Skip to main content
SpringerLink
Log in

Deep shape-aware descriptor for nonrigid 3D object retrieval

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

Deep learning is a rapidly growing discipline that models high-level features in data as multilayered neural networks. In this paper, we propose a deep learning approach for 3D shape retrieval using a multi-level feature learning methodology. We first extract low-level features or local descriptors from a 3D shape using spectral graph wavelets. Then, we construct mid-level features from these local descriptors via the bag-of-features paradigm by employing locality-constrained linear coding as a feature coding method, together with the biharmonic distance as a measure of the spatial relationship between each pair of bag-of-feature descriptors. Finally, high-level shape features are learned via a deep auto-encoder, resulting in a deep shape-aware descriptor that is compact, geometrically informative and efficient to compute. The proposed 3D shape retrieval approach is evaluated on SHREC-2014 and SHREC-2015 datasets through extensive experiments, and the results show compelling superiority of our approach over the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. ARPACK (ARnoldi PACKage) is a MATLAB library for computing the eigenvalues and eigenvectors of large matrices.

References

  1. Pickup D, Sun X, Rosin P, Martin R, Cheng Z, Lian Z, Aono M, Ben Hamza A, Bronstein A, Bronstein M, Bu S, Castellani U, Cheng S, Garro V, Giachetti A, Godil A, Han J, Johan H, Lai L, Li B, Li C, Li H, Litman R, Liu X, Liu Z, Lu Y, Tatsuma A, Ye J (2014) SHREC’14 track: shape retrieval of non-rigid 3D human models. In: Proceedings of the Eurographics Workshop on 3D Object Retrieval, pp 1–10

  2. Lian JZZ, Choi S, ElNaghy H, El-Sana J, Furuya T, Giachetti A, Isaia RGL, Lai L, Li C, Li H, Limberger F, Martin R, Nakanishi R, Nonato ANL, Ohbuchi R, Pevzner K, Pickup D, Rosin P, Sharf A, Sun L, Sun X, Tari S, Unal G, Wilson R (2015) SHREC’15 track: non-rigid 3D shape retrieval. In: Proceedings of the Eurographics Workshop on 3D Object Retrieval, pp 1–14

  3. Rustamov R (2007) Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In: Proc. Symp. Geometry Processing, pp 225–233

  4. Sun J, Ovsjanikov M, Guibas L (2009) A concise and provably informative multi-scale signature based on heat diffusion. Comput Graph Forum 28(5):1383–1392

    Article  Google Scholar 

  5. Bronstein M, Kokkinos I (2010) Scale-invariant heat kernel signatures for non-rigid shape recognition. In: Proceedings of the CVPR, pp 1704–1711

  6. Aubry M, Schlickewei U, Cremers D (2011) The wave kernel signature: a quantum mechanical approach to shape analysis. In: Proceedings of the Computational Methods for the Innovative Design of Electrical Devices, pp 1626–1633

  7. Li C, Ben Hamza A (2013) A multiresolution descriptor for deformable 3D shape retrieval. Visual Comput 29:513–524

    Article  Google Scholar 

  8. Reuter M, Wolter F, Peinecke N (2006) Laplace-Beltrami spectra as ‘Shape-DNA’ of surfaces and solids. Comput Aided Design 38(4):342–366

    Article  Google Scholar 

  9. Chaudhari A, Leahy R, Wise B, Lane N, Badawi R, Joshi A (2014) Global point signature for shape analysis of carpal bones. Phys Med Biol 59:961–973

    Article  Google Scholar 

  10. Ye J, Yu Y (2015) A fast modal space transform for robust nonrigid shape retrieval. Visual Comput 32(5):553–568

    Article  MathSciNet  Google Scholar 

  11. Bronstein A, Bronstein M, Guibas L, Ovsjanikov M (2011) Shape google: geometric words and expressions for invariant shape retrieval. ACM Trans Graph 30(1):1–20

  12. Litman R, Bronstein A, Bronstein M, Castellani U (2014) Supervised learning of bag-of-features shape descriptors using sparse coding. Comput Graph Forum 33(5):127–136

    Article  Google Scholar 

  13. LeCun Y, Boser B, Denker J, Henderson D, Howard R, Hubbard W, Jackel L (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  14. Ciresan D, Meier U, Masci J, Gambardella L, Schmidhuber J (2011) Flexible, high performance convolutional neural networks for image classification. In: Proceedings of the IJAC, pp 1237–1242

  15. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y. OverFeat: integrated recognition, localization and detection using convolutional networks. In: Proceedings of the ICLR

  16. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the CVPR

  17. Lee H, Grosse R, Ranganath R, Ng A, Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the ICML, pp 609–616

  18. Masci J, Meier U, Ciresan D, Schmidhuber J (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: Proceedings of the International Conference on Artificial Neural Networks, p 5259

  19. Hinton G, Osindero S, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  20. Hinton G (2012) A practical guide to training restricted Boltzmann machines. In: Montavon G, Orr GB, Müller K (eds) Neural networks: tricks of the trade. Springer, Berlin, pp 599–619

  21. Xie J, Xu L, Chen E (2012) Image denoising and inpainting with deep neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp 350–358

  22. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp 1097–1105

  23. Karpathy A, Fei-Fei L (2015) Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the CVPR, pp 3128–3137

  24. Zhang J, Shan S, Kan M, Chen X (2014) Coarse-to-fine autoencoder networks (CFAN) for real-time face alignment. In: Proceedings of the ECCV, pp 1–16

  25. Eslami S, Heess N, Williams C, Winn J (2014) The shape Boltzmann machine: a strong model of object shape. Int J Comput Vis 107(2):155–176

    Article  MathSciNet  MATH  Google Scholar 

  26. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the CVPR, pp 1912–1920

  27. Su H, Maji S, Kalogerakis E, Learned-Miller E (2015) Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the ICCV, pp 945–953

  28. Zhu Z, Wang X, Bai S, Yao C, Bai X (2016) Deep learning representation using autoencoder for 3D shape retrieval. Neurocomputing 204(2016):41–50

  29. Qi C, Su H, Nießner M, Dai A, Yan M, Guibas L (2016) Volumetric and multi-view CNNs for object classification on 3D data. In: Proceedings of the CVPR

  30. Savva M, Yu F, Su H, Aono M, Chen B, Cohen-Or D, Deng W, Su H, Bai S, Bai X, Fish JHN, Kalogerakis E, Learned-Miller E, Li Y, Liao M, Maji S, Wang Y, Zhang N, Zhou Z (2016) SHREC’16 track: large-scale 3D shape retrieval from ShapeNet Core55. In: Proceedings of the Eurographics Workshop on 3D Object Retrieval

  31. Fang Y, Xie J, Dai G, Wang M, Zhu F, Xu T, Wong E (2015) 3D deep shape descriptor. In: Proceedings of the CVPR, pp 2319–2328

  32. Bu S, Liu Z, Han J, Wu J, Ji R (2014) Learning high-level feature by deep belief networks for 3-D model retrieval and recognition. IEEE Trans Multimed 24(16):2154–2167

  33. Lipman Y, Rustamov R, Funkhouser T (2010) Biharmonic distance. ACM Trans Graph 29(3):1–11

    Article  Google Scholar 

  34. Rosenberg S (1997) The Laplacian on a Riemannian manifold. Cambridge University Press

  35. Meyer M, Desbrun M, Schröder P, Barr A (2003) Discrete differential-geometry operators for triangulated 2-manifolds. Vis Math III 3(7):35–57

    MathSciNet  MATH  Google Scholar 

  36. Li C, Ben Hamza A (2013) Intrinsic spatial pyramid matching for deformable 3D shape retrieval. Int J Multimed Inf Retr 2:261–271

    Article  Google Scholar 

  37. Dong W, Li X, Zhang D, Shi G (2010) Sparsity-based image denoising via dictionary learning and structural clustering. In: Proceedings of the CVPR, pp 3360–3367

  38. Ben Hamza A, Krim H (2006) Geodesic matching of triangulated surfaces. IEEE Trans Image Process 15(8):2249–2258

    Article  Google Scholar 

  39. Lian Z, Godil A, Bustos B, Daoudi M, Hermans J, Kawamura S, Kurita Y, Lavoué G, Nguyen H, Ohbuchi R, Ohkita Y, Ohishi Y, Porikli F, Reuter M, Sipiran I, Smeets D, Suetens P, Tabia H, Vandermeulen D (2011) SHREC’11 track: shape retrieval on non-rigid 3D watertight meshes. In: Proceedings of the Eurographics/ACM SIGGRAPH Symposium on 3D Object Retrieval, pp 79–88

  40. Giachetti A, Lovato C (2012) Radial symmetry detection and shape characterization with the multiscale area projection transform. Comput Graph Forum 31(5):1669–1678

    Article  Google Scholar 

  41. Pickup D, Sun X, Rosin P, Martin R (2015) Geometry and context for semantic correspondences and functionality recognition in manmade 3D shapes. Pattern Recognit 48(8):2500–2512

    Article  Google Scholar 

  42. Shilane P, Min P, Kazhdan M, Funkhouser T (2004) The Princeton shape benchmark. In: Proceedings of the SMI, pp 167–178

Download references

Author information

Authors and Affiliations

  1. Concordia Institute for Information Systems Engineering, Concordia University, Montreal, QC, Canada

    Hamed Ghodrati & A. Ben Hamza

Authors
  1. Hamed Ghodrati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Ben Hamza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Ben Hamza.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghodrati, H., Ben Hamza, A. Deep shape-aware descriptor for nonrigid 3D object retrieval. Int J Multimed Info Retr 5, 151–164 (2016). https://doi.org/10.1007/s13735-016-0103-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-016-0103-x

Keywords

Search

Navigation