Skip to main content

A fast modal space transform for robust nonrigid shape retrieval

Abstract

Nonrigid or deformable 3D objects are common in many application domains. Retrieval of such objects in large databases based on shape similarity is still a challenging problem. In this paper, we take advantages of functional operators as characterizations of shape deformation, and further propose a framework to design novel shape signatures for encoding nonrigid geometries. Our approach constructs a context-aware integral kernel operator on a manifold, then applies modal analysis to map this operator into a low-frequency functional representation, called fast functional transform, and finally computes its spectrum as the shape signature. In a nutshell, our method is fast, isometry-invariant, discriminative, smooth and numerically stable with respect to multiple types of perturbations. Experimental results demonstrate that our new shape signature for nonrigid objects can outperform all methods participating in the nonrigid track of the SHREC’11 contest. It is also the second best performing method in the real human model track of SHREC’14.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. 1.

    See Appendix 6 for the definition of perturbed kernel.

  2. 2.

    Geodesic distance is computed using the fast marching Matlab toolbox, http://www.mathworks.com/matlabcentral/fileexchange/6110.

  3. 3.

    This code can be downloaded at https://mirc.uzleuven.be/MedicalImageComputing/downloads/meshSIFT.php.

  4. 4.

    This dataset can be downloaded at https://code.google.com/p/tri-mesh-toolkit/.

  5. 5.

    A hybrid track combining SD-GDM and meshSIFT in SHREC’11 did achieve a better performance, but it falls out of scope in our state-of-art evaluation.

  6. 6.

    This dataset can be downloaded at http://visionlab.uta.edu/shape_data.htm.

References

  1. 1.

    Aflalo, Y., Kimmel, R.: Spectral multidimensional scaling. Proc. Natl. Acad. Sci. 110(45), 18052–18057 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Agathos, A., Pratikakis, I., Papadakis, P., Perantonis, S., Azariadis, P., Sapidis, N.S.: 3D articulated object retrieval using a graph-based representation. Vis Comput. 26(10), 1301–1319 (2010)

    Article  Google Scholar 

  3. 3.

    Alfakih, A.Y.: On the eigenvalues of Euclidean distance matrices. Comput. Appl. Math. 27(3), 237–250 (2008)

    MathSciNet  Google Scholar 

  4. 4.

    Arnoldi, W.E.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Q. Appl. Math 9(1), 17–29 (1951)

    MathSciNet  MATH  Google Scholar 

  5. 5.

    Attene, M., Falcidieno, B.: Remesh: an interactive environment to edit and repair triangle meshes. In: IEEE International Conference on Shape Modeling and Applications, 2006. SMI 2006, pp. 41–41. IEEE (2006)

  6. 6.

    Aubin, T.: Some Nonlinear Problems in Riemannian Geometry. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  7. 7.

    Bogomolny, E., Bohigas, O., Schmit, C.: Spectral properties of distance matrices. J. Phys. A: Math. Gen 36(12), 3595 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    Bronstein, A.M., Bronstein, M.M., Guibas, L.J., Ovsjanikov, M.: ShapeGoogle: geometric words and expressions for invariant shape retrieval. ACM Trans. Graph. (TOG) 30(1), 1 (2011)

    Article  Google Scholar 

  9. 9.

    Bronstein, M.M., Bronstein, A.M.: Shape recognition with spectral distances. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 1065–1071 (2011)

    Article  Google Scholar 

  10. 10.

    Buser, P., Conway, J., Doyle, P., Semmler, K.D.: Some planar isospectral domains. Int. Math. Res. Not. 1994(9), 391 (1994)

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Castro, M., Menegatto, V.: Eigenvalue decay of positive integral operators on the sphere. Math. Comput. 81(280), 2303–2317 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Cignoni, P., Corsini, M., Ranzuglia, G.: Meshlab: an open-source 3D mesh processing system. ERCIM News 73, 45–46 (2008)

    Google Scholar 

  13. 13.

    Elad, A., Kimmel, R.: Bending invariant representations for surfaces. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001, vol. 1, pp. I–168. IEEE (2001)

  14. 14.

    Friedrichs, K.O.: Perturbation of Spectra in Hilbert Space, vol. 3. American Mathematical Society, Providence (1965)

    MATH  Google Scholar 

  15. 15.

    Giachetti, A., Lovato, C.: Radial symmetry detection and shape characterization with the multiscale area projection transform. Comput. Graph. Forum 31(5), 1669–1678 (2012)

    Article  Google Scholar 

  16. 16.

    Gordon, C., Webb, D., Wolpert, S.: Isospectral plane domains and surfaces via Riemannian orbifolds. Invent. Math. 110(1), 1–22 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Hu, J., Hua, J.: Salient spectral geometric features for shape matching and retrieval. Vis Comput 25(5–7), 667–675 (2009)

    Article  Google Scholar 

  18. 18.

    Kato, T.: Perturbation Theory for Linear Operators, vol. 132. Springer, Berlin (1995)

    MATH  Google Scholar 

  19. 19.

    Kurtek, S., Srivastava, A., Klassen, E., Laga, H.: Landmark-guided elastic shape analysis of spherically-parameterized surfaces. Comput. Graph. Forum 32(2pt4), 429–438 (2013)

    Article  Google Scholar 

  20. 20.

    Laga, H., Mortara, M., Spagnuolo, M.: Geometry and context for semantic correspondences and functionality recognition in man-made 3D shapes. ACM Trans. Graph. (TOG) 32(5), 150 (2013)

    Article  Google Scholar 

  21. 21.

    Lavoué, G.: Combination of bag-of-words descriptors for robust partial shape retrieval. Vis. Comput. 28(9), 931–942 (2012)

    Article  Google Scholar 

  22. 22.

    Li, C., Hamza, A.B.: Intrinsic spatial pyramid matching for deformable 3D shape retrieval. Int. J. Multimed. Inf. Retr. 2(4), 261–271 (2013)

    Article  Google Scholar 

  23. 23.

    Li, C., Hamza, A.B.: A multiresolution descriptor for deformable 3D shape retrieval. Vis. Comput. 29(6–8), 513–524 (2013)

    Article  Google Scholar 

  24. 24.

    Lian, Z., Godil, A., Bustos, B., Daoudi, M., Hermans, J., Kawamura, S., Kurita, Y., Lavoue, G., van Nguyen, H., Ohbuchi, R., Ohkita, Y., Ohishi, Y., Porikli, F., Reuter, M., Sipiran, I., Smeets, D., Suetens, P., Tabia, H., Vandermeulen, D.: A comparison of methods for non-rigid 3D shape retrieval. Pattern Recognit. 46(1), 449–461 (2013)

    Article  Google Scholar 

  25. 25.

    Lidskii, V.B.: Perturbation theory of non-conjugate operators. USSR Comput. Math. Math. Phys. 6(1), 73–85 (1966)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Lipman, Y., Rustamov, R.M., Funkhouser, T.A.: Biharmonic distance. In: ACM Transactions on Graphics, ACM 0730–0301//10-ART (2007)

  27. 27.

    Litman, R., Bronstein, A., Bronstein, M., Castellani, U.: Supervised learning of bag-of-features shape descriptors using sparse coding. Comput. Graph. Forum 33(5), 127–136 (2014)

    Article  Google Scholar 

  28. 28.

    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  29. 29.

    Maes, C., Fabry, T., Keustermans, J., Smeets, D., Suetens, P., Vandermeulen, D.: Feature detection on 3D face surfaces for pose normalisation and recognition. In: 2010 Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp. 1–6. IEEE (2010)

  30. 30.

    Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. ACM Trans. Graph. (TOG) 31(4), 30 (2012)

    Article  Google Scholar 

  31. 31.

    Pickup, D., Sun, X., Rosin, P.L., Martin, R.R., Cheng, Z., Lian, Z., Aono, M., Hamza, A.B., Bronstein, A., Bronstein, M., et al.: Shrec’14 track: shape retrieval of non-rigid 3D human models. Proc. 3DOR 4(7), 8 (2014)

    Google Scholar 

  32. 32.

    Quarteroni, A., Sacco, R., Saleri, F.: Numerical Mathematics, vol. 37. Springer, Berlin (2007)

    Book  MATH  Google Scholar 

  33. 33.

    Raviv, Dan, Kimmel, Ron: Affine invariant geometry for non-rigid shapes. Int. J. Comput. Vis. 111(1), 1–11 (2015)

    MathSciNet  Article  Google Scholar 

  34. 34.

    Reade, J.B.: Eigenvalues of positive definite kernels. SIAM J. Math. Anal. 14(1), 152–157 (1983)

    MathSciNet  Article  MATH  Google Scholar 

  35. 35.

    Reuter, M., Wolter, F.-E., Peinecke, N.: Laplace–Beltrami spectra as ’shape-DNA’ of surfaces and solids. Comput.-Aided Des. 38(4), 342–366 (2006)

    Article  Google Scholar 

  36. 36.

    Rustamov, R.M.: Laplace–Beltrami eigenfunctions for deformation invariant shape representation. In: Proceedings of the Fifth Eurographics Symposium on Geometry Processing, pp. 225–233. Eurographics Association (2007)

  37. 37.

    Shilane, P., Min, P., Kazhdan, M.M., Funkhouser, T.A.: The princeton shape benchmark. In: Shape Modeling International, pp. 167–178. IEEE Computer Society (2004)

  38. 38.

    Smeets, D., Fabry, T., Hermans, J., Vandermeulen, D., Suetens, P.: Isometric deformation modelling for object recognition. In: CAIP 2009, LNCS 5702, pp. 757–765 (2009)

  39. 39.

    Stewart, G.W.: A Krylov–Schur algorithm for large eigenproblems. SIAM J. Matrix Anal. Appl. 23(3), 601–614 (2001)

    MathSciNet  Article  MATH  Google Scholar 

  40. 40.

    Sun, J., Ovsjanikov, M., Guibas, L.J.: A concise and provably informative multi-scale signature based on heat diffusion. Comput. Graph. Forum 28(5), 1383–1392 (2009)

    Article  Google Scholar 

  41. 41.

    Sunada, T.: Riemannian coverings and isospectral manifolds. Ann. Math. 121(1), 169–186 (1985)

    MathSciNet  Article  MATH  Google Scholar 

  42. 42.

    Tabia, H., Daoudi, M., Vandeborre, J.-P., Colot, O.: A new 3D-matching method of nonrigid and partially similar models using curve analysis. IEEE Trans. Pattern Anal. Mach. Intell. 33(4), 852–858 (2011)

    Article  Google Scholar 

  43. 43.

    Tabia, H., Laga, H., Picard, D., Gosselin, P.-H.: Covariance descriptors for 3D shape matching and retrieval. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4185–4192. IEEE (2014)

  44. 44.

    Thakoor, N., Gao, J., Jung, S.: Hidden Markov model-based weighted likelihood discriminant for 2-D shape classification. Image Process. IEEE Trans. 16(11), 2707–2719 (2007)

    MathSciNet  Article  Google Scholar 

  45. 45.

    Weyl, H.: Uber die asymptotische verteilung der eigenwerte. Nachrichten der Königlichen Gesellschaft der Wissenschaften zu Göttingen. Mathematisch-Naturwissenschaftliche Klasse, pp. 110–117 (1911)

  46. 46.

    Ye, J., Yan, Z., Yu, Y.: Fast nonrigid 3D retrieval using modal space transform. In: International Conference on Multimedia Retrieval, pp. 121–126 (2013)

  47. 47.

    Ying, X., Wang, X., He, Y.: Saddle vertex graph (svg): a novel solution to the discrete geodesic problem. ACM Trans. Graph. 32(6), 1–170 (2013)

    Article  Google Scholar 

  48. 48.

    Zaharescu, A., Boyer, E., Varanasi, K., Horaud, R.: Surface feature detection and description with applications to mesh matching. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 373–380. IEEE (2009)

  49. 49.

    Zahn, C.T., Roskies, R.Z.: Fourier descriptors for plane closed curves. Comput. IEEE Trans. 100(3), 269–281 (1972)

    MathSciNet  Article  MATH  Google Scholar 

  50. 50.

    Zienkiewicz, O.C., Taylor, R.L., Zhu, J.Z.: The Finite Element Method—its Basis and Fundamentals, vol. 1. Elsevier Butterworth-Heinemann, Amsterdam, London (2005)

    MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Maks Ovsjanikov and Dirk Smeets for sharing their software implementations.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yizhou Yu.

Appendices

Appendix A: Error bounds under deformation

In this section, we investigate error bounds of the spectral signature of integral kernel operators in the presence of “smooth” manifold deformations using perturbation theory [14, 18]. The general result, intuitively speaking, is that the total deviation of a spectral signature with an arbitrary number of dimensions is finitely bounded under smooth deformations. Given a general matrix \(A\) and a perturbation parameter \(\epsilon \) within a neighborhood of zero, the perturbed matrix is written as \(A+\epsilon B\), where \(B\) is an arbitrary matrix. In this situation, each eigenvalue or eigenvectors of \(A+\epsilon B\) admits an expansion in fractional powers of \(\epsilon \), and the zeroth order term of this expansion is an eigenvalue or eigenvector of the unperturbed matrix \(A\) (Lidskii theorem, 1965 [25]). This is a well-known result in regular perturbation theory. In particular, one can actually derive a Lipschitz condition number for this continuity:

Theorem 6.1

(A special case of Lidskii theorem [18]) Assuming \(\mu (\epsilon )\) is an eigenvalue of the perturbed matrix \(A+\epsilon B\) for a sufficiently small \(\epsilon \), it admits a first-order expansion

$$\begin{aligned} \mu (\epsilon ) = \mu + \left\| \Phi ^T B\Phi \right\| _{\max } \epsilon + o(\epsilon ), \end{aligned}$$

where \(\Phi \) is the eigenvector of \(A\) corresponding to eigenvalue \(\mu \), \(A\) and \(B\) are conjugated matrices, \(\left\| \cdot \right\| _{\max }\) represents the largest singular value. The first-order coefficient \(\left\| \Phi ^T B\Phi \right\| _{\max }\) is called the Lipschitz condition number.

For the rest of this section, \(A\) is called characterization, and \(B\) is called perturbation. The Lipschitz condition number characterizes the total amount of deviation in a signature with respect to a certain amount of perturbation. In the ideal case of isometric perturbation, where \(A\) is isometric and \(B\) is zero, the Lipschitz condition number is degenerated. Let \(M\) and \(M'\) be two compact manifold domains, and \(H\) and \(H'\) be the spaces of bounded and continuous linear functionals defined on \(M\) and \(M'\). In the current context, we consider \(M'\) as a “gently” deformed version of \(M\). We assume that implicitly there exists a matching (or registration) between these two domains such that it induces a mapping between \(H\) and \(H'\), which is a closed and densely defined operator \(\mathcal D\) from \(H\) to \(H'\). The characterization of this mapping is a self-adjoint operator \(\mathcal O\). Suppose \(\mathcal O'(\epsilon )\) has a first-order approximate perturbed (infinitely dimensional) matrix representation \(A + \epsilon B\) with respect to an underlying domain deformation \(\mathcal D_{\epsilon }\).

Definition 6.1

(Perturbated \((\alpha ,p)\) -smoothness) Consider a perturbated matrix \(A + \epsilon B\) with a spectral family \(\{\mu _i(\epsilon )\}\) admitting first-order expansions: \(\mu _i(\epsilon ) = \mu _i + \left| \phi _i^T B\phi _i \right| \epsilon + o(\epsilon )\). \(A + \epsilon B\) is perturbated \((\alpha ,p)\)-smooth if

$$\begin{aligned} \left\| A^{-\frac{\alpha }{2}}B^pA^{-\frac{\alpha }{2}} \right\| _1=: \sum \limits _{i=1}^{\infty }\left| \mu _i \right| ^{-\alpha }\left\| \phi _i^T B\phi _i \right\| ^p < C(\alpha , p), \end{aligned}$$

for some \(\alpha \ge 0\) and \(p> 0\), where \(\left\| \cdot \right\| _1\) is Schatten p-norms.

Perturbed smoothness actually admits a substantially larger class of deformations of practical interest. It allows the characterization to be varied with respect to the domain deformation, but it bounds the amount of deviation.

Theorem 6.2

(Error bound of spectral signature) Consider a discrepancy function for a pair of original and perturbed spectra

$$\begin{aligned} d_n(\mu ,\mu ';\alpha , p) :=\sum _{i=1}^n\left| \mu _i \right| ^{-\alpha }\left| \mu _i-\mu _i' \right| ^p, \end{aligned}$$

and two Hilbert spaces \(H\) and \(H'\) admitting a perturbation path \(\mathcal D_{\epsilon }\) for \(\epsilon \in [0,1]\). If the perturbation path is uniformly perturbed \((\alpha , p)\)-smooth with a bound \(C(\alpha ,p)\) with respect to a characterization operator \(\mathcal O(\epsilon )\), and order preserving, aka \(\mu _i\ge \mu _j\iff \mu _i'\ge \mu _j'\) for any \(i,j\), we have

$$\begin{aligned} d_n(\mu ,\mu ';\alpha , p)\le \kappa ^{-\alpha } C(\alpha , p), \quad \forall n \end{aligned}$$

where \(\kappa = \max \limits _{i}\{\left| \frac{\mu _i}{\mu _{i+1}} \right| ,\left| \frac{\mu _{i+1}}{\mu _i} \right| \}\).

Proof

(Sketch) Take the integral w.r.t \(\epsilon \): \(\mu _i'-\mu _i=\int _0^1 \mathrm {d}\mu _i(\epsilon ) \) and use the condition of perturbated \((\alpha ,p)\)-smoothness (Definition 6.1). \(\square \)

Integral kernel operator Experimental evidences suggest that distance maps/matrices, more generally, integral operators, could be a useful source for shape descriptor construction, and are potentially better than differential operators (e.g., LBO). As a rigorous treatment justifying the capability of integral kernel operators, we establish another definition for the smoothness of perturbations:

Definition 6.2

(\(\alpha \) -Hölder class of perturbed kernel) Consider integral kernel operator \(k_{\epsilon }(\cdot ,\cdot )\) with perturbation parameter \(\epsilon \), and define the perturbed kernel

$$\begin{aligned} \gamma _{\epsilon }(x,y; k) = \frac{1}{\epsilon } (k_{\epsilon }(\tilde{x},\tilde{y})-k(x,y)), \end{aligned}$$

where \((\tilde{x},\tilde{y})\in M'(\epsilon )\) is the perturbed version of \((x,y)\in M\). Let \(d_M\) be the geodesic distance on \(M\). If there exists \(C>0\) such that

$$\begin{aligned} \left| \gamma _{\epsilon }(x,y; k) - \gamma _{\epsilon }(x,z; k) \right| \le C_{\alpha } \cdot d_M(y,z)^{\alpha }, \end{aligned}$$

and

$$\begin{aligned} \left| \gamma _{\epsilon }(x,y; k) \right| \le C_{\alpha }, \end{aligned}$$

\(\forall x\in M\) and any \(\epsilon >0\) in a sufficiently small neighborhood of zero.

Remark

If \(k(\cdot ,\cdot ) = d_M(\cdot ,\cdot )\), the infimum of bound \(C_{\alpha }\) that makes \(k_{\epsilon }\)’s perturbed kernel belong to \(\alpha \)-Hölder class, aka

$$\begin{aligned} C_{\alpha }^* := \sup _{x,y,z,\epsilon } \left\{ \dfrac{\left| \gamma _{\epsilon }(x,y; k) - \gamma _{\epsilon }(x,z; k) \right| }{d_M^{\alpha }(\cdot ,\cdot )}, \left| \gamma _{\epsilon }(x,y; k) \right| \right\} , \end{aligned}$$

is a “natural” quantity indicator of deformations.

In the sequel, we connect \(\alpha \)-Hölder class of perturbed kernel with perturbed smoothness.

Theorem 6.3

(Main result) Given an integral kernel operator \(\mathcal K\) as described in Sect. 2.2, we assume the distance map is in accordance with the geodesics of power \(q\), aka, there exist positive constants \(C_1\), \(C_2\) and some \(\delta \ge 0\) such that

$$\begin{aligned} C_1/n^{1+q+\delta } \le \lambda _n(k) \le C_2/n^{1+q},\quad \forall n, \end{aligned}$$
(12)

for some integer \(q\ge 1\).

If its perturbed kernels with respect to a class of deformations, i.e., \(\gamma _{\epsilon }(\cdot ,\cdot ; k)\), are of \(\varepsilon \)-Hölder class (Definition 6.2) for some \(1\ge \varepsilon > 0\), \(\mathcal K\) or the transformed operator \(\widetilde{\mathcal K} = \Phi ^T \mathcal K \Phi \) (Sect. 2.2, Eq. 3), as a characterization of the deformed manifold \(M'(\epsilon )\), is perturbed \((\alpha ,p)\)-smooth (Definition 6.1) as long as

$$\begin{aligned} \left( \frac{1}{2} + \varepsilon \right) p > (1+q+\delta )\alpha + 1. \end{aligned}$$

Thus, spectral signatures (with an arbitrary number of dimensions) of both \(\mathcal K\) and \(\widetilde{\mathcal K}\) have finite error bounds under smooth manifold deformations (Theorem 6.2).

Proof

For a general Hilbert–Schmidt operator \(k\) satisfying uniformly \(\varepsilon \)-Hölder condition, it has already been shown in [34] that the decay rate of eigenvalues of \(k(x,y)\) is at least \(O(n^{-1/2-\varepsilon })\). And if \(k\) is positive definite, the decay rate is improved to at least \(O(n^{-{1}-\varepsilon })\). It immediately follows that for \(\varepsilon > \frac{1}{2}\), \(\varepsilon \)-Hölder continuous kernel \(\gamma _{\epsilon }\) is perturbed \((0,1)\)-smooth.

For \(\alpha >0\), we require the eigenvalues of the characterization kernel that have lower bounds on its decay rate. It is generally not true, for example if \(k(x,y)=d_M(x,y)^2\), the resulting Gram matrix has a finite number of nonzero eigenvalues if sample points are uniformly drawn from the Euclidean space. But it is typically tangible, as preliminary work has shown examples where condition (12) holds when \(k(x,y)=d_M(x,y)\) (e.g., see [7, 11] examples). Then following the main result on Hölder condition and the decay of eigenvalues [34], one has \(\dfrac{\left| \lambda _n(\gamma _{\epsilon }) \right| ^p}{\left| \lambda _n(k) \right| ^{\alpha }}\sim O(n^{-(\frac{1}{2} +\varepsilon )p+\alpha (1+q+\delta )})\).

Appendix B: Spectral convergence of R-BiHDM

Since we know eigenvalues of the Laplace–Beltrami operator follow \(\lambda _m \sim \frac{4\pi }{A} m\) for surfaces (Weyl-asymptotic growth of eigenvalue [45]), we can therefore deduce the convergence of a spectral sequence under weak assumptions. Note \(\left\| \cdot \right\| \) always denotes the \(L_2\)-norm of a vector, a function or a matrix, by default.

Lemma 1

Let \(A\in \mathbb {C}^{n\times n}\) be an Hermitian matrix, and let \((\hat{\lambda },\hat{x})\) be the computed approximation of an eigenvalue/eigenvector pair \((\lambda ,x)\) of \(A\). By defining the residual

$$\begin{aligned} \hat{r} = A \hat{x} - \hat{\lambda }\hat{x}, \hat{x} \ne 0, \end{aligned}$$

it follows that

$$\begin{aligned} \min _{\lambda _i\in \sigma (A)} \left| \hat{\lambda }- \lambda _i \right| \le \dfrac{\left\| \hat{r} \right\| _2}{\left\| \hat{x} \right\| _2}. \end{aligned}$$

Proof

See [32] page 195. \(\square \)

Theorem 7.1

(Spectral convergence) Let \(\widetilde{K}_{m}\) be an \((m+1)\times (m+1)\) square matrix formed by the following steps: (i) take the first \(m+1\) rows and first \(m+1\) columns of \(\widetilde{K}\); (ii) when calculating each \(a_j\) in this truncated matrix, every infinite summation in (5) is approximated by the first \(m\) terms. Set \(\mu _i^{(m)}\) be the \(i\)-th eigenvalue of \(\widetilde{K}_m\), for \(i\le m+1\) and \(\mu _i{(m)}=0\), for any \(i> m+1\). Let \(f_m = \sqrt{A}\sum _{i=1}^m\phi _i^2/\lambda _i^2\) converge (pointwisely) to \(f\). If \(f\) is bounded and square integrable, i.e., eigenpairs \(\{(\lambda _i, \phi _i)\}_{i=1}^\infty \) satisfy

$$\begin{aligned} \sum _{i=1}^\infty \dfrac{\left\| \phi _i^2 \right\| ^2}{\lambda _i^4} < \infty , \end{aligned}$$
(13)

it then follows that for any \(i\),

$$\begin{aligned} \mu _i^{(m)}\rightarrow \mu _i,\quad \text{ as } m\rightarrow \infty . \end{aligned}$$

Proof

By definition, we have \(\widetilde{K}_{m+n} - \widetilde{K}_{m-1}=\)

$$\begin{aligned} \sum _{k=m}^{m+n} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} \frac{2}{\lambda ^2_k}&{}\frac{C_{k,1}}{\lambda _k^2}&{}\ldots &{}\frac{C_{k,m-1}}{\lambda _k^2}&{} 0&{}\ldots &{}a^{(m+n)}_k&{}\ldots \\ \frac{C_{k,1}}{\lambda _k^2}&{}0\\ \vdots &{}&{}\ddots &{}&{}\\ \frac{C_{k,m-1}}{\lambda _k^2}&{}&{}&{}0\\ 0&{}&{}&{}&{}0\\ \vdots &{}&{}&{}&{}&{}\ddots \\ a^{(m+n)}_k&{}&{}&{}&{}&{}&{}-\frac{2}{\lambda ^2_k}\\ \vdots &{}&{}&{}&{}&{}&{}&{}\ddots \end{array}\right] , \end{aligned}$$

where extra rows and columns of zeros are padded to \(\widetilde{K}_{m-1}\), and

$$\begin{aligned} C_{m,j} = \sqrt{A}\int _M \phi _m^2\phi _j, \end{aligned}$$

which is the Fourier coefficient of \(\sqrt{A}\phi _m^2\) w.r.t series \(\{\phi _j\}_{j=1}^{m-1}\). By Bessel’s inequality we have \(1+\sum _{j=1}^{m-1}C_{m,j}^2\le A\left\| \phi _m^2 \right\| ^2\), and we also have

$$\begin{aligned} \sum _{k=m}^{m+n}\left| a^{(m+n)}_k \right| ^2\le & {} \sum _{k=m}^{\infty }\left| \int _Mf_{m+n}\phi _k \right| ^2\\\le & {} \left\| f_{m+n} -f \right\| ^2 + \sum _{k=m}^{\infty }\left| \int _M f\phi _k \right| ^2. \end{aligned}$$

By assumptions about \(f\) and the norm convergence of Fourier series (as shown for example in [1]), we know that \(\left\| f_{m+n} -f \right\| ^2\rightarrow 0\) and \(\sum _{k=m}^{\infty }\left| \int _M f\phi _k \right| ^2\rightarrow 0\) as \(m\rightarrow \infty \). Since the \(L_2\)-norm of a matrix is less than its Frobenius norm, we have

$$\begin{aligned}&\!\!\! \left\| \widetilde{K}_{m+n} - \widetilde{K}_{m-1} \right\| ^2_2\\&\quad \le \sum _{k=m}^{m+n}\left( \frac{2}{\lambda _k^4} (4+\sum _{j=1}^{m-1}C^2_{k,j}) + 2\left| a^{(m+n)}_k \right| ^2\right) \\&\quad \le \sum _{k=m}^{m+n}\left( \frac{2}{\lambda _k^4}(3+A\left\| \phi _k^2 \right\| ^2) + 2\left| a^{(m+n)}_k \right| ^2\right) \rightarrow 0 \end{aligned}$$

as \(m\rightarrow \infty \). By Lemma 1, we have

$$\begin{aligned} \left| \mu _i^{(m+n)}-\mu _i^{(m-1)} \right|\le & {} \dfrac{\left\| (\widetilde{K}_{m+n} - \widetilde{K}_{m-1})u_i^{(m+n)} \right\| }{\left\| u^{(m+n)}_i \right\| }\\\le & {} \left\| \widetilde{K}_{m+n} - \widetilde{K}_{m-1} \right\| _2 \end{aligned}$$

for any \(i<m\), where \(u^{(m)}_i\) is the eigenvector associated with \(\mu ^{(m)}_i\) of \(\widetilde{K}_{m}\). \(\{\mu _i^{(m)}\}_{m=1}^{\infty }\) is a Cauchy sequence, that converges. \(\square \)

Note that the condition, i.e., Eq. (13), used in the spectral convergence theorem is rather weak. Observe that for two-dimensional Riemannian manifold \(M\), \(f(x)\) is given by the green function \(G(x,y)\) with \(f(x):=G(x,x)\). If \(M\) is a compact manifold (without boundary), it satisfies that \(G(x,y)\) is bounded [6], hence \(f(x)\) is also bounded for any \(x\in M\). So is \(\left\| f \right\| ^2\) (square integrable). An experimental validation is also provided in Fig. 7.

Fig. 7
figure7

A experimental validation: convergence of the first 30 eigenvalues of an R-BiHDM

Appendix C: Matrices for linear and cubic FEM

Fig. 8
figure8

Node configuration of linear and cubic FEM

Table 6 The four matrices of integrals on the unit triangle for linear FEM
Table 7 The four matrices of integrals on the unit triangle for cubic FEM

Here we provide elementary matrices used in linear and cubic FEM. Every FEM has its own set of nodes. There is a finite element associated with every node. The finite element at a specific node is a piecewise polynomial basis function whose value is equal to 1 at the node and 0 at all other nodes. The support of a basis function includes all triangle faces surrounding the node corresponding to the basis function. Consider a triangle face in a mesh with two node configurations shown in Fig. 8. These node configurations are defined for linear and cubic FEM, respectively. Each node in either configuration is associated with a piecewise linear or cubic polynomial basis function. If we focus on a triangle face, an arbitrary bivariate polynomial over the triangle face can be defined as a linear combination of the basis functions associated with the nodes either inside or on the edges of the triangle. This polynomial can interpolate arbitrary prescribed values at the nodes. It reduces to a univariate polynomial along each triangle edge. The two polynomials defined over two adjacent triangles agree with each other along their shared triangle edge. The stiff and mass matrices used in FEM are given as follows.

$$\begin{aligned} L_{ij}(T)= & {} \frac{1}{a(T)}\left( \sum _{k=1}^3 l_k^2(T)J_k\left( t_i,t_j\right) \right) , \quad i,j\in N(T)\\ D_{ij}(T)= & {} a(T)J_4\left( t_i,t_j\right) , \quad i,j\in N(T)\\ L_{ij}(T)= & {} D_{ij}(T) = 0, \quad i,j\not \in N(T) \end{aligned}$$

and

$$\begin{aligned} L = \sum _{T} L(T), \quad D = \sum _{T} D(T), \end{aligned}$$

where \(T\) denotes a triangle face, \(N(T)\) denotes the set of incidence nodes on \(T\), \(t_i\) denotes the local index (used for \(T\)) of the \(i\)-th node used in the FEM, \(l_k(T)\) \((k=1,2,3)\) denotes the length of the \(k\)-th edge of \(T\), \(a(T)\) denotes the area of \(T\), and \(J_k(t_i,t_j)\) denotes the entry with indices \((t_i,t_j)\) of the matrix \(J_k\). \(J_k\) \((k=1,2,3,4)\) for linear and cubic FEM are given in Tables 6 and 7, respectively. Details on the construction of \(J_k\) can be found in [50]. Note that these matrices are assembled triangle-wise rather than element-wise in a typical implementation.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ye, J., Yu, Y. A fast modal space transform for robust nonrigid shape retrieval. Vis Comput 32, 553–568 (2016). https://doi.org/10.1007/s00371-015-1071-5

Download citation

Keywords

  • Content-based object retrieval
  • Shape retrieval
  • Biharmonic distance
  • Functional map
  • Shape signature