Shape-Aware Matching of Implicit Surfaces Based on Thin Shell Energies
- First Online:
- Received:
- Revised:
- Accepted:
- 230 Downloads
Abstract
A shape sensitive, variational approach for the matching of surfaces considered as thin elastic shells is investigated. The elasticity functional to be minimized takes into account two different types of nonlinear energies: a membrane energy measuring the rate of tangential distortion when deforming the reference shell into the template shell, and a bending energy measuring the bending under the deformation in terms of the change of the shape operators from the undeformed into the deformed configuration. The variational method applies to surfaces described as level sets. It is mathematically well-posed, and an existence proof of an optimal matching deformation is given. The variational model is implemented using a finite element discretization combined with a narrow band approach on an efficient hierarchical grid structure. For the optimization, a regularized nonlinear conjugate gradient scheme and a cascadic multilevel strategy are used. The features of the proposed approach are studied for synthetic test cases and a collection of geometry processing examples.
Keywords
Variational shape matching Implicit surfaces Thin shells Weak lower semicontinuityMathematics Subject Classification
Primary 49J45 Secondary 65D18 74K25 68U051 Introduction
We present a variational model for the matching of surfaces implicitly represented as level sets. The approach is inspired by the mathematical theory of nonlinear elasticity of thin shells. The model consists in an energy functional, which is to be minimized among deformations of a computational domain in which two given surfaces are embedded. A minimizer of this functional is a deformation that closely maps one (reference) surface onto the other (template) surface. As the underlying model we consider the reference surface as a thin elastic shell, i.e., a layer of an elastic material embedded in a volume of another several orders of magnitude softer isotropic elastic material. Subject to matching forces the volume is deformed in such a way that the thin shell is mapped onto the template surface. The functional reflects desired phenomena like resistance to compression and expansion of the surface, resistance to bending, and rotational invariance, while solely involving the deformation and the Jacobian of the deformation. The model is formulated in terms of projected derivatives from the tangent space of the reference surface onto the expected tangent space of the template surface. Taking into account a suitable factorization of the natural pullback under a deformation of shape operators enables us to formulate a model with appropriate convexity properties. The actual surface matching constraint is handled through a penalty, allowing for efficient numerical computation.
Through arguments of compensated compactness, we are able to show weak lower semicontinuity of the energy and consequently existence of minimizing deformations. We present a numerical approach based on a multilinear finite element ansatz for the deformation implemented on adaptive octree grids. The resulting discrete energy is minimized in a multiscale fashion applying a regularized gradient descent.
In the conference article [32], a preliminary version of this approach was presented. For the functional in that paper, lower semicontinuity could not be ensured for either the membrane or bending energies. This lack of lower semicontinuity manifests itself in applications, where compression of the surface is expected, and leads to undesired oscillations in almost-minimizing deformations, which we explore in the present work through explicit examples and computations. Additionally, to increase the efficiency the computational meshes are in the present paper adapted to the surfaces. Consequently, the number of degrees of freedom scales asymptotically almost like that of a surface problem.
Related work Linear elasticity has been extensively used in computer vision and in graphics. Prominent applications are image registration [33, 34, 38, 48, 55], optical flow extraction [35], and shape modeling [29]. Recently, theories of nonlinear elasticity have been applied in many computer vision and graphics problems such as mesh deformation [13], shape averaging [56], registration of medical images [12]. The advantage of nonlinear models is that they allow for intuitive deformations when the displacements are large.
In this paper, we present a model for nonlinear elastic matching of thin shells. A finite element method for the discretization of bending energies of biological membranes has been introduced in [5]. Their approach uses quadratic isoparametric finite elements to approximate the interface on which the gradient flow of an elastic energy of Helfrich type is considered. The papers [9, 10] discuss accurate convex relaxation of higher-order variational problems on curves described as jump sets of functions of bounded variation. In particular, it enables the numerical treatment of elastic energies on such curves.
One challenge in polyhedral surface processing is to provide consistent notions of curvatures and second fundamental forms, i.e., notions that converge (in an appropriate topology or in a measure theoretic sense) to their smooth counterparts, given a smooth limit surface. One computationally popular model for discretizing the second fundamental form is Grinspun’s et al. [30] discrete shells model. Another efficient and robust method for nonlinear surface deformation and shape matching is PriMo [6]. This approach is based on replacing the triangles of a polyhedral surfaces by thin prisms. During a deformation, these prisms are required to stay rigid, while nonlinear elastic forces are acting between neighboring prisms to account for bending, twisting, and stretching of the surface. We refer to Botsch and Sorkine [7] for a discussion of pros and cons for various such methods. In comparison with methods based on polyhedral surfaces, level set approaches like ours are not dependent on specific triangulations of the shapes.
The matching of surfaces with elastic energies has recently been studied in [61]. Their energy contains a membrane energy depending on the Cauchy–Green strain tensor and a bending-type energy comparing the mean curvatures on the surfaces. The matching problem is formulated in terms of a binary linear program in the product space of sets of surface patches. For computations, a relaxation approach is used.
A different direction is the use of parametric approaches to reduce shape matching problems to the matching of functions on a fixed domain. For example, the methods presented in [60, 62] are based on conformal maps from the unit disk. A more general variant using conformal maps on surfaces with arbitrary topology is presented in [42]. Within the family of parametric methods, a surface matching approach related to ours is presented in [44], where nonlinear elastic energies are used for matching parametrized surface patches. In comparison with all these methods, our level set approach is nonparametric and allows surfaces of any topology, which does not need to be fixed in advance.
In [59], face matching based on a matching of corresponding level set curves on the facial surfaces is investigated. To match pairs of curves an optimal deformation between them is computed using an elastic shape analysis of curves. Compared to our approach, this model does not take into account bending dissipation of the curves.
A different direction in shape recognition and matching is exploiting the intrinsic geometry of the surfaces only, thereby producing isometry-invariant methods based on the first fundamental form, like those in [11, 24]. In comparison, bending is penalized in our model and we use all curvatures of the surfaces and their directions to be able to better match regions of edges and creases correctly.
A method for matching and blending of curves represented by level sets has been presented in [49]. Thereby, a level set evolution generates an interpolating family of curves, where the associated propagation speed of the level sets depends on differences of level set curvatures. In this class of approaches, geometric evolution problems are formulated, whereas here we focus on variational models for matching deformations. Variational registration of implicit surfaces was also considered in [40], but only through volume elasticity, in contrast to our shell terms.
To summarize, the main novelty of our contribution is the combination of independence of mesh topologies arising from the use of level sets, penalization of tangential distortion in a rotationally invariant framework, and awareness both of curvatures and curvature directions of the surfaces in the matching. We are not aware of any other methods possessing all of these features simultaneously.
Our approach is inspired by the articles [21, 22] in which surface PDE models are derived in terms of the signed distance function. Shape warping based on the framework of [21] has been discussed from a geometric perspective in [14].
Outline The paper is organized as follows. In Sect. 2, we review the required preliminaries about distance functions and formulate the geometric nondistortion and matching conditions that inspire our model. In Sect. 3, we present the different contributions to our energy. Section 4 is devoted to proving the existence of minimizing deformations under suitable Dirichlet and Neumann boundary conditions. Furthermore, the strong convergence of solutions for vanishing matching penalty parameter is discussed and counterexamples showing the lack of lower semicontinuity of related simpler models are given. In Sect. 5, a numerical strategy for minimizing the energy on adaptive octree grids is presented. Finally, Sect. 6 contains a range of numerical examples demonstrating the behavior of solutions corresponding to our design criteria and presents several potential applications.
|B| stands for the Lebesgue measure of \(B \subset \mathbb {R}^n\), and \({\text {diam}}\,B = \sup _{x,y \in B}|x-y|\) for its diameter.
Generic matrices are denoted by A, B, M, N. We use \({\mathbb {1}}\) for the identity matrix. The set of rotations is denoted by \(\text {O}(n)\), and \(\text {SO}(n)\) is the set of orientation-preserving rotations. The set of all symmetric and positive definite matrices is \(\text {SPD}(n)\).
Components of vectors are denoted with subindices. For \(v\in \mathbb {R}^n\), |v| denotes its Euclidean norm. The \((n-1)\)-dimensional sphere is \(\mathbb {S}^{n-1}\). For a matrix M, |M| is the Frobenius norm.
For two column vectors \(v,w \in \mathbb {R}^n\), \(v \otimes w\) is the tensor product of v and w, that is, the square matrix \(v w^T\). In particular, if \(|w|=1\) we have the identity \((v \otimes w) w = v\).
\(\text {P}(e)={\mathbb {1}}- e \otimes e\) is the projection onto vectors orthogonal to \(e \in \mathbb {S}^{n-1}\).
Deformations on \(\mathbb {R}^n\) are denoted by \(\phi \), and deformations defined on a hypersurface \(\mathcal {M}\subset \mathbb {R}^n\) by \(\varphi \). The identity deformation is denoted by \({\text {id}}\).
\(\Omega \subset \mathbb {R}^n\) denotes the computational domain. Every relevant deformation \(\phi \) maps \(\Omega \) into \(\mathbb {R}^n\). \(\Omega \) has to contain all computationally relevant manifolds \(\mathcal {M}\). \(\Omega \) has Lipschitz boundary, is open and bounded.
We use the notation \(\partial _i\) for partial derivatives, \(\nabla \) for the gradient of a scalar function, \(\mathcal {D}\) for the Jacobian matrix of a vector function and \(\mathcal {D}^2\) for the Hessian matrix of a scalar function.
\(\mathcal {M}_1, \mathcal {M}_2 \subset \Omega \) are \(C^{2,1}\) compact hypersurfaces. The inside and outside components of \(\Omega \setminus \mathcal {M}_i\) are well defined by the Jordan–Brouwer separation theorem ([31], Chapter 2, Section 5).
\(T_x\mathcal {M}_i^{\mathbf{{d}}_i(x)}\) denotes the tangent space to \(\mathcal {M}_i^{\mathbf{{d}}_i(x)}\) at x. The outwards normal to \(\mathcal {M}_i^{\mathbf{{d}}_i(x)}\) is given by \(\mathbf{{n}}_i(x)\), and the set of points where \(\mathbf{{d}}_i\) is not differentiable is denoted by \({\text {sing}}\,\mathbf{{d}}_i\).
\(\lambda , \mu \) are the Lamé coefficients of an isotropic material in linearized elasticity.
\(C^0(\Omega ;\mathbb {R}^n)\) is the space of continuous functions from the domain \(\Omega \) to the range \(\mathbb {R}^n\), \(C^{k,\alpha }\) the Hölder spaces in which the k-th derivative is \(\alpha \)-Hölder continuous, including the Lipschitz case \(\alpha =1\). The range of the spaces is specified unless it is \(\mathbb {R}\). Sobolev spaces are denoted by \(W^{1,p}\) and the closure of compactly supported smooth functions in them by \(W_0^{1,p}\).
The letter C is reserved for a generic positive constant that may have different values in each appearance. Sequence indexing is usually denoted by a superscript k, and limits by an overline, e.g., \(\phi ^k \rightarrow \overline{\phi }\).
2 Deformation and Matching of Level Set Hypersurfaces
We are given two compact, connected embedded hypersurfaces \(\mathcal {M}_1,\mathcal {M}_2\) of class \(C^{2,1}\), which are diffeomorphic to each other, and both of which are contained in a bounded Lipschitz domain \(\Omega \subset \mathbb {R}^{n}\). In this section, we deal with the tangential distortion and the change of the shape operator under a deformation \(\phi : \Omega \rightarrow \mathbb {R}^n\).
For any \(c \in \mathbb {R}\), we denote the c-offsets to the hypersurface \(\mathcal {M}_i\) by \(\mathcal {M}_i^c := \{ x \in \Omega \,|\, \mathbf{{d}}_i(x) = c \}\,\). Furthermore, we define the singularity set \({\text {sing}}\,\mathbf{{d}}_i\) as the set of points where \(\mathbf{{d}}_i\) is not differentiable. With the regularity of \(\mathcal {M}_i\) that we have assumed, it is well known (e.g., Theorem 1.1, Corollary 1.3 and Remark 1.4 of [43]) that \({\text {sing}}\,\mathbf{{d}}_i\) has Lebesgue measure zero and \({\text {dist}}(\mathcal {M}_i, {\text {sing}}\,\mathbf{{d}}_i) > 0\). Furthermore, combining [21, Theorem 5.6] and [45, Proposition 4.6, 7.] we see that \(\mathbf{{d}}_i \in C^2(\overline{\Omega } \setminus \overline{{\text {sing}}\,\mathbf{{d}}_i})\).
2.1 Tangential Derivative and Area and Length Distortion
2.2 Bending and Curvature Mismatch
Now, we quantify the change of curvature directions and magnitudes under the deformation \(\phi \). Our approach is motivated by models describing bending of elastic shells, because in our application the hypersurfaces are considered as thin shells.
3 Energy Functional
A membrane deformation energy \(E_{\text {mem}}\) penalizes the tangential distortion measured through \(\mathcal {D}_{\text {tg}}\phi \).
A bending energy \(E_{\text {bend}}\) penalizes bending as reflected by the relative shape operator.
A matching penalty \(E_{\text {match}}\) ensures a proper matching of the two hypersurfaces \(\mathcal {M}_1\) onto \(\mathcal {M}_2\) via a narrow band approach.
A volume energy \(E_{\text {vol}}\) enforces a regular deformation on the whole computational domain \(\Omega \).
In what follows we introduce the four energy contributions separately.
3.1 Tangential Distortion Energy
The energy (3.1) vanishes only on deformations \(\phi \) whose Jacobian matrix \(\mathcal {D}\phi (x)\) maps \(T\mathcal {M}_1^{\mathbf{{d}}_1(x)}\) isometrically onto \(T\mathcal {M}_2^{\mathbf{{d}}_2(\phi (x))}\) for every point \(x \in {\text {supp}}\,{\eta _\sigma \circ \mathbf{{d}}_1}\). In consequence, both tangential expansion and compression are penalized.
3.2 Bending Energy
Thus, we are asking for an alternative lower semicontinuous energy functional which gives preference to deformations \(\phi \) for which \(\mathcal {D}\phi ^T ( {\mathcal {S}}_2\circ \phi ) \mathcal {D}\phi \) is close to \({\mathcal {S}}_1\). We show that this can be achieved with the extended shape operators \({\mathcal {S}}^{ext}_i = \text {P}_i \mathcal {D}^2 \mathbf{{d}}_i \text {P}_i + \mathbf{{n}}_i \otimes \mathbf{{n}}_i\) for \(i=1,2\) and factorization. For proving this, we make use of the following lemma.
Lemma 3.1
Proof
A simple choice is \({\mathcal {C}}({\mathcal {S}}_i^{ext})= {\mathcal {S}}_i^{ext} + \mu {\mathbb {1}}\), where \(-\mu \) is a strict lower bound of the principal curvatures. But in applications surfaces are frequently characterized by strong creases or rather sharp edges, leading to very large \(\mu \). As a consequence, the relative difference of the eigenvalues is significantly reduced when dealing with the resulting curvature classification matrices. Thus, the variational approach is less sensitive to different principal curvatures of the input hypersurfaces.
- Another option is to use a truncation of the absolute value function for the eigenvalues of symmetric matrices. For a symmetric matrix \(B\in \mathbb {R}^{n,n}\) with eigenvalues \(\lambda _1,\ldots , \lambda _n\) and a diagonalization \(B=Q^T \mathrm {diag}(\lambda _1,\ldots , \lambda _n) Q\) we use the classification operatorwhere \(|\lambda |_\tau = \max \{|\lambda |, \tau \}\) for some \(\tau >0\). This approach properly represents the exact shape operator matching objective in case of principal curvatures of equal sign and absolute value larger than \(\tau \). A disadvantage of this construction is that it is not able to force the deformation to correctly match curvature directions on the hypersurface with the same absolute value of the principal curvatures but with different signs. That is, locally a saddle point of the hypersurface may be mistaken for an elliptical point. However, this effect is usually compensated globally, and in applications the ansatz performs well, in particular in matching regions of edges and creases (see Sect. 6).$$\begin{aligned} {\mathcal {C}}(B) = Q^T \mathrm {diag}(|\lambda _1|_\tau ,\ldots , |\lambda _n|_\tau ) Q\,, \end{aligned}$$
3.3 Mismatch Penalty and Volumetric Regularization Energies
3.4 Total Energy
4 Existence of Optimal Matching Deformations
First, we prove the following weak continuity lemma, which is a generalization of the classical result given in [50, Theorem 4.1]. Here, the coefficients may depend on the deformed configuration.
Lemma 4.1
Proof
We are now in a position to prove existence of a minimizing deformation for the hypersurface matching energy E in a suitable set of admissible deformations. Of particular difficulty is that derivatives of \(\mathbf{{d}}_2\) are not defined in the whole of \(\Omega \) and that in the functional these derivatives are evaluated at deformed positions. We handle this by ensuring that the involved deformations are such that terms involving these derivatives are not evaluated near the singularities. We obtain the following theorem:
Theorem 4.2
(Existence of minimizing deformations) Let \(\mathcal {M}_1,\mathcal {M}_2\) be \(C^{2,1}\) compact embedded hypersurfaces in \(\mathbb {R}^n\) such that a \(C^1\) diffeomorphism \(\varphi : \mathcal {M}_1 \rightarrow \mathcal {M}_2\) exists between them.
Proof
We proceed in several steps.
\(E_{\text {vol}}[\phi ]< +\infty \),
\(\det \mathcal {D}\phi (x) > 0\) for a.e. \(x \in \Omega \), and
for all \(x \in {\text {supp}}\,\big ( \eta _\sigma \circ \mathbf{{d}}_1 \big )\) and every \(y \in {\text {sing}}\,(\mathbf{{d}}_2)\), we have \(\left| \phi (x) - y \right| \ge \rho \).
Step 3 Existence of minimizers restricted to admissible deformations Since we have already seen that the set of \(\rho \)-admissible deformations is weakly closed and weakly compact, and that every term of E is weakly lower semicontinuous on this set, we just need to check that for all fixed \(\nu > 0\), the set of \(\rho \)-admissible deformations, with adequate \(\rho \), is not empty.
Step 4 A priori estimate to remove the constraint Next, we show that for any \(\rho \) satisfying (4.12) there exists a parameter \(\nu _0 > 0\) such that for all \(0< \nu < \nu _0\) the constrained minimizers of \(E_\nu \) subject to (4.4) solves the unconstrained optimization problem, consisting in minimizing \(E_\nu \) on \(W_0^{1,p}+{\text {id}}\).
We have particularized the statement of Theorem 4.2 to the case of Dirichlet boundary conditions to ensure global invertibility. In fact, we also have existence of minimizing deformations for the case of Neumann boundary conditions.
Corollary 4.3
Proof
The proof of the estimate for \(\Vert \mathbf{{d}}_2 \circ \phi \Vert _{L^\infty (\{|\mathbf{{d}}_1| \le \sigma \})}\) (to ensure that deformations stay away from the singularities of \(\mathbf{{d}}_2\)) is still valid with minor modifications, since \(\nu \) appears in (4.24) multiplicatively. \(\square \)
We conclude this section with the following proposition, which explores the penalization limit in which the parameter \(\nu \) tends to zero.
Proposition 4.4
Proof
That \(\phi \) is a minimizer of the constrained problem follows directly ([8], Theorem 1.21) from the fact that the \(E_{\nu _k}\) are an equicoercive family of functionals, \(\Gamma \)-converging in the weak topology of \(W^{1,p}\). Indeed, equicoercivity follows easily from the above, while \(\Gamma \)-convergence is implied by the fact that \(E_{\nu _k}\) is an increasing sequence ([8], Remark 1.40), because \(\nu _k \rightarrow 0\) appears as a denominator in \(E_{\text {match}}\). \(\square \)
Remark 4.5
By the coercivity estimate (4.24) of Corollary 4.3, an entirely analogous result holds for minimizers with Neumann boundary conditions.
Remark 4.6
Contrary to what might be expected, the limit problem we have obtained is not a surface problem, since all the level sets are still coupled through the volume energy \(E_{\text {vol}}\). The line of reasoning above depends heavily on the fact that the coefficients of the volume term are held fixed, since the equicoercivity and uniform strict quasiconvexity (in the language of [25]) both require the presence of \(\Vert \mathcal {D}\phi \Vert ^p_{L^p(\Omega )}\) in the functional.
4.1 Oscillations and Lack of Rank-One Convexity for the Naive Approach
Example 4.7
The celebrated Nash–Kuiper theorem [37, 51] states that it is possible to uniformly approximate any short \(C^\infty \) immersion by \(C^1\) isometric ones. Our explicit oscillations around \(r_0 \mathbb {S}^1\) is just one example of this phenomenon. Notice that a bending term of the type \(E_{\text {bend}}\) introduced in our model only compares the curvatures of \(\mathcal {M}_1^{\mathbf{{d}}_1(x)}\) and \(\mathcal {M}_2^{\mathbf{{d}}_2(\phi (x))}\). It therefore does not penalize oscillations, since it does not detect the curvature of \(\phi (\mathcal {M}_1)\) at all.
Example 4.8
(Lack of rank-one convexity) We present an additional example of a configuration for which the integrand of an energy of the type \({\tilde{E}}_{\text {mem}}\) is not rank-one convex. Rank-one convexity of the complete energy density, i.e., , convexity in \(t \in \mathbb {R}\) when composed with the function \(A+tB\) for any matrix A and any rank-one matrix B, is known to be a necessary condition for quasiconvexity ([20], Theorem 5.3). Quasiconvexity, in turn, is necessary for weak lower semicontinuity of integral functionals in Sobolev spaces ([20], Theorem 8.1 and Remark 8.2).
5 Finite Element Discretization Based on Adaptive Octrees
We adopt a ‘discretize, then optimize’ approach and consider a finite element approximation and optimize for the coefficients of the solution. Since the energy \(E_\nu \) is highly nonlinear and nonconvex, we use a cascadic multilevel minimization scheme in which the solution for one grid level is used as the initial data for the minimization on the next finer grid. We use adaptive refinement of the underlying meshes around the surfaces \(\mathcal {M}_1, \mathcal {M}_2 \subset (0,1)^n\) for \(n=2,3\) (Algorithm 1).
Octrees and the access to degrees of freedom via hashtables Even though the tree structure gives a natural hierarchical structure to the elements of the mesh, maintaining consistent linear indices for degrees of freedom, hanging nodes, and elements can be delicate. Consistent rules could be devised to maintain consistency with the element octree for a given mesh, but these would not be easy to update when the grid is refined. In order to keep track of vertex indices in a simple manner without sacrificing efficiency, hash maps ([19], Chapter 11) are maintained to keep track of the indices of degrees of freedom, hanging nodes, and cells. The keys used in the hashmap are a combination of a level value \(\ell \) and point coordinates as integer multiples of \(h=2^{-\ell }\). These keys uniquely identify nodes or elements, with the convention that an element is identified with its lower-left-back corner. Whenever a query for a node or cell is made, there are two possible outcomes. If it is already contained in the corresponding hash table, a linear index for it can be retrieved. Otherwise, a new entry of the hash table is created and the node or cell is given the next unused index. Since we do not require coarsening of the mesh, this scheme guarantees a consistent linear set of indices with a computational cost for insertions and queries that is, on average, independent of the mesh size.
Computing distance functions on octrees In our model, we have assumed that the distance functions to our surfaces are given. In practice, especially when using adaptive grids, we need to compute signed distance functions on such grids. This has been accomplished by a straightforward adaptation of the Fast Marching Method on cartesian grids [57] exploiting the fact that our grids still are subgrids of a regular cartesian grid. In the implemented variant hanging nodes are not taken into account for the propagation, their values being linearly interpolated to accommodate the constraints needed for conformality. The initialization for the distance computation has been performed starting from triangular meshes of the surfaces (for \(n=3\); for \(n=2\) two-bit segmentation of interior and exterior of the curves has been used). The signs of the distance functions have to be computed separately, by detecting which points of the grid are inside (resp. outside) the initial surface data. In our case, they have been computed with the provably correct algorithm given in [2].
Computation of the coefficients The discretization for the unknown deformation \(\phi \), as already mentioned, is done by multilinear finite elements. However, the coefficients of our model include first and second derivatives of the signed distance functions \(\mathbf{{d}}_i\), for the normal vectors \(\mathbf{{n}}_i\) and shape operators \({\mathcal {S}}_i\) (\(i=1,2\)), respectively. The approximations are required to be robust, since they appear in the highest order terms of the model. For the normal vectors \(\mathbf{{n}}_i\), we compute the \(L^2\) projection of the finite element derivative of \(\mathbf{{d}}_i\) to recover the nodal values of a piecewise multilinear function, followed by a orthogonal projection to the unit sphere to restore the constraint \(| \mathbf{{n}}_i | = 1\).
For the computation of matrix square roots and their inverses, we have used the method described in [26], taking appropriate care to truncate almost-singular matrices, since the resulting square roots also appear inverted.
Minimization strategy For the minimization at each level, we have opted for a Fletcher-Reeves nonlinear conjugate gradient method ([52], Section 5.2). The \(L^2\) gradient of \(E_\nu \), whose computation is involved but elementary, was implemented directly. The parameter \(\alpha \) is progressively reduced when a further feasible descent step is not found, according to an Armijo line search ([52], Section 3.1).
6 Numerical Results
All of our results have been computed on the unit cube \(\Omega =[0,1]^3\) for the matching of surfaces in 3D, and the unit square \([0,1]^2\) for the matching of contour curves in 2D. In practice, we have used homogeneous Neumann boundary conditions, since this allows to have relatively large shapes \(\mathcal {M}_i\) in comparison with the size of the domain \(\Omega \) without creating excessive volume energies near the boundary (for the justification we refer to Corollary 4.3). However, if the boundary is not fixed, the deformed domain \(\phi (\Omega )\) is not necessarily contained in \(\Omega \), so evaluation of coefficients on deformed positions has to be appropriately handled numerically. We use a projection of outside position onto the boundary of \(\Omega \) for sufficient large \({\text {dist}}(\mathcal {M}_2, \partial \Omega )\).
In all examples, we have used the identity as the initial deformation. It should be noted that although the energy is geometric by design, we are using a first-order descent method for its minimization. In consequence, an adequate rigid pre-alignment can be beneficial for intricate shapes. Figure 8 shows results for the matching of two different dolphin shapes. Our variational approach is highly nonlinear and nonconvex. Thus, the numerical approximation of the globally optimal deformation depends on the initialization of the deformation. Figure 9 shows that the identity deformation as the initial deformation is advisable only if the expected optimal deformation is not too large. This is demonstrated by applying different rigid body motions to \(\mathcal {M}_1\).
All figures have been produced by deforming the input data (polygonal curve or triangulated surface) via the resulting deformation \(\phi \). This is in contrast to deforming the grid and plotting the resulting extracted level sets (which effectively visualizes the inverse deformation), as commonly done in the registration literature, and also in [32].
Test case First, we present a simple test case to underline the qualitative properties of our model. Figure 5 shows a configuration in which a high amount of compression, combined with rotation, is required. Our model finds the intuitively correct deformation, but oscillations typical for the lack of lower semicontinuity of the underlying energy are induced when \(\text {P}_2\) is not used in the membrane and bending terms. The bending term assists in matching the curvatures even if the deformation is not rigid. Note, however, that for the optimal match the curvature energy \(E_{\text {bend}}\) is not expected to vanish, as can easily be seen from (2.6), (3.4) and the related discussion in Sect. 3.
Parameters and running times on a workstation with a single Intel Xeon E5-1650 CPU (6 cores, 3.2 Ghz)
Figs. | \(\ell _{\text {min}}, \ell _{\text {max}}\) | \(\delta \) | \(c_{\text {vol}}, \nu \) at \(\ell _{\text {min}}\) | Time, \(\ell \le (\ell _{\text {max}}-1)\) | Time, \(\ell = \ell _{\text {max}}\) | DOFs at \(\ell _{\text {max}}\) (K) |
---|---|---|---|---|---|---|
3, 8 | 0.5 | 0.025, 0.002 | 1 h 04 min | 4 h 34 min | 695 | |
2, 8 | 0.71 | 0.05, 0.1 | 30 min 10 s | 1 h 27 min | 313 | |
3, 8 | 1 | 0.025, 0.002 | 20 min 04 s | 50 min 50 s | 179 | |
3, 8 | 0.5 | 0.025, 0.002 | 28 min 56 s | 1 h 25 min | 408 |
Notes
Acknowledgements
Open access funding provided by University of Vienna. This research was supported by the Austrian Science Fund (FWF) through the National Research Network ‘Geometry+Simulation’ (NFN S117) and Doctoral Program ‘Dissipation and Dispersion in Nonlinear PDEs’ (W1245). Furthermore, the authors acknowledge support of the Hausdorff Center for Mathematics at the University of Bonn. We would like to thank the anonymous reviewers for comments that have led to substantial improvements in this paper. The shapes for Fig. 8 are originally from the McGill 3D Shape Benchmark [58]. The scanned faces of Fig. 6 are part of the 3D Basel Face Model dataset [53]. The laser-scanned sugar beets of Fig. 10 and the original shapes for Fig. 7 were kindly provided by Behrend Heeren.
References
- 1.R. A. Adams and J. J. F. Fournier, Sobolev spaces, vol. 140 of Pure and Applied Mathematics (Amsterdam), Elsevier/Academic Press, Amsterdam, second ed., 2003.Google Scholar
- 2.J. A. Baerentzen and H. Aanaes, Signed distance computation using the angle weighted pseudonormal, IEEE Trans. Vis. Comput. Graphics, 11 (2005), pp. 243–253.CrossRefGoogle Scholar
- 3.J. M. Ball, Convexity conditions and existence theorems in nonlinear elasticity, Arch. Ration. Mech. Anal., 63 (1977), pp. 337–403.MathSciNetCrossRefMATHGoogle Scholar
- 4.J. M. Ball, Global invertibility of Sobolev functions and the interpenetration of matter, Proc. Roy. Soc. Edinburgh Sect. A, 88 (1981), pp. 315–328.MathSciNetCrossRefMATHGoogle Scholar
- 5.A. Bonito, R. H. Nochetto, and M. S. Pauletti, Parametric FEM for geometric biomembranes, J. Comput. Phys., 229 (2010), pp. 3171–3188.MathSciNetCrossRefMATHGoogle Scholar
- 6.M. Botsch, M. Pauly, M. Gross, and L. Kobbelt, PriMo: Coupled prisms for intuitive surface modeling, in Proceedings of the Fourth Eurographics Symposium on Geometry Processing, Eurographics Association, 2006, pp. 11–20.Google Scholar
- 7.M. Botsch and O. Sorkine, On linear variational surface deformation methods, IEEE Trans. Vis. Comput. Graph., 14 (2008), pp. 213–230.CrossRefGoogle Scholar
- 8.A. Braides, \(\Gamma \)-convergence for beginners, vol. 22 of Oxford Lecture Series in Mathematics and its Applications, Oxford University Press, Oxford, 2002.Google Scholar
- 9.K. Bredies, T. Pock, and B. Wirth, Convex relaxation of a class of vertex penalizing functionals, J. Math. Imaging Vis., 47 (2013), pp. 278–302.MathSciNetCrossRefMATHGoogle Scholar
- 10.K. Bredies, T. Pock, and B. Wirth, A convex, lower semicontinuous approximation of Euler’s elastica energy, SIAM J. Math. Anal., 47 (2015), pp. 566–613.MathSciNetCrossRefMATHGoogle Scholar
- 11.A. Bronstein, M. Bronstein, and R. Kimmel, Numerical Geometry of Non-Rigid Shapes, Monographs in Computer Science, Springer, 2008.Google Scholar
- 12.M. Burger, J. Modersitzki, and L. Ruthotto, A hyperelastic regularization energy for image registration, SIAM J. Sci. Comput., 35 (2013), pp. B132–B148.MathSciNetCrossRefMATHGoogle Scholar
- 13.I. Chao, U. Pinkall, P. Sanan, and P. Schröder, A simple geometric model for elastic deformations, ACM Trans. Graph., 29 (2010), pp. 38:1–38:6.CrossRefGoogle Scholar
- 14.G. Charpiat, O. Faugeras, and R. Keriven, Approximations of shape metrics and application to shape warping and empirical shape statistics, Found. Comp. Math., 5 (2004), pp. 1–58.MathSciNetCrossRefMATHGoogle Scholar
- 15.Z. Chen and Z. Huan, On the continuity of the m-th root of a continuous nonnegative definite matrix-valued function, J. Math. Anal. Appl., 209 (1997), pp. 60–66.Google Scholar
- 16.P. Ciarlet and G. Geymonat, Sur les lois de comportement en lasticit non linaire compressible, CR Acad. Sci. Paris Sr. II, 295 (1982), pp. 423–426.MATHGoogle Scholar
- 17.P. G. Ciarlet, Mathematical elasticity, volume I: Three-dimensional elasticity, North-Holland, Amsterdam, 1988.Google Scholar
- 18.P. G. Ciarlet, Mathematical elasticity, volume III: Theory of shells, North-Holland, Amsterdam, 2000.Google Scholar
- 19.T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 3rd Edition, The MIT Press, 3rd ed., 2009.Google Scholar
- 20.B. Dacorogna, Direct methods in the calculus of variations, vol. 78 of Applied Mathematical Sciences, Springer, New York, second ed., 2008.Google Scholar
- 21.M. C. Delfour and J.-P. Zolésio, Shape analysis via oriented distance functions, J. Funct. Anal., 123 (1994), pp. 129–201.MathSciNetCrossRefMATHGoogle Scholar
- 22.M. C. Delfour and J.-P. Zolésio, A boundary differential equation for thin shells, J. Differential Equations, 119 (1995), pp. 426–449.MathSciNetCrossRefMATHGoogle Scholar
- 23.M. C. Delfour and J.-P. Zolésio, Shapes and geometries, vol. 22 of Advances in Design and Control, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, second ed., 2011.Google Scholar
- 24.A. Elad and R. Kimmel, On bending invariant signatures for surfaces, IEEE Trans. Pattern Anal. Mach. Intell., 25 (2003), pp. 1285–1295.CrossRefGoogle Scholar
- 25.L. C. Evans and R. F. Gariepy, Some remarks concerning quasiconvexity and strong convergence, Proc. Roy. Soc. Edinburgh Sect. A, 106 (1987), pp. 53–61.MathSciNetCrossRefMATHGoogle Scholar
- 26.L. P. Franca, An algorithm to compute the square root of a \(3\times 3\) positive definite matrix, Comput. Math. Appl., 18 (1989), pp. 459–466.MathSciNetCrossRefMATHGoogle Scholar
- 27.S. Friedland, Variation of tensor powers and spectra, Linear and Multilinear Algebra, 12 (1982/83), pp. 81–98.Google Scholar
- 28.G. Friesecke, R. D. James, M. G. Mora, and S. Müller, Derivation of nonlinear bending theory for shells from three-dimensional nonlinear elasticity by Gamma-convergence, C.R.A.S. Ser. I: Math, 336 (2003), pp. 697 – 702.MathSciNetMATHGoogle Scholar
- 29.M. Fuchs, B. Jüttler, O. Scherzer, and H. Yang, Shape metrics based on elastic deformations, J. Math. Imaging Vis., 35 (2009), pp. 86–102.MathSciNetCrossRefGoogle Scholar
- 30.E. Grinspun, A. N. Hirani, M. Desbrun, and P. Schröder, Discrete shells, in Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, SCA ’03, Eurographics Association, 2003, pp. 62–67.Google Scholar
- 31.V. Guillemin and A. Pollack, Differential topology, Prentice-Hall, Inc., Englewood Cliffs, N.J., 1974.MATHGoogle Scholar
- 32.J. Iglesias, B. Berkels, M. Rumpf, and O. Scherzer, A thin shell approach to the registration of implicit surfaces, in Proceedings of the Vision, Modeling, and Visualization Workshop 2013, Eurographics Association, 2013, pp. 89–96.Google Scholar
- 33.S. Kabus, A. Franz, and B. Fischer, Variational image registration with local properties, in Biomedical Image Registration, J. P. W. Pluim, B. Likar, and F. A. Gerritsen, eds., vol. 4057 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2006, pp. 92–100.Google Scholar
- 34.S. Kabus and C. Lorenz, Fast elastic image registration, in Proceedings of the Medical Image Analysis For The Clinic: A Grand Challenge, MICCAI, Beijing, China, 2010, pp. 81–89.Google Scholar
- 35.S. L. Keeling and W. Ring, Medical image registration and interpolation by optical flow with maximal rigidity, J. Math. Imaging Vision, 23 (2005), pp. 47–65.MathSciNetCrossRefGoogle Scholar
- 36.W. T. Koiter, On the nonlinear theory of elastic shells, Proc. Kon. Ned. Akad. Wetesnch., (1966), pp. 1–54.Google Scholar
- 37.N. H. Kuiper, On \(C^1\)-isometric imbeddings I, Nederl. Akad. Wetensch. Proc. Ser. A., 58 (1955), pp. 545–556.MathSciNetCrossRefMATHGoogle Scholar
- 38.J. Kybic and M. Unser, Fast parametric elastic image registration, IEEE Trans. Image Process., 12 (2003), pp. 1427–1442.CrossRefGoogle Scholar
- 39.H. Le Dret and A. Raoult, The quasiconvex envelope of the Saint Venant-Kirchhoff stored energy function, Proc. Roy. Soc. Edinburgh Sect. A, 125 (1995), pp. 1179–1192.MathSciNetCrossRefMATHGoogle Scholar
- 40.T.-Y. Lee and S.-H. Lai, 3D non-rigid registration for MPU implicit surfaces, in CVPR Workshop on Non-Rigid Shape Analysis and Deformable Image Alignment, 2008.Google Scholar
- 41.G. Leoni, A first course in Sobolev spaces, vol. 105 of Graduate Studies in Mathematics, American Mathematical Society, Providence, RI, 2009.Google Scholar
- 42.X. Li, Y. Bao, X. Guo, M. Jin, X. Gu, and H. Qin, Globally optimal surface mapping for surfaces with arbitrary topology, IEEE Trans. Vis. Comput. Graph., 14 (2008), pp. 805–819.CrossRefGoogle Scholar
- 43.Y. Li and L. Nirenberg, The distance function to the boundary, Finsler geometry, and the singular set of viscosity solutions of some Hamilton-Jacobi equations, Comm. Pure Appl. Math., 58 (2005), pp. 85–146.MathSciNetCrossRefMATHGoogle Scholar
- 44.N. Litke, M. Droske, M. Rumpf, and P. Schröder, An image processing approach to surface matching, in Symposium on Geometry Processing, M. Desbrun and H. Pottmann, eds., 2005, pp. 207–216.Google Scholar
- 45.C. Mantegazza and A. C. Mennucci, Hamilton-Jacobi equations and distance functions on Riemannian manifolds, Appl. Math. Optim., 47 (2003), pp. 1–25.MathSciNetCrossRefMATHGoogle Scholar
- 46.M. Marcus and V. J. Mizel, Absolute continuity on tracks and mappings of Sobolev spaces, Arch. Ration. Mech. Anal., 45 (1972), pp. 294–320.MathSciNetCrossRefMATHGoogle Scholar
- 47.R. E. Megginson, An introduction to Banach space theory, vol. 183 of Graduate Texts in Mathematics, Springer, 1998.Google Scholar
- 48.J. Modersitzki, Numerical Methods for Image Registration, OUP Oxford, 2004.Google Scholar
- 49.D. P. Mukherjee and N. Ray, Contour interpolation using level-set analysis, Int. J. Img. Graph., 12 (2012), p. 1250004.MathSciNetCrossRefGoogle Scholar
- 50.F. Murat, A survey on compensated compactness, in Contributions to modern calculus of variations (Bologna, 1985), vol. 148 of Pitman Res. Notes Math. Ser., Longman Sci. Tech., Harlow, 1987, pp. 145–183.Google Scholar
- 51.J. Nash, \(C^1\)-isometric imbeddings, Ann. Math., 60 (1954), pp. 383–396.MathSciNetCrossRefMATHGoogle Scholar
- 52.J. Nocedal and S. Wright, Numerical Optimization, Springer, second ed., 2006.Google Scholar
- 53.P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter, A 3D face model for pose and illumination invariant face recognition, in Proc. Advanced Video and Signal based Surveillance, 2009.Google Scholar
- 54.M. Renardy and R. Rogers, An Introduction to Partial Differential Equations, vol. 13 of Texts in Applied Mathematics, Springer, second ed., 2004.Google Scholar
- 55.P. Risholm, E. Samset, and W. Wells III, Bayesian estimation of deformation and elastic parameters in non-rigid registration, in Biomedical Image Registration, B. Fischer, B. M. Dawant, and C. Lorenz, eds., vol. 6204 of Lecture Notes in Computer Science, Springer, 2010, pp. 104–115.Google Scholar
- 56.M. Rumpf and B. Wirth, A nonlinear elastic shape averaging approach, SIAM J. Imaging Sci., 2 (2009), pp. 800–833.MathSciNetCrossRefMATHGoogle Scholar
- 57.J. A. Sethian, Level set methods and fast marching methods, Cambridge University Press, second ed., 1999.Google Scholar
- 58.K. Siddiqi, J. Zhang, D. Macrini, A. Shokoufandeh, S. Bouix, and S. Dickinson, Retrieving articulated 3-D models using medial surfaces, Machine Vision and Applications, 19 (2008), pp. 261–275.CrossRefMATHGoogle Scholar
- 59.A. Srivastava, C. Samir, S. H. Joshi, and M. Daoudi, Elastic shape models for face analysis using curvilinear coordinates, J. Math. Imaging. Vis., 33 (2009), pp. 253–265.MathSciNetCrossRefGoogle Scholar
- 60.S. Wang, Y. Wang, M. Jin, X. Gu, and D. Samaras, 3d surface matching and recognition using conformal geometry, in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, IEEE, 2006, pp. 2453–2460.Google Scholar
- 61.T. Windheuser, U. Schlickewei, F. R. Schmidt, and D. Cremers, Geometrically consistent elastic matching of 3D shapes: A linear programming solution, in International Conference on Computer Vision, 2011, pp. 2134–2141.Google Scholar
- 62.D. Zhang and M. Hebert, Harmonic maps and their applications in surface matching, in 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’99), vol. 2, IEEE, 1999, pp. 524–530.Google Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.