Learning the spatiotemporal variability in longitudinal shape data sets

Abstract

In this paper, we propose a generative statistical model to learn the spatiotemporal variability in longitudinal shape data sets, which contain repeated observations of a set of objects or individuals over time. From all the short-term sequences of individual data, the method estimates a long-term normative scenario of shape changes and a tubular coordinate system around this trajectory. Each individual data sequence is therefore (i) mapped onto a specific portion of the trajectory accounting for differences in pace of progression across individuals, and (ii) shifted in the shape space to account for intrinsic shape differences across individuals that are independent of the progression of the observed process. The parameters of the model are estimated using a stochastic approximation of the expectation–maximization algorithm. The proposed approach is validated on a simulated data set, illustrated on the analysis of facial expression in video sequences, and applied to the modeling of the progressive atrophy of the hippocampus in Alzheimer’s disease patients. These experiments show that one can use the method to reconstruct data at the precision of the noise, to highlight significant factors that may modulate the progression, and to simulate entirely synthetic longitudinal data sets reproducing the variability of the observed process.

Meshes represented as currents

The theory of currents has been introduced in Vaillant and Glaunès (2005), and is used in this paper to define a distance metric between pairs of meshes without any assumption on their topology, and in particular without assuming point-to-point correspondence. See also Charon et al. (2020) for more details.

1.1 A.1 Continuous theory

Let y be a surface mesh, that we represent as an infinite set of tuples (x, n(x)) where x is a point of $\mathbb {R}^3$, and n(x) the normal vector of y at this point. Let $g_\mathcal {E}: \mathbb {R}^3 \times \mathbb {R}^3 \rightarrow \mathbb {R}$ be a positive-definite kernel operator, and $\mathcal {E}$ the associated reproducing kernel Hilbert space.

We define the current transform $\mathcal {C}(y):\mathbb {R}^3\rightarrow \mathbb {R}^3 \in \mathcal {E}$ of y as:

$$\begin{aligned} \mathcal {C}(y)(.) = \int _y g_\mathcal {E}(x, .) \cdot n(x) \cdot d\sigma (x) \end{aligned}$$

where $d \sigma (x)$ denotes an infinitesimal surface element of y. The inner product of $\mathcal {E}$ on currents therefore writes:

$$\begin{aligned} \big \langle \mathcal {C}(y), \mathcal {C}(y') \big \rangle _\mathcal {E}{=} \int _y \int _{y'} n'(x')^\top \cdot g_\mathcal {E}(x, x') \cdot n(x) \cdot \sigma (x) \cdot d\sigma '(x') \end{aligned}$$

where $(.)^\top$ is the transposition operator. This inner product defines in turn a distance metric on currents:

$$\begin{aligned} d_\mathcal {E}(\mathcal {C}, \mathcal {C}') = \big \langle \mathcal {C}, \mathcal {C} \big \rangle _\mathcal {E}+ \big \langle \mathcal {C}', \mathcal {C}' \big \rangle _\mathcal {E}- 2\cdot \big \langle \mathcal {C}, \mathcal {C}' \big \rangle _\mathcal {E}. \end{aligned}$$

1.2 A.2 Practical discrete case

In practice, y is described by a finite set of T triangles in $\mathbb {R}^3$ of centers $c_1, \ldots , c_T$ and corresponding surface normal vectors $n_1,\ldots , n_T$. We further assume that $g_\mathcal {E}$ is a Gaussian kernel of radius $\sigma _\mathcal {E}$. The current transform equation then writes:

$$\begin{aligned} \mathcal {C}(y)(x) = \sum _{k=1}^T \exp \frac{- \left\| x - c_k\right\| ^2_{\ell ^2}}{\sigma _\mathcal {E}^2}\cdot n_k \end{aligned}$$

for any $x\in \mathbb {R}^3$. Similarly, the inner product formula becomes:

$$\begin{aligned} \Big \langle \mathcal {C}(y), \mathcal {C}(y') \Big \rangle _\mathcal {E}= \sum _{k=1}^T \sum _{l=1}^{T'} \exp \frac{- \left\| c_l' - c_k\right\| ^2_{\ell ^2}}{\sigma _\mathcal {E}^2} \cdot n_k^\top n_l' \end{aligned}$$

which fully specifies the distance metric $d_\mathcal {E}$ that can be implemented in practice to measure the discrepancy between any pair of currents.