Optical Flow on Evolving Surfaces with Space and Time Regularisation

Abstract

We extend the concept of optical flow with spatiotemporal regularisation to a dynamic non-Euclidean setting. Optical flow is traditionally computed from a sequence of flat images. The purpose of this paper is to introduce variational motion estimation for images that are defined on an evolving surface. Volumetric microscopy images depicting a live zebrafish embryo serve as both biological motivation and test data.

Appendix

We first sketch a proof about the statement from Sect. 2.1 that the normal velocity of an evolving surface is independent of $\phi$.

Proposition 1

Let $\phi$ be a Langrangian specification of $\mathcal {M}$ and $\mathbf{V}$ the corresponding velocity as defined in (3). Then $\mathbf{V} \cdot \mathbf{N}$ is independent of the chosen specification.

Proof

We can represent $\bar{\mathcal {M}}$ locally as the level set of a real-valued function $G(t,x)$, whose gradient does not vanish, see e.g. [20, Prop. 5.16]. We now express $\mathbf{V} \cdot \mathbf{N}$ solely in terms of $G$ and thereby prove the assertion. Observing that the composition of $G$ with $\phi$ is constant, we calculate

$$\begin{aligned} 0 = \frac{\hbox {d}}{\hbox {d} t} G(t,\phi (t,x_0)) = \partial _t G + \nabla _{\mathbb {R}^3} G \cdot \mathbf{V} = \partial _t G + |\nabla _{\mathbb {R}^3} G| \mathbf{V} \cdot \mathbf{N}. \end{aligned}$$

The second equality holds, because $\nabla _{\mathbb {R}^3} G$ is normal to the surface. We conclude that

$$\begin{aligned} \mathbf{V} \cdot \mathbf{N} = - \frac{\partial _t G}{|\nabla _{\mathbb {R}^3} G|}. \end{aligned}$$

$\square$

In other words, different specifications of a surface can only differ in their respective tangential velocities.

Next we prove the transformation law (11), (20) for the connection coefficients $\tilde{\varGamma }^j_{\mu j}$.

Lemma 3

The symbols defined by (10) are given by (11).

Proof

Take inner products on both sides of (11) with $\mathbf{e}_j$ to get

$$\begin{aligned} \mathbf{e}_j \cdot \nabla _{\mathbf{e}_i} \mathbf{e}_k = \tilde{\varGamma }^j_{ik}. \end{aligned}$$

Next express both terms on the left hand side in the coordinate basis by using $\mathbf{e}_j = \alpha _j^m \partial _m \mathbf{x}$ and formula (7). The assertion follows now immediately. $\square$

An analogous calculation yields formula (20).

For our implementation the Euler–Lagrange equations (29) are needed in the following form

$$\begin{aligned} \begin{aligned}&d^{\nu \sigma } \partial _{\nu \sigma }w^m + c^{\sigma m}_{i} \partial _\sigma w^i + b^m_{i} w^i = a^m , \quad \text {in } D, \\&q^{\nu \sigma } \partial _\sigma w^m + p_{i}^{\nu m} w^i = 0, \quad \text {on } \{\xi ^\nu = 0\} \cup \{\xi ^\nu = 1\}, \end{aligned} \end{aligned}$$

(33)

where we assumed $D=(0,1)^3$. As usual the system is to be understood for $m=1,2$ and $\nu =0,1,2$. Below we give the exact coefficients.

$$\begin{aligned} \begin{aligned} a^m&= -\alpha _m^i \partial _i f \partial _t f\\ b^m_{i}&= \alpha _m^j \alpha _i^k \partial _j f \partial _k f \\&\quad + \sum \limits _{\mu } \lambda _\mu \Bigg ( \sum _{j}\tilde{\varGamma }^j_{\mu m} \tilde{\varGamma }^j_{\mu i} \!-\! G_\nu \alpha ^\nu _\mu \tilde{\varGamma }^m_{\mu i} \!+\! \partial _\nu \Big ( \alpha ^\nu _\mu \tilde{\varGamma }^m_{\mu i} \Big ) \Bigg )\\ c^{\sigma m}_{i}&= \sum \limits _{\mu } \lambda _\mu \Bigg ( \alpha ^\sigma _\mu \tilde{\varGamma }^i_{\mu m} - \alpha ^\sigma _\mu \tilde{\varGamma }^m_{\mu i} \\&\quad -\delta _{im} \left( G_\nu \alpha ^\nu _\mu \alpha ^\sigma _\mu + \partial _\nu ( \alpha ^\nu _\mu \alpha ^\sigma _\mu ) \right) \Bigg )\\ d^{\nu \sigma }&= -\sum \limits _{\mu } \lambda _\mu \alpha ^\nu _\mu \alpha ^\sigma _\mu \\ p_{i}^{\nu m}&= \sum \limits _{\mu } \lambda _\mu \alpha ^\nu _\mu \tilde{\varGamma }^m_{\mu i} \\ q^{\nu \sigma }&= \sum \limits _{\mu } \lambda _\mu \alpha ^\nu _\mu \alpha ^\sigma _\mu \end{aligned} \end{aligned}$$

(34)

Here we used the shorthand

$$\begin{aligned} G_\nu = \frac{\partial _\nu \sqrt{ \det g}}{2 \sqrt{ \det g}}. \end{aligned}$$

Recall that the functional without time regularisation (25) leads to a sequence of decoupled systems for every instant $t$. Each of those has the form

$$\begin{aligned} \begin{aligned}&d^{jk} \partial _{jk}w^m + c^{k m}_{i} \partial _k w^i + b^m_{i} w^i = a^m , \qquad \text {in } D,\\&q^{jk} \partial _k w^m + p_{i}^{ mj} w^i = 0, \qquad \text {on } \{\xi ^j = 0\} \cup \{\xi ^j = 1\}. \end{aligned} \end{aligned}$$

Note that, in comparison to system (33), we only replaced Greek indices by Latin ones. The coefficients $a,b,c,d,p,q$ of this simpler system can be obtained from the list above by setting $\lambda _0 = 0$.