Total roto-translational variation

Abstract

We consider curvature depending variational models for image regularization, such as Euler’s elastica. These models are known to provide strong priors for the continuity of edges and hence have important applications in shape- and image processing. We consider a lifted convex representation of these models in the roto-translation space: in this space, curvature depending variational energies are represented by means of a convex functional defined on divergence free vector fields. The line energies are then easily extended to any scalar function. It yields a natural generalization of the total variation to curvature-dependent energies. As our main result, we show that the proposed convex representation is tight for characteristic functions of smooth shapes. We also discuss cases where this representation fails. For numerical solution, we propose a staggered grid discretization based on an averaged Raviart–Thomas finite elements approximation. This discretization is consistent, up to minor details, with the underlying continuous model. The resulting non-smooth convex optimization problem is solved using a first-order primal-dual algorithm. We illustrate the results of our numerical algorithm on various problems from shape- and image processing.

Appendices

Appendix A: Consistency of the discretization

In this appendix, we study the consistency of the discrete approximation of the problem which is used in Sect. 4.

1.1 A.1 Preliminary results

Let us introduce, for $(s,t)\in \mathbb {R}^2$,

$$\begin{aligned} \bar{h}(s,t) = {\left\{ \begin{array}{ll} s f(t/s) &{} \quad \mathrm{if }\quad s>0,\\ f^\infty (t) &{} \quad \mathrm{if }\quad s=0,\\ +\infty &{} \quad \mathrm{else,} \end{array}\right. } \end{aligned}$$

which is such that

$$\begin{aligned} \bar{h}(s,t) = \sup _{a+f^*(b)\le 0} as+bt. \end{aligned}$$

Observe that if the convex function f is differentiable, then for $s>0$, $\partial _s \bar{h}(s,t)=f(t/s)-(t/s)f'(t/s)\le f(0)$.

Consider a vector-valued measurable function $\sigma (\theta ) =(\lambda (\theta )\underline{\theta },\mu (\theta ))$ where $\lambda \ge 0$. Let then $\bar{\theta }\in \mathbb {S}^1$, $\delta _\theta >0$ and

$$\begin{aligned} \bar{\sigma } =\frac{1}{\delta _\theta }\int _{\bar{\theta }-\frac{\delta _\theta }{2}}^{\bar{\theta }+\frac{\delta _\theta }{2}} \sigma (\theta )d\theta . \end{aligned}$$

We first observe that

$$\begin{aligned} \bar{\sigma }\cdot \bar{\underline{\theta }}=\frac{1}{\delta _\theta }\int _{\bar{\theta }-\frac{\delta _\theta }{2}}^{\bar{\theta }+\frac{\delta _\theta }{2}} \lambda (\theta )\underline{\theta }\cdot \bar{\underline{\theta }} d\theta \in \left[ \bar{\lambda } \cos \tfrac{\delta _\theta }{2},\bar{\lambda }\right] \end{aligned}$$

where

$$\begin{aligned} \bar{\lambda }=\frac{1}{\delta _\theta }\int _{\bar{\theta }-\frac{\delta _\theta }{2}}^{\bar{\theta }+\frac{\delta _\theta }{2}} \lambda (\theta )d\theta . \end{aligned}$$

Assuming f is smooth, it follows that

$$\begin{aligned} \bar{h} \Big (\tfrac{\bar{\sigma }\cdot \bar{\underline{\theta }}}{\cos \frac{\delta _\theta }{2}},\bar{\sigma }^\theta \Big )&= \bar{h}(\bar{\lambda },\bar{\sigma }^\theta ) +\int _0^1 \partial _s \bar{h} \Big (\bar{\lambda }+s\Big (\tfrac{\bar{\sigma }\cdot \bar{\underline{\theta }}}{\cos \frac{\delta _\theta }{2}}-\bar{\lambda }\Big ),\bar{\sigma }^\theta \Big ) \Big (\tfrac{\bar{\sigma }\cdot \bar{\underline{\theta }}}{\cos \frac{\delta _\theta }{2}}-\bar{\lambda }\Big ) ds\\&\le \bar{h}(\bar{\lambda },\bar{\sigma }^\theta ) + f(0)\bar{\lambda }\frac{1-\cos \frac{\delta _\theta }{2}}{\cos \frac{\delta _\theta }{2}}. \end{aligned}$$

Using that $\bar{h}$ is 1-homogeneous and that $\bar{h}(s,t)\ge \gamma \sqrt{s^2+t^2}$, it follows

$$\begin{aligned} \bar{h} \big (\bar{\sigma }\cdot \bar{\underline{\theta }},\cos \tfrac{\delta _\theta }{2}\bar{\sigma }^\theta \big )&\le \cos \tfrac{\delta _\theta }{2} \bar{h}(\bar{\lambda },\bar{\sigma }^\theta ) + \big (1-\cos \tfrac{\delta _\theta }{2}\big ) \frac{f(0)}{\gamma } \bar{h}(\bar{\lambda },\bar{\sigma }^\theta ) \\&\le \left( 1+\frac{\delta _\theta ^2}{4}(f(0)/\gamma -1)\right) \bar{h}(\bar{\lambda },\bar{\sigma }^\theta ) \end{aligned}$$

if $\delta _\theta$ is small enough. Observe that if f is not smooth, this still holds by approximation. It follows, thanks to Jensen’s inequality and the fact that $h(\theta ,\sigma )=\bar{h}(\lambda ,\sigma ^\theta )$ for all $\theta$, that provided $\delta _\theta$ is small enough,

$$\begin{aligned} \bar{h} \big (\bar{\sigma }\cdot \bar{\underline{\theta }},\cos \tfrac{\delta _\theta }{2}\bar{\sigma }^\theta \big ) \le (1+C{\delta _\theta ^2})\frac{1}{\delta _\theta } \int _{\bar{\theta }-\frac{\delta _\theta }{2}}^{\bar{\theta }+\frac{\delta _\theta }{2}} h(\theta ,\sigma (\theta ))d\theta , \end{aligned}$$

(44)

where the constant $C=(f(0)/\gamma -1)/4$ only depends on the function f.

Remark A.1

If $\sigma$ is a bounded measure such that $\int h(\theta ,\sigma )<+\infty$, then (44) still holds with now all integrals and averages replaced with integrals over $[\bar{\theta }-\delta _\theta /2,\bar{\theta }+\delta _\theta /2)$. Indeed, in this case, approximating $\sigma$ with smooth measures by convolution one easily deduces that it holds for almost all $\bar{\theta }$ (whenever $|\sigma |(\{\bar{\theta }-\delta _\theta /2,\bar{\theta }+\delta _\theta /2\})=0$). Then, for the other values, it is enough to find a sequence $\varepsilon _n>0$ with $\varepsilon _n\downarrow 0$ such that the result holds for $\bar{\theta }-\varepsilon _n$ and use the fact that for any bounded measure $\mu$,

$$\begin{aligned} \lim _{n\rightarrow \infty } \mu ([\bar{\theta }-\varepsilon _n-\tfrac{\delta _\theta }{2},\bar{\theta }-\varepsilon _n+\tfrac{\delta _\theta }{2}))= \mu ([\bar{\theta }-\tfrac{\delta _\theta }{2},\bar{\theta }+\tfrac{\delta _\theta }{2})) \end{aligned}$$

as $\mu ([\theta -\varepsilon _n,\theta ))\rightarrow 0$ as $n\rightarrow \infty$ for any $\theta \in \mathbb {S}^1$.

We can now show the following lemma:

Lemma A.2

Let $\delta _x,\delta _\theta >0$ small enough, $\sigma \in \mathcal {M}^1({\Omega _{}\times \mathbb {S}^1};\mathbb {R}^3)$ with $\text {div}\,\sigma =0$ and such that $\int _{{\Omega _{}\times \mathbb {S}^1}} h(\theta ,\sigma )<\infty$. Define, for $(\bar{x},\bar{\theta })\in {\Omega _{}\times \mathbb {S}^1}$,

$$\begin{aligned} S=\left[ \bar{x}_1-\tfrac{\delta _x}{2},\bar{x}_1+\tfrac{\delta _x}{2}\right) \times \left[ \bar{x}_2-\tfrac{\delta _x}{2},\bar{x}_2+\tfrac{\delta _x}{2}\right) \end{aligned}$$

and

$$\begin{aligned} \bar{\sigma } = \frac{1}{\delta _x^2\delta _\theta }\sigma (S\times [\bar{\theta }-\tfrac{\delta _\theta }{2},\bar{\theta }+\tfrac{\delta _\theta }{2})). \end{aligned}$$

Then,

$$\begin{aligned} \bar{h}(\bar{\sigma }\cdot \bar{\underline{\theta }},\cos \tfrac{\delta _\theta }{2}\bar{\sigma }^\theta ) \le \frac{1+C\delta _\theta ^2}{\delta _x^2\delta _\theta } \int _{S\times [\bar{\theta }-\frac{\delta _\theta }{2},\bar{\theta }+\frac{\delta _\theta }{2})} h(\theta ,\sigma ) \end{aligned}$$

(45)

where C depends only on f. Moreover, $\bar{\sigma }^x/|\bar{\sigma }|$ lies in the cone $\mathbb {R}_+(\underline{\theta -\delta _\theta /2})+\mathbb {R}_+(\underline{\theta +\delta _\theta /2})$.

Proof

If we introduce the (averaged) marginal $\sigma '\in \mathcal {M}^1(\mathbb {S}^1;\theta )$ defined by $\int _{\mathbb {S}^1}\psi \sigma ' = \frac{1}{\delta _x^2}\int _{S\times \mathbb {S}^1}\psi \sigma$ for all $\psi \in C^0(\mathbb {S}^1)$, then one observes that

$$\begin{aligned} \bar{\sigma } = \frac{1}{\delta _\theta }\sigma '({[\bar{\theta }-\tfrac{\delta _\theta }{2},\bar{\theta }+\tfrac{\delta _\theta }{2})}) \end{aligned}$$

and

$$\begin{aligned} \int _{[\bar{\theta }-\frac{\delta _\theta }{2},\bar{\theta }+\frac{\delta _\theta }{2})} h(\theta ,\sigma ') \le \int _{S\times [\bar{\theta }-\frac{\delta _\theta }{2},\bar{\theta }+\frac{\delta _\theta }{2})} h(\theta ,\sigma ). \end{aligned}$$

This follows by a disintegration argument and using Jensen’s inequality in each “slice” corresponding to a fixed value of $\theta$. The result then follows from (44), together with Remark A.1.

The last statement comes from the fact that $\sigma _x$ is the average of measures all contained in the cone, which is convex. $\square$

Consider now a measure $\sigma$ admissible for some function u, and assume that $\sigma$ is “smooth” in x: we assume for instance that it is the result of a convolution $\rho _\varepsilon *\sigma '$ for some $\sigma '$ admissible (possibly extended in a larger domain), with $\rho _\varepsilon (x)$ a rotationally symmetric mollifier, as in the Proof of Proposition 3.1. In this case, $x\mapsto \sigma$ can be seen as a $\mathcal {M}^1(\mathbb {S}^1;\mathbb {R}^3)$-valued smooth function.

Consider $(\bar{x},\bar{\theta })\in {\Omega _{}\times \mathbb {S}^1}$, $\delta _x,\delta _\theta$ small, and define in the volume $V=S\times [\bar{\theta }-\delta _\theta /2,\bar{\theta } +\delta _\theta /2)$ the average $\bar{\sigma }$ as before and the “Raviart–Thomas” approximation of $\sigma$ defined by the average fluxes through the 6 facets of V (linearly extended inside the volume, as in Eq. 21). There are several ways to define this properly, at least for all $\bar{\theta }$ but a countable number. In our case, one can disintegrate the measure in ${\Omega _{}\times \mathbb {S}^1}$ as $\sigma =\sigma _\theta d\mu$ where $\mu$ is a bounded positive measure in $\mathbb {S}^1$ and for all $\theta$, $\sigma _\theta \in C^\infty (\Omega )$. Then

$$\begin{aligned} \tau ^1_\pm = \frac{1}{\delta _x\delta _\theta }\int _{[\bar{\theta }-\delta _\theta /2,\bar{\theta } +\delta _\theta /2)}\left( \int _{\bar{x}_2-\frac{\delta _x}{2}}^{\bar{x}_2+\frac{\delta _x}{2}} \sigma ^1_\theta (\bar{x}_1\pm \tfrac{\delta _x}{2},x_2,\theta )dx_2 \right) \mu , \end{aligned}$$

$$\begin{aligned} \tau ^2_\pm = \frac{1}{\delta _x\delta _\theta }\int _{[\bar{\theta }-\delta _\theta /2,\bar{\theta } +\delta _\theta /2)}\left( \int _{\bar{x}_1-\frac{\delta _x}{2}}^{\bar{x}_1+\frac{\delta _x}{2}} \sigma ^1_\theta (x_1,\bar{x}_2\pm \tfrac{\delta _x}{2},\theta )dx_1 \right) \mu . \end{aligned}$$

To define the vertical fluxes $\tau ^\theta _\pm$ we assume in addition that $\mu (\{\bar{\theta }\pm \delta _\theta /2\})=0$ (which is true for all values but a countable number). In this case, observe that if $\phi \in C_c^1(\mathring{S}\times \{\theta =-\delta _\theta /2\})$, it can be extended into a $C^1$ function in V vanishing near the 5 other boundaries, and then

$$\begin{aligned} \int _V \nabla \phi \cdot \sigma \end{aligned}$$

defines a measure $\tilde{\tau }^\theta _-$ on $\mathring{S}\times \{\theta =-\delta _\theta /2\}$. Then one simply let $\tau ^\theta _- = \tilde{\tau }^\theta _-(\mathring{S}\times \{\theta =-\delta _\theta /2\})/\delta _x^2$. The value $\tau ^\theta _+$ is defined in the same way. The assumption that $\mu (\{\bar{\theta }-\delta _\theta /2\})=0$ guarantees that the same construction from below will build the same measure and the same value, and that one actually has $(\tau ^1_+-\tau ^1_-+\tau ^2_+-\tau ^2_-)/\delta _x+(\tau ^\theta _+-\tau ^\theta _-)/\delta _\theta =0$.

We can show the following lemma.

Lemma A.3

Let $\tau = (\tau ^1_a,\tau ^2_b,\tau ^\theta _c)^T$ for any $(a,b,c)\in \{-,+\}^3$. Then, for all $\bar{\theta }$ but a countable number,

$$\begin{aligned} \delta _x^2\delta _\theta |\tau -\bar{\sigma }| \le \sqrt{\delta _x^2+\delta _\theta ^2}\int _{V} |\partial _1\sigma ^1|+|\partial _2\sigma ^2|. \end{aligned}$$

(46)

Proof

We prove the result for $(a,b,c)=(-,-,-)$, the proof in the other cases being identical. We first assume that $\sigma$ is also $C^1$ in $\theta$ (which can be achieved by convolution). In this case, one has for all $(x_1,x_2,\theta )\in V$,

$$\begin{aligned} \sigma ^1(\bar{x}_1-\tfrac{\delta _x}{2},x_2,\theta )= \sigma ^1(x_1,x_2,\theta ) - \int _{\bar{x}_1-\frac{\delta _x}{2}}^{x_1} \partial _1 \sigma ^1 (s,x_2,\theta ) ds \end{aligned}$$

so that

$$\begin{aligned}&\delta _x\sigma ^1(\bar{x}_1-\tfrac{\delta _x}{2},x_2,\theta )\\&\quad = \int _{\bar{x}_1-\frac{\delta _x}{2}}^{\bar{x}_1+\frac{\delta _x}{2}}\sigma ^1(x_1,x_2,\theta ) - \int _{\bar{x}_1-\frac{\delta _x}{2}}^{\bar{x}_1+\frac{\delta _x}{2}} (\bar{x}_1+\tfrac{\delta _x}{2}-s) \partial _1 \sigma ^1 (s,x_2,\theta ) ds \end{aligned}$$

Averaging over $x_2,\theta$, we deduce that

$$\begin{aligned} \tau ^1_- = \bar{\sigma }^1 - \frac{1}{\delta _x^2\delta _\theta }\int _{V} (\bar{x}_1+\tfrac{\delta _x}{2}-x_1) \partial _1 \sigma ^1 dx_1dx_2d\theta . \end{aligned}$$

In the same way,

$$\begin{aligned} \tau ^2_- = \bar{\sigma }^2 - \frac{1}{\delta _x^2\delta _\theta }\int _{V} (\bar{x}_2+\tfrac{\delta _x}{2}-x_2) \partial _2 \sigma ^2 dx_1dx_2d\theta , \end{aligned}$$

$$\begin{aligned} \tau ^\theta _- = \bar{\sigma }^\theta - \frac{1}{\delta _x^2\delta _\theta }\int _{V} (\bar{\theta }+\tfrac{\delta _\theta }{2}-\theta ) \partial _\theta \sigma ^\theta dx_1dx_2d\theta . \end{aligned}$$

Using that $\text {div}\,\sigma =0$, the latter can be rewritten

$$\begin{aligned} \tau ^\theta _- = \bar{\sigma }^\theta +\frac{1}{\delta _x^2\delta _\theta }\int _{V} (\bar{\theta }+\tfrac{\delta _\theta }{2}-\theta ) \text {div}\,_x\sigma ^x dx_1dx_2d\theta . \end{aligned}$$

The estimate (46) follows. If $\sigma$ is not $C^1$ in $\theta$, as before we can smooth $\sigma$, then in the limit we will obtain (46) for all $\bar{\theta }$ such that $\mu (\{\bar{\theta }\pm \delta _\theta /2\})=0$. $\square$

Corollary A.4

Let $\tau (x,\theta )$ be the Raviart–Thomas extension of the fluxes $\tau ^\bullet _\pm$ in V: then it holds

$$\begin{aligned} \int _V|\tau -\bar{\sigma }|dxd\theta \le \sqrt{\delta _x^2+\delta _\theta ^2}\int _{V} |\partial _1\sigma ^1|+|\partial _2\sigma ^2| \end{aligned}$$

(47)

(for the same values of $\bar{\theta }$).

This is proven in the same way, as inside V, $\tau ^\bullet (x,\theta )$ is a convex combination of the two fluxes $\tau ^\bullet _\pm$. Moreover, by construction since $\text {div}\,\sigma =0$, it is easy to check that one also has $\text {div}\,\tau =0$. The following is also immediate:

Corollary A.5

Let $\tau (x,\theta )$ be the Raviart–Thomas extension of the fluxes $\tau ^\bullet _\pm$ in V and $\bar{\tau }$ the value in the middle of the cell (in other words,

$$\begin{aligned} \bar{\tau } = \begin{pmatrix} \frac{\tau ^1_- + \tau ^1_+}{2} \\ \frac{\tau ^2_- + \tau ^2_+}{2} \\ \frac{\tau ^\theta _- + \tau ^\theta _+}{2} \end{pmatrix} \end{aligned}$$

is given by the average of the fluxes through the facets of V). Then

$$\begin{aligned} |V||\bar{\tau }-\bar{\sigma }| \le \sqrt{\delta _x^2+\delta _\theta ^2}\int _{V} |\partial _1\sigma ^1|+|\partial _2\sigma ^2|. \end{aligned}$$

(48)

1.2 A.2 Consistent discretization of the energy F

We now are in a position to define almost consistent approximations of F. We will build a discrete approximation which enjoys a sort of discrete-to-continuum $\Gamma$-convergence property to the limiting functional F.

Assume to simplify $\Omega$ is a convex set,² and even a rectangle $[0,a]\times [0,b]$, $a,b>0$, which further simplifies our notation.

Let $u\in BV(\Omega )$ and $\sigma _u$ be admissible for u, such that $F(u)=\int _{{\Omega _{}\times \mathbb {S}^1}} h(\theta ,\sigma _u)<\infty$ and first, for $\varepsilon >0$ fixed, $\sigma _\varepsilon$ by convolution as in the Proof of Proposition 3.1. In particular,

$$\begin{aligned} \int _{{\Omega _{}\times \mathbb {S}^1}} |\partial _1\sigma ^1_\varepsilon |+|\partial _2\sigma ^2_\varepsilon | \le \frac{c}{\varepsilon }\int _{{\Omega _{}\times \mathbb {S}^1}} |\sigma ^1_u|+|\sigma ^2_u| \end{aligned}$$

where $c=2\pi \int _{B_1}|\nabla \rho |dx$ depends only on the convolution kernel $\rho$. Fix $\delta _x,\delta _\theta$ small enough, assume $\delta _\theta =2\pi /N_\theta$ for some integer $N_\theta$, and consider all the volumes $V_{i,j,k}$, defined in (16) [with $S_{i,j}$ defined by (15)], and which are inside ${\Omega _{}\times \mathbb {S}^1}$, for $(i,j,k)\in \mathcal {J}$ (17). We define a Raviart–Thomas vector field from the (averaged) fluxes of $\sigma _\varepsilon$ through the facets of the volumes: $\sigma ^1_{i-\frac{1}{2},j,k}$ through the facets $\mathscr {F}^1_{i-\frac{1}{2},j,k}$, $\sigma ^2_{i,j-\frac{1}{2},k}$ through the facets $\mathscr {F}^2_{i,j-\frac{1}{2},k}$, and $\sigma ^\theta _{i,j,k-\frac{1}{2}}$ through $\mathscr {F}^\theta _{i,j,k-\frac{1}{2}}$, see (18), (19), (20). The latter flux is well-defined up to an infinitesimal vertical translation of the origin of the discretization in $\theta$ (without loss of generality we thus assume it is well defined). The Raviart–Thomas field inside the cube is defined then as in (21). We also define $\bar{\sigma }_{i,j,k}=(\delta _x^{-2}\delta _\theta ^{-1})\int _{V_{i,j,k}}\sigma _\varepsilon$ as the average of $\sigma _\varepsilon$ in $V_{i,j,k}$, and let $\hat{\sigma }_{i,j,k}$ be defined by (24), which corresponds to averaging the fluxes of the facets, or equivalently to consider the value of the Raviart–Thomas extension in the middle of the volume $V_{i,j,k}$.

From Corollary A.5, letting

$$\begin{aligned} e_{i,j,k} = \hat{\sigma }_{i,j,k}-\bar{\sigma }_{i,j,k}, \end{aligned}$$

one has

$$\begin{aligned} \delta _x^2\delta _\theta \sum _{i,j,k} |e_{i,j,k}| \le c\frac{\sqrt{\delta _x^2+\delta _\theta ^2}}{\varepsilon }\int _{{\Omega _{}\times \mathbb {S}^1}}|\sigma ^x_u|. \end{aligned}$$

Moreover by Lemma A.2, one has (we denote, for every k, $\theta _{k}=k \delta _\theta$ and $\theta _{k+\frac{1}{2}}=(k+\frac{1}{2}) \delta _\theta$)

$$\begin{aligned} \delta _x^2\delta _\theta \sum _{i,j,k} \bar{h}(\bar{\sigma }_{i,j,k}\cdot \underline{\theta }_{k}, \cos \tfrac{\delta _\theta }{2}\bar{\sigma }^\theta _{i,j,k}) \le (1+C\delta _\theta ^2) \int _{{\Omega _{}\times \mathbb {S}^1}} h(\theta ,\sigma _u) \end{aligned}$$

(using that $\int _{{\Omega _{}\times \mathbb {S}^1}}h(\theta ,\sigma _\varepsilon )\le \int _{{\Omega _{}\times \mathbb {S}^1}} h(\theta ,\sigma _u)$, cf the Proof of Proposition 3.1).

Eventually, letting now

$$\begin{aligned} e'_{i,j,k} = e_{i,j,k} + (1-\cos \tfrac{\delta _\theta }{2})\bar{\sigma }^{\theta }_{i,j,k}, \end{aligned}$$

which is such that

$$\begin{aligned} \delta _x^2\delta _\theta \sum _{i,j,k} |e'_{i,j,k}|&\le c\frac{\sqrt{\delta _x^2+\delta _\theta ^2}}{\varepsilon }\int _{{\Omega _{}\times \mathbb {S}^1}}|\sigma ^x_u| + \delta _\theta ^2 \int _{{\Omega _{}\times \mathbb {S}^1}}|\sigma ^\theta _u|\nonumber \\&\le c\frac{\sqrt{\delta _x^2+\delta _\theta ^2}}{\varepsilon }\int _{{\Omega _{}\times \mathbb {S}^1}}|\sigma _u| \le c\frac{\sqrt{\delta _x^2+\delta _\theta ^2}}{\gamma \varepsilon } F(u). \end{aligned}$$

(49)

We deduce that we can find an center-averaged Raviart–Thomas field $\hat{\sigma }$ and an error term $e'$ such that (49) holds and

$$\begin{aligned} \sum _{i,j,k} \bar{h}((\hat{\sigma }_{i,j,k}-e'_{i,j,k})\cdot \underline{\theta }_{k},(\hat{\sigma }_{i,j,k}-e'_{i,j,k})^\theta ) \le (1+C\delta _\theta ^2) F(u) \end{aligned}$$

where $\theta ^{\delta _\theta }=\sum _k \theta _k\chi _{\{k\delta _\theta ,(k+1)\delta _\theta \}}$. In particular if we introduce, for $\delta =(\delta _x,\delta _t)$ small, the inf-convolution

$$\begin{aligned} \bar{h}_\delta (s,t) = \min _{s',t'} \bar{h}(s-s',t-t') + \frac{\gamma }{(\delta _x^2+\delta _\theta ^2)^{1/4}}\sqrt{s'^2+t'^2} \end{aligned}$$

(50)

we find that

$$\begin{aligned} \delta _x^2\delta _\theta \sum _{i,j,k} \bar{h}_\delta (\hat{\sigma }_{i,j,k}\cdot \underline{\theta }_{k},\hat{\sigma }_{i,j,k}^\theta ) \le (1+C\delta _\theta ^2+ \tfrac{c}{\varepsilon }(\delta _x^2+\delta _\theta ^2)^{1/4}) F(u). \end{aligned}$$

Moreover, one easily sees that

$$\begin{aligned} \hat{\sigma }_{i,j,k}^x \in \mathbb {R}_+\underline{\theta }_{k-\frac{1}{2}}+\mathbb {R}_+\underline{\theta }_{k+\frac{1}{2}}. \end{aligned}$$

(51)

Remark A.6

If $\bar{h}$ is L-Lipschitz (as it is the case when f as growth one, for instance if $f(t)=\gamma \sqrt{1+t^2}$), then the inf-convolution step is not necessary. One directly obtains

$$\begin{aligned} \delta _x^2\delta _\theta \sum _{i,j,k} \bar{h}(\hat{\sigma }_{i,j,k}\cdot \underline{\theta }_{k},\hat{\sigma }_{i,j,k}^\theta ) \le \Big (1+C\delta _\theta ^2+ \tfrac{cL}{\gamma \varepsilon }\sqrt{\delta _x^2+\delta _\theta ^2}\Big )F(u). \end{aligned}$$

Now, we check the consistency between $\hat{\sigma }$ and u. By construction, $\delta _x\delta _\theta \sum _k \sigma ^1_{i+\frac{1}{2},j,k}$ is the flux of $Du^\perp$ through the edge $\{i+\frac{1}{2}\delta _x\}\times [(j-\frac{1}{2})\delta _x,(j+\frac{1}{2})\delta _x]$ in $\Omega$, hence it is equal to the value $u((i+\frac{1}{2})\delta _x,(j+\frac{1}{2})\delta _x)-u((i+\frac{1}{2})\delta _x,(j-\frac{1}{2})\delta _x)$. Accordingly, if we let, for all i, j, $u_{i+\frac{1}{2},j+\frac{1}{2}}^{\delta }:=u((i+\frac{1}{2})\delta _x,(j+\frac{1}{2})\delta _x)$, we obtain that (23) holds (with u replaced with $u^\delta$).

Eventually we observe that the free divergence condition simply translates as (22) for all admissible i, j, k, as this is the global flux of $\sigma _\varepsilon$ across the boundaries of $V_{i,j,k}$.

It is now easy to deduce the following upper approximation result:

Proposition A.7

Let $u\in BV(\Omega )$, $\sigma$ be admissible for u and such that

$$\begin{aligned} F(u)=\int _{{\Omega _{}\times \mathbb {S}^1}} h(\theta ,\sigma )<\infty .\end{aligned}$$

Then for $\delta =(\delta _x,\delta _\theta )\rightarrow 0$ one can find a discrete field $(\sigma ^1_{i+\frac{1}{2},j,k},\sigma ^2_{i,j+\frac{1}{2},k},\sigma ^{\theta }_{i,j,k+\frac{1}{2}})$ and a discrete image $u^\delta _{i+\frac{1}{2},j+\frac{1}{2}}$ with

$$\begin{aligned} \sum _{i,j} u^\delta _{i+\frac{1}{2},j+\frac{1}{2}}\chi _{[i\delta _x,(i+1)\delta _x)\times [j\delta _x,(j+1)\delta _x)}\rightarrow u \end{aligned}$$

(52)

(strongly in $L^2(\Omega )$) and such that for all i, j, (23) holds, for all i, j, k, (51) and (22) hold, and:

$$\begin{aligned} \limsup _{\delta \rightarrow 0} \delta _x^2\delta _\theta \sum _{i,j,k} \bar{h}_\delta (\hat{\sigma }_{i,j,k}\cdot \underline{\theta }_{k},\hat{\sigma }_{i,j,k}^\theta ) \le F(u) \end{aligned}$$

(53)

where $\hat{\sigma }$ is defined by (24), and where $\bar{h}_\delta$ is defined in (50) (or is $\bar{h}$ in case it is Lipschitz).

To show that the discretization is consistent, we must now show a similar lower bound: namely that given any u and $u^\delta ,\sigma ,\hat{\sigma }$ which satisfy (24), (51), (23), and (52) (weakly, for instance as distributions), then one has

$$\begin{aligned} \liminf _{\delta \rightarrow 0} \delta _x^2\delta _\theta \sum _{i,j,k} \bar{h}_\delta (\hat{\sigma }_{i,j,k}\cdot \underline{\theta }_{k},\hat{\sigma }_{i,j,k}^\theta ) \ge F(u). \end{aligned}$$

(54)

A first obvious remark is that the field

$$\begin{aligned} \hat{\sigma }^\delta := \sum _{i,j,k} \hat{\sigma }_{i,j,k} \chi _{V_{i,j,k}} \end{aligned}$$

is bounded in measure, and hence, up to subsequences, converges (weakly-$*$) to a measure $\sigma$. It is then easy to deduce from (51) and the convexity of $\bar{h}$ that

$$\begin{aligned} \int _{{\Omega _{}\times \mathbb {S}^1}} h(\theta ,u)\le \liminf _{\delta \rightarrow 0} \delta _x^2\delta _\theta \sum _{i,j,k} \bar{h}_\delta (\hat{\sigma }_{i,j,k}\cdot \underline{\theta }_{k+\frac{1}{2}},\hat{\sigma }_{i,j,k}^\theta ). \end{aligned}$$

One can also check that $\text {div}\,\sigma =0$ by passing to the limit in (22) [after a suitable integration against a smooth test function, exactly as in (55) below]. Hence it is enough to show that the limiting $\sigma$ is compatible with u.

But this is quite obvious from (23), which one can integrate against a smooth test function, then “integrate by part” before passing to the limit. More precisely, for $\varphi \in C^\infty _c(\Omega )$, one has (dropping the superscripts $\delta$ and denoting $\varphi _{i,j}=(1/\delta _x^2)\int _{S_{i,j}} \varphi (x) dx$):

$$\begin{aligned}&\int _{{\Omega _{}\times \mathbb {S}^1}}\varphi (x) \hat{\sigma }^1 dx d\theta \nonumber \\&\quad =\sum _{i,j,k} \hat{\sigma }^1_{i,j,k}\int _{V_{i,j,k}}\varphi (x) dx d\theta \nonumber \\&\quad =\delta _x^2\delta _\theta \sum _{i,j} \varphi _{i,j} \sum _k \delta _\theta \frac{\sigma ^1_{i+\frac{1}{2},j,k}+\sigma ^1_{i-\frac{1}{2},j,k}}{2} \nonumber \\&\quad = \, \frac{\delta _x^2\delta _\theta }{2} \sum _{i,j} \varphi _{i,j} \left( \frac{u_{i+\frac{1}{2},j+\frac{1}{2}}-u_{i+\frac{1}{2},j-\frac{1}{2}}}{\delta _x} +\frac{u_{i-\frac{1}{2},j+\frac{1}{2}}-u_{i-\frac{1}{2},j-\frac{1}{2}}}{\delta _x}\right) \nonumber \\&\quad =-\delta _x^2\delta _\theta \sum _{i,j} \frac{u_{i+\frac{1}{2},j+\frac{1}{2}}+u_{i-\frac{1}{2},j+\frac{1}{2}}}{2} \frac{\varphi _{i,j+1}-\varphi _{i,j}}{\delta _x} \nonumber \\&\qquad \quad \rightarrow -\int _\Omega u(x)\partial _2\varphi (x) dx \end{aligned}$$

(55)

as $\delta \rightarrow 0$.

Eventually, we need to show a compactness property, which is that if

$$\begin{aligned} \sup _{\delta } \delta _x^2\delta _\theta \sum _{i,j,k} \bar{h}(\hat{\sigma }_{i,j,k}\cdot \underline{\theta }_{k},\hat{\sigma }_{i,j,k}^\theta ) <+\infty \end{aligned}$$

(56)

the discrete image $u^\delta$ which is recovered from (23) (up to a constant) converges to a u(x), $x\in \Omega$ (in a weak sense which will be made clear). The point here is that a priori, from (56) and (1), one has only

$$\begin{aligned}&\sup _{\delta } \delta _x^2 \sum _{i,j} \left( \left( \frac{u^\delta _{i+\frac{1}{2},j+\frac{1}{2}}-u^\delta _{i+\frac{1}{2},j-\frac{1}{2}}}{\delta _x} +\frac{u^\delta _{i-\frac{1}{2},j+\frac{1}{2}}-u^\delta _{i-\frac{1}{2},j-\frac{1}{2}}}{\delta _x}\right) ^2\right. \nonumber \\&\left. \quad + \left( \frac{u^\delta _{i+\frac{1}{2},j-\frac{1}{2}}-u^\delta _{i-\frac{1}{2},j-\frac{1}{2}}}{\delta _x} +\frac{u^\delta _{i+\frac{1}{2},j+\frac{1}{2}}-u^\delta _{i-\frac{1}{2},j+\frac{1}{2}}}{\delta _x}\right) ^2 \right) ^{\frac{1}{2}} <+\infty \end{aligned}$$

(57)

so that the discrete total variation of $u^\delta$ is a priori not well controlled. However, one can easily check that the kernel of the operator which appears in the energy (57) is two-dimensional, and made of the oscillating discrete images

$$\begin{aligned} v_{i-\frac{1}{2},j-\frac{1}{2}}^\delta = \alpha + \beta (-1)^{i+j}, \end{aligned}$$

(58)

$\alpha ,\beta \in \mathbb {R}^2$. Hence it is possible to show that one can decompose $u^\delta$ as a sum of a non-oscillating function with zero average $\bar{u}^\delta$ and an oscillation $v^\delta$, and obtain a strong control on the discrete total variation of $\bar{u}^\delta$. Therefore one easily deduce that any suitably built continuous extension of $\bar{u}^\delta$ will converge to some u strongly in $L^p(\Omega )$, for any $p<2$ (as $BV(\Omega )$ is compactly embedded in such spaces), and weakly in $L^2(\Omega )$.

In addition, any control on the average of $u^\delta$ and on its oscillation (which cannot be given by (57) and has to come from other terms in the energy, such as a boundary condition or a penalization: note that it is enough to control two adjacent pixels) will ensure in addition that $v^\delta$ remains bounded and converges (only weakly in $L^p(\Omega )$, if the control is only on the $L^p$ norm, however in this case it is obvious that the oscillating term in (58) goes to zero and $v^\delta$ can only go to a constant).

To sum up, we have shown the following.

Proposition A.8

For $\delta \rightarrow 0$, assume we are given $\sigma ^\delta$, $u^\delta$ and $\hat{\sigma }^\delta$ with (24), (23), which in addition satisfy (51) and (22), and (56). Then, up to an oscillating function $v^\delta$ of the form (58), there is $u\in BV(\Omega )$ such that $u^\delta \rightarrow u$, and (54) holds.

Remark A.9

In practice, we did not use the inf-convolutions $\bar{h}_\delta$ (only $\bar{h}$) in our discrete scheme. Also, we replaced the constraint (51) with the stronger constraint $\hat{\sigma }^x_{i,j,k}\in \mathbb {R}_+\underline{\theta }_k$, after having experimentally observed that there was no qualitative difference in the output. It seems the results we compute are still consistent with what is expected from the energy.

B. Smirnov’s theorem in ${\Omega _{}\times \mathbb {S}^1}$

In this whole paper $\Omega \subset \mathbb {R}^2$ is assumed to be a Lipschitz set. In particular, locally its boundary can be represented as the subgraph $\{(x,y): y<h(x)\}$ of a Lipschitz function h. Consider a ball B where this representation holds and assume first h is $C^1$, then one can extend in B a bounded Radon measure $\sigma$ with $\text {div}\,\sigma =0$ into $\tilde{\sigma }$ defined (for $\psi \in C_c^0(B;\mathbb {R}^2)$)

$$\begin{aligned} \int _B \tilde{\sigma }\cdot \psi := \int _{B\cap \Omega }\sigma \cdot \left( \psi (x,y)- \begin{pmatrix}1 &{} 2h'(x)\\ 0 &{} -1\end{pmatrix}\psi (x,2h(x)-y)\right) . \end{aligned}$$

Then, it is standard that $\text {div}\,\tilde{\sigma }=0$ in B, indeed, if $\varphi \in C_c^1(B)$, one has that

$$\begin{aligned} \int _B \tilde{\sigma }\cdot \nabla \varphi = \int _{B\cap \Omega } \sigma \cdot \nabla \left[ \varphi (x,y)-\varphi (x,2h(x)-y)\right] . \end{aligned}$$

The function $\varphi ^s(x,y):=\varphi (x,y)-\varphi (x,2h(x)-y)$ is $C^1$ and vanishes on $\partial \Omega$, hence this expression is zero: Indeed if for $\tau >0$ one lets $\varphi ^s_\tau (x,y)=S_\tau (\varphi ^s(x,y))$ where $S_\tau \in C^\infty (\mathbb {R})$ is a smooth approximation of a “shrinkage operator”:

$$\begin{aligned} S_\tau (t) ={\left\{ \begin{array}{ll} t-\tau &{} \quad \mathrm{if }\; t\ge \tfrac{3}{2}\tau , \\ 0 &{} \quad \mathrm{if }\; |t|< \tfrac{1}{2}\tau , \\ t+\tau &{} \quad \mathrm{if }\; t\le - \tfrac{3}{2}\tau , \end{array}\right. } \end{aligned}$$

with smooth and 1-Lipschitz interpolation in $\pm [\tau /2,3\tau /2]$, then $\varphi ^s_\tau \in C_c^1(B\cap \Omega )$ so that

$$\begin{aligned} \int _{B\cap \Omega } \sigma \cdot \nabla \varphi ^s_\tau =0 \end{aligned}$$

and, using $\nabla \varphi ^s_\tau = S'_\tau (\varphi ^s)\nabla \varphi ^s$

$$\begin{aligned} \int _{B\cap \Omega }\sigma \cdot (\nabla \varphi ^s-\nabla \varphi ^s_\tau ) \le C |\sigma |(B\cap \Omega \cap \{0<|\varphi ^s|< 3\tau /2\})\rightarrow 0 \end{aligned}$$

as $\tau \rightarrow 0$, showing our claim. Hence $\text {div}\,\tilde{\sigma }=0$. If h is not $C^1$ but just Lipschitz, one can approximate it from below by smooth functions $h_n$, build in such a way a sequence $\sigma _n$ of extensions of $\sigma {|} {}_{\{y<h_n(x)\}}$ and pass to the limit to deduce that the extension still exists.

Using cut-off functions, one can therefore assume that $\sigma$ can be extended into a field $\tilde{\sigma }$ which is a measure in $\mathbb {R}^2$ with free divergence in a neighborhood of $\Omega$.

A similar construction would allow to extend a field $\sigma \in \mathcal {M}(\Omega \times \mathbb {R}^2;\mathbb {R}^4)$ to $\mathcal {M}(\mathbb {R}^4;\mathbb {R}^4)$ with free divergence (either in a neighborhood or $\Omega \times \mathbb {R}^4$, or even everywhere). This remark allows to localize Smirnov’s theorems in [73].

Consider indeed now a free divergence field $\sigma \in \mathcal {M}({\Omega _{}\times \mathbb {S}^1};\mathbb {R}^3)$. It can be seen, after extension, as a field in $\mathcal {M}(\mathbb {R}^2\times \mathbb {R}^2;\mathbb {R}^4)$ with $\text {spt}\sigma \subseteq \mathbb {R}^2\times \mathbb {S}^1$.

As in Smirnov’s paper [73], for $l>0$ we introduce $\mathfrak {C}_l$ the set of oriented curves $\gamma$ in $\mathbb {R}^4$ with length l, with the topology corresponding to the weak convergence of the measures . Then, thanks to [73, Theorem A], $\sigma$ can be decomposed as

$$\begin{aligned} \sigma = \int _{\mathfrak {C}_l} \lambda d\mu (\lambda ), \quad |\sigma | = \int _{\mathfrak {C}_l} |\lambda | d\mu (\lambda ), \end{aligned}$$

for some measure $\mu$ on $\mathfrak {C}_l$. Moreover thanks to Remark 5 in [73], $\mu$-a.e. curve in the decomposition lies in $\mathbb {R}^2\times \mathbb {S}^1$.

For this work, it is enough to consider $l=1$. Moreover, if we restrict then all these measures to ${\Omega _{}\times \mathbb {S}^1}$ (and take for $\mu$ the corresponding marginal), we get a decomposition on curves of length less or equal to 1 (possibly entering/exiting the domain). By a slight abuse of notation we still denote $\mathfrak {C}_1$ such a set of curves. One finds that

$$\begin{aligned} \sigma = \int _{\mathfrak {C}_1} \lambda d\mu (\lambda ), \quad |\sigma | = \int _{\mathfrak {C}_1} |\lambda | d\mu (\lambda ), \end{aligned}$$

(59)

with now $\lambda \in \mathfrak {C}_1$, the curves of length at most one in ${\Omega _{}\times \mathbb {S}^1}$.

Remark B.1

Theorem B in [73] is a more precise statement. It shows that one can obtain a similar decomposition with now curves $\lambda$ with $\text {div}\,\lambda =0$ a.e.: being either finite curves entering and exiting the domain, or “elementary solenoids”, which are objects of the form

(in particular one should have $|f'(t)|=1$a.e.), meaning that for any $\varphi \in C_c^1({\Omega _{}\times \mathbb {S}^1})$,

$$\begin{aligned} \lambda (\varphi ) = \lim _{s\rightarrow \infty } \frac{1}{2s}\int _{-s}^s {\left\langle {f'(t)},{\varphi (f(t))}\right\rangle }dt. \end{aligned}$$

(60)

This expresses that either $\lambda$ is defined by the closed curve $f(\mathbb {R})$ (if f is periodic), or $\lambda$ is a limit of curves which densify and do not loose mass in the limit. We do not need such a precise result for our construction.