Sparsity of solutions for variational inverse problems with finite-dimensional data

In this paper we characterize sparse solutions for variational problems of the form minu∈Xϕ(u)+F(Au)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\min _{u\in X} \phi (u) + F(\mathcal {A}u)$$\end{document}, where X is a locally convex space, A\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {A}$$\end{document} is a linear continuous operator that maps into a finite dimensional Hilbert space and ϕ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\phi $$\end{document} is a seminorm. More precisely, we prove that there exists a minimizer that is “sparse” in the sense that it is represented as a linear combination of the extremal points of the unit ball associated with the regularizer ϕ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\phi $$\end{document} (possibly translated by an element in the null space of ϕ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\phi $$\end{document}). We apply this result to relevant regularizers such as the total variation seminorm and the Radon norm of a scalar linear differential operator. In the first example, we provide a theoretical justification of the so-called staircase effect and in the second one, we recover the result in Unser et al. (SIAM Rev 59(4):769–793, 2017) under weaker hypotheses.

inf u∈X φ (u) subjected to Au = y, (1) where φ is the regularizer, A : X → H , H finite-dimensional Hilbert space, models the finite number of observations (that is small compared to the dimension of X ) and y ∈ H is noise-free data. When the domain X is finite-dimensional and the regularizer is the 1 norm, the problem falls into the established theory of compressed sensing [11,19] that has seen a huge development in recent years. In this case, sparsity is intended as a high number of zero coefficients with respect to a certain basis of X .
In an infinite dimensional setting, when the domain X is usually a Banach space, there has been a clear evidence that the action of the regularizers is promoting different notions of sparsity, but there have not been a comprehensive theory explaining this effect.
Nevertheless, the effect of sparsity plays a crucial role in the field of image processing and computer vision: in many cases, the recovered image in a variational model can be interpreted as sparse with respect to a notion of sparsity that is depending on the regularizer. For example, for classical total variation (TV) denoising [32] inf u∈BV ( ) it has been observed that minimizers are characterized by the so called staircase effect (see for example [13,15,29]) which corresponds to the gradient of the considered image having small support. Another classical example of sparsity-promoting regularizers is 1 -penalization. In [17] the authors study the p regularizer with 1 ≤ p ≤ 2 in Hilbert spaces and they note that the case p = 1 promotes sparsity with respect to a given basis of the Hilbert space which means that only a finite number of coefficients in the respective basis representation is non-zero. In [7], 1 -regularization is used in the framework of the least error method to recover a sparse solution with a fixed bound on the number of non-zero coefficients. Finally, it has been noted that suitable 1 -type regularizers enforce sparsity when data are represented in a wavelet basis (see for example [3,20]). The intrinsic sparsity of infinite-dimensional variational models with finite-dimensional data has been investigated by various authors in specific cases and in different contexts. One of the most important instances can be found in [16]: here, the authors notice that the regularizer is linked to the convex hull of the set of sparse vectors that we aim to recover. This was also noticed in optimal control theory (see, for example, [12]) and used in practice for developing efficient algorithms to solve optimization problems that are based on the sparsity of the minimizers [9,10,31].
More recently, several authors have investigated deeply the connection between regularizers and sparsity. In 2016, Unser et al. in [36] have studied the case where φ(u) = Lu M , L is a scalar linear differential operator and · M denotes the Radon norm. They showed the existence of a sparse solution, namely a linear combination of counterimages of Dirac deltas which can be expressed using a fundamental solution of L. Also, the work of Flinth and Weiss [26] is worth mentioning, where they give an alternative proof of the result in [36] with less restrictive hypotheses. In both works, however, the case of a vector-valued differential operator was not treated and therefore, problems involving the total variation regularizer were not covered. After this manuscript was finalized, we discovered a recent preprint [5] where the authors study a similar abstract problem and apply it, in particular, to the TV regularizer in order to justify the staircase effect. We remark that [5] and the present paper were developed independently and differ in terms of the proofs as well as the applications.
In this paper, we provide a theory that characterizes sparsity for minimizers of general linear inverse problems with finite-dimensional data constraints. More precisely, we choose to work with locally convex spaces in order to deal, in particular, with weak* topologies. The latter is necessary in order to treat variational problems with TV regularization or Radon-norm regularization. We consider the following problem: where X is a locally convex space, φ : X → [0, +∞] is a lower semi-continuous seminorm, A : X → H is a linear continuous map with values in a finite-dimensional Hilbert space H and F is a proper, convex, lower semi-continuous functional. (Notice that this generality allows problems of the type (1) for noise-free data as well as soft constraints in case of noisy data.) Additionally we ask that A(dom φ) = H (see Assumption [H0] below) and that φ is coercive when restricted to the quotient space of X with the null-space of φ that we denote by N (see Assumption [H1] below). Under these hypotheses we prove that there exists a sparse minimizer of (3), namely a minimizer that can be written as a linear combination of extremal points of the unit ball associated to φ (in the quotient space X /N ). More precisely, we obtain the following result: Theorem (Theorem 3.3) Under the previous hypotheses there exists u ∈ X , a minimizer of (3) such that: Notice that our result completely characterizes the sparse solution u of (3) and relates the notion of sparsity with structural properties of the regularizer φ. Moreover, our hypotheses are minimal for having a well-posed variational problem (3).
The strategy to prove the previous theorem relies on the application of Krein-Milman's theorem and Carathéodory's theorem in the quotient space of A(X ) that allows to represent any element in the image by A of the unit ball of the regularizer as a convex combination of the extremal points (see Theorem 3.3). In order to prove minimality for the element having the desired representation, we derive optimality conditions for Problem (3) (Proposition 2.12). For this purpose, we need to prove a no gap property in the quotient space between primal and dual problem. In locally convex vector spaces this is not straightforward and requires the notion of Mackey topology [34].
In the second part of our paper we apply the main result to specific examples of popular regularizers. First of all we recover the well-known result (see for example [35]) that by minimizing the Radon norm of a measure under finite-dimensional data constraints, one recovers a minimizer that is made of delta peaks. Indeed, according to our theory which applies when the space of Radon measures M( ) is equipped with the weak* topology, Dirac deltas are extremal points of the unit ball associated with the Radon norm of a measure and our result applies straightforwardly (see Sect. 4.1).
Then, we consider the TV regularizer for BV functions in bounded domains. Also in this case, our result applies when BV ( ) is equipped with the weak* topology. This justifies the usage of locally convex spaces in the general theory. In order to confirm the heuristic observation that sparse minimizers show a peculiar staircase effect, we characterize the extremal points of the unit ball associated to the TV norm (in the quotient space BV ( )/R). In particular, we extend a result of [1,24] to the case where is a bounded domain. In order to achieve that, we need an alternative notion of simple sets of finite perimeter (see Definition 4.5). We prove the following theorem: Finally, we apply our main result to the setting considered in [26,36], i.e., where the regularizer is given by φ(u) = Lu M for a scalar linear differential operator L. We remove the hypotheses concerning the structure of the null-space of L and we work in the space of finite-order distributions equipped with the weak* topology. This allows us to have a general framework for these inverse problems that does not require additional assumptions on the Banach structure of the minimization domain (see [5,26] for comparison). It also justifies once more the usage of locally convex spaces in the abstract theory. In this setting, as an application of our main theorem, we are able to recover the same result as in [26,36].

Basic assumptions on the functionals
Let (X , τ ) be a real locally convex space, i.e., the topology is generated by a separating family of seminorms, and (X * , τ ) its topological dual equipped with the weak* topology. Further, let H be an N -dimensional real Hilbert space and A : X → H a linear continuous operator and we denote by A * : H → X * its continuous adjoint, defined thanks to Riesz's theorem as for every u ∈ X and w ∈ H . Notice that we have denoted by ·, · both the scalar product in the Hilbert space H the duality product between X and X * .
As anticipated in the introduction we deal with a variational problem of the type inf u∈X φ(u) + F(Au).
In the remaining part of this section we describe the assumptions on F and φ separately.
-Assumptions on F: We consider a proper convex function that is coercive, and lower semi-continuous with respect to the topology of H , which is the standard topology on finite-dimensional spaces.
-Assumptions on φ: We consider a seminorm and that is lower semi-continuous with respect to the topology of X . We make the following additional assumption: Defining the null-space of φ as N = {u ∈ X : φ(u) = 0}, which is a closed subspace of X , we consider the following quotient space: endowed with the quotient topology. It is well-known that X N is a locally convex space [33]. We call π N : X → X N the canonical projection onto the quotient space and for simplifying the notation, given u ∈ X we denote by u N = u + N the image of u in the quotient space by π N . Likewise, for U ⊂ X , we tacitly identify the Minkowski sum U + N ⊂ X with its image under π N in X N . Define then φ N : Note that φ N is well-defined as φ is constant on the set u + N for every u ∈ X . Moreover, it is a seminorm in X N . We assume that [H1] φ N is coercive, i.e. the sublevel sets are compact for every α > 0.

Remark 2.1
Note that φ N is lower semi-continuous in X N : Indeed, as φ is lower semicontinuous, the superlevel-sets S + (φ, α) = {u ∈ X : φ(u) > α} are open in X for each α. Now, as φ N (u N ) > α if and only if φ(u) > α, we have S + (φ N , α) = π N (S + (φ, α)). Since π N is an open map (X is a topological group with respect to addition), each S + (φ N , α) is open in X N meaning that φ N is lower semi-continuous. As a consequence, in order to obtain [H1], it suffices that each S − (φ N , α) is contained in a compact set.
From now on we assume that A, F and φ satisfy the properties described above.

Existence of minimizers
We state the following minimization problem: Problem 2.2 (Minimization problem in X ) Given φ, A and F with the assumptions given in the previous section, define for u ∈ X the following functional: We aim at solving min u∈X J (u).
In order to prove the existence of minimizers for Problem 2.2 we state an auxiliary minimization problem in the quotient space X N . Problem 2.3 (Minimization problem in X N ) Given F, φ and A with the assumptions given in the previous section, we define We want to solve Note that the functional J is well-defined in X N as both summands in (11) are constant on u + N for every u ∈ X . We aim at proving existence of minimizers for Problem 2.3. For this reason we firstly prove a lemma about the coercivity of functionals defined in quotient spaces.
Then, f is coercive with respect to the quotient topology of Y M .
Proof By coercivity, the sublevel sets S − ( f , α) are compact for each α. Since the projection Moreover, F N is proper and convex. As F is assumed to be coercive, applying Lemma 2.4 yields that F N is also coercive and lower semi-continuous in particular. Now, where the right-hand side is a composition of continuous linear maps and a lower semicontinuous functional and hence, lower semi-continuous. Obviously, replacing u by u + ϕ, ϕ ∈ N does not change the value of this functional, so by the same argument as in Remark 2.1, we deduce that u N → inf ψ∈N F(A(u+ψ)) and consequently J , is lower semi-continuous. Notice now that Therefore, as J is lower semi-continuous and φ N is coercive due to Hypothesis [H1], we infer that S − (J , α) is compact for every α ∈ R.
We want to prove that J admits a minimizer in X N . Notice that the collection {S − (J , α)} α>inf J has the finite intersection property. As the set S − (J , α 0 ) is compact for an arbitrary α 0 > inf J and each S − (J , α) is closed, we infer that We are now in position to prove the existence of minimizers for Problem 2.2.

Theorem 2.6 Given u
Hence taking the infimum with respect to ψ ∈ N on both sides we obtain that Problems 2.2 and 2.3 have the same infimum. Let u N be a minimizer for Problem 2.3. Then consider the following minimization problem: As F is proper, convex, lower semi-continuous and coercive as well as A(N ) is finitedimensional and hence closed in H , the infimum is realized and finite. Denoting by η a minimizer, we choose ψ ∈ N such that Aψ = η. Then, v := u + ψ is a minimizer for Problem 2.2. Indeed, Then, as the two minimization problems have equal infimum, we conclude.

Remark 2.7
The converse of Theorem 2.6 holds true. Namely, if u a minimizer for Problem 2.2, then u N = u + N is a minimizer of Problem 2.3. Indeed, for every v N = v + N and denoting by ψ a minimizer of ψ → F(A(v + ψ)) in N (that exists for similar arguments as in the previous proof) we have

Optimality conditions
In this section we want to obtain optimality conditions for Problem 2.3 deriving a dual formulation and showing that under our hypotheses we have no gap between the primal and the dual problem.
In order to perform this analysis we need to endow the space X * N equipped with the weak*topology with the associated Mackey topology. For the reader's convenience we remind the definition of the Mackey topology and we refer to [34] for a comprehensive treatment. Given a real locally convex space Y , define the following family of seminorms on Y * : for every A ⊂ Y absolutely convex and weakly compact. This family of seminorms generates a locally convex topology on Y * that is called Mackey topology and it is denoted by τ (Y * , Y ). It is the strongest topology on Y * such that Y is still the dual of Y * (see Theorem 9 in Section A.4 of [4]). Further, we need the notion of Fenchel conjugate functionals which are defined as follows. Given a real locally convex space Y and a proper function f : In order to obtain the optimality conditions we will use the following well-known proposition (see Proposition 5 in Section 3.4.3 of [4]). (i) f * is continuous in zero for the Mackey topology τ (Y * , Y ). (ii) for every α ∈ R, the sublevel-set is compact with respect to the weak topology.

Remark 2.9
In the next proposition, we will apply this result for f = φ N , a proper and lower semi-continuous seminorm. In this case, the proof of Proposition 2.8 is straightforward.
where I is the indicator function and S = {u : φ N (u) ≤ 1}. Hence, if S is weakly compact, then thanks to the definition of the Mackey topology, φ * N is continuous in zero.
Conversely, if φ * N is continuous in zero, then there exist absolutely convex, weakly compact sets A 1 , . . . , A n ⊂ X N and ε 1 , . . . , ε n > 0 such that Indeed, if this were not the case, one could separate a u ∈ S from the absolutely convex and weakly compact set . . , n leading to the contradiction ρ S (u * ) ≤ 1. Due to lower semi-continuity of φ N , S is a closed convex subset of a weakly compact set and hence weakly compact. By positive homogeneity of φ N , the sets S − (φ N , α) are compact for all α ∈ R.
For the following, it is convenient to define the linear operator A N :

Remark 2.10 Notice that
Moreover, it is continuous in X N ; indeed, the following diagram commutes: We denote by A * N : H N → X * N its adjoint that has finite-dimensional image and is hence continuous for each topology that makes X * N a topological vector space. Given w ∈ H , we denote by w N := w + A(N ) an element of H N .
We can equivalently write where

Remark 2.11
Notice again that F N is proper, convex and, applying Lemma 2.4 with f = F and M = A(N ), it is also coercive in H N .
We now derive optimality conditions for Problem 2.3. For that purpose, recall that given a functional f : for each y ∈ Y . In this case, we denote x * ∈ ∂ f (x).

Proposition 2.12 (Optimality conditions) It holds that u N ∈ X N is a minimizer for Problem 2.3 if and only if there exists
Proof We start with transforming Problem 2.3 into the problem (P * ) for which the dual problem in terms of Fenchel-Rockafellar duality (see Remark III.4.2 in [22]) will turn out to be equivalent to the original problem: We now endow X * N with the Mackey topology τ (X * N , X N ). Notice that φ * N is convex, proper and weakly* lower semi-continuous and hence it is lower semi-continuous with respect to τ (X * N , X N ) as well. As previously mentioned, the adjoint A * N : H N → X * N is linear and continuous with respect to any vector space topology and in particular, the weak* topology of X * N as well as the Mackey topology τ (X * N , X N ). Moreover, thanks to Remark 2.11, F * N is convex, proper and lower semi-continuous in H N . Notice that as φ N satisfies Hypothesis [H1] (that implies in particular that the sublevel sets of φ N are weakly compact), using Proposition 2.8, we have that φ * N is continuous in zero. Hence, applying Theorem III.4.1 in [22], the problem (P * ) has zero gap to its dual which coincides, as the dual In order to establish the optimality conditions, we want to prove now that the problem (P * ) has a minimizer, since the existence of a minimizer for Problem 2.3 has already been established in Theorem 2.5. Notice that at this point there is no more need to consider the Mackey topology on X * N and we can use the weak* topology on X * N .
is convex, proper and lower semi-continuous. We aim at showing that it is also coercive. It is enough to prove that φ * N • A * N is the indicator function of a compact convex set as F * N is proper, convex and lower semi-continuous. Notice that We prove that the map G : H N → (−∞, +∞] defined as is convex, proper and coercive in H N . For this purpose, notice that, as a consequence of Hypothesis [H0] and the definition of A N , we have By Hypothesis [H1], φ N is coercive, so Lemma 2.4 yields that φ 0 is coercive. As φ N is a seminorm, φ 0 is proper and convex. It follows that G is proper, convex and lower semicontinuous. As dom G = H N , convexity implies that G is continuous everywhere in H N and in particular, in zero. Consequently, G * = φ * N • A * N is coercive. It follows that G * is the indicator function of a compact convex set as G is one-homogeneous. Hence, applying the direct method of calculus of variations in H N we infer that Problem (P * ) has a minimizer that we denote by w N ∈ H N .
We now derive the optimality conditions for this problem. Applying Proposition III.4.1 in [22] we infer that Therefore, we get with the help of the Fenchel inequality that Finally, Eqs. (17) and (18) are equivalent to as we wanted to prove. Vice versa, if there exist w N and u N that satisfy the optimality conditions (i) and (ii), applying again Proposition III.4.1 in [22] we deduce that u N is a minimizer of Problem 2.3 and w N is a minimizer of (P * ). Definition 3.1 (Extremal points) Given a convex set K of a locally convex space we define the extremal points of K as the points k ∈ K such that if there exists t ∈ (0, 1), k 1 , k 2 ∈ K such that

Remark 2.13 Defining the set
The set of extremal points of K will be denoted by E xt(K ).
First we need a lemma about the behaviour of extremal points under a linear mapping.
Lemma 3.2 Let K be a convex set in a locally convex space X . Given Y a real topological vector space and a linear map L : X → Y the following statements hold: Proof To prove (i) let us consider k ∈ K such that Lk is an extremal point of L K . We want to show that there exists k ∈ E xt(K ) such that Lk = Lk which proves the first claim. Consider the set (k + ker L) ∩ K . As this is a non-empty compact convex set in a locally convex space (ker L is closed by the continuity of L), by the Krein-Milman theorem, it admits an extremal point denoted by k ∈ (k + ker L) ∩ K . In order to conclude the proof we need to prove that k ∈ E xt(K ). Assume the convex combination for k 1 , k 2 ∈ K and t ∈ (0, 1). Then applying the linear operator L we obtain that As Lk ∈ E xt(L K ) and Lk = Lk we infer that Lk = Lk 1 = Lk 2 and so k 1 , k 2 ∈ (k + ker L) ∩ K . From (20) and the extremality of k it follows that k = k 1 = k 2 . Let us prove (ii). To show that E xt(L K ) ⊂ L E xt(K ) take Lk ∈ E xt(L K ) and assume the convex combination for k 1 , k 2 ∈ K , t ∈ (0, 1). Applying L to both sides and using that Lk ∈ E xt(L K ) we obtain that Lk = Lk 1 = Lk 2 . Then the injectivity of L implies that k = k 1 = k 2 , thus k ∈ E xt(K ).
To prove the opposite inclusion let us consider k ∈ E xt(K ). Assume the convex combination for k 1 , k 2 ∈ K and t ∈ (0, 1). As L is injective and using that k ∈ E xt(K ) we conclude that k = k 1 = k 2 and hence Lk = Lk 1 = Lk 2 .
We are now in the position to prove our main theorem.
Proof We apply Propositions 2.5 and 2.12 to find u N ∈ X N a minimizer of Problem 2.3 and w N ∈ H N such that properties (i) and (ii) in Proposition 2.12 hold. If φ N (u N ) = 0, then applying Theorem 2.6, we infer that there exists ψ ∈ N such that ψ is a minimizer of Problem 2.2. Therefore, Eq. (21) holds with p = 0. Hence, we suppose without loss of generality that u N / Hence, using Hypothesis [H1] we infer that B N is compact and thanks to Remark 2.10 we have that A N B N is compact in H N as well. As and p ≤ dim H N + 1. We can assume that p is minimal, in the sense that it is the minimal number such that a decomposition like (23) holds.
Thanks to part (i) in Lemma 3.2 we have that there exist We want to prove that p ≤ dim H N . We claim that for every i = 1, . . . , p we have that Indeed, thanks to Remark 2.13 we have Hilbert space (obviously, w N = 0 in this case). Then, applying Carathéodory's theorem again, we deduce that p ≤ dim H N as a consequence of the minimality of p.

and the linearity of A N we infer that
and in particular A N v N ∈ ∂ F * N (−w N ). In order to deduce that v N is a minimizer for Problem 2.3 using Proposition 2.12, it remains to prove that Indeed, using φ N (v i ) ≤ 1 for each i = 1, . . . , p, Eq. (27) and Remark 2.13, we obtain

On the other hand as
Thus, as a consequence of Proposition 2.12, v N is a minimizer for Problem 2.3. To conclude, notice that there exist u 1 , . . . , u p ∈ X such that u i + N = v i and therefore from (26) we have Then applying Theorem 2.6 we infer that there exists u ∈ X such that u + N = v N and u is a minimizer of Problem 2.2. From the equality we obtain the existence of a ψ ∈ N such that (21) holds.

Remark 3.4
Let us point out similarities and differences to the work [5], where a theorem similar to Theorem 3.3 has been shown. First, instead of seminorms, [5] deals with general convex regularizers. Moreover, in [5], the existence of minimizers for the considered variational inverse problem is assumed a priori, with the goal of disentangling the main result (which is purely geometric) from the topology chosen on X . In contrast, we make suitable assumptions that ensure existence of minimizers for the inverse problem and that the set of extremal points of the balls of the regularizer is non-empty. In such a way, we provide an operative result with hypotheses that can be easily checked. It is worth to notice that both our result and [5] do not provide a sparse representation for every minimizer of the variational inverse problem. However, the points of view are complementary. In [5], the authors characterize, with a help of a theorem by Dubins and Klee [21,28], the minimizers belonging to the finite-dimensional faces of the set of the solutions (we refer to [5] for the definition of the face of a convex set). In particular, when the dimension of a face is zero, i.e., the face is an extremal point, it is possible to obtain a sparse representation of the minimizer in terms of the extremal points and extremal rays of a certain sublevel set of the regularizer (see Section 2 in [5] for the definition of extremal ray). This is still true when the dimension of the face is larger than zero and finite (see Theorem 1 in [5]). Existence of extremal points is then, e.g., obtained by Klee's extension of the Krein-Milman theorem [27] in case of regularizers whose sublevel sets are closed, convex and locally compact in an appropriate locally convex space. On the contrary, our theorem always provides the existence of a minimizer represented as a convex combination of extremal points of the ball of the regularizer. Due to the different techniques used, such a sparse minimizer does not necessarily belong to a finite-dimensional face of the set of the solutions.
Finally, let us point out that in order to obtain sparse representations for solutions of the general variational problem (3), the authors of [5] consider solutions of the optimization problem min u∈X φ(u) subjected to Au = y, (28) i.e., are forced to pass from (3) to (28), then apply Klee-Dubins' theorem, and afterwards use sparse solutions of (28) to construct sparse solutions of (3). We remark that due to the use of different techniques, such a procedure is not required in our paper.

Examples of sparsity for relevant regularizers
In this section we study the structure of the extremal points for relevant regularizers, in order to applying the results of the previous section. The first example is about the Radon norm in the space of measures.

The Radon norm for measures
Given ⊂ R d a non-empty, open, bounded set, we set X = M( ) the set of Radon measures on . We choose φ(u) = u M defined as In order to get more information from Theorem 3.3 we need to characterize the extremal points of B. This result is well-known, but we go through it for the reader's convenience.

Proposition 4.1 Given B defined as above we have that
Proof Let us prove that δ x , −δ x ∈ E xt(B) for every x ∈ . Indeed, let us suppose that there exists u 1 , u 2 ∈ B such that for t ∈ (0, 1). Separating the positive part and negative part of u 1 and u 2 , we can suppose without loss of generality that u 1 ≥ 0 and u 2 ≥ 0. Then supp u 1 ⊂ {x} and hence u 1 = δ x . Similarly one can prove that −δ x ∈ E xt(B) for every x ∈ . On the other hand we prove that there are not other extremal points different from the Dirac deltas. Suppose by contradiction that there exists an extremal point u not supported on a singleton. Then u M = 1 and there exists a measurable set A ⊂ such that 0 < |u|(A) < 1. We have which implies that u is not an extremal point. Hence all the extremal points of B are of the form aδ x where a ∈ R and x ∈ . As the extremal points of B have unit Radon norm we deduce immediately that |a| = 1.
From Proposition 4.1 we obtain immediately the following theorem:

The total variation for BV functions
Let ⊂ R d be a non-empty, bounded Lipschitz domain. We want to apply the result of the previous section for X = BV ( ) and φ(u) = |Du|( ), where This is a relevant setting for inverse problems in image processing as φ(u) = |Du|( ) is the classical TV regularizer which has widely been studied and used in applications. We refer to [2] for the basic definitions regarding BV functions and sets of finite perimeter that we will use.
We equip X with the weak* topology for BV functions by interpreting BV ( ) as a dual space (see, for instance [2,Remark 3.12] Notice that with the chosen topology on X , the functional φ(u) = |Du|( ) is a lower semi-continuous seminorm and A satisfies assumption [H0]. Therefore in order to apply Theorem 3.3 we just need to verify Hypothesis [H1] that is the content of the next lemma. Notice that in this specific case, we have N = R as is connected. Proof We first remark that the metrizability of the space X N on bounded sets is not straightforward to show. Therefore we work with nets instead of sequences (we refer to Sections 1.3, 1.4, 1.6 in [30] for the basic properties of nets).
Consider a net (u β N ) β ⊂ X N such that |Du β |( ) ≤ α. Using the Poincaré inequality for BV functions (see Theorem 3.44 in [2]) we deduce that there exists c β ∈ R such that for every β. Recall now that bounded sets of BV functions are compact with respect to weak* convergence of nets (as mentioned earlier, the space of BV functions is isomorphic to the dual of a separable Banach space according to Remark 3.12 in [2]; this implies compactness of weak*-closed bounded sets by the Banach-Alaoglu theorem). So, thanks to (31), there exists a subnet (not relabelled) (u β + c β ) β in BV ( ) and u ∈ BV ( ) such that u β + c β → u in L 1 ( ) and Du β * Du in M( , R d ) in the sense of nets (see Theorem 1.6.2 in [30]). By the continuity of the projection on the quotient we obtain that see Proposition 1.4.3 in [30]. Then, by the lower semi-continuity of |Du|( ) we have that the sublevel sets of |Du|( ) are weak*-closed. This implies that u ∈ S − (φ N , α) (Proposition 1.3.6 in [30]).
In order to have an explicit representation for the sparse minimizer we aim to characterize the extremal points of the set The result is known for = R d and it was proved in [24] for d = 2 and then extended to all dimensions in [25] and in a slighly different setting in [1]. Our plan is to modify the approach in [1] for the case of bounded. The first definition is taken from [1] and it is a suitable modification of the classical definition for currents given in [23]. Recall that a set of finite perimeter is a measurable set A ⊂ such that χ A ∈ BV ( ) for the characteristic function χ A of A. In this case, we call P(A, ) = |Dχ A |( ) the perimeter of A. In [1], the notion of saturated set is introduced that is suitable in the case = R d . In our case of bounded domains, we do not need this requirement, but we ask that both the set and its complement are indecomposable. In what follows we denote by E 1 the measure theoretic interior of E defined as and by E 0 the measure theoretic exterior: The essential boundary of E is then defined as We will also need the following result due to Dolzmann and Müller [18].

Lemma 4.6 (Constancy theorem) Given u ∈ BV ( ) and E ⊂ an indecomposable set such that
|Du|(E 1 ) = 0, then there exists c ∈ R such that u(x) = c almost everywhere in E.
With the following theorem we are able to characterize the extremal points of B N in a rather straightforward way without relying on indecomposability results for the reduced boundary as in [25].

Theorem 4.7 We have that
Proof We start to prove that Taking u N ∈ E xt(B N ) and choosing u ∈ BV ( ) such that u + N = u N , we have clearly that |Du|( ) = 1. We want to show that u assumes two values almost everywhere. In order to do that we define We have F(−∞) = 0 and by the coarea formula for BV functions, F(+∞) = 1. Moreover, the function t → P({u(x) ≤ t}, ) is integrable on R, so there exists an s ∈ R such that F(s) = 1 2 . Setting u 1 = 2 min(u, s), we see that u = 1 2 u 1 + 1 2 u 2 as well as |Du 1 |( ) = |Du 2 |( ) = 1, the latter again by the coarea formula and the choice of s. As u N is an extremal point of B N , it follows that u N = (u 1 ) N = (u 2 ) N which means that there exist c 1 , c 2 ∈ R such that u = u 1 + c 1 = u 2 + c 2 . Now, for x ∈ such that u(x) ≥ s, this implies u(x) = 2s + c 1 . Likewise, if u(x) ≤ s, then u(x) = c 2 . Hence, u assumes at most two values almost everywhere. However, since |Du|( ) = 1, it assumes exactly two values almost everywhere and 2s +c 1 > c 2 . (Moreover, the set {u(x) = s} must be a null set.) Up to change of the representative u of u N , we can suppose that u(x) ∈ {0, a} almost everywhere, where a > 0. Defining E = {x ∈ : u(x) = a} and using the fact that |Du|( ) = 1 one concludes that u = χ E P (E, ) . Suppose now by contradiction that E is decomposable and let A and B be the sets of finite perimeter given by Definition 4.4. Then, P(A, ) > 0 and P(B, ) > 0 and defining Hence by the properties of A and B given by Definition 4.4, Formula (33) is a non-trivial convex combination of u.
Likewise, suppose by contradiction that \E is decomposable and call A and B its decomposition according to Definition 4.4. Define Notice that So using that {A, B} is a decomposition of \E and the fact that P(E, ) = P( \E, ) we conclude that (34) is a non-trivial convex combination of u.
Thus, E must be a simple set and the first inclusion is proven. Let us prove now the opposite inclusion: Given E ⊂ a simple set, let us suppose that there exists u 1 , u 2 ∈ BV ( ) such that where λ ∈ (0, 1). This means that there exists c ∈ R such that χ E P(E, ) Notice that for every A ⊂ measurable one has ) we would arrive at the contradiction In other words, χ E P(E, ) + N = u 1 + N = u 2 + N , so χ E P(E, ) + N is indeed an extremal point.
We have shown the following theorem.

Radon norm of a scalar differential operator
In this section we consider the case where φ(u) = Lu M , namely the Radon norm of a linear, translation-invariant scalar differential operator L. This was already treated in [26,36] in different settings. Our goal is to show that our theory applies straightforwardly to this case. We start some useful properties of scalar differential operators that we are going to use. In what follows we denote by α = (α 1 , . . . , α d ) ∈ N d a multi-index and we employ the standard multi-index notation and conventions.

Some technical lemmas
We consider a non-zero differential operator with linear coefficients of order q ∈ N of the form where each c α ∈ R and c α = 0 for some |α| = q. We also denote by L * the operator defined formally by The existence of a fundamental solution G for L is ensured by virtue of the classical Malgrange-Ehrenpreis theorem (see for example Theorem 8.5 in [33]).

Lemma 4.11
There exists C ∈ R and s ∈ N such that for every μ ∈ M( ) one has for every test function ϕ ∈ D( ). In particular, for each μ ∈ M( ), T μ can be extended to a unique element in C s 0 ( ) * such that T : M( ) → C s 0 ( ) * becomes a linear and continuous mapping.
Proof Consider a test function ϕ ∈ D( ) and denote by ϕ its zero extension to R d . Then, the order of G is finite on bounded sets which means that there exists s ∈ N such that whose support is contained in a compact set that only depends on . Notice now that for every x ∈ R d we have meaning that T μ can be extended, by density, to an element in C s 0 ( ) * . The latter also establishes the claimed continuity of T : M( ) → C s 0 ( ) * .

Existence of a sparse minimizer
Recall that we consider the differential operator L given in Eq. (37). With s is given by Lemma 4.11, we set X = C s 0 ( ) * , the space of distributions of order s equipped with the weak* topology. From now on we consider the weak differential operator L mapping between X → C s+q 0 ( ) * . Notice that with this definition, L is a continuous operator when X and C s+q 0 ( ) * are equipped with the weak* topology. Indeed, the adjoint L * according to (38) maps continuously between the spaces C s+q 0 ( ) → C s 0 ( ) as a classical differential operator. Thus, considering u n * u in X and ϕ ∈ C s+q 0 ( ) we have L * ϕ ∈ C s 0 ( ) and hence, which establishes the weak*-continuity as due to separability of C s 0 ( ) and C s+q 0 ( ), it suffices to consider sequences.

Remark 4.12
Notice that φ is a seminorm and it is lower semi-continuous in X (with respect to the weak* topology). Indeed, once again, as C s 0 ( ) is separable we know that weak* lower semi-continuity for L is equivalent to weak* sequential lower semi-continuity. Therefore, we consider a sequence (u n ) n ⊂ X such that u n * u in X and we suppose without loss of generality that lim inf n→+∞ Lu n M < +∞ and lim n→+∞ Lu n M = C.
Then, by weak* sequential compactness of measures there exists v ∈ M( ) such that, up to subsequences, Lu n * v in M( ) and in particular in C s+q 0 ( ) * . As L is weak*-weak* closed we infer that v = Lu and from the lower semi-continuity of the Radon norm with respect to weak* convergence in M( ) we conclude that Lu M ≤ C.
In order to apply Theorem 3.3, it remains to verify Assumption [H1]. This is the content of the next proposition. We remind that X N = X + N (equipped with the quotient of the weak* topology of X ) where N is the null-space of L and φ N (u N ) = φ N (u + N ) := φ(u) (for notational convenience we denote by L the operator acting on X N in the natural way).
Proof Similarly to the proof of Lemma 4.3 we employ nets since metrizability of the space X N does not play a role in this context.
Given u N ∈ S − (φ N , α) we have thanks to Remark 4.10 that Therefore, there exists ψ ∈ N such that u + ψ = T Lu N , where u ∈ X is such that u + N = u N . Moreover, with the help of Lemma 4.11, it follows that for every ϕ ∈ C s 0 ( ). Hence, Consider now a net (u for every β. Thus, there exists a net (ψ β ) β in N andC > 0 such that Applying the Banach-Alaoglu theorem we extract a subnet (not relabelled) of (u β + ψ β ) β that is converging to u ∈ X in the weak* topology of X (Theorem 1.6.2 in [30]). As the projection on the quotient is a continuous operation we deduce also that (Proposition 1.4.3 in [30]). It remains to show that u N ∈ S − (φ N , α). Thanks to Remarks 4.12 and 2.1, the functional φ N : X N → [0, ∞] is lower semi-continuous with respect to the quotient topology in X N and therefore, its sublevel sets are closed. This implies that u N ∈ S − (φ N , α) (Proposition 1.3.6 in [30]).
We are now in position to apply Theorem 3.3. Consider A : X → H a linear continuous operator such that [H0] holds and F : H → (−∞, +∞] satisfying the assumptions in Sect. 2.1.
We set the following variational problem: Thanks to Proposition 4.13, Theorem 3.3 is applicable. We can furthermore characterize the extremal points of the ball associated to φ N according to the following theorem. Note that a similar result was also obtained by [26,36] in different settings and more restrictive hypotheses. For this purpose, for x ∈ R d , denote by G x the fundamental solution G translated by x, i.e., such that LG x = δ x .
Proof With π N denoting the quotient map X → X N , we have due to Remark 4.10 that Notice that π N • T : M( ) → X N is a linear, injective map. Indeed, let us suppose that (π N • T )(μ) = 0. Then there exists ψ ∈ N such that T μ = ψ. Applying L on both sides and using Remark 4.10 we deduce that μ = 0. Hence we can apply part (ii) of Lemma 3. So applying Theorem 3.3 and noting that T δ x = G x one concludes.

Conclusions and open problems
The abstract main result of this paper contained in Theorem 3.3 about the structure of a minimizer of a variational problem with finite dimensional data appears to be widely applicable, thanks to its generality. The usability of this theorem to concrete problems relies, however, on the characterization of the extremal points of the unit ball associated with the given regularizer. Such a characterization appears to be fundamental for devising suitable algorithms that rely on the structure of the minimizers given by Theorem 3.3.
In this paper we essentially carried out this characterization for two specific regularizers: • The total variation of a function with bounded variation.
• The Radon norm of a scalar differential operator.
In the meantime, a follow-up paper also provides the characterization of extremal points for the Benamou-Brenier energy in optimal transport [6]. A challenging direction of further research is the study of the extremal points of balls associated with other classes of regularizers. For example, it would be of great interest to be able to treat the case of the Radon norm for general vector-valued differential operators. This would lead to the consideration, as an instance among others, of T V 2 regularization (see for example [14]) which is defined as T V 2 (u) = sup ∇u · div ϕ dx : ϕ ∈ C 1 c ( , R d×d ), ϕ ∞ ≤ 1 , that is the total variation of the weak gradient of an L 1 function. As a consequence, it would be possible to compare the regularizing effect of the T V 2 seminorm and the T V seminorm, leading, e.g., to a better understanding of how higher-order regularizers reduce the staircase effect.
Additionally, one can also consider more complex regularizers that were studied to overcome the limitations of T V and T V 2 models. For example, in [8], the so called total generalized variation was introduced, which is defined in the following way: where Sym k (R d ) is the space of symmetric tensors of order k and α = (α 0 , . . . , α k−1 ) are positive parameters. The characterization of extremal points of the ball associated with these particular regularizers is, up to our knowledge, still not known and would lead to a deep understanding of the regularization effects in respective variational models.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.