Maximal Solutions of Sparse Analysis Regularization

Abstract

This paper deals with the non-uniqueness of the solutions of an analysis—Lasso regularization. Most previous works in this area are concerned with the case, where the solution set is a singleton, or to derive guarantees to enforce uniqueness. Our main contribution consists in providing a geometrical interpretation of a solution with a maximal analysis support: such a solution abides in the relative interior of the solution set. Our result allows us to provide a way to exhibit a maximal solution using a primal-dual interior point algorithm.

Appendix

In this section, we propose to express some of our results in a general framework. More precisely, we consider the following optimization problem

$$\begin{aligned} \min _{x\in E} \{f(x) + g(x)\} . \end{aligned}$$

(14)

where E is a Banach space, $f, g : E\mapsto {\mathbb {R}}\cup \{+\infty \}$ are convex lower semicontinuous functions. The dual space of E and the pairing between E and $E^*$ will be denoted by $E^*$ and $\langle \cdot , \cdot \rangle$, respectively. The Fenchel subdifferential of f at ${\bar{x}}$ is defined by

$$\begin{aligned} \partial f({\bar{x}}) \mathrel {\mathop :}=\{ x^*\in E^*: \, \langle x^*, x-{\bar{x}}\rangle \le f(x) - f({\bar{x}}) \, \forall x\in E\}. \end{aligned}$$

The aim of the following proposition is to give a characterization of solutions of the problem (14).

Proposition A.1

Let ${\bar{x}}\in E$ be a fixed solution of the problem (14) and $x^*\in \partial g({\bar{x}})$ be such that $-x^* \in \partial f({\bar{x}})$. Then the following assertions are equivalent:

• (1) u is a solution of the problem (14),

• (2) $g(u) \le g({\bar{x}}) + \langle x^*, u-{\bar{x}}\rangle$ and u is a solution of the problem

  $$\begin{aligned} \min _{x\in E} \{f(x) +\langle x^*, x\rangle \}. \end{aligned}$$

  (15)

Consequently, if $\{ x\in E : \, g(x) \le g({\bar{x}}) + \langle x^*, x-{\bar{x}}\rangle \}$ is a polyhedral set and the function f is polyhedral (supremum of a finite affine family), then so is $\underset{x \in {\mathbb {R}}^n}{{{\mathrm{Argmin}}}}\;\{f(x)+g(x)\}$.

Proof

Since the implication $(2) \Longrightarrow (1)$ is obvious, we will establish only the implication $(1) \Longrightarrow (2)$. First note that, because of our assumptions, assertion (2) is equivalent to say that $x^*\in \partial g(u)$ and $-x^*\in \partial f(u)$. So if u is a solution of the problem (14), we have

$$\begin{aligned} f(u)+g(u) = f({\bar{x}})+g({\bar{x}}). \end{aligned}$$

(16)

Since $x^*\in \partial g({\bar{x}})$ and $-x^*\in \partial f({\bar{x}})$, we easily obtain, by using relation (16), that $x^*\in \partial g(u)$ and $-x^*\in \partial f(u)$ and the proof is completed. $\square$

A particular and interesting case is the Hilbert setting with a special form of g.

Corollary A.1

Suppose that E (resp. F) is a Hilbert endowed with a scalar product denoted by $\langle \cdot , \cdot \rangle$ and the associated norm $\Vert \cdot \Vert$. Let ${\varPhi }: E\mapsto F$ be a linear continuous operator and $y\in F$. Define the function $g : E \mapsto {\mathbb {R}}$ by

$$\begin{aligned} g(x) = {1\over 2}\Vert {\varPhi }x-y\Vert ^2. \end{aligned}$$

Let ${\bar{x}}\in E$ be a fixed solution of the problem (14) and put $x^* = {\varPhi }^*({\varPhi }{\bar{x}}-y)$. Then the following assertions are equivalent:

• (1) u is a solution of the problem (14),

• (2) ${\varPhi }u={\varPhi }{\bar{x}}$ and u is a solution of the problem

  $$\begin{aligned} \min _{x\in E} \{f(x) +\langle x^*, x\rangle \}. \end{aligned}$$

  (17)

Consequently, each solution u of the problem (14) satisfies ${\varPhi }u={\varPhi }{\bar{x}}$ and $f(u) = f({\bar{x}})$.

Proof

It suffices to see that the (in)equality $g(u) \le g({\bar{x}}) + \langle x^*, u-{\bar{x}}\rangle$ is equivalent to ${\varPhi }u={\varPhi }{\bar{x}}$ and to apply Proposition A.1. $\square$

The following corollary asserts that knowing one solution of (14), we can determine all the other ones.

Corollary A.2

Let the assumptions of Corollary A.1 be satisfied. Then

$$\begin{aligned} \underset{x \in {\mathbb {R}}^n}{{{\mathrm{Argmin}}}}\;\{f(x)+g(x)\} = \{ x \in E: \, {\varPhi }x={\varPhi }{\bar{x}}, \, f(x) = f({\bar{x}})\}. \end{aligned}$$