Should We Search for a Global Minimizer of Least Squares Regularized with an ℓ₀ Penalty to Get the Exact Solution of an under Determined Linear System?

Abstract

We study objectives ${\mathcal{F}}_d$ combining a quadratic data-fidelity and an ℓ₀ regularization. Data d are generated using a full-rank M ×N matrix A with N > M. Our main results are listed below.

Minimizers of ${\mathcal{F}}_d$ are strict if and only if length(support())$\leqslant M$ and the submatrix of A whose columns are indexed by support() is full rank. Their continuity in data is derived. Global minimizers are always strict.

We adopt a weak assumption on A and show that it holds with probability one. Data read $d=A{\ddot{u}}$ where length(support(${\ddot{u}}$))$\leqslant M-1$ and the submatrix whose columns are indexed by support(${\ddot{u}}$) is full rank. Among all strict (local) minimizers of ${\mathcal{F}}_d$ with support shorter than M − 1, the exact solution is the unique vector that cancels the residual. The claim is independent of the regularization parameter. This is usually a strict local minimizer where ${\mathcal{F}}_d$ does not reach its global minimum. Global minimization of ${\mathcal{F}}_d$ can then prevent the recovery of ${\ddot{u}}$.

A numerical example (A is 5 ×10) illustrates our main results.