Choice of the parameters in a primal-dual algorithm for Bregman iterated variational regularization

Abstract

Focus of this work is solving a non-smooth constraint minimization problem by a primal-dual splitting algorithm involving proximity operators. The problem is penalized by the Bregman divergence associated with the non-smooth total variation (TV) functional. We analyze two aspects: Firstly, the convergence of the regularized solution of the minimization problem to the minimum norm solution. Second, the convergence of the iteratively regularized minimizer to the minimum norm solution by a primal-dual algorithm. For both aspects, we use the assumption of a variational source condition (VSC). This work emphasizes the impact of the choice of the parameters in stabilization of a primal-dual algorithm. Rates of convergence are obtained in terms of some concave, positive definite index function. The algorithm is applied to a simple two-dimensional image processing problem. Sufficient error analysis profiles are provided based on the size of the forward operator and the noise level in the measurement.

Appendices

Appendix A: VSC as upper bound for the Bregman distance

The total error estimation can also be stabilized due to the following assumption that has been derived in the literature listed in Section 4.2,

$$ E({u}_{\alpha}^{\delta} , u^{\dagger}) \leq D_{\mathcal{J}}({u}_{\alpha}^{\delta} , u^{\dagger}). $$

(1.1)

Therefore, for stabilization of E, we seek a stable upper bound for the Bregman distance (1.1).

Assumption A.1

[Variational Source Condition] Let $T : \mathcal {X} \rightarrow \mathcal {Y}$ be linear, injective forward operator and v^‡∈range(T). There exists some constant σ ∈ (0, 1] and a concave, monotonically increasing index function Ψ with Ψ(0) = 0 and ${\Psi } : [0 , \infty ) \rightarrow [0 , \infty )$ such that for $q^{\dagger } \in \partial \mathcal {J}(u^{\dagger })$ the minimum norm solution u^‡∈BV(Ω) satisfies

$$ \sigma D_{\mathcal{J}}(u,u^{\dagger}) \leq \mathcal{J}(u) - \mathcal{J}(u^{\dagger}) + {\Psi}\left( \Vert T u - T u^{\dagger}\Vert \right) \text{, for all } u \in \mathcal{X} . $$

(1.2)

Recall that quantitative stability analysis in the continuous mathematical setting aims to find upper bound for the total error estimation functional E in (4.4). According to (1.1), by means of finding stable upper bound for the Bregman distance between the regularized minimizer ${u}_{\alpha }^{\delta }$ and the minimum norm solution u^‡ will yield one of the two convergence results of this section. With the established choice of the regularization parameter and the asserted $\mathcal {J}$ difference estimation in Lemma 6.2, the last ingredient of the Bregman distance following up the Assumption A2.1 is formulated below.

Lemma A.2

Let $\alpha (\delta ,v^{\delta })\in \overline {S}\cap \underline {S}$ be the regularization parameter for the regularized solution ${u}_{\alpha }^{\delta }$ to the problem (4.2). If the minimum norm solution u^‡ satisfies Assumption A2.1, then

$$ - \langle D^{\ast}w^{\dagger},{u}_{\alpha}^{\delta} - u^{\dagger} \rangle = \mathcal{O}({\Psi}(\delta)), $$

holds.

Proof

It follows from VSC (4.5) that

$$ \begin{array}{@{}rcl@{}} &&\frac{\sigma}{2}\left( \mathcal{J}({u}_{\alpha}^{\delta}) - \mathcal{J}(u^{\dagger}) - \langle D^{\ast}w^{\dagger},{u}_{\alpha}^{\delta} - u^{\dagger}\rangle\right)\\ &\leq& \mathcal{J}({u}_{\alpha}^{\delta}) - \mathcal{J}(u^{\dagger}) + {\Psi}(\Vert T {u}_{\alpha}^{\delta} - T u^{\dagger}\Vert). \end{array} $$

(1.3)

After arranging the terms,

$$ \begin{array}{@{}rcl@{}} -\langle D^{\ast}w^{\dagger},{u}_{\alpha}^{\delta}-u^{\dagger}\rangle & \leq & \frac{2}{\sigma}\left( 1 - \frac{\sigma}{2}\right) \left( \mathcal{J}({u}_{\alpha}^{\delta}) - \mathcal{J}(u^{\dagger}) \right) + {\Psi}(\Vert T {u}_{\alpha}^{\delta} - T u^{\dagger}\Vert) \\ & \overset{(6.4)}{\leq} & \frac{2}{\sigma}\left( 1 - \frac{\sigma}{2}\right) \left( \frac{1}{\underline{\tau}-1}\right){\Psi}(\delta) + {\Psi}(\Vert T {u}_{\alpha}^{\delta} - T u^{\dagger}\Vert) \\ & \overset{(4.15)}{\leq} & \frac{2}{\sigma}\left( 1 - \frac{\sigma}{2}\right) \left( \frac{1}{\underline{\tau}-1}\right){\Psi}(\delta) + {\Psi}((\overline{\tau} + 1)\delta) \\ & \overset{(4.7)}{\leq} & \frac{2}{\sigma}\left( 1 - \frac{\sigma}{2}\right) \left( \frac{1}{\underline{\tau}-1}\right){\Psi}(\delta) + (\overline{\tau} + 1){\Psi}(\delta) \\ & = &\left( \frac{2}{\sigma} -1\right) \left( \frac{1}{\underline{\tau}-1}\right){\Psi}(\delta) + (\overline{\tau} + 1){\Psi}(\delta). \end{array} $$

(1.4)

□

Theorem A.3

Let the minimum norm solution u^‡∈Ω satisfy the VSC given by Assumption A2.1. Then, in the light of the assumptions of lemmata 6.1, 6.2 and finally A2.2, the following estimation holds:

$$ D_{\mathcal{J}}({u}_{\alpha}^{\delta} , u^{\dagger}) = \mathcal{O}({\Psi}(\delta)). $$

Proof

The proof simply follows from verifying the previously established estimations on each components of the Bregman distance as shown below:

$$ \begin{array}{@{}rcl@{}} D_{\mathcal{J}}({u}_{\alpha}^{\delta} , u^{\dagger}) & = & \mathcal{J}({u}_{\alpha}^{\delta}) - \mathcal{J}(u^{\dagger})- \langle D^{\ast}w^{\dagger},{u}_{\alpha}^{\delta} - u^{\dagger} \rangle \\ &\overset{(1.4)}{\leq}& \mathcal{J}({u}_{\alpha}^{\delta}) - \mathcal{J}(u^{\dagger}) + \left( \frac{2}{\sigma} - 1\right) \left( \frac{1}{\underline{\tau} - 1}\right){\Psi}(\delta) + (\overline{\tau} + 1){\Psi}(\delta) \\ &\overset{(6.4)}{\leq}& \left( \frac{1}{\underline{\tau} - 1} \right){\Psi}(\delta)+ \left( \frac{2}{\sigma} - 1 \right) \left( \frac{1}{\underline{\tau} - 1} \right){\Psi}(\delta) + (\overline{\tau} + 1){\Psi}(\delta). \end{array} $$

(1.5)

□

Appendix B: Further numerical results

In this section, we will present some further numerical results to emphasize the condition on the step length μ in Theorem 7.2. Although the formulated condition allows one to choose the step length as $\mu = \frac {2}{\Vert T \Vert ^{2}},$ we have observed divergence when we have made this choice of μ (see Fig. 7).

Fig. 7

Error analysis profiles and the data visualization: although the noise amount is sufficiently small and it is a full-rank system, with some different choice of the step length that is defined as $\mu = \frac {2}{\Vert T\Vert ^{2}},$ insufficient reconstruction and instability have been observed