A Note on the Morozov Principle via Lagrange Duality

Abstract

Considering a general linear ill-posed equation, we explore the duality arising from the requirement that the discrepancy should take a given value based on the estimation of the noise level, as is notably the case when using the Morozov principle. We show that, under reasonable assumptions, the dual function is smooth, and that its maximization points out the appropriate value of Tikhonov’s regularization parameter. The numerical relevance of our approach is established by means of an illustrative example from nonparametric instrumental regression, a standard problem in statistics.

Appendix: Algorithmic Details

In order to maximize D, which we reformulate here as the minimization of $\bar {D}:=-D$, we combine a Quasi-Newton method with Wolfe-Lemaréchal line-search (see the algorithms 1 and 2 below).

Recall that at each iteration k, we have an update of the form λ_k+ 1 = λ _k + α _k d _k , where d _k denotes the direction of descent computed via $\bar {D}^{\prime }$ and an approximation of $\bar {D}^{\prime \prime }$. The stepsize α _k is chosen so as to satisfy the two Wolfe conditions (C1) and (C2) (see [11]) in order to guarantee the monotonicity of the sequence $\bar {D}(\lambda _{k})$:

$$\begin{array}{ll} (C1) :\quad \bar{D} (\lambda_{k} + \alpha_{k} d_{k}) - \bar{D}(\lambda_{k}) - \alpha_{k} \beta_{1} \bar{D}^{\prime}(\lambda_{k}) d_{k} <0 \\ (C2) :\quad \bar{D}^{\prime} (\lambda_{k} + \alpha_{k} d_{k}) d_{k} - \beta_{2} \bar{D}^{\prime}(\lambda_{k}) d_{k} \geq 0. \end{array} $$

In (C1) and (C2), the parameters β₁ and β₂ are taken in (0,1) (see [3, 15]). In Algorithm 1, the stopping criterion is $| \bar {D}^{\prime }(\lambda _{k})| < \epsilon$, with 𝜖 > 0, defining the tolerance. For computing α _k at line 6, we propose Algorithm 2, which is based on the line-search algorithm (see [11]).

In Algorithm 2, (α _g ,α _d ) is the interval in which the step α _k will be chosen. Here, M is a large number which emulates ∞. For example, we may set M = 10¹⁰. Recall that the failure of Condition (C1) means that α _k is too large, while the failure of Condition (C2) means that α _k is too small. If it happens that $|\alpha _{d} - \alpha _{g}| \thickapprox 0$, indicating that β₁ is too big or β₂ is too small, one may then merely adjust these parameters. This situation occurs rarely. For instance, β₁ = 0.25 and β₂ = 0.75 worked perfectly well for all our simulations.