Regularized Linear Inversion with Randomized Singular Value Decomposition

Abstract

In this work, we develop efficient solvers for linear inverse problems based on randomized singular value decomposition (RSVD). This is achieved by combining RSVD with classical regularization methods, e.g., truncated singular value decomposition, Tikhonov regularization, and general Tikhonov regularization with a smoothness penalty. One distinct feature of the proposed approach is that it explicitly preserves the structure of the regularized solution in the sense that it always lies in the range of a certain adjoint operator. We provide error estimates between the approximation and the exact solution under canonical source condition, and interpret the approach in the lens of convex duality. Extensive numerical experiments are provided to illustrate the efficiency and accuracy of the approach.

Appendix A: Iterative refinement

Proposition 1 enables iteratively refining the inverse solution when RSVD is not sufficiently accurate. This idea was proposed in [29, 34] for standard Tikhonov regularization, and we describe the procedure in a slightly more general context. Suppose $\mathscr {N}(L)=\{0\}$. Given a current iterate $x^j$, we define a functional $J_\alpha ^j(\delta x)$ for the increment $\delta x$ by

$$\begin{aligned} J_\alpha ^j(\delta x) : = \Vert A(\delta x+x^j)-b\Vert ^2 + \alpha \Vert L(\delta x+x^j)\Vert ^2. \end{aligned}$$

Thus the optimal correction $\delta x_\alpha$ satisfies

$$\begin{aligned} (A^*A + \alpha L^*L)\delta x_\alpha = A^*(b-Ax^j)-\alpha L^*Lx^j, \end{aligned}$$

i.e.,

$$\begin{aligned} (B^* B + \alpha I) L\delta x_\alpha = B^* (b-Ax^j)-\alpha Lx^j, \end{aligned}$$

(20)

with $B=AL^\dag$. However, its direct solution is expensive. We employ RSVD for a low-dimensional space $\tilde{V}_k$ (corresponding to B), parameterize the increment $L\delta x$ by $L\delta x=\tilde{V}_k^*z$ and update z only. That is, we minimize the following functional in z

$$\begin{aligned} J_\alpha ^j(z) : = \Vert A(L^{\dag }\tilde{V}_k^*z+x^j)-b\Vert ^2 + \alpha \Vert z+\tilde{V}_k L x^j\Vert ^2. \end{aligned}$$

Since $k\ll m$, the problem can be solved efficiently. More precisely, given the current estimate $x^j$, the optimal z solves

$$\begin{aligned} (\tilde{V}_k B^* B\tilde{V}_k^* + \alpha I) z = \tilde{V}_k B^*(b-A x^j)-\alpha \tilde{V}_kLx^j. \end{aligned}$$

(21)

It is the Galerkin projection of (20) for $\delta x_\alpha$ onto the subspace $\tilde{V}_k$. Then we update the dual $\xi$ and the primal x by the duality relation in Sect. 6:

$$\begin{aligned} \xi ^{j+1}&= b-Ax^j - B\tilde{V}_k^* z^j,\end{aligned}$$

(22)

$$\begin{aligned} x^{j+1}&= \alpha ^{-1}\varGamma A^*\xi ^{j+1}. \end{aligned}$$

(23)

Summarizing the steps gives Algorithm 3. Note that the duality relation (17) enables A and $A^*$ to enter into the play, thereby allowing progressively improving the accuracy. The main extra cost lies in matrix-vector products by A and $A^*$.

The iterative refinement is a linear fixed-point iteration, with the solution $x_\alpha$ being a fixed point and the iteration matrix being independent of the iterate. Hence, if the first iteration is contractive, i.e., $\Vert x^1-x_\alpha \Vert \le c\Vert x^0-x_\alpha \Vert$, for some $c\in (0,1)$, then Algorithm 3 converges linearly to $x_\alpha$. It can be satisfied if the RSVD approximation $(\tilde{U}_k,\tilde{\varSigma }_k,\tilde{V}_k)$ is reasonably accurate to B.