Multiscale Support Vector Approach for Solving Ill-Posed Problems

Abstract

Based on the use of compactly supported radial basis functions, we extend in this paper the support vector approach to a multiscale support vector approach (MSVA) scheme for approximating the solution of a moderately ill-posed problem on bounded domain. The Vapnik’s $\epsilon$-intensive function is adopted to replace the standard $l^2$ loss function in using the regularization technique to reduce the error induced by noisy data. Convergence proof for the case of noise-free data is then derived under an appropriate choice of the Vapnik’s cut-off parameter and the regularization parameter. For noisy data case, we demonstrate that a corresponding choice for the Vapnik’s cut-off parameter gives the same order of error estimate as both the a posteriori strategy based on discrepancy principle and the noise-free a priori strategy. Numerical examples are constructed to verify the efficiency of the proposed MSVA approach and the effectiveness of the parameter choices.

Appendix: Proofs of Lemmas 4–11

Proof of Lemma 4

Referring to Algorithm 1, the local reconstructed solution $s_k^{\epsilon ,\gamma ,\delta }$ at each level $k$ is determined by following minimization problem:

$$\begin{aligned} \min \limits _{s\in H^{\tau }(\mathbb {R}^d)}\mathcal {J}^{\delta }_k(s)&= \min \limits _{s\in H^{\tau }(\mathbb {R}^d)}\left( ~\sum _{j=1}^{N_k}\left| e_{k-1}^{\delta } (x_{k,j})-A^{(k)}s(x_{k,j})\right| _{\epsilon _k} +\gamma _k\Vert s\Vert _{\varPhi _k}^2\right) . \end{aligned}$$

Taking $Es_k^{\delta ,*}\in H^{\tau }(\mathbb {R}^d)$ as a feasible candidate, note for each $k$

$$\begin{aligned} \left| e_{k-1}^{\delta }(x_{k,j})-A^{(k)}s(x_{k,j})\right| _{\epsilon _k}\le \left( \Vert e_{k-1}^{\delta }-A^{(k)}s\Vert _{l^{\infty }(X_k)}-\epsilon _k\right) _+. \end{aligned}$$

Then, the estimate

$$\begin{aligned} \Vert e_{k-1}^{\delta } - A^{(k)}s_k^{\delta ,*}\Vert _{l^{\infty }(X_k)}&= \Vert g^{\delta } - A^{(k)}f_{k-1}^{\epsilon ,\gamma ,\delta } - A^{(k)}(f^*-f_{k-1}^{\epsilon ,\gamma ,\delta })\Vert _{l^{\infty }(X_k)}\\&= \Vert g^{\delta } -A^{(k)}f^* \Vert _{l^{\infty }(X_k)}\\&\le \Vert g^{\delta }-g\Vert _{l^{\infty }(X_k)} + \Vert (A-A^{(k)})f^*\Vert _{l^{\infty }(X_k)}\\&\le \delta _k + Kh_k^{r}{\varrho }\, \end{aligned}$$

and the minimizing property of the objective functional yield the estimates.$\square$

Proof of Lemma 5

The first estimate is derived consequently from the minimality of the functional $\mathcal {J}_k(s_k^{\epsilon ,\gamma })$ in (15), i.e.,

$$\begin{aligned} \gamma _k\Vert s_k^{\epsilon ,\gamma }\Vert _{\varPhi _k}\le \mathcal {J}_k(s_k^{\epsilon ,\gamma })\le \mathcal {J}_k(Es_k^*) = \gamma _k\Vert Es_k^{*}\Vert _{\varPhi _k}. \end{aligned}$$

The second estimate follows from

$$\begin{aligned} \Vert s_{k+1}^*\Vert _{H^{\tau }(\varOmega )}&= \Vert f^*-f_{k}^{\epsilon ,\gamma }\Vert _{H^{\tau }(\varOmega )} = \Vert f^{*}- (f_{k-1}^{\epsilon ,\gamma } + s_k^{\epsilon ,\gamma })\Vert _{H^{\tau }(\varOmega )} \\&\le \Vert s_k^*\Vert _{H^{\tau }(\varOmega )} + \Vert s_k^{\epsilon ,\gamma }\Vert _{H^{\tau }(\varOmega )}\le \Vert Es_k^*\Vert _{H^{\tau }(\mathbb {R}^d)} + \Vert s_k^{\epsilon ,\gamma }\Vert _{H^{\tau }(\mathbb {R}^d)} \\&\le c_2^{1/2}\eta _k^{-\tau }\left( \Vert Es_k^*\Vert _{\varPhi _k} + \Vert s_k^{\epsilon ,\gamma }\Vert _{\varPhi _k}\right) \\&\le 2c_2^{1/2}\eta _k^{-\tau }\Vert Es_k^*\Vert _{\varPhi _k}. \end{aligned}$$

Utilizing [A5] in Assumption 1, the second estimate and noticing the fact that $\Vert \cdot \Vert _{l^{\infty }(X_k)}\le |\cdot |_{\epsilon _k}+\epsilon _k$, derivation for the third estimate is given as follow:

$$\begin{aligned} \Vert As_{k+1}^*\Vert _{l^{\infty }(X_k)}&= \Vert g - Af_{k}^{\epsilon ,\gamma }\Vert _{l^{\infty }(X_k)} \\&\le \Vert g-A^{(k)}f_{k}^{\epsilon ,\gamma }\Vert _{l^{\infty }(X_k)} + \Vert (A-A^{(k)})f_{k}^{\epsilon ,\gamma }\Vert _{l^{\infty }(X_k)}\\&\le \max _{j=1,\ldots ,N_k}\left| g_j - A^{(k)}f_{k}^{\epsilon ,\gamma }(x_{k,j})\right| _{\epsilon _k}+ \epsilon _k + \Vert (A-A^{(k)})f_{k}^{\epsilon ,\gamma }\Vert _{l^{\infty }(X_k)}\\&\le \sum _{j=1}^{N_k}\left| g_j - A^{(k)}f_{k}^{\epsilon ,\gamma }(x_{k,j})\right| _{\epsilon _k} + \epsilon _k + Kh_k^r\Vert f_{k}^{\epsilon ,\gamma }\Vert _{H^{\tau }(\varOmega )}\\&\le \epsilon _k + \gamma _k\Vert Es_k^*\Vert _{\varPhi _k}^2 + Kh_k^r\left( {\varrho }+ \Vert f^*-f_{k}^{\epsilon ,\gamma }\Vert _{H^{\tau }(\varOmega )}\right) \\&\le \epsilon _k + \gamma _k\Vert Es_k^*\Vert _{\varPhi _k}^2 + Kh_k^r\left( {\varrho }+\Vert s_{k+1}^*\Vert _{H^{\tau }(\varOmega )}\right) \\&\le \epsilon _k + \gamma _k\Vert Es_k^*\Vert _{\varPhi _k}^2 + Kh_k^r\left( {\varrho }+ 2c_2^{1/2}\eta _k^{-\tau }\Vert Es_k^*\Vert _{\varPhi _k}\right) . \end{aligned}$$

$\square$

Proof of Lemma 6

For sufficiently small $h_1$ satisfying $\eta _{k+1}<\cdots <\eta _1 = \nu h_1^{\beta }\in (0,1)$, utilizing the inequality (13), we have

$$\begin{aligned} \Vert Es_{k+1}^{*}\Vert _{\varPhi _{k+1}}^2&= \int _{\mathbb {R}^d}\frac{|\widehat{Es_{k+1}^{*}}(\omega )|^2}{\widehat{\varPhi _{k+1}}(\omega )}d\omega \\&\le \frac{1}{c_1} \int _{\mathbb {R}^d} |\widehat{Es_{k+1}^{*}}(\omega )|^2(1+\eta _{k+1}^2 \Vert \omega \Vert _2^2)^{\tau } d\omega := \frac{1}{c_1}\left( I_1+I_2\right) \end{aligned}$$

with

$$\begin{aligned} I_1&= \int _{\Vert \omega \Vert _2 \le \frac{1}{\eta _{k+1}}} |\widehat{Es_{k+1}^{*}}(\omega )|^2(1+\eta _{k+1}^2 \Vert \omega \Vert _2^2)^{\tau } d\omega ,\\ I_2&= \int _{\Vert \omega \Vert _2 \ge \frac{1}{\eta _{k+1}}} |\widehat{Es_{k+1}^{*}}(\omega )|^2(1+\eta _{k+1}^2 \Vert \omega \Vert _2^2)^{\tau } d\omega . \end{aligned}$$

From the extension operator in (5), Assumption 1, estimates in Lemma 3, and the choice strategies for two parameters $\epsilon _k$ and $\gamma _k$, the integration $I_1$ can be estimated as

$$\begin{aligned} I_1&\le 2^{\tau } \Vert Es_{k+1}^{*}\Vert _{L^2(\mathbb {R}^d)}^2 \le 2^{\tau } C_{0}^2 \Vert s_{k+1}^{*}\Vert _{L^2(\varOmega )}^2 \le \frac{ 2^{\tau } C_{0}^2}{c_{3}^2} \Vert As_{k+1}^*\Vert _{H^{\alpha }(\varOmega )}^2\\&\le \frac{2^{\tau } C_{0}^2 C_d^2}{c_{3}^2}\left( c_{4}h_{k}^{\tau }\Vert s_{k+1}^{*}\Vert _{H^{\tau }(\varOmega )} +h_{k}^{-\alpha }\Vert As_{k+1}^{*}\Vert _{l^{\infty }(X_{k})}\right) ^2.\\&\le \frac{2^{\tau } C_{0}^2 C_d^2}{c_{3}^2}\left( \frac{2c_2^{1/2}\left( c_{4}h_{k}^{(1-\beta )\tau } +Kh_k^{r-\alpha -\beta \tau }\right) \mu ^{\beta \tau }}{T^{\tau }}\Vert Es_{k}^{*}\Vert _{\varPhi _{k}}\right. \nonumber \\&\qquad \left. +\frac{\kappa \mu ^{\beta \tau }}{T^{\tau }}\Vert Es_{k}^{*}\Vert _{\varPhi _{k}}^2+ 2K {\varrho }h_k^{r-\alpha }\right) ^2. \end{aligned}$$

Noting that the choice of the parameter $\beta =\min \{1, \frac{r-\alpha }{\tau }\}$ guarantees that $1-\beta \ge 0$ and $r-\alpha -\beta \tau \ge 0$, we thus obtain

$$\begin{aligned} \sqrt{I_1}&\le \frac{2^{\tau /2}C_{0} C_d}{c_{3}}\left( \frac{2c_2^{1/2}\left( c_{4}+K\right) \mu ^{\beta \tau }}{T^{\tau }}\Vert Es_{k}^{*}\Vert _{\varPhi _{k}}+ \frac{\kappa \mu ^{\beta \tau }}{T^{\tau }}\Vert Es_{k}^{*}\Vert _{\varPhi _{k}}^2+ 2K {\varrho }h_k^{r-\alpha }\right) . \end{aligned}$$

For the second integral $I_2$, we observe that $\delta _{k+1}\Vert \omega \Vert _2\ge 1$ implies,

$$\begin{aligned} \left( 1+\eta _{k+1}^2\Vert \omega \Vert _2^2\right) ^{\tau } \le 2^{\tau }\eta _{k+1}^{2\tau }\Vert \omega \Vert _2^{2\tau }\le 2^{\tau }\eta _{k+1}^{2\tau }\left( 1+\Vert \omega \Vert _2^2\right) ^{\tau }. \end{aligned}$$

Thus, we have,

$$\begin{aligned} \sqrt{I_2}&\le 2^{\tau /2}\eta _{k+1}^{\tau }\Vert Es_{k+1}^*\Vert _{H^{\tau }(\mathbb {R}^d)} \le 2^{\tau /2}\eta _{k+1}^{\tau }C_{\tau }\Vert s_{k+1}^*\Vert _{H^{\tau }(\varOmega )}\\&\le 2^{\tau /2+1}c_2^{1/2}C_{\tau } \left( \frac{\eta _{k+1}}{\eta _k}\right) ^{\tau }\Vert Es_{k}^{*}\Vert _{\varPhi _k}\\&\le 2^{\tau /2+1}c_2^{1/2}C_{\tau }\mu ^{\beta \tau }\Vert Es_{k}^{*}\Vert _{\varPhi _k}. \end{aligned}$$

Combining the estimates of $I_1$ and $I_2$ yields the estimate (18).

Noticing that the inequality (14) and the additional assumption on $\Vert f^*\Vert _{H^{\tau }(\varOmega )}$ imply

$$\begin{aligned} \Vert Es_1^*\Vert _{\varPhi _1} \le c_1^{-1/2}\Vert Es_1^*\Vert _{H^{\tau }(\mathbb {R}^d)} = c_1^{-1/2} \Vert Ef^*\Vert _{H^{\tau }(\mathbb {R}^d)} \le c_1^{-1/2} C_{\tau } \Vert f^*\Vert _{H^{\tau }(\varOmega )} \le 1. \end{aligned}$$

By induction, we can derive a more precise estimate (19) and the lemma is proven. $\square$

Proof of Lemma 7

Referring to the basic estimate (16) in Lemma 4, the minimality of the functional $\mathcal {J}_k^{\delta }(s_k^{\epsilon ,\gamma ,\delta })$ yields

$$\begin{aligned} \Vert s_k^{\epsilon ,\gamma ,\delta }\Vert _{\varPhi _k}^2 \le \frac{N_k}{\gamma _k}(\delta _k+K{\varrho }h_k^r -\epsilon _k)_+ + \Vert Es_k^{\delta ,*}\Vert _{\varPhi _k}^2. \end{aligned}$$

The first estimate follows directly after the choice of cut-off parameter $\epsilon _k \ge \delta _k+K{\varrho }h_k^r$.

The second estimate is obtained by the fact that

$$\begin{aligned} \Vert s_{k+1}^{\delta ,*}\Vert _{H^{\tau }(\varOmega )}&= \Vert f^* - f_{k-1}^{\epsilon ,\gamma ,\delta } - s_k^{\epsilon ,\gamma ,\delta }\Vert _{H^{\tau }(\varOmega )}\le \Vert s_k^{\delta ,*}\Vert _{H^{\tau }(\varOmega )}+\Vert s_k^{\epsilon ,\gamma ,\delta }\Vert _{H^{\tau }(\varOmega )}\\&\le 2c_2^{1/2}\eta _k^{-\tau }\Vert Es_k^{\delta ,*}\Vert _{\varPhi _k}. \end{aligned}$$

$\square$

Proof of Lemma 8

Referring to the definition of $s_{k+1}^{\delta ,*}$, utilizing the second estimate in Lemma 7, it follows that,

$$\begin{aligned} \Vert As_{k+1}^{\delta ,*}\Vert _{l^{\infty }(X_k)}&= \Vert g-Af_k^{dis}\Vert _{l^{\infty }(X_k)} \\&\le \delta _k + \Vert g^{\delta } - Af_k^{dis}\Vert _{l^{\infty }(X_k)}\\&\le \delta _k + \Vert g^{\delta } - A^{(k)}f_k^{dis}\Vert _{l^{\infty }(X_k)}+ \Vert ( A^{(k)}-A)f_k^{dis}\Vert _{l^{\infty }(X_k)}\\&\le (C_{dis}+1)\delta _k + Kh_k^r\left( {\varrho }+2c_2^{1/2}\eta _k^{-\tau }\Vert Es_k^{\delta ,*}\Vert _{\varPhi _k}\right) . \end{aligned}$$

$\square$

Proof of Lemma 9

The proof follows the similar arguments in Lemma 6. For sufficiently small $h_1$, we have

$$\begin{aligned} \Vert Es_{k+1}^{\delta ,*}\Vert _{\varPhi _{k+1}}^2&\le \frac{1}{c_1} \int _{\mathbb {R}^d} |\widehat{Es_{k+1}^{\delta ,*}}(\omega )|^2(1+\eta _{k+1}^2 \Vert \omega \Vert _2^2)^{\tau } d\omega := \frac{1}{c_1}\left( I_1+I_2\right) \end{aligned}$$

with

$$\begin{aligned} I_1&= \int _{\Vert \omega \Vert _2 \le \frac{1}{\eta _{k+1}}} |\widehat{Es_{k+1}^{\delta ,*}}(\omega )|^2(1+\eta _{k+1}^2 \Vert \omega \Vert _2^2)^{\tau } d\omega ,\\ I_2&= \int _{\Vert \omega \Vert _2 \ge \frac{1}{\eta _{k+1}}} |\widehat{Es_{k+1}^{\delta ,*}}(\omega )|^2(1+\eta _{k+1}^2 \Vert \omega \Vert _2^2)^{\tau } d\omega . \end{aligned}$$

Skipping the detailed calculation, the terms $\sqrt{I_1}$ and $\sqrt{I}_2$ can be estimated as

$$\begin{aligned} \sqrt{I_1}&\le \frac{2^{\tau /2}C_{0}C_d}{c_{3}}\left( \frac{2c_2^{1/2}\left( c_{4} +K\right) \mu ^{\beta \tau }}{T^{\tau }}\Vert Es_{k}^{\delta ,*}\Vert _{\varPhi _{k}}+ K {\varrho }h_k^{r-\alpha } + (C_{dis}+1)h_k^{-\alpha }\delta _k\right) \, \end{aligned}$$

and

$$\begin{aligned} \sqrt{I_2}&\le 2^{\tau /2+1}c_2^{1/2}C_{\tau }\mu ^{\beta \tau }\Vert Es_{k}^{\delta ,*}\Vert _{\varPhi _k}. \end{aligned}$$

The combination of both estimates yields the result. It is worth to note that, the norm constraint on the exact solution $\Vert f^{*}\Vert _{H^{\tau }(\varOmega )}$ is not required. $\square$

Proof of Lemma 10

Similarly to the proof of Lemma 5, it follows that,

$$\begin{aligned} \Vert As_{k+1}^{\delta ,*}\Vert _{l^{\infty }(X_k)}&= \Vert g-Af_k^{\epsilon ,\gamma ,\delta }\Vert _{l^{\infty }(X_k)} \le \delta _k + \Vert g^{\delta }-Af_k^{\epsilon ,\gamma ,\delta }\Vert _{l^{\infty }(X_k)}\\&\le \delta _k + \Vert g^{\delta }-A^{(k)}f_k^{\epsilon ,\gamma ,\delta }\Vert _{l^{\infty }(X_k)} + \Vert (A-A^{(k)})f_k^{\epsilon ,\gamma ,\delta }\Vert _{l^{\infty }(X_k)}\\&\le \delta _k + \max _{j=1,\ldots ,N_k}\left| g^{\delta }_j-A^{(k)}f_k^{\epsilon ,\gamma ,\delta }(x_{k,j}) \right| _{\epsilon _k}+\epsilon _k + Kh_k^r\Vert f_k^{\epsilon ,\gamma ,\delta }\Vert _{H^{\tau }(\varOmega )}\\&\le \delta _k + \epsilon _k + \gamma _k\Vert Es_k^{\delta ,*}\Vert _{\varPhi _k}^2 + Kh_k^r\left( {\varrho }+ 2c_2^{1/2}\eta _k^{-\tau }\Vert Es_k^{\delta ,*}\Vert _{\varPhi _k}\right) . \end{aligned}$$

The result follows directly after the choice of the cut-off parameter. $\square$

Proof of Lemma 11

Again, for sufficiently small $h_1$, there exists

$$\begin{aligned} \Vert Es_{k+1}^{\delta ,*}\Vert _{\varPhi _{k+1}}^2&\le \frac{1}{c_1} \int _{\mathbb {R}^d} |\widehat{Es_{k+1}^{\delta ,*}}(\omega )|^2\left( 1+\eta _{k+1}^2 \Vert \omega \Vert _2^2\right) ^{\tau } d\omega := \frac{1}{c_1}\left( I_1+I_2\right) \end{aligned}$$

with

$$\begin{aligned} I_1&= \int _{\Vert \omega \Vert _2 \le \frac{1}{\eta _{k+1}}} |\widehat{Es_{k+1}^{\delta ,*}}(\omega )|^2\left( 1+\eta _{k+1}^2 \Vert \omega \Vert _2^2\right) ^{\tau } d\omega ,\\ I_2&= \int _{\Vert \omega \Vert _2 \ge \frac{1}{\eta _{k+1}}} |\widehat{Es_{k+1}^{\delta ,*}}(\omega )|^2\left( 1+\eta _{k+1}^2 \Vert \omega \Vert _2^2\right) ^{\tau } d\omega . \end{aligned}$$

Similarly we obtain, for $\sqrt{I_1}$ and $\sqrt{I_2}$, respectively

$$\begin{aligned} \sqrt{I_1}&\le 2^{\tau /2}\Vert Es_{k+1}^{\delta ,*}\Vert _{L^2(\mathbb {R}^d)} \le 2^{\tau /2}C_{0}\Vert s_{k+1}^{\delta ,*}\Vert _{L^2(\varOmega )}\le \frac{2^{\tau /2}C_{0}}{c_{3}} \Vert As_{k+1}^{\delta ,*}\Vert _{H^{\alpha }(\varOmega )}\\&\le \frac{2^{\tau /2}C_{0}C_d}{c_{3}}\left( \frac{2c_2^{1/2}\left( c_{4} +K\right) \mu ^{\beta \tau }}{T^{\tau }}\Vert Es_k^{\delta ,*}\Vert _{\varPhi _k}+ \frac{\kappa \mu ^{\beta \tau }}{T^{\tau }}\Vert Es_k^{\delta ,*}\Vert _{\varPhi _k}^2\right. \nonumber \\&\qquad \left. + 2K {\varrho }h_k^{r-\alpha } + 2h_k^{-\alpha }\delta _k\right) , \end{aligned}$$

and

$$\begin{aligned} \sqrt{I_2}&\le 2^{\tau /2}\eta _{k+1}^{\tau }\Vert Es_{k+1}^{\delta ,*}\Vert _{H^{\tau }(\mathbb {R}^d)} \le 2^{\tau /2}\eta _{k+1}^{\tau }C_{\tau }\Vert s_{k+1}^{\delta ,*}\Vert _{H^{\tau }(\varOmega )}\\&\le 2^{\tau /2+1}c_2^{1/2}C_{\tau }\mu ^{\beta \tau }\Vert Es_{k}^{\delta ,*}\Vert _{\varPhi _k}. \end{aligned}$$

The combination of both integrals yields the estimate (25). Moreover, since we have assumed that $\Vert f^*\Vert _{H^{\tau }(\varOmega )} \le \frac{c_1^{1/2}}{C_{\tau }}$, it follows that,

$$\begin{aligned} \Vert Es_{1}^{\delta ,*}\Vert _{\varPhi _1} \le c_1^{-1/2}C_{\tau }\Vert f^*\Vert _{H^{\tau }(\varOmega )}\le 1. \end{aligned}$$

Then, the estimate (26) is consequently obtained by induction.$\square$