Adaptive three-term family of conjugate residual methods for system of monotone nonlinear equations

Abstract

Two new conjugate residual algorithms are presented and analyzed in this article. Specifically, the main functions in the system considered are continuous and monotone. The methods are adaptations of the scheme presented by Narushima et al. (SIAM J Optim 21: 212–230, 2011). By employing the famous conjugacy condition of Dai and Liao (Appl Math Optim 43(1): 87–101, 2001), two different search directions are obtained and combined with the projection technique. Apart from being suitable for solving smooth monotone nonlinear problems, the schemes are also ideal for non-smooth nonlinear problems. By employing basic conditions, global convergence of the schemes is established. Report of numerical experiments indicates that the methods are promising.

Appendices

Appendix 1

By applying the same approach as for Algorithm 3.1, we obtain that the sequence $\{x_k\}$ generated by Algorithm 3.2 is bounded.

Lemma 6.1

Given that Assumption 3.3 holds and $\{x_{k}\}$ is the sequence generated by Algorithm 3.2, then a constant ${\hat{M}}>0$ exists such that

$$\begin{aligned} \Vert d_k^{(2)}\Vert \le {\hat{M}},\quad \forall k. \end{aligned}$$

(6.1)

Proof

From (3.33) we analyze two cases: Case(1):   $t\frac{\left\langle F_k,s_{k-1}\right\rangle \left\langle d_{k-1},{\bar{y}}_{k-1}\right\rangle }{\Vert {\bar{y}}_{k-1}\Vert ^2\left\langle F_k,d_{k-1}\right\rangle }>\xi -\psi \frac{\Vert {\bar{y}}_{k-1}\Vert ^2\Vert s_{k-1}\Vert ^2}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2}$. Then

$$\begin{aligned} \theta _k^{*}= t\frac{\left\langle F_k,s_{k-1}\right\rangle \left\langle d_{k-1},{\bar{y}}_{k-1}\right\rangle }{\Vert {\bar{y}}_{k-1}\Vert ^2\left\langle F_k,d_{k-1}\right\rangle }, \end{aligned}$$

(6.2)

which from (3.11), (3.34), (4.15) and the Cauchy Schwartz inequality, we have

$$\begin{aligned} \begin{aligned} \Vert \beta _k^{(2)}s_{k-1}\Vert&=\left\| \frac{\left\langle F_k,{\bar{y}}_{k-1}\right\rangle }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }s_{k-1}+t\frac{\left\langle F_k,s_{k-1}\right\rangle \left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }{\Vert {\bar{y}}_{k-1}\Vert ^2\left\langle F_k,d_{k-1}\right\rangle }\frac{\Vert {\bar{y}}_{k-1}\Vert ^2\left\langle F_k,d_{k-1}\right\rangle }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2}s_{k-1}\right\| \\&\le \frac{\Vert F_k\Vert \Vert {\bar{y}}_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\Vert s_{k-1}\Vert +t\frac{\Vert F_k\Vert \Vert s_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\Vert s_{k-1}\Vert \\&\le \frac{\kappa (L+m)\Vert s_{k-1}\Vert }{m\Vert s_{k-1}\Vert ^2}\Vert s_{k-1}\Vert +t\frac{\kappa \Vert s_{k-1}\Vert }{m\Vert s_{k-1}\Vert ^2}\Vert s_{k-1}\Vert \\&=\frac{\kappa (L+m)}{m}+t\frac{\kappa }{m}=\left( (L+m)+t \right) \frac{\kappa }{m}. \end{aligned} \end{aligned}$$

(6.3)

Similarly, applying (3.11), (3.34), (4.15) and the Cauchy inequality, we can write

$$\begin{aligned} \begin{aligned} \left\| \beta _k^{(2)}\frac{\left\langle F_k,s_{k-1}\right\rangle }{\Vert F_k\Vert ^2}F_k\right\|&\le \frac{\Vert F_k\Vert \Vert {\bar{y}}_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\frac{\Vert F_k\Vert \Vert s_{k-1}\Vert }{\Vert F_k\Vert ^2}\Vert F_k\Vert +t\frac{\Vert F_k\Vert \Vert s_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\frac{\Vert F_k\Vert \Vert s_{k-1}\Vert }{\Vert F_k\Vert ^2}\Vert F_k\Vert \\&=\frac{\Vert F_k\Vert \Vert {\bar{y}}_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\Vert s_{k-1}\Vert +t\frac{\Vert F_k\Vert \Vert s_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\Vert s_{k-1}\Vert \\&\le \frac{\kappa (L+m)\Vert s_{k-1}\Vert ^2}{m\Vert s_{k-1}\Vert ^2}+t\frac{\kappa \Vert s_{k-1}\Vert ^2}{m\Vert s_{k-1}\Vert ^2}\\&=\frac{\kappa (L+m)}{m}+t\frac{\kappa }{m}=\left( (L+m)+t \right) \frac{\kappa }{m}. \end{aligned} \end{aligned}$$

(6.4)

So, using (3.35), (6.3), (6.4) and the Cauchy Schwartz inequality, we obtain

$$\begin{aligned} \begin{aligned} \Vert d_k^{(2)}\Vert&\le \Vert F(x_k)\Vert +\Vert \beta _k^{(2)}s_{k-1}\Vert +\left\| \beta _k^{(2)}\frac{\left\langle F_k,s_{k-1}\right\rangle }{\Vert F_k\Vert ^2}F_k\right\| \\&\le \kappa +2\left( (L+m)+t \right) \frac{\kappa }{m}\\&=\left( 1+2\left( \frac{(L+m)}{m}+\frac{t}{m}\right) \right) \kappa . \end{aligned} \end{aligned}$$

(6.5)

Setting $\left( 1+2\left( \frac{(L+m)}{m}+\frac{t}{m}\right) \right) \kappa =M_1$, we obtain

$$\begin{aligned} \Vert d_k^{(2)}\Vert \le M_1. \end{aligned}$$

(6.6)

Case(2):   $t\frac{\left\langle F_k,s_{k-1}\right\rangle \left\langle d_{k-1},{\bar{y}}_{k-1}\right\rangle }{\Vert {\bar{y}}_{k-1}\Vert ^2\left\langle F_k,d_{k-1}\right\rangle }<\xi -\psi \frac{\Vert {\bar{y}}_{k-1}\Vert ^2\Vert s_{k-1}\Vert ^2}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2}$. Then

$$\begin{aligned} \theta _k^{*}= \xi -\psi \frac{\Vert {\bar{y}}_{k-1}\Vert ^2\Vert s_{k-1}\Vert ^2}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2}. \end{aligned}$$

(6.7)

So, using (3.11), (3.34), (4.15) and the Cauchy Schwartz inequality, we have

$$\begin{aligned} \begin{aligned} \Vert \beta _k^{(2)}s_{k-1}\Vert&\le \frac{|\left\langle F_k,{\bar{y}}_{k-1}\right\rangle |}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\Vert s_{k-1}\Vert +\left( \xi -\psi \frac{\Vert {\bar{y}}_{k-1}\Vert ^2\Vert s_{k-1}\Vert ^2}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2}\right) \frac{\Vert {\bar{y}}_{k-1}\Vert ^2|\left\langle F_k,s_{k-1}\right\rangle |}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2}\\&\Vert s_{k-1}\Vert \\&\le \frac{\Vert F_k\Vert \Vert {\bar{y}}_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\Vert s_{k-1}\Vert +\xi \frac{\Vert {\bar{y}}_{k-1}\Vert ^2\Vert F_{k}\Vert \Vert s_{k-1}\Vert ^2}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2}+\psi \frac{\Vert {\bar{y}}_{k-1}\Vert ^4\Vert F_{k}\Vert \Vert s_{k-1}\Vert ^4}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^4}\\&\le \frac{\kappa (L+m)\Vert s_{k-1}\Vert ^2}{m\Vert s_{k-1}\Vert ^2}+\xi \frac{\kappa (L+m)^2\Vert s_{k-1}\Vert ^4}{m^2\Vert s_{k-1}\Vert ^4}+\psi \frac{\kappa (L+m)^4\Vert s_{k-1}\Vert ^8}{m^4\Vert s_{k-1}\Vert ^8}\\&\le \frac{\kappa (L+m)}{m}+\xi \frac{\kappa (L+m)^2}{m^2}+\psi \frac{\kappa (L+m)^4}{m^4}\\&=\left( 1+\xi \frac{(L+m)}{m}+\psi \frac{(L+m)^3}{m^3} \right) \frac{\kappa (L+m)}{m}. \end{aligned} \end{aligned}$$

(6.8)

Similarly, applying (3.11), (3.34), (4.15) and the Cauchy Schwartz inequality, we can write

$$\begin{aligned} \begin{aligned} \left\| \beta _k^{(2)}\frac{\left\langle F_k,s_{k-1}\right\rangle }{\Vert F_k\Vert ^2}F_k\right\|&\le \frac{\Vert F_k\Vert \Vert {\bar{y}}_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\frac{\Vert F_k\Vert ^2\Vert s_{k-1}\Vert }{\Vert F_k\Vert ^2}+\xi \frac{\Vert {\bar{y}}_{k-1}\Vert ^2\Vert F_k\Vert ^3\Vert s_{k-1}\Vert ^2}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2\Vert F_k\Vert ^2}\\&+|\psi |\frac{\Vert {\bar{y}}_{k-1}\Vert ^4\Vert F_k\Vert ^3\Vert s_{k-1}\Vert ^4}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^4\Vert F_k\Vert ^2}\\&=\frac{\Vert F_k\Vert \Vert {\bar{y}}_{k-1}\Vert }{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle }\Vert s_{k-1}\Vert +\xi \frac{\Vert {\bar{y}}_{k-1}\Vert ^2\Vert F_{k}\Vert \Vert s_{k-1}\Vert ^2}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^2}\\&+\psi \frac{\Vert {\bar{y}}_{k-1}\Vert ^4\Vert F_{k}\Vert \Vert s_{k-1}\Vert ^4}{\left\langle s_{k-1},{\bar{y}}_{k-1}\right\rangle ^4}\\&\le \frac{\kappa (L+m)\Vert s_{k-1}\Vert ^2}{m\Vert s_{k-1}\Vert ^2}+\xi \frac{\kappa (L+m)^2\Vert s_{k-1}\Vert ^4}{m^2\Vert s_{k-1}\Vert ^4}+\psi \frac{\kappa (L+m)^4\Vert s_{k-1}\Vert ^8}{m^4\Vert s_{k-1}\Vert ^8}\\&\le \frac{\kappa (L+m)}{m}+\xi \frac{\kappa (L+m)^2}{m^2}+\psi \frac{\kappa (L+m)^4}{m^4}\\&=\left( (L+m)+\xi \frac{(L+m)}{m}+\psi \frac{(L+m)^3}{m^3} \right) \frac{\kappa (L+m)}{m}\\&=\left( 1+\frac{\xi (L+m)}{m}+\psi \frac{(L+m)^3}{m^3}\right) \frac{\kappa (L+m)}{m}. \end{aligned} \end{aligned}$$

(6.9)

So, using (3.35), (6.8), (6.9) and the Cauchy Schwartz inequality, we obtain

$$\begin{aligned} \begin{aligned} \Vert d_k^{(2)}\Vert&\le \Vert F(x_k)\Vert +\Vert \beta _k^{(2)}s_{k-1}\Vert +\left\| \beta _k^{(2)}\frac{\left\langle F_k,s_{k-1}\right\rangle }{\Vert F_k\Vert ^2}F_k\right\| \\&\le \kappa +2\left( (L+m)+\xi \frac{(L+m)}{m}+\psi \frac{(L+m)^3}{m^3} \right) \frac{\kappa (L+m)}{m}\\&=\left( 1+2\left( \frac{(L+m)^2}{m}+\xi \frac{(L+m)^2}{m^2}+\psi \frac{(L+m)^4}{m^4}\right) \right) \kappa . \end{aligned} \end{aligned}$$

(6.10)

Again setting $\left( 1+2\left( \frac{(L+m)^2}{m}+\xi \frac{(L+m)^2}{m^2}+\psi \frac{(L+m)^4}{m^4}\right) \right) \kappa =M_2$, we obtain

$$\begin{aligned} \Vert d_k^{(2)}\Vert \le M_2. \end{aligned}$$

(6.11)

Therefore, from (6.6) and (6.11), we have

$$\begin{aligned} \Vert d_k^{(2)}\Vert \le \max \{M_1,M_2 \}. \end{aligned}$$

(6.12)

So, setting ${\hat{M}}=\max \{M_1,M_2 \}$, we obtain the result. $\square$

Theorem 6.2

Given Assumptions 3.1–3.3 hold and the sequences $\{x_k\}$ and $\{{z}_k\}$ are generated by Algorithm 3.2. Then

$$\begin{aligned} \liminf _ {k \, \rightarrow \, \infty } \Vert F_k\Vert = 0. \end{aligned}$$

(6.13)

The proof is established in a similar manner as Algorithm 3.1.

Appendix 2

See Tables 1, 2, 3, 4.

Table 1 Results of numerical experiment for problem 5.1–5.3

Table 2 Results of numerical experiment for problem 5.4–5.6

Table 3 Results of Numerical experiment for Problem 5.7–5.9

Table 4 Results of Numerical experiment for Problem 5.10–5.12

Appendix 3

See Table 5.

Table 5 Summary of report from Tables 1, 2, 3, 4 showing the number of problems/percentage solved with least iteration numbers, function evaluations and processor time by each of the four methods

The twelve test problems used for the experiments reported in Tables 1, 2, 3, 4.

Problem 5.1

Strictly Convex Function [83]. $F_i(x)=e^{x_i}-1, \quad i=1,2,\ldots ,n$.

Problem 5.2

[85].

• $F_1(x)=2x_1+\sin {x_1}-1$,

• $F_i(x)=2x_i-x_{i-1}+\sin {x_i}-1$,

• $F_n(x)=2x_n+\sin {x_n}-1,\quad i=2,3,\ldots ,n-1$.

Problem 5.3

[86].

• $F_1(x)=2.5x_{1}+x_{2}-1$,

• $F_i(x)=x_{i-1}+2.5x_{i}+x_{i+1}-1, \quad i=2,\ldots ,n-1$,

• $F_n(x)=2x_{n-1}+2.5x_{n}-1$.

Problem 5.4

Exponential Function [87].

• $F_1(x)=e^{x_1}-1$,

• $F_i(x)=e^{x_i}+x_i-1$, $\quad i=2,\ldots ,n$.

Problem 5.5

Non-smooth Function [42]. $F_i(x)=2x_i-\sin {|x_i|}$, $\quad i=1,2, \ldots ,n$.

Problem 5.6

Decretized Chandrasekhar Equation [88].

• $F_i(x)=x_i-\left( 1-\frac{c}{2n} \displaystyle \sum _{j=1}^{n} \frac{\mu _ix_j}{\mu _i+\mu _j}\right) ^{-1},\quad i=1,2,\ldots ,n$,

• with $c\in [0,1)$ and $\mu =\frac{ i-0.5}{n},$ for $1\le i \le n.$ (c is taken as 0.9 in the experiment).

Problem 5.7

[89] The Function F(x) is given by: $F(x) = A_1x + b_2$,

where $b_2 = (\sin x_1-1, \ldots ,\sin x_n-1)^T$, and

$A_1= \left( \begin{matrix} 2&-1&&&\\ 0&2&-1&&\\ &\ddots & \ddots & \ddots &\\ & &\ddots & \ddots & -1\\ &&&0&2 \end{matrix} \right)$.

Problem 5.8

[85] Nonsmooth Function. $F_i(x)=x_i-\sin |x_i-1|$, $\quad i=1, \ldots ,n$.

Problem 5.9

Nonsmooth Function [85]. $F_i(x)=x_i-2\sin |x_i-1|$, $\quad i=1, \ldots ,n$.

Problem 5.10

Tridiagonal Exponential Function [90].

• $F_1(x)= x_{1}-e^{\left( cos{\frac{x_{1}+x_{2}}{n+1}}\right) }$,

• $F_i(x)=x_{i}-e^{\left( cos{\frac{x_{i-1}+x_{i}+x_{i+1}}{n+1}}\right) }, \quad i=2,3,\ldots ,n-1$,

• $F_n(x)=x_{n}-e^{\left( cos{\frac{x_{n-1}+x_{n}}{n+1}}\right) }$.

Problem 5.11

The Logarithmic Function obtained from [83]. $F_i(x)=\ln {(x_i+1)}-\frac{x_i}{n}$, $\quad i=2, \dots n$.

Problem 5.12

[91]. $F_i(x)=x_{i}-\frac{1}{n}x_{i}^2+\frac{1}{n}\displaystyle \sum _{j=1}^{n}x_{i}+i$, $\quad i=1,2,\ldots ,n.$

Remark 6.3

From the selected 12 problems, it can be observed that some have a diagonal Jacobian. Such a separability of the variables turns the outcomes quite independent of the dimensions, as can be observed in the results of Tables 1, 2, 3, 4. Second, although Problem 5.9 is not globally monotone, it seems that local monotonicity is enough to the good behavior of the proposed algorithms, at least with the selected initial points.