An Adaptive Orthogonal Basis Method for Computing Multiple Solutions of Differential Equations with Polynomial Nonlinearities

Abstract

This paper presents an innovative approach, the Adaptive Orthogonal Basis Method, tailored for computing multiple solutions to differential equations characterized by polynomial nonlinearities. Departing from conventional practices of predefining candidate basis pools, our novel method adaptively computes bases, considering the equation’s nature and structural characteristics of7 the solution. It further leverages companion matrix techniques to generate initial guesses for subsequent computations. Thus this approach not only yields numerous initial guesses for solving such equations but also adapts orthogonal basis functions to effectively address discretized nonlinear systems. Through a series of numerical experiments, this paper demonstrates the method’s effectiveness and robustness. By reducing computational costs in various applications, this novel approach opens new avenues for uncovering multiple solutions to differential equations with polynomial nonlinearities.

Appendix

We present the detailed process of the trust region method to solve (2.6). For this purpose, we introduce a region around the current best solution, and approximate the objective function by a quadratic form which boils down to solving a sequence of trust-region subproblems:

$$\begin{aligned} \begin{aligned}&\min _{\varvec{s}\in {\mathbb B}_{h_k}} q^{(k)}({\varvec{s}}) := Q({\varvec{{a}}}_{k}) + {\varvec{g}}({\varvec{{a}}}_{k})^{\top }{\varvec{s}} + \frac{1}{2}{\varvec{s}}^{\top }{\varvec{G}}({\varvec{{a}}}_{k}){\varvec{s}}, \quad \quad k \ge 0, \end{aligned} \end{aligned}$$

(5.1)

where the trust region ${\mathbb B}_{h_k}:=\{\varvec{s}\in {\mathbb R}^n\,:\,\Vert \varvec{s}\Vert \le h_k\}$. When $h_{k}$ is given and ${\varvec{s}}_{k}$ is the minimizer of $q^{(k)}({\varvec{s}})$ in (5.1), we can update ${\varvec{{a}}}_{k+1} = {\varvec{{a}}}_{k} + {\varvec{s}}_{k}$. Obviously, it is one of the most critical steps to choose a proper $h_k$ at each iteration. Based on a good agreement between $q^{(k)}(s_{k})$ and the objective function value $Q({\varvec{a}}_{k+1})$, we should choose $h_{k}$ as large as possible. To be specific, we define a ratio

$$\begin{aligned} r_{k} = \frac{Q({\varvec{{a}}}_{k}) - Q({\varvec{{a}}}_{k+1})}{q^{(k)}(\varvec{0}) - q^{(k)}({\varvec{s}}_{k})}. \end{aligned}$$

(5.2)

The ratio $r_{k}$ is an indicator for expanding and contracting the trust region. If $r_{k}$ is negative, the current value of $Q({\varvec{a}}_{k})$ is less than the new objective value $Q({\varvec{a}}_{k} + {\varvec{s}}_{k})$, consequently the step should be rejected. If $r_{k}$ is close to 1, it means there is a good agreement between the model $q^{(k)}$ and the objective function Q over this step, we can expand the trust region for the next iteration. If $r_{k}$ is close to zero, the trust region should be contracted. Otherwise, we do not alter the trust region at the next iteration. Moreover, the process is also summarized in the following algorithm 1.

For simplicity, in general we choose $\epsilon = 10^{-13}, \delta _1 = 0.25, \delta _2 = 0.75, \tau _1 = 0.5,$ and $\tau _2 = 2$ throughout the paper. Moreover, in the Algorithm 1 (see Line 4), the subproblem (5.1) needs to be solved. Here the so-called dogleg method (see [11, 26]) is used to solve it, and the process is as follows: Let $s:= {\varvec{a}}_{k} - d_{k}{\varvec{g}}_{k}$, and substituting it into (5.1) yields

$$\begin{aligned} q^{(k)}({\varvec{a}}_{k} - d_{k} {\varvec{g}}_{k}) = Q({\varvec{a}}_{k}) - d_{k}\Vert {\varvec{g}}_{k}\Vert ^{2}_2 + \frac{1}{2}d^2_{k} {\varvec{g}}^{\top }_{k}{\varvec{G}}_{k}{\varvec{g}}_{k}. \end{aligned}$$

Based on the exact line search, the step size $d_{k}$ becomes

$$\begin{aligned} d_{k} = \frac{\Vert {\varvec{g}}_{k}\Vert ^{2}_2}{{\varvec{g}}^{\top }_{k}{\varvec{G}}_{k}{\varvec{g}}_{k}}. \end{aligned}$$

Consequently the corresponding step along the steepest descent direction is

$$\begin{aligned} {\varvec{s}}^C_{k} = - d_{k}{\varvec{g}}_{k} = -\frac{{\varvec{g}}^{\top }_{k}{\varvec{g}}_{k}}{{\varvec{g}}^{\top }_{k}{\varvec{G}}_{k}{\varvec{g}}_{k}}{\varvec{g}}_{k}. \end{aligned}$$

On the other hand, the Newtonian step is

$$\begin{aligned} {\varvec{s}}^{N}_{k} = -{\varvec{G}}^{-1}_{k}{\varvec{g}}_{k}. \end{aligned}$$

If $\Vert {\varvec{s}}^{C}_{k}\Vert _{2} = \Vert d_{k}{\varvec{g}}_{k}\Vert _{2} \ge h_{k}$, the solution of (5.1) can be obtained, i.e.,

$$\begin{aligned} {\varvec{s}}_{k} = -\frac{{\varvec{g}}_{k}}{\Vert {\varvec{g}}_{k}\Vert _2}h_{k}, \end{aligned}$$

(5.3)

which leads to ${\varvec{a}}_{k+1} = {\varvec{a}}_{k} + {\varvec{s}}_{k}$. If $\Vert {\varvec{s}}^{C}_{k}\Vert _2 < h_{k}$ and $\Vert {\varvec{s}}^{N}_{k}\Vert _2 > h_{k}$, a dogleg path consisting of two line segments is used to approximate ${\varvec{s}}$ in (5.1), i.e.,

$$\begin{aligned} {\varvec{s}}_{k}(\lambda ) = {\varvec{s}}^{C}_{k} + \lambda ({\varvec{s}}^{N}_{k} - {\varvec{s}}^{C}_{k}),\quad 0 \le \lambda \le 1. \end{aligned}$$

(5.4)

Obviously, when $\lambda = 0$, ${\varvec{s}}_{k}(\lambda )$ reduces to the steepest descent direction. While $\lambda = 1$, it becomes the Newtonian direction. To exactly obtain $\lambda$ in (5.4), we will solve the following equation:

$$\begin{aligned} \Vert {\varvec{s}}^{C}_{k} + \lambda ({\varvec{s}}^{N}_{k} - {\varvec{s}}^{C}_{k})\Vert _2 = h_{k}. \end{aligned}$$

As a result, we have

$$\begin{aligned} {\varvec{a}}_{k+1} = {\varvec{a}}_{k} + {\varvec{s}}_{k}(\lambda ) = {\varvec{a}}_{k} + {\varvec{s}}^{C}_{k} + \lambda ({\varvec{s}}^{N}_{k} - {\varvec{s}}^{C}_{k}), \end{aligned}$$

Otherwise, we choose

$$\begin{aligned} {\varvec{s}}_{k} = {\varvec{s}}^{N}_{k} = -{\varvec{G}}^{-1}_{k}{\varvec{g}}_{k}. \end{aligned}$$

(5.5)

In summary, with (5.3), (5.4) and (5.5), the solution ${\varvec{s}}_{k}$ in (5.1) becomes

$$\begin{aligned} {\varvec{s}}_{k}={\left\{ \begin{array}{ll}-\frac{{\varvec{g}}_{k}}{\Vert {\varvec{g}}_{k}\Vert _2} h_{k}, &{} \text {if}\;\; \Vert {\varvec{s}}^{C}_{k}\Vert _{2} \ge h_{k}, \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad (\text {I}) \\ {\varvec{s}}^{C}_{k} + \lambda ({\varvec{s}}^{N}_{k} - {\varvec{s}}^{C}_{k}), &{} \text {if}\;\; \Vert {\varvec{s}}^{C}_{k} \Vert _{2}< h_{k}\;\; \text {and}\;\; \Vert {\varvec{s}}^{N}_{k}\Vert _{2} > h_{k}, \quad \quad \quad (\text {II}) \\ -{\varvec{G}}^{-1}_{k}{\varvec{g}}_{k}, &{} \text {if}\;\; \Vert {\varvec{s}}^{C}_{k}\Vert _{2} < h_{k}\;\; \text {and}\;\; \Vert {\varvec{s}}^{N}_{k}\Vert _{2} \le h_{k}. \quad \quad \quad (\text {III}) \end{array}\right. } \end{aligned}$$

(5.6)

Fig. 18

Exact trajectory and dogleg approximation

Next, we remark the trust region method for solving nonlinear algebraic system (2.5). As mentioned in [26], the trust region enjoys the desirable global convergence with a local superlinear rate of convergence as follows.

Theorem 5.1

Assume that

• (i) the function $Q({\varvec{{a}}})$ is bounded below on the level set

  $$\begin{aligned} H := \{{\varvec{{a}}}\in R^{n}\; :\; Q({\varvec{{a}}}) \le Q({\varvec{{a}}}_{0})\}, \quad \forall \, {\varvec{{a}}}_{0}\in \mathbb R^n, \end{aligned}$$

  (5.7)

  and is Lipschitz continuously differentiable in H; 

• (ii) the Hessian matrixes $G({\varvec{{x}}}^{(k)})$ are uniformly bounded in 2-norm, i.e., $\Vert G({\varvec{{a}}}_{k})\Vert \le \beta$ for any k and some $\beta >0$.

If ${\varvec{g}}({\varvec{{a}}}_{k}) \ne {\varvec{0}}$, then

$$\begin{aligned} \lim _{k \rightarrow \infty }\inf \Vert {\varvec{g}}({\varvec{{a}}}_{k})\Vert = 0. \end{aligned}$$

(5.8)

Moreover, if ${\varvec{g}}({\varvec{{a}}}^{*}) = {\varvec{0}}$, and ${\varvec{G}}({\varvec{{a}}}^{*})$ is positive definite, then the convergence rate of the trust region method is quadratic.

Remark 5.1

When k is large enough, the trust region method becomes the Newtonian iteration. As a result, it has the same convergence rate as the Newtonian method. $\square$

Remark 5.2

In practice, the gradient and Hessian matrices might be appropriately approximated by some numerical means. We refer to Zhang et al. [37] for such derivative-free methods for (2.6) with $\varvec{f}$ being twice continuously differentiable, but none of their first-order or second-order derivatives being explicitly available. $\square$