1 Introduction

In applied sciences, we need to solve a system of equations, and in many cases these systems are nonlinear. We cannot always solve this type of systems in an exact way, due to the complexity of the problem. For this reason, we obtain an approach to the solution instead of solving it exactly. These systems are described as \(F(x)=0\), where \(F: D \subseteq {\mathbb {R}}^m \rightarrow {\mathbb {R}}^r\) is a vectorial function of m variables.

One way to obtain these approximations is by using iterative methods. Iterative methods create a sequence of approximations starting from a seed that, under specific conditions, tends to the solution.

But, what if instead of searching for only one of the solutions, we aim to obtain more? One option would be trying to obtain different solutions by using several different initial approximations. But with this we have a problem, what happens if both estimates converge to the same root? In the following, we are going to show this problem by representing the dynamical planes for Newton’s and Steffensen’s methods to illustrate what we have just mentioned. What we observe in these cases is if the initial points converge or not to the roots of our problems. In this case, both methods are applied to \(p(x) = x^2 - 1\), with roots 1 and \(-1\).

We chose a mesh of \(400 \times 400\) points to create the dynamical planes, and what we do then is apply our methods to each one of these points, taking it as an initial estimation. The X and Y axes correspond to the real and imaginary part of the initial estimate, respectively. In addition, we set the maximum number of iterations to 80, and we consider the convergence of the initial point to a solution if the distance to that root is less than \(10^{-3}\). In orange, we mark the initial points that converge to \(-1\), in green those that converge to 1, and in black those that do not converge to any root.

Fig. 1
figure 1

Newton’s and Steffensen’s dynamical planes for p(x)

Full size image

In this case, if we take in Fig. 1 two different initial approximations and both are in the same basin of attraction, we obtain two sequences converging to the same solution.

Therefore, if we design a method that calculates simultaneously as many sequences of approximations as solutions the problem has, we will avoid this problem. This is due to the process of calculating an iterate takes into account who are the other iterates.

Schemes to find simultaneously roots of scalar equations are usually defined for polynomial functions. They generate, starting with an initial set of estimations, sequences of approximations that simultaneously tend to several roots. See, for instance (Petković et al. 2014; Proinov 2015; Proinov and Vassileva 2016; Petković et al. 1997; Proinov and Ivanov 2019; Proinov and Vassileva 2019) and their references inside.

From Ehrlich’s method (Ehrlich 1967), the authors in Cordero et al. (2022), design an iterative procedure that can be added to any root-finding scheme for solving arbitrary nonlinear functions. In addition, if the original iterative method has order of convergence p, the new scheme will duplicate its order to 2p. In addition, the new procedure can be used for obtaining the solutions of the equation simultaneously.

Next, we build the dynamical planes of the modified Newton’ and Steffensen’s methods, applied to the same equation with the same convergence criterion. In this case, one axis corresponds to one component of the initial estimation \(x_1^{\left( 0\right) }\), being the other axis component \(x_2^{\left( 0\right) }\). We represent the initial point in purple if the component of the point on the \(x_1^{\left( 0\right) }\) converges to the root \(-1\) and the component in the axis \(x_2^{\left( 0\right) }\) converges to the root 1. We represent the initial point in yellow if the part of the point on the \(x_1^{\left( 0\right) }\) converges to the root 1 and the part in the axis \(x_2^{\left( 0\right) }\) converges to the root \(-1\). If there is no convergence, we mark the starting point in blue.

Fig. 2
figure 2

Dynamical plane of Newton and modified Newton schemes

Full size image

In Fig. 2, we show Newton’s and modified Newton’s procedures applied in the polynomial dynamical planes. The basins of attraction cover the entire space in case of Newton’s scheme, and also for its variant for finding roots simultaneously.

Fig. 3
figure 3

Dynamical plane of Steffensen and modified Steffensen

Full size image

In Fig. 3, dynamical planes are shown for Steffensen’ and modified Steffensen’s methods. In this case, as we can see, there is no convergence for Steffensen’s method in some black zones, for example at \(z = -5\), but its variant converges to the roots at any point, except a small zone where \(x^{(0)}_1 = x^{(0)}_2 = 0\).

As we can see in the dynamical planes of the modified Newton’ and Steffensen’s methods, taking suitable initial estimations \(x_1^{\left( 0\right) }=2\) and \(x_2^{\left( 0\right) }=5\) we converge to both roots. This can not be achieved by iterating both non simultaneous methods in parallel, since both were in the basin of the attraction of the root 1.

In this paper, we generalize this idea to nonlinear systems, thus designing an iterative procedure that can be added to any iterative method for solving systems of nonlinear equations. Indeed, the new resultant method duplicates the order of convergence and can get several solutions simultaneously.

In Sect. 2, we design the mentioned step and study the order of convergence of the resulting method. In the third Section, we carry out some numerical experiments with known iterative procedures to which we add the simultaneous step, to see the performance of the new schemes. We finish the work in Sect. 4 with some conclusions, derived from the study.

2 Convergence analysis

Let \(F(x)=0\), \(F:{\mathbb {R}}^m \rightarrow {\mathbb {R}}^r\), be a system of nonlinear equations, where the number of unknowns is m and the number of equations is r. Note that the system is written as a column vector of size \(r\times 1\).

Suppose that this system has n solutions, which are denoted by

$$\begin{aligned} \alpha _i=\left( \alpha _{i_1},\alpha _{i_2},\ldots ,\alpha _{i_m}\right) \text { for }i=1,\ldots ,n. \end{aligned}$$

We design an iterative scheme to obtain all the solutions simultaneously. Therefore, we take a set of n initial approximations which we denote by \(x_i^{\left( 0\right) }=\left( x_{i_1}^{\left( 0\right) },x_{i_2}^{\left( 0\right) },\ldots , x_{i_m}^{\left( 0\right) }\right) \), \(i=1,..,n\). If we define

$$\begin{aligned} \dfrac{1}{x_i^{\left( k\right) }-x_j^{\left( k\right) }}:=\left( \dfrac{1}{x_{i_1}^{\left( k\right) }-x_{j_1}^{\left( k\right) }},\dfrac{1}{x_{i_2}^{\left( k\right) }-x_{j_2}^{\left( k\right) }},\ldots ,\dfrac{1}{x_{i_m}^{\left( k\right) }-x_{j_m}^{\left( k\right) }}\right) , \end{aligned}$$

then we design the iterative step as:

$$\begin{aligned} \left\{ \begin{aligned} x_i^{\left( k+1\right) }=x_i^{\left( k\right) }-\left( F'\left( x_i^{\left( k\right) }\right) -F\left( x_i^{\left( k\right) }\right) \sum _{j\ne i}\dfrac{1}{x_i^{\left( k\right) }-x_j^{\left( k\right) }}\right) ^{-1}F\left( x_i^{\left( k\right) }\right) , \end{aligned}\right. \end{aligned}$$
(1)

for \(i=1,\ldots ,n\) and \(k=0,1,\ldots \)

As we can see, the size of matrix \(F'\left( x_i^{\left( k\right) }\right) \) is \(r\times m\) which matches with the size of the product of \(F\left( x_i^{\left( k\right) }\right) \) and \(\dfrac{1}{x_i^{\left( k\right) }-x_j^{\left( k\right) }}\), \(j\ne i\), as the above vectors are a column vector of size \(r\times 1\) and a row vector of size \(1\times m\), respectively.

We denote this method by PS. Now, we prove that this iterative procedure has quadratic order.

Theorem 1

Let \(F: {\mathbb {R}} ^m \longrightarrow {\mathbb {R}}^r\) be an enough differentiable function in a neighbourhood of \(\alpha _i\), \(D_i \subset {\mathbb {R}}^m\). Let us also assume that \(F(\alpha _i)=0\), for \(i=1,...,n\) and \(F'(\alpha _i)\) is nonsingular for \(i=1,...,n\). Then, by taking an estimation \(x_i^{(0)}\) close enough to \(\alpha _i\), \(i=1,...,n\), sequence \(\{x_i^{(k)}\}\) of approximations generated by the PS scheme converges to \(\alpha _i\) with order 2.

Proof

We denote by \(F=(F_1,F_2,\ldots ,F_r)\), where \(F_p:{\mathbb {R}}^m\rightarrow {\mathbb {R}}\), for \(p=1,2,\ldots , r\). Now, we consider the Taylor expansion of \(F_p\left( x_i^{\left( k\right) }\right) \) around \(\alpha _i\) for \(p=1,2,\ldots , r\),

$$\begin{aligned} F_p\left( x_i^{\left( k\right) }\right) =\sum _{j_1=1}^m\dfrac{\partial F_p\left( \alpha _i\right) }{\partial x_{j_1}}e_{{i,k}_{j_1}}+\sum _{j_1=1}^m\sum _{j_2=1}^m\dfrac{\partial ^2 F_p\left( \alpha _i\right) }{\partial x_{j_1}\partial x_{j_2}}e_{{i,k}_{j_1}}e_{{i,k}_{j_2}}+O_3\left( e_{i,k}\right) , \end{aligned}$$
(2)

where \(e_{{i,k}_{j_1}}=x_{i_{j_1}}^{\left( k\right) }-\alpha _{i_{j_1}}\) for \(j_1\in \{1,2,\ldots ,m\}\) and \(i\in \{1,\ldots ,n\}\), and where \(O_3\left( e_{i,k}\right) \) the elements where the sums of the exponents of \(e_{{i,k}_{j_1}}\) is greater than or equal to 3 with \(j_1\in \{1,2,\ldots ,m\}\).

If we derive this with respect to the variable \(x_{i_q}\), with \(q=1,2,\ldots , m\), we get

$$\begin{aligned} \dfrac{\partial F_p\left( x_i^{\left( k\right) }\right) }{\partial x_{i_q}}=\dfrac{\partial F_p\left( \alpha _i\right) }{\partial x_{i_q}}+\sum _{j_1=1}^m\dfrac{\partial ^2 F_p\left( \alpha _i\right) }{\partial x_{i_q}\partial x_{j_1}}e_{{i,k}_{j_1}}+O_2\left( e_{{i,k}}\right) . \end{aligned}$$
(3)

Then,

$$\begin{aligned}&\dfrac{\partial F_p\left( x_i^{\left( k\right) }\right) }{\partial x_{i_q}}- F_p\left( x_i^{\left( k\right) }\right) \sum _{j\ne i}^m \dfrac{1}{x_{i_q}-x_{j_q}}=\dfrac{\partial F_p\left( \alpha _i\right) }{\partial x_{i_q}}+\sum _{j_1=1}^m\dfrac{\partial ^2 F_p\left( \alpha _i\right) }{\partial x_{i_q}\partial x_{j_1}}e_{{i,k}_{j_1}}\\&\quad -\sum _{j_1=1}^m\dfrac{\partial F_p\left( \alpha _i\right) }{\partial x_{j_1}}e_{{i,k}_{j_1}}\sum _{j\ne i}^m \dfrac{1}{e_{i,k_q}-e_{j,k_q}+\alpha _{i_q}-\alpha _{j_q}}\\&\quad +O_2\left( e_{i,k}\right) . \end{aligned}$$

Then, we can rewrite the last equation as

$$\begin{aligned} \dfrac{\partial F_p\left( x_i^{\left( k\right) }\right) }{\partial x_{i_q}}- F_p\left( x_i^{\left( k\right) }\right) \sum _{j\ne i}^m \dfrac{1}{x_{i_q}-x_{j_q}}=&=\dfrac{\partial F_p\left( \alpha _i\right) }{\partial x_{i_q}}+\sum _{j_1=1}^mA_{i_q,j_1}e_{{i,k}_{j_1}}+O_2\left( e_{i,k}\right) , \end{aligned}$$

being

$$\begin{aligned} A_{i_q,j_1}:=\left( \dfrac{\partial ^2 F_p\left( \alpha _i\right) }{\partial x_{i_q}\partial x_{j_1}}-\dfrac{\partial F_p\left( \alpha _i\right) }{\partial x_{j_1}}\sum _{j\ne i}^m \dfrac{1}{e_{i,k_q}-e_{j,k_q}+\alpha _{i_q}-\alpha _{j_q}}\right) , \end{aligned}$$

for \(j_1\in \{1,2,\ldots ,m\}\).

If we write \(A_{i_q}:=\left( A_{i_q,1},A_{i_q,2},\ldots , A_{i_q,m}\right) \), and then write A which rows are \(A_{i_1}\), \(A_{i_2}\), \(\ldots \), \(A_{i_m}\) we obtain that

$$\begin{aligned} F'\left( x_i^{\left( k\right) }\right) -F\left( x_i^{k}\right) \sum _{j\ne i}\dfrac{1}{x_i^k-x_j^k}&=F'\left( \alpha _i\right) \left( I+F'\left( \alpha _i\right) ^{-1}Ae_{i,k}\right) +O_2\left( e_{i,k}\right) . \end{aligned}$$

From the above relationship, it follows that

$$\begin{aligned} \left( F'\left( x_i^{\left( k\right) }\right) -F\left( x_i^{k}\right) \sum _{j\ne i}\dfrac{1}{x_i^k-x_j^k}\right) ^{-1}&=\left( I-F'\left( \alpha _i\right) ^{-1}Ae_{i,k}\right) F'\left( \alpha _i\right) ^{-1}+O_2\left( e_{i,k}\right) . \end{aligned}$$

If we apply the above relation, then \(e_{i,k+1}\) has the following form

$$\begin{aligned} e_{i,k+1}&=e_{i,k}-\left( I-F'\left( \alpha _i\right) ^{-1}Ae_{i,k}\right) \left( e_{i,k}+C_2e_{i,k}^2\right) +O_3\left( e_{i,k}\right) \\&=\left( F'\left( \alpha _i\right) ^{-1}A-C_2\right) e_{i,k}^2+O_3\left( e_{i,k}\right) . \end{aligned}$$

It is, therefore, proven that PS has convergence order 2. \(\square \)

Let \(\phi (x)\) be the fixed point of an iterative method. We define \(PS_{\phi }\) as follows

$$\begin{aligned} \left\{ \begin{aligned} x_i^{\left( k+1\right) }&=\phi \left( x_i^{\left( k\right) }\right) ,\\ x_i^{\left( k+2\right) }&=PS\left( x_1^{\left( k+1\right) },\ldots ,x_n^{\left( k+1\right) }\right) , \text { for } i=1,\ldots ,n \text { and } k=0,1,\ldots \end{aligned}\right. \end{aligned}$$

that is, an iterative scheme in which we first perform method described by \(\phi \) and then the PS scheme.

Theorem 2

Let \(F: {\mathbb {R}} ^m \longrightarrow {\mathbb {R}}^r\) be a sufficiently differentiable function in a neighbourhood \(D_i \subset {\mathbb {R}}^m\), of \(\alpha _i\). Let us assume that \(F(\alpha _i)=0\), for \(i=1,...,n\) and \(F'(\alpha _i)\) is nonsingular for \(i=1,...,n\). Then, by taking \(x_i^{(0)}\), an seed close enough to \(\alpha _i\), for \(i=1,...,n\), the sequence of iterates \(\{x_i^{(k)}\}\) generated by \(PS_{\phi }\) method converges to \(\alpha _i\) with order 2p, where p is the order of convergence of scheme described by \(\phi \).

Proof

By Theorem 1,

$$\begin{aligned} e_{i,k+2}=\left( F'\left( \alpha _i\right) ^{-1}A-C_2\right) e_{i,k+1}^2+O_3\left( e_{i,k+1}\right) . \end{aligned}$$
(4)

Since we have that \(\phi \) has order p, this means that \(e_{i,k+1}\sim e_{i,k}^p\). Substituting the last relation into the equation (4) we obtain that

$$\begin{aligned} e_{i,k+2}&\sim e_{i,k+1}^2\sim \left( e_{i,k}^p\right) ^2\sim e_{i,k}^{2p}. \end{aligned}$$

Thus, it is proven that procedure \(PS_{\phi }\) has order 2p. \(\square \)

3 Numerical experimentation

We use Matlab R2021b with arithmetic precision of one thousand digits for the computational calculations. As a stopping criterion we compare the mean of the norm of the function evaluated at the last iterations with a tolerance of \(10^{-50}\), that is, if we try to find n solutions,

$$\begin{aligned} \dfrac{1}{n}\sum _{i=1}^{n}\left\| F\left( x_i^{(k+1)}\right) \right\| <10^{-50}. \end{aligned}$$

We denote by

$$\begin{aligned} \left\| F\left( x^{(k+1)}\right) \right\| :=\dfrac{1}{n}\sum _{i=1}^{n}\left\| F\left( x_i^{(k+1)}\right) \right\| \end{aligned}$$

and \(x^{(k+1)}=\left( x_1^{(k+1)},x_2^{(k+1)},\ldots ,x_n^{(k+1)}\right) \).

We also use a maximum of 100 iterations as a stopping criterion.

For the numerical experiments, we use three different algorithms. The first one is the method PS, that we have defined in (1). The others ones are \(PS_{N}\) and \(PS_{N2}\) that are the composition of PS with N and N2, where N denote Newton’s scheme and N2 denote Newton’s method composed with itself, that is

$$\begin{aligned} \left\{ \begin{aligned} x_i^{\left( k+1\right) }&=x_i^{\left( k\right) }-F'\left( x_i^{\left( k\right) }\right) ^{-1}F\left( x_i^{\left( k\right) }\right) ,\\ x_i^{\left( k+2\right) }&=x_i^{\left( k+1\right) }-F'\left( x_i^{\left( k+1\right) }\right) ^{-1}F\left( x_i^{\left( k+1\right) }\right) , \text { for } i=1,\ldots ,n \text { and } k=0,1,\ldots \end{aligned}\right. \end{aligned}$$

The numerical results we are going to compare the methods in the different examples are:

  • the obtained approximation,

  • the mean of the norm of the system evaluated in that set of approximations,

  • the distance between the last two sets of iterates,

  • the number of iterations needed to achieve the tolerance,

  • the computational time and the ACOC (approximate computational convergence order), defined by Cordero and Torregrosa in Cordero and Torregrosa (2007), defined as follows

    $$\begin{aligned} p \approx ACOC =\frac{\ln \left( \Vert x^{(k+1)}-x^{(k)} \Vert /\Vert x^{(k)}-x^{(k-1)}\Vert \right) }{\ln \left( \Vert x^{(k)}-x^{(k-1)} \Vert /\Vert x^{(k-1)}-x^{(k-2)}\Vert \right) }. \end{aligned}$$

We are going to solve the following nonlinear problems.

Example 1

We aim to find the intersection points of the circle centred at \(\left( 0,0\right) \) with radius \(\sqrt{2}\) and the ellipse \(\{ \left( x,y\right) \in {\mathbb {R}}^2 : 3x^2+2xy+3y^2=5\}\), that is, find the points (x, y) that are solutions of the following system denoted by S(x, y):

$$\begin{aligned} x^2+y^2&=2, \\ 3x^2+2xy+3y^2&=5. \end{aligned}$$

The exact solutions of this problem are

$$\begin{aligned} \scriptstyle \left\{ \left( \sqrt{1+\frac{\sqrt{3}}{2}},\frac{-1}{\sqrt{4+2\sqrt{3}}}\right) , \left( -\sqrt{1+\frac{\sqrt{3}}{2}},\frac{1}{\sqrt{4+2\sqrt{3}}}\right) , \left( \sqrt{1-\frac{\sqrt{3}}{2}},\frac{-1}{\sqrt{4-2\sqrt{3}}}\right) , \left( -\sqrt{1-\frac{\sqrt{3}}{2}},\frac{1}{\sqrt{4-2\sqrt{3}}}\right) \right\} . \end{aligned}$$

Then, we take as initial estimations \(x^{\left( 0\right) }_1=\left( 1, -0.5\right) \), \(x^{\left( 0\right) }_2=\left( -1,0.5\right) \), \(x^{\left( 0\right) }_3=\left( 0.5, -1\right) \) and \(x^{\left( 0\right) }_4=\left( -0.5,1\right) \).

Table 1 Output for intersection of ellipse and circle
Full size table

As we can see in Table 1, all the methods obtain good results for the chosen initial points. The approximate computational convergence order coincides with the theoretical one or is greater than that one.

Example 2

The second academical problem to be solved is to obtain all the critical points of function \(g\left( x,y\right) =\frac{1}{3}x^3+y^2+2xy-6x-3y+4\).

To calculate the critical points, we solve \(\nabla g\left( x,y\right) =0\).

As initial approximations we take \(x^{\left( 0\right) }_1=\left( 0, 1\right) \) and \(x^{\left( 0\right) }_2=\left( 2,-1\right) \).

Table 2 Output of the equation \(\nabla g\left( x,y\right) =0\)
Full size table

In this case, for all methods, vectors \(\left( -1,\frac{5}{2}\right) \) and \(\left( 3,-\frac{3}{2}\right) \) are obtained as approximations to the solutions.

As we can see in Table 2, all the methods obtain good results for the chosen initial points. The approximate computational convergence order matches the theoretical one.

Example 3

Now, we solve the following nonlinear system with size \(200 \times 200\)

$$\begin{aligned} F_i(x)&=(x_i^2-1)(x_{i+1}^2-1)=0,\quad i=1,\ldots ,199 \\ F_{200}(x)&=(x_1^2-1)(x_{200}^2-1)=0, \end{aligned}$$

where \(x=\left( x_1,x_2,\ldots , x_{199},x_{200}\right) \in {\mathbb {R}}^{200}\).

This system has infinite solutions, but we aim to find two of them. As initial points we choose \(x_1^{(0)}=0.8(1,1,\ldots ,1)\) and \(x_2^{(0)}=-0.8(1,1,\ldots ,1)\).

In this case, we use Matlab2021b with arithmetic precision of 10 digits and \(10^{-5}\) as tolerance. As we can see in Table 3, the method that does not use a predictor needs 17 iterations to achieve that tolerance.

We conclude that in many cases is very useful to use a predictor scheme, due to the fact that \(PS_{N2}\) obtains the exact solution in only 2 iterations. It is clear that methods with a higher order of convergence converge faster.

Table 3 Output of the system \(F(x)=0\)
Full size table

4 Outcomes and conclusions

In this paper, we define an iterative step that can be added to any iterative scheme for nonlinear systems obtaining a new iterative method that simultaneously find several roots, which has twice the order of the original scheme.

Different known iterative procedures were selected to add this step, and some experiments were carried out numerically to confirm the good properties of the new iterative procedures.