On the order of convergence of Broyden’s method

We present two theoretical results and two surprising conjectures concerning convergence properties of Broyden’s method for smooth nonlinear systems of equations. First, we show that when Broyden’s method is applied to a nonlinear mapping F:Rn→Rn\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F:\mathbb {R}^n\rightarrow \mathbb {R}^n$$\end{document} with n-d\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n-d$$\end{document} affine component functions and the initial matrix B0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B_0$$\end{document} is chosen suitably, then the generated sequence (uk,F(uk),Bk)k≥1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(u^k,F(u^k),B_k)_{k\ge 1}$$\end{document} can be identified with a lower-dimensional sequence that is also generated by Broyden’s method. This property enables us to prove, second, that for such mixed linear–nonlinear systems of equations a proper choice of B0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B_0$$\end{document} ensures 2d-step q-quadratic convergence, which improves upon the previously known 2n steps. Numerical experiments of high precision confirm the faster convergence and show that it is not available if B0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B_0$$\end{document} deviates from the correct choice. In addition, the experiments suggest two surprising possibilities: It seems that Broyden’s method is (2d-1)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(2d-1)$$\end{document}-step q-quadratically convergent for d>1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d>1$$\end{document} and that it admits a q-order of convergence of 21/(2d)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2^{1/(2d)}$$\end{document}. These conjectures are new even for d=n\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d=n$$\end{document}.


Introduction
Given a smooth nonlinear mapping F ∶ ℝ n → ℝ n , Broyden's method aims at finding a root ū of F, It is one of the most widely used quasi-Newton methods for systems of nonlinear equations and converges locally q-superlinearly, as was shown by Broyden, Dennis and Moré in their seminal paper [7]. We state the method as Algorithm BROY.
Before discussing the content of this paper in more detail, let us outline its main contributions: -A well-known result of Gay [11,Theorem 3.1] asserts local 2n-step q-quadratic convergence of Broyden's method under appropriate assumptions. We show under the same assumptions that if n − d of the equations are affine and the corresponding n − d rows of B 0 agree with the corresponding n − d rows of F ′ , then Broyden's method is locally 2d-step q-quadratically convergent. -We provide high-precision numerical experiments that confirm the improved convergence speed and observe that it is lost if the relevant rows of B 0 are perturbed. -The experiments suggest that Broyden's method enjoys a q-order of convergence no smaller than 2 1∕(2d) . This is the first time that the q-order of Broyden's method is studied numerically, and even for d = n the conjecture that a q-order larger than one may exist is novel.
The starting point of this work is the property of Algorithm BROY that if n − d rows of F ′ are constant and the initial guess B 0 matches these rows exactly, then there exist d-dimensional subspaces S ⊂ ℝ n and ℝ n d ⊂ ℝ n such that (s k ) k≥1 ⊂ S and (F(u k )) k≥1 ⊂ ℝ n d , where ℝ n d ∶= {(y 1 , … , y n ) T ∈ ℝ n ∶ y j = 0 ∀j > d} and where we have assumed without loss of generality that the constant rows of F ′ are the last n − d ones. This subspace property is well-known and appears, for instance, in the classical book of Dennis and Schnabel [9]. To the best of the author's knowledge, however, it has not been noted before that in this situation the sequence (u k , F(u k ), B k ) k≥1 can be identified with a sequence (w k , G(w k ), C k ) k≥0 ⊂ ℝ d × ℝ d × ℝ d×d that is generated by applying Broyden's method to a suitable mapping G ∶ ℝ d → ℝ d . We stress that many well-known quasi-Newton methods do not have this property, e.g., the BFGS update, cf. Remark 1 in Sect. 2. We will use it to show that (u k ) k≥1 is 2d-step q-quadratically convergent under the assumptions of Gay's theorem on 2n-step q-quadratic convergence (which coincide with the classical assumptions for local q-superlinear convergence of Broyden's method). As a corollary we obtain that (u k ) exhibits an r-order of convergence [30,Sect. 9.2] no smaller than 2 1 2d . We emphasize that no modification of Algorithm BROY is necessary to enjoy the faster convergence; it is only required to choose B 0 in such a way that it agrees with F ′ on the rows that correspond to affine components of F. On the other hand, it may be numerically advantageous to carry out Algorithm BROY for G instead of F, for instance because smaller linear systems have to be solved; cf. also Remark 2 (2).
It is clear that there is an abundance of practically relevant nonlinear systems of equations that contain some linear equations; these systems are covered by the result on 2d-step q-quadratic convergence. In addition, this result supports two standard suggestions for the choice of B 0 , which are to use either B 0 = F � (u 0 ) or a componentwise finite difference approximation of F � (u 0 ) , cf. for instance [28,comment after Theorem 11.5] and [25,Sect. 10], since both choices imply that B 0 and F ′ agree on rows associated to affine component functions of F.
We confirm the 2d-step convergence in numerical experiments of high precision and observe that it is lost if the rows of B 0 that correspond to affine components of F are perturbed, while perturbations in other rows have no such effect. This shows that choosing B 0 to match the constant rows of F ′ (if possible) will usually decrease both iteration count and runtime.
Even for d = n (i.e., fully nonlinear systems) these conjectures are new and perhaps somewhat surprising. We therefore stress that the conjectured lower bound of 2 1∕(2n) for the q-order of Broyden's method is consistent with further numerical experiments of the author, e.g. in [20][21][22]. Also, we are aware that the 2n-step q-quadratic convergence is derived from the 2n-step convergence of Broyden's method on regular linear systems [11,12,29] and that it is more or less accepted that fewer than 2n iterations rarely suffice in the case of linear systems. On the other hand, since the present work is the first to numerically assess the 2n-step q-quadratic convergence and the q-order of convergence of Broyden's method, no numerical evidence that contradicts the above conjectures is available.
Broyden's method is one of the most prominent quasi-Newton methods for solving nonlinear equations, and there is an ever-growing body of literature available. For the case of smooth systems of equations, the surveys [2,8,16,25] cover many aspects and provide further references, so we restrict ourselves to mentioning [7,15,31] that develop the local convergence theory of Broyden's method. Works that are too recent to be included in the surveys are, for instance, concerned with the extension of Broyden's method to constrained nonlinear systems of equations [24], to set-valued mappings [3], and to single-and set-valued nonsmooth problems in infinite-dimensional spaces [1,23,27]. Broyden's method is also used in implicit deep learning, cf. [5,6], which further underlines that it continues to play a role nowadays. Despite the vast amount of contributions, we are not aware of works that contain the identification of (u k , F(u k ), B k ) k≥1 with (w k , G(w k ), C k ) k≥0 or that exploit the presence of affine equations to prove faster convergence. We acknowledge that the use of exact initialization of constant rows can be regarded as a special case of the least-change update theory [10], of structured Broyden methods [4,18,26,32], and of reduced quasi-Newton methods [13,14], but in these and similar contributions it is not specified how the rate of convergence depends on the dimension of the underlying spaces, so they yield no improvement for the situation at hand.
There is also a considerable amount of numerical studies available for Broyden's method. Since many of the aforementioned works contain numerical experiments and the surveys include references to a number of studies, we mention only [33] and the monograph [19].
This paper is organized as follows. In Sect. 2 we present and prove the relationship between (u k , F(u k ), B k ) k≥1 and its lower-dimensional counterpart (w k , G(w k ), C k ) k≥0 , and in Sect. 3 we use it to establish the result on local 2d-step q-quadratic convergence. Section 4 is devoted to numerical experiments, and Sect. 5 provides a summary.
Notation. We use ℕ = {1, 2, 3, …} and ℕ 0 ∶= ℕ ∪ {0} . By ‖ ⋅ ‖ we indicate the Euclidean norm for vectors, respectively, the spectral norm for matrices. For A ∈ ℝ n×n , A j indicates the jth row of A, regarded as a row vector, while A i,j ∈ ℝ is an entry of A and A k is a member of the matrix sequence (A k ) . The linear hull of C ⊂ ℝ n is denoted by ⟨C⟩ and its orthogonal complement is denoted by C ⟂ .

A subspace property of Broyden's method
The following assumption presents the setting that we are interested in.
where a j ∈ ℝ n and b j ∈ ℝ for all j ∈ J . Let u 0 ∈ ℝ n and B 0 ∈ ℝ n×n invertible with B j 0 = a T j for all j ∈ J.
The first lemma describes the behavior of Algorithm BROY on affine components of F provided that B 0 is initialized with the rows of F ′ for these components. It appears in [9, Sect. 8.5, Exercise 10], albeit without proof. Lemma 1 Let Assumption 1 hold and let (u k ) and (B k ) be generated by Algorithm BROY with initial guess (u 0 , B 0 ) . Then we have for all j ∈ J and all k ≥ 1 the identities B j k = a T j , F j (u k ) = 0 , a T j s k = 0 and B k a j = B 1 a j .

Remark 1
Several other quasi-Newton methods do not have the property described in Lemma 1. Let us consider the BFGS method as an example. We choose It is easy to confirm that u 1 = F(u 1 ) = (0, 1 2 ) T , but To state the main result of this section we introduce the following notation.

Definition 1 Let Assumption 1 hold. For any matrix B ∈ ℝ n×n we set
We now establish the fundamental property of Broyden's method that under Assumption 1 the sequence (u k , F(u k ), B k ) k≥1 can be identified with a sequence (w k , G(w k ), C k ) k≥0 that is generated by applying Broyden's method to a suitable mapping G from ℝ d into ℝ d .
Theorem 1 Let Assumption 1 hold and let (u k ) and (B k ) be generated by Algorithm BROY with initial guess (u 0 , B 0 ) . Suppose that each B k is invertible. Let S ∈ ℝ n×d be a matrix whose columns form an orthonormal basis of ⟨{a j } j∈J ⟩ ⟂ . Define as well as Then C 0 is invertible and the application of Algorithm BROY to G with initial guess (w 0 , C 0 ) generates sequences (w k ) and (C k ) with the following properties: (1) Each C k is invertible and for all k ≥ 1 there hold (2) The iterates (u k ) converge to ū ∈ ℝ n if and only if there is w ∈ ℝ d such that (w k ) converges to w . If (u k ) and (w k ) converge to ū and w , respectively, then we have for all k ≥ 1 Proof Before we prove (1) and (2), let us point out that S is well-defined. This follows since the invertibility of We now prove that Algorithm BROY generates sequences (w k ) and (C k ) , that each C k is invertible, and that the first and last of the three asserted equalities in (1) hold. To proceed by induction, we note that w 0 = 0 , so u 1 + Sw 0 = u 1 holds. Also, C 0 =B 1 S by definition, so C 0 is invertible by the first part of the proof. For the induction step let k ∈ ℕ and assume that w k−1 and C k−1 satisfying u k = u 1 + Sw k−1 and C k−1 =B k S have been generated and that C k−1 is invertible. Since s k ∈ S by Lemma 1 and since the columns of S are a basis of S , there is a vector ∈ ℝ d such that , n} , where we used the induction assumption. By definition, the first d of these equations can be expressed as B k S = −G(w k−1 ) . Since C k−1 =B k S by induction assumption and C k−1 is regular, it follows that the system C k−1 s k−1 w = −G(w k−1 ) in line 3 of Algorithm BROY has the unique solution s k−1 w = , hence w k = w k−1 + s k−1 w exists, and we obtain s k = S = Ss k−1 w = S(w k − w k−1 ) . Adding u k and using the induction assumption u k = u 1 + Sw k−1 this yields u k+1 = u 1 + Sw k , which is the first equality in (1). We observe that where the last equality follows since the columns of S are orthonormal. To conclude the induction it is left to demonstrate the third equality in (1) and the invertibility of where we used the first equality from (1) to rewrite the argument of F as u k+1 , respectively, u k . Since F(u k ) ≠ 0 implies s k ≠ 0 , we conclude from (3) that C k exists and from (4) that it satisfies as desired. By the first part of the proof this representation of C k also implies that C k is invertible, which concludes the induction. The remaining second equality of (1) follows from the first and the definition of G. Proof of (2) All claims follow readily by use of u k = u 1 + Sw k−1 . ◻

Remark 2
(1) To illustrate Theorem 1, let us consider the special case that S is given by the first d columns of the n × n identity matrix. In the notation of the previous proof this corresponds to Broyden update affects only the entries of this submatrix of B k for k ≥ 1 . Similarly, for k ≥ 1 only the first d entries of u k , respectively, of F(u k ) change, and w k−1 , respectively, G(w k−1 ) contain exactly these entries.
(2) Theorem 1 (1) shows that once u 1 is computed, all further iterates (u k , B k , F k ) , k ≥ 2 , can be obtained by an application of Broyden's method to G with initial guess (w 0 , C 0 ) . Using G instead of F may reduce the numerical costs, for instance because the linear systems that have to be solved involve the d × d matrix C k rather than the n × n matrix B k . Moreover, if F is used it is possible that rounding errors destroy the properties (s k ) k≥1 ⊂ S and (F(u k )) k≥1 ⊂ ℝ n d , which cannot happen if G is used instead. The properties (s k ) k≥1 ⊂ S and (F(u k )) k≥1 ⊂ ℝ n d are crucial to obtain the improved convergence result of this work, both in theory and practice; loosing them slows down the convergence.

Improved convergence for mixed linear-nonlinear systems
In this section we show that Gay's result on local 2n-step q-quadratic convergence of Broyden's method can be improved if F has some affine component functions. This requires the notion of multi-step q-quadratic converge.
Remark 3 For m = 1 this is q-quadratic convergence in the usual sense.
Gay's theorem is based on the following assumption.

Remark 4
(1) It is well known that under the assumptions of Theorem 2, (u k ) is q-superlinearly convergent and the sequences (‖B k ‖) and (‖B −1 k ‖) are bounded. (2) While it is not required by Definition 2, we note that the constant C in Theorem 2 is independent of (u 0 , B 0 ).
The following result improves Theorem 2 in the presence of linear equations.
j for all j ∈ J , Algorithm BROY either terminates with output u * =ū or it generates a sequence (u k ) that converges to ū and satisfies Proof If Algorithm BROY terminates with u * =ū , then there is nothing to show, so we can assume that this termination does not occur. Since the assumptions of Theorem 2 are satisfied, it follows that Algorithm BROY successfully generates (u k ) and (B k ) and that (u k ) converges to ū . Since each B k is invertible, we can apply Theorem 1. In particular, this endows us with a matrix S, a mapping G, sequences (w k ) and (C k ) , and a point w , all satisfying the properties stated in that theorem. We observe that G(w) = � F(u 1 + Sw) = � F(ū) = 0 by the definition of G and (2). Using again that u 1 + Sw =ū , we infer that for all w sufficiently close to w . Thus, Theorem 2 is applicable to G if G � (w) = � F � (ū)S is invertible and if the norms ‖w 0 −w‖ and ‖C 0 − G � (w)‖ are sufficiently small. Once these properties are established, Theorem 2 yields from which (5) follows by virtue of the second identity in (2). It remains to show the aforementioned properties. By (3.5) in Gay's proof of [11, Theorem 3.1] (or, alternatively, by the q-linear convergence of Broyden's method) we can assume without loss of generality that ‖u 1 −ū‖ ≤ ‖u 0 −ū‖ . Therefore, it is evident that ‖w 0 −w‖ = ‖u 1 −ū‖ becomes sufficiently small if is small enough. Using that B j 1 = a T j = F � j (ū) for all j ∈ J by Lemma 1, we deduce Thus, it follows from (3.6) in the proof of [11, Theorem 3.1] (or, alternatively, from the well-known bounded deterioration principle) that ‖C 0 − G � (w)‖ ≤ 2 . The invertibility of F � (ū) implies the invertibility of G � (w) = � F � (ū)S verbatim as in the first part of the proof of Theorem 1. ◻
Theorem 3 yields the following bound for the r-order of convergence [30, Sect. 9.2] of Broyden's method.
Corollary 1 Any sequence (u k ) that satisfies (5) and converges to ū admits an r-order of convergence of at least Proof Without loss of generality we can assume that the constant C appearing in for all t and j as before. Setting ∶= max T≤t≤T+D−1 t it follows that for all k ≥ T , which proves the claim. ◻ Remark 6 Corollary 1 shows that in the setting of Theorem 3 the r-order of convergence of Broyden's method is bounded from below by 2 1∕(2d) for d ≥ 1 . In addition, the numerical experiments in Sect. 4 suggest that for d > 1 , Broyden's method may be (2d − 1)-step q-quadratically convergent, which would imply that the r-order is no smaller than 2 1∕(2d−1) . While the exact r-order of Broyden's method remains unknown, cf. also [16, pp. 308-310], we mention that the exact r-order is known for the adjoint Broyden method, cf. [17].

Numerical experiments
We provide numerical results to verify the 2d-step q-quadratic convergence established in Theorem 3. We also assess the q-order of convergence of Broyden's method, which has not been done before. We will see that the numerical results are consistent with the following conjectures C1 and C2: Here, as before, we have supposed that n − d of n equations are linear. Both conjectures are new even for d = n and, in fact, the existence of a nontrivial q-order for Broyden's method has not been proposed before. In Sect. 4.1 we present the design of the experiments, Sect. 4.2 contains the examples and results.

Implementation and accuracy
The experiments are carried out in Matlab 2017a using its variable precision arithmetic (vpa) with a precision of 1000 digits. We replace the termination criterion F(u k ) = 0 in Algorithm BROY by ‖F(u k )‖ ≤ 10 −320 . The rather small residual tolerance of 10 −320 ensures that the number of iterations is large enough to meaningfully assess asymptotic properties such as the q-order of convergence. Of course, the small residual tolerance necessitates the usage of sufficiently many digits. By k ≥ 0 we denote the final value of k in Algorithm BROY.

Known solution and random initialization
All examples have an explicitly available solution ū and the initial guesses are such that convergence to ū takes place in all runs. In all examples F � (ū) is invertible. The initial point u 0 is generated using Matlab's function rand, which produces uniformly distributed random numbers, and satisfies u 0 ∈ū + [−10 −3 , 10 −3 ] n . For B 0 we choose B 0 = F � (u 0 ) +̂‖F � (u 0 )‖R , where ̂∈ {0, 10 −30 , 10 −10 , 10 −3 } and where R ∈ ℝ n×n is a random matrix whose entries belong to [−1, 1] , but that has a particular structure in which many entries are zero. Specifically, we use two schemes for R: Either R affects only those rows of B 0 that correspond to nonlinear components of F, in which case R j = 0 for all j ∈ J (cf. Assumption 1 for notation), or it affects only those rows that correspond to affine components, in which case R j = 0 for all j ∈ {1, … , d} . In the first case we want to demonstrate that the perturbation has essentially no effect, so we allow R to be nonzero in the entire rows, i.e., for each j ∈ {1, … , d} the row R j is randomly drawn from [−1, 1] n . In the second case the aim is to show that even minimal perturbations significantly decrease the order of convergence, so we modify only one entry of B 0 per row, i.e., for each j ∈ J all entries of R j except one are zero. The nonzero entry is taken to be a random number in [−1, 1] and its position within the row is random, too. We denote ̂=̂n (nonlinear) in the first case and ̂=̂l (linear) in the second.

Quantities of interest
Let (u k ) be generated by Algorithm BROY. For k ≥ 0 we define where m ∈ {1, … , 2n} will be specified for each example. Whenever any of these quantities is undefined we set it to −1 ; e.g., m k ∶= −1 for all k ∈ {0, … , m − 1}.

Single runs and cumulative runs
We perform single runs and cumulative runs. For single runs we display the quantities of interest during the course of Algorithm BROY. For cumulative runs we perform 10,000 single runs with varying initial data as described in Sect. 4.1.2. For each of the 10,000 single runs we compute where j ∈ I ∶= {1, … , 10000} indicates the respective single run and we always use the value k 0 (j) ∶= ⌊0.75k(j)⌋ . As outcome of a cumulative run we display for several values of m. In case of m-step q-quadratic convergence we expect (Ĉ j m ) j to be bounded from above (due to the uniformity of C discussed in Remarks 4 and 5) and (̂j m ) j to be bounded from below by (approximately) 2, and this should be reflected in C + m and − m , respectively. Correspondingly, C − , + m−1 should be clearly above, respectively, below 2. If lim inf k→∞ 1 k ≥ , then the q-order of (u k ) is no smaller than . The r-order is at least as large as the q-order, cf. [30, 9.3.3.]. We view − 1 as lower bound for the q-and r-order of Algorithm BROY and we expect the actual q-order to belong to the interval [ − 1 , + 1 ] , while the r-order may be larger.

Example 1 a
The first example is [9, Example 8.2.6]. Let The mapping F has the root ū = (1, 1) T . Since F does not have affine component functions, Theorem 3 and Corollary 1 assert 4-step q-quadratic convergence and an r-order no smaller than 2 1∕4 ≈ 1.189 . Table 1 displays the numerical outcome of a single run with ̂= 0 , while Table 2 shows the data from the cumulative runs with ̂= 0 and ̂n = 10 −3 . The data suggest that the iterates converge 3-step q-quadratically rather than 4-step, which fits with conjecture C2. The worst-case estimate for the q-and r-order is − 1 ≈ 1.20 , which confirms the proven lower bound 2 1∕4 for the r-order and is in line with conjecture C1 that 2 1∕4 may also be a lower bound for the q-order. Comparing the first and second row in Table 2 shows that perturbing the rows of B 0 that correspond to nonlinear components of F has essentially no effect on the rate of convergence. In Example 1 b we will see that this is very different if rows are perturbed that correspond to affine components. The iteration numbers range from 14 to 16.

Example 1 b
We linearize the first component function of F from Example 1 a) and obtain with unchanged root ū = (1, 1) T . From Theorem 3 we expect 2-step q-quadratic convergence if B 1 0 = (2 2) , resulting in a lower bound of 1.41 for the r-order. Table 3 shows a single run with ̂= 0 , while Table 4 provides the data from the cumulative runs conducted with ̂= 0 , ̂n = 10 −3 and ̂l ∈ {10 −30 , 10 −10 , 10 −3 } . The results are very clear: For ̂= 0 and ̂n = 10 −3 the fact that only one component function of    F is not affine induces a reduction of Algorithm BROY to one dimension, so its convergence rate is the same as that of the one-dimensional secant method, i.e., convergence with exact q-and r-order 1+ [20,34]. This implies that the error decays faster than 2-step q-quadratically, which can also be seen from C − 2 and C + 2 . In contrast, even a deviation of ̂l = 10 −30 in only one entry of B 1 0 = (2 2) slows down the convergence to 3-step q-quadratic, which is the same as in the fully nonlinear Example 1 a, cf. Table 2. While this fits well with conjecture C2, we notice that for ̂l = 10 −30 the worst-case estimate − 1 = 1.16 of the q-order is somewhat smaller than our conjecture C1 of 2 1∕4 ≈ 1.189 ; on the other hand, [ − 1 , + 1 ] = [1.16, 1.24] comfortably includes 1.189. The iteration numbers vary between 9 and 10 if B 1 0 = (2 2) and between 10 and 16 otherwise.

Example 1 c
We modify Example 1 a by inserting an additional equation. Let The mapping F has the root ū = (1, 1, 1) T . Theorem 3 implies 4-step q-quadratic convergence for ̂= 0 and ̂n = 10 −3 , yielding an r-order of at least 1.189, whereas conjectures C1 and C2 predict that the latter is also the q-order and that 3 steps are sufficient for q-quadratic convergence. Table 5 shows a single run with ̂= 0 , while Tables 6 and 7 provide the data from the cumulative runs conducted with ̂= 0 and ̂n = 10 −3 , respectively, ̂l ∈ {10 −30 , 10 −10 , 10 −3 } . The results in Table 6 are similar to those from Example 1 a in Table 2 and confirm the 3-step q-quadratic convergence and the q-order of 1.19. Table 7 shows that perturbations in any entry of the third row of B 0 have a strong detrimental effect as they induce a rate between 4-step and 5-step q-quadratic convergence. This convergence is, however, consistent with conjecture C2 for d = n , and so is the worst-case estimate − 1 = 1.13 with conjecture C1. The iteration numbers range from 13 to 16 for ̂= 0 and ̂n = 10 −3 , respectively, from 15 to 20 otherwise. Table 4 Example 1 b: Cumulative runs with ̂= 0 (top), ̂n = 10 −3 (second to top), ̂l = 10 −3 (third to top), ̂l = 10 −10 (second to last), ̂l = 10 −30 (last)

Example 3
Let F ∶ ℝ 10 → ℝ 10 be given by The mapping F has the root ū = 0 . We expect no more than 6 steps for q-quadratic convergence if ̂= 0 and ̂n = 10 −3 as well as an r-order no smaller than 1.122. Table 11 displays a single run with ̂= 0 and Table 12 provides the data from the cumulative runs conducted with ̂= 0 and ̂n = 10 −3 . Five steps are sufficient for quadratic convergence and − 1 = 1.13 is compatible with the conjectured lower bound 1.122 for the q-order. Table 13 provides the data for ̂l ∈ {10 −30 , 10 −10 , 10 −3 } , but in contrast to previous experiments we have only perturbed three of the seven rows of B 0 that correspond to affine components of F, so Theorem 3 and Corollary 1 ensure 12-step q-quadratic convergence and an r-order of at least 1.06, while C1 and C2 predict 11 steps and a q-order of 1.06. Table 13 confirms the q-order and indicates that 6 to 8 steps are enough for quadratic convergence depending on the magnitude of the perturbation. The iteration numbers range from 17 to 23 for ̂= 0 and ̂n = 10 −3 , respectively, from 19 to 33 otherwise.

Example 4
As final example we consider F ∶ ℝ 6 → ℝ 6 given by where A ∈ ℝ 4×6 is a random matrix with entries in [−1, 1] that is changed after each of the 10,000 runs of the cumulative run. The root of F is ū = 0 and A is chosen such that F � (ū) is invertible. Theorem 3 ensures 4-step q-quadratic convergence for ̂= 0 and this is clearly confirmed in Table 14, that actually suggests 3-step q-quadratic convergence. In accordance with conjecture C1 the worst-case estimate − 1 = 1.20 of the q-order is slightly larger than 2 1∕4 . The iteration numbers lie between 14 and 16.  In passing, let us point out that the values depicted in Table 14 are quite similar to those in Table 2, which illustrates the key point of Theorem 3 that d determines the behavior of Broyden's method rather than n.

Summary
We have demonstrated that the local convergence of Broyden's method improves from 2n-step q-quadratic to 2d-step q-quadratic if F ∶ ℝ n → ℝ n has n − d affine component functions and the corresponding n − d rows of B 0 match those of the Jacobian of F. We have confirmed the faster convergence in numerical experiments and observed that it is stable under perturbations of the d rows of B 0 associated to nonlinear component functions of F, but not under perturbations of the remaining n − d rows. Based on the numerical results we have proposed the conjectures that Broyden's method enjoys (2d − 1)-step q-quadratic convergence for d ∈ {2, … , n} and admits a q-order of convergence that is bounded from below by 2 1∕(2d) .
Funding Open access funding provided by University of Graz.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.