A global convergent quasi-Newton method for systems of monotone equations

In this paper, we propose a quasi-Newton method for solving systems of monotone equations. The method is a combination of the Broyden method and the hyperplane projection method. Under appropriate conditions, we prove that the proposed method is globally convergent. Preliminary numerical results show that the proposed method is promising.


Introduction
In this paper, we consider the problem of finding a solution of the nonlinear system of equations where F : R n → R n is continuous and monotone. By monotonicity, we mean F (x) − F (y), x − y ≥ 0, ∀x, y ∈ R n .
Nonlinear monotone equations have many practical background such as the first order necessary condition of the unconstrained convex optimization problem and the subproblems in the generalized proximal algorithms with Bregman distances [9]. Some monotone variational inequality problems can also be converted into the form of the problem by means of fixed point maps or normal maps if the underlying function satisfies some coercive conditions [12]. Many methods for solving (1.1) fall into the class of the quasi-Newton methods since they converge rapidly from sufficiently good initial guess. Since Boryden [1] proposed the first quasi-Newton method for solving nonlinear equations, there has been significant progress in the theoretical study on quasi-Newton methods, especially in local convergence analysis [2,3]. To ensure global convergence, some line search strategy for some merit function are used. Recently, Solodov and Svaiter [10] presented a Newton-type algorithm for solving systems of monotone equations. By using hybrid projection method, they showed that their method converges globally. For nonlinear equations, Griewank [4] obtained a global convergence results for Broyden's rank one method. By introduce a new line search process, Li and Fukushima [7] have developed a globally convergent Broyden-like method for solving nonlinear equations and [6] presented a globally convergent Gauss-Newton-based BFGS method for solving symmetric nonlinear equations. The method in [6,7] is not norm descent. Gu, Li, Qi and Zhou [5] generalized the method in [6] and proposed a globally convergent and norm descent BFGS method for solving symmetirc equations. Quite recently, Zhou and Li [13] proposed a global convergence BFGS method for systems of monotone equations without use of merit functions. We refer to papers [8,11] for a review on recent advances in this area.
In this paper, based on the hyperplane projection method [10], we propose a quasi-Newton method for solving systems of monotone equations without use of merit functions. The method is a combination of the Broyden method and the hyperplane projection method [10]. Under appropriate conditions, we show that the proposed method is globally convergent. Preliminary numerical results show that the method is promising.
The paper is organized as follows. In Sect. 2, after simply recalling hyperplane projection method, we present the algorithm. In Sect. 3, we establish the global convergence of the algorithm. We report some numerical results in the last section.

Algorithm
In this section, we describe the method in detail. Firstly, let us first recall the hyperplane projection method in [10]. Note that by the monotonicity of F , for anyx such that F (x) = 0, we have Let x k be the current iterate. By performing some kind of line search procedure along a directiond k , a point z k = x k + α kdk can be computed such that Thus the hyperplane strictly separates the current iterate x k from zeros of (1.1). Therefore, it is reasonable to let the next iterate x k+1 be the projection of x k onto the hyperplane. Now, we state the steps of the algorithm as follow.
Step 3. Determine steplength α k = σ k β m k such that m k is the smallest nonnegative integer m satisfying Step 4. Compute the projection of x k on H k by Step 5. Compute B k+1 by the following Broyden update formula Step 6. Compute d k+1 by solving the linear equation If the system (2.4) is not solvable or the condition is not satisfied, set d k+1 = −F (x k+1 ) and σ k+1 = max{σ min , min{ s k 2 s T k y k , σ max }}; else set σ k+1 = 1.
Remark It is easy to see from Step 6 of Algorithm 2.1 that Therefore after a finite number of reductions of α k , the line search condition (2.1) necessarily holds. Consequently, Algorithm 2.1 is well-defined.

Convergence property
This section is devoted to the global convergence of Algorithm 2.1. To establish global convergence of Algorithm 2.1, we need the following assumption.
where ∇F denotes the Jacobian of F .
Before proving global convergence of Algorithm 2.1, we first give three preliminary lemmas. The following lemma is from [10].
Then for anyx ∈ R n such that F (x) = 0, it holds that Then by the mean-value theorem, we have y k = A k+1 s k and hence Moreover, by the update formula (2.3), we have In a similar way to Lemma 2.6 in [7], it is not difficult to prove the following useful lemma.

Lemma 3.2 Suppose that Assumption 3.1 holds and the sequence {x
Since lim k∈K 1 δ k = 0, the last inequality implies that there is a constant C 1 > 0 such that for all k ∈ K 1 sufficiently large On the other hand, applying Cauchy-Schwarz inequality to (2.5), we obtain In particular, {x k } is bounded. Furthermore, it holds that either {x k } is finite and the last iterate is a solution, or the sequence is infinite and lim k→∞ x k+1 − x k = 0. Moreover, {x k } converges to some solution of (1.1).
Proof We first note that if the algorithm terminates at some iteration k, then F (z k ) = 0 or F (x k ) = 0. This means that x k or z k is a solution of (1.1). Suppose that F (z k ) = 0 and F (x k ) = 0 for all k. Then an infinite sequence {x k } is generated. It follows from (2.1) that Letx be any solution such that F (x) = 0. By (2.2), (3.4) and Lemma 3.1, we obtain Hence the sequence { x k −x 2 } is decreasing and convergent. In particular, the sequence { x k −x } is convergent and the sequence {x k } is bounded. Again by (3.5), we have Summing both side of (3.6), since the sequence { x k −x 2 } is convergent, we have (3.1). In particular, we have By (2.2) and (3.4), we obtain The last inequality together with (3.7) implies lim k→∞ α k d k = 0. (3.8) Now we consider the following two possible cases: By the line search rule, we have for all k ∈ K 1 sufficiently large, σ k β m k −1 will not satisfy (2.1). This means The boundedness of {x k } k∈K 1 implies that there exist an accumulation pointx and an infinite index set K 2 ⊂ K 1 such that lim k∈K 2 x k =x. Since the sequence {d k } k∈K 2 is also bounded, there exist an infinite index set K 3 ⊂ K 2 and an accumulation pointd such that lim k∈K 3 d k =d. Taking limit in (3.10) for k ∈ K 3 , we obtain However, it is easy to see from (2.5) that This yields a contradiction. Consequently, the case (ii) is not possible. The proof is complete.

Numerical results
In this section, we tested Algorithm 2.1 and compared it with the BFGS method in [13] and the INM method in [10]. We implemented Algorithm 2.1 with the following parameters: β = 0.6, ξ = 10 −8 and η = 10 −4 . If σ k / ∈ [σ min , σ max ], we replace it by where σ min = 10 −10 and σ max = 10 10 . We stop the iteration if the iteration number exceeds 500 or the inequality is satisfied. The BFGS method in [13] was implemented with the following parameters: β = 0.6, σ = 10 −5 , h = 10 −4 and r = 0. For the INM method in [10], we set or the iteration number exceeds 500. The codes were written in FORTRAN 90 with double precision arithmetic and carried out on a PC (CPU 3.0 GHz, 512M memory) with Windows operation system. The efficiency of the proposed method was tested on the following two problems with various dimensions and different initial points.

Problem 1 Function F is given by
Problem 2 Function F is given by We note that Problem 1 is symmetric while Problem 2 is nonsymmetric. The results are listed in Tables 1-2 Tables 1-2, we report the problem number along with the initial point number (Pro(initial)), the dimension of each test problem (dim), the number of iterations (iter), the number of function evaluations (fun) and the CPU time in seconds (time). We claim that two method fails, and use the symbol 'F', when some of the following options hold: (a) the number of iterations is greater than or equal to 500; or (b) the number of backtracking required by the line search along a step is greater than or equal to 20.
We tested each problem 100 times with the same initial point. The CPU time reported in Tables 1-2 is the average value. In the tables, "method 1" and "method 2" represent Algorithm 2.1 and the BFGS method in [13] respectively.
From Tables 1-2, we observe that method 1 performed much better than method 2 and in most cases the INM method performs best. Simultaneously, during the numerical experiments, it is interesting to note that the step d k = −F (x k ) never appeared when k > 0 in method 1. In other words, the condition d T k F k ≤ −ξ F k 2 was always satisfied.

Conclusion
In this paper, we propose an algorithm for solving nonlinear monotone equations, which combines the Broyden method and the hyperplane projection method. Under appropriate conditions, we prove that the proposed method is globally convergent. We also report some numerical results to show efficiency of the proposed method.