Abstract
In this paper, we study a class of derivative-free nonmonotone line search methods for solving nonlinear systems of equations, which includes the method N-DF-SANE proposed in Cheng and Li (IMA J Numer Anal 29:814–825, 2009). These methods correspond to derivative-free optimization methods applied to the minimization of a suitable merit function. Assuming that the mapping defining the system of nonlinear equations has Lipschitz continuous Jacobian, we show that the methods in the referred class need at most \({\mathcal {O}}\left( |\log (\epsilon )|\epsilon ^{-2}\right) \) function evaluations to generate an \(\epsilon \)-approximate stationary point to the merit function. For the case in which the mapping is strongly monotone, we present two methods with evaluation-complexity of \({\mathcal {O}}\left( |\log (\epsilon )|\right) \).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
For example, condition (3) for N-DF-SANE is obtained taking \(\nu _{k}=C_{k}-f(x_{k})\).
By derivative-based methods we mean methods that make explicit use of the Jacobian matrix of \(F(\,\cdot \,)\).
By Jacobian-free methods we mean methods that only require the results of matrix-vector products with J(x) (\(x\in {\mathbb {R}}^{n}\)), without the need to store J(x).
See, e.g., Geiger and Kanzow (1996).
References
Bergou EH, Diouane Y, Kungurtsev V (2020) Convergence and complexity analysis of a Levenberg-Marquardt algorithm for inverse problems. J Optim Theory Appl 185:927–944
Birgin EG, Martínez JM (2020) Secant acceleration of sequential residual methods for solving large-scale nonlinear systems of equations. arXiv: 2012.13251 [math.NA]
Birgin EG, Gardenghi JL, Marcondes DS, Martínez JM (2021) Accelerated derivative-free spectral residual method for nonlinear systems of equations. arXiv: 2104.13447 [math.NA]
Cartis C, Gould NIM, Toint PhL (2011) On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming. SIAM J Optim 21:1721–1739
Cheng W, Li D-H (2009) A derivative-free nonmonotone line search and its application to the spectral residual method. IMA J Numer Anal 29:814–825
Garmanjani R, Júdice D, Vicente LN (2016) Trust-region methods without derivatives: Wosrt case complexity and the nonsmooth case. SIAM J Optim 26:1987–2011
Geiger C, Kanzow C (1996) On the resolution of monotone complementarity problems. Comput Optim Appl 5:155–173
Gorman RP, Sejnowski TJ (1988) Analysis of hidden units in a layered network trained to classify sonar targets. Neural Netw 1:75–89
Grapiglia GN, Sachs EW (2017) On the worst-case evaluation complexity of non-monotone line search algorithms. Comput Optim Appl 68:555–577
Grapiglia GN, Yuan J, Yuan Y (2016) A derivative-free trust-region algorithm for composite nonsmooth optimization. Comput Appl Math 35:475–499
Grippo L, Lampariello F, Lucidi S (1986) A nonmonotone line search technique for Newton’s method. SIAM J Numer Anal 23:707–716
Huang J-C, Fan J-Y (2018) Global Complexity Bound of the Inexact Levenberg-Marquardt Method. J Oper Res Soc China 6:417–428
Karas EW, Santos SA, Svaiter BF (2016) Algebraic rules for computing the regularization parameter of the Levenberg-Marquardt method. Comput Optim Appl 65:723–751
Knoll DA, Keyes DE (2004) Jacobian-free Newton-Krylov methods: a survey of approaches and applications. J Comput Phys 193:357–397
La Cruz W, Raydan M (2003) Nonmonotone spectral methods for large-scale nonlinear systems. Optim Methods Softw 18:583–599
La Cruz W, Martínez JM, Raydan M (2006) Spectral residual method without gradient information for solving large-scale nonlinear systems of equations. Math Comput 75:1429–1448
La Cruz W (2017) A spectral algorithm for large-scale systems of nonlinear monotone equations. Numer Algorithms 76:1109–1130
Losh M, Fuchs A, Lemieux J-F, Vanselow A (2014) A parallel Jacobian-free Newton-Krylov solver for a coupled sea ice-ocean model. J Comput Phys 257:901–911
Lovsletten O, Rypdal M (2012) Approximated maximum likelihood estimation in multifractal random walks. Phys Rev E 85:046705046705
Miller NH, Osborne M (2014) Spatial differentiation and price discrimination in the cement industry: evidence from a structural model. Rand J Econ 45:221–247
Mohammad H, Santos SA (2018) A structured diagonal Hessian approximation method with evaluation complexity analysis for nonlinear least squares. Comput Appl Math 37:6619–6653
Moré JJ, Garbow BS, Hillstrom KE (1981) Testing unconstrained optimization software. ACM Trans Math Softw 7:17–41
Moré JJ, Wild SM (2009) Benchmarking derivative-Free optimization algorithms. SIAM J Optim 20:172–191
Ortega JM, Rheinboldt WC (1970) Iterative solution of nonlinear equations in several variables. Academic Press, New York
Sachs EW, Sachs SM (2011) Nonmonotone line searches for optimization algorithms. Control Cybern 40:1059–1075
Ueda K, Yamashita N (2010) On a global complexity bound of the Levenberg-Marquardt method. J Optim Theory Appl 147:443–453
Varadhan R, Gilbert PD (2009) BB: an R package for solving a large system of nonlinear equations and for optimizing a high-dimensional nonlinear objective function. J Stat Softw 32:1–26
Viallet M, Goffrey T, Baraffe I, Folini D, Geroux C, Popov MV, Pratt J, Wolder R (2016) A Jacobian-free Newton-Krylov method for time-implicit multidimensional hydrodynamics. Astron Astrophys 586:A153
Yildirim A, Kenway GKW, Mader CA, Martins JRRA (2019) A Jacobian-free approximate Newton-Krylov startup strategy for RANS simulations. J Comput Phys 397:108741
Zhang HC, Hager WW (2004) A nonmonotone line search technique for unconstrained optimization. SIAM J Optim 14:1043–1056
Zhao R, Fan J (2016) Global complexity bound of the Levenberg-Marquardt method. Optim Methods Softw 31:805–814
Acknowledgements
We are very grateful to the two anonymous referees, whose comments helped to improve the paper. We are also grateful to Sandra A. Santos and Orizon P. Ferreira for their insightful comments on the first version of this work.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Ernesto G. Birgin.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
G.N. Grapiglia was supported by the National Council for Scientific and Technological Development (CNPq)—Brazil (Grants 406269/2016-5 and 312777/2020-5). F. Chorobura was supported by the Coordination for the Improvement of Higher Education Personnel (CAPES)—Brazil.
Rights and permissions
About this article
Cite this article
Grapiglia, G.N., Chorobura, F. Worst-case evaluation complexity of derivative-free nonmonotone line search methods for solving nonlinear systems of equations. Comp. Appl. Math. 40, 259 (2021). https://doi.org/10.1007/s40314-021-01621-4
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s40314-021-01621-4