A Levenberg–Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients
In this paper we consider large scale nonlinear least-squares problems for which function and gradient are evaluated with dynamic accuracy and propose a Levenberg–Marquardt method for solving such problems. More precisely, we consider the case in which the exact function to optimize is not available or its evaluation is computationally demanding, but approximations of it are available at any prescribed accuracy level. The proposed method relies on a control of the accuracy level, and imposes an improvement of function approximations when the accuracy is detected to be too low to proceed with the optimization process. We prove global and local convergence and complexity of our procedure and show encouraging numerical results on test problems arising in data assimilation and machine learning.
Mathematics Subject Classification65K05 90C30 90C26 90C06
We thank the authors of  for providing us the Matlab code for the data assimilation test problem.
- 6.Blanchet, J., Cartis, C., Menickelly, M., Scheinberg, K.: Convergence rate analysis of a stochastic trust region method for nonconvex optimization (2016) arXiv preprint arXiv:1609.07428
- 7.Bollapragada, R., Byrd, R., Nocedal, J.: Exact and inexact subsampled Newton methods for optimization (2016) arXiv preprint arXiv:1609.08502
- 8.Bottou, L., Curtis, F.E., Nocedal, J.: Optimization methods for large-scale machine learning (2016) arXiv preprint arXiv:1606.04838
- 9.Causality Workbench Team. A Marketing Dataset. http://www.causality.inf.ethz.ch/data/CINA.html (2008). Accessed 23 Jan 2017
- 19.Kelley, C.T.: Iterative Methods for Optimization: Matlab Codes. http://www4.ncsu.edu/~ctk/matlab_darts.html. Accessed 12 Mar 2017
- 23.Roosta-Khorasani, F., Mahoney, M.W.: Sub-sampled Newton methods 1: globally convergent algorithms (2016) arXiv preprint arXiv:1601.04737
- 24.Saunders, M.: Systems Optimization Laboratory. http://web.stanford.edu/group/SOL/software/cgls/. Accessed 15 Nov 2016