Structured Two-Point Stepsize Gradient Methods for Nonlinear Least Squares
In this paper, we present two choices of structured spectral gradient methods for solving nonlinear least squares problems. In the proposed methods, the scalar multiple of identity approximation of the Hessian inverse is obtained by imposing the structured quasi-Newton condition. Moreover, we propose a simple strategy for choosing the structured scalar in the case of negative curvature direction. Using the nonmonotone line search with the quadratic interpolation backtracking technique, we prove that these proposed methods are globally convergent under suitable conditions. Numerical experiment shows that the methods are competitive with some recently developed methods.
KeywordsNonlinear least squares problems Spectral gradient method Nonmonotone line search Global convergence
Mathematics Subject Classification49M37 65K05 90C56
We thank Professor Sandra Augusta Santos of the University of Campinas (UNICAMP) SP, Brazil, for providing us with useful comments and suggestions that we used for improving the earlier version of this manuscript. We are also grateful to the suggestions of the anonymous reviewer and the Editor-in-Chief, which lead us to improve the presentation of the paper. We are indebted to Hasan Shahid, graduate student at Brown University, USA, for his careful reading and helpful comments. This research was conducted during a one year stay of the first author as a sandwich Ph.D. student at the University of Campinas (UNICAMP), SP, Brazil, supported by the Tertiary Education Trust Fund (TETFund) Academic Staff Training and Development (AST&D) intervention.
- 2.Kim, S., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: An interior-point method for large-scale \(\ell _1 \)-regularized least squares. IEEE J. Sel. Top. Signal Process. 1(4), 606–617 (2007)Google Scholar
- 8.Dennis, J.: Some computational techniques for the nonlinear least squares problem. In: Byrne, G.D., Hall, C.A. (eds.) Numerical Solution of Systems of Nonlinear Algebraic Equations, pp. 157–183. Academic Press, New York (1973)Google Scholar
- 15.Al-Baali, M.: Quasi-Newton algorithms for large-scale nonlinear least-squares. In: Pillo, G., Murli, A. (eds.) High Performance Algorithms and Software for Nonlinear Optimization, pp. 1–21. Springer, Berlin (2003)Google Scholar
- 23.Nazareth, L.: An adaptive method for minimizing a sum of squares of nonlinear functions. Working Paper WP-83-099, IIASA, Laxenburg, Austria (1983)Google Scholar
- 33.Mohammad, H., Santos, S.A.: A structured diagonal Hessian approximation method with evaluation complexity analysis for nonlinear least squares. Comput. Appl. Math. (2018). https://doi.org/10.1007/s40314-018-0696-1
- 43.La Cruz, W., Martínez, J.M., Raydan, M.: Spectral residual method without gradient information for solving large-scale nonlinear systems: theory and experiments. Technical Report RT-04-08, Universidad Central de Venezuela, Venezuela (2004). http://kuainasi.ciens.ucv.ve/mraydan/download_papers/TechRep.pdf
- 45.Lukšan, L., Vlcek, J.: Test problems for unconstrained optimization. Technical Report 897, Academy of Sciences of the Czech Republic, Institute of Computer Science (2003)Google Scholar