Abstract
This paper investigates scaling a modified limited-memory algorithm to solve unconstrained optimization problems. The basic idea was to combine the damped techniques for the limited-memory update and the technique of equilibrating the inverse Hessian matrix. Enhanced curvature information about the objective function is stored in the form of a diagonal matrix and plays the dual roles of providing an initial matrix and equilibrating for damped limited-memory iterations. Numerical experiments indicated that the new algorithm is very effective.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Liu, D., Nocedal, J.: On the limited memory BFGS method for large-scale optimization. Math. Program. 45, 503–528 (1989)
Yuan, Y.: A modified BFGS algorithm for unconstrained optimization. IMA J. Numer. Anal. 11, 325–332 (1991)
Zhang, J.Z., Deng, N.Y., Chen, L.H.: Quasi-Newton equation and related methods for unconstrained optimization. J. Optim. Theory Appl. 102, 147–167 (1999)
Xu, C., Zhang, J.: A survey of Quasi-Newton equations and Quasi-Newton methods for optimization. A. Oper. Res. 103, 213–234 (2001)
Wei, Z., Yu, G., Lian, Z., Yuan, G.: The superlinear convergence of a modified BFGS-type method for unconstrained optimization. Comput. Optim. Appl. 29, 315–332 (2004)
Li, D., Fukushima, M.: A modified BFGS method and its global convergence in nonconvex minimization. J. Compt. Appl. Math. 129, 15–35 (2001)
Yabe, H., Ogasawara, H., Yoshino, M.: Local and superlinear convergence of Quasi-Newton methods based on modified secant conditions. J. Compt. Appl. Math. 205, 617–632 (2007)
Li, D., Qi, L., Roshchina, V.: A new class of quasi-newton updating formulas. OMS 23, 237–249 (2008)
Biglari, F., Ebadian, A.: Limited memory BFGS method based on a high-order tensor model. Comput. Optim. Appl. 60, 413–422 (2015)
Al-Baali, M., Grandinetti, L.: On practical modifications of the Quasi-Newton BFGS method. Adv. Model. Optim 11, 63–76 (2009)
Al-Baali, M., Grandinetti, L., Pisacane, O.: Damped techniques for the limited memory BFGS method for large-scale optimization. J. Optim. Theory. Appl. 161, 688–699 (2014)
Powell, M.J.D.: Algorithms for nonlinear constraints that use Lagrange functions. Math. Program. 14, 224–248 (1978)
Oren, S.S.: On the selection of parameters in self-scaling variable metric algorithms. Math. Program. 3, 351–367 (1974)
Oren, S.S., Luenberger, G.: Self-scaling variable metric algorithm. Part I. Manag. Sci. 20, 845–862 (1974)
Oren, S.S., Spedicato, E.: Optimal conditioning of self-scaling variable metric algorithms. Math. Program. 10, 70–90 (1976)
Spedicato: computational experience with quasi-Newton algorithms for minimization problems of moderately large size. Report CISE-N-175, CISE documentation service, Segrate (Milano) (1975)
Dennis, J.E., Wolkowicz, H.: Sizing and least change secant methods. Technical report, Department of Mathematical Sciences, Rice University (Houston,TX) (1991)
Shanno, D.F.: Conditioning of Quasi-Newton methods for function minimization. Math. Comput. 24, 647–656 (1970)
Shanno, D.F., Phua, K.H.: Matrix conditioning and nonlinear optimization. Math. Program. 14, 149–160 (1978)
Al-Baali, M.: New initial Hessian approximations for the limited memory BFGS method for large scale optimization. J. Fac. Sci. UAE Univ. 14, 167–175 (1995)
Al-Baali, M.: Global and superlinear convergence of a restricted class of self-scaling methods with inexact line search, for convex functions. Comput. Optim. Appl. 9, 191–203 (1998)
Al-Baali, M., Conforti, D., Musmanno, R.: Computational experiments with scaled initial Hessian approximation for the Broyden family methods. Optimization 48, 375–389 (2000)
Al-Baali, M., Khalfan, H.F.: A combined class of self-scaling and modified Quasi-Newton methods. Comput. Optim. Appl. 52(2), 393–408 (2012)
Gilbert, J., Lemaréchal, C.: Some numerical experiments with variable-storage Quasi-Newton algorithms. Math. Program. Ser. B.45, 407–435 (1989)
Van der Sluis, A.: Condition number and equilibration of matrices. Numer. Math. 14, 14–23 (1969)
Zhu, M., Nazareth, J.L., Wolkowicz, H.: The quasi-Cauchy relation and diagonal updating. SIAM J. Optim. 9(4), 1192–1204 (1999)
Veersé, F., Auroux, D.: Some numerical experiments on scaling and updating L-BFGS diagonal preconditioners. INRIA Research Report 3858 (2000)
Roma, M.: Dynamic scaling based preconditioning for truncated Newton methods in large scale unconstrained optimization. Optim. Method. Softw. 20(6), 693–713 (2005)
Wilkinson, J.H.: Error analysis of direct methods of matrix inversion. J. ACM 8, 281–330 (1961)
Bunch, J.R.: Equilibration of symmetric matrices in the max-norm. J. Assoc. Comput. Mach. 18(4), 566–572 (1971)
Biglari, F.: Dynamic scaling on the limited memory BFGS method. Eur. J. Oper. Res. 243, 697–702 (2015)
Nocedal, J., Wright, S.J. (eds.): Numerical Optimization. Springer, New York (2006)
Schneider, M.H., Zenios, S.: A comparative study of algorithms for matrix balancing. Oper. Res. 38, 439–455 (1990)
Higham, N.J. (ed.): Accuracy and Stability of Numerical Algorithms. SIAM, Philadelphia (2002)
Moré, J.J., Thuente, D.J.: Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw. 20, 286–307 (1994)
Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10, 147–161 (2008)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–203 (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Ilio Galligani.
Rights and permissions
About this article
Cite this article
Biglari, F., Mahmoodpur, F. Scaling Damped Limited-Memory Updates for Unconstrained Optimization. J Optim Theory Appl 170, 177–188 (2016). https://doi.org/10.1007/s10957-016-0940-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-016-0940-z
Keywords
- Large-scale optimization
- Nonlinear programming
- Limited-memory quasi-Newton methods
- Damped technique
- Scaling technique