Abstract
Quasi-Newton methods are often used in the frame of non-linear optimization. In those methods, the quality and cost of the estimate of the Hessian matrix has a major influence on the efficiency of the optimization algorithm, which has a huge impact for computationally costly problems. One strategy to create a more accurate estimate of the Hessian consists in maximizing the use of available information during this computation. This is done by combining different characteristics. The Powell-Symmetric-Broyden method (PSB) imposes, for example, the satisfaction of the last secant equation, which is called secant update property, and the symmetry of the Hessian (Powell in Nonlinear Programming 31–65, 1970). Imposing the satisfaction of more secant equations should be the next step to include more information into the Hessian. However, Schnabel proved that this is impossible (Schnabel in quasi-Newton methods using multiple secant equations, 1983). Penalized PSB (pPSB), works around the impossibility by giving a symmetric Hessian and penalizing the non-satisfaction of the multiple secant equations by using weight factors (Gratton et al. in Optim Methods Softw 30(4):748–755, 2015). Doing so, he loses the secant update property. In this paper, we combine the properties of PSB and pPSB by adding to pPSB the secant update property. This gives us the secant update penalized PSB (SUpPSB). This new formula that we propose also avoids matrix inversions, which makes it easier to compute. Next to that, SUpPSB also performs globally better compared to pPSB.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahookhosh, M., Ghaderi, S.: On efficiency of nonmonotone Armijo-type line searches. Appl. Math. Model. 43, 170–190 (2017)
Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16(1), 1–3 (1966)
Bartels, R.H., Stewart, G.W.: Solution of the matrix equation AX + XB = C [F4]. Commun. ACM 15(9), 820–826 (1972)
Broyden, C.: On the discovery of the “good Broyden” method. Math. Program. 87(2), 209–213 (2000)
Broyden, C.G.: A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19(92), 577–593 (1965)
Broyden, C.G.: Quasi-Newton methods and their application to function minimisation. Math. Comput. 21(99), 368–381 (1967)
Chen, C., Luo, L., Han, C., Chen, Y.: Global convergence of an extended descent algorithm without line search for unconstrained optimization. Parameters 1, 2 (2018)
Degroote, J., Bathe, K.J., Vierendeels, J.: Performance of a new partitioned procedure versus a monolithic procedure in fluid-structure interaction. Comput. Struct. 87(11–12), 793–801 (2009)
Degroote, J., Hojjat, M., Stavropoulou, E., Wüchner, R., Bletzinger, K.U.: Partitioned solution of an unsteady adjoint for strongly coupled fluid-structure interactions and application to parameter identification of a one-dimensional problem. Struct. Multidiscip. Optim. 47(1), 77–94 (2013)
Ding, Y., Lushi, E., Li, Q.: Investigation of Quasi-Newton Methods for Unconstrained Optimization. Simon Fraser University, Burnaby (2004)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)
Errico, R.M.: What is an adjoint model? Bull. Am. Meteorol. Soc. 78(11), 2577–2591 (1997)
Gratton, S., Malmedy, V., Toint, P.L.: Quasi-Newton updates with weighted secant equations. Optim. Methods Softw. 30(4), 748–755 (2015)
Haelterman, R.: Analytical study of the least squares quasi-Newton method for interaction problems. Ph.D. thesis, Ghent University (2009)
Haelterman, R., Bogaers, A., Degroote, J., Boutet, N.: Quasi-Newton methods for the acceleration of multi-physics codes. IAENG Int. J. Appl. Math. 47(3), 352–360 (2017)
Jarlebring, E.: KTH royal institute of technology in Stockholm, lecture notes: numerical methods for Lyapunov equations. https://people.kth.se/~eliasj/NLA/matrixeqs.pdf. Last visited on 07 Jan 2018
Kelley, C.T.: Iterative Methods for Optimization. SIAM, Raleigh (1999)
Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. (TOMS) 7(1), 17–41 (1981)
Neumaier, A.: Universität Wien, Personnal Webpage: Global Optimization Test Problems.http://www.mat.univie.ac.at/~neum/glopt/test.html and http://www.mat.univie.ac.at/~neum/glopt/bounds.html. Last visited on 04 Feb 2018
Patelli, E., Pradlwarter, H.J.: Monte Carlo gradient estimation in high dimensions. Int. J. Numer. Methods Eng. 81(2), 172–188 (2010)
Plessix, R.E.: A review of the adjoint-state method for computing the gradient of a functional with geophysical applications. Geophys. J. Int. 167(2), 495–503 (2006)
Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Nonlinear Programming, pp. 31–65. Elsevier (1970)
Rheinboldt, W.C.: University of Pittsburgh, Lecture Notes: Quasi-Newton Methods. Url: https://www-m2.ma.tum.de/foswiki/pub/M2/Allgemeines/SemWs09/quasi-newt.pdf. Last visited on 07 Jan 2018
Ruiz, D.: A scaling algorithm to equilibrate both rows and columns norms in matrices. Technical report, CM-P00040415 (2001)
Schnabel, R.B.: Quasi-Newton Methods Using Multiple Secant Equations. Technical report, DTIC Document (1983)
Zhang, J., Xu, C.: Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations. J. Comput. Appl. Math. 137(2), 269–278 (2001)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Boutet, N., Haelterman, R. & Degroote, J. Secant update version of quasi-Newton PSB with weighted multisecant equations. Comput Optim Appl 75, 441–466 (2020). https://doi.org/10.1007/s10589-019-00164-z
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-019-00164-z