Abstract
In this paper, we propose a filter sequential adaptive regularization algorithm using cubics (ARC) for solving nonlinear equality constrained optimization. Similar to sequential quadratic programming methods, an ARC subproblem with linearized constraints is considered to obtain a trial step in each iteration. Composite step methods and reduced Hessian methods are employed to tackle the linearized constraints. As a result, a trial step is decomposed into the sum of a normal step and a tangential step which is computed by a standard ARC subproblem. Then, the new iteration is determined by filter methods and ARC framework. The global convergence of the algorithm is proved under some reasonable assumptions. Preliminary numerical experiments and comparison results are reported.
Similar content being viewed by others
References
Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Methods. Society for Industrial and Applied Mathematics, Philedalphia (2000)
Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91(2, Ser. A), 239–269 (2002)
Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127(2, Ser. A), 245–295 (2011)
Griewank, A.: The modification of Newton’s method for unconstrained optimization by bounding cubic terms. Technical report Technical report NA/12 (1981). Department of Applied Mathematics and Theoretical Physics, University of Cambridge (1981)
Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108(1, Ser. A), 177–205 (2006)
Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program. 130(2, Ser. A), 295–319 (2011)
Bellavia, S., Gurioli, G., Morini, B.: Adaptive cubic regularization methods with dynamic inexact Hessian information and applications to finite-sum minimization. IMA J. Numer. Anal. 41(1), 764–799 (2021)
Benson, H.Y., Shanno, D.F.: Cubic regularization in symmetric rank-1 quasi-Newton methods. Math. Program. Comput. 10(4), 457–486 (2018)
Bergou, E.H., Diouane, Y., Gratton, S.: A line-search algorithm inspired by the adaptive cubic regularization framework and complexity analysis. J. Optim. Theory Appl. 178(3), 885–913 (2018)
Bianconcini, T., Sciandrone, M.: A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques. Optim. Methods Softw. 31(5), 1008–1035 (2016)
Cartis, C., Gould, N.I.M., Toint, P.L.: A concise second-order complexity analysis for unconstrained optimization using high-order regularized models. Optim. Methods Softw. 35(2), 243–256 (2020)
Dehghan Niri, T., Heydari, M., Hosseini, M.M.: An improvement of adaptive cubic regularization method for unconstrained optimization problems. Int. J. Comput. Math. 98(2), 271–287 (2021)
Dussault, J.-P.: ARCq: a new adaptive regularization by cubics. Optim. Methods Softw. 33(2), 322–335 (2018)
Martínez, J.M., Raydan, M.: Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization. J. Global Optim. 68(2), 367–385 (2017)
Park, S., Jung, S.H., Pardalos, P.M.: Combining stochastic adaptive cubic regularization with negative curvature for nonconvex optimization. J. Optim. Theory Appl. 184(3), 953–971 (2020)
Lu, S., Wei, Z., Li, L.: A trust region algorithm with adaptive cubic regularization methods for nonsmooth convex minimization. Comput. Optim. Appl. 51(2), 551–573 (2012)
Zhao, T., Liu, H., Liu, Z.: New subspace minimization conjugate gradient methods based on regularization model for unconstrained optimization. Numer. Algoritm. 87(4), 1501–1534 (2021)
Bergou, E., Diouane, Y., Gratton, S.: On the use of the energy norm in trust-region and adaptive cubic regularization subproblems. Comput. Optim. Appl. 68(3), 533–554 (2017)
Gould, N.I.M., Porcelli, M., Toint, P.L.: Updating the regularization parameter in the adaptive cubic regularization algorithm. Comput. Optim. Appl. 53(1), 1–22 (2012)
Cartis, C., Gould, N.I.M., Toint, P.L.: An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32(4), 1662–1695 (2012)
Cartis, C., Gould, N.I., Toint, P.L.: Universal regularization methods: varying the power, the smoothness and the accuracy. SIAM J. Optim. 29(1), 595–615 (2019)
Agarwal, N., Boumal, N., Bullins, B., Cartis, C.: Adaptive regularization with cubics on manifolds. Math. Program. 188(1, Ser. A), 85–134 (2021)
Lubkoll, L., Schiela, A., Weiser, M.: An affine covariant composite step method for optimization with PDEs as equality constraints. Optim. Methods Softw. 32(5), 1132–1161 (2017)
Ahmadzadeh, H., Mahdavi-Amiri, N.: A competitive inexact nonmonotone filter SQP method: convergence analysis and numerical results. Optim. Methods Softw., 1–34 (2021)
Birgin, E.G., Martínez, J.M.: On regularization and active-set methods with complexity for constrained optimization. SIAM J. Optim. 28(2), 1367–1395 (2018)
Jia, X., Liang, X., Shen, C., Zhang, L.-H.: Solving the cubic regularization model by a nested restarting Lanczos method. SIAM J. Matrix Anal. Appl. 43(2), 812–839 (2022)
Hsia, Y., Sheu, R. -L., Yuan, Y.-x.: Theory and application of p-regularized subproblems for p > 2. Optim. Methods Softw. 32(5), 1059–1077 (2017)
Jiang, R., Yue, M.-C., Zhou, Z.: An accelerated first-order method with complexity analysis for solving cubic regularization subproblems. Comput. Optim. Appl. 79(2), 471–506 (2021)
Lieder, F.: Solving large-scale cubic regularization by a generalized eigenvalue problem. SIAM J. Optim. 30(4), 3345–3358 (2020)
Fletcher, R., Gould, N.I.M., Leyffer, S., Toint, P.L., Wächter, A.: Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming. SIAM J. Optim. 13(3), 635–6592003 (2002)
Ulbrich, M., Ulbrich, S., Vicente, L.N.: A globally convergent primal-dual interior-point filter method for nonlinear programming. Math. Program. 100(2, Ser. A), 379–410 (2004)
Walther, A., Biegler, L.: On an inexact trust-region SQP-filter method for constrained nonlinear optimization. Comput. Optim. Appl. 63(3), 613–638 (2016)
Wächter, A., Biegler, L.T.: Line search filter methods for nonlinear programming: motivation and global convergence. SIAM J. Optim. 16 (1), 1–31 (2005)
Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1, Ser. A), 25–57 (2006)
Gould, N.I.M., Loh, Y., Robinson, D.P.: A nonmonotone filter SQP method: local convergence and numerical results. SIAM J. Optim. 25 (3), 1885–1911 (2015)
Shen, C., Leyffer, S., Fletcher, R.: A nonmonotone filter method for nonlinear optimization. Comput. Optim. Appl. 52(3), 583–607 (2012)
Pei, Y.G., Zhu, D.T.: On the global convergence of a projective trust region algorithm for nonlinear equality constrained optimization. Acta Math. Sin. (Engl. Ser.) 34(12), 1804–1828 (2018)
Gould, N.I.M., Orban, D., Toint, P.L.: Cutest: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3), 545–557 (2015)
Chen, Z., Dai, Y.-H., Liu, J.: A penalty-free method with superlinear convergence for equality constrained optimization. Comput. Optim. Appl. 76(3), 801–833 (2020)
Liu, X., Yuan, Y.: A sequential quadratic programming method without a penalty function or a filter for nonlinear equality constrained optimization. SIAM J. Optim. 21(2), 545–571 (2011)
Dolan, E.D., Moré, J. J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2, Ser. A), 201–213 (2002)
Acknowledgements
The authors are grateful to editor and reviewer for their valuable suggestions, insightful comments, and encouragement regarding this paper, which greatly help us to improve the presentation of this paper.
Funding
This study was financially supported by the National Natural Science Foundation of China (12071133) and Key Scientific Research Project for Colleges and Universities in Henan Province (21A110012).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Author contribution
Y. Pei and S. Song wrote the main manuscript text and did numerical experiments. D. Zhu presented some idea of the proofs of convergence. Y. Pei and S. Song revised the manuscript according to the reviewer’s comments. All authors reviewed the manuscript.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Shaofang Song and Detong Zhu contributed equally to this work.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pei, Y., Song, S. & Zhu, D. A filter sequential adaptive cubic regularization algorithm for nonlinear constrained optimization. Numer Algor 93, 1481–1507 (2023). https://doi.org/10.1007/s11075-022-01475-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-022-01475-9