Mathematical Programming

, Volume 151, Issue 1, pp 249–281 | Cite as

Recent advances in trust region algorithms

Full Length Paper Series B

Abstract

Trust region methods are a class of numerical methods for optimization. Unlike line search type methods where a line search is carried out in each iteration, trust region methods compute a trial step by solving a trust region subproblem where a model function is minimized within a trust region. Due to the trust region constraint, nonconvex models can be used in trust region subproblems, and trust region algorithms can be applied to nonconvex and ill-conditioned problems. Normally it is easier to establish the global convergence of a trust region algorithm than that of its line search counterpart. In the paper, we review recent results on trust region methods for unconstrained optimization, constrained optimization, nonlinear equations and nonlinear least squares, nonsmooth optimization and optimization without derivatives. Results on trust region subproblems and regularization methods are also discussed.

Keywords

Trust region algorithms Nonlinear optimization Subproblem Complexity Convergence 

Mathematics Subject Classification

65K05 90C30 

References

  1. 1.
    Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7, 303–330 (2007)MATHMathSciNetGoogle Scholar
  2. 2.
    Ai, W.B., Zhang, S.Z.: Strong duality for the CDT subproblem: a necessary and sufficient condition. SIAM J. Optim. 19, 1735–1756 (2009)MATHMathSciNetGoogle Scholar
  3. 3.
    Bandeira, A.S., Scheinberg, K., Vicente, L.N.: Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization. Math. Program. 134, 223–257 (2012)MATHMathSciNetGoogle Scholar
  4. 4.
    Bandeira, A.S., Scheinberg, K., Vicente, L.N.: Convergence of trust region methods based on probabilistic models. SIAM J. Optim. 24, 1238–1264 (2014)MATHMathSciNetGoogle Scholar
  5. 5.
    Bastin, F., Malmedy, F., Mouffe, M., Toint, PhL, Tomanos, D.: A retrospective trust-region method for unconstrained optimization. Math. Program. 123, 395–418 (2010)MATHMathSciNetGoogle Scholar
  6. 6.
    Bienstock, D.: A Note on Polynomial Solvability of the CDT Problem. Technical Report, Columbia University, USA, arXiv:1406.6429v2 (2013)
  7. 7.
    Billups, S.C., Larson, J., Graf, P.: Derivative-free optimization of expensive functions with computational error using weighted regression. SIAM J. Optim. 23, 27–53 (2013)MATHMathSciNetGoogle Scholar
  8. 8.
    Boumal, N., Absil, P.-A.: RTRMC: A Riemannian Trust-Region Method for Low-Rank Matrix Completion. Technical Report, ICTEAM Institute Universit e catholique de Louvain B-1348 Louvain-la-Neuve (2014) (http://www.optimization-online.org/DB_HTML/2012/09/3619.html)
  9. 9.
    Burdakov, O., Gong, L.J., Yuan, Y., Zikrin, S.: On Efficiently Combining Limited Memory and Trust-Region Techniques. Techinical Report, Linkoping University, Sweden (2014)Google Scholar
  10. 10.
    Burke, J.: Descent methods for composite nondifferentiable optimization problem. Math. Program. 33, 260–279 (1985)MATHMathSciNetGoogle Scholar
  11. 11.
    Byrd, R.H., Nocedal, J., Waltz, R.A.: An Integrated Package for Nonlinear Optimization. In: Di Pillo, G., Roma, M. (eds.) Large-scale nonlinear optimization, pp. 35–59. Springer, Berlin (2006)Google Scholar
  12. 12.
    Byrd, R., Schnabel, R.B., Shultz, G.A.: A trust region algorithm for nonlinear constrained optimization. SIAM J. Numer. Anal. 24, 1152–1170 (1987)MATHMathSciNetGoogle Scholar
  13. 13.
    Cartis, C., Gould, N.I.M., Toint, Ph.L.: On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming. SIAM J. Optim. 21, 1721–1739 (2011)Google Scholar
  14. 14.
    Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127, 245–295 (2011)Google Scholar
  15. 15.
    Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program. 130, 295–319 (2011)Google Scholar
  16. 16.
    Cartis, C., Gould, N.I.M., Toint, Ph.L.: An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32(4), 1662–1695 (2012)Google Scholar
  17. 17.
    Cartis, C., Gould, N.I.M., Toint, Ph.L.: On the Evaluation Complexity of Constrained Nonlinear Least-Squares and General Constrained Nonlinear Optimization Using Second-Order Methods. Report naXys-01-2013, Dept of Mathematics, FUNDP, Namur (B) (2013)Google Scholar
  18. 18.
    Celis, M.R., Dennis, J.E., Tapia, R.A.: A trust region algorithm for nonlinear equality constrained optimization. In: Boggs, P.T., Byrd, R.H., Schnabel, R.B. (eds.) Numerical Optimization, pp. 71–82. SIAM, Philadelphia (1985)Google Scholar
  19. 19.
    Chen, X.D., Yuan, Y.: On local solutions of the CDT subproblem. SIAM J. Optim. 10, 359–383 (1999)MATHMathSciNetGoogle Scholar
  20. 20.
    Chen, X.J., Niu, L.F., Yuan, Y.: Optimality conditions and smoothing trust region Newton method for non-Lipschitz optimization. SIAM J. Optim. 23(3), 1528–1552 (2013)MATHMathSciNetGoogle Scholar
  21. 21.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM, Philadelphia (1980)Google Scholar
  22. 22.
    Coleman, T.F., Li, Y.: On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds. Math. Program. 67, 189–224 (1994)MATHMathSciNetGoogle Scholar
  23. 23.
    Coleman, T.F., Li, Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Opt. 6, 418–445 (1996)MATHMathSciNetGoogle Scholar
  24. 24.
    Conn, A.R., Gould, N.I.M., Toint, Ph.L.: Trust-Region Methods. MPS-SIAM Series on Optimization. SIAM, Philedalphia (2000)Google Scholar
  25. 25.
    Conn, A.R., Scheinberg, K., Toint, Ph.L.: Recent progress in unconstrained nonlinear optimization without derivatives. Math. Program. 79, 397–414 (1997)Google Scholar
  26. 26.
    Conn, A.R., Scheinberg, K., Toint, Ph.L.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization Tributes to M.J.D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)Google Scholar
  27. 27.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of interpolation sets in derivative free optimization. Math. Program. 111, 141–172 (2008)MATHMathSciNetGoogle Scholar
  28. 28.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of sample sets in derivative-free optimization: polynomial regression and underdetermined interpolation. IMA J. Numer. Anal. 28, 721–748 (2008)MATHMathSciNetGoogle Scholar
  29. 29.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2009)Google Scholar
  30. 30.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points. SIAM J. Optim. 20, 387–415 (2009)MATHMathSciNetGoogle Scholar
  31. 31.
    Conn, A.R., Toint, Ph.L.: An algorithm using quadratic interpolation for unconstrained derivative free optimization. In: Di Pillo, G., Gianessi, F. (eds.) Nonlinear Optimization and Applications, pp. 27–47. Plenum, New York (1996)Google Scholar
  32. 32.
    Conn, A.R., Vicente, L.N.: Bilevel derivative-free optimization its application to robust optimization. Optim. Methods Softw. 27(3), 559–575 (2012)MathSciNetGoogle Scholar
  33. 33.
    Curtis, F., Gould, N.I.M., Robinson, D., Toint, Ph.L.: An Interior-Point Trust-Funnel Algorithm for Nonlinear Optimization. Report naXys-02-2014, Dept. of Mathematics, UNamur, Namur (B), (2014)Google Scholar
  34. 34.
    Curtis, F., Jiang, H., Robinson, D.P.: An adaptive augmented Lagrangian method for large-scale constrained optimization. Math. Program. Ser. A (2014). doi:10.1007/s10107-014-0784-y
  35. 35.
    Custodio, A.L., Rocha, H., Vicente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. 46, 265–278 (2010)MATHMathSciNetGoogle Scholar
  36. 36.
    Dennis, J.E., Li, S.-B., Tapia, R.A.: A unified approach to global convergence of trust region methods for nonsmooth optimization. Math. Program. 68, 319–346 (1995)MATHMathSciNetGoogle Scholar
  37. 37.
    Dennis, J.E., Mei, H.H.: Two new unconstrained optimization algorithms which use function and gradient values. J. Optim. Theory Appl. 28, 453–482 (1979)MATHMathSciNetGoogle Scholar
  38. 38.
    Di, S., Sun, W.Y.: Trust region method for conic model to solve unconstrained optimization problems. Optim. Methods Softw. 6, 237–263 (1996)Google Scholar
  39. 39.
    Diouanne, Y., Gratton, S., Vicente, L.N.: Globally Convergent Evolutionary Strategies for Constrained Optimization. Technical Report TR-PA-14-50, CERFACS, Toulouse, France (2014)Google Scholar
  40. 40.
    Duff, I.S., Nocedal, J., Reid, J.K.: The use of linear programming for the solution of sparse sets of nonlinear equations. SIAM J. Sci. Stat. Comput. 8, 99–108 (1987)MATHMathSciNetGoogle Scholar
  41. 41.
    El Hallabi, M., Tapia, R.A.: A Global Convergence Theory for Arbitrary Norm Trust Region Methods for Nonlinear Equations. Technical Report 87–25, Dept. Math. Sciences, Rice University, USA (1987)Google Scholar
  42. 42.
    Fan, J.Y.: Convergence rate of the trust region method for nonlinear equations under local error bound condition. Comput. Optim. Appl. 34, 215–227 (2006)MATHMathSciNetGoogle Scholar
  43. 43.
    Fan, J.Y., Lu, N.: On the modified trust region algorithm. Optim. Methods Softw. (2014). doi:10.1080/10556788.2014.932943 Google Scholar
  44. 44.
    Fan, J.Y., Pan, J.Y.: An improved trust region algorithm for nonlinear equations. Comput. Optim. Appl. 48, 59–70 (2011)MATHMathSciNetGoogle Scholar
  45. 45.
    Fan, J.Y., Yuan, Y.: A new trust region algorithm with trust region radius converging to zero. in: D. Li ed. Proceedings of the 5th International Conference on Optimization: Techniques and Applications (December 2001, Hongkong), pp. 786–794 (2001)Google Scholar
  46. 46.
    Fasano, G., Morales, J.L., Nocedal, J.: On the geometry phase in model-based algorithms for derivative-free optimization. Optim. Methods Softw. 24, 145–154 (2009)MATHMathSciNetGoogle Scholar
  47. 47.
    Fletcher, R.: An Efficient, Global Convergent Algorithm for Unconstrained and Linearly Constrained Optimization Problems. Technical Report TP 431, AERE Harwell Laboratory, Oxfordshire, England (1970)Google Scholar
  48. 48.
    Fletcher, R.: Practical Methods of Optimization, Volume 1: Unconstrained Optimization. Wiley, Chichester (1980)Google Scholar
  49. 49.
    Fletcher, R.: Practical Methods of Optimization, Volume 2: Constrained Optimization. Wiley, Chichester (1981)Google Scholar
  50. 50.
    Fletcher, R.: A model algorithm for composite NDO problem. Math. Prog. Study 17, 67–76 (1982)MATHMathSciNetGoogle Scholar
  51. 51.
    Fletcher, R.: Second order correction for nondifferentiable optimization. In: Watson, G.A. (ed.) Numerical Analysis, pp. 85–115. Springer, Berlin (1982)Google Scholar
  52. 52.
    Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91, 239C269 (2002)MathSciNetGoogle Scholar
  53. 53.
    Gay, D.M.: Computing optimal local constrained steps. SIAM J. Sci. Comput. 2, 186–197 (1981)MATHMathSciNetGoogle Scholar
  54. 54.
    Gong, L.J.: Trust Region Algorithms for Unconstrained Optimization. Ph.D. thesis, Academy of Mathematics and Systems Science, Chinese Academy of Sciences (2011)Google Scholar
  55. 55.
    Gould, N.I.M., Orban, D., Sartenar, A., Toint, PhL: Sensitive of trust-region algorithms on their parameters. 4OR, Q. J. Italian Fr. Belgian OR Soc 3, 227–241 (2005)MATHGoogle Scholar
  56. 56.
    Gould, N.I.M., Orban, D., Toint, PhL: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. (TOMS) 29(4), 373–394 (2003)MATHMathSciNetGoogle Scholar
  57. 57.
    Gould, N.I.M., Robinson, D.P., Thorne, H.S.: On solving trust-region and other regularised subproblems in optimization. Math. Program. Comput. 2, 21–57 (2010)MATHMathSciNetGoogle Scholar
  58. 58.
    Gould, N.I.M., Toint, PhL: Nonlinear programming without a penalty function or a filter. Math. Program. 122, 155–196 (2010)MATHMathSciNetGoogle Scholar
  59. 59.
    Grapiglia, G.N., Yuan, J.Y., Yuan, Y.: A subspace version of the Powell–Yuan trust region algorithm for equality constrained optimization. J. Oper. Res. Soc. China 1(4), 425–451 (2013)MATHGoogle Scholar
  60. 60.
    Grapiglia, G.N., Yuan, J.Y., Yuan, Y.: On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization. Math. Program. (2014). doi:10.1007/s10107-014-0794-9
  61. 61.
    Grapiglia, G.N., Yuan, J.Y., Yuan, Y.: A derivative-free trust-region algorithm for composite nonsmooth optimization. Comput. Appl. Math. (2014). doi:10.1007/s40314-014-0201-4
  62. 62.
    Gratton, S., Royer, C.W., Vicente, L.N., Zhang, Z.: Direct Search Based on Probabilistic Descent. Technical Report 14–11, Dept. Math., University of Coimbra, Portugal (2014)Google Scholar
  63. 63.
    Gratton, S., Sartenaer, A., Toint, Ph.L.: Recursive trust-region methods for multiscale nonlinear optimization. SIAM J. Optim. 19, 414–444 (2008)Google Scholar
  64. 64.
    Gratton, S., Toint, PhL, Troeltzsch, A.: An active-set trust region method for derivative-free nonlinear bound-constrained optimization. Optim. Methods Softw. 26, 873–894 (2011)MATHMathSciNetGoogle Scholar
  65. 65.
    Gratton, S., Vicente, L.N.: A surrogate management framework using rigorous trust-region steps. Optim. Methods Softw. 29, 10–23 (2014)MATHMathSciNetGoogle Scholar
  66. 66.
    Griewank, A.: The Modification of Newtons Method for Unconstrained Optimization by Bounding Cubic Terms. Technical Report NA/12, DAMTP, University of Cambridge, England (1981)Google Scholar
  67. 67.
    Hebden, M.D.: An Algorithm for Minimization Using Exact Second Order Derivatives. Technical Report, T.P. 515, AERE Harwell Laboratory, Harwell, Oxfordshire, England (1973)Google Scholar
  68. 68.
    Hei, L.: A self-adaptive trust region algorithm. J. Comput. Math. 21, 229–236 (2003)MATHMathSciNetGoogle Scholar
  69. 69.
    Heinkenschloss, M., Vicente, L.N.: Analysis of inexact trust-region SQP algorithms. SIAM J. Optim. 12, 283–302 (2001)MATHMathSciNetGoogle Scholar
  70. 70.
    Hsia, Y., Sheu, R.L., Yuan, Y.: On the p-Regularized Trust Region Subproblem. arXiv:1409.4665 (2014)
  71. 71.
    Kanzow, C., Klug, A.: An interior-point affine-scaling trust-region method for semismooth equations with box constraints. Comput. Optim. Appl. 37, 329C353 (2007)MathSciNetGoogle Scholar
  72. 72.
    Levenberg, K.: A method for solution of certain problems in least squares. Q. J. Appl. Math. 2, 164–168 (1944)MATHMathSciNetGoogle Scholar
  73. 73.
    Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989)MATHMathSciNetGoogle Scholar
  74. 74.
    Madsen, K.: An algorithm for the minimax solution of overdetermined systems of nonlinear equations. J. Inst. Math. Its Appl. 16, 321–328 (1975)MATHMathSciNetGoogle Scholar
  75. 75.
    Marazzi, M., Nocedal, J.: Wedge trust region methods for derivative free optimization. Math. Program. 91, 289–305 (2002)MATHMathSciNetGoogle Scholar
  76. 76.
    Marquardt, D.: An algorithm for least-squares estimation on nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)MATHMathSciNetGoogle Scholar
  77. 77.
    Martinez, J.M.: Local minimizers of quadratic functions on Euclidean balls and spheres. SIAM J. Optim. 4, 159–176 (1994)MATHMathSciNetGoogle Scholar
  78. 78.
    Moré, J.: The Levenberg–Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Numerical Analysis, pp. 105–116. Springer, Berlin (1978)Google Scholar
  79. 79.
    Moré, J.J.: Recent developments in algorithms and software for trust region methods. In: Bachem, A., Grötschel, M., Korte, B. (eds.) Mathematical Programming: The State of the Art, pp. 258–287. Springer, Berlin (1983)Google Scholar
  80. 80.
    Morrison, D.D.: Methods for nonlinear least squares problems and convergence proofs. In: Lorell, J., Yagi, F. (eds.) Proceedings of the Seminar on Tracking Programs and Orbit Determination, pp. 1–9. Jet Propulsion Laboratory, Pasadena (1960)Google Scholar
  81. 81.
    Nesterov, Yu., Polyak, B.T.: Cubic regularization of Newtons method and its global performance. Math. Program. 108, 177–205 (2006)MATHMathSciNetGoogle Scholar
  82. 82.
    Niu, L.F., Yuan, Y.: A new trust region algorithm for nonlinear constrained optimization. J. Comput. Math. 28, 72–86 (2010)MATHMathSciNetGoogle Scholar
  83. 83.
    Nocedal, J., Yuan, Y.: Combining trust region and line search techniques. In: Yuan, Y. (ed.) Advances in Nonlinear Programming, pp. 153–175. Kluwer, Netherlands (1998)Google Scholar
  84. 84.
    Oeuvray, R., Bierlaire, M.: BOOSTER: a derivative-free algorithm based on radial basis functions. Inter. J. Model. Simul. 29, 349–371 (2009)Google Scholar
  85. 85.
    Omojokun, E.O.: Trust Region Algorithms for Optimization with Nonlinear Equality and Inequality Constraints. Ph.D. Thesis, University of Colorado at Boulder, USA (1989)Google Scholar
  86. 86.
    Osborne, M.R.: Nonlinear least squares—the Levenberg–Marquardt algorithm revisited. J. Aust. Math. Soc. B 19, 343–357 (1976)MATHMathSciNetGoogle Scholar
  87. 87.
    Peng, J., Yuan, Y.: Optimality conditions for the minimization of a quadratic with two quadratic constraints. SIAM J. Optim. 7, 579–594 (1997)MATHMathSciNetGoogle Scholar
  88. 88.
    Powell, M.J.D.: A hybrid method for nonlinear equations. In: Rabinowitz, P. (ed.) Numerical Methods for Nonlinear Algebraic Equations, pp. 87–114. Gordon and Breach, London (1970)Google Scholar
  89. 89.
    Powell, M.J.D.: A Fortran subroutine for solving systems of nonlinear algebraic equations. In: Robinowitz, P. (ed.) Numerical Methods for Nonlinear Algebraic Equations, pp. 115–161. Gordon and Breach, London (1970)Google Scholar
  90. 90.
    Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Rosen, J.B., Mangasarian, O.L., Ritter, K. (eds.) Nonlinear Programming, pp. 31–65. Academic Press, London (1970)Google Scholar
  91. 91.
    Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming 2, pp. 1–27. Academic Press, New York (1975)Google Scholar
  92. 92.
    Powell, M.J.D.: Some global convergence properties of a variable metric algorithm for minimization without exact line searches. In: Cottle, R.W., Lemke, C.E. (eds.) Nonlinear Programming, SIAM-AMS Proceedings, vol. IX, pp. 53–72. SIAM, Philadelphia (1976)Google Scholar
  93. 93.
    Powell, M.J.D.: General algorithms for discrete nonlinear approximation calculations. In: Schumaker, L.L. (ed.) Approximation Theory IV, pp. 187–218. Academy Press, New York (1984)Google Scholar
  94. 94.
    Powell, M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez, S., Hennart, J.P. (eds.) Advances in Optimization and Numercial Analysis. Mathematics and Its Applications, vol. 275, pp. 51–67. Springer, Netherland (1994)Google Scholar
  95. 95.
    Powell, M.J.D.: Direct search algorithms for optimization calculations. Acta Numer. 7, 287–336 (1998)Google Scholar
  96. 96.
    Powell, M.J.D.: UOBYQA: unconstrained optimization by quadratic approximation. Math. Program. 92, 555–582 (2002)MATHMathSciNetGoogle Scholar
  97. 97.
    Powell, M.J.D.: On trust region methods for unconstrained minimization without derivatives. Math. Program. 97, 605–623 (2003)MATHMathSciNetGoogle Scholar
  98. 98.
    Powell, M.J.D.: Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Math. Program. 100, 183–215 (2004)MATHMathSciNetGoogle Scholar
  99. 99.
    Powell, M.J.D.: On the use of quadratic models in unconstrained minimization without derivatives. Optim. Methods Softw. 19, 399–411 (2004)MATHMathSciNetGoogle Scholar
  100. 100.
    Powell, M.J.D.: The NEWOUA software for unconstrained optimization without derivatives. In: Di Pillo, G., Roma, M. (eds.) Large Scale Nonlinear Optimization. Nonconvex Optimization and Its Applications, vol. 83, pp. 255–297. Springer, Berlin (2006)Google Scholar
  101. 101.
    Powell, M.J.D.: Developments of NEWUOA for minimization without derivatives. IMA J. Numer. Anal. 28, 649–664 (2008)MATHMathSciNetGoogle Scholar
  102. 102.
    Powell, M.J.D.: The BOBYQA Algorithym for Bound Constrained Optimization Without Derivatives. Technical Report DAMTP 2009/NA06, CMS, University of Cambridge, UK (2009)Google Scholar
  103. 103.
    Powell, M.J.D.: On the convergence of trust region algorithm for unconstrained minimization without derivatives. Comput. Optim. Appl. 53, 527–555 (2012)MATHMathSciNetGoogle Scholar
  104. 104.
    Powell, M.J.D.: Beyond symetric Broyden for updating quadratic models in minimization without derivatives. Math. Program. 138, 475–500 (2013)MATHMathSciNetGoogle Scholar
  105. 105.
    Powell, M.J.D.: On fast trust region methods for quadratic models with linear constraints. DAMTP 2014/NA02, CMS, University of Cambridge (2014)Google Scholar
  106. 106.
    Powell, M.J.D., Yuan, Y.: Conditions for superlinear convergence in \(l_1\) and \(l_\infty \) solutions of overdetermined non-linear equations. IMA J. Numer. Anal. 4, 241–251 (1984)MATHMathSciNetGoogle Scholar
  107. 107.
    Powell, M.J.D., Yuan, Y.: A trust region algorithm for equality constrained optimization. Math. Program. 49, 189–211 (1991)MathSciNetGoogle Scholar
  108. 108.
    Qi, L., Sun, J.: A trust region algorithm for minimization of locally Lipschitzian functions. Math. Program. 66, 25–43 (1994)MATHMathSciNetGoogle Scholar
  109. 109.
    Sampaio, Ph.R., Toint, Ph.L.: A Derivative-Free Trust-Funnel Method for Equality-Constrained Nonlinear Optimization. Report naXys-08-2014, Dept of Mathematics, UNamur, Namur (B), (2014)Google Scholar
  110. 110.
    Sampaio, R.J.B., Yuan, J.Y., Sun, W.Y.: Trust region algorithm for nonsmooth optimization. Appl. Math. Comput. 85, 109–116 (1997)MATHMathSciNetGoogle Scholar
  111. 111.
    Scheinberg, K., Toint, Ph.L.: Self-correcting geometry in model-based algorithms for derivative-free unconstrained optimization. SIAM J. Optim. 20, 3512–3532 (2010)Google Scholar
  112. 112.
    Siegel, D.: Implementing and Modifying Broyden Class Updates for Large Scale Optimization. Report DAMPT1992/NA12, University of Cambridge, Department ofApplied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, England (1992)Google Scholar
  113. 113.
    Sorensen, D.C.: Newton’s method with a model trust region modification. SIAM J. Numer. Anal. 19, 409–426 (1982)MATHMathSciNetGoogle Scholar
  114. 114.
    Sorensen, D.C.: Trust region methods for unconstrained optimization. In: Powell, M.J.D. (ed.) Nonlinear Optimization 1981, pp. 29–39. Academic Press, London (1982)Google Scholar
  115. 115.
    Steihaug, T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–637 (1983)MATHMathSciNetGoogle Scholar
  116. 116.
    Sturm, J.F., Zhang, S.Z.: On cones of nonnegative quadratic functions. Math. Oper. Res. 28, 246–267 (2003)MATHMathSciNetGoogle Scholar
  117. 117.
    Sun, W.Y., Yuan, Y.: A conic trust-region method for nonlinearly constrained optimization. Ann. Oper. Res. 103, 175–191 (2001)MATHMathSciNetGoogle Scholar
  118. 118.
    Toint, Ph.L.: Some numerical result using a sparse matrix updating formula in unconstrainded optimization. Math. Comput. 32, 839–851 (1978)Google Scholar
  119. 119.
    Toint, Ph.L.: On the superlinear convergence of an algorithm for solving a sparse minimization problem. SIAM J. Numer. Anal. 16, 1036–1045 (1979)Google Scholar
  120. 120.
    Toint, Ph.L.: Sparsity exploiting quasi-Newton methods for unconstrained optimization. In: Dixon, L.C.W., Spedicato, E., Szego, G.P. (eds.) Nonlinear Optimization: Theory and Algorithms, pp. 65–90. Birkhauser, Belgium (1980)Google Scholar
  121. 121.
    Toint, Ph.L.: Convergence Properties of a Class of Minimization Algorithms That Use a Possibly Unbounded Sequences of Quadratic Approximation. Technical Report 81/1, Dept. Math., University of Namur, Belgium (1981)Google Scholar
  122. 122.
    Toint, Ph.L.: Towards an efficient sparsity exploiting Newton method for minimization. In: Duff, I.S. (ed.) Sparse Matrices and Their Uses, pp. 57–88. Academic Press, Lodon (1978)Google Scholar
  123. 123.
    Toint, Ph.L.: A non-monotone trust region algorithm for nonlinear optimization subject to convex constraints. Math. Program. 77, 69–94 (1997)Google Scholar
  124. 124.
    Toint, Ph.L.: Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization. Optim. Methods Sofw. 28, 82–95 (2013)Google Scholar
  125. 125.
    Troeltzsch, A.: A sequential quadratic programming algorithm for equality-constrained optimization without derivatives. Optim. Lett. (2014). doi:10.1007/s11590-014-0830-y
  126. 126.
    Vardi, A.: A trust region algorithm for equality constrained minimization: convergence properties and implementation. SIAM J. Numer. Anal. 22, 575–591 (1985)MATHMathSciNetGoogle Scholar
  127. 127.
    Villacorta, K.D.V., Oliveira, P.R., Soubeyran, A.: A trust-region method for unconstrained multiobjective problems with applications in satisficing processes. J. Opt. Theory Appl. 160, 865–889 (2014)MATHMathSciNetGoogle Scholar
  128. 128.
    Vicente, L.N.: A comparison between line searches and trust regions for nonlinear optimization. Investigacao Operacional 16, 173–179 (1996)Google Scholar
  129. 129.
    Waltz, R.A., Morales, J.L., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math. Program. 107, 391C408 (2006)MathSciNetGoogle Scholar
  130. 130.
    Wang, Z.H., Yuan, Y.: A subspace implementation of quasi-Newton trust region methods for unconstrained optimization. Numer. Math. 104, 241–269 (2006)MATHMathSciNetGoogle Scholar
  131. 131.
    Wang, X., Yuan, Y.: A trust region method based on a new affine scaling technique for simple bounded optimization. Optim. Methods Soft. 28, 871–888 (2013)MATHMathSciNetGoogle Scholar
  132. 132.
    Wang, X., Yuan, Y.: An augmented Lagrangian trust region method for equality constrained optimization. Optim. Methods Softw. (2014). doi:10.1080/10556788.2014.940947
  133. 133.
    Wang X., Zhang H.: An augmented Lagrangian affine scaling method for nonlinear programming. https://www.math.lsu.edu/hozhang/papers/ALAS, (2014)
  134. 134.
    Weiser, M., Deuflhard, P., Erdmann, B.: Affine conjugate adaptive Newton methods for nonlinear elastomechanics. Optim. Methods Softw. 22(3), 413–431 (2007)MATHMathSciNetGoogle Scholar
  135. 135.
    Wild, S.M., Regis, R.G., Shoemaker, C.A.S.: ORBIT: Optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30, 3197–3219 (2008)MATHMathSciNetGoogle Scholar
  136. 136.
    Wild, S.M., Shoemaker, C.A.S.: Global convergence of radial basis function trust-region algorithms for derivative-free optimization. SIAM Rev. 55, 349–371 (2013)MATHMathSciNetGoogle Scholar
  137. 137.
    Winfield, D.H.: Function and Functional Minimization by Interpolation in Data Tables. Ph.D. Thesis, Harvard Unviersity, Cambridge, MA, USA (1969)Google Scholar
  138. 138.
    Winfield, D.H.: Function minimization by interpolation in date table. IMA J. Numer. Anal. 12, 339–347 (1973)MATHMathSciNetGoogle Scholar
  139. 139.
    Yamashita, H., Yabe, H., Tanabe, T.: A globally and superlinearly convergent primal-dual interior point trust region method for large scale constrained optimization. Math. Program. 102, 111–151 (2005)MATHMathSciNetGoogle Scholar
  140. 140.
    Yuan, Y.: An example of only linearly convergence of trust region algorithms for nonsmooth optimization. IMA J. Numer. Anal. 4, 327–335 (1984)MATHMathSciNetGoogle Scholar
  141. 141.
    Yuan, Y.: Conditions for convergence of trust region algorithms for nonsmooth optimization. Math. Program. 31(2), 220–228 (1985)MATHGoogle Scholar
  142. 142.
    Yuan, Y.: On the superlinear convergence of a trust region algorithm for nonsmooth optimization. Math. Program. 31(3), 269–285 (1985)MATHGoogle Scholar
  143. 143.
    Yuan, Y.: Some Theories and Algorithms in Nonlinear Programming. Ph.D. Thesis, University of Cambridge, UK (1985)Google Scholar
  144. 144.
    Yuan, Y.: On a subproblem of trust region algorithms for constrained optimization. Math. Program. 47, 53–63 (1990)MATHGoogle Scholar
  145. 145.
    Yuan, Y.: A dual algorithm for minimizing a quadratic function with two quadratic constraints. J. Comput. Math. 9, 348–359 (1991)MATHMathSciNetGoogle Scholar
  146. 146.
    Yuan, Y.: On the convergence of a new trust region algorithm. Numer. Math. 70, 515–539 (1995)MATHMathSciNetGoogle Scholar
  147. 147.
    Yuan, Y.: Trust region algorithms for nonlinear equations. Information 1, 7–20 (1998)MATHMathSciNetGoogle Scholar
  148. 148.
    Yuan, Y.: A review of trust region algorithms for optimization. In: Ball, J.M., Hunt, J.C.R. (eds.) ICIAM 99, Proceedingss of the Fourth International Cogress on Industrial and Applied Mathematics, pp. 271–282. Oxford University Press, Oxford (2000)Google Scholar
  149. 149.
    Yuan, Y.: On the truncated conjugate gradient method. Math. Program. 87, 561–571 (2000)MATHMathSciNetGoogle Scholar
  150. 150.
    Yuan, Y.: A trust region algorithm for Nash equilibrium problems. Pac. J. Optim. 7, 125–138 (2011)MATHMathSciNetGoogle Scholar
  151. 151.
    Yuan, Y.: A review on subspace methods for nonlinear optimization. In: Proceedings of the International Congress of Mathematics 2014, Seoul, Korea, pp. 807–827 (2014)Google Scholar
  152. 152.
    Zhang, H.C., Conn, A.R.: On the local convergence of a derivative-free algorithm for least-squares minimization. Comput. Optim. Appl. 51(2), 481–507 (2012)MATHMathSciNetGoogle Scholar
  153. 153.
    Zhang, H.C., Conn, A.R., Scheinberg, K.: A derivative-free algorithm for least squares minimization. SIAM J. Optim. 20(6), 3555–3576 (2012)MathSciNetGoogle Scholar
  154. 154.
    Zhang, J.: A new trust region method for nonlinear equations. Math. Methods Oper. Res. 58, 283–298 (2003)MATHMathSciNetGoogle Scholar
  155. 155.
    Zhang, L.P.: A new trust region algorithm for nonsmooth convex minimization. Appl. Math. Comput. 193, 135–142 (2007)MATHMathSciNetGoogle Scholar
  156. 156.
    Zhang, Y.: Computing a Celis–Dennis–Tapia trust region step for equality constrained optimization. Math. Program. 55, 109–124 (1992)MATHGoogle Scholar
  157. 157.
    Zhang, Z.: Sobolev seminorm of quadratic functions with applications to derivative-free optimization. Math. Program. 146, 77–96 (2014)MATHMathSciNetGoogle Scholar
  158. 158.
    Zikrin, S.: Large-Scale Optimization Mehods with Applications to Design of Filter Networks. Linkoping Studies in Science and Technology, Dissertation No. 151, Likoping University, Sweden (2014)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2015

Authors and Affiliations

  1. 1.State Key Laboratory of Scientific/Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems ScienceChinese Academy of SciencesBeijingChina

Personalised recommendations