Skip to main content
SpringerLink
Log in

A new simple model trust-region method with generalized Barzilai-Borwein parameter for large-scale optimization

  • Articles
  • Published:
Science China Mathematics Aims and scope Submit manuscript

Abstract

In this paper a new trust region method with simple model for solving large-scale unconstrained nonlinear optimization is proposed. By employing the generalized weak quasi-Newton equations, we derive several schemes to construct variants of scalar matrices as the Hessian approximation used in the trust region subproblem. Under some reasonable conditions, global convergence of the proposed algorithm is established in the trust region framework. The numerical experiments on solving the test problems with dimensions from 50 to 20,000 in the CUTEr library are reported to show efficiency of the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Barzilai J, Borwein J M. Two point step size gradient method. IMA J Numer Anal, 1988, 8: 141–148

    Article  MathSciNet  MATH  Google Scholar 

  2. Bongartz I, Conn A R, Gould N I M, et al. CUTE: Constrained and unconstrained testing environment. ACM Trans Math Software, 1995, 21: 123–160

    Article  MATH  Google Scholar 

  3. Byrd R H, Schnabel R B, Schultz G A. Approximate solution of the trust regions problem by minimization over two-dimensional subspaces. Math Program, 1988, 40: 247–263

    Article  MATH  Google Scholar 

  4. Chen Y, Sun W. A dwindling filter line search method for unconstrained optimization. Math Comp, 2015, 84: 187–208

    Article  MathSciNet  MATH  Google Scholar 

  5. Conn A R, Gould N I M, Toint P L. Trust Region Methods. Philadelphia: SIAM, 2000

    Book  MATH  Google Scholar 

  6. Dai Y H, Fletcher R. Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming. Numer Math, 2005, 100: 21–47

    Article  MathSciNet  MATH  Google Scholar 

  7. Dai YH, Liao LZ. R-linear convergence of the Barzilai and Borwein gradient method. IMA J Numer Anal, 2002, 22: 1–10

    Article  MathSciNet  MATH  Google Scholar 

  8. Dai Y H, Yuan J Y, Yuan Y X. Modified two-point stepsize gradient methods for unconstrained optimization. Comput Optim Appl, 2002, 22: 103–109

    Article  MathSciNet  MATH  Google Scholar 

  9. Dai Y H, Zhang H. An adaptive two-point stepsize gradient algorithm. Numer Algor, 2001, 27: 377–385

    Article  MathSciNet  MATH  Google Scholar 

  10. Deng N Y, Xiao Y, Zhou F J. Nonmontonic trust region algorithm. J Optim Theory Appl, 1993, 26: 259–285

    Article  MathSciNet  MATH  Google Scholar 

  11. Dennis J E, Wolkowicz H. Sizing and least-change secant methods. SIAM J Numer Anal, 1993, 30: 1291–1314

    Article  MathSciNet  MATH  Google Scholar 

  12. Di S, Sun W. Trust region method for conic model to solve unconstrained optimization. Optim Methods Softw, 1996, 6: 237–263

    Article  Google Scholar 

  13. Dolan E D, Moré J J. Benchmarking optimization software with performance profiles. Math Program, 2002, 91: 201–213

    Article  MathSciNet  MATH  Google Scholar 

  14. Farid M, Leong W J, Hassan M A. A new two-step gradient-type method for large-scale unconstrained optimization. Comput Math Appl, 2010, 59: 3301–3307

    Article  MathSciNet  MATH  Google Scholar 

  15. Grippo L, Lamparillo F, Lucidi S. A nonmonotone line search technique for Newton’s method. SIAM J Numer Anal, 1986, 23: 707–716

    Article  MathSciNet  MATH  Google Scholar 

  16. Hager W W. Minimizing a quadratic over a sphere. SIAM J Optim, 2001, 12: 188–208

    Article  MathSciNet  MATH  Google Scholar 

  17. Levenberg K. A method for the solution of certain nonlinear problems in least squares. Q Appl Math, 1994, 2: 164–168

    MathSciNet  MATH  Google Scholar 

  18. Li G D, Yuan Y. Computing Celis-Dennis-Tapia step. J Comput Math, 2005, 23: 463–478

    MathSciNet  MATH  Google Scholar 

  19. Liu D C, Nocedal J. On the limited memory BFGS method for large scale optimization. Math Program, 1989, 45: 503–528

    Article  MathSciNet  MATH  Google Scholar 

  20. Maratos N. Exact penalty function algorithms for finite dimensional and control optimization problems. PhD thesis. London: University of London, 1978

    Google Scholar 

  21. Marquardt D W. An algorithm for least-squares estimation of nonlinear inequalities. SIAM J Appl Math, 1963, 11: 431–441

    Article  MathSciNet  MATH  Google Scholar 

  22. Mo J, Liu C, Yan S. A nonmonotone trust region method based on nonincreasing technique of weighted average of the successive function values. J Comput Appl Math, 2007, 209: 97–108

    Article  MathSciNet  MATH  Google Scholar 

  23. Moré J J. Recent developments in algorithms and software for trust region methods. In: Mathematical Programming. The State of the Art. Berlin: Springer-Verlag, 1983, 258–287

    Chapter  Google Scholar 

  24. Moré J J, Sorensen D C. Computing a trust region step. SIAM J Sci Stat Comp, 1983, 4: 553–572

    Article  MathSciNet  MATH  Google Scholar 

  25. Nocedal J, Wright S J. Numerical Optimization. New York: Springer, 1999

    Book  MATH  Google Scholar 

  26. Powell M J D. A new algorithm for unconstrained optimization. In: Rosen J B, Mangasarian O L, Ritter K, eds. Nonlinear Programming. New York: Academic Press, 1970, 31–65

    Chapter  Google Scholar 

  27. Powell M J D. Convergence properties of a class of minimization algorithms. In: Mangasarian O L, Meyer R R, Robinson S M, eds. Nonlinear Programming 2. New York: Academic Press, 1975, 1–27

    Chapter  Google Scholar 

  28. Raydan M. The Barzilai and Borwein gradient method for large scale unconstrained minimization problem. SIAM J Optim, 1997, 7: 26–33

    Article  MathSciNet  MATH  Google Scholar 

  29. Schubert L K. Modification of a quasi-Newton method for nonlinear equations with sparse Jacobian. Math Comput, 1970, 24: 27–30

    Article  MathSciNet  MATH  Google Scholar 

  30. Schultz G A, Schnabel R B, Byrd R H. A family of trust-region-based algorithms for unconstrained minimization with strong global convergence properties. SIAM J Numer Anal, 1985, 22: 47–67

    Article  MathSciNet  MATH  Google Scholar 

  31. Sidi A. A new approach to vector-valued rational interpolation. J Approx Theory, 2004, 130: 177–187

    Article  MathSciNet  MATH  Google Scholar 

  32. Sun W. Optimization methods for non-quadratic model. Asia Pacific J Oper Res, 1996, 13: 43–63

    MathSciNet  MATH  Google Scholar 

  33. Sun W. Non-monotone trust region method for optimization. Appl Math Comput, 2004, 156: 159–174

    Article  MathSciNet  MATH  Google Scholar 

  34. Sun W, Du Q, Chen J. Computational Methods. Beijing: Higher Education Press, 2007

    Google Scholar 

  35. Sun W, Hou L, Dang C. A modified trust region method with Beale’s PCG technique for optimization. Comput Optim Appl, 2008, 40: 59–62

    Article  MathSciNet  MATH  Google Scholar 

  36. Sun W, Yuan Y. Optimization Theory and Methods: Nonlinear Programming. New York: Springer, 2006

    MATH  Google Scholar 

  37. Toint P L. Global convergence of a class of trust region methods for nonconvex minimization in Hilbert space. IMA J Numer Anal, 1988, 8: 231–252

    Article  MathSciNet  MATH  Google Scholar 

  38. Toint P L. Towards an efficient sparsity exploiting Newton method for minimization. In: Sparse Matrices and Their Uses. New York: Academic Press, 1990, 57–87

    Google Scholar 

  39. Wang L, Sun W, Sampaio R J B, et al. A Barzilai and Borwein scaling conjugate gradient method for unconstrained optimization problems. Appl Math Comput 2015, 262: 136–144

    Article  MathSciNet  Google Scholar 

  40. Wang Z H, Yuan Y. A subspace implementation of quasi-Newton trust-region methods for unconstrained optimization. Numer Math, 2006, 104: 241–269

    Article  MathSciNet  MATH  Google Scholar 

  41. Winfield D. Function and functional optimization by interpolation in data tables. PhD thesis. Cambridge: Havard University, 1969

    Google Scholar 

  42. Winfield D. Function minimization by interpolation in a data table. J Inst Math Appl, 1973, 12: 339–347

    Article  MathSciNet  MATH  Google Scholar 

  43. Yuan Y. On a subproblem of trust-region algorithm for constrained optimization. Math Program, 1990, 47: 53–63

    Article  MATH  Google Scholar 

  44. Yuan Y. A modified BFGS algorithm for unconstrained optimization. IMA J Numer Anal, 1991, 11: 325–332

    Article  MathSciNet  MATH  Google Scholar 

  45. Zhang H C, Hager W W. A nonmonotone line search technique and its application to unconstrained optimization. SIAM J Optim, 2004, 14: 1043–1056

    Article  MathSciNet  MATH  Google Scholar 

  46. Zhang J Z, Xu C X. Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations. J Comput Appl Math, 2001, 137: 269–278

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang Y, Sun W, Qi L. A nomonotone filter Barzilai-Borwein method for optimization. Asia Pacific J Oper Res, 2010, 27: 55–69

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics and Physics, Jiangsu University of Technology, Changzhou, 213001, China

    QunYan Zhou

  2. School of Mathematical Sciences, Jiangsu Key Laboratory for NSLSCS, Nanjing Normal University, Nanjing, 210023, China

    WenYu Sun

  3. Department of Mathematics, Louisiana State University, Baton Rouge, LA, 70803-4918, USA

    HongChao Zhang

Authors
  1. QunYan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. WenYu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. HongChao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to WenYu Sun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Q., Sun, W. & Zhang, H. A new simple model trust-region method with generalized Barzilai-Borwein parameter for large-scale optimization. Sci. China Math. 59, 2265–2280 (2016). https://doi.org/10.1007/s11425-015-0734-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11425-015-0734-2

Keywords

MSC(2010)

Search

Navigation