Computational Optimization and Applications

, Volume 71, Issue 1, pp 251–271 | Cite as

An effective adaptive trust region algorithm for nonsmooth minimization

  • Zhou Sheng
  • Gonglin Yuan


In this paper, an adaptive trust region algorithm that uses Moreau–Yosida regularization is proposed for solving nonsmooth unconstrained optimization problems. The proposed algorithm combines a modified secant equation with the BFGS update formula and an adaptive trust region radius, and the new trust region radius utilizes not only the function information but also the gradient information. The global convergence and the local superlinear convergence of the proposed algorithm are proven under suitable conditions. Finally, the preliminary results from comparing the proposed algorithm with some existing algorithms using numerical experiments reveal that the proposed algorithm is quite promising for solving nonsmooth unconstrained optimization problems.


Nonsmooth problems Moreau–Yosida regularization Trust region algorithm Global convergence Superlinear convergence 

Mathematical subject classification

65K05 90C26 



The authors would like to thank the referees and the editor for their valuable comments that greatly improved this paper. This work is supported by the National Natural Science Foundation of China (11661009 and 11261006), the Guangxi Science Fund for Distinguished Young Scholars (No. 2015GXNSFGA139001), the Guangxi Natural Science Key Fund (No. 2017GXNSFDA198046), and the Basic Ability Promotion Project of Guangxi Young and Middle-aged Teachers (No. 2017KY0019).


  1. 1.
    Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific Publishing, Singapore (1992)CrossRefzbMATHGoogle Scholar
  2. 2.
    Bagirov, A., Karmitsa, N., Mäkelä, M.M.: Introduction to Nonsmooth Optimization: Theory, Practice and Software. Springer, Berlin (2014)CrossRefzbMATHGoogle Scholar
  3. 3.
    Li, J., Li, X., Yang, B., Sun, X.: Segmentation-based image copy-move forgery detection scheme. IEEE Trans. Inf. Forensics Secur. 10, 507–518 (2015)CrossRefGoogle Scholar
  4. 4.
    Luks̆an, L., Vlc̆ek, J.: A bundle-Newton method for nonsmooth unconstrained minimization. Math. Program. 83, 373–391 (1998)MathSciNetGoogle Scholar
  5. 5.
    Mäkelä, M.M.: Survey of bundle methods for nonsmooth optimization. Optim. Methods Softw. 17, 1–29 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Shor, N.Z.: Minimization Methods for Non-differentiable Functions. Springer, Berlin (2012)Google Scholar
  7. 7.
    Polak, E., Royset, J.O.: Algorithms for finite and semi-infinite min-max-min problems using adaptive smoothing techniques. J. Optim. Theory Appl. 119, 421–457 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Grapiglia, G.N., Yuan, J., Yuan, Y.: A derivative-free trust-region algorithm for composite nonsmooth optimization. Comput. Appl. Math. 32, 1–25 (2014)zbMATHGoogle Scholar
  9. 9.
    Fukushima, M., Qi, L.: A globally and superlinearly convergent algorithm for nonsmooth convex minimization. SIAM J. Optim. 6, 1106–1120 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Sagara, N., Fukushima, M.: A trust region method for nonsmooth convex optimization. J. Ind. Manag. Optim. 1, 171–180 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Zhang, L.: A new trust region algorithm for nonsmooth convex minimization. Appl. Math. Comput. 193, 135–142 (2007)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Yuan, G., Wei, Z.: The Barzilai and Borwein gradient method with nonmonotone line search for nonsmooth convex optimization problems. Math. Model. Anal. 17, 203–216 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Yuan, G., Wei, Z., Wang, Z.: Gradient trust region algorithm with limited memory BFGS update for nonsmooth convex minimization. Comput. Optim. Appl. 54, 45–64 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Cui, Z., Yuan, G., Sheng, Z., Liu, W., Wang, X., Duan, X.: A modified BFGS formula using a trust region model for nonsmooth convex minimizations. PLoS ONE 10, 1–15 (2015)Google Scholar
  15. 15.
    Ou, Y., Lin, H.: An ODE-like nonmonotone method for nonsmooth convex optimization. J. Appl. Math. Comput. 52, 1–21 (2015)MathSciNetGoogle Scholar
  16. 16.
    Yuan, G., Meng, Z., Li, Y.: A modified Hestenes and Stiefel conjugate gradient algorithm for large-scale nonsmooth minimizations and nonlinear equations. J. Optim. Theory Appl. 168, 129–152 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Yuan, G., Sheng, Z., Liu, W.: The modified HZ conjugate gradient algorithm for large-scale nonsmooth optimization. PLoS ONE 11, 1–15 (2016)Google Scholar
  18. 18.
    Yuan, G., Sheng, Z.: Nonsmooth Optimization Algorithms. Press of Science, Beijing (2017)Google Scholar
  19. 19.
    Qi, L., Sun, J.: A trust region algorithm for minimization of locally Lipschitzian functions. Math. Program. 66, 25–43 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Akbari, Z., Yousefpour, R., Peyghami, M.R.: A new nonsmooth trust region algorithm for locally Lipschitz unconstrained optimization problems. J. Optim. Theory Appl. 164, 733–754 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Rosen, J.B., Mangasarian, O.L., Ritter, K. (eds.) Nonlinear Programming, pp. 31–65. Academic Press (1970).Google Scholar
  22. 22.
    Zhang, X., Zhang, J., Liao, L.: An adaptive trust region method and its convergence. Sci. China Ser. A Math. 45, 620–631 (2002)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Cui, Z., Wu, B.: A new modified nonmonotone adaptive trust region method for unconstrained optimization. Comput. Optim. Appl. 53, 795–806 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Li, G.: A trust region method with automatic determination of the trust region radius. Chin. J. Eng. Math. 23, 843–848 (2006)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Shi, Z.J., Guo, J.H.: A new trust region method for unconstrained optimization. J. Comput. Appl. Math. 213, 509–520 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Shi, Z., Wang, S.: Nonmonotone adaptive trust region method. Eur. J. Oper. Res. 208, 28–36 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Cui, Z., Wu, B.: A new trust region method with adaptive radius for unconstrained optimization. Optim. Methods Softw. 27, 419–429 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Zhou, Q., Hang, D.: Nonmonotone adaptive trust region method with line search based on new diagonal updating. Appl. Numer. Math. 91, 75–88 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Kamandi, A., Amini, K., Ahookhosh, M.: An improved adaptive trust-region algorithm. Optim. Lett. 11, 1–15 (2016)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Esmaeili, H., Kimiaei, M.: A new adaptive trust-region method for system of nonlinear equations. Appl. Math. Model. 38, 3003–3015 (2014)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Kimiaei, M., Esmaeili, H.: A trust-region approach with novel filter adaptive radius for system of nonlinear equations. Numer. Algoritm. 73, 1–18 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Program. 58, 353–367 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM, Bangkok (1983)zbMATHGoogle Scholar
  34. 34.
    Pang, J.S., Qi, L.: A globally convergent Newton method for convex \(SC^1\) minimization problems. J. Optim. Theory Appl. 85, 633–648 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II: Advanced Theory and Bundle Methods, vol. 306 of Grundlehren der Mathematischen Wissenschaften. Springer, Berlin (1993)CrossRefzbMATHGoogle Scholar
  36. 36.
    Qi, L.: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 18, 227–244 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Fukushima, M.: A descent algorithm for nonsmooth convex optimization. Math. Program. 30, 163–175 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Auslender, A.: Numerical Methods for Nondifferentiable Convex Optimization. Nonlinear Analysis and Optimization, pp. 102–126. Springer, Berlin (1987)zbMATHGoogle Scholar
  39. 39.
    Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62, 261–275 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Rauf, A.I., Fukushima, M.: Globally convergent BFGS method for nonsmooth convex optimization. J. Optim. Theory Appl. 104, 539–558 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Wei, Z., Li, G., Qi, L.: New quasi-Newton methods for unconstrained optimization problems. Appl. Math. Comput. 175, 1156–1188 (2006)MathSciNetzbMATHGoogle Scholar
  42. 42.
    Yuan, G., Wei, Z., Li, G.: A modified Polak–Ribière–Polyak conjugate gradient algorithm for nonsmooth convex programs. J. Comput. Appl. Math. 255, 86–96 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Yuan, G., Wei, Z., Lu, X.: A BFGS trust-region method for nonlinear equations. Computing 92, 317–333 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Powell, M.J.D.: Convergence properties of a class of minimization algorithms. Nonlinear Program. 2, 1–27 (1975)Google Scholar
  45. 45.
    Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Methods. SIAM, Bangkok (2000)CrossRefzbMATHGoogle Scholar
  46. 46.
    Steihaug, T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–637 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Di Pillo, G., Grippo, L., Lucidi, S.: A smooth method for the finite minimax problem. Math. Program. 60, 187–214 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Bagirov, A.: A method for minimization of quasidifferentiable functions. Optim. Methods Softw. 17, 31–60 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Mathematics and Information ScienceGuangxi UniversityNanningPeople’s Republic of China

Personalised recommendations