Skip to main content
SpringerLink
Log in

New proximal bundle algorithm based on the gradient sampling method for nonsmooth nonconvex optimization with exact and inexact information

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

In this paper, we focus on a descent algorithm for solving nonsmooth nonconvex optimization problems. The proposed method is based on the proximal bundle algorithm and the gradient sampling method and uses the advantages of both. In addition, this algorithm has the ability to handle inexact information, which creates additional challenges. The global convergence is proved with probability one. More precisely, every accumulation point of the sequence of serious iterates is either a stationary point if exact values of gradient are provided or an approximate stationary point if only inexact information of the function and gradient values is available. The performance of the proposed algorithm is demonstrated using some academic test problems. We further compare the new method with a general nonlinear solver and two other methods specifically designed for nonconvex nonsmooth optimization problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

Data availability not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Bagirov, A.M.: Continuous subdifferential approximations and their applications. J. Math. Sci., 115, 2567–2609 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bagirov, A.M., Hoseini Monjezi, N., Taheri, S.: An augmented subgradient method for minimizing nonsmooth DC functions. Comput. Optim. Appl. 80, 411–438 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bagirov, A.M., Jin, L., Karmitsa, N., Nuaimat, A.Al., Sultanova N.: Subgradient method for nonconvex nonsmooth optimization. J. Optim. Theory Appl., 157, 416–435 (2013)

    MathSciNet  Google Scholar 

  4. Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: Introduction to Nonsmooth Optimization: theory, practice and software. Springer (2014)

    Book  MATH  Google Scholar 

  5. Bagirov, A.M., Taheri, S., Joki, K., Karmitsa, N., Mäkelä, M.M.: Aggregate subgradient method for nonsmooth DC optimization. Optim. Lett., 15, 83–96 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  6. Burke, J., Lewis, A., Overton, M.: A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J. Optim., 15, 571–779 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  7. Curtis, F.E., Overton, M.L.: A sequential quadratic programming algorithm for nonconvex, nonsmooth constrained optimization. SIAM J. Optim., 22(2), 474–500 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. de Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Convex proximal bundle methods in depth: a unified analysis for inexact oracles. Math. Program., 148(1–2), 241–277 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  9. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program., 91(2), 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  10. Ferrier, C.: Bornes Duales de Problémes d’Optimisation Polynomiaux, Ph.D. thesis, Laboratoire Approximation et Optimisation, Université Paul Sabatier, France (1997)

    Google Scholar 

  11. Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim., 20, 2442–2473 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  12. Hare, W., Sagastizábal, C., Solodov, M.: A proximal bundle method for nonsmooth nonconvex functions with inexact information. Comput. Optim. Appl., 63, 1–28 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  13. Hintermüller, M.: A proximal bundle method based on approximate subgradients. Comput. Optim. Appl., 20, 245–266 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hoseini, N., Nobakhtian, S.: A new trust region method for nonsmooth nonconvex optimization. Optimization, 67, 1265–1286 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hoseini Monjezi, N., Nobakhtian, S.: A filter proximal bundle method for nonsmooth nonconvex constrained optimization. J. Glob. Opti., 79, 1–37 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  16. Hoseini Monjezi, N., Nobakhtian, S.: A new infeasible proximal bundle algorithm for nonsmooth nonconvex constrained optimization. Comput. Optim. Appl., 74(2), 443–480 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  17. Hoseini Monjezi, N., Nobakhtian, S.: A proximal bundle-based algorithm for nonsmooth constrained multiobjective optimization problems with inexact data. Numer. Algor., 89, 637–674 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hoseini Monjezi, N., Nobakhtian, S.: An inexact multiple proximal bundle algorithm for nonsmooth nonconvex multiobjective optimization problems. Ann. Oper. Res., 311, 1123–1154 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hoseini Monjezi, N., Nobakhtian, S.: Convergence of the proximal bundle algorithm for nonsmooth nonconvex optimization problems. Optim. Lett., 16, 1495–1511 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  20. Hoseini Monjezi, N., Nobakhtian, S. Pouryayevali, M.R.: Proximal bundle algorithm for nonsmooth optimization on riemannian manifolds, IMA J. Numer. Anal., 43(1), 293–325 (2023)

  21. Kiwiel, K.C.: Approximations in proximal bundle methods and decomposition of convex programs. J. Optim. Theory Appl., 84, 529–548 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kiwiel, K.C.: Convergence of approximate and incremental subgradient methods for convex optimization. SIAM. J. Optim., 14(3), 807–840 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  23. Kiwiel, K.C.: Convergence of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim., 18, 379–388 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  24. Kiwiel, K.C.: Efficiency of Proximal Bundle Methods. J. Optim. Theory Appl., 104, 589–603 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  25. Lemaréchal, C.: Bundle methods in nonsmooth optimization, in: Nonsmooth Optimization (Laxenburg, 1977), Lemaréchal C., Mifflin, R. (eds.), IIASA Proc. Ser. 3, Pergamon Press, Oxford, 79–102 (1978)

  26. Lewis, A.S., Overton, M.L.: Nonsmooth Optimization via Quasi-Newton Methods. Math. Program., 141(1–2), 135–163 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  27. Lv, J., Pang, LP., Xu, N., Xiao, Z.H.: An infeasible bundle method for nonconvex constrained optimization with application to semi-infinite programming problems. Numer. Algor., 80, 397–427 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  28. Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific, Singapore (1992)

    Book  MATH  Google Scholar 

  29. Nesterov, Y.: Primal-dual subgradient methods for convex problems. Math. Program., 120(1), 221–259 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  30. Noll, D.: Bundle method for non-convex minimization with inexact subgradients and function values. In: Computational and Analytical Mathematics, vol. 50, pp. 555–592. Springer Proceedings in Mathematics (2013)

  31. Noll, D.: Cutting plane oracles to minimize non-smooth non-convex functions. Set-Valued Var. Anal., 18, 531–568 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  32. Pang, LP., Wu, Q.: A feasible proximal bundle algorithm with convexification for nonsmooth, nonconvex semi-infinite programming. Numer. Algor., 90, 387–422 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  33. Qi, L., Sun, J.: A trust region algorithm for minimization of locally Lipschitzian functions. Math. Program., 66, 25–43 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  34. Shor, N.Z.: Minimization methods for non-differentiable functions. Springer (1985)

    Book  MATH  Google Scholar 

  35. Solodov, M.V., Zavriev, S.K.: Error stability properties of generalized gradient-type algorithms. J. Optim. Theory Appl., 98, 663–680 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  36. Tang, C., Liu, S., Jian, J., Li. J.: A feasible SQP-GS algorithm for nonconvex, nonsmooth constrained optimization. Numer. Algor., 65, 1–22 (2014)

  37. Yang, Y., Pang, L., Ma, X., Shen, J.: Constrained Nonconvex Nonsmooth Optimization via Proximal Bundle Method. J. Optim. Theory Appl., 163, 900–925 (2014)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank anonymous referees for their comments that helped to improve the quality of the paper.

Funding

The research was supported by a grant from IPM.

Author information

Authors and Affiliations

  1. School of Mathematics, Institute for Research in Fundamental Sciences (IPM), P.O. Box: 19395-5746, Tehran, Iran

    N. Hoseini Monjezi

  2. Department of Applied Mathematics and Computer Science, Faculty of Mathematics and Statistics, University of Isfahan, Isfahan, Iran

    S. Nobakhtian

Authors
  1. N. Hoseini Monjezi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Nobakhtian
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

N. Hoseini Monjezi and S. Nobakhtian contributed equally to drafting this manuscript (material preparation and data collection, design, analysis and implementation of the algorithm, writing and editing the first draft of the manuscript). Both authors read and approved the final manuscript.

Corresponding author

Correspondence to S. Nobakhtian.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Human and animal ethics

Not applicable

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoseini Monjezi, N., Nobakhtian, S. New proximal bundle algorithm based on the gradient sampling method for nonsmooth nonconvex optimization with exact and inexact information. Numer Algor 94, 765–787 (2023). https://doi.org/10.1007/s11075-023-01519-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-023-01519-8

Keywords

Mathematics Subject Classification (2010)

Search

Navigation