Advertisement

Mathematical Programming

, Volume 156, Issue 1–2, pp 125–159 | Cite as

A doubly stabilized bundle method for nonsmooth convex optimization

  • Welington de OliveiraEmail author
  • Mikhail Solodov
Full Length Paper Series A

Abstract

We propose a bundle method for minimizing nonsmooth convex functions that combines both the level and the proximal stabilizations. Most bundle algorithms use a cutting-plane model of the objective function to formulate a subproblem whose solution gives the next iterate. Proximal bundle methods employ the model in the objective function of the subproblem, while level methods put the model in the subproblem’s constraints. The proposed algorithm defines new iterates by solving a subproblem that employs the model in both the objective function and in the constraints. One advantage when compared to the proximal approach is that the level set constraint provides a certain Lagrange multiplier, which is used to update the proximal parameter in a novel manner. We also show that in the case of inexact function and subgradient evaluations, no additional procedure needs to be performed by our variant to deal with inexactness (as opposed to the proximal bundle methods that require special modifications). Numerical experiments on almost 1,000 instances of different types of problems are presented. Our experiments show that the doubly stabilized bundle method inherits useful features of the level and the proximal versions, and compares favorably to both of them.

Keywords

Nonsmooth optimization Proximal bundle method  Level bundle method Inexact oracle 

Mathematics Subject Classification

90C25 90C30 65K05 

Notes

Acknowledgments

The authors would like to acknowledge helpful comments of Claudia Sagastizábal. The authors also thank the anonymous referees for constructive suggestions that considerably improved the original version of this article.

References

  1. 1.
    Astorino, A., Frangioni, A., Gaudioso, M., Gorgone, E.: Piecewise-quadratic approximations in convex numerical optimization. SIAM J. Optim. 21(4), 1418–1438 (2011). doi: 10.1137/100817930 CrossRefMathSciNetzbMATHGoogle Scholar
  2. 2.
    Bello-Cruz, J.Y., de Oliveria, W.: Level bundle-like algorithms for convex optimization. J. Glob. Optim. 59(4), 787–809 (2014). doi: 10.1007/s10898-013-0096-4 Google Scholar
  3. 3.
    Ben-Tal, A., Nemirovski, A.: Non-euclidean restricted memory level method for large-scale convex optimization. Math. Program. 102, 407–456 (2005). doi: 10.1007/s10107-004-0553-4. http://dl.acm.org/citation.cfm?id=1057781.1057789
  4. 4.
    Bonnans, J., Gilbert, J., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization. Theoretical and Practical Aspects. Universitext. Springer, Berlin (2006). Second edition, xiv+490 ppGoogle Scholar
  5. 5.
    Brannlund, U., Kiwiel, K.C., Lindberg, P.O.: A descent proximal level bundle method for convex nondifferentiable optimization. Oper. Res. Lett. 17(3), 121–126 (1995). doi: 10.1016/0167-6377(94)00056-C. http://www.sciencedirect.com/science/article/pii/016763779400056C
  6. 6.
    Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(2), 261–275 (1993)CrossRefzbMATHGoogle Scholar
  7. 7.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002). doi: 10.1007/s101070100263 CrossRefMathSciNetzbMATHGoogle Scholar
  8. 8.
    de Oliveira, W.: Combining level and proximal bundle methods for convex optimization in energy problems. In: 3rd International Conference on Engineering Optimization—EngOpt, Rio de Janeiro, Mathematical Optimization Techniques, vol. 404, pp. 1–10 (2012). http://www.engopt.org/authors/404.html
  9. 9.
    de Oliveira, W., Sagastizábal, C.: Level bundle methods for oracles with on-demand accuracy. Optim. Methods Softw. 29(6), 1180–1209 (2014)CrossRefMathSciNetzbMATHGoogle Scholar
  10. 10.
    de Oliveira, W., Sagastizábal, C., Scheimberg, S.: Inexact bundle methods for two-stage stochastic programming. SIAM J. Optim. 21(2), 517–544 (2011). doi: 10.1137/100808289 CrossRefMathSciNetzbMATHGoogle Scholar
  11. 11.
    de Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Convex bundle methods in depth: a unified analysis for inexact oracles. Math. Program. 148(1–2), 241–277 (2014)CrossRefMathSciNetzbMATHGoogle Scholar
  12. 12.
    Fábián, C.: Bundle-type methods for inexact data. Cent. Eur. J. Oper. Res. 8, 35–55 (2000)zbMATHGoogle Scholar
  13. 13.
    Frangioni, A.: Generalized bundle methods. SIAM J. Optim. 13(1), 117–156 (2002). doi: 10.1137/S1052623498342186 CrossRefMathSciNetzbMATHGoogle Scholar
  14. 14.
    Frangioni, A., Gorgone, E.: Bundle methods for sum-functions with ‘easy’ components: applications to multicommodity network design. Math. Program. 145(1–2), 133–161 (2014)CrossRefMathSciNetzbMATHGoogle Scholar
  15. 15.
    Goffin, J.L., Haurie, A., Vial, J.P.: Decomposition and nondifferentiable optimization with the projective algorithm. Manag. Sci. 38(2), 284–302 (1992). http://EconPapers.repec.org/RePEc:inm:ormnsc:v:38:y:1992:i:2:p:284-302
  16. 16.
    Goffin, J.L., Vial, J.P.: Interior points methods for nondifferentiable optimization. In: Proceedings of the Operations Research 1997 (Jena), pp. 35–49 (1998). http://EconPapers.repec.org/RePEc:fth:ehecge:97.24
  17. 17.
    Hintermüller, M.: A proximal bundle method based on approximate subgradients. Comput. Optim. Appl. 20, 245–266 (2001)CrossRefMathSciNetzbMATHGoogle Scholar
  18. 18.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. No. 305–306 in Grund. der math. Wiss (two volumes). Springer, Berlin (1993)Google Scholar
  19. 19.
    Kiwiel, K.C.: Proximity control in bundle methods for convex nondiferentiable minimization. Math. Program. 46, 105–122 (1990)CrossRefMathSciNetzbMATHGoogle Scholar
  20. 20.
    Kiwiel, K.C.: Proximal level bubdle methods for convex nondiferentiable optimization, saddle-point problems and variational inequalities. Math. Program. 69(1), 89–109 (1995). doi: 10.1007/BF01585554 MathSciNetzbMATHGoogle Scholar
  21. 21.
    Kiwiel, K.C.: A proximal bundle method with approximate subgradient linearizations. SIAM J. Optim. 16(4), 1007–1023 (2006)CrossRefMathSciNetzbMATHGoogle Scholar
  22. 22.
    Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69(1), 111–147 (1995). doi: 10.1007/BF01585555 CrossRefzbMATHGoogle Scholar
  23. 23.
    Lemaréchal, C., Sagastizábal, C.: Variable metric bundle methods: from conceptual to implementable forms. Math. Program. 76, 393–410 (1997)CrossRefzbMATHGoogle Scholar
  24. 24.
    Sagastizábal, C.: Composite proximal bundle method. Math. Program. 140(1), 189–233 (2013)CrossRefMathSciNetzbMATHGoogle Scholar
  25. 25.
    Sagastizábal, C., Solodov, M.: An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter. SIAM J. Optim. 16(1), 146–169 (2005). doi: 10.1137/040603875 CrossRefMathSciNetzbMATHGoogle Scholar
  26. 26.
    Solodov, M.: On approximations with finite precision in bundle methods for nonsmooth optimization. J. Optim. Theory Appl. 119(1), 151–165 (2003). doi: 10.1023/B:JOTA.0000005046.70410.02 CrossRefMathSciNetzbMATHGoogle Scholar
  27. 27.
    van Ackooij, W., de Oliveira, W.: Level bundle methods for constrained convex optimization with various oracles. Comput. Optim. Appl. 57(3), 555–597 (2014). doi: 10.1007/s10589-013-9610-3 CrossRefMathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2015

Authors and Affiliations

  1. 1.IMPA – Instituto de Matemática Pura e AplicadaRio de JaneiroBrazil

Personalised recommendations