Advertisement

Journal of Global Optimization

, Volume 59, Issue 4, pp 787–809 | Cite as

Level bundle-like algorithms for convex optimization

  • J. Y. Bello Cruz
  • W. de Oliveira
Article

Abstract

We propose two restricted memory level bundle-like algorithms for minimizing a convex function over a convex set. If the memory is restricted to one linearization of the objective function, then both algorithms are variations of the projected subgradient method. The first algorithm, proposed in Hilbert space, is a conceptual one. It is shown to be strongly convergent to the solution that lies closest to the initial iterate. Furthermore, the entire sequence of iterates generated by the algorithm is contained in a ball with diameter equal to the distance between the initial point and the solution set. The second algorithm is an implementable version. It mimics as much as possible the conceptual one in order to resemble convergence properties. The implementable algorithm is validated by numerical results on several two-stage stochastic linear programs.

Keywords

Convex minimization Nonsmooth optimization Level bundle method Strong convergence 

Notes

Acknowledgments

The authors are grateful to the reviewers for their insightful comments and remarks. Research for this paper was partially supported by PROCAD-nf—UFG/UnB/IMPA research and PRONEX—CNPq-FAPERJ—Optimization research, and by project CAPES-MES-CUBA 226/2012 “Modelos de Otimização e Aplicações ”. The first author thanks CNPq and the Institute for Pure and Applied Mathematics (IMPA), in Rio de Janeiro, Brazil, where he was a visitor while working on this paper.

References

  1. 1.
    Alber, Y.I., Iusem, A.N.: Extension of subgradient techniques for nonsmooth optimization in banach spaces. Set Valued Anal. 9, 315–335 (2001). doi: 10.1023/A:1012665832688 CrossRefGoogle Scholar
  2. 2.
    Alber, Y.I., Iusem, A.N., Solodov M.V.: On the projected subgradient method for nonsmooth convex optimization in a Hilbert space. Math. Program. 81, 23–35 (1998). doi: 10.1007/BF01584842 Google Scholar
  3. 3.
    Bazaraa, M.S., Sherali, H.D.: On the choice of step size in subgradient optimization. Eur. J. Oper. Res. 7, 380–388 (1981)CrossRefGoogle Scholar
  4. 4.
    Bello Cruz, J.Y., Iusem, A.N.: A strongly convergent method for nonsmooth convex minimization in hilbert spaces. Numer. Funct. Anal. Optim. 32, 1009–1018 (2011)CrossRefGoogle Scholar
  5. 5.
    Ben-Tal, A., Nemirovski, A.: Non-euclidean restricted memory level method for large-scale convex optimization. Math. Program. 102, 407–456 (2005)CrossRefGoogle Scholar
  6. 6.
    Brannlund, U., Kiwiel, K.C., Lindberg, P.O.: A descent proximal level bundle method for convex nondifferentiable optimization. Oper. Res. Lett. 17, 121–126 (1995)CrossRefGoogle Scholar
  7. 7.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)CrossRefGoogle Scholar
  8. 8.
    Fábián, C.: Bundle-type methods for inexact data. Central Eur. J. Oper. Res. 8, 35–55 (special issue, T. Csendes and T. Rapcsák, eds.) (2000)Google Scholar
  9. 9.
    Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. no. 305-306 in Grund. der math. Wiss, Springer (two volumes) (1993)Google Scholar
  10. 10.
    Kim, S., Ahn, H., Cho, S.-C.: Variable target value subgradient method. Math. Program. 49, 359–369 (1990). doi: 10.1007/BF01588797 CrossRefGoogle Scholar
  11. 11.
    Kiwiel, K.C.: Proximal level bubdle methods for convex nondiferentiable optimization, saddle-point problems and variational inequalities. Math. Program. 69, 89–109 (1995)Google Scholar
  12. 12.
    Lan, G.: Bundle-type methods uniformly optimal for smooth and nonsmooth convex optimization. Technical report 2796, University of Florida, Departament of Industrial and Systems Engineering, November 2010. Optimization OnlineGoogle Scholar
  13. 13.
    Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69, 111–147 (1995)CrossRefGoogle Scholar
  14. 14.
    Mak, W.-K., Morton, D., Wood, R.: Monte Carlo bounding techniques for determining solution quality in stochastic programs. Oper. Res. Lett. 24, 47–56 (1999) cited By (since 1996)140Google Scholar
  15. 15.
    Nesterov, Y.: Subgradient methods for huge-scale optimization problems. Math. Program., 1–23 (2013). doi: 10.1007/s10107-013-0686-4
  16. 16.
    Norkin, V., Pflug, G., Ruszczyński, A.: A branch and bound method for stochastic global optimization. Math. Program. Ser B 83, 425–450 (1998) cited By (since 1996)89Google Scholar
  17. 17.
    Oliveira, W., Sagastizábal, C.: Level bundle methods for oracles with on-demand accuracy. Preprint, Instituto Nacional de Matemática Pura e Aplicada. http://www.optimization-online.org/DB_HTML/2012/03/3390.html (2012)
  18. 18.
    Oliveira, W., Sagastizábal, C., Scheimberg, S.: Inexact bundle methods for two-stage stochastic programming. SIAM J. Optim. 21, 517–544 (2011)CrossRefGoogle Scholar
  19. 19.
    Polyak, B.: A general method for solving extremal problems. Soviet Mathematics Doklady 8, 593–597 (1967)Google Scholar
  20. 20.
    Polyak, B.: Minimization of unsmooth functionals. USSR Comput. Math. Math. Phys. 9, 14–29 (1969)CrossRefGoogle Scholar
  21. 21.
    Richtárik, P.: Approximate level method for nonsmooth convex minimization. J. Optim. Theory Appl. 152, 334–350 (2012) cited By (since 1996)1Google Scholar
  22. 22.
    Shapiro, A.: Monte Carlo sampling methods. Handbooks Oper. Res. Manag. Sci. 10, 353–425 (2003) cited By (since 1996)78Google Scholar
  23. 23.
    Shapiro, A., Dentcheva, D., Ruszczyński, A.: Lectures on Stochastic Programming, Modeling and Theory. MPS-SIAM Series on Optimization, SIAM—Society for Industrial and Applied Mathematics and Mathematical Programming Society, Philadelphia (2009)Google Scholar
  24. 24.
    Shor, N.Z.: Minimization methods for non-differentiable functions. ZAMM J. Appl. Math. Mech. 66, 575–575 (1986)Google Scholar
  25. 25.
    Zverovich, V., Fábián, C., Ellison, E., Mitra, G.: A computational study of a solver system for processing two-stage stochastic lps with enhanced benders decomposition. Math. Program. Comput. 4, 211–238 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Instituto de Matemática e EstatísticaUniversidade Federal de GoiásGoiâniaBrazil
  2. 2.Instituto Nacional de Matemática Pura e AplicadaRio de JaneiroBrazil

Personalised recommendations