Mathematical Programming

, Volume 109, Issue 1, pp 181–205 | Cite as

Globally convergent limited memory bundle method for large-scale nonsmooth optimization

  • Napsu Haarala
  • Kaisa Miettinen
  • Marko M. Mäkelä
Article

Abstract

Many practical optimization problems involve nonsmooth (that is, not necessarily differentiable) functions of thousands of variables. In the paper [Haarala, Miettinen, Mäkelä, Optimization Methods and Software, 19, (2004), pp. 673–692] we have described an efficient method for large-scale nonsmooth optimization. In this paper, we introduce a new variant of this method and prove its global convergence for locally Lipschitz continuous objective functions, which are not necessarily differentiable or convex. In addition, we give some encouraging results from numerical experiments.

Keywords

Nondifferentiable programming Large-scale optimization Bundle methods Variable metric methods Limited memory methods Global convergence 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bacaud, L., Lemaréchal, C., Renaud, A., Sagastizábal, C.: Bundle methods in stochastic optimal power management: A disaggregated approach using preconditioners. Comput. Optim. Appl. 20 (3), 227–244 (2001)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bihain, A.: Optimization of upper semidifferentiable functions. J. Optimization Theory Appl. 4, 545–568 (1984)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasi-Newton matrices and their use in limited memory methods. Math. Program. 63, 129–156 (1994)MATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley-Interscience, New York, 1983Google Scholar
  5. 5.
    Clarke, F.H., Ledyaev, Y.S., Stern, R.J., Wolenski, P.R.: Nonsmooth Analysis and Control Theory. Springer, New York, 1998Google Scholar
  6. 6.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MATHMathSciNetCrossRefGoogle Scholar
  7. 7.
    Fletcher, R.: Practical Methods of Optimization, 2nd edn. John Wiley and Sons, Chichester, 1987Google Scholar
  8. 8.
    Gilbert, J.C., Lemaréchal, C.: Some numerical experiments with variable-storage quasi-Newton algorithms. Math. Program. 45, 407–435 (1989)MATHCrossRefGoogle Scholar
  9. 9.
    Gröwe-Kuska, N., Kiwiel, K.C., Nowak, M.P., Römisch, W., Wegner, I.: Power management in a hydro-thermal system under uncertainty by Lagrangian relaxation. In: C. Greengard, A. Ruszczynski (eds.) Decision Making under Uncertainty: Energy and Power, IMA Volumes in Mathematics and its Applications, vol. 128, Springer, New York, 2002, pp. 39–70Google Scholar
  10. 10.
    Haarala, M.: Large-scale nonsmooth optimization: Variable metric bundle method with limited memory. Ph.D. thesis, University of Jyväskylä, Department of Mathematical Information Technology (2004)Google Scholar
  11. 11.
    Haarala, M., Miettinen, K., Mäkelä, M.M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. 19 (6), 673–692 (2004)MathSciNetGoogle Scholar
  12. 12.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II. Springer-Verlag, Berlin, 1993Google Scholar
  13. 13.
    Kärkkäinen, T., Majava, K., Mäkelä, M.M.: Comparison of formulations and solution methods for image restoration problems. Reports of the Department of Mathematical Information Technology, Series B. Scientific Computing, B 14/2000 University of Jyväskylä, Jyväskylä (2000)Google Scholar
  14. 14.
    Kärkkäinen, T., Majava, K., Mäkelä, M.M.: Comparison of formulations and solution methods for image restoration problems. Inverse Probl. 17 (6), 1977–1995 (2001)Google Scholar
  15. 15.
    Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics 1133. Springer-Verlag, Berlin, 1985Google Scholar
  16. 16.
    Kiwiel, K.C.: A method for solving certain quadratic programming problems arising in nonsmooth optimization. IMA J. Numer. Anal. 6, 137–152 (1986)MATHMathSciNetGoogle Scholar
  17. 17.
    Lemaréchal, C.: Nondifferentiable optimization. In: G.L. Nemhauser, A.H.G. Rinnooy Kan, M.J. Todd (eds.) Optimization. Elsevier North-Holland, Inc., New York, 1989, pp. 529–572Google Scholar
  18. 18.
    Lemaréchal, C., Strodiot, J.J., Bihain, A.: On a bundle algorithm for nonsmooth optimization. In: O.L. Mangasarian, R.R. Mayer, S.M. Robinson (eds.) Nonlinear Programming. Academic Press, New York, 1981, pp. 285–281Google Scholar
  19. 19.
    Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989)MATHCrossRefGoogle Scholar
  20. 20.
    Lukšan, L., Vlček, J.: Globally convergent variable metric method for convex nonsmooth unconstrained minimization. J. Optimization Theory Appl. 102, 593–613 (1999)CrossRefGoogle Scholar
  21. 21.
    Mäkelä, M.M.: Issues of implementing a Fortran subroutine package NSOLIB for nonsmooth optimization. Technical Report 5/1993, Department of Mathematics, Laboratory of Scientific Computing, University of Jyväskylä (1993)Google Scholar
  22. 22.
    Mäkelä, M.M.: Survey of bundle methods for nonsmooth optimization. Optim. Methods Softw. 17 (1), 1–29 (2002)Google Scholar
  23. 23.
    Mäkelä, M.M., Miettinen, M., Lukšan, L., Vlček, J.: Comparing nonsmooth nonconvex bundle methods in solving hemivariational inequalities. J. Glob. Optim. 14, 117–135 (1999)MATHCrossRefGoogle Scholar
  24. 24.
    Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific Publishing Co., Singapore, 1992Google Scholar
  25. 25.
    Mifflin, R.: A modification and an extension of Lemaréchal's algorithm for nonsmooth minimization. Math. Program. Study 17, 77–90 (1982)MATHMathSciNetGoogle Scholar
  26. 26.
    Mistakidis, E.S., Stavroulakis, G.E.: Nonconvex Optimization in Mechanics. Smooth and Nonsmooth Algorithms, Heuristics and Engineering Applications by the F.E.M. Kluwert Academic Publishers, Dordrecht, 1998Google Scholar
  27. 27.
    Moreau, J.J., Panagiotopoulos, P.D., Strang, G. (eds.): Topics in Nonsmooth Mechanics. Birkhäuser Verlag, Basel, 1988Google Scholar
  28. 28.
    Naniewicz, Z., Panagiotopoulos, P.D.: Mathematical Theory of Hemivariational Inequalities and Applications. Marcel Dekker, New York, 1995Google Scholar
  29. 29.
    Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35 (151), 773–782 (1980)MathSciNetGoogle Scholar
  30. 30.
    Nowak, M.P., Nürnberg, R., Römisch, W., Schultz, R., Westphalen, M.: Stochastic programming for power production and trading under uncertainty. In: W. Jäger, H.J. Krebs (eds.) Mathematics –- Key Technology for the Future. Springer, Berlin, 2003, pp. 623–636Google Scholar
  31. 31.
    Outrata, J., Kočvara, M., Zowe, J.: Nonsmooth Approach to Optimization Problems With Equilibrium Constraints. Theory, Applications and Numerical Results. Kluwert Academic Publisher, Dordrecht, 1998Google Scholar
  32. 32.
    Panagiotopoulos, P.D.: Hemivariational Inequalities. Springer-Verlag, New York, 1993Google Scholar
  33. 33.
    Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: Conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2 (1), 121–152 (1992)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Vlček, J., Lukšan, L.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J. Optimization Theory Appl. 111 (2), 407–430 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Napsu Haarala
    • 1
  • Kaisa Miettinen
    • 2
  • Marko M. Mäkelä
    • 3
  1. 1.School of Computational & Applied MathematicsUniversity of the WitwatersrandJohannesburgSouth Africa
  2. 2.Helsinki School of EconomicsHelsinkiFinland
  3. 3.Department of Mathematical Information TechnologyUniversity of JyväskyläFinland

Personalised recommendations