Skip to main content
SpringerLink
Log in

Composite proximal bundle method

  • Full Length Paper
  • Series B
  • Published:
Mathematical Programming Submit manuscript

Abstract

We consider minimization of nonsmooth functions which can be represented as the composition of a positively homogeneous convex function and a smooth mapping. This is a sufficiently rich class that includes max-functions, largest eigenvalue functions, and norm-1 regularized functions. The bundle method uses an oracle that is able to compute separately the function and subgradient information for the convex function, and the function and derivatives for the smooth mapping. With this information, it is possible to solve approximately certain proximal linearized subproblems in which the smooth mapping is replaced by its Taylor-series linearization around the current serious step. Our numerical results show the good performance of the Composite Bundle method for a large class of problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Bonnans, J., Gilbert, J., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization. Theoretical and Practical Aspects. Universitext, 2nd edn, pp xiv+423. Springer, Berlin (2006)

  2. Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer Series in Operations Research, Springer, New York (2000)

  3. Burke, J., Lewis, A., Overton, M.: Two numerical methods for optimizing matrix stability. Linear Algebra Appl. 351–352, 117–145 (2002)

    Article  MathSciNet  Google Scholar 

  4. Burke, J.V., Ferris, M.C.: A Gauss-Newton method for convex composite optimization. Math. Program. 71, 179–194 (1995). doi:10.1007/BF01585997

    MathSciNet  MATH  Google Scholar 

  5. Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(2), 261–275 (1993)

    Article  MATH  Google Scholar 

  6. Daniilidis, A., Sagastizábal, C., Solodov, M.: Identifying structure of nonsmooth convex functions by the bundle technique. SIAM J. Optim. 20(2), 820–840 (2009). doi:10.1137/080729864. http://link.aip.org/link/?SJE/20/820/1

    Google Scholar 

  7. Emiel, G., Sagastizábal, C.: Incremental-like bundle methods with application to energy planning. Comput. Optim. Appl. 46, 305–332 (2010). doi:10.1007/s10589-009-9288-8

    Article  MathSciNet  MATH  Google Scholar 

  8. Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, Chichester (1987)

    MATH  Google Scholar 

  9. Haarala, M., Miettinen, K., Makela, M.M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. (6), 673–692 (2004). http://www.informaworld.com/10.1080/10556780410001689225

  10. Hare, W.: Nonsmoth Optimization with Smooth Substructure. Ph.D. thesis, Department of Mathematics, Simon Fraser University (2003)

  11. Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010). doi:10.1137/090754595

    Article  MathSciNet  MATH  Google Scholar 

  12. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. No. 305–306 in Grund. der math. Wiss, vol. 2. Springer, Berlin (1993)

  13. Hiriart-Urruty, J.B., Lemaréchal, C.: Fundamentals of Convex Analysis. Grundlehren Text Editions, Springer, Berlin (2001)

  14. Karmitsa, N., Bagirov, A., Makela, M.M.: Comparing different nonsmooth minimization methods and software. Optim. Methods Softw. (2010). http://www.informaworld.com/10.1080/10556788.2010.526116

  15. Kiwiel, K.: A method for solving certain quadratic programming problems arising in nonsmooth optimization. IMA J. Numer. Anal. 6, 137–152 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  16. Lemaréchal, C., Oustry, F., Sagastizábal, C.: The \({\cal U}\)-Lagrangian of a convex function. Trans. Am. Math. Soc. 352(2), 711–729 (2000)

    Google Scholar 

  17. Lemaréchal, C., Sagastizábal, C.: Practical aspects of the Moreau-Yosida regularization: theoretical preliminaries. SIAM J. Optim. 7(2), 367–385 (1997). http://link.aip.org/link/?SJE/7/367/1

  18. Lewis, A.: Active sets, nonsmoothness and sensitivity. SIAM J. Optim. 13(3), 702–725 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  19. Lewis, A., Wright, S.: A Proximal Method for Composite Minimization (2008). Available at http://www.optimization-online.org/DB_HTML/2008/12/2162.html

  20. Lewis, A.S., Overton, M.L.: Nonsmooth optimization via quasi-Newton methods. Math. Program. (2011). Accepted for publication

  21. Li, C., Wang, X.: On convergence of the gauss-newton method for convex composite optimization. Math. Program. 91, 349–356 (2002). doi:10.1007/s101070100249

    Article  MathSciNet  MATH  Google Scholar 

  22. Luksan, L., Vlcek, J.: Test Problems for Nonsmooth Unconstrained and Linearly Constrained Optimization. Technical Report 798, Institute of Computer Science, Academy of Sciences of the, Czech Republic (2000)

  23. Mifflin, R.: Semi-smooth and semi-convex functions in constrained optimization. SIAM J. Control Optim. 15, 959–972 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  24. Mifflin, R., Sagastizábal, C.: On \({\cal VU}\)-theory for functions with primal-dual gradient structure. SIAM J. Optim. 11(2), 547–571 (2000). http://siamdl.aip.org/getabs/servlet/GetabsServlet?prog=normal&id=SJOPE8000011000002000547000001&idtype=cvips&gifs=Yes

  25. Mifflin, R., Sagastizábal, C.: Primal-dual gradient structured functions: second-order results; links to epi-derivatives and partly smooth functions. SIAM J. Optim. 13(4), 1174–1194 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  26. Mifflin, R., Sagastizábal, C.: \({\cal VU}\)-smoothness and proximal point results for some nonconvex functions. Optim. Methods Softw. 19(5), 463–478 (2004)

  27. Mifflin, R., Sagastizábal, C.: A \({\cal V}U\)-algorithm for convex minimization. Math. Program. Ser. A 104(2–3), 583–608 (2005). doi:10.1007/s10107-005-0630-3

  28. Nemirovsky, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. A Wiley-Interscience Publication (1983)

  29. Nesterov, Y.: Gradient methods for minimizing composite objective functions. Math. Program. (2011). Accepted for publication

  30. Noll, D., Prot, O., Rondepierre, A.: A proximity control algorithm to minimize nonsmooth and nonconvex functions. Pac. J. Optim. 4(3), 569–602 (2008)

    MathSciNet  Google Scholar 

  31. Oustry, F.: A second-order bundle method to minimize the maximum eigenvalue function. Math. Program. 89(1, Ser. A), 1–33 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  32. Shapiro, A.: On a class of nonsmooth composite functions. Math. Oper. Res. 28(4), 677–692 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  33. Skajaa, A.: Limited Memory BFGS for Nonsmooth Optimization. Master’s thesis, Courant Institute of Mathematical Science (2010). http://cs.nyu.edu/overton/mstheses/skajaa/msthesis.pdf

  34. Womersley, R.: Local properties of algorithms for minimizing nonsmooth composite functions. Math. Program. 32, 69–89 (1985)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IMPA, Estrada Dona Castorina 110, Jardim Botânico, Rio de Janeiro, RJ, 22460-320, Brazil

    Claudia Sagastizábal

Authors
  1. Claudia Sagastizábal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudia Sagastizábal.

Additional information

Research partially supported by Grants CNPq 303840/2011-0, AFOSR FA9550-08-1-0370, and NSF DMS 0707205, as well as by PRONEX-Optimization and FAPERJ.

To C. Lemaréchal, a nonsmooth optimization giant who kindly guided the author through ridges of nondifferentiability until a superlinear path of knowledge. Merci, Claude.

On leave from INRIA, France Visiting researcher at IMPA.

Appendix

Appendix

See Tables 8, 9, 10, and 11.

Table 8 Optimal (opt) or best (best) function values, for problems in Groups 2 and 3
Full size table
Table 9 Results for Group 1: problems in Table 1
Full size table
Table 10 Results for Group 2: problems in Table 3, \(n\le 50\)
Full size table
Table 11 Results for Group 3: problems in Table 3, \(n=100\) and \(n=500\)
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sagastizábal, C. Composite proximal bundle method. Math. Program. 140, 189–233 (2013). https://doi.org/10.1007/s10107-012-0600-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-012-0600-5

Keywords

Mathematical Subject Classification

Search

Navigation