Skip to main content
SpringerLink
Log in

A Second Order Bundle Algorithm for Nonsmooth, Nonconvex Optimization Problems

  • Chapter
  • First Online:
Numerical Nonsmooth Optimization

Abstract

In this chapter we extend the SQP-approach of the well-known bundle-Newton method for nonsmooth unconstrained minimization and the second order bundle method by Fendl and Schichl (A feasible second order bundle algorithm for nonsmooth, nonconvex optimization problems with inequality constraints: I. derivation and convergence. arXiv:1506.07937, 2015, preprint) to the general nonlinearly constrained case. Instead of using a penalty function or a filter or an improvement function to deal with the presence of constraints, the search direction is determined by solving a convex quadratically constrained quadratic program to obtain good iteration points. Furthermore, global convergence of the method is shown under certain mild assumptions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bagirov, A.: A method for minimization of quasidifferentiable functions. Optim. Methods Softw. 17, 31–60 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bagirov, A.: Continuous subdifferential approximations and their applications. J. Math. Sci. 115, 2567–2609 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bagirov, A., Ganjehlou, A.: A quasisecant method for minimizing nonsmooth functions. Optim. Methods Softw. 25(1), 3–18 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bagirov, A., Beliakov, G., Mammadov, M., Rubinov, A.: Programming library GANSO (Global And Non-Smooth Optimization). Centre for Informatics and Applied Optimization, University of Ballarat, Australia (2005). http://www.ballarat.edu.au/ard/itms/CIAO/ganso/. Version 5.1.0.0

    Google Scholar 

  5. Bagirov, A., Karasözen, B., Sezer, M.: Discrete gradient method: derivative-free method for nonsmooth optimization. J. Optim. Theory Appl. 137(2), 317–334 (2008)

    MathSciNet  MATH  Google Scholar 

  6. Bonnans, J., Gilbert, J., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization: Theoretical and Practical Aspects, 2nd edn. Springer, Berlin (2006)

    MATH  Google Scholar 

  7. Borwein, J., Lewis, A.: Convex Analysis and Nonlinear Optimization: Theory and Examples, 2nd edn. Springer, Berlin (2006)

    Book  MATH  Google Scholar 

  8. Boţ, R. I., Csetnek, E. R.: An inertial Tseng’s type proximal algorithm for nonsmooth and nonconvex optimization problems. J. Optim. Theory Appl. 171(2), 600–616 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  9. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  10. Burke, J., Lewis, A., Overton, M.: Approximating subdifferentials by random sampling of gradients. Math. Oper. Res. 27(3), 567–584 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  11. Burke, J., Lewis, A., Overton, M.: A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J. Optim. 15(3), 751–779 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  12. Byrd, R., Nocedal, J., Schnabel, R.: Representations of quasi-Newton matrices and their use in limited memory methods. Math. Program. 63, 129–156 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  13. Curtis, F., Overton, M.: A robust sequential quadratic programming algorithm for nonconvex, nonsmooth constrained optimization. SIAM J. Optim. 22(2), 474–500 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  14. Eckstein, J., Bertsekas, D.P.: On the douglas–rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1–3), 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  15. Fendl, H., Schichl, H.: A feasible second order bundle algorithm for nonsmooth, nonconvex optimization problems with inequality constraints: I. derivation and convergence. arXiv:1506.07937 (2015, preprint)

    Google Scholar 

  16. Fendl, H., Schichl, H.: A feasible second order bundle algorithm for nonsmooth nonconvex optimization problems with inequality constraints: II. implementation and numerical results. arXiv:1506.08021 (2015, preprint)

    Google Scholar 

  17. Fletcher, R., Leyffer, S.: A bundle filter method for nonsmooth nonlinear optimization. Numerical Analysis Report NA/195, University of Dundee, Department of Mathematics (1999)

    Google Scholar 

  18. Gill, P., Murray, W.: Newton-type methods for unconstrained and linearly constrained optimization. Math. Program. 28, 311–350 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gill, P., Murray, W., Saunders, M.: SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM Rev. 47(1), 99–131 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  20. Griewank, A., Corliss, G. (eds.): Automatic Differentiation of Algorithms: Theory, Implementation, and Application. SIAM, Philadelphia (1991)

    MATH  Google Scholar 

  21. Haarala, M.: Large-scale nonsmooth optimization: variable metric bundle method with limited memory. Ph.D. Thesis, University of Jyväskylä, Department of Mathematical Information Technology (2004)

    Google Scholar 

  22. Haarala, M., Miettinen, M., Mäkelä, M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. 19(6), 673–692 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  23. Haarala, N., Miettinen, M., Mäkelä, M.: Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math. Program. 109(1), 181–205 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  24. Herskovits, J.: Feasible direction interior-point technique for nonlinear optimization. J. Optim. Theory Appl. 99(1), 121–146 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  25. Herskovits, J., Santos, G.: On the computer implementation of feasible direction interior point algorithms for nonlinear optimization. Struct. Optim. 14, 165–172 (1997)

    Article  Google Scholar 

  26. Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes. J. Global Optim. 68(3), 501–535 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  27. Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M., Taheri, S.: Double bundle method for finding Clarke stationary points in nonsmooth DC programming. SIAM J. Optim. 28(2), 1892–1919 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  28. Jourani, A.: Constraint qualifications and lagrange multipliers in nondifferentiable programming problems. J. Optim. Theory Appl. 81(3), 533–548 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  29. Kappel, F., Kuntsevich, A.: An implementation of Shor’s r-algorithm. Comput. Optim. Appl. 15(2), 193–205 (2000). https://doi.org/10.1023/A:1008739111712

    Article  MathSciNet  MATH  Google Scholar 

  30. Karas, E., Ribeiro, A., Sagastizábal, C., Solodov, M.: A bundle-filter method for nonsmooth convex constrained optimization. Math. Program. B(116), 297–320 (2009)

    Google Scholar 

  31. Karmitsa, N.: Decision tree for nonsmooth optimization software. http://napsu.karmitsa.fi/solveromatic/

  32. Karmitsa, N., Mäkelä, M.: Adaptive limited memory bundle method for bound constrained large-scale nonsmooth optimization. Optim. J. Math. Program. Oper. Res. 59(6), 945–962 (2010)

    MathSciNet  MATH  Google Scholar 

  33. Karmitsa, N., Mäkelä, M.: Limited memory bundle method for large bound constrained nonsmooth optimization: convergence analysis. Optim. Methods Softw. 25(6), 895–916 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  34. Karmitsa, N., Mäkelä, M., Ali, M.: Limited memory interior point bundle method for large inequality constrained nonsmooth minimization. Appl. Math. Comput. 198(1), 382–400 (2008)

    MathSciNet  MATH  Google Scholar 

  35. Karmitsa, N., Bagirov, A., Mäkelä, M.: Comparing different nonsmooth minimization methods and software. Optim. Methods Softw. 27(1), 131–153 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  36. Kiwiel, K.: Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics, vol. 1133. Springer, Berlin (1985)

    Google Scholar 

  37. Kiwiel, K.: A constraint linearization method for nondifferentiable convex minimization. Numer. Math. 51, 395–414 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  38. Kiwiel, K.: Restricted step and Levenberg-Marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization. SIAM J. Optim. 6(1), 227–249 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  39. Kiwiel, K., Stachurski, A.: Issues of effectiveness arising in the design of a system of nondifferentiable optimization algorithms. In: Lewandowski, A., Wierzbicki, A. (eds.) Aspiration Based Decision Support Systems: Theory, Software and Applications. Lecture Notes in Economics and Mathematical Systems, vol. 331, pp. 180–192. Springer, Berlin (1989)

    Chapter  Google Scholar 

  40. Kroshko, D.: ralg. http://openopt.org/ralg/. Software package

  41. Lawrence, C., Tits, A.: A computationally efficient feasible sequential quadratic programming algorithm. SIAM J. Optim. 11(4), 1092–1118 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  42. Lewis, A., Overton, M.: Nonsmooth optimization via quasi–Newton methods. Math. Program. 141, 135–163 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  43. Lukšan, L.: An implementation of recursive quadratic programming variable metric methods for linearly constrained nonlinear minimax approximation. Kybernetika 21(1), 22–40 (1985)

    MathSciNet  MATH  Google Scholar 

  44. Lukšan, L., Vlček, J.: PBUN, PNEW – bundle-type algorithms for nonsmooth optimization. Technical report 718, Institute of Computer Science, Academy of Sciences of the Czech Republic, Prague (1997). http://www.uivt.cas.cz/~luksan/subroutines.html

  45. Lukšan, L., Vlček, J.: PMIN – a recursive quadratic programming variable metric algorithm for minimax optimization. Technical report 717, Institute of Computer Science, Academy of Sciences of the Czech Republic, Prague (1997). http://www.uivt.cas.cz/~luksan/subroutines.html

  46. Lukšan, L., Vlček, J.: A bundle-Newton method for nonsmooth unconstrained minimization. Math. Program. 83, 373–391 (1998)

    MathSciNet  MATH  Google Scholar 

  47. Lukšan, L., Vlček, J.: Globally convergent variable metric method for convex nonsmooth unconstrained minimization. J. Optim. Theory Appl. 102(3), 593–613 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  48. Lukšan, L., Vlček, J.: NDA: algorithms for nondifferentiable optimization. Technical report 797, Institute of Computer Science, Academy of Sciences of the Czech Republic, Prague (2000). http://www.uivt.cas.cz/~luksan/subroutines.html

  49. Mäkelä, M.: Multiobjective proximal bundle method for nonconvex nonsmooth optimization: FORTRAN subroutine MPBNGC 2.0. Reports of the Department of Mathematical Information Technology, Series B. Scientific computing, B 13/2003 University of Jyväskylä, Jyväskylä (2003). http://napsu.karmitsa.fi/proxbundle/

  50. Mäkelä, M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific, Singapore (1992)

    Book  MATH  Google Scholar 

  51. Mangasarian, O.L., Fromovitz, S.: The Fritz John necessary optimality conditions in the presence of equality and inequality constraints. J. Math. Anal. Appl. 17(1), 37–47 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  52. Maratos, N.: Exact penalty function algorithms for finite dimensional and control optimization problems. Ph.D. Thesis, University of London (1978)

    Google Scholar 

  53. Merrill, O.: Applications and extensions of an algorithm that computes fixed points of certain upper semicontinuous point to set mappings. Ph.D. Thesis, University of Michigan, Ann Arbor (1972)

    Google Scholar 

  54. Mifflin, R.: An algorithm for constrained optimization with semismooth functions. Math. Oper. Res. 2(2), 191–207 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  55. Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15(6), 959–972 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  56. Mifflin, R.: A modification and an extension of Lemarechal’s algorithm for nonsmooth minimization. Math. Program. Study 17, 77–90 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  57. Mifflin, R., Sun, D., Qi, L.: Quasi-Newton bundle-type methods for nondifferentiable convex optimization. SIAM J. Optim. 8(2), 583–603 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  58. Nesterov, Y., Vial, J.P.: Homogeneous analytic center cutting plane methods for convex problems and variational inequalities. SIAM J. Optim. 9, 707–728 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  59. Nocedal, J., Wright, S.: Numerical Optimization. Springer Series in Operations Research and Financial Engineering, 2nd edn. Springer, New York (2006)

    Google Scholar 

  60. Oustry, F.: A second-order bundle method to minimize the maximum eigenvalue function. Math. Program. 89(1), 1–33 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  61. Overton, M.: Hanso: Hybrid algorithm for non-smooth optimization (2010). http://www.cs.nyu.edu/overton/software/hanso/. New York University, Department of Computer Science

  62. Parikh, N., Boyd, S., et al.: Proximal algorithms. Found. Trends Optim. 1(3), 127–239 (2014)

    Article  Google Scholar 

  63. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  64. Sagastizábal, C., Solodov, M.: An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter. SIAM J. Optim. 16, 146–169 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  65. Schramm, H.: Eine Kombination von Bundle- und Trust-Region-Verfahren zur Lösung nichtdifferenzierbarer Optimierungsprobleme. Ph.D. Thesis, Universität Bayreuth (1989)

    Google Scholar 

  66. Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2(1), 121–152 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  67. Shor, N.: Minimization Methods for Non-Differentiable Functions. Springer, Berlin (1985)

    Book  MATH  Google Scholar 

  68. Solodov, M.: On the sequential quadratically constrained quadratic programming methods. Math. Oper. Res. 29(1), 1–90 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  69. Tits, A.: Feasible sequential quadratic programming. In: Floudas, C., Pardalos, P. (eds.) Encyclopedia of Optimization, 2nd edn., pp. 1001–1005. Springer, Berlin (2009)

    Chapter  Google Scholar 

  70. Vial, J.P., Sawhney, N.: OBOE User Guide Version 1.0 (2007). https://projects.coin-or.org/OBOE/

  71. Vlček, J.: Bundle algorithms for nonsmooth unconstrained minimization. Research Report 608, Institute of Computer Science, Academy of Sciences of the Czech Republic, Prague (1995)

    Google Scholar 

  72. Vlček, J., Lukšan, L.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J. Optim. Theory Appl. 111(2), 407–430 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  73. Zowe, J.: The BT-algorithm for minimizing a nonsmooth functional subject to linear constraints. In: Clarke, F., Demyanov, V., Giannessi, F. (eds.) Nonsmooth Optimization and Related Topics, pp. 459–480. Plenum Press, New York (1989)

    Chapter  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Vienna, Faculty of Mathematics, Wien, Austria

    Hermann Schichl & Hannes Fendl

Authors
  1. Hermann Schichl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hannes Fendl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hermann Schichl .

Editor information

Editors and Affiliations

  1. School of Science, Engineering and Information Technology, Federation University Australia, Ballarat, VIC, Australia

    Adil M. Bagirov

  2. Department of Informatics, Modeling, Electronics and System Engineering, University of Calabria, Rende (CS), Italy

    Manlio Gaudioso

  3. Department of Mathematics and Statistics, University of Turku, Turku, Finland

    Napsu Karmitsa

  4. Department of Mathematics and Statistics, University of Turku, Turku, Finland

    Marko M. Mäkelä

  5. School of Science, Engineering and Information Technology, Federation University Australia, Ballarat, VIC, Australia

    Sona Taheri

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Schichl, H., Fendl, H. (2020). A Second Order Bundle Algorithm for Nonsmooth, Nonconvex Optimization Problems. In: Bagirov, A., Gaudioso, M., Karmitsa, N., Mäkelä, M., Taheri, S. (eds) Numerical Nonsmooth Optimization. Springer, Cham. https://doi.org/10.1007/978-3-030-34910-3_4

Download citation

Publish with us

Policies and ethics

Search

Navigation