An algorithm for nonsmooth optimization by successive piecewise linearization
- 54 Downloads
Abstract
We present an optimization method for Lipschitz continuous, piecewise smooth (PS) objective functions based on successive piecewise linearization. Since, in many realistic cases, nondifferentiabilities are caused by the occurrence of abs(), max(), and min(), we concentrate on these nonsmooth elemental functions. The method’s idea is to locate an optimum of a PS objective function by explicitly handling the kink structure at the level of piecewise linear models. This piecewise linearization can be generated in its abs-normal-form by minor extension of standard algorithmic, or automatic, differentiation tools. In this paper it is shown that the new method when started from within a compact level set generates a sequence of iterates whose cluster points are all Clarke stationary. Numerical results including comparisons with other nonsmooth optimization methods then illustrate the capabilities of the proposed approach.
Keywords
Piecewise smoothness Nonsmooth optimization Algorithmic differentiation Abs-normal form Clarke stationaryMathematics Subject Classification
49J52 90C56Notes
Acknowledgements
The authos are indebted to the two referees for their careful reading of the first submission and their many objections and suggestions, which helped to greatly improve the consistency and the readability of the paper.
References
- 1.Burke, J., Lewis, A., Overton, M.: A robust gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 15(3), 751–779 (2005)MathSciNetCrossRefMATHGoogle Scholar
- 2.Clarke, F.: Optimization and Nonsmooth Analysis. SIAM, Philadelphia (1990)CrossRefMATHGoogle Scholar
- 3.de Oliveira, W., Sagastizábal, C.: Bundle methods in the XXIst century: a birds’-eye view. Pesqui. Oper. 34(3), 647–670 (2014)CrossRefGoogle Scholar
- 4.Griewank, A.: On stable piecewise linearization and generalized algorithmic differentiation. Optim. Methods Softw. 28(6), 1139–1178 (2013)MathSciNetCrossRefMATHGoogle Scholar
- 5.Griewank, A., Bernt, J.U., Radons, M., Streubel, T.: Solving piecewise linear systems in abs-normal form. Linear Algebra Appl. 471, 500–530 (2015)MathSciNetCrossRefMATHGoogle Scholar
- 6.Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. SIAM, Philadelphia (2008)CrossRefMATHGoogle Scholar
- 7.Griewank, A., Walther, A.: First and second order optimality conditions for piecewise smooth objective functions. Optim. Methods Softw. 31(5), 904–930 (2016)MathSciNetCrossRefMATHGoogle Scholar
- 8.Griewank, A., Walther, A., Fiege, S., Bosse, T.: On Lipschitz optimization based on gray-box piecewise linearization. Math. Program. Ser. A 158, 383–415 (2016)MathSciNetCrossRefMATHGoogle Scholar
- 9.Gurbuzbalaban, M., Overton, M.L.: On Nesterov’s nonsmooth Chebyshev–Rosenbrock functions. Nonlinear Anal. Theory Methods Appl. 75(3), 1282–1289 (2012)MathSciNetCrossRefMATHGoogle Scholar
- 10.Haarala, M., Miettinen, K., Mäkelä, M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. 19(6), 673–692 (2004)MathSciNetCrossRefMATHGoogle Scholar
- 11.Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I. Springer, Berlin (1993)CrossRefMATHGoogle Scholar
- 12.Karmitsa, N., Mäkelä, M.: Limited memory bundle method for large bound constrained nonsmooth optimization: convergence analysis. Optim. Methods Softw. 25(6), 895–916 (2010)MathSciNetCrossRefMATHGoogle Scholar
- 13.Lemaréchal, C., Sagastizábal, C.: Variable metric bundle methods: from conceptual to implementable forms. Math. Program. 76(3), 393–410 (1997)MathSciNetCrossRefMATHGoogle Scholar
- 14.Lewis, A., Overton, M.: Nonsmooth optimization via quasi-Newton methods. Math. Program. 141(1–2), 135–163 (2013)MathSciNetCrossRefMATHGoogle Scholar
- 15.Lukšan, L., Vlček, J.: Test problems for nonsmooth unconstrained and linearly constrained optimization. Technical report 798, Institute of Computer Science, Academy of Sciences of the Czech Republic (2000)Google Scholar
- 16.Mäkelä, M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific Publishing Co., Singapore (1992)CrossRefMATHGoogle Scholar
- 17.Mäkelä, M.M.: Multiobjective proximal bundle method for nonconvex nonsmooth optimization: Fortran subroutine MPBNGC 2.0. Reports of the Department of Mathematical Information Technology, Series B, Scientific computing No. B 13/2003, University of Jyväskylä (2003)Google Scholar
- 18.Nesterov, Y.: Lexicographic differentiation of nonsmooth functions. Math. Program. 104(2–3), 669–700 (2005)MathSciNetCrossRefMATHGoogle Scholar
- 19.Scholtes, S.: Introduction to Piecewise Differentiable Functions. Springer, Berlin (2012)CrossRefMATHGoogle Scholar
- 20.Shor, N.: Nondifferentiable Optimization and Polynomial Problems. Kluwer, Amsterdam (1998)CrossRefMATHGoogle Scholar