Mathematical Programming

, Volume 141, Issue 1, pp 135–163

Nonsmooth optimization via quasi-Newton methods

Authors

  • Adrian S. Lewis
    • School of Operations Research and Information EngineeringCornell University
    • Courant Institute of Mathematical SciencesNew York University
Full Length Paper Series A

DOI: 10.1007/s10107-012-0514-2

Cite this article as:
Lewis, A.S. & Overton, M.L. Math. Program. (2013) 141: 135. doi:10.1007/s10107-012-0514-2
  • 1.5k Views

Abstract

We investigate the behavior of quasi-Newton algorithms applied to minimize a nonsmooth function f, not necessarily convex. We introduce an inexact line search that generates a sequence of nested intervals containing a set of points of nonzero measure that satisfy the Armijo and Wolfe conditions if f is absolutely continuous along the line. Furthermore, the line search is guaranteed to terminate if f is semi-algebraic. It seems quite difficult to establish a convergence theorem for quasi-Newton methods applied to such general classes of functions, so we give a careful analysis of a special but illuminating case, the Euclidean norm, in one variable using the inexact line search and in two variables assuming that the line search is exact. In practice, we find that when f is locally Lipschitz and semi-algebraic with bounded sublevel sets, the BFGS (Broyden–Fletcher–Goldfarb–Shanno) method with the inexact line search almost always generates sequences whose cluster points are Clarke stationary and with function values converging R-linearly to a Clarke stationary value. We give references documenting the successful use of BFGS in a variety of nonsmooth applications, particularly the design of low-order controllers for linear dynamical systems. We conclude with a challenging open question.

Keywords

BFGSNonconvexLine searchR-linear convergenceClarke stationaryPartly smooth

Mathematics Subject Classification (2000)

90C3065K05
Download to read the full article text

Copyright information

© Springer and Mathematical Optimization Society 2012