Abstract
Subgradient and Newton algorithms for nonsmooth optimization require generalized derivatives to satisfy subtle approximation properties: conservativity for the former and semismoothness for the latter. Though these two properties originate in entirely different contexts, we show that in the semi-algebraic setting they are equivalent. Both properties for a generalized derivative simply require it to coincide with the standard directional derivative on the tangent spaces of some partition of the domain into smooth manifolds. An appealing byproduct is a new short proof that semi-algebraic maps are semismooth relative to the Clarke Jacobian.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Data sharing not applicable to this article as no datasets were generated or analysed during the current study.
References
Bolte, J., Daniilidis, A., Lewis, A.: Tame functions are semismooth. Math. Program. 117(1), 5–19 (2009)
Bolte, J., Daniilidis, A., Lewis, A., Shiota, M.: Clarke subgradients of stratifiable functions. SIAM J. Optim. 18(2), 556–572 (2007)
Bolte, J., Pauwels, E: Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning. Mathematical Programming (2020)
Bolte, J., Pauwels, E.: A mathematical model for automatic differentiation in machine learning. arXiv:2006.02080 (2020)
Clarke, F. H., Ledyaev, Y. u., Stern, R. I., Wolenski, P. R.: Nonsmooth Analysis and Control Theory Texts in Math, vol. 178. Springer, New York (1998)
Coste, M.: An introduction to o-minimal geometry. RAAG Notes, 81 pages. Institut de Recherche Mathématiques de Rennes (1999)
Coste, M.: An introduction to semialgebraic geometry. RAAG notes, 78 pages, Institut de Recherche Mathématiques de Rennes (2002)
Daniilidis, A., Drusvyatskiy, D.: Pathological subgradient dynamics. SIAM J. Optim. 30(2), 1327–1338 (2020)
Davis, D., Drusvyatskiy, D., Kakade, S., Lee, J. D.: Stochastic subgradient method converges on tame functions. Foundations of Computational Mathematics 20(1), 119–154 (2020)
Drusvyatskiy, D., Ioffe, A., Lewis, A.: Curves of descent. SIAM J. Control. Optim. 53(1), 114–138 (2015)
Drusvyatskiy, D., Lewis, A.: Semi-algebraic functions have small subdifferentials. Math. Program. 140(1), 5–29 (2013)
Ioffe, A. D.: Variational Analysis of Regular Mappings. Springer Monographs in Mathematics. Springer, Cham (2017). Theory and applications
Klatte, D., Kummer, B.: Nonsmooth Equations in Optimization, volume 60 of Nonconvex Optimization and its Applications. Kluwer Academic Publishers, Dordrecht (2002). Regularity, calculus, methods and applications
Lewis, A., Tian, T.: The structure of conservative gradient fields. arXiv:2101.00699 (2021)
Majewski, S., Miasojedow, B., Moulines, E.: Analysis of nonsmooth stochastic approximation: the differential inclusion approach. arXiv:1805.01916 (2018)
Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control. Optim. 15(6), 959–972 (1977)
Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I: Basic Theory. Grundlehren der mathematischen Wissenschaften, vol. 330. Springer, Berlin (2006)
Norkin, V: Generalized-differentiable functions. Cybernetics 16(1), 10–12 (1980)
Norkin, V: Stochastic generalized-differentiable functions in the problem of nonconvex nonsmooth stochastic optimization. Cybernetics 22(6), 804–809 (1986)
Penot, J. -P.: Calculus without derivatives, volume 266 of Graduate Texts in Mathematics. Springer, New York (2013)
Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Mathematical Programming 58(1-3), 353–367 (1993)
Rockafellar, R. T., Wets, R. J. -B.: Variational Analysis Grundlehren der mathematischen Wissenschaften, vol. 317. Springer, Berlin (1998)
Ruszczyński, A.: Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization. Optim. Lett. 1–11 (2020)
Shapiro, A.: On concepts of directional differentiability. J. Optim. Theory Appl. 66(3), 477–487 (1990)
van den Dries, L., Miller, C.: Geometric categories and o-minimal structures. Duke Math. J. 84, 497–540 (1996)
Acknowledgements
We thank Adrian Lewis, Tonghua Tian, and the two anonymous reviewers for insightful comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Research of Davis supported by an Alfred P. Sloan research fellowship and NSF DMS award 2047637.
Research of Drusvyatskiy was supported by the NSF DMS 1651851 and CCF 1740551 awards.
Rights and permissions
About this article
Cite this article
Davis, D., Drusvyatskiy, D. Conservative and Semismooth Derivatives are Equivalent for Semialgebraic Maps. Set-Valued Var. Anal 30, 453–463 (2022). https://doi.org/10.1007/s11228-021-00594-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11228-021-00594-0