Skip to main content
SpringerLink
Log in

Newton methods for nonsmooth convex minimization: connections among -Lagrangian, Riemannian Newton and SQP methods

  • Published:
Mathematical Programming Submit manuscript

Abstract

This paper studies Newton-type methods for minimization of partly smooth convex functions. Sequential Newton methods are provided using local parameterizations obtained from -Lagrangian theory and from Riemannian geometry. The Hessian based on the -Lagrangian depends on the selection of a dual parameter g; by revealing the connection to Riemannian geometry, a natural choice of g emerges for which the two Newton directions coincide. This choice of g is also shown to be related to the least-squares multiplier estimate from a sequential quadratic programming (SQP) approach, and with this multiplier, SQP gives the same search direction as the Newton methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Bonnans, J., Gilbert, J., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization. Universitext. Springer-Verlag, 2003

  2. Chamberlain, R., Lemaréchal, C., Pedersen, H., Powell, M.: The watchdog technique for forcing convergence in algorithms for constrained optimization. Math. Program. Study Series 16, 1–17 (1982)

    Google Scholar 

  3. Dedieu, J.-P., Priouret, P., Malajovich, G.: Newton's method on Riemannian manifolds: covariant alpha-theory. IMA J. Numer. Anal. 23 (3), 395–419 (2003)

    Article  Google Scholar 

  4. Dennis, J.E., Moré, J.J.: Quasi-Newton methods, motivation and theory. SIAM Review 19(1), 46–89 (1977)

    Article  Google Scholar 

  5. do Carmo, M.P.: Riemannian Geometry. Mathematics: Theory and Applications. Birkhäuser, 1992

  6. Edelman, A., Arias, T., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20 (2), 303–353 (1999)

    Article  Google Scholar 

  7. Gabay, D.: Minimizing a differentiable function over a differentiable manifold. J. Optimization Theory Appl. 37(2), 177–219 June 1982

    Article  Google Scholar 

  8. Hare, W.: Recent functions and sets of smooth substructure: Relationships and examples. To appear in J. Comput. Optim. Appl. available at http://www.cecm.sfu.ca/~whare/Subsmooth-relations.ps, Mar. 17, 2005

  9. Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. Number 305–306 in Grundlehren der mathematischen Wissenschaften. Springer-Verlag, Berlin, 1993

  10. Lee, J.M.: Riemannian Manifolds: An Introduction to Curvature. Number 176 in Graduate Texts in Mathematics. Springer-Verlag, New York, 1997

  11. Lemaréchal, C., Oustry, F., Sagastizábal, C.: The -Lagrangian of a convex function. Trans. AMS 352(2), 711–729 (1999)

    Article  Google Scholar 

  12. Lewis, A.S.: Active sets, nonsmoothness and sensitivity. SIAM J. Optim. 13, 702–725 (2003)

    Article  Google Scholar 

  13. Maratos, N.: Exact penalty function algorithms for finite dimensional and control optimization problems. PhD thesis, Imperial College, London, 1978

  14. Mayne, D.Q.: On the use of exact penalty functions to determine step length in optimization algorithms. In: Lecture Notes in Mathematics, volume 773, Springer Verlag, 1980 pp 98–109

  15. Mifflin, R., Sagastizábal, C.: -decomposition derivatives for convex max-functions. In: M. Théra and R. Tichatschke, editors, Ill-Posed Variational Problems and Regularization Techniques, Springer-Verlag, Berlin, 1999 pp 167–186

  16. Mifflin, R., Sagastizábal C.: On -theory for functions with primal-dual gradient structure. SIAM J. Optim. 11 (2), 547–571 (2000)

    Article  Google Scholar 

  17. Mifflin, R., Sagastizábal, C.: Proximal points are on the fast track. J. Convex Anal. 9 (2), 563–579 (2002)

    Google Scholar 

  18. Mifflin, R., Sagastizábal, C.: Primal-dual gradient structured functions: second-order results; links to epi-derivatives and partly smooth functions. SIAM J. Optim. 13 (4), 1174–1194 (2003)

    Article  Google Scholar 

  19. Mifflin, R., Sagastizábal, C.: -smoothness and proximal point results for some nonconvex functions. Optim. Methods Softw. 19 (5), 463–478 (2004)

    Article  Google Scholar 

  20. Mifflin, R., Sagastizábal, C.: A -algorithm for convex minimization. Preprint, available at http://www.sci.wsu.edu/math/faculty/mifflin/vualgo.ps, revised Feb. 14, 2005

  21. Oustry, F.: The -Lagrangian of the maximum eigenvalue function. SIAM J. Optim. 9 (2), 526–549 (1999)

    Article  Google Scholar 

  22. Overton, M.L.: Large-scale optimization of eigenvalues. SIAM J. Optim. 2, 88–120 (1992)

    Article  Google Scholar 

  23. Overton, M.L., Ye, X.: Towards second-order methods for structured nonsmooth optimization. In: S. Gomez and J. P. Hennart, editors, Advances in Optimization and Numerical Analysis, Kluwer, 1994 pp 97–110

  24. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Number 317 in Grundlehren der mathematischen Wissenschaften. Springer-Verlag, Berlin, 1998

  25. Shapiro, A., Fan, M.K.H.: On eigenvalue optimization. SIAM J. Optim. 5 (3), 552–569 (1995)

    Article  Google Scholar 

  26. Smith, S.T.: Optimization techniques on Riemannian manifolds. Fields Inst. Comm. 3, 113–136 (1994)

    Google Scholar 

  27. Udrişte, C.: Convex functions and optimization methods on Riemannian manifolds. Number 297 in Mathematics and its Applications. Kluwer, 1994

Download references

Author information

Authors and Affiliations

  1. Numerica Corp., P.O. Box 271246, Ft. Collins, CO, 80527-1246, USA

    Scott A. Miller

  2. INRIA, 655 avenue de l'Europe, 38334, Saint Ismier Cedex, France

    Jérôme Malick

Authors
  1. Scott A. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jérôme Malick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Scott A. Miller.

Additional information

This paper is dedicated to R.T. Rockafellar, on the occasion of his 70th birthday.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Miller, S., Malick, J. Newton methods for nonsmooth convex minimization: connections among -Lagrangian, Riemannian Newton and SQP methods. Math. Program. 104, 609–633 (2005). https://doi.org/10.1007/s10107-005-0631-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-005-0631-2

Keywords

Search

Navigation