Skip to main content
SpringerLink
Log in

Descentwise inexact proximal algorithms for smooth optimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The proximal method is a standard regularization approach in optimization. Practical implementations of this algorithm require (i) an algorithm to compute the proximal point, (ii) a rule to stop this algorithm, (iii) an update formula for the proximal parameter. In this work we focus on (ii), when smoothness is present—so that Newton-like methods can be used for (i): we aim at giving adequate stopping rules to reach overall efficiency of the method.

Roughly speaking, usual rules consist in stopping inner iterations when the current iterate is close to the proximal point. By contrast, we use the standard paradigm of numerical optimization: the basis for our stopping test is a “sufficient” decrease of the objective function, namely a fraction of the ideal decrease. We establish convergence of the algorithm thus obtained and we illustrate it on some ill-conditioned problems. The experiments show that combining the proposed inexact proximal scheme with a standard smooth optimization algorithm improves the numerical behaviour of the latter for those ill-conditioned problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. http://www-rocq.inria.fr/~gilbert/modulopt.

References

  1. Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16, 1–3 (1966)

    MathSciNet  MATH  Google Scholar 

  2. Bellman, R.E., Kalaba, R.E., Lockett, J.: Numerical Inversion of the Laplace Transform. Elsevier, New York (1966), pp. 143–144

    MATH  Google Scholar 

  3. Bongartz, I., Conn, A.R., Gould, N., Toint, Ph.L.: CUTE: Constrained and Unconstrained Testing Environment. ACM Trans. Math. Softw. 21(1), 123–260 (1995)

    Article  MATH  Google Scholar 

  4. Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62, 261–275 (1993)

    Article  MATH  Google Scholar 

  5. Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs (1983)

    MATH  Google Scholar 

  6. Gilbert, J.C., Lemaréchal, C.: Some numerical experiments with variable-storage quasi-Newton algorithms. Math. Program. 45, 407–435 (1989)

    Article  MATH  Google Scholar 

  7. Goldstein, A.A., Price, J.F.: An effective algorithm for minimization. Numer. Math. 10, 184–189 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  8. Hager, W.W., Zhang, H.: Self-adaptive inexact proximal point methods. Comput. Optim. Appl. 39(2), 161–181 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  9. Humes, C., Silva, P.J.S.: Inexact proximal point algorithms and descent methods in optimization. Optim. Eng. 6, 257–271 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  10. Lemaréchal, C., Sagastizábal, C.: Variable metric bundle methods: from conceptual to implementable forms. Math. Program. 76(3), 393–410 (1997)

    Article  MATH  Google Scholar 

  11. Levenberg, K.: A method for the solution of certain nonlinear problems in least squares. Q. Appl. Math. 2, 164–168 (1944)

    MathSciNet  MATH  Google Scholar 

  12. Maréchal, P., Rondepierre, A.: A proximal approach to the inversion of ill-conditioned matrices. C. R. Acad. Sci. Paris 347(23–24), 1435–1438 (2009)

    Article  MATH  Google Scholar 

  13. Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  14. Martinet, B.: Régularisation d’inéquations variationelles par approximation successives. Rev. Fr. Inform. Rech. Oper. R3, 154–158 (1970)

    MathSciNet  Google Scholar 

  15. Moré, J.J.: Recent developments in algorithms and software for trust region methods. In: Bachem, A., Grötschel, M., Korte, B. (eds.) Mathematical Programming, the State of the Art, pp. 258–287. Springer, Berlin (1983)

    Chapter  Google Scholar 

  16. Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35, 773–782 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  17. Qi, L., Sun, J.: nonsmooth version of Newton’s method. Math. Program. (1993)

  18. Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97–116 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  19. Solodov, M.V., Svaiter, B.F.: A hybrid projection-proximal point algorithm. J. Convex Anal. 6(1), 059–070 (1999)

    MathSciNet  Google Scholar 

  20. Solodov, M.V., Svaiter, B.F.: A unified framework for some inexact proximal point algorithms. Numer. Funct. Anal. Optim. 22, 1013–1035 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  21. Todd, M.: On convergence properties of algorithms for unconstrained minimization. IMA J. Numer. Anal. 9, 435–441 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  22. Wolfe, P.: Convergence conditions for ascent methods. SIAM Rev. 11, 226–235 (1969)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Inria—Rhône-Alpes, 655 avenue de l’Europe, 38334, Montbonnot, France

    Marc Fuentes & Claude Lemaréchal

  2. CNRS, LJK (Grenoble), 655 avenue de l’Europe, 38334, Montbonnot, France

    Jérôme Malick

Authors
  1. Marc Fuentes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jérôme Malick
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claude Lemaréchal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jérôme Malick.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fuentes, M., Malick, J. & Lemaréchal, C. Descentwise inexact proximal algorithms for smooth optimization. Comput Optim Appl 53, 755–769 (2012). https://doi.org/10.1007/s10589-012-9461-3

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-012-9461-3

Keywords

Search

Navigation