Skip to main content
SpringerLink
Log in

Generalizations of the proximal method of multipliers in convex optimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The proximal method of multipliers, originally introduced as a way of solving convex programming problems with inequality constraints, is a proximally stabilized alternative to the augmented Lagrangian method that is sometimes called the proximal augmented Lagrangian method. It has gained attention as a vehicle for deriving decomposition algorithms for wider formulations of problems in convex optimization than just convex programming. Here those themes are developed further. The basic algorithm is articulated in several seemingly different formats that are equivalent under exact computations, but diverge when minimization steps are executed only approximately. Stopping criteria are demonstrated to maintain convergence to a particular solution despite such approximations. Q-linear convergence is obtained from a metric regularity property of the Lagrangian mapping at the solution that acts as a mildly enhanced condition for local optimality on top of convexity and is generically available, in a sense. Moreover, all this is brought about with the proximal terms allowed to vary in their underlying metric from one iteration to the next. That generalization enables the results to be translated to the theory of the progressive decoupling algorithm, significantly adding to its versatility and providing linear convergence guarantees in its broad applicability to techniques for problem decomposition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data availability:

No data was generated or analyzed.

Notes

  1. The ambiguity in the Lagrangian formula when \(f(x)=\infty \) and \(g^*(y)=\infty \) is resolved by (1.1) as \(l(x,y)=\infty \).

  2. An earlier variable-metric PPA of Parente, Lotito and Solodov [5], despite its notable innovations, treats approximation in a completely different manner that rules out the possibility of deriving implementable stopping criteria in our context. This is explained in [18].

  3. The possibility of \(\alpha _k=0\) can be covered through a work-around that for every \(r'>r\) requires \(\alpha _{k+1}\le r'\alpha _k\) when k is sufficiently large.

  4. For the sake of these extremes, the expression for r would better be taken in the form \(a_\infty /\sqrt{a_\infty ^2+c_\infty ^2}\), but the form in (2.25) is superior otherwise for emphasizing that the rate only depends on the ratio between \(c_\infty \) and \(a_\infty \) when they are finite.

  5. Because the nonexpansivity is actually “firm nonexpansvity.”

  6. If either the projection on S or the projection on \(S^\perp \) is readily computable, then so too is the other, according to the complementarity in (4.3).

  7. Of course, it could be assumed for simplicity that these bounds hold for all z and w in H, but only this much is needed in our analysis.

References

  1. Dontchev, A.D., Rockafellar, R.T.: Implicit Functions and Solution Mappings, 2nd edn. Springer (2014)

    Book  Google Scholar 

  2. Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  Google Scholar 

  3. Ismailov, A.F., Solodov, M.V.: Perturbed augmented Lagrangian method framework with applications to proximal and smoothed variants. J. Optim. Theory Appl. 193, 491–522 (2022)

    Article  MathSciNet  Google Scholar 

  4. Luque, F.J.: Asymptotic convergence analysis of the proximal point algorithm. SIAM J. Control Opt. 22, 277–293 (1984)

    Article  MathSciNet  Google Scholar 

  5. Parente, L.A., Lotito, P.A., Solodov, M.V.: A class of inexact variable metric proximal point algorithms. SIAM J. Optim. 19, 240–260 (2008)

    Article  MathSciNet  Google Scholar 

  6. Powell, M.J.D.: A method for nonlinear optimization in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press (1969)

    Google Scholar 

  7. Rockafellar, R.T.: Convex Analysis. Princeton University Press (1970)

    Book  Google Scholar 

  8. Rockafellar, R.T.: Conjugate Duality and Optimization. No. 16 in Conference Board of Mathematical Sciences Series, SIAM Publications (1974)

  9. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    Article  MathSciNet  Google Scholar 

  10. Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97–116 (1976)

    Article  MathSciNet  Google Scholar 

  11. Rockafellar, R.T., Wets, R.J.-B.: Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16, 119–147 (1991)

    Article  MathSciNet  Google Scholar 

  12. Rockafellar, R.T.: Progressive decoupling of linkages in monotone variational inequalities and convex optimization. In: Hojo, M., Hoshino, M., Takahashi, W. (eds.) Proceedings of the 10th International Conference on Nonlinear Analysis and Convex Analysis (Chitose, Japan, 2017), Yokohama Publishers, pp. 271–291 (2019)

  13. Rockafellar, R.T.: Progressive decoupling of linkages in optimization and variational inequalities with elicitable convexity or monotonicity. Set-Valued Var. Anal. 27, 863–893 (2019)

    Article  MathSciNet  Google Scholar 

  14. Rockafellar, R.T.: Augmented Lagrangians and hidden convexity in sufficient conditions for local optimality. Math. Program. 198, 159–194 (2023). https://doi.org/10.1007/s10107-022-01768-w

    Article  MathSciNet  Google Scholar 

  15. Rockafellar, R.T.: Advances in convergence and scope of the proximal point algorithm. J. Nonlinear Convex Anal. 22, 2347–2375 (2021)

    MathSciNet  Google Scholar 

  16. Rockafellar, R.T.: Convergence of augmented Lagrangian methods in extensions beyond nonlinear programming. Math. Program. 199, 375–420 (2023). https://doi.org/10.1007/s10107-022-01832-5

    Article  MathSciNet  Google Scholar 

  17. Rockafellar R.T.: Metric regularity properties of monotone mappings. Mathematica Serdica (memorial issue for Asen Dontchev). Accepted

  18. Rockafellar R.T.: Generic linear convergence through metric subregularity in a variable-metric extension of the proximal point algorithm. Comput. Optim. Appl. https://doi.org/10.1007/s10589-023-00494-z

  19. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. No. 317 in the series Grundlehren der Mathematischen Wissenschaften, Springer (1997)

  20. Sefi, R., Teboulle, M.: Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim 24, 269–297 (2014)

    Article  MathSciNet  Google Scholar 

  21. Spingarn, J.: Partial inverse of a monotone operator. Appl. Math. Optim. 10, 247–265 (1983)

    Article  MathSciNet  Google Scholar 

  22. Spingarn, J.: Applications of the method of partial inverses to convex programming: decomposition. Math. Program. 32, 199–223 (1985)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Washington, Box 354350, Seattle, WA, 98195-4350, USA

    R. Tyrrell Rockafellar

Authors
  1. R. Tyrrell Rockafellar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. Tyrrell Rockafellar.

Ethics declarations

Conflict of interest:

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rockafellar, R.T. Generalizations of the proximal method of multipliers in convex optimization. Comput Optim Appl 87, 219–247 (2024). https://doi.org/10.1007/s10589-023-00519-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-023-00519-7

Keywords

Search

Navigation