Skip to main content
SpringerLink
Log in

A proximal point algorithm revisit on the alternating direction method of multipliers

  • Articles
  • Published:
Science China Mathematics Aims and scope Submit manuscript

Abstract

The alternating direction method of multipliers (ADMM) is a benchmark for solving convex programming problems with separable objective functions and linear constraints. In the literature it has been illustrated as an application of the proximal point algorithm (PPA) to the dual problem of the model under consideration. This paper shows that ADMM can also be regarded as an application of PPA to the primal model with a customized choice of the proximal parameter. This primal illustration of ADMM is thus complemental to its dual illustration in the literature. This PPA revisit on ADMM from the primal perspective also enables us to recover the generalized ADMM proposed by Eckstein and Bertsekas easily. A worst-case O(1/t) convergence rate in ergodic sense is established for a slight extension of Eckstein and Bertsekas’s generalized ADMM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bertsekas D P. Constrained Optimization and Lagrange Multiplier Methods. New York: Academic Press, 1982

    MATH  Google Scholar 

  2. Blum E, Oettli W. Mathematische Optimierung, Econometrics and Operations Research XX. Berlin: Springer Verlag, 1975

    Book  Google Scholar 

  3. Boyd S, Parikh N, Chu E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn, 2010, 3: 1–122

    Article  MATH  Google Scholar 

  4. Douglas J, Rachford H H. On the numerical solution of the heat conduction problem in 2 and 3 space variables. Trans Amer Math Soc, 1956, 82: 421–439

    Article  MathSciNet  MATH  Google Scholar 

  5. Eckstein J, Bertsekas D P. On the Douglas-Rachford splitting method and the proximal points algorithm for maximal monotone operators. Math Program, 1992, 55: 293–318

    Article  MathSciNet  MATH  Google Scholar 

  6. Facchinei F, Pang J S. Finite-Dimensional Variational Inequalities and Complementarity Problems. Vol. I. Springer Ser Oper Res. Berlin: Springer, 2003

    Google Scholar 

  7. Fukushima M. The primal Douglas-Rachford splitting algorithm for a class of monotone mappings with application to the traffic equilibrium problem. Math Program, 1996, 72: 1–15

    MathSciNet  MATH  Google Scholar 

  8. Gabay D, Mercier B. A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput Math Appl, 1976, 2: 17–40

    Article  MATH  Google Scholar 

  9. Glowinski R, Marrocco A. Approximation par éléments finis d’ordre un et résolution par pénalisation-dualité d’une classe de problémes non linéaires. R.A.I.R.O., R2, 1975, 41–76

    MathSciNet  Google Scholar 

  10. Gol’shtein E G, Tret’yakov N V. Modified Lagrangian in convex programming and their generalizations. Math Program Study, 1979, 10: 86–97

    Article  MathSciNet  Google Scholar 

  11. He B S, Yuan X M. On the O(1/n) convergence rate of Douglas-Rachford alternating direction method. SIAM J Numer Anal, 2012, 50: 700–709

    Article  MathSciNet  MATH  Google Scholar 

  12. He B S, Yuan X M, Zhang WX. A customized proximal point algorithm for convex minimization with linear constraints. Comput Optim Appl, under revision

  13. Hestenes M R. Multiplier and gradient methods. J Optim Theory Appl, 1969, 4: 303–320

    Article  MathSciNet  MATH  Google Scholar 

  14. Lions P L, Mercier B. Splitting algorithms for the sum of two nonlinear operators. SIAM J Numer Anal, 1979, 16: 964–979

    Article  MathSciNet  MATH  Google Scholar 

  15. Martinet B. Regularision d’inéquations variationnelles par approximations successive. Revue Francaise d’Automatique et Informatique Recherche Opérationnelle, 1970, 126: 154–159

    MathSciNet  Google Scholar 

  16. Powell M J D. A method for nonlinear constraints in minimization problems. In: Optimization. Fletcher R, ed. New York: Academic Press, 1969, 283–298

    Google Scholar 

  17. Rockafellar R T. Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math Oper Res, 1976, 1: 97–116

    Article  MathSciNet  MATH  Google Scholar 

  18. Rockafellar R T. Monotone operators and the proximal point algorithm. SIAM J Control Optim, 1976, 14: 877–898

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Nanjing University, Nanjing, 210093, China

    XingJu Cai, GuoYong Gu & BingSheng He

  2. Department of Mathematics, Hong Kong Baptist University, Hong Kong, 100002, China

    XiaoMing Yuan

Authors
  1. XingJu Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. GuoYong Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. BingSheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. XiaoMing Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to XiaoMing Yuan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cai, X., Gu, G., He, B. et al. A proximal point algorithm revisit on the alternating direction method of multipliers. Sci. China Math. 56, 2179–2186 (2013). https://doi.org/10.1007/s11425-013-4683-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11425-013-4683-0

Keywords

MSC(2010)

Search

Navigation