Mathematical Programming

, Volume 64, Issue 1–3, pp 81–101 | Cite as

A proximal-based decomposition method for convex minimization problems

  • Gong Chen
  • Marc Teboulle
Article

Abstract

This paper presents a decomposition method for solving convex minimization problems. At each iteration, the algorithm computes two proximal steps in the dual variables and one proximal step in the primal variables. We derive this algorithm from Rockafellar's proximal method of multipliers, which involves an augmented Lagrangian with an additional quadratic proximal term. The algorithm preserves the good features of the proximal method of multipliers, with the additional advantage that it leads to a decoupling of the constraints, and is thus suitable for parallel implementation. We allow for computing approximately the proximal minimization steps and we prove that under mild assumptions on the problem's data, the method is globally convergent and at a linear rate. The method is compared with alternating direction type methods and applied to the particular case of minimizing a convex function over a finite intersection of closed convex sets.

AMS Subject Classification

90C25 

Key words

Convex programming Proximal methods Augmented Lagrangian Decomposition-splitting methods 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    A. Auslender, “Numerical methods for nondifferentiable convex optimization,”Mathematical Programming Study 30 (1987) 102–126.Google Scholar
  2. [2]
    D.P. Bertsekas,Constrained Optimization and Lagrangian Multipliers (Academic Press, New York, 1982).Google Scholar
  3. [3]
    D.P. Bertsekas and J.N. Tsitsiklis,Parallel and Distributed Computation: Numerical Methods (Prentice-Hall, Englewood Cliffs, NJ, 1989).Google Scholar
  4. [4]
    J. Eckstein and D.P. Bertsekas, “On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators,”Mathematical Programming 55 (1992) 293–318.Google Scholar
  5. [5]
    W. Findeisen, F.N. Bailey, M. Brdys, K. Malinowski, P. Tatjewski and A. Wozniak,Control and Coordination in Hierarchical Systems (Wiley, New York, 1980).Google Scholar
  6. [6]
    M. Fortin and R. Glowinski,Augmented Lagrangian Methods: Applications to the Solution of Boundary-Valued Problems (North-Holland, Amsterdam, 1983).Google Scholar
  7. [7]
    M. Fukushima, “Application of the alternating direction method of multipliers to separable convex programming problems,”Computational Optimization and Applications 1 (1992) 93–111.Google Scholar
  8. [8]
    D. Gabay, “Applications of the method of multipliers to variational inequalities,” in: M. Fortin and R. Glowinski, eds.,Augmented Lagrangian Methods: Applications to the Solution of Boundary-Valued Problems (North-Holland, Amsterdam, 1983) pp. 299–331.Google Scholar
  9. [9]
    D. Gabay and B. Mercier, “A dual algorithm for the solution of nonlinear variational problems via finiteelement approximations,”Computors and Mathematics with Applications 2 (1976) 17–40.Google Scholar
  10. [10]
    R. Glowinski and P. Le Tallec, “Augmented lagrangian and operator-splitting methods in nonlinear mechanics,” in:SIAM Studies in Applied Mathematics (SIAM, Philadelphia, PA, 1989).Google Scholar
  11. [11]
    R. Glowinski and A. Marrocco, “Sur l'approximation par éléments finis d'ordre un, et la résolution par pénalisation-dualité d'une classe de problèmes de Dirichlet nonlinéaires,”Revue Française d'Automatique, Informatique et Recherche Opérationelle 2 (1975) 41–76.Google Scholar
  12. [12]
    O. Guler, “On the convergence of the proximal point algorithm for convex minimization,”SIAM Journal on Control and Optimization 29 (1991) 403–419.Google Scholar
  13. [13]
    S.P. Han and G. Lou, “A parallel algorithm for a class of convex programs,”SIAM Journal on Control and Optimization 26 (1988) 345–355.Google Scholar
  14. [14]
    S. Ibaraki, M. Fukushima and T. Ibaraki, “Primal—dual proximal point algorithm for linearly constrained convex programming problems,” Technical Report, No. 91016, Kyoto University (Kyoto, Japan, 1991).Google Scholar
  15. [15]
    A.N. Iusem and A.R. De Pierro, “On the convergence of Han's method for convex programming with quadratic objective,”Mathematical Programming 52 (1991) 265–284.Google Scholar
  16. [16]
    L.S. Lasdon,Optimization Theory for Large Systems (Macmillan, New York, 1970).Google Scholar
  17. [17]
    B. Lemaire, “The proximal algorithm,”International Series of Numerical Mathematics 87 (1989) 73–87.Google Scholar
  18. [18]
    P.L. Lions and B. Mercier, “Splitting algorithms for the sum of two nonlinear operators,”SIAM Journal on Numerical Analysis 16 (1979) 964–979.Google Scholar
  19. [19]
    K. Mouallif, V.H. Nguyen and J.J. Strodiot, “A perturbed parallel decomposition method for a class of nonsmooth convex minimization problems,”SIAM Journal on Control and Optimization 29 (1991) 829–847.Google Scholar
  20. [20]
    B.T. Polyak,Introduction to Optimization (Optimization Software, New York, 1987).Google Scholar
  21. [21]
    R.T. Rockafellar,Convex Analysis (Princeton University Press, Princeton, NJ, 1970).Google Scholar
  22. [22]
    R.T. Rockafellar, “Augmented lagrangians and applications of the proximal point algorithm in convex programming,”Mathematics of Operations Research 1 (1976) 97–116.Google Scholar
  23. [23]
    R.T. Rockafellar, “Monotone operators and the proximal point algorithm,”SIAM Journal on Control and Optimization 14 (1976) 877–898.Google Scholar
  24. [24]
    J.E. Spingarn, “Applications of the method of partial inverses to convex programming: decomposition,”Mathematical Programming 32 (1985) 199–223.Google Scholar
  25. [25]
    G. Stephanopoulos and A.W. Westerberg, “The use of Hestenes' method of multipliers to resolve dual gaps in engineering system optimization,”Journal of Optimization Theory and Applications 15 (1975) 285–309.Google Scholar
  26. [26]
    A. Tanikawa and H. Mukai, “A new technique for nonconvex primal—dual decomposition,”IEEE Transactions on Automatic Control AC-30 (1985) 133–143.Google Scholar
  27. [27]
    P. Tseng, “Applications of a splitting algorithm to decomposition in convex programming and variational inequalities,”SIAM Journal on Control and Optimization 29 (1991) 119–138.Google Scholar
  28. [28]
    H. Uzawa, “Iterative methods for concave programming,” in: K.J. Arrow, L. Hurwicz and H. Uzawa, eds.,Studies in Linear and Nonlinear Programming (Stanford University Press, Stanford, CA, 1958) pp. 154–165.Google Scholar

Copyright information

© The Mathematical Programming Society, Inc 1994

Authors and Affiliations

  • Gong Chen
    • 1
  • Marc Teboulle
    • 1
  1. 1.Department of Mathematics and StatisticsUniversity of Maryland, Baltimore County CampusBaltimoreUSA

Personalised recommendations