# On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators

- 3.6k Downloads
- 983 Citations

## Abstract

This paper shows, by means of an operator called a*splitting operator,* that the Douglas—Rachford splitting method for finding a zero of the sum of two monotone operators is a special case of the proximal point algorithm. Therefore, applications of Douglas—Rachford splitting, such as the alternating direction method of multipliers for convex programming decomposition, are also special cases of the proximal point algorithm. This observation allows the unification and generalization of a variety of convex programming algorithms. By introducing a modified version of the proximal point algorithm, we derive a new,*generalized* alternating direction method of multipliers for convex programming. Advances of this sort illustrate the power and generality gained by adopting monotone operator theory as a conceptual framework.

## Key words

Monotone operators proximal point algorithm decomposition## Preview

Unable to display preview. Download preview PDF.

## References

- [1]D.P. Bertsekas, “Necessary and sufficient conditions for a penalty method to be exact,”
*Mathematical Programming*9 (1975) 87–99.Google Scholar - [2]D.P. Bertsekas,
*Constrained Optimization and Lagrange Multiplier Methods*(Academic Press, New York, 1982).Google Scholar - [3]D.P. Bertsekas and J. Tsitsiklis,
*Parallel and Distributed Computation: Numerical Methods*(Prentice-Hall, Englewood Cliffs, NJ, 1989).Google Scholar - [4]H. Brézis,
*Opérateurs Maximaux Monotones et Semi-Groupes de Contractions dans les Espaces de Hilbert*(North-Holland, Amsterdam, 1973).Google Scholar - [5]H. Brézis and P.-L. Lions, “Produits infinis de resolvantes,”
*Israel Journal of Mathematics*29 (1978) 329–345.Google Scholar - [6]V. Doležal,
*Monotone Operators and Applications in Control and Network Theory*(Elsevier, Amsterdam, 1979).Google Scholar - [7]J. Douglas and H.H. Rachford, “On the numerical solution of heat conduction problems in two and three space variables,”
*Transactions of the American Mathematical Society*82 (1956) 421–439.Google Scholar - [8]R. Durier, “On locally polyhedral convex functions,” Working paper, Université de Dijon (Dijon, 1986).Google Scholar
- [9]R. Durier and C. Michelot, “Sets of efficient points in a normed space,”
*Journal of Mathematical Analysis and its Applications*117 (1986) 506–528.Google Scholar - [10]J. Eckstein, “The Lions—Mercier algorithm and the alternating direction method are instances of the proximal point algorithm,” Report LIDS-P-1769, Laboratory for Information and Decision Sciences, MIT (Cambridge, MA, 1988).Google Scholar
- [11]J. Eckstein, “Splitting methods for monotone operators with applications to parallel optimization,” Doctoral dissertation, Department of Civil Engineering, Massachusetts Institute of Technology. Available as Report LIDS-TH-1877, Laboratory for Information and Decision Sciences, MIT (Cambridge, MA, 1989).Google Scholar
- [12]M. Fortin and R. Glowinski, “On decomposition-coordination methods using an augmented Lagrangian,” in: M. Fortin and R. Glowinski, eds.,
*Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems*(North-Holland, Amsterdam, 1983).Google Scholar - [13]D. Gabay, “Applications of the method of multipliers to variational inequalities,” in: M. Fortin and R. Glowinski, eds.,
*Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems*(North-Holland, Amsterdam, 1983).Google Scholar - [14]D. Gabay and B. Mercier, “A dual algorithm for the solution of nonlinear variational problems via finite element approximations,”
*Computers and Mathematics with Applications*2 (1976) 17–40.Google Scholar - [15]R. Glowinski and P. Le Tallec, “Augmented Lagrangian methods for the solution of variational problems,” MRC Technical Summary Report #2965, Mathematics Research Center, University of Wisconsin-Madison (Madison, WI, 1987).Google Scholar
- [16]R. Glowinski and A. Marroco, “Sur l'approximation, par elements finis d'ordre un, et la resolution, par penalisation-dualité, d'une classe de problemes de Dirichlet non lineares,”
*Revue Française d'Automatique, Informatique et Recherche Opérationelle*9(R-2) (1975) 41–76.Google Scholar - [17]E.G. Gol'shtein, “Method for modification of monotone mappings,”
*Ekonomika i Matemacheskaya Metody*11 (1975) 1142–1159.Google Scholar - [18]E.G. Gol'shtein, “Decomposition methods for linear and convex programming problems,”
*Matekon*22(4) (1985) 75–101.Google Scholar - [19]E.G. Gol'shtein, “The block method of convex programming,”
*Soviet Mathematics Doklady*33 (1986) 584–587.Google Scholar - [20]E.G. Gol'shtein, “A general approach to decomposition of optimization systems,”
*Soviet Journal of Computer and Systems Sciences*25(3) (1987) 105–114.Google Scholar - [21]E.G. Gol'shtein and N.V. Tret'yakov, “The gradient method of minimization and algorithms of convex programming based on modified Lagrangian functions,”
*Ekonomika i Matemacheskaya Metody*11(4) (1975) 730–742.Google Scholar - [22]E.G. Gol'shtein and N.V. Tret'yakov, “Modified Lagrangians in convex programming and their generalizations,”
*Mathematical Programming Study*10 (1979) 86–97.Google Scholar - [23]M.R. Hestenes, “Multiplier and gradient methods,”
*Journal of Optimization Theory and Applications*4 (1969) 303–320.Google Scholar - [24]M.C. Joshi and R.K. Bose,
*Some Topics in Nonlinear Functional Analysis*(Halsted/Wiley, New Delhi, 1985).Google Scholar - [25]R.I. Kachurovskii, “On monotone operators and convex functionals,”
*Uspekhi Matemacheskaya Nauk*15(4) (1960) 213–215.Google Scholar - [26]R.I. Kachurovskii, “Nonlinear monotone operators in Banach space,”
*Russian Mathematical Surveys*23(2) (1968) 117–165.Google Scholar - [27]B. Kort and D.P. Bertsekas, “Combined primal—dual and penalty methods for convex programming,”
*SIAM Journal on Control and Optimization*14 (1976) 268–294.Google Scholar - [28]J. Lawrence and J.E. Spingarn, “On fixed points of non-expansive piecewise isometric mappings,”
*Proceedings of the London Mathematical Society*55 (1987) 605–624.Google Scholar - [29]O. Lefebvre and C. Michelot, “About the finite convergence of the proximal point algorithm,” in: K.-H. Hoffmann et al., eds.,
*Trends in Mathematical Optimization: 4th French—German Conference on Optimization. International Series of Numerical Mathematics No. 84*(Birkhäuser, Basel, 1988).Google Scholar - [30]P.-L. Lions, “Une méthode itérative de resolution d'une inequation variationnelle,”
*Israel Journal of Mathematics*31 (1978) 204–208.Google Scholar - [31]P.-L. Lions and B. Mercier, “Splitting algorithms for the sum of two nonlinear operators,”
*SIAM Journal on Numerical Analysis*16 (1979) 964–979.Google Scholar - [32]F.J. Luque, “Asymptotic convergence analysis of the proximal point algorithm,”
*SIAM Journal on Control and Optimization*22 (1984) 277–293.Google Scholar - [33]G.I. Marchuk,
*Methods of Numerical Mathematics*(Springer, New York, 1975).Google Scholar - [34]B. Martinet, “Regularisation d'inequations variationelles par approximations successives,”
*Revue Française d'Informatique et de Recherche Operationelle*4(R-3) (1970) 154–158.Google Scholar - [35]B. Martinet, “Determination approchée d'un point fixe d'une application pseudo-contractante. Cas de l'application prox,”
*Comptes Rendus de l'Academie des Sciences, Paris, Série A*274 (1972) 163–165.Google Scholar - [36]G.J. Minty, “On the maximal domain of a ‘monotone’ function,”
*Michigan Mathematical Journal*8 (1961) 135–137.Google Scholar - [37]G.J. Minty, “Monotone (nonlinear) operators in Hilbert space,”
*Duke Mathematics Journal*29 (1962) 341–346.Google Scholar - [38]G.J. Minty, “On the monotonicity of the gradient of a convex function,”
*Pacific Journal of Mathematics*14 (1964) 243–247.Google Scholar - [39]D. Pascali and S. Sburlan,
*Nonlinear Mappings of Monotone Type*(Editura Academeie, Bucarest, 1978).Google Scholar - [40]G.B. Passty, “Ergodic convergence to a zero of the sum of monotone operators in Hilbert space,”
*Journal of Mathematical Analysis and Applications*72 (1979) 383–390.Google Scholar - [41]M.J.D. Powell, “A method for nonlinear constraints in minimization problems,” in: R. Fletcher, ed.,
*Optimization*(Academic Press, New York, 1969).Google Scholar - [42]R.T. Rockafellar, “Characterization of the subdifferentials of convex functions,”
*Pacific Journal of Mathematics*17 (1966) 497–510.Google Scholar - [43]R.T. Rockafellar, “Local boundedness of nonlinear, monotone operators,”
*Michigan Mathematical Journal*16 (1969) 397–407.Google Scholar - [44]R.T. Rockafellar,
*Convex Analysis*(Princeton University Press, Princeton, NJ, 1970).Google Scholar - [45]R.T. Rockafellar, “On the maximal monotonicity of subdifferential mappings,”
*Pacific Journal of Mathematics*33 (1970) 209–216.Google Scholar - [46]R.T. Rockafellar, “On the maximality of sums of nonlinear monotone operators,”
*Transactions of the American Mathematical Society*149 (1970) 75–88.Google Scholar - [47]R.T. Rockafellar, “On the virtual convexity of the domain and range of a nonlinear maximal monotone operator,”
*Mathematische Annalen*185 (1970) 81–90.Google Scholar - [48]R.T. Rockafellar, “Monotone operators and the proximal point algorithm,”
*SIAM Journal on Control and Optimization*14 (1976) 877–898.Google Scholar - [49]R.T. Rockafellar, “Augmented Lagrangians and applications of the proximal point algorithm in convex programming,”
*Mathematics of Operations Research*1 (1976) 97–116.Google Scholar - [50]R.T. Rockafellar, “Monotone operators and augmented Lagrangian methods in nonlinear programming,” in: O.L. Mangasarian, R.M. Meyer and S.M. Robinson, eds.,
*Nonlinear Programming 3*(Academic Press, New York, 1977).Google Scholar - [51]R.T. Rockafellar and R.J.-B. Wets, “Scenarios and policy aggregation in optimization under uncertainty,”
*Mathematics of Operations Research*16(1) (1991) 119–147.Google Scholar - [52]J.E. Spingarn, “Partial inverse of a monotone operator,”
*Applied Mathematics and Optimization*10 (1983) 247–265.Google Scholar - [53]J.E. Spingarn, “A primal—dual projection method for solving systems of linear inequalities,”
*Linear Algebra and its Applications*65 (1985) 45–62.Google Scholar - [54]J.E. Spingarn, “Application of the method of partial inverses to convex programming: decomposition,”
*Mathematical Programming*32 (1985) 199–233.Google Scholar - [55]J.E. Spingarn, “A projection method for least-squares solutions to overdetermined systems of linear inequalities,”
*Linear Algebra and its Applications*86 (1987) 211–236.Google Scholar - [56]P. Tseng, “Applications of a splitting algorithm to decomposition in convex programming and variational inequalities,”
*SIAM Journal on Control and Optimization*29(1) (1991) 119–138.Google Scholar