Abstract
This paper describes two specializations of the alternating direction method of multipliers: the alternating step method and the epigraphic projection method. The alternating step method applies to monotropic programs, while the epigraphic method applies to general block-separable convex programs, including monotropic programs as a special case. The epigraphic method resembles an earlier parallel method due to Spingarn, but solves a larger number of simpler subproblems at each iteration. This paper gives convergence results for both the alternating step and epigraphic methods, and compares their performance on random dense separable quadratic programs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bertsekas, D. P.,Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, New York, 1982.
Glowinski, R., andMarroco, A.,Sur l'Approximation, par Elements Finis d'Ordre Un, et la Resolution, par Penalisation-Dualité, d'une Classe de Problèmes de Dirichlet Nonlineares, Revue Française d'Automatique, Informatique et Recherche Opérationelle, Series R, Vol. 9, pp. 41–76 1975.
Gabay, D., andMercier, B.,A Dual Algorithm for the Solution of Nonlinear Variational Problems via Finite Element Approximations, Computers and Mathematics with Applications, Vol. 2, pp. 17–40, 1976.
Fortin, M., andGlowinski, R.,On Decomposition-Coordination Methods Using an Augmented Lagrangian, Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems, Edited by M. Fortin and R. Glowinski, North-Holland, Amsterdam, Holland, pp. 97–145, 1983.
Gabay, D.,Applications of the Method of Multipliers to Variational Inequalities, Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems, Edited by M. Fortin and R. Glowinski, North-Holland, Amsterdam, Holland, pp. 299–331, 1983.
Lions, P. L., andMercier, B.,Splitting Algorithms for the Sum of Two Nonlinear Operators, SIAM Journal on Numerical Analysis, Vol. 16, pp. 964–979, 1979.
Eckstein, J., andBertsekas, D. P.,On the Douglas-Rachford Splitting Method and the Proximal Point Algorithm for Maximal Monotone Operators, Mathematical Programming, Vol. 55, pp. 293–318, 1992.
Glowinski, R., andLe Tallec, P.,Augmented Lagrangian Methods for the Solution of Variational Problems, MRC Technical Summary Report No. 2965, University of Wisconsin, 1987.
Bertsekas, D. P., andTsitsiklis, J.,Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, Englewood Cliffs, New Jersey, 1989.
Rockafellar, R. T.,Network Flows and Monotropic Optimization, John Wiley, New York, New York, 1984.
Eckstein, J.,Splitting Methods for Monotone Operators with Applications to Parallel Optimization, PhD Thesis, Massachusetts Institute of Technology, 1989.
Spingarn, J. E.,Application of the Method of Partial Inverses to Convex Programming: Decomposition, Mathematical Programming, Vol. 32, pp. 199–233, 1985.
Rockafellar, R. T.,Convex Analysis, Princeton University Press, Princeton, New Jersey, 1970.
Eckstein, J.,The Alternating Step Method for Monotropic Programming on the Connection Machine CM-2, ORSA Journal on Computing, Vol. 5, pp. 84–96, 1993.
Nielsen, S. S., andZenios, S. A.,Massively Parallel Algorithms for Singly Constrained Nonlinear Programs, ORSA Journal on Computing, Vol. 4, pp. 166–1981, 1992.
Zenios, S. A., andCensor, Y.,Massively Parallel Row Action Algorithms for Some Nonlinear Transportation Problems, SIAM Journal on Optimization, Vol. 3, pp. 373–400, 1991.
Eckstein, J., andFerris, M. C.,Operator Splitting Methods for Monotone Linear Complementarity Problems, Technical Report TMC-239, Thinking Machines Corporation, 1992.
Eckstein, J., andBertsekas, D. P.,An Alternating Direction Method for Linear Programming, Working Paper 90-063, Harvard Business School, 1990.
Klingman, D., Napier, A., andStutz, J.,NETGEN: A Program for Generating Large-Scale Uncapacitated Assignment, Transportation, and Minimum Cost Network Problems, Management Science, Vol. 20, pp. 814–822, 1974.
Rockafellar, R. T.,Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming, Mathematics of Operations Research, Vol. 1, pp. 97–116, 1976.
Spingarn, J. E.,Partial Inverse of a Monotone Operator, Applied Mathematics and Optimization, Vol. 10, pp. 247–265, 1983.
Author information
Authors and Affiliations
Additional information
Communicated by O. L. Mangasarian
Some of the research described here was performed at the Massachusetts Institute of Technology and was supported by the Army Research Office under Grant DAAL03-86-K-0171 and the National Science Foundation under Grant ECS-85-19058. This portion of the work was supervised by Dimitri P. Bertsekas, for whose support the author is grateful.
Rights and permissions
About this article
Cite this article
Eckstein, J. Parallel alternating direction multiplier decomposition of convex programs. J Optim Theory Appl 80, 39–62 (1994). https://doi.org/10.1007/BF02196592
Issue Date:
DOI: https://doi.org/10.1007/BF02196592