Abstract
We consider the alternating direction method of multipliers decomposition algorithm for convex programming, as recently generalized by Eckstein and Bert- sekas. We give some reformulations of the algorithm, and discuss several alternative means for deriving them. We then apply these reformulations to a number of optimization problems, such as the minimum convex-cost transportation and multicommodity flow. The convex transportation version is closely related to a linear-cost transportation algorithm proposed earlier by Bertsekas and Tsitsiklis. Finally, we construct a simple data-parallel implementation of the convex-cost transportation algorithm for the CM-5 family of parallel computers, and give computational results. The method appears to converge quite quickly on sparse quadratic-cost transportation problems, even if they are very large; for example, we solve problems with over a million arcs in roughly 100 iterations, which equates to about 30 seconds of run time on a system with 256 processing nodes. Substantially better timings can probably be achieved with a more careful implementation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ahuja, R. K., Magnanti, T. L., and Orlin, J. B. (1989), “Network Flows,” in Optimization, Handbooks in Operations Research and Management Science, Vol. 1, A. H. G. Rinooy Kan and M. J. Todd eds., North-Holland, Amsterdam, 211–369.
Bertsekas, D. P. and Tsitsiklis, J. N. (1989), Parallel and Distributed,Computation: Numerical Methods, Prentice-Hall, Englewood Cliffs, New Jersey.
Blelloch, G. E. (1990), Vector Models for Data-Parallel Computing, MIT Press, Cambridge, Massachusetts.
Brézis, H. (1973), Opérateurs Maximaux Monotones et Semi-groupes de Contrac¬tions dans les Espaces de Hilbert, North-Holland, Amsterdam.
Eckstein, J. (1989), Splitting Methods for Monotone Operators with Applications to Parallel Optimization, Ph. D. thesis, Department of Civil Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts.
Eckstein, J. and Bertsekas, D. P. (1992), “On the Douglas-Rachford Splitting Method and the Proximal Point Algorithm for Maximal Monotone Operators,” Mathematical Programming, Vol. 55, 293–318.
Eckstein, J. (1993), “The Alternating Step Method for Monotropic Programming on the Connection Machine CM-2,” ORSA Journal on Computing, Vol. 5, 84–96.
Eckstein, J. (1994), “Alternating Direction Multiplier Decomposition of Convex Programs,” Journal of Optimization Theory and Applications, Vol. 80, to appear.
Fortin, M. and Glowinski, R. (1983), “On Decomposition-Coordination Methods using an Augmented Lagrangian,” Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems, M. Fortin and R. Glowinski eds., North-Holland, Amsterdam, 97–146.
Fukushima, M. (1992), “Application of the Alternating Direction Method of Multipliers to Separable Convex Programming Problems”, Computational Optimization and Applications, Vol. 1, 93–112.
Gabay, D. (1983), “Applications of the Method of Multipliers to Variational Inequalities,” Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, M. Fortin and R. Glowinski (eds.), North Holland, Amsterdam, 299–331.
Gabay, D. and Mercier, B. (1976), “A Dual Approach for the Solution of Nonlinear Variational Problems via Finite Element Approximation,” Computers and Mathematics with Applications, Vol. 2, 17–40.
Glowinski, R. and Le Tallec, P. (1989), Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics, SIAM, Philadelphia, Pennsylvania.
Glowinski, R. and Marroco, A. (1975), “Sur l’Approximation, par Elements Finis d’Ordre Un, et la Resolution, par Pénalisation-Dualité, d’une Classe de Problèmes de Dirichlet non Lineares,” Revue Française dfAutomatique, Informatique et Recherche Opérationelle, Vol. 9 (R-2), 41–76.
Ibaraki, T. and Katoh, N. (1988), Resource Allocation Problems: Algorithmic Approaches, MIT Press, Cambridge, Massachusetts.
Klingman, D., Napier, A., and Stutz, J. (1974), “NETGEN — A Program for Generating Large-Scale (Un)Capacitated Assignment, Transportation, and Minimum Cost Network Problems,” Management Science, Vol. 20, 814–822.
Lions, P.-L. and Mercier, B. (1979), “Splitting Algorithms for the Sum of Two Nonlinear Operators,” SIAM Journal on Numerical Analysis, Vol. 16, 964 - 979.
Moré, J. J. (1990), “On the Performance of Algorithms for Large-Scale Bound- Constrainted Problems,” Large-Scale Numerical Optimization, T. F. Coleman and Y. Li eds., SIAM, Philadelphia, Pennsylvania, 32–45.
Nielsen, S. S. and Zenios, S. A. (1992), “Massively Parallel Algorithms for Singly Constrained Convex Programs,” ORSA Journal on Computing, Vol. 4, 166–181.
Rockafellar, R. T. (1970), Convex Analysis, Princeton University Press, Princeton, New Jersey.
Rockafellar, R. T. (1976), “Monotone Operators and the Proximal Point Algorithm,” SIAM Journal on Control and Optimization, Vol. 14, 877–898.
Rockafellar, R. T. (1976), “Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming,” Mathematics of Operations Research, Vol. 1, 97–116.
Thinking Machines Corporation (1992), The Connection Machine CM-5 Technical Summary, Cambridge, Massachusetts.
Tseng, P. (1990), “Dual Ascent Methods for Problems with Strictly Convex Costs and Linear Constraints: a Unified Approach,” SIAM Journal on Control and Optimization, Vol. 28, 214–242.
Zenios, S. A. and Censor, Y. (1991), “Massively Parallel Row Action Algorithms for Some Nonlinear Transportation Problems,” SIAM Journal on Optimization, Vol. 3, 373–400.
Zenios, S. A. and Lasken, R. A. (1988), “Nonlinear Network Optimization on a Massively Parallel Connection Machine,” Annals of Operations Research, Vol. 14, 147–165.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Kluwer Academic Publishers
About this chapter
Cite this chapter
Eckstein, J., Fukushima, M. (1994). Some Reformulations and Applications of the Alternating Direction Method of Multipliers. In: Hager, W.W., Hearn, D.W., Pardalos, P.M. (eds) Large Scale Optimization. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-3632-7_7
Download citation
DOI: https://doi.org/10.1007/978-1-4613-3632-7_7
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-3634-1
Online ISBN: 978-1-4613-3632-7
eBook Packages: Springer Book Archive