Abstract
A parallel Uzawa-type algorithm, for solving unconstrained minimization of large-scale partially separable functions, is presented. Using auxiliary unknowns, the unconstrained minimization problem is transformed into a (linearly) constrained minimization of a separable function.The augmented Lagrangian of this problem decomposes into a sum of partially separable augmented Lagrangian functions. To take advantage of this property, a Uzawa block relaxation is applied. In every iteration, unconstrained minimization subproblems are solved in parallel before updating Lagrange multipliers. Numerical experiments show that the speed-up factor gained using our algorithm is significant.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bouaricha, A., Moré, J.J.: Impact of partial separability on large-scale optimization. Comput. Optim. Appl. 7, 27–40 (1997)
Conforti, D., Musmano, R.: Parallel algorithm for unconstrained optimization based on decomposition techniques. J. Optim. Theory Appl. 95, 531–544 (1997)
Fukushima, M.: Parallel variable transformation in unconstrained optimization. SIAM J. Optim. 8, 658–672 (1998)
Kibardin, V.M.: Decomposition into functions in minimization problem. Autom. Remote Control 40, 1311–1323 (1980)
Liu, C.S., Tseng, S.H.: Parallel synchronous and asynchronous space-decomposition algorithms for large-scale minimizations problems. Comput. Optim. Appl. 17, 85–107 (2000)
Mangasarian, O.: Parallel gradient distribution in uncostrained optimization. SIAM J. Control Optim. 33, 1916–1925 (1995)
Phua, P.K.H., Fan, W., Zeng, Y.: Parallel algorithms for large scale nonlinear optimization. Int. Trans. Oper. Res. 5, 67–77 (1998)
Koko, J., Moukrim, A.: Parallel implementation of a generalized conjugate gradient algorithm. Informatica 9, 437–448 (1998)
Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Application to the Numerical Solution of Boundary-Value Problems. North-Holland, Amsterdam (1983)
Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. Studies in Applied Mathematics. SIAM, Philadelphia (1989)
Gunzburger, M.D., Heinkenschloss, M., Lee, H.K.: Solution of elliptic partial differential equations by an optimization-based domain decomposition method. Appl. Math. Comput. 113, 111–139 (2000)
Gunzburger, M.D., Peterson, J., Kwon, H.: An optimization based domain decomposition method for partial differential equations. Comput. Math. Appl. 37(10), 77–93 (1999)
Koko, J.: Lagrange multiplier based domain decomposition methods for a nonlinear sedimentary basin problem. Comput. Geosci. 11, 307–317 (2007)
Koko, J.: Uzawa block relaxation domain decomposition method for the two-body contact problem with Tresca friction. Comput. Methods Appl. Mech. Eng. 198, 420–431 (2008)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011)
Nocedal, J., Wright, S.: Numerical Optimization. Springer, Berlin (2006)
Quarteroni, A., Valli, A.: Domain Decomposition Methods for Partial Differential Equations. Oxford University Press, London (1999)
Conn, A.R., Gould, N.I.M., Toint, P.: Testing a class of methods for solving minimization problems with simple bounds on variables. Math. Comput. 50, 399–430 (1988)
Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Springer, Berlin (1984)
Cea, J., Glowinski, R.: Sur des méthodes d’optimisation par relaxation. ESAIM: Math. Model. Numer. Anal. R-3, 5–32 (1973)
Byrd, R.H., Lu, P., Nodedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16(5), 1190–1208 (1995)
Zhu, C., Byrd, R.H., Lu, P., Nocedal, J.: L-BFGS-B: a limited memory fortran code for solving bound constrained optimization problems. Technical report NAM-11, EECS Department, Northwestern University (1994)
Buckley, A., Lenir, A.: QN-link variable storage conjugate gradients. Math. Program. 27, 155–175 (1983)
Dixon, L.C., Price, R.C.: Numerical experience with truncated Newton method. Technical report 169, Numerical Optimization Center, Hatfield Polytecnic (1986)
Lukšan, L., Vlček, J.: Sparse and partially separable test problems for unconstrained and equality constrained optimization. Technical report 767, Institute of Computer Science, Academy of Sciences of the Czech Republic (1999)
Moré, J.J., Garbow, B.S., Hillström, K.E.: Testing unconstrained optimization sofware. ACM Trans. Math. Softw. 7, 17–41 (1981)
Gill, P.E., Murray, W.: The numerical solution of problem in the calculus of variations. Technical report NAC-28, National Physical Laboraty, Teddington, England (1972)
Chandra, K.S., Rao, M.V.C.: A direct search package for unconstrained minimization. Int. J. Numer. Methods Eng. 20, 1643–1660 (1984)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Masao Fukushima.
Rights and permissions
About this article
Cite this article
Koko, J. Parallel Uzawa Method for Large-Scale Minimization of Partially Separable Functions. J Optim Theory Appl 158, 172–187 (2013). https://doi.org/10.1007/s10957-012-0059-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-012-0059-9