Abstract
Alternating direction method of multipliers has been well studied in the context of linearly constrained convex optimization. In the last few years, we have witnessed a number of novel applications arising from image processing, compressive sensing and statistics, etc., where the approach is surprisingly efficient. In the early applications, the objective function of the linearly constrained convex optimization problem is separable into two parts. Recently, the alternating direction method of multipliers has been extended to the case where the number of the separable parts in the objective function is finite. However, in each iteration, the subproblems are required to be solved exactly. In this paper, by introducing some reasonable inexactness criteria, we propose two inexact alternating-direction-based contraction methods, which substantially broaden the applicable scope of the approach. The convergence and complexity results for both methods are derived in the framework of variational inequalities.
Similar content being viewed by others
References
Floudas, C.A., Pardalos, P.M. (eds.): Encyclopedia of Optimization, 2nd edn. Springer, Berlin (2009)
Pardalos, P.M., Resende, M.G.C. (eds.): Handbook of Applied Optimization. Oxford University Press, Oxford (2002)
Ng, M.K., Weiss, P., Yuan, X.: Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods. SIAM J. Sci. Comput. 32(5), 2710–2736 (2010)
Yang, J., Zhang, Y.: Alternating direction algorithms for ℓ 1-problems in compressive sensing. SIAM J. Sci. Comput. 33(1), 250–278 (2011)
Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Program. Comput. 2(3–4), 203–230 (2010)
He, B., Xu, M., Yuan, X.: Solving large-scale least squares semidefinite programming by alternating direction methods. SIAM J. Matrix Anal. Appl. 32(1), 136–152 (2011)
Yuan, X.: Alternating direction method for covariance selection models. J. Sci. Comput. 51(2), 261–273 (2012)
Yuan, X., Yang, J.: Sparse and low-rank matrix decomposition via alternating direction method. Pac. J. Optim. 9(1), 167–180 (2013)
Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Computer Science and Applied Mathematics. Academic Press/Harcourt Brace Jovanovich Publishers, New York (1982)
Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer Series in Operations Research and Financial Engineering. Springer, New York (2006)
Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
Chan, R.H., Yang, J., Yuan, X.: Alternating direction method for image inpainting in wavelet domains. SIAM J. Imaging Sci. 4(3), 807–826 (2011)
Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)
Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)
Douglas, J. Jr., Rachford, H.H. Jr.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)
He, B., Tao, M., Xu, M., Yuan, X.: An alternating direction-based contraction method for linearly constrained separable convex programming problems. Optimization 62(4), 573–596 (2013)
Blum, E., Oettli, W.: Grundlagen und Verfahren. Springer, Berlin (1975). Grundlagen und Verfahren, Mit einem Anhang “Bibliographie zur Nichtlinearer Programmierung”, Ökonometrie und Unternehmensforschung, No. XX
Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Prentice-Hall, Upper Saddle River (1989)
He, B., Liao, L., Qian, M.: Alternating projection based prediction-correction methods for structured variational inequalities. J. Comput. Math. 24(6), 693–710 (2006)
He, B., Liao, L., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program., Ser. A 92(1), 103–118 (2002)
Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program., Ser. A 64(1), 81–101 (1994)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)
Teboulle, M.: Convergence of proximal-like algorithms. SIAM J. Optim. 7(4), 1069–1083 (1997)
Tseng, P.: Alternating projection-proximal methods for convex programming and variational inequalities. SIAM J. Optim. 7(4), 951–965 (1997)
He, B., Xu, M.: A general framework of contraction methods for monotone variational inequalities. Pac. J. Optim. 4(2), 195–212 (2008)
Nemirovski, A.: Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15(1), 229–251 (2004) (Electronic)
Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. I. Springer Series in Operations Research. Springer, New York (2003)
Cai, X., Gu, G., He, B.: On the O(1/t) convergence rate of the projection and contraction methods for variational inequalities with Lipschitz continuous monotone operators. Comput. Optim. Appl. (2013). doi:10.1007/s10589-013-9599-7
Acknowledgements
Authors wish to thank Prof. Panos M. Pardalos (the associate editor) and Prof. Cornelis Roos (Delft University of Technology) for useful comments and suggestions.
G. Gu was supported by the NSFC grant 11001124.
B. He was supported by the NSFC grant 91130007.
J. Yang was supported by the NSFC grant 11371192.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Panos M. Pardalos.
Rights and permissions
About this article
Cite this article
Gu, G., He, B. & Yang, J. Inexact Alternating-Direction-Based Contraction Methods for Separable Linearly Constrained Convex Optimization. J Optim Theory Appl 163, 105–129 (2014). https://doi.org/10.1007/s10957-013-0489-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-013-0489-z