Global Convergence of Unmodified 3-Block ADMM for a Class of Convex Minimization Problems
The alternating direction method of multipliers (ADMM) has been successfully applied to solve structured convex optimization problems due to its superior practical performance. The convergence properties of the 2-block ADMM have been studied extensively in the literature. Specifically, it has been proven that the 2-block ADMM globally converges for any penalty parameter \(\gamma >0\). In this sense, the 2-block ADMM allows the parameter to be free, i.e., there is no need to restrict the value for the parameter when implementing this algorithm in order to ensure convergence. However, for the 3-block ADMM, Chen et al. (Math Program 155:57–79, 2016) recently constructed a counter-example showing that it can diverge if no further condition is imposed. The existing results on studying further sufficient conditions on guaranteeing the convergence of the 3-block ADMM usually require \(\gamma \) to be smaller than a certain bound, which is usually either difficult to compute or too small to make it a practical algorithm. In this paper, we show that the 3-block ADMM still globally converges with any penalty parameter \(\gamma >0\) if the third function \(f_3\) in the objective is smooth and strongly convex, and its condition number is in [1, 1.0798), besides some other mild conditions. This requirement covers an important class of problems to be called regularized least squares decomposition (RLSD) in this paper.
KeywordsADMM Global convergence Convex minimization Regularized least squares decomposition
The research of Shiqian Ma is supported in part by a startup package in Department of Mathematics at UC Davis. The research of Shuzhong Zhang is supported in part by NSF Grant with Grant No. CMMI-1462408.
- 5.Chen, C., Shen, Y., You, Y.: On the convergence analysis of the alternating direction method of multipliers with three blocks. Abstr. Appl. Anal. Article ID 183961 (2013)Google Scholar
- 6.Davis, D., Yin, W.: A three-operator splitting scheme and its optimization applications, pp. 15–13. Technical Report, UCLA CAM Report (2015)Google Scholar
- 11.Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary Value Problems. North-Holland, Amsterdam (1983)Google Scholar
- 19.Hong, M., Chang, T.-H., Wang, X., Razaviyayn, M., Ma, S., Luo, Z.-Q.: A block successive upper bound minimization method of multipliers for linearly constrained convex optimization. arXiv preprint arXiv:1401.7079 (2014)
- 28.Lin, Z., Chen, M., Wu, L., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC Technical Report UILU-ENG-09-2215. arXiv:1009.5055v2 (2009)
- 34.Sun, R., Luo, Z.-Q., Ye, Y.: On the expected convergence of randomly permuted ADMM. https://arxiv.org/abs/1503.06387 (2015)
- 38.Waters, A., Sankaranarayanan, A., Baraniuk, R.: Sparcs: recovering low-rank and sparse matrices from compressive measurements. In: NIPS (2011)Google Scholar
- 40.Wright, J., Ganesh, A., Min, K., Ma, Y.: Compressive principal component pursuit. Inf. Inference 2(1), 32–68 (2013)Google Scholar
- 42.Zhou, Z., Li, X., Wright, J., Candès, E.J., Ma, Y.: Stable principal component pursuit. In: Proceedings of International Symposium on Information Theory (2010)Google Scholar