Abstract
Recent years have witnessed the rapid development of the first order methods for multi-block separable programming involving large-scale date-set. The purpose of this paper is to introduce two new first order methods, the proximal splitting methods (PSMs), for the model under consideration. The first PSM fully utilizes the desired property of such problems and adopts fully Jacobian updating rule, which often results in easy subproblems in practice. The global convergence and the worst-case \(\mathcal {O}(1/t)\) convergence rate in an ergodic sense of the first PSM are proved under the condition that the involved functions are assumed to be strongly convex. Applying the hybrid Jacobian and Gauss–Seidel updating rule to the first PSM, we derive the second PSM, whose global convergence can be guaranteed only under the condition that the involved functions are convex. Furthermore, its worst-case \(\mathcal {O}(1/t)\) convergence rate in both the ergodic and non-ergodic senses is also established. Finally, numerical results on stable principal component pursuit are reported to testify the accuracy and speed of the second PSM, and some numerical comparisons are also reported.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)
Martinet, B.: Regularision d’inéquations variationnelles par approximations successive. Revue Francaise d’Automatique et Informatique Recherche Opérationnelle. 126, 154–159 (1970)
Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97–116 (1976)
Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal points algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)
Hou, L.S., He, H.J., Yang, J.F.: A partially parallel splitting method for multiple-block separable convex programming with applications to robust PCA. Comput. Optim. Appl. 63(1), 273–303 (2016)
He, B.S., Tao, M., Yuan, X.M.: Alternating direction method with Gaussian-back substitution for separable convex programming. SIAM J. Optim. 22, 313–340 (2012)
He, B.S., Hou, L.S., Yuan, X.M.: On full Jacobian decomposition of the augmented lagrangian method for separable convex programming. SIAM J. Optim. 25(4), 2274–2312 (2015)
Han, D.R., Yuan, X.M.: A note on the alternating direction method of multipliers. J. Optim. Theory Appl. 155, 227–238 (2012)
Wang, K., Desai, J., He, H.J.: A note on augmented Lagrangian-based parallel splitting method. Optim. Lett. 9, 1199–1212 (2015)
Chao, M.T., Cheng, C.Z.: A note on the convergence of alternating proximal gradient method. Appl. Math. Comput. 228(1), 258–263 (2014)
Chen, C.H., He, B.S., Ye, Y.Y., Yuan, X.M.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1), 57–79 (2016)
Parikh, N., Boyd, S.P.: Proximal algorithms. Found Trends Opt. 1, 127–239 (2014)
Wang, K., Desai, J., He, H.J.: A proximal partially parallel splitting method for separable convex programs. Optim. Methods Softw. 32(1), 39–68 (2017)
Deng, W., Lai, M.J., Peng, Z.M., Yin, W.T.: Parallel multi-block ADMM with \(o(1/k)\) convergence. J. Sci. Comput. (2016) (in press)
He, B.S., Tao, M., Yuan, X.M.: A splitting method for separate convex programming with linking linear constraints (2010) (manuscript)
Lin, Z.C., Liu, R.S., Li, H.: Linearized alternating direction method with parallel splitting and adaptive penalty for separable convex programs in machine learning. Mach. Learn. 95(2), 287–325 (2015)
He, B.S., Yuan, X.M., Zhang, W.X.: A customized proximal point algorithm for convex minimization with linear constraints. Comput. Optim. Appl. 56, 559–572 (2013)
Gu, G.Y., He, B.S., Yuan, X.M.: Customized proximal point algorithms for linearly constrained convex minimization and saddle-point problems: a unified approach. Comput. Optim. Appl. 59, 135–161 (2014)
Huai, K.Z., Ni, M.F., Ma, F., Yu, Z.K.: A customized proximal point algorithm for stable principal component pursuit with nonnegative constraint. J. Inequal. Appl. 2015, 148 (2015)
Li, Q., Xu, Y.S., Zhang, N.: Two-step fixed-point proximity algorithms for multi-block separable convex problems. J. Sci. Comput. (2016) (in press)
Chen, P.J., Huang, J.G., Zhang, X.Q.: A primal-dual fixed point algorithm for minimization of the sum of three convex separable functions. Fixed Point Theory Appl. 2016(1), 1–18 (2016)
Han, D.R., Yuan, X.M., Zhang, W.X., Cai, X.J.: An ADM-based splitting method for separable convex programming. Comput. Optim. Appl. 54, 343–369 (2013)
Xu, Y.Y.: Accelerated first-order primal-dual proximal methods for linearly constrained composite convex programming (2016) (manuscript)
He, B.S., Liu, H., Wang, Z.R., Yuan, X.M.: A strictly contractive Peaceman–Rachford splitting method for convex programming. SIAM J. Optim. 24(3), 1011–1040 (2014)
Ma, S.Q.: Alternating proximal gradient method for convex minimization. J. Sci. Comput. 68(2), 546–572 (2016)
He, B.S., Tao, M., Yuan, X.M.: A splitting method for separable convex programming. IMA J. Numer. Anal. 35, 394–426 (2015)
Tao, M., Yuan, X.M.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21, 57–81 (2011)
Acknowledgements
The authors would like to express their thanks to the anonymous reviewers who helped improve the quality of the manuscript and Doctor H.J. He for his kind offer of the Matlab codes used in this paper. This work is supported by the foundation of National Natural Science Foundation of China (No. 11601475), and Scientific Research Project of Shandong Universities (No. J15LI11).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sun, M., Sun, H. & Wang, Y. Two proximal splitting methods for multi-block separable programming with applications to stable principal component pursuit. J. Appl. Math. Comput. 56, 411–438 (2018). https://doi.org/10.1007/s12190-017-1080-9
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12190-017-1080-9