Abstract
In this work, we study fixed point algorithms for finding a zero in the sum of \(n\ge 2\) maximally monotone operators by using their resolvents. More precisely, we consider the class of such algorithms where each resolvent is evaluated only once per iteration. For any algorithm from this class, we show that the underlying fixed point operator is necessarily defined on a d-fold Cartesian product space with \(d\ge n-1\). Further, we show that this bound is unimprovable by providing a family of examples for which \(d=n-1\) is attained. This family includes the Douglas–Rachford algorithm as the special case when \(n=2\). Applications of the new family of algorithms in distributed decentralised optimisation and multi-block extensions of the alternation direction method of multipliers (ADMM) are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Strictly speaking, (5) should be written as \(\textbf{y}=(B\otimes {{\,\textrm{Id}\,}})\textbf{z}+(L\otimes {{\,\textrm{Id}\,}})\textbf{x}\) where \(\otimes \) denotes the Kronecker product. In the special case when \(\mathcal {H}={\mathbb {R}}\), \(B\otimes {{\,\textrm{Id}\,}}=B\) and \(L\otimes {{\,\textrm{Id}\,}}=L\), and the two expressions coincide.
We emphasise that the coefficient matrices \(T_z\) and \(T_x\) are constant and do not have any dependence on \(\textbf{z}\) or \(\textbf{x}\). The subscripts are merely labels to indicate which variables the coefficients belong to. An analogous remark applies to the coefficient matrices \(S_z\) and \(S_x\) in (7).
References
Aragón Artacho, F. J., Bot, R. I. & Torregrosa-Belén, D.: A primal-dual splitting algorithm for composite monotone inclusions with minimal lifting. Numer. Algorithms arXiv:2202.09665v1 (2022) (in press)
Aragón Artacho, F.J., Campoy, R., Tam, M.K.: Strengthened splitting methods for computing resolvents. Comput. Optim. Appl. 80, 549–585 (2021)
Bauschke, H.H., Combettes, P.L.: CMS Books in Mathematics. In: Convex analysis and monotone operator theory in Hilbert spaces, 2nd edn. Springer, Berlin (2017)
Bertsekas, D.P.: Convex optimization algorithms. Athena Scientific, Massachuesetts (2015)
Campoy, R.: A product space reformulation with reduced dimension for splitting algorithms. Comput. Optim. Appl. (2022). https://doi.org/10.1007/s10589-022-00395-7
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)
Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155, 57–79 (2016)
Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)
Condat, L., Kitahara, D., Contreras, A. Hirabayashi, A.: Proximal splitting algorithms for convex optimization: a tour of recent advances, with new twists. SIAM Rev., to appear (2022)
Combettes, P., Pesquet, J.-C.: Proximal Splitting Methods in Signal Processing. In: Bauschke, H.H., Burachik, R.S.S., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-point algorithms for inverse problems in science and engineering, pp. 185–212. Springer-Verlag, New York (2011)
Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1), 293–318 (1992)
Gabay, D.: Applications of the method of multipliers to variational inequalities. Stud. Math. Appl. 15, 299–331 (1983)
Giselsson, P., Fält, M., Boyd, S.: Line search for averaged operator iteration. In: 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 1015–1022, (2016)
He, B., Yuan, X.: On the \(O(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)
He, B., Yuan, X.: Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imaging Sci. 5(1), 119–149 (2012)
Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)
Minty, G.J.: Monotone (nonlinear) operators in Hilbert space. Duke Math. J. 29(3), 341–346 (1962)
Monteiro, R.D., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J. Optim. 23(1), 475–507 (2013)
Pazy, A.: Asymptotic behavior of contractions in Hilbert space. Isr. J. Math. 9(2), 235–240 (1971)
Peaceman, D.W., Rachford, H.H., Jr.: The numerical solution of parabolic and elliptic differential equations. J. Soc. Ind. Appl. Math. 3(1), 28–41 (1955)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)
Ryu, E.K.: Uniqueness of DRS as the \(2\)-operator resolvent-splitting and impossibility of \(3\)-operator resolvent-splitting. Math. Program. 182(1), 233–273 (2020)
Ryu, E.K., Liu, Y., Yin, W.: Douglas-Rachford splitting and ADMM for pathological convex optimization. Comput. Optim. Appl. 74(3), 747–778 (2019)
Tao, M., Yuan, X.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21(1), 57–81 (2011)
Wright, J., Ganesh, A., Rao, S.R., Peng, Y., Ma, Y.: Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. NIPS 58, 289–298 (2009)
Acknowledgements
The work of YM was supported by the Wallenberg Al, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. The project number is 305286. MKT is supported in part by Australian Research Council grant DE200100063. The authors would like to thank the anoymous referees for helpful comments, which included the improved PDHG formulation given in (29).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Malitsky, Y., Tam, M.K. Resolvent splitting for sums of monotone operators with minimal lifting. Math. Program. 201, 231–262 (2023). https://doi.org/10.1007/s10107-022-01906-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-022-01906-4