We consider the monotone inclusion problem with a sum of 3 operators, in which 2 are monotone and 1 is monotone-Lipschitz. The classical Douglas–Rachford and forward–backward–forward methods, respectively, solve the monotone inclusion problem with a sum of 2 monotone operators and a sum of 1 monotone and 1 monotone-Lipschitz operators. We first present a method that naturally combines Douglas–Rachford and forward–backward–forward and show that it solves the 3-operator problem under further assumptions, but fails in general. We then present a method that naturally combines Douglas–Rachford and forward–reflected–backward, a recently proposed alternative to forward–backward–forward by Malitsky and Tam (A forward–backward splitting method for monotone inclusions without cocoercivity, 2018. arXiv:1808.04162). We show that this second method solves the 3-operator problem generally, without further assumptions.
This is a preview of subscription content,to check access.
Access this article
Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)
Tseng, P.: A modified forward–backward splitting method for maximal monotone mappings. SIAM J. Control Optim. 38(2), 431–446 (2000)
Bruck, R.E.: On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space. J. Math. Anal. Appl. 61(1), 159–164 (1977)
Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl. 72(2), 383–390 (1979)
Raguet, H., Fadili, J., Peyré, G.: Generalized forward–backward splitting. SIAM J. Imaging Sci. 6(3), 1199–1226 (2013)
Raguet, H.: A note on the forward-Douglas–Rachford splitting for monotone inclusion and convex optimization. Optim. Lett. 13(4), 717–740 (2018)
Briceño-Arias, L.M.: Forward-Douglas–Rachford splitting and forward–partial inverse method for solving monotone inclusions. Optimization 64(5), 1239–1261 (2015)
Davis, D., Yin, W.: A Three-operator splitting scheme and its optimization applications. Set Valued Var. Anal. 25(4), 829–858 (2017)
Briceño-Arias, L.M., Davis, D.: Forward–Backward–Half forward algorithm for solving monotone inclusions. SIAM J. Optim. 28(4), 2839–2871 (2018)
Banert, S.: A relaxed forward–backward splitting algorithm for inclusions of sums of monotone operators. Master’s Thesis, Technische Universität Chemnitz (2012)
Briceño-Arias, L.M.: Forward–partial inverse–forward splitting for solving monotone inclusions. J. Optim. Theory Appl. 166(2), 391–413 (2015)
Malitsky, Y., Tam, M.K.: A forward–backward splitting method for monotone inclusions without cocoercivity. arXiv:1808.04162 (2018)
Combettes, P.L., Pesquet, J.-C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set Valued Var. Anal. 20(2), 307–330 (2012)
Combettes, P.L.: Systems of structured monotone inclusions: duality, algorithms, and applications. SIAM J. Optim. 23(4), 2420–2447 (2013)
Rockafellar, R.T.: Monotone operators associated with saddle-functions and minimax problems. In: Browder, F.E. (ed.) Nonlinear Functional Analysis, Part 1, Volume 18 of Proceedings of Symposia in Pure Mathematics, vol. 18, pp. 241–250 (1970)
Boţ, R.I., Hendrich, C.: A Douglas–Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM J. Optim. 23(4), 2541–2565 (2013)
Latafat, P., Patrinos, P.: Asymmetric forward–backward–adjoint splitting for solving monotone inclusions involving three operators. Comput. Optim. Appl. 68(1), 57–93 (2017)
Ryu, E.K.: Uniqueness of DRS as the 2 operator resolvent-splitting and impossibility of 3 operator resolvent-splitting. Math. Program. (2019). https://doi.org/10.1007/s10107-019-01403-1
Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)
Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38(3), 667–681 (2013)
Yan, M.: A new primal-dual algorithm for minimizing the sum of three functions with a linear operator. J. Sci. Comput. 76(3), 1698–1717 (2018)
Johnstone, P.R., Eckstein, J.: Projective splitting with forward steps: asynchronous and block-iterative operator splitting. arXiv:1803.07043 (2018)
Johnstone, P.R., Eckstein, J.: Projective splitting with forward steps only requires continuity. arXiv:1809.07180 (2018)
Bùi, M. N. and Combettes, P. L.: Warped proximal iterations for monotone inclusions. arXiv:1908.07077 (2019)
Giselsson, P.: Nonlinear forward–backward splitting with projection correction. arXiv:1908.07449 (2019)
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, New York (2017)
Ryu, E.K., Boyd, S.: Primer on monotone operator methods. Appl. Comput. Math. 15(1), 3–43 (2016)
Ryu, E.K., Hannah, R., Yin, W.: Scaled relative graph: nonexpansive operators via 2D euclidean geometry. arXiv:1902.09788 (2019)
Combettes, P.L., Glaudin, L.E.: Proximal activation of smooth functions in splitting algorithms for convex minimization. SIAM J. Imaging Sci. arXiv:1803.02919v2 (2018)
Drori, Y., Teboulle, M.: Performance of first-order methods for smooth convex minimization: a novel approach. Math. Program. 145(1–2), 451–482 (2014)
Taylor, A.B., Hendrickx, J.M., Glineur, F.: Smooth strongly convex interpolation and exact worst-case performance of first-order methods. Math. Program. 161(1–2), 307–345 (2017)
Ryu, E.K., Taylor, A.B., Bergeling, C., Giselsson, P.: Operator splitting performance estimation: tight contraction factors and optimal parameter selection. arXiv:1812.00146 (2018)
Ernest Ryu was partially supported by AFOSR MURI FA9550-18-1-0502, NSF Grant DMS-1720237, and ONR Grant N000141712162. Bằng Công Vũ’s research work was funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under Grant No. 102.01-2017.05.
Communicated by Jalal Fadili.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Ryu, E.K., Vũ, B.C. Finding the Forward-Douglas–Rachford-Forward Method. J Optim Theory Appl 184, 858–876 (2020). https://doi.org/10.1007/s10957-019-01601-z