Skip to main content
SpringerLink
Log in

Finding the Forward-Douglas–Rachford-Forward Method

Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Cite this article

Abstract

We consider the monotone inclusion problem with a sum of 3 operators, in which 2 are monotone and 1 is monotone-Lipschitz. The classical Douglas–Rachford and forward–backward–forward methods, respectively, solve the monotone inclusion problem with a sum of 2 monotone operators and a sum of 1 monotone and 1 monotone-Lipschitz operators. We first present a method that naturally combines Douglas–Rachford and forward–backward–forward and show that it solves the 3-operator problem under further assumptions, but fails in general. We then present a method that naturally combines Douglas–Rachford and forward–reflected–backward, a recently proposed alternative to forward–backward–forward by Malitsky and Tam (A forward–backward splitting method for monotone inclusions without cocoercivity, 2018. arXiv:1808.04162). We show that this second method solves the 3-operator problem generally, without further assumptions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)

    Article  MathSciNet  Google Scholar 

  2. Tseng, P.: A modified forward–backward splitting method for maximal monotone mappings. SIAM J. Control Optim. 38(2), 431–446 (2000)

    Article  MathSciNet  Google Scholar 

  3. Bruck, R.E.: On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space. J. Math. Anal. Appl. 61(1), 159–164 (1977)

    Article  MathSciNet  Google Scholar 

  4. Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl. 72(2), 383–390 (1979)

    Article  MathSciNet  Google Scholar 

  5. Raguet, H., Fadili, J., Peyré, G.: Generalized forward–backward splitting. SIAM J. Imaging Sci. 6(3), 1199–1226 (2013)

    Article  MathSciNet  Google Scholar 

  6. Raguet, H.: A note on the forward-Douglas–Rachford splitting for monotone inclusion and convex optimization. Optim. Lett. 13(4), 717–740 (2018)

    Article  MathSciNet  Google Scholar 

  7. Briceño-Arias, L.M.: Forward-Douglas–Rachford splitting and forward–partial inverse method for solving monotone inclusions. Optimization 64(5), 1239–1261 (2015)

    Article  MathSciNet  Google Scholar 

  8. Davis, D., Yin, W.: A Three-operator splitting scheme and its optimization applications. Set Valued Var. Anal. 25(4), 829–858 (2017)

    Article  MathSciNet  Google Scholar 

  9. Briceño-Arias, L.M., Davis, D.: Forward–Backward–Half forward algorithm for solving monotone inclusions. SIAM J. Optim. 28(4), 2839–2871 (2018)

    Article  MathSciNet  Google Scholar 

  10. Banert, S.: A relaxed forward–backward splitting algorithm for inclusions of sums of monotone operators. Master’s Thesis, Technische Universität Chemnitz (2012)

  11. Briceño-Arias, L.M.: Forward–partial inverse–forward splitting for solving monotone inclusions. J. Optim. Theory Appl. 166(2), 391–413 (2015)

    Article  MathSciNet  Google Scholar 

  12. Malitsky, Y., Tam, M.K.: A forward–backward splitting method for monotone inclusions without cocoercivity. arXiv:1808.04162 (2018)

  13. Combettes, P.L., Pesquet, J.-C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set Valued Var. Anal. 20(2), 307–330 (2012)

    Article  MathSciNet  Google Scholar 

  14. Combettes, P.L.: Systems of structured monotone inclusions: duality, algorithms, and applications. SIAM J. Optim. 23(4), 2420–2447 (2013)

    Article  MathSciNet  Google Scholar 

  15. Rockafellar, R.T.: Monotone operators associated with saddle-functions and minimax problems. In: Browder, F.E. (ed.) Nonlinear Functional Analysis, Part 1, Volume 18 of Proceedings of Symposia in Pure Mathematics, vol. 18, pp. 241–250 (1970)

  16. Boţ, R.I., Hendrich, C.: A Douglas–Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM J. Optim. 23(4), 2541–2565 (2013)

    Article  MathSciNet  Google Scholar 

  17. Latafat, P., Patrinos, P.: Asymmetric forward–backward–adjoint splitting for solving monotone inclusions involving three operators. Comput. Optim. Appl. 68(1), 57–93 (2017)

    Article  MathSciNet  Google Scholar 

  18. Ryu, E.K.: Uniqueness of DRS as the 2 operator resolvent-splitting and impossibility of 3 operator resolvent-splitting. Math. Program. (2019). https://doi.org/10.1007/s10107-019-01403-1

  19. Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)

    Article  MathSciNet  Google Scholar 

  20. Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38(3), 667–681 (2013)

    Article  MathSciNet  Google Scholar 

  21. Yan, M.: A new primal-dual algorithm for minimizing the sum of three functions with a linear operator. J. Sci. Comput. 76(3), 1698–1717 (2018)

    Article  MathSciNet  Google Scholar 

  22. Johnstone, P.R., Eckstein, J.: Projective splitting with forward steps: asynchronous and block-iterative operator splitting. arXiv:1803.07043 (2018)

  23. Johnstone, P.R., Eckstein, J.: Projective splitting with forward steps only requires continuity. arXiv:1809.07180 (2018)

  24. Bùi, M. N. and Combettes, P. L.: Warped proximal iterations for monotone inclusions. arXiv:1908.07077 (2019)

  25. Giselsson, P.: Nonlinear forward–backward splitting with projection correction. arXiv:1908.07449 (2019)

  26. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, New York (2017)

    Book  Google Scholar 

  27. Ryu, E.K., Boyd, S.: Primer on monotone operator methods. Appl. Comput. Math. 15(1), 3–43 (2016)

    MathSciNet  MATH  Google Scholar 

  28. Ryu, E.K., Hannah, R., Yin, W.: Scaled relative graph: nonexpansive operators via 2D euclidean geometry. arXiv:1902.09788 (2019)

  29. Combettes, P.L., Glaudin, L.E.: Proximal activation of smooth functions in splitting algorithms for convex minimization. SIAM J. Imaging Sci. arXiv:1803.02919v2 (2018)

  30. Drori, Y., Teboulle, M.: Performance of first-order methods for smooth convex minimization: a novel approach. Math. Program. 145(1–2), 451–482 (2014)

    Article  MathSciNet  Google Scholar 

  31. Taylor, A.B., Hendrickx, J.M., Glineur, F.: Smooth strongly convex interpolation and exact worst-case performance of first-order methods. Math. Program. 161(1–2), 307–345 (2017)

    Article  MathSciNet  Google Scholar 

  32. Ryu, E.K., Taylor, A.B., Bergeling, C., Giselsson, P.: Operator splitting performance estimation: tight contraction factors and optimal parameter selection. arXiv:1812.00146 (2018)

Download references

Acknowledgements

Ernest Ryu was partially supported by AFOSR MURI FA9550-18-1-0502, NSF Grant DMS-1720237, and ONR Grant N000141712162. Bằng Công Vũ’s research work was funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under Grant No. 102.01-2017.05.

Author information

Authors and Affiliations

  1. University of California, Los Angeles, CA, USA

    Ernest K. Ryu

  2. Department of Mathematics, Vietnam National University, Hanoi, Vietnam

    Bằng Công Vũ

Authors
  1. Ernest K. Ryu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bằng Công Vũ
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ernest K. Ryu.

Additional information

Communicated by Jalal Fadili.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ryu, E.K., Vũ, B.C. Finding the Forward-Douglas–Rachford-Forward Method. J Optim Theory Appl 184, 858–876 (2020). https://doi.org/10.1007/s10957-019-01601-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-019-01601-z

Keywords

Mathematics Subject Classification

search

Navigation