Advertisement

Comparing Averaged Relaxed Cutters and Projection Methods: Theory and Examples

  • Reinier Díaz Millán
  • Scott B. LindstromEmail author
  • Vera Roshchina
Conference paper
  • 49 Downloads
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 313)

Abstract

We focus on the convergence analysis of averaged relaxations of cutters, specifically for variants that—depending upon how parameters are chosen—resemble alternating projections, the Douglas–Rachford method, relaxed reflect-reflect, or the Peaceman–Rachford method. Such methods are frequently used to solve convex feasibility problems. The standard convergence analysis of projection algorithms is based on the firm nonexpansivity property of the relevant operators. However, if the projections onto the constraint sets are replaced by cutters (which may be thought of as maps that project onto separating hyperplanes), the firm nonexpansivity is lost. We provide a proof of convergence for a family of related averaged relaxed cutter methods under reasonable assumptions, relying on a simple geometric argument. This allows us to clarify fine details related to the allowable choice of the relaxation parameters, highlighting the distinction between the exact (firmly nonexpansive) and approximate (strongly quasinonexpansive) settings. We provide illustrative examples and discuss practical implementations of the method.

Notes

Acknowledgements

We are grateful to Heinz Bauschke for pointing out useful references. We also extend our gratitude to the organisers and participants of the workshop Splitting Algorithms, Modern Operator Theory and Applications held at Casa Matematica Oaxaca, Mexico, from 17–22 September 2017 and to the Australian Research Council for supporting our travel expenses via project DE150100240. We are also grateful for the helpful feedback of two anonymous referees, whose suggestions greatly improved this manuscript.

We dedicate this work to the memory of Jonathan M. Borwein, whose influence on the present authors and on the Australian mathematical community cannot be overstated.

References

  1. 1.
    Aleyner, A., Reich, S.: Block-iterative algorithms for solving convex feasibility problems in Hilbert and in Banach spaces. J. Math. Anal. Appl. 343(1), 427–435 (2008).  https://doi.org/10.1016/j.jmaa.2008.01.087
  2. 2.
    Aragón Artacho, F.J., Borwein, J.M.: Global convergence of a non-convex Douglas-Rachford iteration. J. Glob. Optim. 57(3), 753–769 (2012)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Douglas–Rachford feasibility methods for matrix completion problems. ANZIAM J. 55(4), 299–326 (2014).  https://doi.org/10.1017/S1446181114000145
  4. 4.
    Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Recent results on Douglas–Rachford methods for combinatorial optimization problems. J. Optim. Theory Appl. 163(1), 1–30 (2014).  https://doi.org/10.1007/s10957-013-0488-0
  5. 5.
    Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Global behavior of the Douglas–Rachford method for a nonconvex feasibility problem. J. Global Optim. 65(2), 309–327 (2016).  https://doi.org/10.1007/s10898-015-0380-6
  6. 6.
    Aragón Artacho, F.J., Campoy, R.: Solving graph coloring problems with the Douglas–Rachford algorithm. Set-Valued and Variational Analysis pp. 1–28 (2017)Google Scholar
  7. 7.
    Aragón Artacho, F.J., Campoy, R.: A new projection method for finding the closest point in the intersection of convex sets. Comput. Optim. Appl. 69(1), 99–132 (2018).  https://doi.org/10.1007/s10589-017-9942-5
  8. 8.
    Artacho, F.J.A., Campoy, R.: Optimal rates of linear convergence of the averaged alternating modified reflections method for two subspaces (2017). arXiv:1711.06521
  9. 9.
    Baillon, J.B., Combettes, P.L., Cominetti, R.: There is no variational characterization of the cycles in the method of periodic projections. J. Funct. Anal. 262(1), 400–408 (2012).  https://doi.org/10.1016/j.jfa.2011.09.002
  10. 10.
    Bauschke, H.H., Combettes, P.L.: A weak-to-strong convergence principle for fejér-monotone methods in hilbert spaces. Mathematics of Operations Research 26(2), 248–264. http://www.jstor.org/stable/3690618 (2001)
  11. 11.
    Bauschke, H.H., Combettes, P.L.: Convex analysis and monotone operator theory in Hilbert spaces, 2nd edn. CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-48311-5. With a foreword by Hédy Attouch
  12. 12.
    Bauschke, H.H., Combettes, P.L., Luke, D.R.: Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. J. Opt. Soc. Am. A 19(7), 1334–1345 (2002).  https://doi.org/10.1364/JOSAA.19.001334
  13. 13.
    Bauschke, H.H., Dao, M.N., Noll, D., Phan, H.M.: On Slater’s condition and finite convergence of the Douglas–Rachford algorithm for solving convex feasibility problems in Euclidean spaces. J. Global Optim. 65(2), 329–349 (2016).  https://doi.org/10.1007/s10898-015-0373-5
  14. 14.
    Bauschke, H.H., Wang, C., Wang, X., Xu, J.: On subgradient projectors. SIAM J. Optim. 25(2), 1064–1082 (2015)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Bauschke, H.H., Wang, C., Wang, X., Xu, J.: On the finite convergence of a projected cutter method. J. Optim. Theory Appl. 165(3), 901–916 (2015)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Bauschke, H.H., Wang, C., Wang, X., Xu, J.: Subgradient projectors: extensions, theory, and characterizations. Set-Valued Var. Anal. 1–70 (2017)Google Scholar
  17. 17.
    Bello Cruz, J.Y., Díaz Millán, R.: A relaxed-projection splitting algorithm for variational inequalities in Hilbert spaces. J. Global Optim. 65(3), 597–614 (2016).  https://doi.org/10.1007/s10898-015-0397-x
  18. 18.
    Bello Cruz, J.Y., Iusem, A.N.: An explicit algorithm for monotone variational inequalities. Optimization 61(7), 855–871 (2012).  https://doi.org/10.1080/02331934.2010.536232
  19. 19.
    Benoist, J.: The Douglas-Rachford algorithm for the case of the sphere and the line. J. Glob. Optim. 63, 363–380 (2015)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Borwein, J.M.: The life of modern homo habilis mathematicus: experimental computation and visual theorems. Tools and Mathematics. Mathematics Education Library, vol. 347, pp. 23–90. Springer, Berlin (2016)CrossRefGoogle Scholar
  21. 21.
    Borwein, J.M., Li, G., Tam, M.K.: Convergence rate analysis for averaged fixed point iterations in common fixed point problems. SIAM J. Optim. 27(1), 1–33 (2017)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Borwein, J.M., Li, G., Yao, L.: Analysis of the convergence rate for the cyclic projection algorithm applied to basic semialgebraic convex sets. SIAM J. Optim. 24(1), 498–527 (2014).  https://doi.org/10.1137/130919052
  23. 23.
    Borwein, J.M., Lindstrom, S.B., Sims, B., Schneider, A., Skerritt, M.P.: Dynamics of the Douglas-Rachford method for ellipses and p-spheres. Set-Valued Var. Anal. 26(2), 385–403 (2018)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Borwein, J.M., Sims, B.: The Douglas–Rachford algorithm in the absence of convexity. Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Springer Optimization and Its Applications, vol. 49, pp. 93–109. Springer, New York (2011).  https://doi.org/10.1007/978-1-4419-9569-8_6
  25. 25.
    Cegielski, A.: Iterative Methods for Fixed Point Problems in Hilbert Spaces. Lecture Notes in Mathematics, vol. 2057. Springer, Heidelberg (2012)zbMATHGoogle Scholar
  26. 26.
    Cegielski, A., Reich, S., Zalas, R.: Regular sequences of quasi-nonexpansive operators and their applications (2017)Google Scholar
  27. 27.
    Censor, Y., Lent, A.: Cyclic subgradient projections. Math. Program. 24(1), 233–235 (1982)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Censor, Y., Zenios, S.A.: Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press on Demand (1997)Google Scholar
  29. 29.
    Combettes, P.L.: Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections. IEEE Trans. Image Process. 6(4), 493–506 (1997)CrossRefGoogle Scholar
  30. 30.
    Combettes, P.L.: Quasi-Fejérian analysis of some optimization algorithms 8, 115–152 http://www.sciencedirect.com/science/article/pii/S1570579X01800100 (2001)
  31. 31.
    Cominetti, R., Roshchina, V., Williamson, A.: A counterexample to De Pierro’s conjecture on the convergence of under-relaxed cyclic projections (2018)Google Scholar
  32. 32.
    Dao, M.N., Phan, H.M.: Linear convergence of projection algorithms (2016). arXiv:1609.00341
  33. 33.
    Dao, M.N., Phan, H.M.: Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems (2017). arXiv:1710.09814
  34. 34.
    De Pierro, A.R.: From parallel to sequential projection methods and vice versa in convex feasibility: results and conjectures. Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications (Haifa, 2000). Studies in Computational Mathematics, vol. 8, pp. 187–201. North-Holland, Amsterdam (2001).  https://doi.org/10.1016/S1570-579X(01)80012-4
  35. 35.
    Douglas Jr., J., Rachford Jr., H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956).  https://doi.org/10.2307/1993056
  36. 36.
    Drusvyatskiy, D., Li, G., Wolkowicz, H.: A note on alternating projections for ill-posed semidefinite feasibility problems. Math. Program. 162(1-2, Ser. A), 537–548 (2017).  https://doi.org/10.1007/s10107-016-1048-9
  37. 37.
    Elser, V.: Matrix product constraints by projection methods. J. Global Optim. 68(2), 329–355 (2017)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Fukushima, M.: A relaxed projection method for variational inequalities. Math. Program. 35(1), 58–70 (1986).  https://doi.org/10.1007/BF01589441
  39. 39.
    Hundal, H.S.: An alternating projection that does not converge in norm. Nonlinear Analysis: Theory, Methods & Applications 57(1), 35–61. http://www.sciencedirect.com/science/article/pii/S0362546X03004218 (2004).  https://doi.org/10.1016/j.na.2003.11.004
  40. 40.
    Kruger, A.Y., Luke, D.R., Thao, N.H.: About subtransversality of collections of sets. Set-Valued Var. Anal. 25(4), 701–729 (2017).  https://doi.org/10.1007/s11228-017-0436-5
  41. 41.
    Lamichhane, B.P., Lindstrom, S.B., Sims, B.: Application of projection algorithms to differential equations: boundary value problems (2017). arXiv:1705.11032
  42. 42.
    Li, G., Pong, T.K.: Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems. Math. Program. 159(1-2, Ser. A), 371–401 (2016).  https://doi.org/10.1007/s10107-015-0963-5
  43. 43.
    Lindstrom, S.B., Sims, B.: Survey: sixty years of Douglas–Rachford (2018). arXiv:1809.07181
  44. 44.
    Lindstrom, S.B., Sims, B., Skerritt, M.P.: Computing intersections of implicitly specified plane curves. Nonlinear Conv. Anal. 18(3), 347–359 (2017)MathSciNetzbMATHGoogle Scholar
  45. 45.
    Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979).  https://doi.org/10.1137/0716071
  46. 46.
    Littlewood, J.E.: A Mathematician’s Miscellany. Methuen London (1953)Google Scholar
  47. 47.
    Pierra, G.: Decomposition through formalization in a product space. Math. Program. 28(1), 96–115 (1984)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Polyak, B.T.: Introduction to optimization. Translations Series in Mathematics and Engineering. Optimization Software (1987)Google Scholar
  49. 49.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)Google Scholar
  50. 50.
    Spingarn, J.E.: Partial inverse of a monotone operator. Appl. Math. Optim. 10(1), 247–265 (1983)MathSciNetCrossRefGoogle Scholar
  51. 51.
    Svaiter, B.F.: On weak convergence of the Douglas-Rachford method. SIAM J. Control Optim. 49(1), 280–287 (2011)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Reinier Díaz Millán
    • 1
  • Scott B. Lindstrom
    • 2
    Email author
  • Vera Roshchina
    • 3
  1. 1.Federal Institute of GoiásGoiásBrazil
  2. 2.School of Mathematical and Physical SciencesCentre for Computer-assisted Research Mathematics and its Applications (CARMA), The University of NewcastleCallaghanAustralia
  3. 3.UNSW SydneySydneyAustralia

Personalised recommendations