Abstract
Recently, circumcentering reflection method (CRM) has been introduced for solving the feasibility problem of finding a point in the intersection of closed constraint sets. It is closely related with Douglas–Rachford method (DR). We prove local convergence of CRM in the same prototypical settings of most theoretical analysis of regular nonconvex DR, whose consideration is made natural by the geometry of the phase retrieval problem. For the purpose, we show that CRM is related to the method of subgradient projections. For many cases when DR is known to converge to a feasible point, we establish that CRM locally provides a better convergence rate. As a root finder, we show that CRM has local convergence whenever Newton–Raphson method does, has quadratic rate whenever Newton–Raphson method does, and exhibits superlinear convergence in many cases when Newton–Raphson method fails to converge at all. We also obtain explicit regions of convergence. As an interesting aside, we demonstrate local convergence of CRM to feasible points in cases when DR converges to fixed points that are not feasible. We demonstrate an extension in higher dimensions, and use it to obtain convergence rate guarantees for sphere and subspace feasibility problems. Armed with these guarantees, we experimentally discover that CRM is highly sensitive to compounding numerical error that may cause it to achieve worse rates than those guaranteed by theory. We then introduce a numerical modification that enables CRM to achieve the theoretically guaranteed rates. Any future works that study CRM for product space formulations of feasibility problems should take note of this sensitivity and account for it in numerical implementations.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Data Availability
The code used to generate the numerical results is available at [27].
References
Cinderella (software): Available at https://cinderella.de/tiki-index.php (2016)
Artacho, F.J. A., Borwein, J.M.: Global convergence of a non-convex douglas–Rachford iteration. J. Glob. Optim. 57(3), 753–769 (2013)
Artacho, F.J.A., Borwein, J.M., Tam, M.K.: Recent results on Douglas-R,achford methods. Serdica Math. J. 39, 313–330 (2013)
Artacho, F.J.A., Borwein, J.M., Tam, M.K.: Douglas–Rachford feasibility methods for matrix completion problems. ANZIAM J. 55(4), 299–326 (2014)
Artacho, F.J.A., Borwein, J.M., Tam, M.K.: Recent results on Douglas–R,achford methods for combinatorial optimization problems. J. Optim Theory Appl. 163(1), 1–30 (2014)
Artacho, F.J.A., Campoy, R.: Solving graph coloring problems with the douglas–Rachford algorithm. Set-Valued and Variational Analysis 26(2), 277–304 (2018)
Artacho, F.J.A., Campoy, R, Elser, V.: An enhanced formulation for solving graph coloring problems with the douglas–Rachford algorithm. J. Glob. Optim. 77(2), 383–403 (2020)
Arefidamghani, R., Behling, R., Bello-Cruz, Y., Iusem, A.N., Santos, L.R.: The circumcentered-reflection method achieves better rates than alternating projections. Comput. Optim. Appl. 79(2), 383–403 (2021)
Artacho, F.J.A., Campoy, R., Tam, M.K.: The douglas–Rachford algorithm for convex and nonconvex feasibility problems. Math. Methods Oper. Rese. 91(2), 201–240 (2020)
Bauschke, H.H., Combettes, P.L., Luke, D.R.: Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. J. Opt. Soc. Amer. A 19(7), 1334–1345 (2002)
Bauschke, H.H., Dao, M.N., Lindstrom, S.B.: The douglas–Rachford algorithm for a hyperplane and a doubleton. J. Glob. Optim. 74(1), 79–93 (2019)
Bauschke, H.H., Ouyang, H., Wang, X.: On circumcenter mappings induced by nonexpansive operators, Pure and Applied Functional Analysis, in press (2018)
Bauschke, H.H., Ouyang, H., Wang, X.: On circumcenters of finite sets in Hilbert spaces. Linear and Nonlinear Anal. 4, 271–295 (2018)
Bauschke, H.H., Ouyang, H., Wang, X.: On the linear convergence of circumcentered isometry methods. Numer. Algo. 87(1), 263–297 (2021)
Behling, R, Bello-Cruz, J.Y., Santos, L.R.: On the linear convergence of the circumcentered-reflection method. Oper. Res. Lett. 46(2), 159–162 (2018)
Behling, R., Bello-Cruz, J.Y., Santos, L.R.: On the circumcentered-reflection method for the convex feasibility problem. Numer. Algo. 86(4), 1475–1494 (2021)
Behling, R., Bello-Cruz, J.Y., Santos, L.R.: Circumcentering the douglas–Rachford method. Numer. Algo. 78, 759–776 (2018)
Benoist, J.: The douglas–Rachford algorithm for the case of the sphere and the line. J. Glob. Optim. 63, 363–380 (2015)
Borwein, J.M., Lindstrom, S.B., Brailey, S., Skerritt, M.P., Schneider, A.: Dynamics of the Douglas–Rachford method for ellipses and p-spheres. Set-Valued Anal. 26(2), 385–403 (2018)
Borwein, J.M., Brailey, S., The Douglas–Rachford algorithm in the absence of convexity. In: Heinz H. Bauschke, Regina S. Burachik, Patrick L. Combettes, Veit Elser, D. Russell Luke, and Henry Wolkowicz, editors, Fixed Point Algorithms for Inverse Problems in Science and Engineering, volume 49 of Springer Optimization and Its Applications, pages 93–109. Springer Optimization and Its Applications (2011)
Borwein, J.M., Tam, M.K.: Reflection Methods for Inverse Problems with Applications to Protein Conformation Determination. In: Springer Volume on the CIMPA School Generalized Nash Equilibrium Problems, Bilevel Programming and MPEC, New Delhi, India (2012)
Borwein, J.M., Tam, M.K.: Douglas–Rachford iterations in the absence of convexity: Jonathan Borwein’s lecture at 2015 spring school on variational analysis in Czech Republic. Available at https://carma.newcastle.edu.au/{DR}methods/paseky/{III}-nonconvex{DRS}mall.pdf (2015)
Burden, R.L., Douglas Faires, J.: Numerical analysis. cengage learning (2016)
Dao, M.N., Tam, M.K.: A Lyapunov-type approach to convergence of the Douglas–R,achford algorithm. J. Glob Optim. 73(1), 83–112 (2019)
Dizon, N., Hogan, J., Lindstrom, S.B.: Centering projection methods for wavelet feasibility problems. ISAAC 2019: The 30th International Symposium on Algorithms and Computation proceeding volume Current Trends in Analysis, its Applications and Computation, in press (2020)
Dizon, N., Hogan, J. , Lindstrom, S.B.: Circumcentered reflections method for wavelet feasibility problems. ANZIAM J. 22 (2022)
Dizon, N., Hogan, J., Lindstrom, S.B.: Code for the article “Circumcentering reflection methods for nonconvex feasibility problems”. Available at https://github.com/lindstromscott/Circumcentering--Reflection--Methods--for--Nonconvex--Feasibility--Problems (2021)
Douglas Jr, J., Rachford Jr, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Amer. Math. Soc. 82, 421–439 (1956)
Elser, V.: The complexity of bit retrieval. IEEE Trans. Inf. Theory 64(1), 412–428 (2018)
Fienup, J.R.: Phase retrieval algorithms: a comparison. Appl. optics 21(15), 2758–2769 (1982)
Franklin, D.J.: Projection Algorithms for Non-separable Wavelets and Clifford Fourier Analysis. PhD thesis, University of Newcastle. Available at http://hdl.handle.net/1959.13/1395028 (2018)
Gabay, D.: Applications of the Method of Multipliers to Variational Inequalities. In: Studies in Mathematics and Its Applications, Volume 15, Chapter Ix, Pages 299–331. Elsevier (1983)
Giladi, O., Rüffer, B.S.: A Lyapunov function construction for a non-convex douglas–Rachford iteration. J. Optim. Theory Appl. 180(3), 729–750 (2019)
Gravel, S, Elser, V: Divide and concur: A general approach to constraint satisfaction. Phy. Rev. E 78(3), 036706 (2008)
Lakshmikantham, V., Trigiante, D.: Theory of difference equations - numerical methods and applications marcel dekker (2002)
Lamichhane, B.P., Lindstrom, S.B., Brailey, S.: Application of projection algorithms to differential equations: boundary value problems. The ANZIAM Journal 61(1), 23–46 (2019)
Lindstrom, S.B.: Proximal Point algorithms, dynamical systems, and Associated Operators: Modern Perspectives from Experimental Mathematics. PhD thesis, University of Newcastle (2019)
Lindstrom, S.B.: Computable centering methods for spiraling algorithms and their duals, with motivations from the theory of Lyapunov functions. arXiv:2001.10784 (2020)
Lindstrom, S.B., Brailey, S.: Survey: sixty years of douglas–Rachford. J. Aust. Math. Soc. 110(3), 333–370 (2021)
Lindstrom, S.B., Brailey, S., Skerritt, M.P.: Computing intersections of implicitly specified plane curves. Nonlinear and Conv. Anal. 18(3), 347–359 (2017)
Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer Anal. 16(6), 964–979 (1979)
Boris, S.M.: Variational analysis and generalized differentiation i: Basic theory, volume 330 Springer Science & Business Media (2006)
Pierra, G.: Decomposition through formalization in a product space. Math. Program. 28(1), 96–115 (1984)
Poon, C., Liang, J.: Trajectory of alternating direction method of multipliers and adaptive acceleration. NeurIPS 2019 (2019)
Acknowledgements
SBL was supported in part by Hong Kong Research Grants Council PolyU153085/16p and by an AustMS Lift-Off Fellowship; his collaboration in this project was also made possible in part by funding from CARMA Priority Research Centre at University of Newcastle. JAH and NDD are supported by Australian Research Council Grant DP160101537.
Funding
Open Access funding enabled and organized by CAUL and its Member Institutions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Dizon, N.D., Hogan, J.A. & Lindstrom, S.B. Circumcentering Reflection Methods for Nonconvex Feasibility Problems. Set-Valued Var. Anal 30, 943–973 (2022). https://doi.org/10.1007/s11228-021-00626-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11228-021-00626-9