# Alternating Projections on Nontangential Manifolds

- 284 Downloads
- 5 Citations

## Abstract

We consider sequences \((B_{k})_{k=0}^{\infty}\) of points obtained by projecting a given point *B*=*B* _{0} back and forth between two manifolds \(\mathcal{M}_{1}\) and \(\mathcal{M}_{2}\), and give conditions guaranteeing that the sequence converges to a limit \(B_{\infty}\in\mathcal{M}_{1}\cap\mathcal{M}_{2}\). Our motivation is the study of algorithms based on finding the limit of such sequences, which have proved useful in a number of areas. The intersection is typically a set with desirable properties but for which there is no efficient method for finding the closest point *B* _{ opt } in \(\mathcal{M}_{1}\cap\mathcal{M}_{2}\). Under appropriate conditions, we prove not only that the sequence of alternating projections converges, but that the limit point is fairly close to *B* _{ opt }, in a manner relative to the distance ∥*B* _{0}−*B* _{ opt }∥, thereby significantly improving earlier results in the field.

## Keywords

Alternating projections Convergence Non-convexity Low-rank approximation Manifolds## Mathematics Subject Classification

41A65 49Q99 53B25## Notes

### Acknowledgements

This work was supported by the Swedish Research Council and the Swedish Foundation for International Cooperation in Research and Higher Education, as well as Dirección de Investigación Científica y Technológica del Universidad de Santiago de Chile, Chile. Part of the work was conducted while Marcus Carlsson was employed at Universidad de Santiago de Chile. We thank Arne Meurman for fruitful discussions. We also would like to thank one of the reviewers for the constructive criticism and useful suggestions that have helped to improve the quality of the paper.

## References

- 1.Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008) MATHGoogle Scholar
- 2.Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Lojasiewicz inequality. Math. Oper. Res.
**35**, 438–457 (2010) MathSciNetCrossRefMATHGoogle Scholar - 3.Badea, C., Grivaux, S., Müller, V.: A generalization of the Friedrichs angle and the method of alternating projections. C. R. Math.
**348**, 53–56 (2010) CrossRefMATHGoogle Scholar - 4.Basu, S., Pollack, R., Roy, M.-F.: Algorithms in Real Algebraic Geometry. Algorithms and Computation in Mathematics, vol. 10. Springer, Berlin (2006) MATHGoogle Scholar
- 5.Bauschke, H.H., Borwein, J.M.: On the convergence of von Neumann’s alternating projection algorithm for two sets. Set-Valued Anal.
**1**, 185–212 (1993) MathSciNetCrossRefMATHGoogle Scholar - 6.Bauschke, H.H., Borwein, J.M.: Dykstra’s alternating projection algorithm for two sets. J. Approx. Theory
**79**, 418–443 (1994) MathSciNetCrossRefMATHGoogle Scholar - 7.Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev.
**38**, 367–426 (1996) MathSciNetCrossRefMATHGoogle Scholar - 8.Bauschke, H.H., Noll, D., Celler, A., Borwein, J.M.: An EM algorithm for dynamic SPECT. IEEE Trans. Med. Imag
**18**, 252–262 (1999) CrossRefGoogle Scholar - 9.Berger, M., Gostiaux, B.: Differential Geometry: Manifolds, Curves and Surfaces. Springer, Berlin (1988) CrossRefMATHGoogle Scholar
- 10.Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers. Now Publishers, Hanover (2011) Google Scholar
- 11.Boyd, S., Xiao, L.: Least squares covariance matrix adjustment. SIAM J. Matrix Anal. Appl.
**27**, 532–546 (2005) MathSciNetCrossRefMATHGoogle Scholar - 12.Brègman, L.M.: Finding the common point of convex sets by the method of successive projection. Dokl. Akad. Nauk SSSR
**162**, 487–490 (1965) MathSciNetGoogle Scholar - 13.Cadzow, J.A.: Signal enhancement—a composite property mapping algorithm. IEEE Trans. Acoust. Speech Signal Process.
**36**, 49–62 (1988) CrossRefMATHGoogle Scholar - 14.Combettes, P.L., Trussell, H.J.: Method of successive projections for finding a common point of sets in metric spaces. J. Optim. Theory Appl.
**67**, 487–507 (1990) MathSciNetCrossRefMATHGoogle Scholar - 15.Cox, D.A., Little, J.B., O’Shea, D.: Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, vol. 10. Springer, Berlin (2007) CrossRefGoogle Scholar
- 16.Deutsch, F.: Rate of convergence of the method of alternating projections. In: Parametric Optimization and Approximation, pp. 96–107 (1985) Google Scholar
- 17.Deutsch, F.: Best Approximation in Inner Product Spaces. CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, vol. 7. Springer, New York (2001) CrossRefMATHGoogle Scholar
- 18.Deutsch, F., Hundal, H.: The rate of convergence for the method of alternating projections, ii. J. Math. Anal. Appl.
**205**, 381–405 (1997) MathSciNetCrossRefMATHGoogle Scholar - 19.Dykstra, R.L.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc.
**78**, 837–842 (1983) MathSciNetCrossRefMATHGoogle Scholar - 20.Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrika
**1**, 211–218 (1936) CrossRefMATHGoogle Scholar - 21.Franklin, J.N.: Matrix Theory. Dover Publications in Mineola, New York (2000) Google Scholar
- 22.Friedrichs, K.: On certain inequalities and characteristic value problems for analytic functions and for functions of two variables. Trans. Am. Math. Soc.
**41**, 321–364 (1937) MathSciNetCrossRefGoogle Scholar - 23.Gilbarg, D., Trudinger, N.: Elliptic Partial Differential Equations of Second Order, vol. 224. Springer, Berlin (2001) MATHGoogle Scholar
- 24.Grigoriadis, K.M., Frazho, A.E., Skelton, R.E.: Application of alternating convex projection methods for computation of positive Toeplitz matrices. IEEE Trans. Signal Process.
**42**, 1873–1875 (1994) CrossRefGoogle Scholar - 25.Grubišić, I., Pietersz, R.: Efficient rank reduction of correlation matrices. Linear Algebra Appl.
**422**, 629–653 (2007) MathSciNetCrossRefMATHGoogle Scholar - 26.Gubin, L.G., Polyak, B.T., Raik, E.V.: The method of projections for finding the common point of convex sets. USSR Comput. Math. Math. Phys.
**7**, 1–24 (1967) CrossRefGoogle Scholar - 27.Higham, N.J.: Computing the nearest correlation matrix—a problem from finance. IMA J. Numer. Anal.
**22**, 329–343 (2002) MathSciNetCrossRefMATHGoogle Scholar - 28.Hörmander, L.: The Analysis of Linear Partial Differential Operators. III. Springer, Berlin (2007) MATHGoogle Scholar
- 29.Kayalar, S., Weinert, H.L.: Error bounds for the method of alternating projections. Math. Control Signals Syst. (MCSS)
**1**, 43–59 (1988) MathSciNetCrossRefMATHGoogle Scholar - 30.Kendig, K.: Elementary Algebraic Geometry. Springer, Berlin (1977) CrossRefMATHGoogle Scholar
- 31.Krantz, S.G., Parks, H.R.: Distance to
*C*^{k}hypersurfaces. J. Differ. Equ.**40**, 116–120 (1981) MathSciNetCrossRefMATHGoogle Scholar - 32.Lax, P.D.: Linear Algebra and Its Applications, 2nd edn. Pure and Applied Mathematics (Hoboken). Wiley-Interscience/Wiley, Hoboken (2007) MATHGoogle Scholar
- 33.Levi, A., Stark, H.: Signal restoration from phase by projections onto convex sets. J. Opt. Soc. Am.
**73**, 810–822 (1983) MathSciNetCrossRefGoogle Scholar - 34.Lewis, A.S., Luke, D., Malick, J.: Local linear convergence for alternating and averaged nonconvex projections. Found. Comput. Math.
**9**, 485–513 (2009) MathSciNetCrossRefMATHGoogle Scholar - 35.Lewis, A.S., Malick, J.: Alternating projections on manifolds. Math. Oper. Res.
**33**, 216–234 (2008) MathSciNetCrossRefMATHGoogle Scholar - 36.Li, Y., Liu, K., Razavilar, J.: A parameter estimation scheme for damped sinusoidal signals based on low-rank Hankel approximation. IEEE Trans. Signal Process.
**45**, 481–486 (1997) CrossRefGoogle Scholar - 37.Lu, B., Wei, D., Evans, B.L., Bovik, A.C.: Improved matrix pencil methods. In: Conference Record of the Thirty-Second Asilomar Conference on Signals, Systems Computers, vol. 2, pp. 1433–1437 (1998) Google Scholar
- 38.Markovsky, I.: Structured low-rank approximation and its applications. Automatica
**44**, 891–909 (2008) MathSciNetCrossRefGoogle Scholar - 39.Marks, R.J.: II, Alternating projections onto convex sets. In: Jansson, P.A. (ed.) Deconvolution of Images and Spectra, 2nd edn., pp. 476–501. Academic Press, San Diego (1997) Google Scholar
- 40.Montiel, S., Ros, A., Babbitt, D.G.: Curves and Surfaces, vol. 69. Am. Math. Soc., Providence (2009) MATHGoogle Scholar
- 41.Munkres, J.R.: Topology, 2nd edn. Prentice Hall, Upper Saddle River (2000) MATHGoogle Scholar
- 42.Nikolski, N.K.: Operators, Functions, and Systems: An Easy Reading. Vol. 1. Mathematical Surveys and Monographs, vol. 92. Am. Math. Soc., Providence (2002). Hardy, Hankel, and Toeplitz, translated from the French by Andreas Hartmann Google Scholar
- 43.Prabhu, V.U., Jalihal, D.: An improved ESPRIT based time-of-arrival estimation algorithm for vehicular OFDM systems. In: IEEE 69th Vehicular Technology Conference, 2009 VTC Spring 2009, April 2009, pp. 1–4. (2009) CrossRefGoogle Scholar
- 44.Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Grundlehren der Mathematischen Wissenschaften, vol. 317. Springer, Berlin (1998) CrossRefMATHGoogle Scholar
- 45.Sard, A.: The measure of the critical values of differentiable maps. Bull. Am. Math. Soc.
**48**, 883–890 (1942) MathSciNetCrossRefMATHGoogle Scholar - 46.Schmidt, E.: Zur theorie der linearen und nichtlinearen integralgleichungen. iii, teil. Math. Ann.
**65**, 370–399 (1908) MathSciNetCrossRefMATHGoogle Scholar - 47.Shafarevich, I.R.: Basic Algebraic Geometry. Springer, New York (1974). Translated from the Russian by K.A. Hirsch, Die Grundlehren der mathematischen Wissenschaften, Band 213 CrossRefMATHGoogle Scholar
- 48.Toh, K.C., Todd, M.J., Tütüncü, R.H.: SDPT3—a MATLAB software package for semidefinite programming, version 1.3. Optim. Methods Softw.
**11**, 545–581 (1999) MathSciNetCrossRefGoogle Scholar - 49.Toh, K.C., Todd, M.J., Tütüncü, R.H.: On the implementation and usage of SDPT3—a Matlab software package for semidefinite-quadratic-linear programming, version 4.0. In: Handbook on Semidefinite, Conic and Polynomial Optimization, pp. 715–754 (2012) CrossRefGoogle Scholar
- 50.Trotman, D.: Singularity Theory. Scientific Publishing, Oxford (2007). Chap. Lectures on real stratification theory MATHGoogle Scholar
- 51.Tütüncü, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program.
**95**, 189–217 (2003) MathSciNetCrossRefMATHGoogle Scholar - 52.von Neumann, J.: Functional Operators, Volume II: The Geometry of Orthogonal Spaces. Princeton University Press, Princeton (1950) Google Scholar
- 53.Whitney, H.: Elementary structure of real algebraic varieties. Ann. Math.
**66**, 545–556 (1957) MathSciNetCrossRefMATHGoogle Scholar - 54.Xu, J., Zikatanov, L.: The method of alternating projections and the method of subspace corrections in Hilbert space. J. Am. Math. Soc.
**15**, 573–598 (2002) MathSciNetCrossRefMATHGoogle Scholar - 55.Zangwill, W.I.: Nonlinear Programming. Prentice Hall, Englewood Cliffs (1969) MATHGoogle Scholar