Constructive Approximation

, Volume 38, Issue 3, pp 489–525 | Cite as

Alternating Projections on Nontangential Manifolds



We consider sequences \((B_{k})_{k=0}^{\infty}\) of points obtained by projecting a given point B=B 0 back and forth between two manifolds \(\mathcal{M}_{1}\) and \(\mathcal{M}_{2}\), and give conditions guaranteeing that the sequence converges to a limit \(B_{\infty}\in\mathcal{M}_{1}\cap\mathcal{M}_{2}\). Our motivation is the study of algorithms based on finding the limit of such sequences, which have proved useful in a number of areas. The intersection is typically a set with desirable properties but for which there is no efficient method for finding the closest point B opt in \(\mathcal{M}_{1}\cap\mathcal{M}_{2}\). Under appropriate conditions, we prove not only that the sequence of alternating projections converges, but that the limit point is fairly close to B opt , in a manner relative to the distance ∥B 0B opt ∥, thereby significantly improving earlier results in the field.


Alternating projections Convergence Non-convexity Low-rank approximation Manifolds 

Mathematics Subject Classification

41A65 49Q99 53B25 



This work was supported by the Swedish Research Council and the Swedish Foundation for International Cooperation in Research and Higher Education, as well as Dirección de Investigación Científica y Technológica del Universidad de Santiago de Chile, Chile. Part of the work was conducted while Marcus Carlsson was employed at Universidad de Santiago de Chile. We thank Arne Meurman for fruitful discussions. We also would like to thank one of the reviewers for the constructive criticism and useful suggestions that have helped to improve the quality of the paper.


  1. 1.
    Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008) MATHGoogle Scholar
  2. 2.
    Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Lojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010) MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Badea, C., Grivaux, S., Müller, V.: A generalization of the Friedrichs angle and the method of alternating projections. C. R. Math. 348, 53–56 (2010) CrossRefMATHGoogle Scholar
  4. 4.
    Basu, S., Pollack, R., Roy, M.-F.: Algorithms in Real Algebraic Geometry. Algorithms and Computation in Mathematics, vol. 10. Springer, Berlin (2006) MATHGoogle Scholar
  5. 5.
    Bauschke, H.H., Borwein, J.M.: On the convergence of von Neumann’s alternating projection algorithm for two sets. Set-Valued Anal. 1, 185–212 (1993) MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Bauschke, H.H., Borwein, J.M.: Dykstra’s alternating projection algorithm for two sets. J. Approx. Theory 79, 418–443 (1994) MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996) MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Bauschke, H.H., Noll, D., Celler, A., Borwein, J.M.: An EM algorithm for dynamic SPECT. IEEE Trans. Med. Imag 18, 252–262 (1999) CrossRefGoogle Scholar
  9. 9.
    Berger, M., Gostiaux, B.: Differential Geometry: Manifolds, Curves and Surfaces. Springer, Berlin (1988) CrossRefMATHGoogle Scholar
  10. 10.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers. Now Publishers, Hanover (2011) Google Scholar
  11. 11.
    Boyd, S., Xiao, L.: Least squares covariance matrix adjustment. SIAM J. Matrix Anal. Appl. 27, 532–546 (2005) MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Brègman, L.M.: Finding the common point of convex sets by the method of successive projection. Dokl. Akad. Nauk SSSR 162, 487–490 (1965) MathSciNetGoogle Scholar
  13. 13.
    Cadzow, J.A.: Signal enhancement—a composite property mapping algorithm. IEEE Trans. Acoust. Speech Signal Process. 36, 49–62 (1988) CrossRefMATHGoogle Scholar
  14. 14.
    Combettes, P.L., Trussell, H.J.: Method of successive projections for finding a common point of sets in metric spaces. J. Optim. Theory Appl. 67, 487–507 (1990) MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Cox, D.A., Little, J.B., O’Shea, D.: Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, vol. 10. Springer, Berlin (2007) CrossRefGoogle Scholar
  16. 16.
    Deutsch, F.: Rate of convergence of the method of alternating projections. In: Parametric Optimization and Approximation, pp. 96–107 (1985) Google Scholar
  17. 17.
    Deutsch, F.: Best Approximation in Inner Product Spaces. CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, vol. 7. Springer, New York (2001) CrossRefMATHGoogle Scholar
  18. 18.
    Deutsch, F., Hundal, H.: The rate of convergence for the method of alternating projections, ii. J. Math. Anal. Appl. 205, 381–405 (1997) MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Dykstra, R.L.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78, 837–842 (1983) MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrika 1, 211–218 (1936) CrossRefMATHGoogle Scholar
  21. 21.
    Franklin, J.N.: Matrix Theory. Dover Publications in Mineola, New York (2000) Google Scholar
  22. 22.
    Friedrichs, K.: On certain inequalities and characteristic value problems for analytic functions and for functions of two variables. Trans. Am. Math. Soc. 41, 321–364 (1937) MathSciNetCrossRefGoogle Scholar
  23. 23.
    Gilbarg, D., Trudinger, N.: Elliptic Partial Differential Equations of Second Order, vol. 224. Springer, Berlin (2001) MATHGoogle Scholar
  24. 24.
    Grigoriadis, K.M., Frazho, A.E., Skelton, R.E.: Application of alternating convex projection methods for computation of positive Toeplitz matrices. IEEE Trans. Signal Process. 42, 1873–1875 (1994) CrossRefGoogle Scholar
  25. 25.
    Grubišić, I., Pietersz, R.: Efficient rank reduction of correlation matrices. Linear Algebra Appl. 422, 629–653 (2007) MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Gubin, L.G., Polyak, B.T., Raik, E.V.: The method of projections for finding the common point of convex sets. USSR Comput. Math. Math. Phys. 7, 1–24 (1967) CrossRefGoogle Scholar
  27. 27.
    Higham, N.J.: Computing the nearest correlation matrix—a problem from finance. IMA J. Numer. Anal. 22, 329–343 (2002) MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Hörmander, L.: The Analysis of Linear Partial Differential Operators. III. Springer, Berlin (2007) MATHGoogle Scholar
  29. 29.
    Kayalar, S., Weinert, H.L.: Error bounds for the method of alternating projections. Math. Control Signals Syst. (MCSS) 1, 43–59 (1988) MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Kendig, K.: Elementary Algebraic Geometry. Springer, Berlin (1977) CrossRefMATHGoogle Scholar
  31. 31.
    Krantz, S.G., Parks, H.R.: Distance to C k hypersurfaces. J. Differ. Equ. 40, 116–120 (1981) MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Lax, P.D.: Linear Algebra and Its Applications, 2nd edn. Pure and Applied Mathematics (Hoboken). Wiley-Interscience/Wiley, Hoboken (2007) MATHGoogle Scholar
  33. 33.
    Levi, A., Stark, H.: Signal restoration from phase by projections onto convex sets. J. Opt. Soc. Am. 73, 810–822 (1983) MathSciNetCrossRefGoogle Scholar
  34. 34.
    Lewis, A.S., Luke, D., Malick, J.: Local linear convergence for alternating and averaged nonconvex projections. Found. Comput. Math. 9, 485–513 (2009) MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    Lewis, A.S., Malick, J.: Alternating projections on manifolds. Math. Oper. Res. 33, 216–234 (2008) MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Li, Y., Liu, K., Razavilar, J.: A parameter estimation scheme for damped sinusoidal signals based on low-rank Hankel approximation. IEEE Trans. Signal Process. 45, 481–486 (1997) CrossRefGoogle Scholar
  37. 37.
    Lu, B., Wei, D., Evans, B.L., Bovik, A.C.: Improved matrix pencil methods. In: Conference Record of the Thirty-Second Asilomar Conference on Signals, Systems Computers, vol. 2, pp. 1433–1437 (1998) Google Scholar
  38. 38.
    Markovsky, I.: Structured low-rank approximation and its applications. Automatica 44, 891–909 (2008) MathSciNetCrossRefGoogle Scholar
  39. 39.
    Marks, R.J.: II, Alternating projections onto convex sets. In: Jansson, P.A. (ed.) Deconvolution of Images and Spectra, 2nd edn., pp. 476–501. Academic Press, San Diego (1997) Google Scholar
  40. 40.
    Montiel, S., Ros, A., Babbitt, D.G.: Curves and Surfaces, vol. 69. Am. Math. Soc., Providence (2009) MATHGoogle Scholar
  41. 41.
    Munkres, J.R.: Topology, 2nd edn. Prentice Hall, Upper Saddle River (2000) MATHGoogle Scholar
  42. 42.
    Nikolski, N.K.: Operators, Functions, and Systems: An Easy Reading. Vol. 1. Mathematical Surveys and Monographs, vol. 92. Am. Math. Soc., Providence (2002). Hardy, Hankel, and Toeplitz, translated from the French by Andreas Hartmann Google Scholar
  43. 43.
    Prabhu, V.U., Jalihal, D.: An improved ESPRIT based time-of-arrival estimation algorithm for vehicular OFDM systems. In: IEEE 69th Vehicular Technology Conference, 2009 VTC Spring 2009, April 2009, pp. 1–4. (2009) CrossRefGoogle Scholar
  44. 44.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Grundlehren der Mathematischen Wissenschaften, vol. 317. Springer, Berlin (1998) CrossRefMATHGoogle Scholar
  45. 45.
    Sard, A.: The measure of the critical values of differentiable maps. Bull. Am. Math. Soc. 48, 883–890 (1942) MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Schmidt, E.: Zur theorie der linearen und nichtlinearen integralgleichungen. iii, teil. Math. Ann. 65, 370–399 (1908) MathSciNetCrossRefMATHGoogle Scholar
  47. 47.
    Shafarevich, I.R.: Basic Algebraic Geometry. Springer, New York (1974). Translated from the Russian by K.A. Hirsch, Die Grundlehren der mathematischen Wissenschaften, Band 213 CrossRefMATHGoogle Scholar
  48. 48.
    Toh, K.C., Todd, M.J., Tütüncü, R.H.: SDPT3—a MATLAB software package for semidefinite programming, version 1.3. Optim. Methods Softw. 11, 545–581 (1999) MathSciNetCrossRefGoogle Scholar
  49. 49.
    Toh, K.C., Todd, M.J., Tütüncü, R.H.: On the implementation and usage of SDPT3—a Matlab software package for semidefinite-quadratic-linear programming, version 4.0. In: Handbook on Semidefinite, Conic and Polynomial Optimization, pp. 715–754 (2012) CrossRefGoogle Scholar
  50. 50.
    Trotman, D.: Singularity Theory. Scientific Publishing, Oxford (2007). Chap. Lectures on real stratification theory MATHGoogle Scholar
  51. 51.
    Tütüncü, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program. 95, 189–217 (2003) MathSciNetCrossRefMATHGoogle Scholar
  52. 52.
    von Neumann, J.: Functional Operators, Volume II: The Geometry of Orthogonal Spaces. Princeton University Press, Princeton (1950) Google Scholar
  53. 53.
    Whitney, H.: Elementary structure of real algebraic varieties. Ann. Math. 66, 545–556 (1957) MathSciNetCrossRefMATHGoogle Scholar
  54. 54.
    Xu, J., Zikatanov, L.: The method of alternating projections and the method of subspace corrections in Hilbert space. J. Am. Math. Soc. 15, 573–598 (2002) MathSciNetCrossRefMATHGoogle Scholar
  55. 55.
    Zangwill, W.I.: Nonlinear Programming. Prentice Hall, Englewood Cliffs (1969) MATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Centre for Mathematical SciencesLund UniversityLundSweden

Personalised recommendations