Foundations of Computational Mathematics

, Volume 16, Issue 2, pp 457–492 | Cite as

A Quadratically Convergent Algorithm for Structured Low-Rank Approximation

Article

Abstract

Structured Low-Rank Approximation is a problem arising in a wide range of applications in Numerical Analysis and Engineering Sciences. Given an input matrix \(M\), the goal is to compute a matrix \(M'\) of given rank \(r\) in a linear or affine subspace \(E\) of matrices (usually encoding a specific structure) such that the Frobenius distance \(\left\| M-M' \right\| \) is small. We propose a Newton-like iteration for solving this problem, whose main feature is that it converges locally quadratically to such a matrix under mild transversality assumptions between the manifold of matrices of rank \(r\) and the linear/affine subspace \(E\). We also show that the distance between the limit of the iteration and the optimal solution of the problem is quadratic in the distance between the input matrix and the manifold of rank \(r\) matrices in \(E\). To illustrate the applicability of this algorithm, we propose a Maple implementation and give experimental results for several applicative problems that can be modeled by Structured Low-Rank Approximation: univariate approximate GCDs (Sylvester matrices), low-rank matrix completion (coordinate spaces) and denoising procedures (Hankel matrices).

Keywords

Structured Low-Rank Approximation Newton iteration  Quadratic convergence Approximate GCD Matrix completion 

Mathematics Subject Classification

65B99 65Y20 15A83 

References

  1. 1.
    Absil, P.A., Amodei, L., Meyer, G.: Two Newton methods on the manifold of fixed-rank matrices endowed with Riemannian quotient geometries. Computational Statistics (2013)Google Scholar
  2. 2.
    Allgower, E., Georg, K.: Numerical continuation methods, vol. 13. Springer-Verlag Berlin (1990)MATHGoogle Scholar
  3. 3.
    Arbarello, E., Cornalba, M., Griffiths, P., Harris, J.: Geometry of algebraic curves I, vol. 268. Springer (1984)Google Scholar
  4. 4.
    Ben-Israel, A.: A modified Newton-Raphson method for the solution of systems of equations. Israel Journal of Mathematics 3(2), 94–98 (1965)CrossRefMathSciNetMATHGoogle Scholar
  5. 5.
    Bini, D., Boito, P.: Structured matrix-based methods for polynomial-GCD: analysis and comparisons. In: Proceedings of the 2007 international symposium on Symbolic and algebraic computation, pp. 9–16. ACM (2007)Google Scholar
  6. 6.
    Bruns, W., Vetter, U.: Determinantal Rings. Springer (1988)Google Scholar
  7. 7.
    Cadzow, J.: Signal enhancement-a composite property mapping algorithm. IEEE Transactions on Acoustics, Speech and Signal Processing 36(1), 49–62 (1988)CrossRefMathSciNetMATHGoogle Scholar
  8. 8.
    Candes, E., Plan, Y.: Matrix completion with noise. Proceedings of the IEEE 98(6), 925–936 (2010)CrossRefGoogle Scholar
  9. 9.
    Candès, E., Recht, B.: Exact matrix completion via convex optimization. Foundations of Computational Mathematics 9(6), 717–772 (2009)CrossRefMathSciNetMATHGoogle Scholar
  10. 10.
    Candès, E., Tao, T.: The power of convex relaxation: Near-optimal matrix completion. Information Theory, IEEE Transactions on 56(5), 2053–2080 (2010)CrossRefGoogle Scholar
  11. 11.
    Chèze, G., Yakoubsohn, J.C., Galligo, A., Mourrain, B.: Computing nearest GCD with certification. In: Proceedings of the 2009 conference on Symbolic numeric computation, pp. 29–34. ACM (2009)Google Scholar
  12. 12.
    M., R., R. (2003) Structured Low Rank Approximation. Linear algebra and its applications 366:157–172CrossRefMathSciNetMATHGoogle Scholar
  13. 13.
    Condat, L., Hirabayashi, A.: Cadzow denoising upgraded: A new projection method for the recovery of Dirac pulses from noisy linear measurements (2012). PreprintGoogle Scholar
  14. 14.
    Corless, R., Watt, S., Zhi, L.: QR factoring to compute the GCD of univariate approximate polynomials. IEEE Transactions on Signal Processing 52(12), 3394–3402 (2004)CrossRefMathSciNetGoogle Scholar
  15. 15.
    Dedieu, J.P.: Points fixes, zéros et la méthode de Newton, vol. 54. Springer (2006)MATHGoogle Scholar
  16. 16.
    Dedieu, J.P., Kim, M.H.: Newton’s method for analytic systems of equations with constant rank derivatives. Journal of Complexity 18(1), 187–209 (2002)CrossRefMathSciNetMATHGoogle Scholar
  17. 17.
    Deutsch, F.: Best approximation in inner product spaces. Springer (2001)CrossRefMATHGoogle Scholar
  18. 18.
    Draisma, J., Horobet, E., Ottaviani, G., Sturmfels, B., Thomas, R.R.: The Euclidean distance degree of an algebraic variety. Foundations of Computational Mathematics (2015). To appearGoogle Scholar
  19. 19.
    Eisenbud, D.: Linear sections of determinantal varieties. American Journal of Mathematics 110(3), 541–575 (1988)CrossRefMathSciNetMATHGoogle Scholar
  20. 20.
    Emiris, I., Galligo, A., Lombardi, H.: Certified approximate univariate GCDs. Journal of Pure and Applied Algebra 117, 229–251 (1997)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Friedrichs, K.: On certain inequalities and characteristic value problems for analytic functions and for functions of two variables. Transactions of the American Mathematical Society 41(3), 321–364 (1937)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Gao, S., Kaltofen, E., May, J., Yang, Z., Zhi, L.: Approximate factorization of multivariate polynomials via differential equations. In: Proceedings of the 2004 International Symposium on Symbolic and Algebraic Computation, pp. 167–174. ACM (2004)Google Scholar
  23. 23.
    Golubitsky, M., Guillemin, V.: Stable mappings and their singularities, vol. 314. Springer-Verlag New York (1973)CrossRefMATHGoogle Scholar
  24. 24.
    Hogben, L. (ed.): Handbook of Linear Algebra. Discrete Mathematics and Its Applications. Taylor & Francis (2006)Google Scholar
  25. 25.
    Jain, P., Netrapalli, P., Sanghavi, S.: Low-rank matrix completion using alternating minimization. In: Proceedings of STOC’2013, pp. 665–674. ACM (2013)Google Scholar
  26. 26.
    Kaltofen, E., May, J., Yang, Z., Zhi, L.: Approximate factorization of multivariate polynomials using singular value decomposition. Journal of Symbolic Computation 43(5), 359–376 (2008)CrossRefMathSciNetMATHGoogle Scholar
  27. 27.
    Kaltofen, E., Yang, Z., Zhi, L.: Approximate greatest common divisors of several polynomials with linearly constrained coefficients and singular polynomials. In: Proceedings of the 2006 International Symposium on Symbolic and Algebraic Computation, pp. 169–176. ACM (2006)Google Scholar
  28. 28.
    Kaltofen, E., Yang, Z., Zhi, L.: Structured low rank approximation of a Sylvester matrix. In: Symbolic-numeric computation, pp. 69–83. Springer (2007)Google Scholar
  29. 29.
    Karmarkar, N., Lakshman, Y.: Approximate polynomial greatest common divisors and nearest singular polynomials. In: Proceedings of the 1996 International Symposium on Symbolic and Algebraic Computation, pp. 35–39. ACM (1996)Google Scholar
  30. 30.
    Karmarkar, N., Lakshman, Y.: On approximate GCDs of univariate polynomials. Journal of Symbolic Computation 26(6), 653–666 (1998)CrossRefMathSciNetMATHGoogle Scholar
  31. 31.
    Lewis, A.S., Malick, J.: Alternating projections on manifolds. Mathematics of Operations Research 33(1), 216–234 (2008)CrossRefMathSciNetMATHGoogle Scholar
  32. 32.
    Li B., Yang Z., Zhi L. (2005) Fast low rank approximation of a Sylvester matrix by structured total least norm. J. Japan Soc. Symbolic and Algebraic Comp 11:165–174Google Scholar
  33. 33.
    Markovsky, I.: Structured low-rank approximation and its applications. Automatica 44(4), 891–909 (2008)CrossRefMathSciNetMATHGoogle Scholar
  34. 34.
    Ottaviani, G., Spaenlehauer, P.J., Sturmfels, B.: Exact solutions in structured low-rank approximation. SIAM Journal on Matrix Analysis and Applications 35(4), 1521 – 1542 (2014)CrossRefMathSciNetMATHGoogle Scholar
  35. 35.
    Pan, V.: Computation of approximate polynomial GCDs and an extension. Information and Computation 167(2), 71–85 (2001)CrossRefMathSciNetMATHGoogle Scholar
  36. 36.
    Park, H., Zhang, L., Rosen, J.: Low rank approximation of a Hankel matrix by structured total least norm. BIT Numerical Mathematics 39(4), 757–779 (1999)CrossRefMathSciNetMATHGoogle Scholar
  37. 37.
    Recht, B.: A simpler approach to matrix completion. The Journal of Machine Learning Research pp. 3413–3430 (2011)Google Scholar
  38. 38.
    Recht, B., Xu, W., Hassibi, B.: Necessary and sufficient conditions for success of the nuclear norm heuristic for rank minimization. In: 47th IEEE Conference on Decision and Control, 2008., pp. 3065–3070. IEEE (2008)Google Scholar
  39. 39.
    Rosen, J., Park, H., Glick, J.: Structured total least norm for nonlinear problems. SIAM Journal on Matrix Analysis and Applications 20(1), 14–30 (1998)CrossRefMathSciNetMATHGoogle Scholar
  40. 40.
    Ruppert, W.: Reducibility of polynomials \(f(x, y)\) modulo \(p\). Journal of Number Theory 77, 62–70 (1999)CrossRefMathSciNetMATHGoogle Scholar
  41. 41.
    Schönhage, A.: Quasi-GCD computations. Journal of Complexity 1(1), 118–137 (1985)CrossRefMathSciNetMATHGoogle Scholar
  42. 42.
    Terui, A.: An iterative method for calculating approximate GCD of univariate polynomials. In: Proceedings of the 2009 International Symposium on Symbolic and Algebraic Computation, pp. 351–358. ACM (2009)Google Scholar
  43. 43.
    Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM Journal on Optimization (2013). AcceptedGoogle Scholar
  44. 44.
    Winkler, J., Allan, J.: Structured low rank approximations of the Sylvester resultant matrix for approximate GCDs of Bernstein basis polynomials. Electronic Transactions on Numerical Analysis 31, 141–155 (2008)MathSciNetMATHGoogle Scholar
  45. 45.
    Yakoubsohn, J.C., Masmoudi, M., Cheze, G., Auroux, D.: Approximate GCD a la Dedieu. Applied Mathematics E-Notes 11, 244–248 (2011)MathSciNetMATHGoogle Scholar
  46. 46.
    Zeng, Z., Dayton, B.: The approximate GCD of inexact polynomials. In: Proceedings of the 2004 International Symposium on Symbolic and Algebraic Computation, pp. 320–327. ACM (2004).Google Scholar

Copyright information

© SFoCM 2015

Authors and Affiliations

  1. 1.ORCCA and Department of Computer Science, Middlesex CollegeWestern UniversityLondonCanada
  2. 2.Inria Nancy Grand-Est, Université de Lorraine, CNRS, CARAMEL ProjectVillers-lès-Nancy CedexFrance
  3. 3.Max Planck Institute for MathematicsBonnGermany

Personalised recommendations