Skip to main content
SpringerLink
Log in

A Quadratically Convergent Algorithm for Structured Low-Rank Approximation

  • Published:
Foundations of Computational Mathematics Aims and scope Submit manuscript

Abstract

Structured Low-Rank Approximation is a problem arising in a wide range of applications in Numerical Analysis and Engineering Sciences. Given an input matrix \(M\), the goal is to compute a matrix \(M'\) of given rank \(r\) in a linear or affine subspace \(E\) of matrices (usually encoding a specific structure) such that the Frobenius distance \(\left\| M-M' \right\| \) is small. We propose a Newton-like iteration for solving this problem, whose main feature is that it converges locally quadratically to such a matrix under mild transversality assumptions between the manifold of matrices of rank \(r\) and the linear/affine subspace \(E\). We also show that the distance between the limit of the iteration and the optimal solution of the problem is quadratic in the distance between the input matrix and the manifold of rank \(r\) matrices in \(E\). To illustrate the applicability of this algorithm, we propose a Maple implementation and give experimental results for several applicative problems that can be modeled by Structured Low-Rank Approximation: univariate approximate GCDs (Sylvester matrices), low-rank matrix completion (coordinate spaces) and denoising procedures (Hankel matrices).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Absil, P.A., Amodei, L., Meyer, G.: Two Newton methods on the manifold of fixed-rank matrices endowed with Riemannian quotient geometries. Computational Statistics (2013)

  2. Allgower, E., Georg, K.: Numerical continuation methods, vol. 13. Springer-Verlag Berlin (1990)

    MATH  Google Scholar 

  3. Arbarello, E., Cornalba, M., Griffiths, P., Harris, J.: Geometry of algebraic curves I, vol. 268. Springer (1984)

    Google Scholar 

  4. Ben-Israel, A.: A modified Newton-Raphson method for the solution of systems of equations. Israel Journal of Mathematics 3(2), 94–98 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bini, D., Boito, P.: Structured matrix-based methods for polynomial-GCD: analysis and comparisons. In: Proceedings of the 2007 international symposium on Symbolic and algebraic computation, pp. 9–16. ACM (2007)

  6. Bruns, W., Vetter, U.: Determinantal Rings. Springer (1988)

    Google Scholar 

  7. Cadzow, J.: Signal enhancement-a composite property mapping algorithm. IEEE Transactions on Acoustics, Speech and Signal Processing 36(1), 49–62 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  8. Candes, E., Plan, Y.: Matrix completion with noise. Proceedings of the IEEE 98(6), 925–936 (2010)

    Article  Google Scholar 

  9. Candès, E., Recht, B.: Exact matrix completion via convex optimization. Foundations of Computational Mathematics 9(6), 717–772 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  10. Candès, E., Tao, T.: The power of convex relaxation: Near-optimal matrix completion. Information Theory, IEEE Transactions on 56(5), 2053–2080 (2010)

    Article  Google Scholar 

  11. Chèze, G., Yakoubsohn, J.C., Galligo, A., Mourrain, B.: Computing nearest GCD with certification. In: Proceedings of the 2009 conference on Symbolic numeric computation, pp. 29–34. ACM (2009)

  12. M., R., R. (2003) Structured Low Rank Approximation. Linear algebra and its applications 366:157–172

    Article  MathSciNet  MATH  Google Scholar 

  13. Condat, L., Hirabayashi, A.: Cadzow denoising upgraded: A new projection method for the recovery of Dirac pulses from noisy linear measurements (2012). Preprint

  14. Corless, R., Watt, S., Zhi, L.: QR factoring to compute the GCD of univariate approximate polynomials. IEEE Transactions on Signal Processing 52(12), 3394–3402 (2004)

    Article  MathSciNet  Google Scholar 

  15. Dedieu, J.P.: Points fixes, zéros et la méthode de Newton, vol. 54. Springer (2006)

    MATH  Google Scholar 

  16. Dedieu, J.P., Kim, M.H.: Newton’s method for analytic systems of equations with constant rank derivatives. Journal of Complexity 18(1), 187–209 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  17. Deutsch, F.: Best approximation in inner product spaces. Springer (2001)

    Book  MATH  Google Scholar 

  18. Draisma, J., Horobet, E., Ottaviani, G., Sturmfels, B., Thomas, R.R.: The Euclidean distance degree of an algebraic variety. Foundations of Computational Mathematics (2015). To appear

  19. Eisenbud, D.: Linear sections of determinantal varieties. American Journal of Mathematics 110(3), 541–575 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  20. Emiris, I., Galligo, A., Lombardi, H.: Certified approximate univariate GCDs. Journal of Pure and Applied Algebra 117, 229–251 (1997)

    Article  MathSciNet  Google Scholar 

  21. Friedrichs, K.: On certain inequalities and characteristic value problems for analytic functions and for functions of two variables. Transactions of the American Mathematical Society 41(3), 321–364 (1937)

    Article  MathSciNet  Google Scholar 

  22. Gao, S., Kaltofen, E., May, J., Yang, Z., Zhi, L.: Approximate factorization of multivariate polynomials via differential equations. In: Proceedings of the 2004 International Symposium on Symbolic and Algebraic Computation, pp. 167–174. ACM (2004)

  23. Golubitsky, M., Guillemin, V.: Stable mappings and their singularities, vol. 314. Springer-Verlag New York (1973)

    Book  MATH  Google Scholar 

  24. Hogben, L. (ed.): Handbook of Linear Algebra. Discrete Mathematics and Its Applications. Taylor & Francis (2006)

  25. Jain, P., Netrapalli, P., Sanghavi, S.: Low-rank matrix completion using alternating minimization. In: Proceedings of STOC’2013, pp. 665–674. ACM (2013)

  26. Kaltofen, E., May, J., Yang, Z., Zhi, L.: Approximate factorization of multivariate polynomials using singular value decomposition. Journal of Symbolic Computation 43(5), 359–376 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  27. Kaltofen, E., Yang, Z., Zhi, L.: Approximate greatest common divisors of several polynomials with linearly constrained coefficients and singular polynomials. In: Proceedings of the 2006 International Symposium on Symbolic and Algebraic Computation, pp. 169–176. ACM (2006)

  28. Kaltofen, E., Yang, Z., Zhi, L.: Structured low rank approximation of a Sylvester matrix. In: Symbolic-numeric computation, pp. 69–83. Springer (2007)

  29. Karmarkar, N., Lakshman, Y.: Approximate polynomial greatest common divisors and nearest singular polynomials. In: Proceedings of the 1996 International Symposium on Symbolic and Algebraic Computation, pp. 35–39. ACM (1996)

  30. Karmarkar, N., Lakshman, Y.: On approximate GCDs of univariate polynomials. Journal of Symbolic Computation 26(6), 653–666 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  31. Lewis, A.S., Malick, J.: Alternating projections on manifolds. Mathematics of Operations Research 33(1), 216–234 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  32. Li B., Yang Z., Zhi L. (2005) Fast low rank approximation of a Sylvester matrix by structured total least norm. J. Japan Soc. Symbolic and Algebraic Comp 11:165–174

    Google Scholar 

  33. Markovsky, I.: Structured low-rank approximation and its applications. Automatica 44(4), 891–909 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  34. Ottaviani, G., Spaenlehauer, P.J., Sturmfels, B.: Exact solutions in structured low-rank approximation. SIAM Journal on Matrix Analysis and Applications 35(4), 1521 – 1542 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  35. Pan, V.: Computation of approximate polynomial GCDs and an extension. Information and Computation 167(2), 71–85 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  36. Park, H., Zhang, L., Rosen, J.: Low rank approximation of a Hankel matrix by structured total least norm. BIT Numerical Mathematics 39(4), 757–779 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  37. Recht, B.: A simpler approach to matrix completion. The Journal of Machine Learning Research pp. 3413–3430 (2011)

  38. Recht, B., Xu, W., Hassibi, B.: Necessary and sufficient conditions for success of the nuclear norm heuristic for rank minimization. In: 47th IEEE Conference on Decision and Control, 2008., pp. 3065–3070. IEEE (2008)

  39. Rosen, J., Park, H., Glick, J.: Structured total least norm for nonlinear problems. SIAM Journal on Matrix Analysis and Applications 20(1), 14–30 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  40. Ruppert, W.: Reducibility of polynomials \(f(x, y)\) modulo \(p\). Journal of Number Theory 77, 62–70 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  41. Schönhage, A.: Quasi-GCD computations. Journal of Complexity 1(1), 118–137 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  42. Terui, A.: An iterative method for calculating approximate GCD of univariate polynomials. In: Proceedings of the 2009 International Symposium on Symbolic and Algebraic Computation, pp. 351–358. ACM (2009)

  43. Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM Journal on Optimization (2013). Accepted

  44. Winkler, J., Allan, J.: Structured low rank approximations of the Sylvester resultant matrix for approximate GCDs of Bernstein basis polynomials. Electronic Transactions on Numerical Analysis 31, 141–155 (2008)

    MathSciNet  MATH  Google Scholar 

  45. Yakoubsohn, J.C., Masmoudi, M., Cheze, G., Auroux, D.: Approximate GCD a la Dedieu. Applied Mathematics E-Notes 11, 244–248 (2011)

    MathSciNet  MATH  Google Scholar 

  46. Zeng, Z., Dayton, B.: The approximate GCD of inexact polynomials. In: Proceedings of the 2004 International Symposium on Symbolic and Algebraic Computation, pp. 320–327. ACM (2004).

Download references

Acknowledgments

We are grateful to Erich Kaltofen, Giorgio Ottaviani, Olivier Ruatta, Bruno Salvy, Bernd Sturmfels and Agnes Szanto for useful discussions and for pointing out important references. We also wish to thank an anonymous referee for his useful comments which led to the second variant of the algorithm. We acknowledge the financial support of NSERC and of the Canada Research Chairs program.

Author information

Author notes

  1. Pierre-Jean Spaenlehauer

    Present address: Inria Nancy Grand-Est, Université de Lorraine, CNRS, CARAMEL Project, Bâtiment A. 615, rue du Jardin Botanique, 54600, Villers-lès-Nancy Cedex, France

Authors and Affiliations

  1. ORCCA and Department of Computer Science, Middlesex College, Western University, London, ON, N6A 3K7, Canada

    Éric Schost & Pierre-Jean Spaenlehauer

  2. Max Planck Institute for Mathematics, Bonn, Germany

    Pierre-Jean Spaenlehauer

Authors
  1. Éric Schost
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pierre-Jean Spaenlehauer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pierre-Jean Spaenlehauer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schost, É., Spaenlehauer, PJ. A Quadratically Convergent Algorithm for Structured Low-Rank Approximation. Found Comput Math 16, 457–492 (2016). https://doi.org/10.1007/s10208-015-9256-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10208-015-9256-x

Keywords

Mathematics Subject Classification

Search

Navigation