Sparse Solutions of Underdetermined Systems

  • Simon Foucart
  • Holger Rauhut
Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


The notions of sparsity and compressibility are formally defined in this chapter, and some useful inequalities are established along the way. Then the question about the minimal number of linear equations to solve underdetermined systems admitting s-sparse solutions is answered. This is equivalent to the question about the minimal number of linear measurements to recover s-sparse vectors via \(\ell_{0}\)-minimization, which is shown to be NP-hard in general. A practical (but unstable) procedure to recover all s-sparse vectors using the minimal number of Fourier measurements is also presented.


0-norm sparsity compressibility best s-term approximation nonincreasing rearrangement p-space weak p-space 0-minimization partial Fourier matrix Prony method NP-hardness exact cover by 3-sets problem 


  1. 14.
    F. Andersson, M. Carlsson, M.V. de Hoop, Nonlinear approximation of functions in two dimensions by sums of exponentials. Appl. Comput. Harmon. Anal. 29(2), 198–213 (2010)MathSciNetMATHCrossRefGoogle Scholar
  2. 15.
    F. Andersson, M. Carlsson, M.V. de Hoop, Sparse approximation of functions using sums of exponentials and AAK theory. J. Approx. Theor. 163(2), 213–248 (2011)MATHCrossRefGoogle Scholar
  3. 19.
    S. Arora, B. Barak, Computational Complexity: A Modern Approach (Cambridge University Press, Cambridge, 2009)CrossRefGoogle Scholar
  4. 45.
    G. Beylkin, L. Monzón, On approximation of functions by exponential sums. Appl. Comput. Harmon. Anal. 19(1), 17–48 (2005)MathSciNetMATHCrossRefGoogle Scholar
  5. 46.
    G. Beylkin, L. Monzón, Approximation by exponential sums revisited. Appl. Comput. Harmon. Anal. 28(2), 131–149 (2010)MathSciNetMATHCrossRefGoogle Scholar
  6. 52.
    R. Blahut, Algebraic Codes for Data Transmission (Cambridge University Press, Cambridge, 2003)MATHCrossRefGoogle Scholar
  7. 55.
    T. Blu, P. Marziliano, M. Vetterli, Sampling signals with finite rate of innovation. IEEE Trans. Signal Process. 50(6), 1417–1428 (2002)MathSciNetCrossRefGoogle Scholar
  8. 123.
    A. Cohen, W. Dahmen, R.A. DeVore, Compressed sensing and best k-term approximation. J. Amer. Math. Soc. 22(1), 211–231 (2009)MathSciNetMATHCrossRefGoogle Scholar
  9. 155.
    D.L. Donoho, M. Elad, Optimally sparse representations in general (non-orthogonal) dictionaries via 1 minimization. Proc. Nat. Acad. Sci. 100(5), 2197–2202 (2003)MathSciNetMATHCrossRefGoogle Scholar
  10. 209.
    S. Foucart, Sparse recovery algorithms: sufficient conditions in terms of restricted isometry constants. In Approximation Theory XIII: San Antonio 2010, ed. by M. Neamtu, L. Schumaker. Springer Proceedings in Mathematics, vol. 13 (Springer, New York, 2012), pp. 65–77Google Scholar
  11. 220.
    D. Ge, X. Jiang, Y. Ye, A note on complexity of l p minimization. Math. Program. 129, 285–299 (2011)MathSciNetMATHCrossRefGoogle Scholar
  12. 225.
    A.C. Gilbert, M. Strauss, J.A. Tropp, R. Vershynin, One sketch for all: fast algorithms for compressed sensing. In Proceedings of the Thirty-ninth Annual ACM Symposium on Theory of Computing, STOC ’07, pp. 237–246, ACM, New York, NY, USA, 2007Google Scholar
  13. 232.
    R.A. Gopinath, Nonlinear recovery of sparse signals from narrowband data. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing. ICASSP ’95, vol. 2, pp. 1237–1239, IEEE Computer Society, 1995Google Scholar
  14. 284.
    R. Hunt, On L(p, q) spaces. Enseignement Math. (2) 12, 249–276 (1966)Google Scholar
  15. 298.
    S. Karlin, Total Positivity Vol. I (Stanford University Press, Stanford, 1968)Google Scholar
  16. 313.
    J. Kruskal, Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Lin. Algebra Appl. 18(2), 95–138 (1977)MathSciNetMATHCrossRefGoogle Scholar
  17. 344.
    S. Marple, Digital Spectral Analysis with Applications (Prentice-Hall, Englewood Cliffs, 1987)Google Scholar
  18. 357.
    T.K. Moon, W.C. Stirling, Mathematical Methods and Algorithms for Signal Processing. (Prentice-Hall, Upper Saddle River, NJ, 2000)Google Scholar
  19. 359.
    B.K. Natarajan, Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 227–234 (1995)MathSciNetMATHCrossRefGoogle Scholar
  20. 389.
    A. Pinkus, Totally Positive Matrices. Cambridge Tracts in Mathematics, vol. 181 (Cambridge University Press, Cambridge, 2010)Google Scholar
  21. 401.
    D. Potts, M. Tasche, Parameter estimation for exponential sums by approximate Prony method. Signal Process. 90(5), 1631–1642 (2010)MATHCrossRefGoogle Scholar
  22. 402.
    R. Prony, Essai expérimental et analytique sur les lois de la Dilatabilité des fluides élastiques et sur celles de la Force expansive de la vapeur de l’eau et de la vapeur de l’alkool, à différentes températures. J. École Polytechnique 1, 24–76 (1795)Google Scholar
  23. 503.
    M. Wakinm, The Geometry of Low-dimensional Signal Models, PhD thesis, Rice University, 2006Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Simon Foucart
    • 1
  • Holger Rauhut
    • 2
  1. 1.Department of MathematicsDrexel UniversityPhiladelphiaUSA
  2. 2.Lehrstuhl C für Mathematik (Analysis)RWTH Aachen UniversityAachenGermany

Personalised recommendations