Advertisement

Pursuit Algorithms – Guarantees

  • Michael Elad
Chapter

Abstract

Assume that the linear system A x = b has a sparse solution with k0 non-zeros, i.e., \(\parallel \mathbf{X}\parallel_{0}=k_0.\) Furthermore, assume that k 0 < spark(A)/2. Will matching pursuit or Basis-Pursuit succeed in recovering the sparsest solution? Clearly, such success cannot be expected for all k 0 and for all matrices A, since this would conflict with the known NP-hardness of the problem in the general case. However, if the equation actually has a "suffciently sparse" solution, the success of these algorithms in addressing the original objective (P 0) can be guaranteed.

Keywords

Sparse Representation Atomic Decomposition Generalize Uncertainty Principle Basis Pursuit Sparse Solution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. 1.
    J.H. Ahlberg and E.N. Nilson, Convergence properties of the spline fit, J. SIAM, 11:95–104, 1963.MATHMathSciNetGoogle Scholar
  2. 2.
    S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Journal on Scientific Computing, 20(1):33–61 (1998).CrossRefMathSciNetGoogle Scholar
  3. 3.
    S.S. Chen, D.L. Donoho, and M.A. Saunders, Atomic decomposition by basis pursuit, SIAM Review, 43(1):129–159, 2001.MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    C. Couvreur and Y. Bresler, On the optimality of the Backward Greedy Algorithm for the subset selection problem, SIAM Journal on Matrix Analysis and Applications, 21(3):797–808, 2000.MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    D.L. Donoho and M. Elad, Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization, Proc. of the National Academy of Sciences, 100(5):219227–02, 2003.CrossRefMathSciNetGoogle Scholar
  6. 6.
    D.L. Donoho and X. Huo, Uncertainty principles and ideal atomic decomposition, IEEE Trans. on Information Theory, 47(7):2845–2862, 1999.CrossRefMathSciNetGoogle Scholar
  7. 7.
    D.L. Donoho and J. Tanner, Sparse nonnegative solutions of underdetermined linear equations by linear programming, Proceedings of the National Academy of Sciences, 102(27):9446–9451, July 2005.CrossRefMathSciNetGoogle Scholar
  8. 8.
    D.L. Donoho and J. Tanner, Neighborliness of randomly-projected Simplices in high dimensions, Proceedings of the National Academy of Sciences, 102(27):9452– 9457, July 2005.Google Scholar
  9. 9.
    M. Elad and A.M. Bruckstein, A generalized uncertainty principle and sparse representation in pairs of bases, IEEE Trans. on Information Theory, 48:2558– 2567, 2002.MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    A. Feuer and A. Nemirovsky, On sparse representation in pairs of bases, IEEE Trans. On Information Theory, 49:1579–1581, June 2002.CrossRefGoogle Scholar
  11. 11.
    J.J. Fuchs, On sparse representations in arbitrary redundant bases, IEEE Trans. On Information Theory, 50:1341–1344, 2004.CrossRefGoogle Scholar
  12. 12.
    R. Gribonval and M. Nielsen, Sparse decompositions in unions of bases, IEEE Trans. On Information Theory, 49(12):3320–3325, 2003.CrossRefMathSciNetGoogle Scholar
  13. 13.
    X. Huo, Sparse Image representation Via Combined Transforms, PhD thesis, Stanford, 1999.Google Scholar
  14. 14.
    S. Mallat and Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Trans. Signal Processing, 41(12):3397–3415, 1993.MATHCrossRefGoogle Scholar
  15. 15.
    N. Morača, Bounds for norms of the matrix inverse and the smallest singular value, Linear Algebra and its Applications, 429:2589–2601, 2008.MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    J.A. Tropp, Greed is good: Algorithmic results for sparse approximation, IEEE Trans. On Information Theory, 50(10):2231–2242, October 2004.CrossRefMathSciNetGoogle Scholar
  17. 17.
    J.M. Varah, A lower bound for the smallest singular value of a matrix, Linear Algebra Appl. 11:3–5, 1975.MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Computer Science DepartmentThe Technion – Israel Institute of TechnologyHaifaIsrael

Personalised recommendations