Basic Algorithms

  • Simon Foucart
  • Holger Rauhut
Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


This chapter outlines several sparse reconstruction techniques analyzed throughout the book. More precisely, we present optimization methods, greedy methods, and thresholding-based methods. In each case, only intuition and basic facts about the algorithms are provided at this point.


1-minimization basis pursuit quadratically constrained basis pursuit orthogonal matching pursuit CoSaMP basic thresholding iterative hard thresholding hard thresholding pursuit 


  1. 70.
    S. Boyd, L. Vandenberghe, Convex Optimization (Cambridge University Press, Cambridge, 2004)MATHCrossRefGoogle Scholar
  2. 98.
    E.J. Candès, T. Tao, The Dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 35(6), 2313–2351, (2007)Google Scholar
  3. 113.
    S. Chen, S. Billings, W. Luo, Orthogonal least squares methods and their application to nonlinear system identification. Intl. J. Contr. 50(5), 1873–1896 (1989)MathSciNetMATHCrossRefGoogle Scholar
  4. 114.
    S.S. Chen, D.L. Donoho, M.A. Saunders, Atomic decomposition by Basis Pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1999)MathSciNetMATHCrossRefGoogle Scholar
  5. 135.
    W. Dai, O. Milenkovic, Subspace Pursuit for Compressive Sensing Signal Reconstruction. IEEE Trans. Inform. Theor. 55(5), 2230–2249 (2009)MathSciNetCrossRefGoogle Scholar
  6. 145.
    G. Davis, S. Mallat, Z. Zhang, Adaptive time-frequency decompositions. Opt. Eng. 33(7), 2183–2191 (1994)CrossRefGoogle Scholar
  7. 162.
    D.L. Donoho, A. Maleki, A. Montanari, Message-passing algorithms for compressed sensing. Proc. Natl. Acad. Sci. USA 106(45), 18914–18919 (2009)CrossRefGoogle Scholar
  8. 214.
    J. Friedman, W. Stuetzle, Projection pursuit regressions. J. Am. Stat. Soc. 76, 817–823 (1981)MathSciNetCrossRefGoogle Scholar
  9. 277.
    J. Högborn, Aperture synthesis with a non-regular distribution of interferometer baselines. Astronom. and Astrophys. 15, 417 (1974)Google Scholar
  10. 342.
    S. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41(12), 3397–3415 (1993)MATHCrossRefGoogle Scholar
  11. 361.
    D. Needell, J. Tropp, CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301–321 (2008)MathSciNetCrossRefGoogle Scholar
  12. 362.
    D. Needell, R. Vershynin, Uniform uncertainty principle and signal recovery via regularized orthogonal matching pursuit. Found. Comput. Math. 9(3), 317–334 (2009)MathSciNetMATHCrossRefGoogle Scholar
  13. 363.
    D. Needell, R. Vershynin, Signal recovery from incomplete and inaccurate measurements via regularized orthogonal matching pursuit. IEEE J. Sel. Top. Signal Process. 4(2), 310–316 (April 2010)CrossRefGoogle Scholar
  14. 369.
    J. Nocedal, S. Wright, Numerical Optimization, 2nd edn. Springer Series in Operations Research and Financial Engineering (Springer, New York, 2006)Google Scholar
  15. 378.
    Y.C. Pati, R. Rezaiifar, P.S. Krishnaprasad, Orthogonal Matching Pursuit: Recursive Function Approximation with Applications to Wavelet Decomposition. In 1993 Conference Record of The Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, Nov. 1–3, 1993., pp. 40–44, 1993Google Scholar
  16. 404.
    S. Qian, D. Chen, Signal representation using adaptive normalized Gaussian functions. Signal Process. 36(1), 1–11 (1994)MATHCrossRefGoogle Scholar
  17. 470.
    V. Temlyakov, Nonlinear methods of approximation. Found. Comput. Math. 3(1), 33–107 (2003)MathSciNetMATHCrossRefGoogle Scholar
  18. 471.
    V. Temlyakov, Greedy approximation. Act. Num. 17, 235–409 (2008)MathSciNetMATHGoogle Scholar
  19. 472.
    V. Temlyakov, Greedy Approximation. Cambridge Monographs on Applied and Computational Mathematics, vol. 20 (Cambridge University Press, Cambridge, 2011)Google Scholar
  20. 473.
    R. Tibshirani, Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. B 58(1), 267–288 (1996)MathSciNetMATHGoogle Scholar
  21. 476.
    J.A. Tropp, Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inform. Theor. 50(10), 2231–2242 (2004)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Simon Foucart
    • 1
  • Holger Rauhut
    • 2
  1. 1.Department of MathematicsDrexel UniversityPhiladelphiaUSA
  2. 2.Lehrstuhl C für Mathematik (Analysis)RWTH Aachen UniversityAachenGermany

Personalised recommendations