Skip to main content
SpringerLink
Log in

Sparse Approximation by Greedy Algorithms

  • Conference paper
  • First Online:
Mathematical Analysis, Probability and Applications – Plenary Lectures (ISAAC 2015)

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 177))

  • 990 Accesses

Abstract

It is a survey on recent results in constructive sparse approximation. Three directions are discussed here: (1) Lebesgue-type inequalities for greedy algorithms with respect to a special class of dictionaries, (2) constructive sparse approximation with respect to the trigonometric system, (3) sparse approximation with respect to dictionaries with tensor product structure. In all three cases constructive ways are provided for sparse approximation. The technique used is based on fundamental results from the theory of greedy approximation. In particular, results in the direction (1) are based on deep methods developed recently in compressed sensing. We present some of these results with detailed proofs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bazarkhanov, D., Temlyakov, V.: Nonlinear tensor product approximation of functions. J. Complexity 31, 867–884 (2015). arXiv:1409.1403v1 [stat.ML] 4 Sep 2014 (to appear in J. Complex. 2015)

  2. Belinskii, E.S.: Decomposition theorems and approximation by a “floating” system of exponentials. Trans. Am. Math. Soc. 350, 43–53 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  3. Dai, W., Milenkovic, O.: Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans. Inf. Theory 55, 2230–2249 (2009)

    Article  MathSciNet  Google Scholar 

  4. Dahmen, W., DeVore, R., Grasedyck, L., Süli, E.: Tensor-sparsity of solutions to high-dimensional elliptic Partial Differential Equations, Manuscript, 23 July 2014

    Google Scholar 

  5. Davis, G., Mallat, S., Avellaneda, M.: Adaptive greedy approximations. Constr. Approx. 13, 57–98 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  6. DeVore, R.A.: Nonlinear approximation. Acta Numerica 7, 51–150 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  7. DeVore, R.A., Temlyakov, V.N.: Nonlinear approximation by trigonometric sums. J. Fourier Anal. Appl. 2, 29–48 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dilworth, S.J., Kalton, N.J.: and Denka Kutzarova. On the existence of almost greedy bases in Banach spaces, Studia Math. 158, 67–101 (2003)

    MathSciNet  Google Scholar 

  9. Dilworth, S.J., Kutzarova, D., Temlyakov, V.N.: Convergence of some Greedy Algorithms in Banach spaces. J. Fourier Anal. Appl. 8, 489–505 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  10. Dilworth, S.J., Soto-Bajo, M., Temlyakov, V.N.: Quasi-greedy bases and Lebesgue-type inequalities. Stud. Math. 211, 41–69 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Garrigós, G., Hernández, E., Oikhberg, T.: Lebesgue type inequalities for quasi-greedy bases. Constr. Approx. 38, 447–479 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  12. Gluskin, E.D.: Extremal properties of orthogonal parallelpipeds and their application to the geometry of Banach spaces. Math USSR Sbornik 64, 85–96 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  13. Hackbusch, W.: Tensor Spaces and Numerical Tensor Calculus. Springer, Heidelberg (2012)

    Book  MATH  Google Scholar 

  14. Ismagilov, R.S.: Widths of sets in normed linear spaces and the approximation of functions by trigonometric polynomials. Uspekhi Mat. Nauk 29, 161–178 (1974); English transl. in Russian Math. Surv. 29 (1974)

    Google Scholar 

  15. Kashin, B.S.: Widths of certain finite-dimensional sets and classes of smooth functions. Izv. AN SSSR 41, 334–351 (1977); English transl. in Math. Izv. 11 (1977)

    Google Scholar 

  16. Kashin, B.S., Temlyakov, V.N.: On best \(m\)-term approximations and the entropy of sets in the space \(L^1\). Math. Notes 56, 1137–1157 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  17. Konyagin, S.V., Temlyakov, V.N.: A remark on greedy approximation in Banach spaces. East. J. Approx. 5, 365–379 (1999)

    MathSciNet  MATH  Google Scholar 

  18. Livshitz, E.D., Temlyakov, V.N.: Sparse approximation and recovery by greedy algorithms, IEEE Trans. Inf. Theory 60, 3989–4000 (2014). arXiv:1303.3595v1 [math.NA] 14 Mar 2013

    Google Scholar 

  19. Maiorov, V.E.: Trigonometric diameters of the Sobolev classes \(W^r_p\) in the space \(L_q\). Math. Notes 40, 590–597 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  20. Makovoz, Y.: On trigonometric \(n\)-widths and their generalizations. J. Approx. Theory 41, 361–366 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  21. Needell, D., Tropp, J.A.: CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26, 301–321 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  22. Needell, D., Vershynin, R.: Uniform uncertainty principle and signal recovery via orthogonal matching pursuit. Found. Comput. Math. 9, 317–334 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  23. Nielsen, M.: An example of an almost greedy uniformly bounded orthonormal basis for \(L_p(0,1)\). J. Approx. Theory 149, 188–192 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  24. Romanyuk, A.S.: Best \(M\)-term trigonometric approximations of Besov classes of periodic functions of several variables, Izvestia RAN, Ser. Mat. 67 (2003), 61–100; English transl. in. Izvestiya: Mathematics 67(2), 265 (2003)

    Article  MathSciNet  Google Scholar 

  25. Shalev-Shwartz, S., Srebro, N., Zhang, T.: Trading accuracy for sparsity in optimization problems with sparsity constrains. SIAM J. Optim. 20(6), 2807–2832 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Schmidt, E.: Zur Theorie der linearen und nichtlinearen Integralgleichungen. I. Math. Ann. 63, 433–476 (1907)

    Article  MathSciNet  Google Scholar 

  27. Schneider, R., Uschmajew, A.: Approximation rates for the hierarchical tensor format in periodic Sobolev spaces. J. Complex. 30, 56–71 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  28. Stechkin, S.B.: On absolute convergence of orthogonal series. Dokl. AN SSSR 102, 37–40 (1955) (in Russian)

    Google Scholar 

  29. Savu, D., Temlyakov, V.N.: Lebesgue-type inequalities for greedy approximation in Banach Spaces. IEEE Trans. Inf. Theory 58, 1098–1106 (2013)

    Article  MathSciNet  Google Scholar 

  30. Temlyakov, V.N.: Approximation of functions with bounded mixed derivative. Trudy MIAN 178, 1–112 (1986). English transl. in Proc. Steklov Inst. Math. 1 (1989)

    Google Scholar 

  31. Temlyakov, V.N.: Approximation of periodic functions of several variables by bilinear forms. Izvestiya AN SSSR 50, 137–155 (1986); English transl. in Math. USSR Izvestija 28, 133–150 (1987)

    Google Scholar 

  32. Temlyakov, V.N.: Estimates of the best bilinear approximations of functions of two variables and some of their applications. Mat. Sb. 134, 93–107 (1987); English transl. in Math. USSR-Sb 62, 95–109 (1989)

    Google Scholar 

  33. Temlyakov, V.N.: Estimates of best bilinear approximations of periodic functions. Trudy Mat. Inst. Steklov 181, 250–267 (1988); English transl. Proc. Steklov Inst. Math. 4, 275–293 (1989)

    Google Scholar 

  34. Temlyakov, V.N.: Estimates of best bilinear approximations of functions and approximation numbers of integral operators. Matem. Zametki 51, 125–134 (1992); English transl. in Math. Notes 51, 510–517 (1992)

    Google Scholar 

  35. Temlyakov, V.N.: Greedy algorithms with regard to multivariate systems with special structure. Constr. Approx. 16, 399–425 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  36. Temlyakov, V.N.: Nonlinear Kolmogorov’s widths. Matem. Zametki 63, 891–902 (1998)

    Article  MathSciNet  Google Scholar 

  37. Temlyakov, V.N.: Greedy algorithms in Banach spaces. Adv. Comput. Math. 14, 277–292 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  38. Temlyakov, V.N.: Nonlinear method of approximation. Found. Comput. Math. 3, 33–107 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  39. Temlyakov, V.N.: Greedy-type approximation in Banach Spaces and applications. Constr. Approx. 21, 257–292 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  40. Temlyakov, V.N.: Greedy Approximation. Cambridge University Press (2011)

    Google Scholar 

  41. Temlyakov, V.N.: Sparse approximation and recovery by greedy algorithms in Banach Spaces. Forum Math. Sigma 2, e12, e26 (2014). arXiv:1303.6811v1 [stat.ML] 27 Mar 2013, 1–27

  42. Temlyakov, V.N.: An inequality for the entropy numbers and its application. J. Approx. Theory 173, 110–121 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  43. Temlyakov, V.N.: Constructive sparse trigonometric approximation and other problems for functions with mixed smoothness. Math. Sbornik 206, 131–160 (2015). arXiv:1412.8647v1 [math.NA] 24 Dec 2014, 1–37 (to appear in Math. Sbornik, 2015)

    Google Scholar 

  44. Temlyakov, V.N.: Constructive sparse trigonometric approximation for functions with small mixed smoothness. arXiv:1503.0282v1 [math.NA] 1 Mar 2015, 1–30

  45. Temlyakov, V.N., Yang, M., Ye, P.: Greedy approximation with regard to non-greedy bases. Adv. Comput. Math. 34, 319–337 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  46. Wojtaszczyk, P.: Greedy algorithm for general biorthogonal systems. J. Approx. Theory 107, 293–314 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang, T.: Sequential greedy approximation for certain convex optimization problems. IEEE Trans. Inf. Theory 49(3), 682–691 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  48. Zhang, T.: Sparse recovery with orthogonal matching pursuit under RIP. IEEE Trans. Inf. Theory 57, 6215–6221 (2011)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

University of South Carolina and Steklov Institute of Mathematics. Research was supported by NSF grant DMS-1160841.

Author information

Authors and Affiliations

  1. University of South Carolina, Columbia, USA

    V. Temlyakov

  2. Steklov Institute of Mathematics, Moscow, Russia

    V. Temlyakov

Authors
  1. V. Temlyakov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. Temlyakov .

Editor information

Editors and Affiliations

  1. Faculty of Science and Technology, University of Macau, Taipa, Macao

    Tao Qian

  2. Department of Mathematics, University of Torino, Torino, Italy

    Luigi G. Rodino

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Temlyakov, V. (2016). Sparse Approximation by Greedy Algorithms. In: Qian, T., Rodino, L. (eds) Mathematical Analysis, Probability and Applications – Plenary Lectures. ISAAC 2015. Springer Proceedings in Mathematics & Statistics, vol 177. Springer, Cham. https://doi.org/10.1007/978-3-319-41945-9_7

Download citation

Publish with us

Policies and ethics

Search

Navigation