Advances in Computational Mathematics

, Volume 12, Issue 2–3, pp 213–227

Weak greedy algorithms[*]This research was supported by National Science Foundation Grant DMS 9970326 and by ONR Grant N00014‐96‐1‐1003.

  • V.N. Temlyakov


Theoretical greedy type algorithms are studied: a Weak Greedy Algorithm, a Weak Orthogonal Greedy Algorithm, and a Weak Relaxed Greedy Algorithm. These algorithms are defined by weaker assumptions than their analogs the Pure Greedy Algorithm, an Orthogonal Greedy Algorithm, and a Relaxed Greedy Algorithm. The weaker assumptions make these new algorithms more ready for practical implementation. We prove the convergence theorems and also give estimates for the rate of approximation by means of these algorithms. The convergence and the estimates apply to approximation from an arbitrary dictionary in a Hilbert space.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A.R. Barron, Universal approximation bounds for superposition of n sigmoidal functions, IEEE Trans. Inform. Theory 39 (1993) 930-945.MATHMathSciNetCrossRefGoogle Scholar
  2. [2]
    C. Darken, M. Donahue, L. Gurvits and E. Sontag, Rate of convex approximation in non-Hilbert spaces, Constr. Approx. 13 (1997) 187-220.MATHMathSciNetCrossRefGoogle Scholar
  3. [3]
    G. Davis, S. Mallat and M. Avellaneda, Adaptive greedy approximations, Constr. Approx. 13 (1997) 57-98.MATHMathSciNetCrossRefGoogle Scholar
  4. [4]
    R.A. DeVore, Nonlinear approximation, Acta Numer. (1998) 51-150.Google Scholar
  5. [5]
    R. DeVore, B. Jawerth and V. Popov, Compression of wavelet decompositions, Amer. J. Math. 114 (1992) 737-785.MATHMathSciNetCrossRefGoogle Scholar
  6. [6]
    R.A. DeVore and V.N. Temlyakov, Nonlinear approximation by trigonometric sums, J. Fourier Anal. Appl. 2 (1995) 29-48.MATHMathSciNetCrossRefGoogle Scholar
  7. [7]
    R.A. DeVore and V.N. Temlyakov, Some remarks on Greedy Algorithms, Adv. Comput. Math. 5 (1996) 173-187.MATHMathSciNetCrossRefGoogle Scholar
  8. [8]
    R.A. DeVore and V.N. Temlyakov, Nonlinear approximation in finite-dimensional spaces, J. Complexity 13 (1997) 489-508.MATHMathSciNetCrossRefGoogle Scholar
  9. [9]
    D.L. Donoho, Unconditional bases are optimal bases for data compression and for statistical estimation, Appl. Comput. Harmon. Anal. 1 (1993) 100-115.MATHMathSciNetCrossRefGoogle Scholar
  10. [10]
    D.L. Donoho, CART and best-ortho-basis: A connection, Preprint (1995) 1-45.Google Scholar
  11. [11]
    V.V. Dubinin, Greedy algorithms and applications, Ph.D. thesis, University of South Carolina (1997).Google Scholar
  12. [12]
    J.H. Friedman and W. Stuetzle, Projection pursuit regression, J. Amer. Statist. Assoc. 76 (1981) 817-823.MathSciNetCrossRefGoogle Scholar
  13. [13]
    P.J. Huber, Projection pursuit, Ann. Statist. 13 (1985) 435-475.MATHMathSciNetGoogle Scholar
  14. [14]
    L. Jones, On a conjecture of Huber concerning the convergence of projection pursuit regression, Ann. Statist. 15 (1987) 880-882.MATHMathSciNetGoogle Scholar
  15. [15]
    L. Jones, A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training, Ann. Statist. 20 (1992) 608-613.MATHMathSciNetGoogle Scholar
  16. [16]
    B.S. Kashin and V.N. Temlyakov, On best m-terms approximations and the entropy of sets in the space L1, Math. Notes 56 (1994) 57-86.MATHMathSciNetGoogle Scholar
  17. [17]
    B.S. Kashin and V.N. Temlyakov, On estimating approximative characteristics of classes of functions with bounded mixed derivative, Math. Notes 58 (1995) 922-925.MATHMathSciNetGoogle Scholar
  18. [18]
    L. Rejtö and G.G. Walter, Remarks on projection pursuit regression and density estimation, Stochast. Anal. Appl. 10 (1992) 213-222.MATHGoogle Scholar
  19. [19]
    E. Schmidt, Zur Theorie der linearen und nichtlinearen Integralgleichungen. I, Math. Ann. 63 (1906-1907) 433-476.CrossRefGoogle Scholar
  20. [20]
    V.N. Temlyakov, Greedy algorithm and m-term trigonometric approximation, Constr. Approx. 14 (1998) 569-587.MATHMathSciNetCrossRefGoogle Scholar
  21. [21]
    V.N. Temlyakov, The best m-term approximation and greedy algorithms, Adv. Comput. Math. 8 (1998) 249-265.MATHMathSciNetCrossRefGoogle Scholar
  22. [22]
    V.N. Temlyakov, Nonlinear m-term approximation with regard to the multivariate Haar system, East J. Approx. 4 (1998) 87-106.MATHMathSciNetGoogle Scholar
  23. [23]
    V.N. Temlyakov, Greedy algorithms with regard to the multivariate systems with a special structure, Preprint (1998) 1-26.Google Scholar
  24. [24]
    V.N. Temlyakov, Greedy algorithms and m-term approximation with regard to redundant dictionaries, J. Approx. Theory 98 (1999) 117-145.MATHMathSciNetCrossRefGoogle Scholar
  25. [25]
    A. Zygmund, Trigonometric Series (Cambridge University Press, Cambridge, 1959).Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • V.N. Temlyakov
    • 1
  1. 1.Department of MathematicsUniversity of South CarolinaColumbiaUSA

Personalised recommendations