Mathematical Notes

, Volume 92, Issue 3–4, pp 485–489 | Cite as

Comparison of the convergence rate of pure greedy and orthogonal greedy algorithms

  • A. V. DereventsovEmail author


The following two types of greedy algorithms are considered: the pure greedy algorithm (PGA) and the orthogonal greedy algorithm (OGA). From the standpoint of estimating the rate of convergence on the entire class A 1(D), the orthogonal greedy algorithm is optimal and significantly exceeds the pure greedy algorithm. The main result in the present paper is the assertion that the situation can also be opposite for separate elements of the class A 1(D) (and even of the class A 0(D)): the rate of convergence of the orthogonal greedy algorithm can be significantly lower than the rate of convergence of the pure greedy algorithm.


pure greedy algorithm orthogonal greedy algorithm dictionary rate of convergence Hilbert space the classes A1(D) and A0(D


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J.H. Friedman and W. Stueuzle, “Projection pursuit regression,” J. Amer. Statist. Assoc. 76(376), 817–823 (1981).MathSciNetCrossRefGoogle Scholar
  2. 2.
    L. K. Jones, “On a conjecture of Huber concerning the convergence of projection pursuit regression,” Ann. Statist. 15(2), 880–882 (1987).MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    S. G. Mallat and Z. Zhang, “Matching pursuit with time-frequency dictionaries,” IEEE Trans. Signal Process. 41(12), 3397–3415 (1993).zbMATHCrossRefGoogle Scholar
  4. 4.
    R. A. DeVore and V. N. Temlyakov, “Some remarks on greedy algorithms,” Adv. Comput. Math. 5(2–3), 173–187 (1996).MathSciNetCrossRefGoogle Scholar
  5. 5.
    V. N. Temlyakov, “Weak greedy algorithms,” Adv. Comput. Math. 12(2–3), 213–227 (2000).MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    V. N. Temlyakov, “Greedy algorithms in Banach spaces,” Adv. Comput. Math. 14(3), 277–292 (2001).MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    V. V. Galatenko and E. D. Livshits, “Generalized approximate weak greedy algorithms,” Mat. Zametki 78(2), 186–201 (2005) [Math. Notes 78 (1–2), 170–184 (2005)].MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    L. Rejtö and G. G. Walter, “Remarks on projection pursuit regression and density estimation,” Stochastic Anal. Appl. 10(2), 213–222 (1992).MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    L. K. Jones, “A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and network training,” Ann. Statist. 20(1), 608–613 (1992).MathSciNetCrossRefGoogle Scholar
  10. 10.
    V. V. Dubinin, Greedy Algorithms and Applications, Ph. D. Thesis (Univ. South Carolina, 1997).Google Scholar
  11. 11.
    S. B. Stechkin, “On absolute convergence of orthogonal series,” Dokl. Akad. Nauk SSSR 102, 37–40 (1955).MathSciNetzbMATHGoogle Scholar
  12. 12.
    A. R. Barron, “Universal approximation bounds for superpositions of a sigmoidal function,” IEEE Trans. Inform. Theory 39(3), 930–945 (1993).MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    S. V. Konyagin and V. N. Temlyakov, “Rate of convergence of Pure Greedy Algorithm,” East J. Approx. 5(4), 493–499 (1999).MathSciNetzbMATHGoogle Scholar
  14. 14.
    A. V. Sil’nichenko, “On the rate of convergence of greedy algorithms,” Mat. Zametki 76(4), 628–632 (2004) [Math. Notes 76 (4), 582–586 (2004)].MathSciNetCrossRefGoogle Scholar
  15. 15.
    E. D. Livshitz and V. N. Temlyakov, “Two lower estimates in greedy approximation,” Constr. Approx. 19(4), 509–523 (2003).MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    E. D. Livshits, “Lower bounds for the rate of convergence of greedy algorithms,” Izv. Ross. Akad. Nauk Ser. Mat. 73(6), 125–144 (2009) [Izv. Math. 73 (6), 1197–1215 (2009)].MathSciNetGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2012

Authors and Affiliations

  1. 1.Moscow State UniversityMoscowRussia

Personalised recommendations