Advertisement

A Closer Look at Adaptive Regret

  • Dmitry Adamskiy
  • Wouter M. Koolen
  • Alexey Chernov
  • Vladimir Vovk
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7568)

Abstract

For the prediction with expert advice setting, we consider methods to construct algorithms that have low adaptive regret. The adaptive regret of an algorithm on a time interval [t 1,t 2] is the loss of the algorithm there minus the loss of the best expert. Adaptive regret measures how well the algorithm approximates the best expert locally, and it is therefore somewhere between the classical regret (measured on all outcomes) and the tracking regret, where the algorithm is compared to a good sequence of experts.

We investigate two existing intuitive methods to derive algorithms with low adaptive regret, one based on specialist experts and the other based on restarts. Quite surprisingly, we show that both methods lead to the same algorithm, namely Fixed Share, which is known for its tracking regret. Our main result is a thorough analysis of the adaptive regret of Fixed Share. We obtain the exact worst-case adaptive regret for Fixed Share, from which the classical tracking bounds can be derived. We also prove that Fixed Share is optimal, in the sense that no algorithm can have a better adaptive regret bound.

Keywords

Online learning adaptive regret Fixed Share specialist experts 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bousquet, O., Warmuth, M.K.: Tracking a small set of experts by mixing past posteriors. Journal of Machine Learning Research 3, 363–396 (2002)MathSciNetGoogle Scholar
  2. 2.
    Cesa-Bianchi, N., Gaillard, P., Lugosi, G., Stoltz, G.: A new look at shifting regret. CoRR abs/1202.3323 (2012)Google Scholar
  3. 3.
    Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press (2006)Google Scholar
  4. 4.
    Chernov, A., Vovk, V.: Prediction with Expert Evaluators’ Advice. In: Gavaldà, R., Lugosi, G., Zeugmann, T., Zilles, S. (eds.) ALT 2009. LNCS, vol. 5809, pp. 8–22. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Freund, Y., Schapire, R.E., Singer, Y., Warmuth, M.K.: Using and combining predictors that specialize. In: Proc. 29th Annual ACM Symposium on Theory of Computing, pp. 334–343. ACM (1997)Google Scholar
  6. 6.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Hazan, E., Seshadhri, C.: Efficient learning algorithms for changing environments. In: ICML, p. 50 (2009)Google Scholar
  8. 8.
    Herbster, M., Warmuth, M.K.: Tracking the best expert. Machine Learning 32(2), 151–178 (1998)zbMATHCrossRefGoogle Scholar
  9. 9.
    Koolen, W.M.: Combining Strategies Efficiently: High-quality Decisions from Conflicting Advice. Ph.D. thesis, Institute of Logic, Language and Computation (ILLC), University of Amsterdam (January 2011)Google Scholar
  10. 10.
    Koolen, W.M., de Rooij, S.: Combining expert advice efficiently. In: Servedio, R., Zang, T. (eds.) Proceedings of the 21st Annual Conference on Learning Theory (COLT 2008), pp. 275–286 (June 2008)Google Scholar
  11. 11.
    Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Inf. Comput. 108(2), 212–261 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Shamir, G.I., Merhav, N.: Low complexity sequential lossless coding for piecewise stationary memoryless sources. IEEE Trans. Info. Theory 45, 1498–1519 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Vovk, V.: Aggregating strategies. In: Proceedings of the Third Annual Workshop on Computational Learning Theory, pp. 371–383. Morgan Kaufmann (1990)Google Scholar
  14. 14.
    Vovk, V.: Competitive on-line statistics. International Statistical Review 69, 213–248 (2001)zbMATHCrossRefGoogle Scholar
  15. 15.
    Vovk, V.: A game of prediction with expert advice. Journal of Computer and System Sciences 56, 153–173 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Zinkevich, M.: Online convex programming and generalized infinitesimal gradient ascent. In: Proc. 20th Int. Conference on Machine Learning (ICML 2003), pp. 928–936 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dmitry Adamskiy
    • 1
  • Wouter M. Koolen
    • 1
  • Alexey Chernov
    • 2
  • Vladimir Vovk
    • 1
  1. 1.Computer Learning Research Centre and Department of Computer ScienceRoyal Holloway, University of LondonSurreyUK
  2. 2.Department Mathematical SciencesDurham UniversityDurhamUK

Personalised recommendations