Discrete Prediction Games with Arbitrary Feedback and Loss (Extended Abstract)

  • Antonio Piccolboni
  • Christian Schindelhauer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2111)

Abstract

We investigate the problem of predicting a sequence when the information about the previous elements (feedback) is onlypartial and possibly dependent on the predicted values. This setting can be seen as a generalization of the classical multi-armed bandit problem and accommodates as a special case a natural bandwidth allocation problem. According to the approach adopted by many authors, we give up any statistical assumption on the sequence to be predicted. We evaluate the performance against the best constant predictor (regret), as it is common in iterated game analysis.

We show that for anydiscrete loss function and feedback function only one of two situations can occur: either there is a prediction strategy that achieves in T rounds a regret of at most O(T3/4(ln T)1/2) or there is a sequence which cannot be predicted by any algorithm without incurring a regret of Ω(T)..

We prove both sides constructively, that is when the loss and feedback functions satisfya certain condition, we present an algorithm that generates predictions with the claimed performance; otherwise we show a sequence that no algorithm can predict without incurring a linear regret with probability at least 1/2.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [ACBFS95]
    Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. Gambling in a rigged casino: the adversarial multi-armed bandit problem. In Proceedings of the 36th Annual Symposium on Foundations of Computer Science, pages 322–331._IEEE Computer SocietyPress, Los Alamitos, CA, 1995.Google Scholar
  2. [Aue00]
    Peter Auer. Using upper confidence bounds for online learning. In Proceedings of the 41th Annual Symposium on Foundations of Computer Science, pages 270–279. IEEE Computer Society Press, Los Alamitos, CA, 2000.Google Scholar
  3. [CBFH+97]
    Nicolò Cesa-Bianchi, Yoav Freund, David P. Helmbold, David Haussler, Robert E. Schapire, and Manfred K. Warmuth. How to use expert advice. Journal of the ACM, 44(3):427–485, 1997.MATHCrossRefMathSciNetGoogle Scholar
  4. [CBFHW93]
    Nicolò Cesa-Bianchi, Yoav Freund, David P. Helmbold, and Manfred Warmuth. On-line prediction and conversion strategies. In EUROCOLT: EUROCOLT, European Conference on Computational Learning Theory, EuroCOLT,. LNCS, 1993.Google Scholar
  5. [CBL99]
    Nicolò Cesa-Bianchi and Gabor Lugosi. Minimax regret under log loss for general classes of experts. In Proceedings of the 12th Annual Conference on Computational Learning Theory. ACM Press, 1999.Google Scholar
  6. [FS97]
    Y. Freund and R. Schapire. Adaptive game playing using multiplicative weights. Games and Economic Behavior, 1997. to appear.Google Scholar
  7. [Han57]
    James Hannan. Approximation to bayes risk in repeated play. In M. Dresher, A.W. Tucker, and P. Wolfe, editors, Contributions to the Theory of Games, volume III, pages 97–139. Princeton University Press, 1957.Google Scholar
  8. [HKW95]
    D. Haussler, J. Kivinen, and M.K. Warmuth. Tight worst-case loss bounds for predicting with expert advice. Lecture Notes in Computer Science, 904:69, 1995.Google Scholar
  9. [HKW98]
    D. Haussler, J. Kivinen, and M.K. Warmuth. Sequential prediction of individual sequences under general loss functions. IEEE Transactions on Information Theory, 44, 1998.Google Scholar
  10. [KKPS00]
    Richard Karp, Elias Koutsopias, Christos Papadimitriou, and Scott Shenker. Optimization Problems in Congestion Control In Proceedings of the 41st Symposium on the Foundation of Computer Science, 2000.Google Scholar
  11. [KL93]
    M. Kearns and M. Li. Learning in the presence of malicious errors. SIAM Journal on Computing,, 22(4):807–837, August 1993.MATHCrossRefMathSciNetGoogle Scholar
  12. [LW94]
    Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm. Information and Computation, 108(2):212–261, 1 February 1994.MATHCrossRefMathSciNetGoogle Scholar
  13. [PS00]
    A. Piccolboni and C. Schindelhauer. Discrete prediction games with arbitraryfeedbac k and loss. Technical Report A-00-18, Schriftenreihe der Institute für Informatik und Mathematik, Universität Lübeck, October 2000.Google Scholar
  14. [Sch99]
    Robert E. Schapire. Drifting games. In Proc. 12th Annu. Conf. on Comput. Learning Theory, pages 114–124. ACM Press, New York, NY, 1999.Google Scholar
  15. [Vov98]
    V. Vovk. A game of prediction with expert advice. Journal of Computer and System Sciences, 56(2):153–173, April 1998.MATHCrossRefMathSciNetGoogle Scholar
  16. [Vov99]
    V. Vovk. Competitive on-line statistics. In The 52nd Session of the International Statistical Institute, 1999.Google Scholar
  17. [WM00]
    T. Weissman and N. Merhav. Universal prediction of binaryindividual sequences in the presence of noise. accepted to IEEE Trans. Inform. Theory, September 2000.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Antonio Piccolboni
    • 1
  • Christian Schindelhauer
    • 1
  1. 1.Dept. of Math. and Comp. Science and Heinz-Nixdorf-InstitutPaderborn UniversityGermany

Personalised recommendations