Discrete Prediction Games with Arbitrary Feedback and Loss (Extended Abstract)
We investigate the problem of predicting a sequence when the information about the previous elements (feedback) is onlypartial and possibly dependent on the predicted values. This setting can be seen as a generalization of the classical multi-armed bandit problem and accommodates as a special case a natural bandwidth allocation problem. According to the approach adopted by many authors, we give up any statistical assumption on the sequence to be predicted. We evaluate the performance against the best constant predictor (regret), as it is common in iterated game analysis.
We show that for anydiscrete loss function and feedback function only one of two situations can occur: either there is a prediction strategy that achieves in T rounds a regret of at most O(T 3/4(ln T)1/2) or there is a sequence which cannot be predicted by any algorithm without incurring a regret of Ω(T)..
We prove both sides constructively, that is when the loss and feedback functions satisfya certain condition, we present an algorithm that generates predictions with the claimed performance; otherwise we show a sequence that no algorithm can predict without incurring a linear regret with probability at least 1/2.
Unable to display preview. Download preview PDF.
- [ACBFS95]Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. Gambling in a rigged casino: the adversarial multi-armed bandit problem. In Proceedings of the 36th Annual Symposium on Foundations of Computer Science, pages 322–331._IEEE Computer SocietyPress, Los Alamitos, CA, 1995.Google Scholar
- [Aue00]Peter Auer. Using upper confidence bounds for online learning. In Proceedings of the 41th Annual Symposium on Foundations of Computer Science, pages 270–279. IEEE Computer Society Press, Los Alamitos, CA, 2000.Google Scholar
- [CBFHW93]Nicolò Cesa-Bianchi, Yoav Freund, David P. Helmbold, and Manfred Warmuth. On-line prediction and conversion strategies. In EUROCOLT: EUROCOLT, European Conference on Computational Learning Theory, EuroCOLT,. LNCS, 1993.Google Scholar
- [CBL99]Nicolò Cesa-Bianchi and Gabor Lugosi. Minimax regret under log loss for general classes of experts. In Proceedings of the 12th Annual Conference on Computational Learning Theory. ACM Press, 1999.Google Scholar
- [FS97]Y. Freund and R. Schapire. Adaptive game playing using multiplicative weights. Games and Economic Behavior, 1997. to appear.Google Scholar
- [Han57]James Hannan. Approximation to bayes risk in repeated play. In M. Dresher, A.W. Tucker, and P. Wolfe, editors, Contributions to the Theory of Games, volume III, pages 97–139. Princeton University Press, 1957.Google Scholar
- [HKW95]D. Haussler, J. Kivinen, and M.K. Warmuth. Tight worst-case loss bounds for predicting with expert advice. Lecture Notes in Computer Science, 904:69, 1995.Google Scholar
- [HKW98]D. Haussler, J. Kivinen, and M.K. Warmuth. Sequential prediction of individual sequences under general loss functions. IEEE Transactions on Information Theory, 44, 1998.Google Scholar
- [KKPS00]Richard Karp, Elias Koutsopias, Christos Papadimitriou, and Scott Shenker. Optimization Problems in Congestion Control In Proceedings of the 41st Symposium on the Foundation of Computer Science, 2000.Google Scholar
- [PS00]A. Piccolboni and C. Schindelhauer. Discrete prediction games with arbitraryfeedbac k and loss. Technical Report A-00-18, Schriftenreihe der Institute für Informatik und Mathematik, Universität Lübeck, October 2000.Google Scholar
- [Sch99]Robert E. Schapire. Drifting games. In Proc. 12th Annu. Conf. on Comput. Learning Theory, pages 114–124. ACM Press, New York, NY, 1999.Google Scholar
- [Vov99]V. Vovk. Competitive on-line statistics. In The 52nd Session of the International Statistical Institute, 1999.Google Scholar
- [WM00]T. Weissman and N. Merhav. Universal prediction of binaryindividual sequences in the presence of noise. accepted to IEEE Trans. Inform. Theory, September 2000.Google Scholar