Abstract
We investigate the problem of predicting a sequence when the information about the previous elements (feedback) is onlypartial and possibly dependent on the predicted values. This setting can be seen as a generalization of the classical multi-armed bandit problem and accommodates as a special case a natural bandwidth allocation problem. According to the approach adopted by many authors, we give up any statistical assumption on the sequence to be predicted. We evaluate the performance against the best constant predictor (regret), as it is common in iterated game analysis.
We show that for anydiscrete loss function and feedback function only one of two situations can occur: either there is a prediction strategy that achieves in T rounds a regret of at most O(T 3/4(ln T)1/2) or there is a sequence which cannot be predicted by any algorithm without incurring a regret of Ω(T)..
We prove both sides constructively, that is when the loss and feedback functions satisfya certain condition, we present an algorithm that generates predictions with the claimed performance; otherwise we show a sequence that no algorithm can predict without incurring a linear regret with probability at least 1/2.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. Gambling in a rigged casino: the adversarial multi-armed bandit problem. In Proceedings of the 36th Annual Symposium on Foundations of Computer Science, pages 322–331._IEEE Computer SocietyPress, Los Alamitos, CA, 1995.
Peter Auer. Using upper confidence bounds for online learning. In Proceedings of the 41th Annual Symposium on Foundations of Computer Science, pages 270–279. IEEE Computer Society Press, Los Alamitos, CA, 2000.
Nicolò Cesa-Bianchi, Yoav Freund, David P. Helmbold, David Haussler, Robert E. Schapire, and Manfred K. Warmuth. How to use expert advice. Journal of the ACM, 44(3):427–485, 1997.
Nicolò Cesa-Bianchi, Yoav Freund, David P. Helmbold, and Manfred Warmuth. On-line prediction and conversion strategies. In EUROCOLT: EUROCOLT, European Conference on Computational Learning Theory, EuroCOLT,. LNCS, 1993.
Nicolò Cesa-Bianchi and Gabor Lugosi. Minimax regret under log loss for general classes of experts. In Proceedings of the 12th Annual Conference on Computational Learning Theory. ACM Press, 1999.
Y. Freund and R. Schapire. Adaptive game playing using multiplicative weights. Games and Economic Behavior, 1997. to appear.
James Hannan. Approximation to bayes risk in repeated play. In M. Dresher, A.W. Tucker, and P. Wolfe, editors, Contributions to the Theory of Games, volume III, pages 97–139. Princeton University Press, 1957.
D. Haussler, J. Kivinen, and M.K. Warmuth. Tight worst-case loss bounds for predicting with expert advice. Lecture Notes in Computer Science, 904:69, 1995.
D. Haussler, J. Kivinen, and M.K. Warmuth. Sequential prediction of individual sequences under general loss functions. IEEE Transactions on Information Theory, 44, 1998.
Richard Karp, Elias Koutsopias, Christos Papadimitriou, and Scott Shenker. Optimization Problems in Congestion Control In Proceedings of the 41st Symposium on the Foundation of Computer Science, 2000.
M. Kearns and M. Li. Learning in the presence of malicious errors. SIAM Journal on Computing,, 22(4):807–837, August 1993.
Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm. Information and Computation, 108(2):212–261, 1 February 1994.
A. Piccolboni and C. Schindelhauer. Discrete prediction games with arbitraryfeedbac k and loss. Technical Report A-00-18, Schriftenreihe der Institute für Informatik und Mathematik, Universität Lübeck, October 2000.
Robert E. Schapire. Drifting games. In Proc. 12th Annu. Conf. on Comput. Learning Theory, pages 114–124. ACM Press, New York, NY, 1999.
V. Vovk. A game of prediction with expert advice. Journal of Computer and System Sciences, 56(2):153–173, April 1998.
V. Vovk. Competitive on-line statistics. In The 52nd Session of the International Statistical Institute, 1999.
T. Weissman and N. Merhav. Universal prediction of binaryindividual sequences in the presence of noise. accepted to IEEE Trans. Inform. Theory, September 2000.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Piccolboni, A., Schindelhauer, C. (2001). Discrete Prediction Games with Arbitrary Feedback and Loss (Extended Abstract). In: Helmbold, D., Williamson, B. (eds) Computational Learning Theory. COLT 2001. Lecture Notes in Computer Science(), vol 2111. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44581-1_14
Download citation
DOI: https://doi.org/10.1007/3-540-44581-1_14
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42343-0
Online ISBN: 978-3-540-44581-4
eBook Packages: Springer Book Archive