Worst Case Prediction over Sequences under Log Loss
We consider the game of sequentially assigning probabilities to future data based on past observations under logarithmic loss. We are not making probabilistic assumptions about the generation of the data, but consider a situation where a player tries to minimize his loss relative to the loss of the (with hindsight) best distribution from a target class for the worst sequence of data. We give bounds on the minimax regret in terms of the metric entropies of the target class with respect to suitable distances between distributions.
KeywordsMinimax Regret Computational Learning Theory Probabilistic Assumption Target Family Good Expert
Unable to display preview. Download preview PDF.
- V.G. Vovk, Aggregating strategies, Proceedings of the 1990 conference on computational learning theory, Morgan Kaufmann, 2 371–381 (1990).Google Scholar
- N. Cesa-Bianchi, Y. Freund, D.H. Helmbold, D. Haussler, R.E. Schapire, and M.K. Warmuth, How to use expert advice, in 25th Annual ACM Symposium on Theory of Computing, 382–392, San Diego, CA (1993).Google Scholar
- Y. Freund, Predicting a binary sequence as well as the optimal biased coin, Proceedings of the ninth annual conference on computational learning theory, ACM Press (1996).Google Scholar
- J. Shtarkov, Coding of discrete sources with unknown statistics, In: Topics in Information Theory, 559–574, I. Csiszar and P. Elias, editors, North Holland, Amsterdam, 1975.Google Scholar
- A.R. Barron and Q. XIE, Asymptotic minimax loss for data compression, gambling, and prediction, Proceedings of the ninth annual conference on computational learning theory, ACM Press (1996).Google Scholar
- AAD W. Van Der Vaart and JON A. Wellner, Weak Convergence and Empirical Processes, Springer Series in Statistics, 1996.Google Scholar
- D. Haussler and M. Opper, Mutual Information, Metric Entropy, and Risk in Estimation of Probability Distributions, Annals of Statistics 25 (6) (December, 1997).Google Scholar