Minimizing Regret with Label Efficient Prediction
We investigate label efficient prediction, a variant of the problem of prediction with expert advice, proposed by Helmbold and Panizza, in which the forecaster does not have access to the outcomes of the sequence to be predicted unless he asks for it, which he can do for a limited number of times. We determine matching upper and lower bounds for the best possible excess error when the number of allowed queries is a constant. We also prove that a query rate of order (ln n) (ln ln n)2/n is sufficient for achieving Hannan consistency, a fundamental property in game-theoretic prediction models. Finally, we apply the label efficient framework to pattern classification and prove a label efficient mistake bound for a randomized variant of Littlestone’s zero-threshold Winnow algorithm.
KeywordsLoss Function Expert Advice Repeated Game Query Rate Cumulative Loss
Unable to display preview. Download preview PDF.
- 2.Auer, P., Cesa-Bianchi, N., Gentile, C.: Adaptive and self-confident on-line learning algorithms. Journal of Computer and System Sciences 64(1) (2002)Google Scholar
- 3.Birgé, L.: A new look at an old result: Fano’s lemma. Technical report, Université Paris 6 (2001) Google Scholar
- 7.Hannan, J.: Approximation to Bayes risk in repeated play. Contributions to the theory of games 3, 97–139 (1957)Google Scholar
- 10.Littlestone, N.: Mistake Bounds and Logarithmic Linear-threshold Learning Algorithms. PhD thesis, University of California at Santa Cruz (1989) Google Scholar
- 11.Massart, P.: Concentration inequalities and model selection. Saint-Flour summer school lecture notes (2003) (to appear) Google Scholar
- 12.Piccolboni, A., Schindelhauer, C.: Discrete prediction games with arbitrary feedback and loss. In: Proceedings of the 14th Annual Conference on Computational Learning Theory, pp. 208–223 (2001)Google Scholar