Leading Strategies in Competitive On-Line Prediction
We start from a simple asymptotic result for the problem of on-line regression with the quadratic loss function: the class of continuous limited-memory prediction strategies admits a “leading prediction strategy”, which not only asymptotically performs at least as well as any continuous limited-memory strategy but also satisfies the property that the excess loss of any continuous limited-memory strategy is determined by how closely it imitates the leading strategy. More specifically, for any class of prediction strategies constituting a reproducing kernel Hilbert space we construct a leading strategy, in the sense that the loss of any prediction strategy whose norm is not too large is determined by how closely it imitates the leading strategy. This result is extended to the loss functions given by Bregman divergences and by strictly proper scoring rules.
KeywordsLoss Function Reproduce Kernel Hilbert Space Prediction Strategy Predictable Process Quadratic Loss Function
Unable to display preview. Download preview PDF.
- 11.Dawid, A.P.: Proper measures of discrepancy, uncertainty and dependence, with applications to predictive experimental design. Technical Report 139, Department of Statistical Science, University College London, November 1994. This technical report was revised (and its title was slightly changed) in August 1998Google Scholar
- 13.Ellul, J.: The Technological Bluff. Eerdmans, Grand Rapids, MI (1990), Translated by Bromiley, G.W.: The French original: Le bluff technologique, Hachette, Paris, 1988Google Scholar
- 16.Kabanov, Y.M., Liptser, R.S., Shiryaev, A.N.: To the question of absolute continuity and singularity of probability measures. Matematicheskii Sbornik 104, 227–247 (1977) (in Russian)Google Scholar
- 23.Ville, J.: Etude critique de la notion de collectif. In: Gauthier-Villars, Paris (1939)Google Scholar
- 25.Vovk, V.: Probability theory for the Brier game. Theoretical Computer Science, 1997 261, 57–79 (2001); Conference version in: Li, M. (ed.) ALT 1997. LNCS, vol. 1316, pp. 57–79. Springer, Heidelberg (1997)Google Scholar
- 26.Vovk, V.: Defensive prediction with expert advice. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS (LNAI), vol. 3734, pp. 444–458. Springer, Heidelberg (2005); Full version: Technical Report arXiv:cs.LG/0506041 “Competitive on-line learning with a convex loss function” (version 3), arXiv.org e-Print archive (September 2005)CrossRefGoogle Scholar
- 28.Vovk, V.: Competing with Markov prediction strategies. Technical report, arXiv.org e-Print archive (July 2006)Google Scholar
- 29.Vovk, V.: Competing with stationary prediction strategies. Technical Report arXiv:cs.LG/0607067, arXiv.org e-Print archive (July 2006)Google Scholar
- 31.Vovk, V.: Leading strategies in competitive on-line prediction. Technical Report arXiv:cs.LG/0607134, arXiv.org e-Print archive (July 2006)Google Scholar
- 32.Vovk, V.: On-line regression competitive with reproducing kernel Hilbert spaces. Technical Report arXiv:cs.LG/00511058 (version 2), arXiv.org e-Print archive (January 2006); Cai, J.-Y., Cooper, S.B., Li, A. (eds.) TAMC 2006. LNCS, vol. 3959, pp. 452–463. Springer, Heidelberg (extended abstract, 2006)Google Scholar
- 34.Vovk, V., Takemura, A., Shafer, G.: Defensive forecasting. In: Cowell, R.G., Ghahramani, Z. (eds.) Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics. Society for Artificial Intelligence and Statistics, pp. 365–372 (2005), Available electronically at: http://www.gatsby.ucl.ac.uk/aistats/