Is There an Elegant Universal Theory of Prediction?
Solomonoff’s inductive learning model is a powerful, universal and highly elegant theory of sequence prediction. Its critical flaw is that it is incomputable and thus cannot be used in practice. It is sometimes suggested that it may still be useful to help guide the development of very general and powerful theories of prediction which are computable. In this paper it is shown that although powerful algorithms exist, they are necessarily highly complex. This alone makes their theoretical analysis problematic, however it is further shown that beyond a moderate level of complexity the analysis runs into the deeper problem of Gödel incompleteness. This limits the power of mathematics to analyse and study prediction algorithms, and indeed intelligent systems in general.
KeywordsPrediction Algorithm Kolmogorov Complexity Input String Computable Sequence Short Program
Unable to display preview. Download preview PDF.
- 6.Gödel, K.: Über formal unentscheidbare Sätze der principia mathematica und verwandter systeme I. Monatshefte für Matematik und Physik 38, 173–198 (1931); English translation by Mendelsohn, E.: On undecidable propositions of formal mathematical systems. In: Davis, M. (ed.) The undecidable, New York, pp. 39–71. Raven Press, Hewlitt (1965) Google Scholar
- 15.Sutton, R., Barto, A.: Reinforcement learning: An introduction. MIT Press, Cambridge, MA (1998)Google Scholar
- 18.Willems, F.M.J., Shtarkov, Y.M., Tjalkens, T.J.: The context-tree weighting method: Basic properties. IEEE Transactions on Information Theory 41(3) (1995)Google Scholar