The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions
Solomonoff’s optimal but noncomputable method for inductive inference assumes that observation sequences x are drawn from an recursive prior distribution μ(x). Instead of using the unknown μ(x) he predicts using the celebrated universal enumerable prior M(x) which for all x exceeds any recursive μ(x), save for a constant factor independent of x. The simplicity measure M(x) naturally implements “Occam’s razor” and is closely related to the Kolmogorov complexity of x. However, M assigns high probability to certain data x that are extremely hard to compute. This does not match our intuitive notion of simplicity. Here we suggest a more plausible measure derived from the fastest way of computing data. In absence of contrarian evidence, we assume that the physical world is generated by a computational process, and that any possibly infinite sequence of observations is therefore computable in the limit (this assumption is more radical and stronger than Solomonoff’s). Then we replace M by the novel Speed Prior S, under which the cumulative a priori probability of all data whose computation through an optimal algorithm requires more than O(n) resources is 1/n. We show that the Speed Prior allows for deriving a computable strategy for optimal prediction of future y, given past x. Then we consider the case that the data actually stem from a nonoptimal, unknown computational process, and use Hutter’s recent results to derive excellent expected loss bounds for S-based inductive inference. We conclude with several nontraditional predictions concerning the future of our universe.
Unable to display preview. Download preview PDF.
- 3.B. Carter. Large number coincidences and the anthropic principle in cosmology. In M. S. Longair, editor, Proceedings of the IAU Symposium 63, pages 291–298. Reidel, Dordrecht, 1974.Google Scholar
- 7.D. F. Galouye. Simulacron 3. Bantam, 1964.Google Scholar
- 8.M. Hutter. Convergence and error bounds of universal prediction for general alphabet. Proceedings of the 12th European Conference on Machine Learning (ECML-2001), (TR IDSIA-07-01, cs.AI/0103015), 2001.Google Scholar
- 9.M. Hutter. General loss bounds for universal sequence prediction. In C. E. Brodley and A. P. Danyluk, editors, Proceedings of the 18th International Conference on Machine Learning (ICML-2001), pages 210–217. Morgan Kaufmann, 2001. TR IDSIA-03-01, IDSIA, Switzerland, Jan 2001, cs.AI/0101019.Google Scholar
- 10.M. Hutter. Towards a universal theory of artificial intelligence based on algorithmic probability and sequential decisions. Proceedings of the 12 th European Conference on Machine Learning (ECML-2001), (TR IDSIA-14-00, cs.AI/0012011), 2001.Google Scholar
- 11.M. Hutter. The fastest and shortest algorithm for all well-defined problems. International Journal of Foundations of Computer Science, (TR IDSIA-16-00, cs.CC/0102018), 2002. In press.Google Scholar
- 12.A. N. Kolmogorov. Three approaches to the quantitative definition of information. Problems of Information Transmission, 1:1–11, 1965.Google Scholar
- 13.L. G. Kraft. A device for quantizing, grouping, and coding amplitude modulated pulses. M.Sc. Thesis, Dept. of Electrical Engineering, MIT, Cambridge, Mass., 1949.Google Scholar
- 14.L. A. Levin. Universal sequential search problems. Problems of Information Transmission, 9(3):265–266, 1973.Google Scholar
- 15.M. Li and P. M. B. Vitányi. An Introduction to Kolmogorov Complexity and its Applications (2nd edition). Springer, 1997.Google Scholar
- 18.C. Schmidhuber. Strings from logic. Technical Report CERN-TH/2000-316, CERN, Theory Division, 2000. http://xxx.lanl.gov/abs/hep-th/0011065.
- 19.J. Schmidhuber. Discovering solutions with low Kolmogorov complexity and high generalization capability. In A. Prieditis and S. Russell, editors, Machine Learning: Proceedings of the Twelfth International Conference, pages 488–496. Morgan Kaufmann Publishers, San Francisco, CA, 1995.Google Scholar
- 20.J. Schmidhuber. A computer scientist’s view of life, the universe, and everything. In C. Freksa, M. Jantzen, and R. Valk, editors, Foundations of Computer Science: Potential-Theory-Cognition, volume 1337, pages 201–208. Lecture Notes in Computer Science, Springer, Berlin, 1997.Google Scholar
- 22.J. Schmidhuber. Algorithmic theories of everything. Technical Report IDSIA-20-00, quant-ph/0011122, IDSIA, Manno (Lugano), Switzerland, 2000.Google Scholar
- 23.J. Schmidhuber. Hierarchies of generalized Kolmogorov complexities and nonenumerable universal measures computable in the limit. International Journal of Foundations of Computer Science, 2002. In press.Google Scholar
- 26.G. ’t Hooft. Quantum gravity as a dissipative deterministic system. Technical Report SPIN-1999/07/gr-gc/9903084, http://xxx.lanl.gov/abs/gr-qc/9903084, Institute for Theoretical Physics, Univ. of Utrecht, and Spinoza Institute, Netherlands, 1999. Also published in Classical and Quantum Gravity 16, 3263.Google Scholar