, Volume 84, Issue 3, pp 633–656 | Cite as

Putnam’s Diagonal Argument and the Impossibility of a Universal Learning Machine

  • Tom F. SterkenburgEmail author


Putnam construed the aim of Carnap’s program of inductive logic as the specification of a “universal learning machine,” and presented a diagonal proof against the very possibility of such a thing. Yet the ideas of Solomonoff and Levin lead to a mathematical foundation of precisely those aspects of Carnap’s program that Putnam took issue with, and in particular, resurrect the notion of a universal mechanical rule for induction. In this paper, I take up the question whether the Solomonoff–Levin proposal is successful in this respect. I expose the general strategy to evade Putnam’s argument, leading to a broader discussion of the outer limits of mechanized induction. I argue that this strategy ultimately still succumbs to diagonalization, reinforcing Putnam’s impossibility claim.


  1. Achinstein, P. (1963). Confirmation theory, order, and periodicity. Philosophy of Science, 30, 17–35.CrossRefGoogle Scholar
  2. Blackwell, D., & Dubins, L. (1962). Merging of opinion with increasing information. The Annals of Mathematical Statistics, 33, 882–886.CrossRefGoogle Scholar
  3. Carnap, R. (1950). Logical foundations of probability. Chicago, IL: The University of Chicago Press.Google Scholar
  4. Carnap, R. (1963a). Replies and systematic expositions. In Schilpp (1963), pp. 859–1013Google Scholar
  5. Carnap, R. (1963b). Variety, analogy, and periodicity in inductive logic. Philosophy of Science, 30(3), 222–227.CrossRefGoogle Scholar
  6. Dawid, A. P. (1985a). Calibration-based empirical probability. The Annals of Statistics, 13(4), 1251–1274.CrossRefGoogle Scholar
  7. Dawid, A. P. (1985b). The impossibility of inductive inference. Comment on Oakes (1985). Journal of the American Statistical Association, 80(390), 339.CrossRefGoogle Scholar
  8. Diaconis, P. W., & Freedman, D. A. (1986). On the consistency of Bayes estimates. The Annals of Statistics, 14(1), 1–26.CrossRefGoogle Scholar
  9. Downey, R. G., & Hirschfeldt, D. R. (2010). Algorithmic randomness and complexity. New York: Springer.Google Scholar
  10. Earman, J. (1992). Bayes or bust? A critical examination of Bayesian confirmation theory. Cambridge, MA: MIT Press.Google Scholar
  11. Gillies, D. A. (2001a). Popper and computer induction. BioEssays, 23, 859–860.CrossRefGoogle Scholar
  12. Gillies, D. A. (2001b). Bayesianism and the fixity of the theoretical framework. In D. Corfield & J. Williamson (Eds.), Foundations of Bayesianism (pp. 363–379). Berlin: Springer.CrossRefGoogle Scholar
  13. Goodman, N. (1946). A query on confirmation. The Journal of Philosophy, 43(14), 383–385.CrossRefGoogle Scholar
  14. Goodman, N. (1947). On infirmities of confirmation-theory. Philosophy and Phenomenological Research, 8(1), 149–151.CrossRefGoogle Scholar
  15. Hintikka, J. (1965). Towards a theory of inductive generalization. In Y. Bar-Hillel (Eds.), Logic, Methodology and philosophy of science. Proceedings of the 1964 international congress (pp. 274–288). North-Holland, Amsterdam.Google Scholar
  16. Howson, C. (2000). Hume’s problem: Induction and the justification of belief. New York: Oxford University Press.CrossRefGoogle Scholar
  17. Huttegger, S. M. (2015). Merging of opinions and probability kinematics. The Review of Symbolic Logic, 8(4), 611–648.CrossRefGoogle Scholar
  18. Hutter, M. (2003). Convergence and loss bounds for Bayesian sequence prediction. IEEE Transactions on Information Theory, 49(8), 2061–2067.CrossRefGoogle Scholar
  19. Hutter, M. (2007). On universal prediction and Bayesian confirmation. Theoretical Computer Science, 384(1), 33–48.CrossRefGoogle Scholar
  20. Kelly, K. T. (2004). Learning theory and epistemology. In I. Niiniluoto, M. Sintonen, J. Woleński (Eds.), Handbook of epistemology (pp. 183–203). Kluwer, Dordrecht, Page numbers refer to reprint in H. Arló-Costa, V. F. Hendricks, J. F. A. K. van Benthem (Eds.), (2016). Readings in formal epistemology.Google Scholar
  21. Kelly, K. T., Juhl, C. F., & Glymour, C. (1994). Reliability, realism, and relativism. In P. Clark & B. Hale (Eds.), Reading Putnam (pp. 98–160). Oxford: Blackwell.Google Scholar
  22. Leike, J., & Hutter, M. (2015). On the computability of Solomonoff induction and knowledge-seeking. In K. Chaudhuri, C. Gentile, S. Zilles (Eds.), Algorithmic learning theory: proceedings of the twenty-sixth international conference (ALT 2015) (pp. 364–378). Springer.Google Scholar
  23. Levin, L. A. (2010). Some theorems on the algorithmic approach to probability theory and information theory. Annals of Pure and Applied Logic, 162, 224–235. Translation of PhD dissertation, 1971. Russia: Moscow State University.Google Scholar
  24. Li, M., & Vitányi, P. M. B. (2008). An introduction to Kolmogorov complexity and its applications (3rd ed.). New York: Springer.CrossRefGoogle Scholar
  25. Nies, A. (2009). Computability and randomness. Oxford: Oxford University Press.CrossRefGoogle Scholar
  26. Oakes, D. (1985). Self-calibrating priors do not exist. Journal of the American Statistical Association, 80(390), 340–341.CrossRefGoogle Scholar
  27. Poland, J., & Hutter, M. (2005). Asymptotics of discrete MDL for online prediction. IEEE Transactions on Information Theory, 51(11), 3780–3795.CrossRefGoogle Scholar
  28. Putnam, H. (1963a) Degree of confirmation’ and inductive logic. In Schilpp (1963), pp. 761–783. Reprinted in Putnam (1975), pp. 270–292.Google Scholar
  29. Putnam, H. (1963b). Probability and confirmation. In The voice of America forum lectures. U.S. Information Agency, Washington, D.C., Page numbers refer to reprint in Putnam (1975), pp. 293–304.Google Scholar
  30. Putnam, H. (1974). The ‘corroboration’ of theories. In P. A. Schilpp (Ed.), The philosophy of Karl Popper, Book I. The Library of Living Philosophers (Vol. 14, pp. 221–240). Open Court, LaSalle, IL, Reprinted in Putnam (1975), pp. 250–269.Google Scholar
  31. Putnam, H. (1975). Mathematics, matter, and method. Cambridge: Cambridge University Press.Google Scholar
  32. Rathmanner, S., & Hutter, M. (2011). A philosophical treatise of universal induction. Entropy, 13(6), 1076–1136.CrossRefGoogle Scholar
  33. Reichenbach, H. (1933). Die logischen Grundlagen des Wahrscheinlichkeitsbegriffs. Erkenntnis, 3, 401–425.CrossRefGoogle Scholar
  34. Reichenbach, H. (1935). Wahrscheinlichkeitslehre: eine Untersuchung Über die Logischen und Mathematischen Grundlagen der Wahrscheinlichkeitsrechnung. Leiden: Sijthoff.Google Scholar
  35. Reichenbach, H. (1938). Experience and prediction. Chicago, IL: University of Chicago Press.Google Scholar
  36. Reimann, J. (2009). Randomness—Beyond Lebesgue measure. In S. B. Cooper, H. Geuvers, A. Pillay, & J. Väänänen (Eds.), Logic colloquium 2006 (pp. 247–279). Chicago, IL: Association for Symbolic Logic.CrossRefGoogle Scholar
  37. Romeijn, J.-W. (2004). Hypotheses and inductive predictions. Synthese, 141(3), 333–364.CrossRefGoogle Scholar
  38. Salmon, W. C. (1967). The foundations of scientific inference. Pittsburgh, PA: University of Pittsburgh Press.CrossRefGoogle Scholar
  39. Salmon, W. C. (1991). Hans Reichenbach’s vindication of induction. Erkenntnis, 35, 99–122.Google Scholar
  40. Schervish, M. J. (1985). Comment on Dawid (1985a). The Annals of Statistics, 13(4), 1274–1282.CrossRefGoogle Scholar
  41. Schilpp, P. A. (Ed.). (1963). The philosophy of Rudolf Carnap. The library of living philosophers (Vol. 11). LaSalle, IL: Open Court.Google Scholar
  42. Shen, A. K., Uspensky, V. A., & Vereshchagin, N. K. (2017). Kolmogorov complexity and algorithmic randomness. Providence, RI: American Mathematical Society.Google Scholar
  43. Skyrms, B. (1991). Carnapian inductive logic for Markov chains. Erkenntnis, 35, 439–460.Google Scholar
  44. Skyrms, B. (1996). Carnapian inductive logic and Bayesian statistics. In T. Ferguson, L. Shapley, & J. MacQueen (Eds.), Statistics, probability and game theory: Papers in honor of David Blackwell (pp. 321–336). Beachwood: Institute of Mathematical Statistics.CrossRefGoogle Scholar
  45. Soare, R. I. (2016). Turing computability: Theory and applications. New York: Springer.Google Scholar
  46. Solomonoff, R. J. (1964). A formal theory of inductive inference. Parts I and II. Information and Control, 7(1–22), 224–254.CrossRefGoogle Scholar
  47. Solomonoff, R. J. (1978). Complexity-based induction systems: Comparisons and convergence theorems. IEEE Transactions on Information Theory, 24(4), 422–432.CrossRefGoogle Scholar
  48. Sterkenburg, T. F. (2016). Solomonoff prediction and Occam’s razor. Philosophy of Science, 83(4), 459–479.CrossRefGoogle Scholar
  49. Tao, T. (2011). An introduction to measure theory. Providence, RI: American Mathematical Society.Google Scholar
  50. Turing, A. M. (1936). On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, 2(42), 230–265.Google Scholar
  51. van Fraassen, B. C. (1989). Laws and symmetry. Oxford: Clarendon Press.CrossRefGoogle Scholar
  52. van Fraassen, B. C. (2000). The false hopes of traditional epistemology. Philosophy and Phenomenological Research, 60(2), 253–280.CrossRefGoogle Scholar
  53. Zvonkin, A. K., & Levin, L. A. (1970). The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms. Russian Mathematical Surveys, 26(6), 83–124. Translation of the Russian original. Uspekhi Matematicheskikh Nauk, 25(6), 85–127, 1970.Google Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Munich Center for Mathematical PhilosophyLMU MunichMunichGermany

Personalised recommendations