Putnam’s Diagonal Argument and the Impossibility of a Universal Learning Machine

Abstract

Putnam construed the aim of Carnap’s program of inductive logic as the specification of a “universal learning machine,” and presented a diagonal proof against the very possibility of such a thing. Yet the ideas of Solomonoff and Levin lead to a mathematical foundation of precisely those aspects of Carnap’s program that Putnam took issue with, and in particular, resurrect the notion of a universal mechanical rule for induction. In this paper, I take up the question whether the Solomonoff–Levin proposal is successful in this respect. I expose the general strategy to evade Putnam’s argument, leading to a broader discussion of the outer limits of mechanized induction. I argue that this strategy ultimately still succumbs to diagonalization, reinforcing Putnam’s impossibility claim.

Appendix

Theorem 2 is in the literature (Li and Vitányi 2008, 352ff; Hutter 2003, 2062; Poland and Hutter 2005, 3781) usually presented as a consequence of (variations of) the following stronger result, first shown by Solomonoff (1978, 426f). Let us introduce as a measure of the divergence between two distributions $P_1$ and $P_2$ over $\{0,1\}$ the squared Hellinger distance

$$\begin{aligned} H(P_1,P_2) := \sum _{x \in \{0,1\}}\left( \sqrt{P_1(x)}-\sqrt{P_2(x)}\right) ^2. \end{aligned}$$

(1)

Then, for every $\mu \in \Delta _1$, the expected infinite sum of divergences between $Q_U$ and $\mu$

$$\begin{aligned} {{\mathrm{{\mathbf {E}}}}}_{X^\omega \sim \mu }\left[ \sum _{n=0}^\infty H\left( \mu (\cdot \mid X^n),Q_U(\cdot \mid X^n) \right) \right] \end{aligned}$$

(2)

is bounded by a constant.

To see how $(\text {I: }\Delta _1)$ follows from this constant bound, suppose that $Q_U$ does not satisfy $(\text {I: }\Delta _1)$: there is a $\mu \in \Delta _1$ such that with probability $\epsilon >0$ there is a $\delta > 0$ such that $\left| \mu (x_{n+1} \mid \pmb {x}^n)-Q_U(x_{n+1} \mid \pmb {x}^n)\right| >\delta$ infinitely often. But that means that with positive probability the infinite sum of squared Hellinger distances is infinite, and the expectation (2) cannot be bounded by a constant.

The proof of the constant bound on (2) starts with the fact that the distance $H(P_1,P_2)$ is bounded by the Kullback-Leibler divergence

$$\begin{aligned} D(P_1 \parallel P_2) := {{\mathrm{{\mathbf {E}}}}}_{X \sim P_1}\left[ -\log \frac{P_2(X)}{P_1(X)} \right] . \end{aligned}$$

(3)

The term $-\log P(\pmb {x})$ expresses the logarithmic loss of P on sequence $\pmb {x}$, a standard measure of prediction error; the difference $-\log P_2(\pmb {x})-\left( -\log P_1(\pmb {x})\right) =-\log \frac{P_2(\pmb {x})}{P_1(\pmb {x})}$ expresses the surplus prediction error or regret of $P_2$ relative to $P_1$ on sequence $\pmb {x}$. Thus the Kullback-Leibler divergence (3) expresses the P₁-expected regret of $P_2$ relative to $P_1$.

Using $H(P_1,P_2) \le D(P_1 \parallel P_2)$ one can work out that (2) is bounded by

$$\begin{aligned} {{\mathrm{{\mathbf {E}}}}}_{X^\omega \sim \mu }\left[ \sum _{n=0}^\infty -\log \frac{Q_U(X_{n+1}\mid X^n)}{\mu (X_{n+1}\mid X^n)}\right] . \end{aligned}$$

(4)

Now by the universality of $Q_U$ in the class of $\Sigma _1$ measures we know that $Q_U$ majorizes $\mu$: for every finite $\pmb {x}$ there is a constant $c \in [0,1]$ such that $Q_U(\pmb {x}) \ge c \cdot \mu (\pmb {x})$. Indeed we can identify c with $w(\mu )$, where w is the prior over hypothesis class ${{\mathcal {H}}_{\Sigma _1}}$ in the classical Bayesian representation $\xi ^{\Sigma _1}_w$ of $Q_U$. This fact allows us to derive that for every sequence$\pmb {x}^m$of any lengthm

$$\begin{aligned} \sum _{n=0}^{m-1} -\log \frac{Q_U(x_{n+1}\mid \pmb {x}^n)}{\mu (x_{n+1}\mid \pmb {x}^n)}&= -\log \prod _{n=0}^{m-1} \frac{Q_U(x_{n+1}\mid \pmb {x}^n)}{\mu (x_{n+1}\mid \pmb {x}^n)} \nonumber \\&= -\log \frac{Q_U(\pmb {x}^m)}{\mu (\pmb {x}^m)} \nonumber \\&\le -\log w(\mu ). \end{aligned}$$

(5)

This concludes the proof that (2) is bounded by a constant: since the bound (5) holds for any individual sequence of any length, it also holds for (4) and thus for (2).

The absolute optimality property mentioned in Sect. 8 is just this individual sequence bound (5), which continues to hold for $\nu$ that are $\Sigma _1$. To reformulate, for any such $\nu$, the sum of surplus prediction errors (regrets) of $Q_U$ relative to $\nu$ will always (for any sequence $\pmb {x}^m$ of any length m) be bounded by a constant:

$$\begin{aligned} \sum _{n=0}^{m-1} \left( - \log Q_U(x_{n+1} \mid \pmb {x}^n) - \left( - \log \nu (x_{n+1} \mid \pmb {x}^n)\right) \right) \le -\log w(\nu ). \end{aligned}$$