von Neumann’s Theorem Revisited

Abstract

According to a popular narrative, in 1932 von Neumann introduced a theorem that intended to be a proof of the impossibility of hidden variables in quantum mechanics. However, the narrative goes, Bell later spotted a flaw that allegedly shows its irrelevance. Bell’s widely accepted criticism has been challenged by Bub and Dieks: they claim that the proof shows that viable hidden variables theories cannot be theories in Hilbert space. Bub’s and Dieks’ reassessment has been in turn challenged by Mermin and Schack. Hereby I critically assess their reply, with the aim of bringing further clarification concerning the meaning, scope and relevance of von Neumann’s theorem. I show that despite Mermin and Schack’s response, Bub’s and Dieks’ reassessment is quite correct, and that this reading gets strongly reinforced when we carefully consider the connection between von Neumann’s proof and Gleason’s theorem.

Appendices

Appendix 1: von Neumann’s Derivation of the Trace Rule

Let $|\phi_1\rangle,|\phi_2\rangle, \ldots$ be a complete orthonormal set of vectors, and $R$ an arbitrary Hermitian operator with matrix elements in that basis given by $r_{\mu \nu } = \langle{\phi_{\nu } {|}R{|}\phi_{\mu }}\rangle$. Von Neumann defined three Hermitian operators in terms of their matrix elements. For the operator $A^{\left( n \right)}$, its elements $a_{\mu \nu }^{\left( n \right)}$ are 1 for $\mu = \nu = n$, and 0 otherwise. For the operator $B^{{\left( {mn} \right)}}$ its elements $b_{\mu \nu }^{{\left( {mn} \right)}}$ (with $m < n$) are 1 for $\mu = m$, $\nu = n$, and for $\mu = n$, $\nu = m$, and 0 otherwise. For the operator $C^{{\left( {mn} \right)}}$ its elements $c_{\mu \nu }^{{\left( {mn} \right)}}$ (with $m < n$) are $i$ for $\mu = m$, $\nu = n$, $- i$ for $\mu = n$, $\nu = m$, and 0 otherwise. The elements $r_{\mu \nu } = r_{\nu \mu }^{*}$ of $R$ are, therefore,

$$r_{\mu \nu } = \mathop \sum \limits_{n} r_{nn} a_{\mu \nu }^{\left( n \right)} + \mathop \sum \limits_{{\begin{array}{*{20}c} {m,n} \\ {m < n} \\ \end{array} }} {\text{Re}}\left( {r_{mn} } \right)b_{\mu \nu }^{{\left( {mn} \right)}} + \mathop \sum \limits_{{\begin{array}{*{20}c} {m,n} \\ {m < n} \\ \end{array} }} {\text{Im}}\left( {r_{mn} } \right)c_{\mu \nu }^{{\left( {mn} \right)}} ,$$

where ${\text{Re}}\left( {r_{mn} } \right)$ and ${\text{Im}}\left( {r_{mn} } \right)$ are the real and imaginary parts of the $r_{mn}$. The first term in the sum determines the diagonal elements of $R$, whereas the second and third term yield the real and imaginary parts of $R$’s off-diagonal elements, respectively. Thus, $R$ can be written as a linear combination of the operators $A^{\left( n \right)}$, $B^{{\left( {mn} \right)}}$ and $C^{{\left( {mn} \right)}}$:

$$R = \mathop \sum \limits_{n} r_{nn} A^{\left( n \right)} + \mathop \sum \limits_{{\begin{array}{*{20}c} {m,n} \\ {m < n} \\ \end{array} }} {\text{Re}}\left( {r_{mn} } \right)B^{{\left( {mn} \right)}} + \mathop \sum \limits_{{\begin{array}{*{20}c} {m,n} \\ {m < n} \\ \end{array} }} {\text{Im}}\left( {r_{mn} } \right)C^{{\left( {mn} \right)}}$$

Then, by II,

$${\mathcal{R}} = \mathop \sum \limits_{n} r_{nn} {\mathcal{A}}^{\left( n \right)} + \mathop \sum \limits_{{\begin{array}{*{20}c} {m,n} \\ {m < n} \\ \end{array} }} {\text{Re}}\left( {r_{mn} } \right){\mathcal{B}}^{{\left( {mn} \right)}} + \mathop \sum \limits_{{\begin{array}{*{20}c} {m,n} \\ {m < n} \\ \end{array} }} {\text{Im}}\left( {r_{mn} } \right){\mathcal{C}}^{{\left( {mn} \right)}} ,$$

and by B′,

$$Exp\left( {\mathcal{R}} \right) = \mathop \sum \limits_{n} r_{{nn}} Exp\left( {{\mathcal{A}}^{{\left( n \right)}} } \right) + \mathop \sum \limits_{{\begin{array}{*{20}c} {m,n} \\ {m < n} \\ \end{array} }} \text{Re} \left( {r_{{mn}} } \right)Exp\left( {{\mathcal{B}}^{{\left( {mn} \right)}} } \right) + \mathop \sum \limits_{{\begin{array}{*{20}c} {m,n} \\ {m < n} \\ \end{array} }} \text{Im} \left( {r_{{mn}} } \right)Exp\left( {{\mathcal{C}}^{{\left( {mn} \right)}} } \right)$$

Von Neumann’s next step was to define the matrix elements $u_{nn} = Exp\left( {{\mathcal{A}}^{\left( n \right)} } \right)$, $u_{mn} = \frac{1}{2}Exp\left( {{\mathcal{B}}^{{\left( {mn} \right)}} } \right) + i\frac{1}{2}Exp\left( {{\mathcal{C}}^{{\left( {mn} \right)}} } \right)$ (with $m < n$), and $u_{nm} = \frac{1}{2}Exp\left( {{\mathcal{B}}^{{\left( {mn} \right)}} } \right) - i\frac{1}{2}Exp\left( {{\mathcal{C}}^{{\left( {mn} \right)}} } \right)$ (with $m < n$). By plugging this in the right hand side of our equation for $Exp\left( {\mathcal{R}} \right)$, he obtained (for the details of this step, see [14, pp. 249–250]):

$$Exp\left( {\mathcal{R}} \right) = \mathop \sum \limits_{m,n} u_{nm} r_{mn}$$

Now, since $u_{mn} = u_{nm}^{*}$, we can define the Hermitian operator $U$ with matrix elements $\langle{\phi_{n} {|}U{|}\phi_{m}}\rangle = u_{mn}$. That is, the $u_{nn}$ determine the diagonal elements of $U$, whereas its off-diagonal elements are given by the $u_{mn}$ and the $u_{nm}$. Thus, we can now express $Exp\left( {\mathcal{R}} \right) = \mathop \sum \limits_{m,n} u_{nm} r_{\mu \nu }$ as

$$Exp\left( {\mathcal{R}} \right) = {\text{Tr}}\left( {UR} \right)$$

Von Neumann then shows that A′ implies that $U$ must be positive semi-definite. For the quantity ${\mathcal{Q}} = {\mathcal{P}}^{2}$, A′ entails that $Exp\left( {\mathcal{Q}} \right) \ge 0$. If the operator of quantity ${\mathcal{P}}$ is a projector $P_{\phi }$ onto a normalized but otherwise arbitrary vector ${\left| \phi \right.}\rangle$, I implies that the operator of ${\mathcal{Q}}$ is $P_{\phi }^{2}$, and since $P_{\phi }^{2} = P_{\phi }$, A′ also enforces that $Exp\left( {\mathcal{Q}} \right) = Exp\left( {\mathcal{P}} \right) \ge 0$. Thus, since $Exp\left( {\mathcal{P}} \right) = {\text{Tr}}\left( {UP_{\phi } } \right) = \langle{\phi {|}U{|}\phi}\rangle$, it must hold that $\langle{\phi {|}U{|}\phi}\rangle \ge 0$. For an arbitrary vector ${\left| \psi \right.}\rangle$, if ${\left| \psi \right.}\rangle \ne 0$, then $\left| \phi \right.\rangle = \left| \psi \right.\rangle/{\parallel}\left| \psi \right.\rangle{\parallel}$, so $\langle\phi {|}U{|}\phi\rangle = \langle\psi {|}U{|}\psi\rangle /{{\parallel}\left| \psi \right.\rangle{\parallel}}^{2}$, and therefore $\langle\psi {|}U{|}\psi\rangle \ge 0$. If $\left| \psi \right.\rangle = 0$, the same result follows trivially. Von Neumann also shows that the operators $U$ in the trace rule yield consistent absolute probabilities only if $Exp\left( {\mathcal{I}} \right) = 1$, where ${\mathcal{I}}$ is the quantity represented by the identity operator $I$, and this constraint leads to ${\text{Tr}}U = 1$.

Appendix 2: No Dispersion-Free States

Let us assume a dispersion-free state represented by $U$, and a quantity ${\mathcal{R}}$. By the definition of dispersion free-states, $Exp\left( {{\mathcal{R}}^{2} } \right)_{U} = \left[ {Exp\left( {\mathcal{R}} \right)_{U} } \right]^{2}$, so ${\text{Tr}}\left( {UR^{2} } \right) = \left[ {{\text{Tr}}\left( {UR} \right)} \right]^{2}$. Let us assume that $R = P_{\phi }$, therefore, ${\text{Tr}}\left( {UR} \right) = \left[ {{\text{Tr}}\left( {UR} \right)} \right]^{2}$ and $\langle\phi {|}U{|}\phi\rangle = \langle\phi {|}U{|}\phi\rangle^{2}$. Consequently, either $\langle\phi {|}U{|}\phi\rangle= 1$, or $\langle\phi {|}U{|}\phi\rangle= 0$. Let $\left| {\phi ^{\prime}} \right.\rangle$ and $\left| {\phi ^{\prime\prime}} \right.\rangle$ be two normalized vectors, and we vary $\left| \phi \right.\rangle$ continuously from $\left| {\phi ^{\prime}} \right.\rangle$ to $\left| {\phi ^{\prime\prime}} \right.\rangle$. von Neumann [1, p. 209, fn. 170] proved that $\left| \phi \right.\rangle$ is normalized along the whole variation, so $\langle\phi {|}U{|}\phi\rangle$ also varies continuously from $\langle\phi ^{\prime}{|}U{|}\phi ^{\prime}\rangle$ to $\langle\phi ^{\prime\prime}{|}U{|}\phi ^{\prime\prime}\rangle$. Now, since either $\langle\phi {|}U{|}\phi\rangle = 1$ or $\langle\phi {|}U{|}\phi\rangle = 0$, $\langle\phi {|}U{|}\phi\rangle$ must be constant along the whole variation, and $\langle\phi ^{\prime}{|}U{|}\phi ^{\prime}\rangle= \langle\phi ^{\prime\prime}{|}U{|}\phi ^{\prime\prime}\rangle$. But precisely because either $\langle\phi {|}U{|}\phi\rangle = 1$ or $\langle\phi {|}U{|}\phi\rangle = 0$ along the whole variation, then either $U = I$ or $U = 0$, respectively. However, ${\text{Tr}}U = 1$. Therefore, $U$ cannot be dispersion-free. Notice that this proof does not make use of A′, B′, I or II.

Appendix 3: Homogeneous States and Projectors

An ensemble $E$ is homogeneous if for any sub-ensembles $E_{1}$ and $E_{2}$, $Exp\left( {\mathcal{R}} \right)_{E} = aExp\left( {\mathcal{R}} \right)_{{E_{1} }} + bExp\left( {\mathcal{R}} \right)_{{E_{2} }}$ (where $a > 0$, $b > 0$, and $a + b = 1$) implies that $Exp\left( {\mathcal{R}} \right)_{E} = Exp\left( {\mathcal{R}} \right)_{{E_{1} }} = Exp\left( {\mathcal{R}} \right)_{{E_{2} }}$. From this definition and the trace rule, it follows that a state $U$ is homogeneous if $U = V + W$ (where $V$ and $W$ are also positive semi-definite) implies that $V = c^{\prime}U$ and that $W = c^{\prime\prime}U$. Here von Neumann is assuming the principle that with respect to relative probabilities and expectation values $U$ and $cU$ (with $c$ a positive constant) are essentially identical.

Let us assume that $U$ is homogeneous. Since $U$ is positive semi-definite, there is a non-zero vector $\left| {\phi_{ + } } \right.\rangle$ such that $\langle\phi_{ + } {|}U{|}\phi_{ + }\rangle > 0$. Then von Neumann introduced the operators $V$ and $W$ (where $\left| \phi \right.\rangle$ is any vector):

$$V\left| \phi \right.\rangle = \frac{{\langle\phi {|}U{|}\phi_{ + } \rangle}}{{\langle\phi_{ + } {|}U{|}\phi_{ + }\rangle }} U\left| {\phi_{ + } } \right.\rangle\quad W\left| \phi \right.\rangle = U\left| \phi \right.\rangle - V\left| \phi \right.\rangle$$

$V$ and $W$ are positive semi-definite, for

$$\langle\phi |V|\phi\rangle = \frac{{\left| {\langle\phi |U|\phi _{ + }\rangle } \right|^{2} }}{{\langle\phi _{ + } |U|\phi _{ + }\rangle }} \ge 0\quad \langle\phi |W|\phi\rangle = \frac{{\langle\phi |U|\phi\rangle \langle\phi _{ + } |U|\phi _{ + }\rangle - \left| {\langle\phi |U|\phi} \rangle\right|^{2} }}{{\langle\phi _{ + } |U|\phi _{ + }\rangle }} \ge 0$$

Now, since by assumption $U$ is homogeneous and $U = V + W$, then $V = c^{\prime}U$, and because $V\left| {\phi_{ + } } \right.\rangle = U\left| {\phi_{ + } } \right.\rangle \ne 0$, $c^{\prime} = 1$ and $V = U$. Then von Neumann defined the unit vector $\left| \psi \right.\rangle = U\left| {\phi_{ + } } \right\rangle/||U\left| {\phi_{ + } } \right.\rangle||$, and the positive constant $c = ||U\left| {\phi_{ + } } \right.\rangle||^{2} /\langle\phi_{ + } {|}U{|}\phi_{ + }\rangle$. With these, he finally obtained $U\left| \phi \right.\rangle = V\left| \phi \right.\rangle = c\left| \psi \right.\rangle\langle\psi {|}\phi\rangle = cP_{\psi } \left| \phi \right.\rangle$, which implies that $U = P_{\psi }$. That is, if $U$ is homogeneous, it is a projector onto a unit vector.

Conversely, assume that $U = cP_{\psi }$, where $\left| \psi \right.\rangle$ is again normalized. If $U = V + W$, where $V$ and $W$ are positive semi-definite, then $0 \le \langle\phi {|}V{|}\phi\rangle \le \langle\phi {|}V{|}\phi\rangle + \langle\phi {|}W{|}\phi\rangle = \langle\phi {|}U{|}\phi\rangle$. Now, if $\left| \phi \right.\rangle$ is orthogonal to $\left| \psi \right.\rangle$, then $U\left| \phi \right.\rangle = 0$, so that $\langle\phi {|}U{|}\phi\rangle = \langle\phi {|}V{|}\phi\rangle = 0$, and $V\left| \phi \right.\rangle = 0$. Now, for any vector $\left| \theta \right.\rangle$, it holds that $\langle\theta {|}V{|}\phi\rangle = \langle\phi {|}V{|}\theta\rangle = 0$, which implies that any vector that is orthogonal to $\left| \psi \right.\rangle$ is also orthogonal to $V\left| \theta \right.\rangle$, i.e., $\left| \psi \right.\rangle$ and $V\left| \theta \right.\rangle$ are collinear, so $V\left| \theta \right.\rangle = c_{\theta } \left| \psi \right.\rangle$, where $c_{\theta }$ depends on $\left| \theta \right.\rangle$. Thus, if $\left| \theta \right.\rangle = \left| \psi \right.\rangle$, then $V\left| \psi \right.\rangle = c^{\prime}\left| \psi \right.\rangle$. Finally, let $\left| {\phi^{\prime}} \right.\rangle$ be an arbitrary vector. It holds that $\left| {\phi^{\prime}} \right.\rangle$ can be written in the form $\langle\psi {|}\phi^{\prime}\rangle\left| \psi \right.\rangle\ + \left| \phi \right.\rangle$. Therefore, $V\left| {\phi^{\prime}} \right.\rangle = \langle\psi {|}\phi^{\prime}\rangle\cdot V\left| \psi \right.\rangle + V\left| \phi \right.\rangle = c^{\prime}\left| \psi \right.\rangle\langle\psi {|}\phi^{\prime}\rangle = c^{\prime}P_{\psi } \left| {\phi^{\prime}} \right.\rangle = c^{\prime}U\left| {\phi^{\prime}} \right.\rangle$. Thus, $V = c^{\prime}U$, and $W = U - V = \left( {1 - c^{\prime}} \right)U$. That is, if $U$ is a projector onto a unit vector, it is homogeneous. Notice that just like in the derivation of the corollary in Appendix 2, the proof of the result that $U$ is homogeneous iff it is a projector onto a unit vector does not make use of any of the premises in the derivation of the trace rule.