How the Weak Variance of Momentum Can Turn Out to be Negative

Abstract

Weak values are average quantities, therefore investigating their associated variance is crucial in understanding their place in quantum mechanics. We develop the concept of a position-postselected weak variance of momentum as cohesively as possible, building primarily on material from Moyal (Mathematical Proceedings of the Cambridge Philosophical Society, Cambridge University Press, Cambridge, 1949) and Sonego (Found Phys 21(10):1135, 1991) . The weak variance is defined in terms of the Wigner function, using a standard construction from probability theory. We show this corresponds to a measurable quantity, which is not itself a weak value. It also leads naturally to a connection between the imaginary part of the weak value of momentum and the quantum potential. We study how the negativity of the Wigner function causes negative weak variances, and the implications this has on a class of ‘subquantum’ theories. We also discuss the role of weak variances in studying determinism, deriving the classical limit from a variational principle.

Weak Values and Variances

Weak Values

Introduction

In the two state vector formalism, a weak value is equal to the transition amplitude between a prepared initial state and a postselection state [1]. We focus in this study on the weak value and the “weak variance” of the momentum operator $\hat{p}$ postselected by position $\left\langle x\right|$,

$$\begin{aligned} {_{x}\hat{p}_{\psi }} =\frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\; ,\quad {_{x}\hat{p}_{\psi }} \in \mathbb {C}. \end{aligned}$$

A number of the following results should generalise to any pair of canonically noncommuting variables, although a general definition of the weak variance is not the objective of this study. The weak value ${_{x}\hat{p}_{\psi }} \in \mathbb {C}$ can be split into real and imaginary parts [2] as

$$\begin{aligned} {_{x}\hat{p}_{\psi }} = \frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }=\hbar \partial _x \left( \arg \psi \right) -i\left( \frac{\hbar }{2}\frac{\partial _{x}\left| \psi \right| ^{2}}{\left| \psi \right| ^{2}}\right) . \end{aligned}$$

(1)

One might naively attempt to define a weak variance as the second central moment out of weak values ${_{x}\hat{p}^2_{\psi }} - {_{x}\hat{p}_{\psi }}^2$. However, the fact that weak values are complex requires one to take real parts of this naive construction—and it makes a difference whether one squares the second term before or after taking real parts: $\mathrm {Re}({_{x}\hat{p}_{\psi }}^2) \ne (\mathrm {Re}\,{_{x}\hat{p}_{\psi }})^2$. This ambiguity motivates the search for a more ‘fundamental’ approach to these statistical quantities, which this first section will attempt to describe.

This study is organised as follows. We begin by reviewing the relation between the real parts of weak values and the phase space formalism of quantum mechanics. The quasistatistical interpretation of this formalism suggests a natural and unique definition for the “weak variance” in terms of the Wigner function. In Sect. 2, the embedding of configuration space formalisms into the phase space formalism then allows the weak variance to be related to the thermodynamics of the Madelung fluid, to the de Broglie–Bohm quantum potential $Q$, and to an experimentally measurable combination $\mathrm {Re}({_{x}\hat{p}^2_{\psi }}-{_{x}\hat{p}_{\psi }}^2)$ of weak values.

In Sect. 3 we discuss the titular problem, that the negativity of the Wigner function can result in a negative weak variance. The weak variance therefore does not define a standard deviation of weak measurements from the weak value. The sign of the weak variance is shown to be related to the local extrema of the probability amplitude. The relation between weak variances and determinism paves the way for a detailed study of the classical limit in Sect. 4. We derive the classical limit from a variational principle involving the average stochastic contribution to the total momentum variance, and demonstrate the existence of a semiclassical limit where particle motion occurs, on average, classically.

The Importance of the Measurement Apparatus

It is important to note [3] that weak values are not measured independently of an interaction Hamiltonian or a meter system. The real part of a weak value is well known to be the conditional average one would measure in the ideal limit of zero disturbance form the measuring apparatus [4–6]. The imaginary part of a weak measurement is unrelated to observables, it arises from the disturbance due to coupling with a measurement apparatus [4]. As such, it is not uniquely defined and not intrinsic to the measured system [3].

Since only the real part informs us about our operator as an observable, we opt to describe the variance of only the real part of the weak value in this study. However, the imaginary part in (1) will emerge naturally during our investigation of weak variances for the real part, without reference to a specific interaction Hamiltonian or meter system. In principle the ‘ideal’ decomposition (1) would be measured as [3, 7]

$$\begin{aligned} {_{x}\hat{p}_{\psi }} = \mathrm {Re}\left( \frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) + \eta \; \mathrm {Im}\left( \frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) , \end{aligned}$$

(2)

such that the theoretically natural imaginary part in (1) is arbitrarily scaled in experiments by adjusting the value of $\eta$. This parameter depends on meter states and observables, as well as the interaction strength with the apparatus [3].

We shall therefore discuss the imaginary part as if it were intrinsic to the system, absorbing all dependence on the measurement protocol into the parameter $\eta$. This has the disadvantage of illusorily disconnecting these results from the experimental setups that give rise to them, which comes hand in hand with the hidden advantage of being a more universal treatment than that derived from any specific experimental setup. As such, the weak variance we define is related to the stochastic nature of quantum theory; it is not the experimental variance of any apparatus (although it may manifest itself as such in a measurement setting). Not without irony, this apparatus-agnostic treatment will suggest multiple possible experimental setups to study the weak variance (Sect. 2.3).

Weak Values as Conditional Expectations

In the quasistatistical interpretation [8–10] of the phase space formalism [11–14], the canonical observables $x,p\in \mathbb {R}$ are treated as random variables. Since these observables are incompatible, no joint probability distribution can reasonably be given for them—the phase space distributions are understood to be bivariate pseudoprobability distributions. For example, the Wigner function $\mathcal {W}\left( x,p\right)$ is not positive definite. For a review of extended probability we refer the reader to [10].

Our choice of the Wigner function to define the weak variance is motivated by this quasidistribution’s special status amongst physically equivalent representations of the phase space formalism (which correspond to different choices of operator ordering, $[\hat{x},\hat{p}]\ne 0$). Indeed, only the Wigner function produces the correct marginals and observable averages by direct integration [13]; other quasidistributions require explicit convolutions over noncommutativity terms. As such, the Wigner function is uniquely singled out by the quasistatistical outlook, where such noncommutativity terms would interfere with the interpretation of observable averages as moments of the pseudodistribution.

The real part of a weak value is a conditional average [5, 6, 15]. Hence, to obtain weak values of momentum, one must consider not the momentum $p$, but instead the mean value of $p$ at a given position, for a statistical ensemble. One may consequently express the experimental protocol of postselection with conditional random variables: the quantity $p|x$ ($p$ conditioned by $x$) is also a random variable. To find the distribution of $p|x$, we need to consider the conditional distribution $\mathcal {W}\left( p|x\right)$. The Wigner function has a marginal distribution

$$\begin{aligned} \mathcal {W}\left( x\right) =\int \mathcal {W}\left( x,p\right) dp=\left| \psi (x)\right| ^{2}, \end{aligned}$$

(3)

and so the conditional distribution for the random variable $p|x$ is simply

$$\begin{aligned} \mathcal {W}\left( p|x\right) =\frac{\mathcal {W}\left( x,p\right) }{\mathcal {W}\left( x\right) }. \end{aligned}$$

(4)

These observables $p$, $x$ are incompatible: in what sense can we condition one on the other if they are not meaningful simultaneously? A random variable is defined by its distribution. Individual realisations of $x$ and $p$ are meaningful positions and momenta only because their distributions (the marginals of the Wigner quasidistribution) are proper distributions. The random variable $p|x$ is defined by the conditional quasidistribution $\mathcal {W}\left( p|x\right)$, which from (4) is clearly not nonnegative. Hence individual realisations of $p|x$ may occur with negative probability. This follows the same logic as stating that joint realisations $(x,p)$ may occur with negative probability as realisations of the full Wigner function.

However, the fact that individual realisations of $p|x$ can occur with negative probabilities does not preclude the use of this conditional variable as an intermediate step in a calculation of a physical quantity [10]. Furthermore, the mean or the variance of $p|x$ need not be confined to intermediate steps in this way, since they are not random variables but instead contribute to the description of the physical state of an ensemble (i.e. they describe the Wigner function as a whole rather than the outcome of a measurement).

The mean value of $p|x$,

$$\begin{aligned} \mathbb {E}_{p}\left( p|x\right) \equiv \tilde{p}, \end{aligned}$$

(5)

can be thought of as a function of $x$ [9]. This mean value $\tilde{p}$ will in Sect. 1.2.1 be shown to equal the real part of the weak value $_{x}\hat{p}_{\psi }$. One may then express the conditional mean momentum in terms of $\mathcal {W}$ as the first conditional moment [8], i.e. the first moment of the conditional distribution:

$$\begin{aligned} \tilde{p}= & {} \int p\mathcal {W}\left( p|x\right) \; dp\end{aligned}$$

(6)

$$\begin{aligned}= & {} \frac{1}{\mathcal {W}\left( x\right) }\int \; p\mathcal {W}\left( x,p\right) \; dp. \end{aligned}$$

(7)

It is possible to formulate complex weak values of arbitrary observables directly in the phase space formalism using cross-Wigner functions [16], but as noted earlier our focus is currently limited to the variance of the real part of position-postselected weak momentum measurements.

Weak Variance as the Second Conditional Cumulant

Through the statistical Wigner–Moyal formalism described above and in [8], the weak value $\tilde{p}$ takes on the role of a statistical average. However, an average is scientifically worthless unless one can provide an associated spread. It is well established that the variance, i.e. the second central moment of a distribution, is equal to the second cumulant of that distribution. With some effort, the same can be shown of conditional variances and conditional cumulants [17], and extended to those of pseudodistributions like the Wigner function [8].

Derivation

Using the characteristic function associated to $\mathcal {W}\left( x,p\right)$ and the polar decomposition $\psi =\sqrt{\rho }e^{iS}$, the conditional cumulants of this distribution can be written for even cumulants ($n=2,4,\ldots$) as [8]

$$\begin{aligned} \kappa _{n}|x=\left( \frac{\hbar }{2i}\right) ^{n}\left( \frac{\partial }{\partial x}\right) ^{n}\ln \rho \left( x\right) , \end{aligned}$$

(8)

and for odd cumulants ($n=1,3,\ldots$) as

$$\begin{aligned} \kappa _{n}|x=\left( \frac{\hbar }{2i}\right) ^{n-1}\left( \frac{\partial }{\partial x}\right) ^{n}\hbar S\left( x\right) . \end{aligned}$$

(9)

The derivation of this result has been reproduced in full as an Appendix.

By definition, the first two conditional cumulants $n=1,2$ are also the first two conditional moments, and one finds a simple relation [8],

$$\begin{aligned} \tilde{p}=\frac{\partial \left( \hbar S\right) }{\partial x}, \end{aligned}$$

for the ($n=1$) average $\tilde{p}=\mathbb {E}_{p}\left( p|x\right)$. Simple comparison of this expression with (1) should confirm that $\tilde{p} = \mathrm {Re}\left( {_{x}\hat{p}_{\psi }}\right)$. The $n=2$ expression then defines the weak variance of the random variable $p|x$:

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =-\frac{\hbar ^{2}}{4}\left( \frac{\partial }{\partial x}\right) ^{2}\ln \rho \left( x\right) . \end{aligned}$$

(10)

Dimensional analysis confirms that this quantity has units of $\left[ \mathrm {momentum}\right] ^{2}$, as expected of a (conditional) momentum variance. Because of the clean splitting between phase-dependent odd cumulants and amplitude-dependent even cumulants, the weak variance does not depend on the wavefunction phase, except indirectly (via the time evolution of the amplitude).

Other Expressions for the Weak Variance

This second conditional cumulant is also expressible directly as the second central conditional moment of the conditional distribution $\mathcal {W}\left( p|x\right)$:

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =\int \mathcal {W}\left( p|x\right) \left( p|x-\mathbb {E}_{p}\left( p|x\right) \right) ^{2}dp. \end{aligned}$$

(11)

Furthermore, as we shall demonstrate in Sect. 2.1.2, the weak variance also takes the following form, more readily related to weak values:

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) = \frac{1}{2}\mathrm {Re}\left( \frac{\left\langle x\right| \hat{p}^2 \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle } -\left( \frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) ^2\right) . \end{aligned}$$

This last expression shows that despite our use of unphysical conditional random variables $p|x$ in its derivation, the weak variance is indeed related to the intuitive form $\langle p^2\rangle - \langle p\rangle ^2$ of a variance. It also shows that the weak variance is measurable in an experimental setting.

The Uncertainty Principle

It is possible to relate the conditional variance $\mathbb {V}_{p}\left( p|x\right)$ to the unconditional variance $\mathbb {V}\left( p\right)$ that appears in the uncertainty relations.

The Law of Total Variance

One may readily verify that the law of total expectation,

$$\begin{aligned} \mathbb {E}\left( p\right) =\mathbb {E}_{x}\left( \mathbb {E}_{p}\left( p|x\right) \right) =\mathbb {E}_{x}\left( \tilde{p}\right) , \end{aligned}$$

(12)

reads in our quantum setting

$$\begin{aligned} \left\langle \psi \right| \hat{p}\left| \psi \right\rangle =\int \left| \psi \right| ^{2}\tilde{p}dx. \end{aligned}$$

(13)

In the same way that one derives the law of total expectation, one may derive the Law of Total Variance:

$$\begin{aligned} \mathbb {V}\left( p\right)= & {} \mathbb {E}_{x}\left( \mathbb {V}_{p}\left( p|x\right) \right) +\mathbb {V}_{x}\left( \mathbb {E}_{p}\left( p|x\right) \right) \nonumber \\= & {} \mathbb {E}_{x}\left( \mathbb {V}_{p}\left( p|x\right) \right) +\mathbb {V}_{x}\left( \tilde{p}\right) , \end{aligned}$$

(14)

where $\mathbb {V}_{x}\left( \tilde{p}\right)$ is the variance of the weak value $\tilde{p}$ as a function of $x$, and where $\mathbb {E}_{x}\left( \mathbb {V}\left( p|x\right) \right)$ is the average value of the weak variance (as a function of $x$). $\mathbb {V}_{x}\left( \tilde{p}\right)$ can be simply given by the marginal variance

$$\begin{aligned} \mathbb {V}_{x}\left( \tilde{p}\right) =\int \mathcal {W}\left( x\right) \left( \tilde{p}-\mathbb {E}_{x}\left( \tilde{p}\right) \right) ^{2}dx, \end{aligned}$$

(15)

where the law of iterated expectation (13) gives $\mathbb {E}_{x}\left( \tilde{p}\right) =\left\langle \psi \right| \hat{p}\left| \psi \right\rangle$. Hence, $\mathbb {V}_{x}\left( \tilde{p}\right)$ measures the standard deviation of the weak value $\tilde{p}$ from the usual expectation value $\left\langle \hat{p} \right\rangle$ as the postselection position $x$ changes. Particularly, it does not measure the variance of $p|x$ (or of any other random variable): the variable $\tilde{p}$ in the variance $\mathbb {V}_{x}$ is deterministically defined by (15). The nonweak variance $\mathbb {V}_{x}\left( \tilde{p}\right)$ contributes to the total variance of $\hat{p}$ of the uncertainty principle, but it is not itself a manifestation of quantum randomness.

By contrast the conditional variance,

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =\int \mathcal {W}\left( p|x\right) \left( p|x-\mathbb {E}_{p}\left( p|x\right) \right) ^{2}dp, \end{aligned}$$

is truly the variance of a random variable: it says how far from the mean $\mathbb {E}_{p}\left( p|x\right) = \tilde{p}$ one may expect to find the random variable $p|x$. Unlike the total variance or the variance of $\tilde{p}$, this “scedastic function” depends on the postselection $x$. It is in this sense that we identify it as the weak variance.

These two parts of the total variance allow interesting limits where their contributions are negligible one with respect to the other. For example, in stationary states there is no variability in $\tilde{p}(x)$ by construction, and the total variance of the state is supported entirely by the weak variance. A more detailed discussion of these limits, and their relevance to the classical limit, is postponed until the relevant properties of the weak variance $\mathbb {V}_{p}\left( p|x\right)$ are introduced (Sect. 4.1).

Apparatus versus Fundamental Weak Variances

This connection between the weak variance and the momentum variance reconfirms that the weak variance is central to understand the predictions of QM. Oddly, we find that the weak variance is related to the fundamental momentum variance $\mathbb {V}(p)$ of a state (as described in the Robertson uncertainty relations) rather than the momentum change $\Delta p$ when a state is disturbed (as originally described by Heisenberg).

Since $\mathbb {V}_{p}\left( p|x\right)$ is given by differentiation of the wavefunction amplitude (10), it depends only on the values of the wavefunction amplitude in the neighbourhood of the postselection point $x$. This choice of postselection point is not apparatus-dependent, although issues such as pointer variance for the postselection variable will certainly affect the analysis of any experimental study.

The weak variance is further distinguished from an ‘apparatus quantity’ by the fact that it is not an ‘observable’, in the sense that it does not correspond to a Hermitean operator (cf Sect. 2.1.3) or equivalently (in the presentation adopted above) to a real random variable. Instead, as a cumulant of the Wigner function, it describes the state of the system, irrespective of how it is measured.

Connection to Other Formalisms

One can recognise [18] the probability flux density $\int p\mathcal {W}\left( x,p\right) \; dp=mj$ in (7), and re-express this equation using the marginal (3), as

$$\begin{aligned} \frac{\tilde{p}}{m}=\frac{j\left( x,t\right) }{\left| \psi \right| ^{2}}. \end{aligned}$$

(16)

The trajectories of the Hydrodynamical and Bohmian pictures are generated from (16), by setting a velocity $\dot{x}=\tilde{p}/m$ and solving the resulting ODE for $x\left( t\right)$. We will present only those elements of these formalisms that are of interest to our weak variance study; a more detailed account of this ‘embedding’ of configuration space descriptions of QM into phase space is given in [9].

The derivation of the real part of the momentum

$$\begin{aligned} \tilde{p}=\partial _x\left( \hbar S\right) \end{aligned}$$

(17)

from the Schrödinger Equation is present in almost every work related to the Hydrodynamical or de Broglie–Bohm formalisms since 1927 [19], therefore no detail will be given here: The wavefunction is polar-decomposed, derivatives of the components split the Schrödinger equation into real and imaginary parts, from which (17) is obtained by either appealing to an analogy between the resulting equations and the Euler equations (in the hydrodynamical setting) or to an analogy between these equations and the Hamilton–Jacobi equation (in the Bohmian setting).

Bohmian Approach to the Weak Variance

Prelude

In the quantum Hamilton–Jacobi formalism [20–23], Hamilton’s Principal Function is $\mathcal {S}\left( x,t\right) =\hbar S\left( x,t\right) +\mathrm {const.}$, such that the polar decomposition (17) becomes the Hamilton–Jacobi definition of a canonical momentum

$$\begin{aligned} \tilde{p}=\partial _{x}\mathcal {S}. \end{aligned}$$

(18)

Since $\tilde{p}$ is not the classical canonical momentum, but an average, the theory remains fully quantum mechanical [24]. Particularly, the Hamiltonian

$$\begin{aligned} \mathcal {H}_\mathrm {Bohm} = -\frac{\partial \mathcal {S}}{\partial t} = \frac{\tilde{p}^{2}}{2m}+V+Q \end{aligned}$$

(19)

has a quantum potential $Q$ [20, 21]:

$$\begin{aligned} Q=-\frac{\hbar ^{2}}{2m}\frac{\partial _x^{2}\left| \psi \right| }{\left| \psi \right| }. \end{aligned}$$

(20)

The form (10) can be re-expressed (using $R=\left| \psi \right| =\sqrt{\rho }$) as:

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) = -\frac{\hbar ^2}{4} \frac{\partial }{\partial x} \left( \frac{\partial _x \rho }{\rho } \right) = -\frac{\hbar ^{2}}{2} \frac{\partial }{\partial x} \left( \frac{\partial _x R}{R}\right) . \end{aligned}$$

(21)

The imaginary part of the weak value (1) is

$$\begin{aligned} \mathrm {Im}\left[ _{x}\hat{p}_{\psi }\right] =-\frac{\hbar }{2}\frac{\partial _x \rho }{\rho }=-\hbar \frac{\partial _x R}{R}, \end{aligned}$$

(22)

where the experimental $\eta$ term from (2) is suppressed. Equation (21) then indicates that the weak variance is proportional (with a dimensionful factor of $\hbar /2$) to the divergence of this imaginary part.

Applying the divergence operator in (21) gives

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =-\frac{\hbar ^2}{4} \left( \frac{\partial _x^2 \rho }{\rho } - \frac{(\partial _x \rho )^2 }{\rho ^2} \right) =-\frac{\hbar ^{2}}{2} \left( \frac{\partial _x^{2} R}{R} - \left( \frac{\partial _x R}{R}\right) ^{2}\right) , \end{aligned}$$

(23)

and by recognising each term above (with (20) and (22)), we find

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =\frac{\left[ \mathrm {Im}_{x}\hat{p}_{\psi }\right] ^{2}}{2} + mQ. \end{aligned}$$

(24)

This nontrivially relates the weak variance to the quantum potential of the Hamilton–Jacobi picture, providing further evidence of the physical importance of $\mathbb {V}_{p}$. The imaginary part of the weak value also appears in the expression above. By combining (21), (22), and (24) we obtain a nonlinear differential (Riccati) equation fixing the form of $\mathrm {Im}_{x}\hat{p}_{\psi }$ at any given time,

$$\begin{aligned} \frac{d\;{\mathrm {Im}{}_{x}\hat{p}_{\psi }}}{dx} - \frac{1}{\hbar }\left[ \;\mathrm {Im}_{x}\hat{p}_{\psi }\right] ^{2} = \frac{2m}{\hbar }Q(x) \end{aligned}$$

(25)

This will be an important element in our discussion of the classical limit (Sect. 4.1).

An Expression for $\mathbb {V}_{p}\left( p|x\right)$ in Terms of Weak Values

Consider the weak value of momentum squared. Substituting once again the polar form $\psi = |\psi |e^{iS}$ and $\hat{p}^2 = -\hbar ^2 \partial _x^2$ gives

$$\begin{aligned} \mathrm {Re}\left( \frac{\left\langle x\right| \hat{p}^2 \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) = -\hbar ^2 \mathrm {Re} \left( \frac{ \partial _x^2 \psi }{\psi } \right) = ( \hbar \partial _x S)^2 + 2mQ. \end{aligned}$$

(26)

Recognising $\hbar \partial _x S$ as the real part of the weak value of momentum, we can re-express the quantum potential $Q$ in a well-known variance-like form (up to a scaling by $2m$):

$$\begin{aligned} Q = \frac{1}{2m} \left( \mathrm {Re}\left( \frac{\left\langle x\right| \hat{p}^2 \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) - \left[ \mathrm {Re}\left( \frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) \right] ^2 \right) . \end{aligned}$$

(27)

Now consider the weak value of momentum, squared:

$$\begin{aligned} \mathrm {Re}\left( \left( \frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) ^2\right) = \mathrm {Re}({_{x}\hat{p}_{\psi }})^2 - \mathrm {Im} ({_{x}\hat{p}_{\psi }})^2 = ( \hbar \partial _x S)^2 - \mathrm {Im} ({_{x}\hat{p}_{\psi }})^2. \end{aligned}$$

Clearly, we can subtract this from (26) above to obtain

$$\begin{aligned} 2\left( mQ + \frac{\mathrm {Im} ({_{x}\hat{p}_{\psi }})^2}{2}\right)= & {} \mathrm {Re}\left( \frac{\left\langle x\right| \hat{p}^2 \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) - \mathrm {Re}\left( \left( \frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) ^2\right) \end{aligned}$$

(28)

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right)= & {} \frac{1}{2}\mathrm {Re}\left( \frac{\left\langle x\right| \hat{p}^2 \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle } -\left( \frac{\left\langle x\right| \hat{p} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) ^2\right) , \end{aligned}$$

(29)

by use of (24) and linearity of $\mathrm {Re\left( \cdots \right) }$. This expression depends on both the first order (${_{x}\hat{p}_{\psi }}$) and second order (${_{x}{\hat{p}^2}_{\psi }}$) weak values, and resolves the ordering ambiguity, discussed in the Introduction, between taking the real part and taking the square of the first order weak value (compare (27) and (29)).

The Weak Variance is Not a Weak Value

We have reached an expression for $\mathbb {V}_{p}\left( p|x\right)$ which superficially has the form $\langle p^2 \rangle - \langle p\rangle ^2$. Let it be noted, however, that this is not a construction in terms of moments of a distribution: the complex weak value ${_{x}\hat{p}_{\psi }}$ is not a moment, it is the real part $\tilde{p} = \mathrm {Re}({_{x}\hat{p}_{\psi }})$ that plays this role.

Consider, now, expanding the square in (11) to obtain

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) = \left( \int p^2 \mathcal {W}(p|x) dp \right) - \mathbb {E}(p)^2 = \mathbb {E}_p(p^2|x) - \mathbb {E}(p)^2. \end{aligned}$$

This also almost looks like $\langle p^2 \rangle - \langle p\rangle ^2$, however the two expectations belong to different (pseudo)distributions ($\mathcal {W}(p|x)$ and $\mathcal {W}(x,p)$ respectively). Using the law of total expectation, we could write both terms using $\mathbb {E}_p$:

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) = \mathbb {E}_p(p^2|x) - \mathbb {E}_p(\mathbb {E}_y(y|p))^2, \end{aligned}$$

where $y|p$ is (formally) a position postselected on momentum. This averaging over all positions $y$ shows that despite being locally valued (in terms of the postselection $x$), $\mathbb {V}_{p}\left( p|x\right)$ encompasses information from the entire configuration space. This is not surprising if one remembers that the wavefunction solves a Schrödinger equation over this same space.

Quantum mechanical expectations are calculated in the Wigner formalism as integrals over the entire phase space,

$$\begin{aligned} \langle \hat{A} \rangle = \int dx \int dp A(x,p) \mathcal {W}(x,p), \end{aligned}$$

(30)

where $A(x,p)$ is the classical observable and Weyl operator ordering is assumed [13]. One may then define configuration space densities

$$\begin{aligned} A_\mathrm {cs} = \int dp A(x,p) \mathcal {W}(x,p), \end{aligned}$$

(31)

which give the expectation $\langle \hat{A} \rangle$ by direct integration, and ‘local densities’ [18, 25, 26]

$$\begin{aligned} A_\mathrm {loc} = |\psi |^{-2} A_\mathrm {cs} = \int dp A(x,p) \mathcal {W}(p|x) = \mathrm {Re}\left( \frac{\left\langle x\right| \hat{A} \left| \psi \right\rangle }{\left\langle x|\psi \right\rangle }\right) , \end{aligned}$$

(32)

which give the expectation by weighting the integration by the probability distribution $|\psi |^{2}$. By writing our classical observable as a polynomial $A(x,p)=\sum _n f_n(x) p^n$, we find that these local densities are nothing more than linear combinations of conditional moments / weak values of $p^n$. We have already seen (29) that the weak variance is quadratic in the weak value ${_{x}\hat{p}_{\psi }}$, therefore it is not a local density and no classical observable associated to it exists. Furthermore this suggests (since classical observables are quantum observables too) that there is no weak variance operator $\widehat{\mathbb {V}_p}$, as argued on interpretational grounds in Sect. 1.3.2. Therefore, the weak variance is not a weak value.

Hydrodynamical Approach to the Weak Variance

The real part $\tilde{p}$ of the weak value is the (Euler-picture) momentum of the Madelung fluid at position $x$. Classically, the fluid momentum is the (local) average momentum of a particle in the fluid; quantum mechanically, $\tilde{p}$ retains this qualitative feature of being a local average as discussed above.

In the Hydrodynamical picture, one can derive [9, 18] a pressure tensor using the same logic [19] from which one derives the fluid momentum (17). We obtain in one dimension (cf. [18, Appendix C]):

$$\begin{aligned} P = \frac{1}{m}\int (p-\tilde{p})^2 \mathcal {W}(x,p) \mathrm {d}p= \frac{\rho }{m}\left( -\frac{\hbar ^2}{4} \partial _x ^2\ln (\rho ) \right) . \end{aligned}$$

(33)

Combining the pressure above and the equation of state of the Madelung fluid $P=\rho k_B T$ gives a definition for the thermal energy [18]:

$$\begin{aligned} k_{B} T=-\frac{\hbar ^{2}}{4m}\partial _x^{2}\ln \rho = \frac{\mathbb {V}_{p}\left( p|x\right) }{m}. \end{aligned}$$

(34)

There is an intuitive picture relating fluid temperatures to momentum variances: In the kinetic theory of gases, a higher temperature corresponds to a wider Maxwell–Boltzmann distribution of momenta, according to $\sigma ^{2}\propto mk_{B}T$. The corresponding quantum relation (34) preserves this intuition at the fluid level, for a temperature distribution $T=T\left( x\right)$ over postselection $x$.

This expression (33) should be compared with the independently motivated expression (10) for the weak variance. In this light, the equation of state may be justified (rather than postulated) by writing the Wigner function from (33) in terms of its marginal, using (10) to find the solution $P=\frac{\rho }{m}\mathbb {V}_{p}\left( p|x\right)$, and then invoking this analogy to kinetic theory to link $P$, $\rho$ and $T$ by an equation of state.

Physical potentials tend to arise from interactions (mediated by gauge fields in the Standard Model or the metric field in General Relativity), but the quantum ‘potential’ $Q$ puzzlingly appears to pop out of the Schrödinger equation. Interpreting the weak variance as a thermal quantity pushes the ‘interactions’ which give rise to $Q$ into a hypothetical microstate description from which postselected quantum mechanics emerges thermodynamically, potentially resolving this philosophical worry.

We purposefully do not propose at this time any underlying statistical thermodynamical picture, instead pointing the reader to the discussion of “subquantum theory” in [18], which suggests that one should analyse the properties of the Wigner Function (such as, in this study, its weak variance) in order to gather some information about such a theory (cf. Sect. 3.4). It is a strength of thermodynamics that it does not formally require any model for microstates.

Measurement of the Weak Variance

We have shown that the weak variance can be described without reference to the properties of a specific apparatus, using only the description of the system’s state through the Wigner quasidistribution or the wavefunction $\psi =Re^{iS}$. However, combining the many equivalent expressions developed in this section with the $\eta$ parameterisation of the imaginary part (2), shows that this quantity should in fact be easy to measure in a number of experimental settings.

Unsurprisingly, the weak variance of momentum can be measured using weak measurements of momentum. The form (29) is available if one has access to the second-order weak value ${_{x}\hat{p}^2_{\psi }}$. With a good calibration of the apparatus’ $\eta$ and a good resolution over $x$, (21) can also be used.

More interestingly, weak measurements of momentum are not necessary to quantify the weak momentum variance. Given the probability density $\rho \left( x\right)$ determined from a series of position measurements, one may simply use the $-\partial _x^{2}\ln \left( \rho \right)$ form to find the weak variance at any position, without performing a single momentum measurement, weak or otherwise. A combination of weak momentum measurements and projective position measurements (possibly the same measurements used for postselection) can be used to cross-check the previous two measurements using a single dataset.

One might also verify experimentally the law of total variance (14): for example, one can measure the total variance $\mathbb {V}\left( p\right)$ from projective momentum measurements, and one can measure each of the partial variances from weak momentum measurements.

Having these many experimentally independent ways to measure $\mathbb {V}_{p}\left( p|x\right)$ should allow this formalism to be stringently tested by contemporary instruments (modulo a calibration or model for the apparatus parameter $\eta$).

Negative Weak Variances

Quantum Mechanics predicts that in some regions, $\mathbb {V}_{p}\left( p|x\right) <0$; this odd behaviour is known to the literature, but often glossed over (cf. e.g. the footnote to the Appendices of our primary source [8]). The Bohmian and hydrodynamical formalisms even allow the ‘weak trajectories’ $x\left( t\right)$ to cross from regions where $\mathbb {V}_{p}\left( p|x\right)$ is positive to regions where it is negative. It is not sufficient to say ‘weak values can lie outside of the eigenspectrum’ to explain why a variance can turn out to be negative.

Furthermore, this is not just an odd but unmeasurable feature of the theory, like the negative probabilities of realisations of $p|x$ [8, 10, 12]. The second cumulant may, as we have just argued, be measurably negative.

Explaining $\mathbb {V}_{p}\left( p|x\right) <0$

The reason we identify $\mathbb {V}_{p}\left( p|x\right)$ as a (conditional) variance is that it appears as the second term in the expansion of the (conditional) characteristic function into cumulants (8). We therefore consider to what extent such an interpretation for this quantity is warranted beyond the (experimentally successful [27]) identification of the first cumulant as a conditional average.

The conditional variance is given by the second central conditional moment of $\mathcal {W}$ (11):

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =\int \mathcal {W}\left( p|x\right) \left( p-\mathbb {E}_{p}\left( p|x\right) \right) ^{2}dp, \end{aligned}$$

(35)

which may be contrasted with the marginal moment (15). The term $\left( p|x-\tilde{p}\right) ^{2}$ is the square of a real quantity, so it must be positive. Any negativity in $\mathbb {V}_{p}\left( p|x\right)$ must therefore come from the distribution $\mathcal {W}\left( p|x\right)$; and indeed, the conditional Wigner function can take negative values [8]. This should be contrasted with $\mathbb {V}_{x}\left( \tilde{p}\right)$, which depends on the marginal distribution $\mathcal {W}(x)=\left| \psi \right| ^{2}\ge 0$ and thus remains positive.

Wigner Negativity and Standard Deviations

Even though the conditional variance may be negative, allowing imaginary standard deviations of $p|x$ would violate the requirement that $p,x\in \mathbb {R}$ (i.e. the requirement that realisations of a real random variable cannot be complex). A standard deviation defined using an ad hoc extension to negative values,

$$\begin{aligned} \sigma _{p|x}=\sqrt{\mathrm {abs}\left( \mathbb {V}_{p}\left( p|x\right) \right) }, \end{aligned}$$

(36)

satisfies $\sigma _{p|x}\in \mathbb {R}^{+}$, such that all our random variables are real, as required. This extension manifestly reproduces the usual standard deviation when $\mathbb {V}_{p}\left( p|x\right) \ge 0$. This last point is necessary not only for mathematical consistency, but also for consistency of the thermodynamical picture (34): The standard deviations of momentum in kinetic theory are obtained from the Maxwell–Boltzmann distribution $\sigma \propto \sqrt{mk_{B}T}$.

However, the conditional distribution is a pseudodistribution, invalidating the simple expression (36). Consider the integral form (35) of the weak variance: even when $\mathbb {V}_{p}\left( p|x\right) = 0$, there can be a spread of $p|x$ away from the weak value, since the negative and positive parts of the pseudodistribution $\mathcal {W}\left( p|x\right)$ may cancel under the integral defining a cumulative distribution, without any information given about the squared-term $(p|x - \tilde{p})^2$. Expressed more formally, deviation-bounding theorems such as the Chebyschev inequality do not apply to quasidistributions. This possibility for stochasticity at zero variance is further emphasised by the fact that higher $n\ge 2$ cumulants (8,9) are generally nonzero, indicating that the pseudoprobability distribution is generally nontrivial.

One might then attempt to push the absolute value inside the integral that defines $\mathbb {V}_{p}\left( p|x\right)$ to take care of this problem, defining instead our standard deviation with

$$\begin{aligned} \sigma _{p|x}^2 = \int \mathrm {abs}(\mathcal {W}\left( p|x\right) )\left( p|x-\mathbb {E}_{p}\left( p|x\right) \right) ^{2}dp. \end{aligned}$$

(37)

By virtue of the triangle inequality $\left| \int A dx\right| \le \int \left| A\right| dx$, this standard deviation gives at least as large a spread in $p|x$ as (36). Furthermore, it clearly reproduces (36) whenever $\mathcal {W}\left( p|x\right) \ge 0$, and so inherits all the desirable properties discussed above.

However, this approach is similarly unacceptable. The expectation $\mathbb {E}_{p}$ is also an integral over $p$, yet if we want to consider deviations away from $\tilde{p}$ then we cannot replace the conditional distribution by its absolute value in this expectation integral: changing the distribution changes the random variable of which we consider the mean! Furthermore, the correct normalisation of the quasidistribution $\int dp \mathcal {W}\left( p|x\right) = 1$ guarantees that $\mathrm {abs}(\mathcal {W}\left( p|x\right) )$ is not normalised, and so it is not a proper probability distribution either.

In light of these failures, it is even possible to dispute whether a standard deviation is a sensible quantity for quasidistributions. Unlike the variance, which pertains to the distribution, the standard deviation is a deviation of realisations of a random variable from their mean. As discussed previously, realisations of $p|x$ can occur with negative probability: building a sample standard deviation from these phantasmagoric realisations would be meaningless as the final step of a calculation, even in a quasistatistical outlook.

Shape of $\psi$

In the description of Bose–Einstein superfluids, a quantum-potential-like term appears in the polar decomposition of the Gross–Pitayevski equation. In this context, $Q$ is interpreted as the kinetic energy cost of curvature of the condensate wavefunction—it costs energy to squeeze a wavepacket.

Given that the weak variance depends in part on the quantum potential (24), one might wonder whether (by analogy) $\mathbb {V}_{p}\left( p|x\right)$ is related to the curvature of the wavefunction.

We now consider how the curvature of the wavefunction amplitude, $R=|\psi |$, determines the sign of the weak variance. The relevant quantity is the convexity $\partial _x^{2}R$ of the amplitude, which is positive for convex intervals of $|\psi |$ and negative for concave intervals.

Non-nodal Points

We use the decomposition (23) to solve the inequality $\mathbb {V}_{p}\left( p|x\right) \ge 0$. We note that $R$ is nonnegative by definition: for non-nodal points (i.e. points $x$ such that $R(x)\ne 0$) the weak variance is nonnegative whenever the convexity satisfies

$$\begin{aligned} \frac{\left( \partial _x R\right) ^{2}}{R}\ge \partial _x^{2}R. \end{aligned}$$

(38)

The left-hand side of (38) is positive, so concave regions ($0>\partial _x^{2}R$) always have positive weak variances (independently of the system under consideration). Regions that have a very small concavity are also very likely to satisfy (38).

Maxima of the wavefunction amplitude are concave so $\mathbb {V}_{p}\left( p|x\right) >0$. Near local minima of $R(x)$, the left-hand side of (38) vanishes and the inequality is never satisfied (since $\partial _x^2 R >0$): Hence we expect $\mathbb {V}_{p}\left( p|x\right) \le 0$ near local minima (again, independently of the specific system under consideration).

In the extremal case, piecewise exponential amplitudes $R\left( x\right) =\exp (\pm kx)$ always give $\mathbb {V}_{p}\left( p|x\right) = 0$. These amplitudes are observed e.g. for particles tunnelling into a constant potential barrier.

Since the position-dependence of the amplitude $R(x)$ is easy to measure with projective position measurements alone, this inequality can serve as a test for the sign of $\mathbb {V}_{p}\left( p|x\right)$ even in an experiment not optimised to calculate the weak variance. We recommend that experimental probes into negative variances should occur near non-nodal minima of the probability density, since the convexity in other regions may not support negative weak variances.

Nodal Points

A slight complication arises around nodal points $R\rightarrow 0$ of the wavefunction: The weak variance at the node itself is infinite, given the logarithmic singularity at $\ln (0)$ that appears in (10).

To show this, we note that continuous differentiability of $\psi$ and $\psi ^{*}$ enforces around nodes. The expansion of the amplitude $R$ about a node at any fixed time is then

$$\begin{aligned} R(x) = ax^2+\mathcal {O}(x^3), \end{aligned}$$

where the condition $\nabla ^{2}R>0$ (that a node is a minimum of the amplitude) gives $a>0$. Around this node, the weak variance (10) tends to:

$$\begin{aligned} -\frac{\hbar ^2}{2}\partial _x^2\ln ax^2=-\hbar ^2 \partial _x^2\ln x =-\hbar ^2 \left( -\frac{1}{x^2}\right) \rightarrow +\infty . \end{aligned}$$

(39)

It is simple to see that this asymptotic result generalises even to pathological expansions $R=0+\cdots +0+\mathcal {O}(x^n)$. Interestingly, this behaviour for the weak variance is always strictly positive, excluding $\mathbb {V}_{p}\left( p|x\right) \le 0$ very close to the node. This will be illustrated in the following section in a number of systems.

Illustration in Toy Systems

The study of coherent states in the context of negativity of the Wigner function is motivated by the result [28] that only these states are nonnegative everywhere.

Consider the coherent states of the quantum harmonic oscillator. We must check (for consistency) that $\mathbb {V}_{p}\left( p|x\right) >0$ (strictly) in these states. The weak variance is

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =-\frac{\hbar ^{2}}{4}\partial _x^{2}\left[ \ln \left( \left( \frac{m\omega }{\hbar \pi }\right) ^{\frac{1}{4}}e^{-\frac{m\omega }{2\hbar }\left( x-\left\langle \hat{x}\left( t\right) \right\rangle \right) ^{2}}\right) ^{2}\right] , \end{aligned}$$

and, after many cancellations, one obtains the simple expression

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =+ m \frac{\hbar \omega }{2}. \end{aligned}$$

This is indeed positive for all $x$, as expected, and manifestly has the required dimensions for a momentum variance:

$$\begin{aligned} \left[ \mathrm {mass}\right] \times \left[ \mathrm {energy}\right] =\left[ \mathrm {momentum}\right] ^{2}. \end{aligned}$$

We note that $\mathbb {V}_{p}\left( p|x\right) /m$ is equal to the zero-point energy $\hbar \omega /2$ in this minimum-uncertainty state. In fact, for any energy eigenstate of the harmonic oscillator, we find a zero-point contribution to the variance:

$$\begin{aligned} \frac{\mathbb {V}_{p}\left( p|x\right) }{m}=+\frac{\hbar \omega }{2}-\frac{\hbar \omega }{2}\left[ \left( \frac{\partial }{\partial y}\right) ^{2}\ln H_{n}\left( y\right) \right] \, ,\; y=\sqrt{\frac{m\omega }{\hbar }}x, \end{aligned}$$

(40)

and this contribution remains even for arbitrary superpositions thereof.

For an energy eigenfunction of a particle in a box (respectively in a $-\alpha /\left| x\right|$ potential), we obtain

$$\begin{aligned} \frac{\mathbb {V}_{p}\left( p|x\right) }{m}= & {} + E_{n} \left[ \csc ^2(\frac{n\pi }{L}[x-x_{0}]) \right] \end{aligned}$$

(41)

$$\begin{aligned} (\text{ resp. })= & {} \frac{\hbar ^{2}}{2m}\frac{1}{x^{2}} + E_{n}\left[ \left( \frac{\partial }{\partial y}\right) ^{2} \ln L_{n-1}^{(1)}\left( y\right) \right] , \end{aligned}$$

(42)

in terms of the energy levels $E_{n}, n\ge 1$ of these two systems. The eigenstate in a box or a harmonic oscillator is clearly not in a coherent state; yet we find that $\mathbb {V}_{p}\left( p|x\right) > 0$ for all $x$ for these toy models, because each $x$ lies in the vicinity of a maximum or a nodal point.

We note in passing that the weak variance becomes positively infinite at the nodal points of these systems, as expected from the discussion in the previous section. More interestingly, the weak variance tends (at certain postselections $x$) to the zero-point energy when the system is in its ground state. We purposefully do not propose at this time an interpretation of this finding.

Negative Temperatures

If the weak variance is the temperature distribution (34) of the Madelung fluid, then the Madelung fluid can have negative temperatures. Negative temperatures may be surprising, although their study is theoretically well-established [29] and the concept has successfully been applied to several experimental systems exhibiting population inversion. Some interesting prerequisites for negative temperatures are that the internal energy of the system must be bounded from above as well as below, and that $T=0$ requires the energy to be extremised to one of these bounds. The ideal gas law $P=\rho k_B T$ shows that the pressure also becomes negative, precisely when the temperature is negative.

One further consequence of weak variances on the thermodynamics of the hydrodynamical formalism is interpretational. Recall that the hydrodynamic momentum $\tilde{p}=\mathbb {E}(p|x)$ is interpreted [18] as the local average momentum of the fluid particles at a position $x$. Yet, we have seen (Sect. 3.2) that negative weak variances do not provide a standard deviation or a Chebyschev inequality for the random variable $p|x$. In the hydrodynamic language, the peculiar momentum of a Madelung fluid particle is not required to be zero, even at zero temperature! Also note that the temperature of a nodal point in the fluid (where we expect to find no fluid particles) is $+\infty$.

These are the sort of features any “subquantum theory” [18] should exhibit, if such a theory admits the Madelung hydrodynamics as its thermodynamic limit.

Nondeterminism in Quantum Mechanics

We have already discussed (Sect. 3.2) how a null weak variance still allows stochasticity and nonzero standard deviations. Although $\mathbb {V}_{p}\left( p|x\right) =0$ does not imply determinism, one might nonetheless consider whether a relation between the weak variance and determinism exists. For example, the highly nonlinear form of the weak variance makes it unlikely that a null weak variance would emerge for superposition states (or, a fortiori, for entangled states). These conditions correspond exactly to the nonclassical regime, where we expect nondeterminism.

The Classical Limit

Consistency of the Classical Limit

It should not be surprising to find that these worrisome negative variances disappear in the classical limit. Heuristically considering the classical limit as the limit of infinitely thin coherent states similarly guarantees $\mathbb {V}_{p}\left( p|x\right) \ge 0$ since the conditional distribution is nonnegative [28].

However, a stronger statement is also true: weak variances should disappear completely in the classical limit, since classical observables are well-defined and motion is deterministic. We have even shown (Sect. 2.1.3) that there is no classical observable associated to it. The quantum potential behaves like $Q\rightarrow 0$ in the classical limit.¹ When $Q=0$, the Riccati equation (25) admits a trivial solution $\mathrm {Im}_{x}\hat{p}_{\psi } = 0$ since the weak momentum ($\in \mathbb {C}$) tends to the classical momentum ($\in \mathbb {R}$). Hence, using (24), $\mathbb {V}_{p}\left( p|x\right) \rightarrow 0$ classically.

Deriving the Classical Limit

Another approach to the classical limit would be to minimise the contribution of the scedastic function $\mathbb {V}_{p}\left( p|x\right)$ in the total variance (14) by our choice of wavefunction, in order to mimic determinism. Since the weak variance does not depend on the wavefunction phase, only the probability density $\rho = |\psi |^2$ is relevant. We can express this extremisation problem in the language of calculus of variations as

$$\begin{aligned} \delta J[\rho ]= & {} \delta \left( \mathbb {E}_{x}\left( \mathbb {V}_{p}\left( p|x\right) \right) \right) \nonumber \\= & {} \delta \int dx \rho (x) \; \partial _x^2 \log (\rho (x)) \nonumber \\= & {} \delta \int dx \left[ \rho ^{\prime \prime } - \frac{(\rho ^\prime )^2}{\rho }\right] \nonumber \\= & {} \int dx \; \delta \mathcal {L}(\rho ,\rho ^\prime ,\rho ^{\prime \prime }) \nonumber \\= & {} 0, \end{aligned}$$

(43)

where unimportant constant factors are suppressed. The resulting Euler–Lagrange equation depends (albeit trivially given our specific $\mathcal {L}$) on the second derivative $\rho ^{\prime \prime } = \partial _x^2 \rho$ of the probability density:

$$\begin{aligned} \frac{\partial \mathcal {L}}{\partial \rho } - \frac{\partial }{\partial x} \frac{\partial \mathcal {L}}{\partial \rho ^\prime } + \frac{\partial ^2}{\partial x^2} \frac{\partial \mathcal {L}}{\partial \rho ^{\prime \prime }} = \frac{\rho ^{\prime \prime }}{\rho } - \frac{1}{2} \left( \frac{\rho ^\prime }{\rho }\right) ^2= 0. \end{aligned}$$

(44)

We notice that the terms of (23) are similar to those of the Euler–Lagrange equation (44). Consequently we find that in this limit, a term of our choice cancels out of (23) to give

$$\begin{aligned} \mathbb {V}_{p}\left( p|x\right) =\left( \mathrm {Im}\left[ _{x}\hat{p}_{\psi }\right] \right) ^2 = -mQ. \end{aligned}$$

However, reinserting this into (24) results in $\mathbb {V}_{p}\left( p|x\right) =+mQ$; this is uniquely possible for $\mathbb {V}_{p}\left( p|x\right) =Q=\mathrm {Im}\left[ _{x}\hat{p}_{\psi }\right] =0$. We have therefore derived that the minimum average contribution of the weak variance to the total variance is zero: negative weak variances do not allow the nonweak variance $\mathbb {V}_x(\tilde{p})$ to exceed the total variance in (14).

In summary, by minimising the stochastic part of the total variance we can derive the classical limit $Q\rightarrow 0$ of the Hamilton–Jacobi formalism.

$\mathbb {V}_{p}\left( p|x\right) \rightarrow 0$ as a Semiclassical Limit

Equation (21) states that $\mathbb {V}_{p}\left( p|x\right)$ is the divergence of the imaginary part of the weak value: when this variance is zero, $\mathrm {Im}({_{x}\hat{p}_{\psi }})$ is a function of time only. In this case, either (24) or the Ricatti equation (25), gives that $Q(x,t)=Q(t\,\mathrm {only}) < 0$. Hence the quantum force $\partial Q /\partial x = 0$: this is [23] the condition that Bohmian trajectories coincide with their classical counterparts. $\mathbb {V}_{p}\left( p|x\right) \rightarrow 0$ therefore yields classical particle dynamics (on average), occurring at nonclassical energies. Of course, these semiclassical dynamics are localised to specific postselections $x$: if $\mathbb {V}_{p}\left( p|x\right) =0,\; \forall x$ then the average weak variance is also zero and we recover the classical limit as derived above.

Consider a particle tunnelling through a constant barrier: The exponential wavefunction amplitude guarantees that $\mathbb {V}_{p}\left( p|x\right) = 0$ inside the barrier, therefore the tunnelling particle lives in this semiclassical limit and will (on average) follow the path of a classical particle with a higher energy: it will go over the barrier. The fact that tunnelling particles behave semiclassically is indeed why tunnelling problems are successfully visualised and formulated in terms of particles.

Weak Variances Do Not ‘Measure’ a Deterministic Particle Ontology

It is sometimes claimed that the weak value $\mathrm {Re}({_{x}\hat{p}_{\psi }})$ ‘measures’ the Bohmian momentum [15, 27]. A naive Bohmian might expect that the weak variance would vanish, being the variance of the nonstochastic Bohmian momenta. In this section we argue against one possible misinterpretation of these quantities: weak values (and their associated weak variances) do not measure the properties of a Bohmian particle.

The only hidden variable of a Bohmian particle is position—it does not even have an energy or even a momentum independently of what the mathematical formalism predicts we should measure [22, 30]. This is of course a reflection of the fact that nonposition measurements in Bohmian mechanics are contextual [22]. As such, a measurement of the weak value, although equal to the Bohmian momentum $\partial _x \mathcal {S}$, is not a measurement of the “particle’s momentum”; similarly, the weak variance does not measure the standard deviation of Bohmian particles’ momentum (both on interpretational and quasistatistical grounds).

With the above caveat in mind, the weak variance allows a deterministic particle interpretation of the ‘semiclassical’ limit $\mathbb {V}_{p}\left( p|x\right) \rightarrow 0$: such a (localised) Bohmian interpretation has already been shown to apply at postselection points $x$ where it is at its strongest (particles tunnelling and in the classical limit), and is guaranteed to not apply where the global Bohmian interpretation is known to break down [22, 23, 31, 32] (since $\mathbb {V}_{p}\left( p|x\right) \rightarrow +\infty \ne 0$ at nodal points). We emphasise that a semiclassically emergent Bohmian ontology is neither supported nor precluded by the absence of a sensible notion of standard deviation for quasidistributions, nor can it be of any instrumental relevance.

Conclusions

The weak variance can be defined naturally and uniquely in terms of the Wigner function, singled out by insisting on a (quasi)statistical interpretation of quantum phase space. A weak variance does not define the standard deviation of individual realisations of $p|x$, which may occur with negative probability, but instead describes the system’s state.

The weak variance $\mathbb {V}_{p}\left( p|x\right)$ contributes to the momentum variance of the uncertainty principle, justifies the ‘ideal gas’ equation of state of the hydrodynamical formalism, and appears to be related to the ground state energy of a few analytically soluble systems. An expression $\mathrm {Re}({_{x}\hat{p}^2_{\psi }}-{_{x}\hat{p}_{\psi }}^2)$ in terms of weak values was derived, which we used to argue that the weak variance is not itself a weak value.

We show that the sign of the weak variance is predetermined for local extrema of the probability density: $\mathbb {V}_{p}\left( p|x\right)$ is positive for nodal points and maxima, and negative for non-nodal extrema. We also derive some features that any hypothetical “subquantum theory” (from which quantum mechanics emerges as a thermodynamic limit) must satisfy.

We show that the weak variance is not only consistent with the classical limit, but can be used to derive the classical limit from a variational principle where the expectation of the weak variance controls the emergence of determinism. This construction also shows that the variability $\mathbb {V}_x(\tilde{p})$ of the weak value cannot exceed the total variance of the state. A semiclassical limit is derived and discussed.

Most importantly, we note that $\mathbb {V}_{p}\left( p|x\right)$ should be measurable by a number of experimental setups (Sect. 2.3).

Appendix: Derivation

The characteristic function of a distribution is its Fourier transform. For the Wigner function we can write the conditional characteristic function [8, 17] as follows:

$$\begin{aligned} M(\tau |x) = \int dp \mathcal {W}(p|x) e^{i\tau p / \hbar } = \frac{ \psi ^*(x-\frac{\hbar }{2} \tau ) \psi (x+\frac{\hbar }{2} \tau )}{\psi ^*\psi (x)}. \end{aligned}$$

(45)

The cumulant-generating function is the logarithm of the characteristic function. This gives

$$\begin{aligned} \ln (M(\tau |x))= & {} \ln \left( \frac{ \psi ^*(x^-) \psi (x^+)}{\rho (x)}\right) = \ln (\psi ^*(x^-)) + \ln (\psi (x^+)) - \ln (\rho (x)) \end{aligned}$$

(46)

$$\begin{aligned}= & {} \frac{1}{2}\left[ \ln (\rho (x^-)) + \ln (\rho (x^+))\right] + i\left[ S(x^+) - S(x^-)\right] - \ln (\rho (x)), \qquad \quad \end{aligned}$$

(47)

which we can then Taylor expand to obtain the cumulants [8].

By inspecting (10), we expect final factors of $(\hbar /2i)^n$ and an expansion in $(i\tau )^n/n!$, so we consider series of the form:

$$\begin{aligned} f(x \pm \frac{\hbar }{2i} (i\tau )) = \sum _{n=0}^\infty \left[ \left( \pm \frac{\hbar }{2i}\right) ^n (\partial ^n f)(x) \right] \frac{(i\tau )^n}{n!}, \end{aligned}$$

where the $\left( \hbar /2i\right) ^n$ come from the chain rule.

From the $i(S^+-S^-)$ term we obtain a contribution to cumulants of

$$\begin{aligned} i \left[ \left( +\frac{\hbar }{2i}\right) ^n - \left( -\frac{\hbar }{2i}\right) ^n \right] \partial ^n S = {\left\{ \begin{array}{ll} \left( \frac{\hbar }{2i} \right) ^{n-1} \partial ^n (\hbar S) &{}\text{ if } n \text{ odd } \\ 0 &{}\text{ if } n \text{ even } \end{array}\right. } \end{aligned}$$

and from the $\ln \rho$ terms we obtain:

$$\begin{aligned} \left[ \frac{1}{2} \left( +\frac{\hbar }{2i}\right) ^n + \frac{1}{2}\left( -\frac{\hbar }{2i}\right) ^n - \delta _{0n} \right] \partial ^n \ln \rho = {\left\{ \begin{array}{ll} 0 &{}\text{ if } n = 0 \\ \left( \frac{\hbar }{2i}\right) ^n \partial ^n \ln \rho &{}\text{ if } n \text{ even } \\ 0 &{}\text{ if } n \text{ odd } \end{array}\right. } \end{aligned}$$