Retrodiction beyond the Heisenberg uncertainty relation

Abstract

In quantum mechanics, the Heisenberg uncertainty relation presents an ultimate limit to the precision by which one can predict the outcome of position and momentum measurements on a particle. Heisenberg explicitly stated this relation for the prediction of “hypothetical future measurements”, and it does not describe the situation where knowledge is available about the system both earlier and later than the time of the measurement. Here, we study what happens under such circumstances with an atomic ensemble containing 10¹¹ rubidium atoms, initiated nearly in the ground state in the presence of a magnetic field. The collective spin observables of the atoms are then well described by canonical position and momentum observables, ${\hat{x}}_{\text{A}}$ and ${\hat{p}}_{\text{A}}$ that satisfy $[{\hat{x}}_{\text{A}},{\hat{p}}_{\text{A}}]=i\hslash$. Quantum non-demolition measurements of ${\hat{p}}_{\text{A}}$ before and of ${\hat{x}}_{\text{A}}$ after time t allow precise estimates of both observables at time t. By means of the past quantum state formalism, we demonstrate that outcomes of measurements of both the ${\hat{x}}_{\text{A}}$ and ${\hat{p}}_{A}$ observables can be inferred with errors below the standard quantum limit. The capability of assigning precise values to multiple observables and to observe their variation during physical processes may have implications in quantum state estimation and sensing.

Introduction

Heisenberg’s uncertainty relation (HUR)¹ is one of the pillars of quantum mechanics and it sets the limit of how precisely one can predict the outcome of the measurements of two non-commuting observables. While the relation itself is a simple consequence of Born’s rule and the operator character of physical observables, it has spurred both foundational discussions of the interpretation of quantum theory and efforts to identify and surpass quantum limits for practical high precision measurements. The HUR deals with the ability to predict the outcomes of measurements of either one of two observables in different experiments with the same quantum state. As a related concept in quantum metrology, the so-called standard quantum limit (SQL) denotes the measurement precision achievable with conventional resources, such as coherent states of light and product states of many particles. Poissonian counting statistics and field amplitude measurements with equal size errors on all quadratures are examples of the SQL, while number states and squeezed states permit precision measurements of a single, relevant observable below the SQL².

Beyond the conventional HUR scenario concerned with measurements of only one observable in each experiment, more complex scenarios have been pursued, where more observables are, simultaneously or sequentially, measured in the same experiment. Measurements generally disturb a quantum system^3,4 and several approaches have been proposed to mitigate effects of this disturbance in high precision measurements of one or more observables over time. These include entanglement based “negative mass” or “quantum-mechanics-free” subsystem measurements^5,6,7,8,9 and quantum dense metrology based on an Einstein–Podolsky–Rosen entangled two-mode system¹⁰. The present work is not aimed at such scenarios but serves to illustrate how the HUR does not account for our ability to retrodict the outcome of a measurement on a system at a past time t, if we have access to the system after the measurement¹¹. Consider for example the particularly simple situation where prior to the measurement at time t the system was prepared in an eigenstate $\left|{a}_{m}\right\rangle$ of an observable $\hat{A}$, while a projective measurement of another observable $\hat{B}$ is applied right after t, yielding the eigenvalue b_n. Clearly, under this circumstance, the outcome of a hypothetical measurement of $\hat{A}$ at time t would have to be a_m, while the outcome of a measurement of $\hat{B}$ would have had to be b_n, in order to be consistent with the subsequent projective measurement. Either outcome can thus be stated with certainty even though $\hat{A}$ and $\hat{B}$ do not commute. Separate formalism has been developed to describe the sometimes paradoxical situations occurring in weak and strong measurement scenarios with prior and posterior measurement information^11,12,13.

Recently, quantum trajectory theory where the density matrix ρ(t) is conditioned on the dynamics and probing of the system until time t, was supplemented by an effect matrix E(t) conditioned on the dynamics and probing of the system in a subsequent time interval [t, T]^14,15. At the final time T, the matrices ρ(t) and E(t) together incorporate all our information about the “past quantum state”, i.e., they yield the probability for the outcome of any general measurement which could have been performed in the laboratory at the earlier time t. Analogous to the forward-backward formalism of hidden Markov models and the similar smoothing procedures in Kalman filtering theory, the past quantum state formalism provides an improved estimate of the system dynamics and allows better estimation of physical parameters and of time-dependent perturbations on the system. This has been demonstrated by the observed conditional dynamics of the photon number evolution in a cavity¹⁶, the excitation and emission dynamics of a superconducting qubit¹⁷ and the motional state of a mechanical oscillator¹⁸. For an alternative application of prior and posterior measurements, see also the definition of quantum smoothing in^19,20. However, experimental studies of such schemes have so far only addressed retrodiction of a single observable, and hence the prospects of “violating” the HUR have not yet been demonstrated.

Here, we demonstrate theoretically and experimentally that the past quantum state formalism “violates” the HUR for a spin oscillator. It is shown that quantum non-demolition (QND) measurements in the earlier and later time intervals [0, t₋] and [t₊, T] provide better estimates of the outcome of measurements at time t of either of the observables than could be inferred from the HUR. Our experiment addresses the collective atomic spin of a ⁸⁷Rb ensemble of 10¹¹ atoms contained in a macroscopic vapor cell. Previously, we used a similar setup to demonstrate measurement-based spin squeezing for a single observable ${\hat{p}}_{\text{A}}$ and showed that subsequent probing of the same observable improved the ability to guess the outcome of measurements of that quantity further below the SQL²¹. The improved estimation of the expected experimental outcome was then exploited to demonstrate RF magnetic field sensitivity better than the SQL²¹. In the experiment reported in the present article, we develop a four-pulse sequence of quantum non-demolition (QND) measurements of different observables, that allows retrodiction of spin quadratures along any direction, and we show that the ability to guess the outcome of a past position and momentum measurement is not generally limited by any HUR. We can, indeed, infer the outcomes of measurements of both the ${\hat{x}}_{\text{A}}$ and ${\hat{p}}_{\text{A}}$ observables with errors below the SQL. This protocol holds the potential for estimation of perturbations causing displacements along any directions in phase space, without change of the preparation and post-selection steps. In future applications, these may be used in magnetometers equally capable of measuring the amplitude and phase of an RF magnetic field below the SQL.

Results

Atom-light Faraday interaction

Consider the collective atomic spin ${\hat{J}}_{i}=\mathop{\sum }\nolimits_{k = 1}^{{N}_{\text{at}}}{\hat{j}}_{i}^{k}$, with i = x, y, z, given by the sum of the total angular momenta ${\hat{j}}_{i}^{k}$ of individual atoms. The macroscopic spin orientation J_x is along the applied bias magnetic field B, and the perpendicular collective spin components ${\hat{J}}_{\text{y,z}}$ oscillate in the lab frame at the Larmor frequency Ω_L. We denote $\left(\begin{array}{l}{\hat{J}}_{\text{y0}}\\ {\hat{J}}_{\text{z0}}\end{array}\right)=\left(\begin{array}{ll}\cos {\Omega }_{\text{L}}t&\sin {\Omega }_{\text{L}}t\\ -\sin {\Omega }_{\text{L}}t&\cos {\Omega }_{\text{L}}t\end{array}\right)\left(\begin{array}{l}{\hat{J}}_{\text{y}}\\ {\hat{J}}_{\text{z}}\end{array}\right)$ as the spin in the rotating frame. Assuming a highly oriented spin state, the Holstein-Primakoff transformation maps the perpendicular spin operators in the rotating frame to the oscillator quadrature operators ${\hat{x}}_{\text{A}}={\hat{J}}_{\text{y0}}/\sqrt{\left|\left\langle {J}_{\text{x}}\right\rangle \right|}$ and ${\hat{p}}_{\text{A}}={\hat{J}}_{\text{z0}}/\sqrt{\left|\left\langle {J}_{\text{x}}\right\rangle \right|}$. The spin commutator $[{{\hat{J}}}_{\text{y0}},{\hat{J}}_{\text{z0}}]=i{J}_{\text{x}}(\hslash =1)$, leads to the HUR, $\Delta {\hat{x}}_{\text{A}}\cdot \Delta {\hat{p}}_{\text{A}}\ge 1/2$. The ground state of the harmonic oscillator corresponds to all atoms being in the $|5{S}_{1/2},F=2,{m}_{F}=-2\rangle$ state, forming the Coherent Spin States (CSS) characterized by $\,{\text{Var}}\,({\hat{J}}_{\text{y0}})=\,{\text{Var}}\,({\hat{J}}_{\text{z0}})={J}_{\text{x}}/2={N}_{\text{at}}F/2$ and $\Delta {\hat{x}}_{\text{A}}\cdot \Delta {\hat{p}}_{\text{A}}=1/2$. The first excited state of the oscillator corresponds to a symmetric superposition state of the ensemble with one atom in the state $|5{S}_{1/2},F=2,{m}_{F}=-1\rangle$^22,23.

In the limit of large probe detuning with respect to the atomic excited-state hyperfine level^22,23, the spin observable ${\hat{J}}_{\text{z}}$ is coupled to the Stokes operators ${\hat{S}}_{\text{z}}$ of an optical probe pulse with N_ph photons and duration τ via the far off-resonance Faraday interaction ${\hat{H}}_{\text{int}}=(\sqrt{2}\kappa /\sqrt{{N}_{\text{ph}}{N}_{\text{at}}}){\hat{J}}_{\text{z}}{\hat{S}}_{\text{z}}=\frac{\kappa }{\tau }{\hat{p}}_{\text{A}}{\hat{p}}_{\text{L}}$, permitting the QND measurement of ${\hat{J}}_{\text{z}}$^{22,24,25,26,27}. Here canonical operators of light are defined as ${\hat{x}}_{\text{L}}={\hat{S}}_{\text{y}}/\sqrt{\left|\left\langle {S}_{\text{x}}\right\rangle \right|}$ and ${\hat{p}}_{\text{L}}={\hat{S}}_{\text{z}}/\sqrt{\left|\left\langle {S}_{\text{x}}\right\rangle \right|}$. The coupling constant κ² ∝ N_phN_at characterizes the strength of the atom-light interaction.

Conditioned dynamics and the past quantum state

The unitary evolution operator of the QND interaction can be written as

$$\hat{U}={e}^{-i{\hat{H}}_{\text{int}}\tau }={e}^{-i\kappa {\hat{p}}_{\text{A}}\otimes {\hat{p}}_{\text{L}}}.$$

(1)

As explained below, the QND measurement of ${\hat{p}}_{\text{A}}$ can be generalized to that of a spin component ${\hat{x}}_{\text{A}}(\theta )={\hat{p}}_{\text{A}}\cos \theta +{\hat{x}}_{\text{A}}\sin \theta$ with an arbitrary direction θ in the oscillator phase space. After the interaction, the field quadrature ${\hat{x}}_{\text{L}}$ is measured, which amounts to an indirect, noisy measurement of ${\hat{x}}_{\text{A}}(\theta )$. Thus, if the field outcome is m, the atomic state is transformed by the operator,

$${\hat{\Omega }}_{\text{m}}=\int{\psi }_{{\hat{x}}_{\text{L}}}(m-\kappa a)\left|a,\theta \right\rangle \left\langle a,\theta \right|{\rm{d}}a$$

(2)

where $\left|a,\theta \right\rangle$ is the eigenstate of ${\hat{x}}_{\text{A}}(\theta )$ with eigenvalue a, while ${\psi }_{{\hat{x}}_{\text{L}}}(m)=\frac{1}{{\pi }^{1/4}}\exp (-\frac{{m}^{2}}{2})$ characterizes the quadrature distribution of the input coherent state of the probe laser beam. The operators $\{{\hat{\Omega }}_{\text{m}}\}$ specify a positive operator valued measurement (POVM)²⁸). For infinite κ, ${\hat{\Omega }}_{\text{m}}$ converges to a projective measurement ${\hat{\Omega }}_{a}=\left|a,\theta \right\rangle \left\langle a,\theta \right|$, which projects the atomic state on the eigenstate $\left|a,\theta \right\rangle$ with a = m/κ.

Conditioned upon the output value m₁ of the first measurement, the atomic oscillator is described by the unnormalized density matrix $\rho ={\Omega }_{{m}_{1}}{\rho }_{0}{\Omega }_{{m}_{1}}^{\dagger }$, with the probability distribution Pr$(a| {m}_{1})\propto {\rm{Tr}}({\hat{\Omega }}_{a}\rho {\hat{\Omega }}_{a}^{\dagger })=\left\langle a,\theta \right|\rho \left|a,\theta \right\rangle$ for a subsequent projective measurement of the atomic observable ${\hat{x}}_{\text{A}}(\theta )$. The distribution is Gaussian and we denote its expectation value and variance by μ_ρ(θ) and ${\sigma }_{\rho }^{2}(\theta )$. The conditional variance does not depend on the outcome of the first measurement.

In the experiment, however, we are restricted to optical measurements employing the finite Faraday interaction described by the POVM ${\hat{\Omega }}_{{m}_{2}}=\int{\psi }_{{\hat{x}}_{\text{L}}}({m}_{2}-{\kappa }_{2}a)\left|a,\theta \right\rangle \left\langle a,\theta \right|{\rm{d}}a$, and the corresponding distribution Pr$({m}_{2}| {m}_{1})\propto {\rm{Tr}}({\hat{\Omega }}_{{m}_{2}}\rho {\hat{\Omega }}_{{m}_{2}}^{\dagger })$. This interaction imprints the atomic observable onto the light observable which acquires the expectation value 〈m₂〉 = κ₂μ_ρ(θ), while shot noise fluctuations contribute to the variance of the optical measurement, ${\rm{Var}}({m}_{2}| {m}_{1})={\kappa }_{2}^{2}{\sigma }_{\rho }^{2}(\theta )+\frac{1}{2}$ (see Supplementary Note 3). It is customary to regard the optical measurement as a noisy measurement of the atomic observable and thus infer its variance by the relation, ${\sigma }_{\rho }^{2}(\theta )=({\rm{Var}}({m}_{2}| {m}_{1})-\frac{1}{2})/{\kappa }_{2}^{2}$.

So far, our discussion was concerned with the usual application of the conditioned quantum state to determine the uncertainty of the outcome of projective and general measurements. Now, we turn to the case where such outcomes are retrodicted by the combination of a prior measurement of ${\hat{p}}_{\text{A}}$ with measurement strength κ₁ and posterior measurements of ${\hat{x}}_{\text{A}}$ and ${\hat{p}}_{\text{A}}$ with measurement strengths κ₃ and κ₄, respectively. The reason we implement the fourth measurement is explained in Supplementary Note 4. If we consider a projective second measurement, represented by ${\hat{\Omega }}_{a}$, the joint probability of all four measurement outcomes is ${\rm{Tr}}({\hat{\Omega }}_{{m}_{4}}{\hat{\Omega }}_{{m}_{3}}{\hat{\Omega }}_{a}{\hat{\Omega }}_{{m}_{1}}{\rho }_{0}{\hat{\Omega }}_{{m}_{1}}^{\dagger }{\hat{\Omega }}_{a}^{\dagger }{\hat{\Omega }}_{{m}_{3}}^{\dagger }{\hat{\Omega }}_{{m}_{4}}^{\dagger })$. Fixing the arguments of this expression by the known values of m₁, m₃ and m₄, this yields the conditional probability $\,\text{Pr}\,(a| {m}_{1},{m}_{3},{m}_{4})\propto {\rm{Tr}}({\hat{\Omega }}_{a}\rho {\hat{\Omega }}_{a}^{\dagger }E)$, where the effect matrix E is defined as $E={\hat{\Omega }}_{{m}_{3}}^{\dagger }{\hat{\Omega }}_{{m}_{4}}^{\dagger }{\hat{\Omega }}_{{m}_{4}}{\hat{\Omega }}_{{m}_{3}}$. It follows that $\,\text{Pr}\,(a| {m}_{1},{m}_{3},{m}_{4})\propto \left\langle a,\theta \right|\rho \left|a,\theta \right\rangle \left\langle a,\theta \right|E\left|a,\theta \right\rangle$, and it is easy to show that by our assumptions, $\left\langle a,\theta \right|E\left|a,\theta \right\rangle$ is a Gaussian function. We denote the centroid and variance of this distribution function by μ_E(θ) and ${\sigma }_{\,\text{E}\,}^{2}(\theta )$ (see Supplementary Note 3).

We thus assign the probability distribution Pr(a∣m₁, m₃, m₄) to the outcome of a past projective atomic measurement, and we readily evaluate its expectation value ${\mu }_{\rho \text{E}}(\theta )=\frac{{\mu }_{\rho }(\theta ){\sigma }_{\,\text{E}\,}^{2}(\theta )\,+\,{\mu }_{\text{E}}(\theta ){\sigma }_{\rho }^{2}(\theta )}{{\sigma }_{\rho }^{2}(\theta )\,+\,{\sigma }_{\,\text{E}\,}^{2}(\theta )}$ and variance ${\rm{Var}}(a| {m}_{1},{m}_{3},{m}_{4})={\sigma }_{\rho \,\text{E}\,}^{2}(\theta )=\frac{1}{1/{\sigma }_{\rho }^{2}(\theta )+1/{\sigma }_{\,\text{E}\,}^{2}(\theta )}$. Polar plots for ${\sigma }_{\rho }^{2}(\theta )$ and ${\sigma }_{\rho \,\text{E}\,}^{2}(\theta )$ based on analytical expressions in Supplementary Note 3 are shown as blue and red curves in Fig. 1 (c). As ${\sigma }_{\rho }^{2}(\theta =0)$ and ${\sigma }_{\,\text{E}\,}^{2}(\theta =\pi /2)$ may be independently reduced well below 1/2, ${\sigma }_{\rho \,\text{E}\,}^{2}(\theta )$ may expose squeezing of both ${\hat{p}}_{\text{A}}$ and ${\hat{x}}_{\text{A}}$.

Fig. 1: Schematics for retrodiction beyond the Heisenberg uncertainty relation.

a Experiment schematics. A paraffin coated 20mm × 7mm × 7mm rectangular vapor cell at 53 °C resides inside a 4-layer magnetic shielding. A coherent spin state (CSS) is created along the quantization axis x by optical pumping, with a pump laser tuned to the Rb D1 transition $5{S}_{1/2},F=2\to 5{P}_{1/2},F^{\prime} =2$ and a repump laser beam stabilized to the Rb D2 transition $5{S}_{1/2},F=1\to 5{P}_{3/2},F^{\prime} =2$, sharing the same circular polarization. A magnetic field of 0.71G induces a ground-state Zeeman splitting of about Ω_L = 2π × 500 kHz and stabilizes the collective spin component along the x direction. A linearly polarized off-resonance D2 laser beam, propagating in the z direction, probes the quantum fluctuations of the spin. The Stokes component S_y that characterizes the linear polarization in the  ± 45^∘ basis is measured using a balanced polarimetry scheme. A Lock-in amplifier (LIA) extracts the signal at the Lamor frequency. b Pulse sequence. The pump lasers prepare the atoms in the CSS. The pump lasers are turned off and the probe laser is turned on to interact with the atoms. The probe pulse sequence is divided in four parts, measuring the quadratures indicated in the figure. c Theoretical polar plot of the variance of the atomic oscillator quadratures conditioned on prior (blue curve) and on prior and posterior measurements (red curve). The plot assumes ideal experiments without decoherence and decay, probing strengths ${\kappa }_{1}^{2}=1.7$, ${\kappa }_{3}^{2}=3.3$ and ${\kappa }_{4}^{2}=2.2$ and a duty factor of 14%, for details see Supplementary Note 4. The radial distance of 0.5 in the polar plot represents the SQL, and the retrodicted variances of the horizontal and vertical ${\hat{p}}_{\text{A}}$ and ${\hat{x}}_{\text{A}}$ quadratures are both reduced in violation of the HUR.

While the theory thus shows that the HUR does not apply for retrodiction of projective measurements, we recall that our experiments are based on optical probing, and for a comparison between theory and experiment, we must address the predictions for the POVM ${\hat{\Omega }}_{{m}_{2}}$, conditioned on m₁, m₃, and m₄. They read (See Supplementary Note 3),

$$\,{\text{Pr}}\,({m}_{2}| {m}_{1},{m}_{3},{m}_{4}) \propto \int\int{\psi }_{{\hat{x}}_{\text{L}}}({m}_{2}-{\kappa }_{2}a){\psi }_{{\hat{x}}_{\text{L}}}({m}_{2}-{\kappa }_{2}a^{\prime} )\\ \cdot \left\langle a,\theta \right|\rho \left|a^{\prime} ,\theta \right\rangle \left\langle a^{\prime} ,\theta \right|E\left|a,\theta \right\rangle {\rm{d}}a{\rm{d}}a^{\prime} .$$

(3)

and we can show (See Supplementary Note 3) that for the special cases of $\theta =0,\frac{\pi }{2}$, we recover the same simple relation between the variances of the atomic and the optical measurements, as we found for the prediction based on the density matrix ρ,

$${\rm{Var}}({m}_{2}| {m}_{1},{m}_{3},{m}_{4})={\kappa }_{2}^{2}{\sigma }_{\rho E}^{2}(\theta )+\frac{1}{2},\,\left(\theta =0,\frac{\pi }{2}\right).$$

(4)

The reduced fluctuations of the optical measurements around their retrodicted mean value thus constitute a test of the past quantum state theory, and if the uncertainty product $\Delta {\hat{x}}_{\text{A}}\cdot \Delta {\hat{p}}_{\text{A}}$ inferred from the optical measurements and Eq. (4) is less than 1/2, it may be taken as a demonstration of the violation of the HUR.

Experimental realization of retrodiction beyond HUR

The core of the experiment is a paraffin coated vapor cell²⁹ containing about 10¹¹ atoms of ⁸⁷Rb, as sketched in Fig. 1(a). The anti-relaxation coating on the inner wall of the cell ensures a relatively long spin coherence lifetime. We initially populate the atoms in the state $|5{S}_{1/2},F=2,{m}_{\text{F}}=-2\rangle$ by optical pumping along the x-direction parallel to the magnetic field B with up to 97.9% polarization, orienting the atoms along the x-direction. It leads to a 6% increase in the measured variance compared to the fully polarized CSS. The projection noise limit is calibrated by measuring the noise of the collective spin of the unpolarized sample, which is 1.25 times that of the ideal CSS state (see Methods).

The linear-polarized probe light propagating along the z-direction is turned on after the optical pumping, to measure quantum spin fluctuations in the transverse y-z plane through Faraday rotation. The intensity of the probe beam is stroboscopically modulated at twice the Larmor frequency in order to probe a spin component ${\hat{x}}_{\text{A}}(\theta )$^21,22. The measurement direction θ is tuned by adjusting the phase of stroboscopic modulation. The phase is experimentally calibrated by radio frequency signal excitation (See Methods). As shown in Fig. 1(b), the probe pulse sequence is divided into four parts. The 1st pulse measures ${\hat{p}}_{\text{A}}$, representing the information of ρ. The 3rd and 4th pulse measure ${\hat{x}}_{\text{A}}$ and ${\hat{p}}_{\text{A}}$ respectively, representing the information of E. ρ and E together provide an estimation of the result at the time of 2nd pulse. In each experimental repetition, we can only obtain the result for one value of θ. Hence, the QND measurement of ${\hat{x}}_{\text{A}}(\theta )$ for different θ are obtained from independent experiments.

While Eq. (3) permits evaluation of the conditional variance of the m₂ measurement, it does not take decoherence, losses, and experimental imperfections into account. In the following, we shall present our bare experimental results based exclusively on analyses of correlations in the measurement data, cf. Eqs. (10), (11) in Methods. These analyses only assume Gaussian correlations between the measurement outcomes, which is compatible with realistic errors and decoherence mechanisms in the system, and they assume that Eq. (4) can be applied to infer the atomic variances from the optical measurement.

Figure 2 depicts the value of the uncertainty product $\Delta {\hat{x}}_{\text{A}}\cdot \Delta {\hat{p}}_{\text{A}}$ as a function of both measurement durations τ₁ and τ₃. The atomic variances are inferred from the variances of the optical measurements and the simple scaling (4) applicable for θ = 0, π/2. We find that the minimum value of $\Delta {\hat{x}}_{\text{A}}\cdot \Delta {\hat{p}}_{\text{A}}=1/2\times (0.680\pm 0.019)$ is smaller than the Heisenberg uncertainty limit of 1/2, and that it is obtained for τ₁ = 1 ms and τ₃ = 2 ms. Fig. 2 shows that $\Delta {\hat{x}}_{\text{A}}\cdot \Delta {\hat{p}}_{\text{A}}$ first decreases with the probing time τ₁ and corresponding measurement strength κ₁, while extending τ₁ beyond 1 ms causes incoherent scattering by spontaneous emission of the atoms, which reduces the effective mean spin projection, breaks the correlations between pairs of spins responsible for the squeezing, and adds random fluctuations to the collective ground state spin^30,31. The later measurements of duration τ₃ and τ₄ do not decohere the spin state at the earlier time t, but we observe a slow rise in the uncertainty product after an optimal probe duration τ₃ in panel (b) of Fig. 2. The optimal probe duration is longer and the increase in uncertainty is slower than for τ₁, which may be due to the reduction of the length of the mean spin, appearing in the definition of ${\hat{x}}_{\text{A}}$ and ${\hat{p}}_{\text{A}}$ and hence in the effective coupling strength of the field and atomic oscillator degrees of freedom. A too long τ₃ depolarizes the spins and hence reduces the efficiency of the final τ₄ measurement which plays an important role for the retrodicted variance (see Supplementary Note 4).

Fig. 2: Experimental results demonstrating “violation” of the Heisenberg uncertainty relation.

Value of $\Delta {\hat{x}}_{\text{A}}\cdot \Delta {\hat{p}}_{\text{A}}$ as a function of (a) τ₁ and (b) τ₃ are shown respectively. The plotted values are inferred by (4) from the correlations between the optical Faraday rotation measurements. The Heisenberg uncertainty relation is represented by the value 1/2. In (a) τ₃ = 2 ms, and in (b) τ₁ = 1 ms, and τ₄ = 1ms for both figures. The insets show the Wineland squeezing parameter $10{\text{log}}_{10}({\xi }_{\,\text{R}\,}^{2})$ in dB units for ${\hat{x}}_{\,\text{A}\,}^{2}$ and ${\hat{p}}_{\,\text{A}\,}^{2}$ (See Supplementary Note 7). The error bars (one standard deviation) are derived from 5 identical experiments for $\Delta {\hat{x}}_{\text{A}}$ and $\Delta {\hat{p}}_{\text{A}}$ respectively, each consisting of 10,000 repetitions of the corresponding pulse sequences.

The outcome of the Faraday rotation measurements by coupling to atomic oscillator quadratures in arbitrary directions are shown by polar plots in Fig. 3. The blue curve shows the variance conditioned on the prior measurements (Eq. (10) in Methods). The result is scaled to atomic units, i.e., we plot $({\rm{Var}}({m}_{2}| {m}_{1})-\frac{1}{2})/{\kappa }_{2}^{2}$, and we observe that the variance of the ${\hat{p}}_{\text{A}}$ quadrature (horizontal direction in the polar plot) is a factor 0.80 ± 0.05 below the SQL, and the anti-squeezed ${\hat{x}}_{\text{A}}$ quadrature is noisier than the SQL (vertical direction in the polar plot). When we implement also the subsequent m₃ and m₄ measurements (Eq. (11) in Methods), the corresponding $({\rm{Var}}({m}_{2}| {m}_{1},{m}_{3},{m}_{4})-\frac{1}{2})/{\kappa }_{2}^{2}$, shown as the red curve in Fig. 3a shows that ${\hat{x}}_{\text{A}}$ is now subject to a similar squeezing effect as observed for ${\hat{p}}_{\text{A}}$. The degree of squeezing is less than theoretically predicted for the ideal experiment in Fig. 1(c) because decay and decoherence in the experiment is not considered in the simple theory.

Fig. 3: Polar plot of the experimentally observed (and rescaled) variances.

a Measured $({\rm{Var}}({m}_{2}| {m}_{1})-\frac{1}{2})/{\kappa }_{2}^{2}$ (blue) and $({\rm{Var}}({m}_{2}| {m}_{1},{m}_{3},{m}_{4})-\frac{1}{2})/{\kappa }_{2}^{2}$ (red) for the Faraday probing of atomic quadratures ${\hat{x}}_{\text{A}}(\theta )$. The data are obtained with optical probe pulse durations corresponding to coupling strengths ${\kappa }_{1}^{2}=1.7$, ${\kappa }_{2}^{2}=0.81$, ${\kappa }_{3}^{2}=3.3$ and ${\kappa }_{4}^{2}=2.2$ between the atoms and the field. A value of 1/2 corresponds to the standard quantum limit (SQL) for the atomic spin variance. b Zoom-in of the red curve in (a). The shaded error bar (one standard deviation) is derived from 5 identical experiments, each consisting of 10,000 repetitions of the pulse sequence.

Another reason for the discrepancy between Figs. 1c and 3 is the more complex relations between the retrodicted variance of the optical measurement and the hypothetical projective atomic measurement for directions in the oscillator phase space, $\theta \,\ne\, 0,\frac{\pi }{2}$. The optical measurement outcome is correlated with the atomic observables, represented by the full matrix character of the operators ρ and E and hence its expectation value and variance in general involve both the probability distribution (diagonal elements) and coherences (off-diagonal elements) in Eq. (3). We thus emphasize that while the determination of the conditional variances of the optical measurement, for $\theta \,\ne\, 0,\frac{\pi }{2}$, are in agreement with our theoretical expectations for those quantities, the angular dependence of the results is different from the one of the hypothetical projective measurement of the atomic quadrature observables. Only in the limit of large κ₂, we recall that the optical Faraday rotation measurement becomes a projective atomic measurements, as shown in Supplementary Note 3.

Discussion

We have demonstrated that measurements of two non-commuting observables of a spin oscillator can both be retrodicted with a precision below the SQL by prior and posterior QND detection of these observables. These detection results respectively condition the Gaussian Wigner function for the density matrix ρ(t) and the effect matrix E(t) which together incorporate all our knowledge about the system at the past time t and thus provide our best estimate of the outcome of any measurement on the system. The past quantum state theory violates the HUR for non-commuting observables of the spin oscillator, and it may also violate error-disturbance relations^3,4 for sequential measurements of non-commuting observables, as the disturbance of the system by the first measurement does not prevent precise retrodiction of the second measurement outcome by the later probing. Our detection method is compatible with spin-based sensing and spectroscopy^32,33,34, and the predicted and retrodicted evolution may offer insight and allow precision estimation of external influences, which can be applied to general quantum metrology² such as interferometers^35,36, magnetometers^21,37,38 and force sensors based on mechanical oscillators^18,39.

Methods

Experiment details

Experimental setup

The experiment setup (Fig. 1) includes a 4-layer magnetic shielding, containing a paraffin-coated 20 mm × 7 mm × 7 mm rectangular vapor cell, and a set of coils for generating a homogeneous bias magnetic field of 0.71G which gives a ground-state Zeeman splitting of about Ω_L = 2π × 500 kHz. The measured decay time for the atomic Zeeman population and coherence are T₁ = 125 ms and T₂ = 20 ms, respectively, with the latter mainly limited by residual magnetic field inhomogeneity. A y-polarized probe laser propagating along the z-axis is blue-detuned by 2.1GHz from the $5{S}_{1/2},F=2\to 5{P}_{3/2},F^{\prime} =3$ transition of the D2 line. Its intensity is modulated at twice the Larmor frequency by an acousto-optic-modulator to implement the stroboscopic quantum back-action evasion protocol²², with an optimal duty cycle of 14%. In this protocol, the variance of ${\hat{S}}_{\text{y}}$ in photon shot noise unit for pulse duration τ after the interaction is²²

$$\,{\text{Var}}\,{\left({\hat{S}}_{\text{y},\tau }^{\text{out}}\right)}_{\text{SN}}\approx \left[1+{\tilde{\kappa }}\,^{2}+\frac{{\tilde{\kappa }}\,^{4}}{3}\frac{1-\,\text{Sinc}\,(\pi D)}{1+\,\text{Sinc}\,(\pi D)}\right],$$

(5)

where D is duty cycle, and ${\tilde{\kappa }}\,^{2}$ is proportional to κ² with a coefficient accounting for the stroboscopic effect (See Supplementary Note 6). The home-made balanced photodetector for measuring the S_y has a quantum efficiency of 92.4% and it operates in the unsaturated regime up to 12 mW.

First, we prepare the atoms in the state $5{S}_{1/2}\left|F=2,{m}_{F}=-2\right\rangle$ (with quantum number m_F associated with the quantization axis along x, the direction of the magnetic field) by applying the circular polarized and spatially overlapped σ⁻ pump and repump lasers propagating along the x-direction. We achieve up to 97.9% degree of spin orientation, as measured by the magneto-optical resonances⁴⁰. The optimized laser powers are 50 mW for the repump and 5 mW for the pump, both having elongated-Gaussian transverse intensity distribution. The probe mode is a symmetric Gaussian with 1/e² beam diameter of 6 mm. All three fields cover nearly the entire cell volume.

Calibration of the spin projection noise limit

The coupling strength ${\tilde{\kappa }}^{2}$ of the atom and field variables is calibrated by measuring the spin noise of the atomic ensemble in thermal equilibrium which is unpolarized and not affected by the tensor interaction. The observed spin noise in the thermal state should be $\frac{5}{4}\cdot \,{\text{Var}}\,{({\hat{x}}_{\text{A}},{\hat{p}}_{\text{A}})}_{CSS}$ by the following reasoning. The thermal state is isotropic, which implies $\langle {\hat{j}}_{\,\text{x}\,}^{2}\rangle =\langle {\hat{j}}_{\,\text{y}\,}^{2}\rangle =\langle {\hat{j}}_{\,\text{z}\,}^{2}\rangle =\frac{F(F+1)}{3}=2$ for F = 2. Meanwhile, all sublevels have the same population, including those in F = 1 which are not observed in the measurement. Since there are 8 sublevels in total for 5S_1/2 hyperfine states, and 5 of them belong to F = 2, the observed noise will thus be $2\cdot \frac{5}{8}=\frac{5}{4}$. This is $\frac{5}{4}$ times the noise variance $\langle {\hat{j}}_{\,\text{z}\,}^{2}\rangle$ in the CSS deduced from the Heisenberg uncertainty relation.

In our system, the linearly polarized probe light is transmitted through the atomic sample and it undergoes a small polarization rotation due to its interaction with the atoms. Our effective polarization homodyne detection employs the original linear polarization component as a local oscillator for the orthogonal component of the field system generated by the interaction with the atoms. The total noise on the optical readout signal includes photon shot noise and spin noise, ${\sigma }_{{x}_{\text{ph}}}^{2}=\frac{1}{2}+{\kappa }^{2}{\sigma }_{{p}_{\text{at}}}^{2}$. In addition to estimating the coupling strength κ from the physical parameters, we can thus infer its value from the noise in the probe experiments. In order to calibrate the photon shot noise level, the Larmor frequency is tuned far away from the lock-in detection bandwidth by changing the DC bias magnetic field, suppressing the noise contribution from the spin oscillator. The photon shot noise depends linearly on the input probe power²¹, since for the coherent state of light the variances of ${\hat{S}}_{\text{y}}$ and ${\hat{S}}_{\text{z}}$ should satisfy $\,{\text{Var}}\,({\hat{S}}_{\text{y}})=\,{\text{Var}}\,({\hat{S}}_{\text{z}})=\frac{{S}_{\text{x}}}{2}$. In addition, as shown in ref. ²¹, the linear scaling of spin noise power as a function of the atomic number indicates a quantum-limited performance and the QND character of the measurement.

Calibration of the measurement direction in the spin oscillator phase space

In the rotating frame, we first measure the quadrature ${\hat{p}}_{\text{A}}$. To measure ${\hat{x}}_{\text{A}}$, we need to wait for the spin oscillator to rotate by π/2, i.e., a quarter of the Larmor period. Here we describe how we verify that the measured quadrature is really ${\hat{x}}_{\text{A}}$, perpendicular to ${\hat{p}}_{\text{A}}$.

As depicted in the main text, we use a homogeneous DC bias magnetic field B_x in the spin orientation direction, which is the x-direction in this paper. This corresponds to an additional Hamiltonian term $\hat{H}={\Omega }_{\text{L}}{\hat{J}}_{\text{x}}$ with Ω_L = g_Fμ_BB_x/ℏ, where g_F is the hyperfine Landé g-factors for the ground state of ⁸⁷Rb, while μ_B and B are the Bohr magneton and the magnitude of the applied magnetic field. If we also add a radio frequency (RF) magnetic field oscillating at frequency Ω along the y-direction such that in the absence of light ${{\bf{B}}}_{\text{ext}}={B}_{\text{x}}{{\bf{e}}}_{\text{x}}+[{B}_{\text{c}}\cos (\Omega t+\phi )+{B}_{\text{s}}\sin (\Omega t+\phi )]{{\bf{e}}}_{\text{y}}$ with constant B_c and B_s one can derive Heisenberg equations of motion for the collective spin components ${\hat{J}}_{\,\text{y}\,}^{\prime}$ and ${\hat{J}}_{\,\text{z}\,}^{\prime}$ in the rotating frame⁴¹,

$$\frac{\partial {\hat{J}}_{\,\text{y}}^{\prime}}{\partial t}=-{\omega }_{\text{s}}\sin ({\Omega }_{\text{L}}t)\sin (\Omega t+\phi ){J}_{\text{x}}$$

(6)

$$\frac{\partial {\hat{J}}_{\,\text{z}}^{\prime}}{\partial t}=-{\omega }_{\text{c}}\cos ({\Omega }_{\text{L}}t)\cos (\Omega t+\phi ){J}_{\text{x}},$$

(7)

with ω_c,s = g_Fμ_BB_c,s/ℏ. Choosing the phase and the frequency of the RF-drive such that ϕ = 0 and Ω = Ω_L, we obtain $\frac{\partial {\hat{J}}_{\,\text{y}}^{\prime}}{\partial t}=-\frac{{\omega }_{\text{s}}{J}_{\text{x}}}{2}$ and $\frac{\partial {\hat{J}}_{\,\text{z}}^{\prime}}{\partial t}=-\frac{{\omega }_{\text{c}}{J}_{\text{x}}}{2}$, given interaction durations T satisfying the condition ω_c,sT ≪ 1 ≪ ΩT. With the RF magnetic field pulses, we are thus able to independently change the spin components ${\hat{J}}_{\,\text{y}\,}^{\prime}$ and ${\hat{J}}_{\,\text{z}\,}^{\prime}$ by an amount controlled by the sine and cosine components B_s and B_c.

Therefore, we can implement the calibration with the pulse sequence depicted in Fig. 4. An RF field pulse is applied between the pump laser and probe laser to create a transverse spin excitation, rotating in the y-z plane. The stroboscopically modulated probe laser is separated into two parts. The first measures the projection component in ${\hat{p}}_{\text{A}}$ and the second measures that in ${\hat{x}}_{\text{A}}$. The phase of the RF field determines the phase of the rotating transverse spin (i.e., the direction in the rotating frame). It is easy to maximize the signal of the first part of the probe by adjusting the phase of the RF field. At this time, the induced transverse spin should be in the ${\hat{p}}_{\text{A}}$ direction (in the rotating frame) and there should be only a minimal spin component in the ${\hat{x}}_{\text{A}}$ direction. We verify that, after a quarter of the Larmor period, the induced spin is in the ${\hat{x}}_{\text{A}}$-direction.

Fig. 4: Pulse sequence for calibration of measurement direction.

The RF field pulse between the pump and probe laser pulses creates a transverse spin excitation, rotating in the y-z plane. The first part of probe pulse measures ${\hat{p}}_{\text{A}}$. By tuning the phase of the RF field to maximize the first probe signal, we can validate that the transverse spin is in the ${\hat{p}}_{\text{A}}$ direction (in the rotating frame). If the second probe signal, after a quarter of the Larmor period, is minimal, we verify that the induced spin is in the ${\hat{x}}_{\text{A}}$-direction.

Data analysis

Since the expectation value μ of the gaussian distribution Pr(m₂∣m₁) is proportional to m₁, experimentally, the variance of m₂ conditioned on the measurement before t, with result m₁, should be obtained from linear numerical feedback²³ as

$$\,{\text{Var}}\,({m}_{2}| {m}_{1}) = \,{\min }_{\alpha }\left[\,{\text{Var}}\,({m}_{2}-\alpha {m}_{1})\right]\\ = \,{\min }_{\alpha }[\,{\text{Var}}\,({m}_{2})+{\alpha }^{2}\,{\text{Var}}\,({m}_{1})-2\alpha \,{\text{Cov}}\,({m}_{2},{m}_{1})]\\ = \,{\text{Var}}\,({m}_{2})-\frac{{{\text{Cov}}}^{2}({m}_{2},{m}_{1})}{\,{\text{Var}}\,({m}_{1})}.$$

(8)

where the minimum is achieved when

$$\alpha =\frac{\,{\text{Cov}}\,({m}_{2},{m}_{1})}{\,{\text{Var}}\,({m}_{1})}$$

(9)

Similarly, since the expectation value μ_ρE of the gaussian distribution Pr(m₂∣m₁, m₃, m₄) is proportional to m₁, m₃, and m₄, the variance of m₂ conditioned on the measurement before and after t, i.e., m₁, m₃, and m₄, should also be obtained from linear numerical feedback as

$$\,{\text{Var}}\,({m}_{2}| {m}_{1},{m}_{3},{m}_{4}) = \,{\min }_{\alpha ,\beta ,\gamma }[\,{\text{Var}}\,({m}_{2}-\alpha {m}_{1}-\beta {m}_{3}-\gamma {m}_{4})]\\ = \,{\min }_{\alpha ,\beta ,\gamma }\left[\,{\text{Var}}\,({m}_{2})+{\alpha }^{2}\,{\text{Var}}\,({m}_{1})+{\beta }^{2}\,{\text{Var}}\,({m}_{3})+{\gamma }^{2}\,{\text{Var}}\,({m}_{4})\right.\\ \,-2\alpha \,{\text{Cov}}\,({m}_{2},{m}_{1})-2\beta \,{\text{Cov}}\,({m}_{2},{m}_{3})-2\gamma \,{\text{Cov}}\,({m}_{2},{m}_{4})\\ \,\left.+\,2\alpha \beta \,{\text{Cov}}\,({m}_{1},{m}_{3})+2\alpha \gamma \,{\text{Cov}}\,({m}_{1},{m}_{4})+2\beta \gamma \,{\text{Cov}}\,({m}_{3},{m}_{4})\right]$$

(10)

The minimal is achieved when

$$\alpha = \, \frac{1}{\Lambda }\left({\text{Cov}}_{14}{\text{Cov}}_{23}{\text{Cov}}_{34}+{\text{Cov}}_{13}{\text{Cov}}_{24}{\text{Cov}}_{34}-{\text{Cov}}_{12}{\text{Cov}\,}_{34}^{2}\right.\\ \,\left.-{\text{Cov}}_{14}{\text{Cov}}_{24}{\text{Cov}}_{33}-{\text{Cov}}_{13}{\text{Cov}}_{23}{\text{Cov}}_{44}+{\text{Cov}}_{12}{\text{Cov}}_{33}{\text{Cov}}_{44}\right)\\ \beta = \,\frac{1}{\Lambda }\left({\text{Cov}}_{13}{\text{Cov}}_{14}{\text{Cov}}_{24}+{\text{Cov}}_{12}{\text{Cov}}_{14}{\text{Cov}}_{34}-{\text{Cov}}_{23}{\text{Cov}\,}_{14}^{2}\right.\\ \,\left.-{\text{Cov}}_{24}{\text{Cov}}_{34}{\text{Cov}}_{11}-{\text{Cov}}_{12}{\text{Cov}}_{13}{\text{Cov}}_{44}+{\text{Cov}}_{11}{\text{Cov}}_{23}{\text{Cov}}_{44}\right)\\ \gamma = \,\frac{1}{\Lambda }\left({\text{Cov}}_{13}{\text{Cov}}_{14}{\text{Cov}}_{23}+{\text{Cov}}_{12}{\text{Cov}}_{13}{\text{Cov}}_{34}-{\text{Cov}}_{24}{\text{Cov}\,}_{13}^{2}\right.\\ \,\left.-{\text{Cov}}_{23}{\text{Cov}}_{34}{\text{Cov}}_{11}-{\text{Cov}}_{12}{\text{Cov}}_{14}{\text{Cov}}_{33}+{\text{Cov}}_{24}{\text{Cov}}_{11}{\text{Cov}}_{33}\right)\\ \Lambda = \,2{\text{Cov}}_{13}{\text{Cov}}_{14}{\text{Cov}}_{34}-{\text{Cov}}_{14}^{2}{\text{Cov}}_{33}-{\text{Cov}}_{13}^{2}{\text{Cov}}_{44}\\ \,+{\text{Cov}}_{11}{\text{Cov}}_{33}{\text{Cov}}_{44}$$

(11)

Here Cov_uv = Cov(m_u, m_v) represent the covariance between m_u and m_v, where u, v = 1, 2, 3. When u = v, Cov_uu = Var(m_u).