1 Introduction

As firstly acknowledged by E. Schrödinger, entanglement is the characteristic aspect of quantum mechanics that enforces its complete departure from classical thought [1]. The physical consequence of entanglement that shows such departure is the violation of Bell inequalities [2] by quantum mechanics, an impossible result for any local and realistic (‘classical’) theory of nature. In consequence, it is of the utmost importance to experimentally test both issues at different scales. It should be noted here that entanglement does not necessarily imply violation of the Bell inequalities, but the opposite is true.

The violation of Bell inequalities in its Clauser-Horne-Shimony-Holt (CHSH) version [3] has been convincingly shown in experiments with low-energy photons and electrons in a number of dramatic experiments [4, 5], which have essentially closed all the conceivable loopholes on the validity of the tests. On the other hand such experimental tests have not been possible yet at high-energy scales.

This goal has been recently addressed in several interesting works [6,7,8,9,10], dealing with the top-antitop system (\(t{\bar{t}}\)) at the Large Hadron Collider (LHC). As stressed in Ref. [8], the qubits associated to the spin states of \(t{\bar{t}}\) pairs produced at the LHC provide a suitable arena to investigate these matters since top quarks decay before their spins are randomised by strong radiation and the spin of the lepton produced in semileptonic decays \(t \rightarrow \ell \nu b\) (\(\ell = e,\mu \)) is completely correlated to that of the mother top. Besides, owing to the large cross section, \(\sigma = 832\) pb at 13 TeV [11] there is a good amount of statistics on \(t{\bar{t}}\) production already at Run 2 with 139 fb\(^{-1}\) of collected data, and there will be much more in the future at its high luminosity upgrade (HL-LHC). In Refs. [6,7,8,9,10], certain sufficient conditions for entanglement and CHSH violation were obtained, together with a phenomenological exploration of their testability. In Ref. [8], it was concluded that, whereas a verification of entanglement is already feasible from the LHC Run 2 dataset, a significant evidence for the violation of Bell-like inequalities is much harder, even in the high luminosity LHC run.

The purpose of this paper is to re-examine the conditions for entanglement and CHSH-violation in the \(t{\bar{t}}\) system, and devise a refined strategy to experimentally test them. To this end, we will make two improvements. First we will consider an upper cut on the velocity of the \(t {\bar{t}}\) system:

$$\begin{aligned} \beta \equiv \left| \frac{p^z_t + p^z_{{\bar{t}}}}{E_t + E_{{\bar{t}}}} \right| \le \beta ^{\mathrm{cut}} \,, \end{aligned}$$
(1)

where \(p^z\), E are the three-momentum in the direction of the beam axis and the energy, respectively, of the top quark and anti-quark in the laboratory frame. An upper cut on \(\beta \) reduces the contribution of \(q \bar{q}\) annihilation processes to \(t{\bar{t}}\) production with respect to that of gluon fusion: the former are less central due to the different parton distribution functions (PDFs) of valence quarks and sea antiquarks inside the protons.Footnote 1 Therefore, an upper cut on \(\beta \) strengthens the entanglement (and thus the CHSH violation) near threshold because the resulting \(t{\bar{t}}\) system is closer to a spin-singlet (and thus maximally entangled) state due to the Landau-Yang theorem. The second improvement is to design physical observables that allow, by a single direct measurement, to evaluate combinations of spin correlations which enter both the tests of entanglement and the ones of CHSH violation.

The paper is organised as follows. In Sect. 2 we re-examine the requirements for entanglement and CHSH violation in the top-antitop system, refining previous sufficient conditions for both, and deriving the sufficient and necessary conditions in physically relevant limits. In Sect. 3 we expound the experimental strategy to probe both phenomena at the LHC, in particular we describe the physical observables that give a direct test of both entanglement and CHSH violation. Section 4 is devoted to the numerical simulation of the described strategy, showing the advantage gained thanks to the aforementioned improvements. Finally, in Sect. 5 we discuss our results and the future prospects. Additional results are presented in two appendices.

2 Conditions for entanglement and Bell inequalities

An entangled state of two subsystems (Alice and Bob) is by definition a non-separable one, i.e. one that cannot be expressed as \(|\psi _1\rangle _A \otimes |\psi _1\rangle _B\). If the state of the global system is a statistical mixture, described by a density matrix, \(\rho \), the separability condition reads

$$\begin{aligned} \rho _{\mathrm{sep}} = \sum _n p_n \rho _n^A \otimes \rho _n^B\ , \end{aligned}$$
(2)

where \(p_n\) are classical probabilities and \(\sum p_n = 1\). If \(\rho \) cannot be expressed as (2), then the state is entangled. Mathematically, a necessary and sufficient condition for entanglement in joint systems of two qubits (i.e. each one having a Hilbert space of dimension 2) is provided by the Peres–Horodecki criterion [14, 15]: from the the initial density matrix, \(\rho \), a new matrix is constructed by transposing only the indices associated to the Bob (or Alice) Hilbert space. If this partially transposed matrix, say \(\rho ^{T_2}\), is not a legal density matrix, which in particular means that it has at least one negative eigenvalue, then the \(\rho \) matrix corresponds to an entangled state.

Concerning the Bell inequalities, it has been shown that the so-called CHSH inequalities are an optimal version of them for joint systems of two qubits [16] when Alice and Bob can measure two different observables each, say \(A, A'\) (Alice) and \(B, B'\) (Bob), which take (or are assigned to take) two possible values, \(\pm 1\). In any local and realistic (‘classical’) theory, the averages of any such observables must fulfill the CHSH inequality

$$\begin{aligned} \left| \langle AB \rangle - \langle AB' \rangle + \langle A'B \rangle + \langle A'B' \rangle \right| \le 2\ . \end{aligned}$$
(3)

For appropriate choices of the \(A, A',B, B'\) observables this inequality can be violated by quantum mechanics in certain entangled \(\rho \) states.

Let us particularise the Peres–Horodecki criterion of entanglement and the CHSH inequalities for a system of two qubits, such as the spin of the \(t{\bar{t}}\) system. An appropriate basis to expand any Hermitian operator acting in the global Hilbert space is provided by the 16 matrices \(\left\{ \mathbbm {1}^A, \sigma _1^A, \sigma _2^A, \sigma _3^A\right\} \otimes \left\{ \mathbbm {1}^B, \sigma _1^B, \sigma _2^B, \sigma _3^B\right\} \). Hence the density matrix of the joint system, \(\rho \), can be written as

$$\begin{aligned} \rho&=\frac{1}{4}\left( \mathbbm {1}\otimes \mathbbm {1} +\sum _i(B_i^+ \sigma _i\otimes \mathbbm {1} + B_i^- \mathbbm {1}\otimes \sigma _i)\right. \nonumber \\&\quad \left. + \sum _{ij}C_{ij} \sigma _i\otimes \sigma _j \right) \ , \end{aligned}$$
(4)

where \(B_i^\pm , C_{ij}\) are real coefficients and we have dropped the A, B superscripts in the matrices. The \(\text {Tr}\ {\rho }=1\) condition is automatically taken into account by the coefficient of the first term. The explicit forms of \(\rho , \rho ^{T_2}\) read

$$\begin{aligned}&\rho = \frac{1}{4}\left[ \begin{array}{cccc} 1+B_3^++B_3^-+C_{33} &{} B_1^-+C_{31}-i(B_2^-+C_{32})&{} B_1^++C_{13}-i(B_2^++C_{23})&{} C_{11}-C_{22}-i(C_{12}+C_{21})\\ B_1^-+C_{31}+i(B_2^-+C_{32})&{} 1+B_3^+-B_3^--C_{33}&{} C_{11}+C_{22}+i(C_{12}-C_{21})&{} B_1^+-C_{13}-i(B_2^+-C_{23})\\ B_1^++C_{13}+i(B_2^++C_{23})&{} C_{11}+C_{22}-i(C_{12}-C_{21})&{} 1-B_3^++B_3^--C_{33}&{} B_1^--C_{31}-i(B_2^--C_{32})\\ C_{11}-C_{22}+i(C_{12}+C_{21})&{} B_1^+-C_{13}+i(B_2^+-C_{23})&{} B_1^--C_{31}+i(B_2^--C_{32})&{} 1-B_3^+-B_3^-+C_{33} \end{array}\right] \end{aligned}$$
(5)
$$\begin{aligned}&\rho ^{T_2} = \frac{1}{4}\left[ \begin{array}{cccc} 1+B_3^++B_3^-+C_{33} &{} B_1^-+C_{31}+i(B_2^-+C_{32})&{} B_1^++C_{13}-i(B_2^++C_{23})&{} C_{11}+C_{22}+i(C_{12}-C_{21})\\ B_1^-+C_{31}-i(B_2^-+C_{32})&{} 1+B_3^+-B_3^--C_{33}&{} C_{11}-C_{22}-i(C_{12}+C_{21})&{} B_1^+-C_{13}-i(B_2^+-C_{23})\\ B_1^++C_{13}+i(B_2^++C_{23})&{} C_{11}-C_{22}+i(C_{12}+C_{21})&{} 1-B_3^++B_3^--C_{33}&{} B_1^--C_{31}+i(B_2^--C_{32})\\ C_{11}+C_{22}-i(C_{12}-C_{21})&{} B_1^+-C_{13}+i(B_2^+-C_{23})&{} B_1^--C_{31}-i(B_2^--C_{32})&{} 1-B_3^+-B_3^-+C_{33} \end{array}\right] \end{aligned}$$
(6)

The eigenvalues of \(\rho ^{T_2}\) are extremely involved combinations of the various parameters, and so are the necessary and sufficient conditions for entanglement (i.e. the existence of at least one negative eigenvalue). Fortunately, very useful sufficient conditions for entanglement are much easier to obtain, by simply probing the negativity of \(v^T \rho ^{T_2}v\) for different 4-vectors v. In particular, using

$$\begin{aligned} v= (1,0,0,\pm 1)^T \,,~ (0,1,\pm 1,0)^T \end{aligned}$$
(7)

we get four completely general sufficient conditions for entanglement, which can be cast as

$$\begin{aligned} |C_{11}+ C_{22}|> & {} 1+ C_{33} \,, \nonumber \\ |C_{11}- C_{22}|> & {} 1- C_{33} \,, \end{aligned}$$
(8)

(it is enough that one of the conditions (8) is fulfilled to guarantee entanglement). We remark that the two conditions (8) are equivalent to the pairs of conditions that can also be obtained for any permutation of the 1,2,3 indices: It can be shown that if neither of (8) hold, then the analogous conditions with index permutations are not fulfilled either.

One may wonder to which extent the previous inequalities (8) are in practice not only sufficient, but also necessary conditions for entanglement. To this end, let us note that one can make some sound approximations in the original density matrix of the \(t{\bar{t}}\) system (4). As shown in Ref. [17], P and CP invariance in the \(t{\bar{t}}\) production leads to \(B_i^\pm =0\), \(C_{ij}=C_{ji}\), thus reducing the number of parameters from 15 to 6. This is an excellent approximation due to the smallness of the weak corrections to the QCD production mechanism of \(t{\bar{t}}\) at the LHC. Besides, using the so-called helicity basis (defined below) as reference system, all the off-diagonal \(C_{ij}\) but one (say \(C_{12}\simeq C_{21}\)), are generated by P-odd absorptive parts of the mixed QCD-weak corrections at one-loop, and are very small [17]. In summary, it is a very good approximation in that basis to keep \(C_{11}, C_{22}, C_{33}\) and \(C_{12}=C_{21}\) as the only non-vanishing parameters in (4). Under this assumption it is possible to extract simple necessary and sufficient conditions for the negativity of \(\rho ^{T_2}\):

$$\begin{aligned} |C_{11}+ C_{22}|> & {} 1+ C_{33} \,, \nonumber \\ |4C_{12}^2+(C_{11}- C_{22})^2|^{1/2}> & {} 1- C_{33} \,, \end{aligned}$$
(9)

(equivalent conditions are obtained by performing cyclic permutations in the indices). Thus, in the previous approximation, it is sufficient and necessary that one of the relations (9) is fulfilled to guarantee entanglement. It is interesting that the first condition in (9) remains unchanged by the inclusion of \(C_{12}\ne 0\) and typically represents the most relevant test of entanglement when \(C_{11}, C_{22}\) have the same sign and \(C_{33}\) is negative, which is precisely the actual situation, as it will be clear in Sect. 4 below. Incidentally, in this approximation the conditions for the semipositivity of \(\rho \) read

$$\begin{aligned} |C_{11} + C_{22}|\le & {} 1- C_{33} , \nonumber \\ |4C_{12}^2 + (C_{11} - C_{22})^2|^{1/2}\le & {} 1+ C_{33} \, \end{aligned}$$
(10)

(both conditions required). These relations, together with Eqs. (9), imply that entanglement requires all \(C_{ii}\ne 0\).

Let us now turn to the CHSH inequalities Eq. (3). As usual, we consider spin observables for Alice and Bob defined by

$$\begin{aligned} A=a_i \sigma _i^A,\quad A'=a'_i \sigma _i^A, \quad B=b_i \sigma _i^B, \quad B'=b'_i \sigma _i^B , \end{aligned}$$
(11)

where, for a moment, we have indicated with a superscript the Alice and Bob operators, and \(a_i, a'_i, b_i, b'_i\) are the components of unit-vectors in space in the same basis as the corresponding \(\sigma -\)matrices. Then the expectation values \(\langle AB \rangle , \langle AB' \rangle , \langle A'B \rangle , \langle A'B' \rangle \) for a state described by the density matrix (4) read

$$\begin{aligned} \langle AB \rangle =\mathrm{Tr}[\rho (A\otimes B)]=\sum _{ij}C_{ij} a_ib_j \ , \end{aligned}$$
(12)

with analogous expressions obtained by replacing \(A\rightarrow A'\), \(a_i\rightarrow a'_i\); \(B\rightarrow B'\), \(b_j\rightarrow b'_j\). Hence the CHSH inequality (3) can be expressed as

$$\begin{aligned} \left| \vec {a}\cdot C (\vec {b}-\vec {b'}) \ +\ \vec {a'}\cdot C (\vec {b}+\vec {b'}) \right| \le 2\ . \end{aligned}$$
(13)

We are interested in choices of \(\vec {a},\vec {a'},\vec {b},\vec {b'}\) that violate (13) in the strongest possible way. In this sense, it was shown in Ref. [18] that the maximum of the l.h.s. of (13) is given by

$$\begin{aligned} \max _{{a},{a'},{b},{b'}}\left| \vec {a}\cdot C (\vec {b}-\vec {b'}) \ +\ \vec {a'}\cdot C (\vec {b}+\vec {b'}) \right| =2\sqrt{\lambda +\lambda '}\ , \end{aligned}$$
(14)

where \(\lambda ,\lambda '\) are the largest eigenvalues of \(C^T C\). This corresponds to choosing \(\vec {b}\pm \vec {b'}\) proportional to the associated eigenvectors with appropriate coefficients, and \(\vec {a},\vec {a'}\) as the unit-vectors proportional to \(C(\vec {b}-\vec {b'}), C(\vec {b}+\vec {b'})\), respectively. Then the violation of any CHSH inequality requires

$$\begin{aligned} \lambda +\lambda '>1\ , \end{aligned}$$
(15)

which thus represents, in principle, the optimal test for CHSH violation. However, as discussed in Ref. [8], the strategy of relying on the largest eigenvalues of \(C^T C\), Eq. (15), to test CHSH violation [7] is in practice problematic and likely biased, due to the uncertainties involved in the measurement. A more conservative approach consists in choosing \(\vec {a},\vec {a'}, \vec {b},\vec {b'}\) in such a way that the l.h.s. of (13) is still potentially larger than 2, but far simpler than (14) and with much more transparent physical meaning. In particular, for any pair of indices \(i\ne j\) we can choose

$$\begin{aligned}&a_k=\delta _{ki},\quad \quad a'_k=\delta _{kj}, \nonumber \\&b_i=-b'_i=\pm \frac{1}{\sqrt{2}}, \quad \quad b_j=b'_j=\pm \frac{1}{\sqrt{2}}, \quad b_{k\ne i,j}=0 \nonumber \\ \end{aligned}$$
(16)

(uncorrelated signs). Using these two sets of vectors the violation of CHSH inequalities reads

$$\begin{aligned} |C_{ii}\pm C_{jj}| > \sqrt{2}\ . \end{aligned}$$
(17)

Then, of course, the strategy is to choose i, j and the ± sign to maximise \(|C_{ii}\pm C_{jj}|\) (the optimal choice depending on their moduli and relative sign). It is interesting to note that, although Eq. (17) involves just two diagonal entries of C, the third one must also be non-vanishing. Actually, it occurs in such a way that at least one of the sufficient conditions for entanglement (8) is fulfilled.Footnote 2

Similarly to our previous discussion on the entanglement test, one may wonder to which extent expression (17) is close in practice to the optimal value (14). As discussed above, an excellent approximation for C is to assume \(C_{ij}=C_{ji}\), so \(\lambda , \lambda '\) are the largest eigenvalues of \(C^2\). Then, from the eigenvalue interlacing theorem, for any \(i\ne j\)

$$\begin{aligned} 2\sqrt{\lambda +\lambda '}\ge & {} 2\sqrt{(C^2)_{ii} + (C^2)_{jj}}\ \ge \ 2\sqrt{(C_{ii})^2 + (C_{jj})^2}\ \nonumber \\\ge & {} \sqrt{2}|C_{ii}\pm C_{jj}|\ . \end{aligned}$$
(18)

Note that the violation of the CHSH inequality (17) implies condition (15), as expected, but not the other way around. Now, the first two inequalities in (18) are usually close to the equality due to the smallness of the off-diagonal \(C-\)entries in the bases we use, as it will be clear below in Sect. 4. The third inequality in (18) is close to the identity, as long as \((|C_{ii}|-|C_{jj}|)^2\ll (|C_{ii}|+|C_{jj}|)^2\), which is also the typical case once we select the two largest \(|C_{ii}|,\ |C_{jj}|\). Anyway, if desired, one can refine the CHSH-violation condition (17) with this correction:

$$\begin{aligned} |C_{ii}|+|C_{jj}|+ \frac{1}{2}\frac{(|C_{ii}|-|C_{jj}|)^2}{|C_{ii}|+ |C_{jj}|} > \sqrt{2}\ , \end{aligned}$$
(19)

which, of course, is easier to satisfy than (17).

3 Experimental strategy

In this section we use the charged leptons from the top and antitop semileptonic decays \(t {\bar{t}} \rightarrow \ell ^+ \nu b \, \ell ^- {\bar{\nu }} {\bar{b}}\) to analyse the spin properties of the mother particles; but, in order to keep the formalism general, we first consider arbitrary decay products ‘a’ from the top quark and ‘b’ from the anti-quark. Their normalised three-momenta, expressed in polar coordinates, in the respective top quark and antiquark rest frames, read

$$\begin{aligned}&{\hat{p}}_{a} = (\sin \theta _a \cos \varphi _a,\sin \theta _a \sin \varphi _a,\cos \theta _a )^T \,, \nonumber \\&{\hat{p}}_{b} = (\sin \theta _b \cos \varphi _b,\sin \theta _b \sin \varphi _b,\cos \theta _b )^T \,. \end{aligned}$$
(20)

where we use he same \(R^3\) basis \(({{\hat{x}}},{{\hat{y}}}, {{\hat{z}}})\) in the top quark and antiquark rest frames. The decay density matrices for the quarks read [19]

$$\begin{aligned} \Gamma _a = \frac{1}{2} \left( \begin{array}{cc} 1 + \alpha _a \cos \theta _a &{} \alpha _a \sin \theta _a e^{i \varphi _a} \\ \alpha _a \sin \theta _a e^{-i \varphi _a} &{} 1 - \alpha _a \cos \theta _a \end{array} \right) \,, \end{aligned}$$
(21)

with an analogous expression for \(\Gamma _b\) with the replacement \(a \rightarrow b\). The constants \(\alpha \) satisfy \(|\alpha | \le 1\) and are called ‘spin analysing power’ of the corresponding top (anti-)quark decay product. For the charged leptons \(\alpha _{\ell ^+} = - \alpha _{\ell ^-} = 1\) at the tree level, with next-to-leading order corrections at the permille level [20]. The joint decay distribution is proportional to the sum [21]

$$\begin{aligned} \sum _{\lambda _t \lambda '_t \lambda _{{\bar{t}}} \lambda '_{{\bar{t}}} = \pm 1/2} \rho (\lambda _t, \lambda '_t, \lambda _{{\bar{t}}}, \lambda '_{\bar{t}}) \Gamma _a(\lambda _t, \lambda '_t) \Gamma _b(\lambda _{{\bar{t}}}, \lambda '_{{\bar{t}}}) \,. \end{aligned}$$
(22)

Omitting for brevity the terms proportional to \(B_i^\pm \), which do not contribute to the observables considered, the quadruple differential distribution reads

$$\begin{aligned}&\frac{1}{\sigma } \frac{d\sigma }{d\Omega _a d\Omega _b} = \frac{1}{(4\pi )^2} \left[ 1 + \alpha _a \alpha _b \sin \theta _a \sin \theta _b \nonumber \right. \\&\qquad \left. \left( C_{11} \cos \varphi _a \cos \varphi _b + C_{22} \sin \varphi _a \sin \varphi _b \right) \right. \nonumber \\&\quad + \alpha _a \alpha _b \sin \theta _a \sin \theta _b \left( C_{12} \cos \varphi _a \sin \varphi _b + C_{21} \sin \varphi _a \cos \varphi _b \right) \nonumber \\&\quad + \alpha _a \alpha _b \left( C_{13} \sin \theta _a \cos \varphi _a \cos \theta _b + C_{31} \cos \theta _a \sin \theta _b \cos \varphi _b \right) \nonumber \\&\quad + \alpha _a \alpha _b \left( C_{23} \sin \theta _a \sin \varphi _a \cos \theta _b + C_{32} \cos \theta _a \sin \theta _b \sin \varphi _b \right) \nonumber \\&\quad \left. + \alpha _a \alpha _b C_{33} \cos \theta _a \cos \theta _b \right] \,, \end{aligned}$$
(23)

with \(d\Omega _a = d\cos \theta _a d\varphi _a\) and likewise for \(d\Omega _b\). Using this distribution, we want to design observables that directly test entanglement and violation of the CHSH inequalities.

For completeness, we point out that the coefficients can be individually measured from two-dimensional distributions or from forward-backward asymmetries, without the need of a fit to the four-dimensional distribution (23). Let us relabel for convenience the polar angles \(\theta _{a,b}\) in (20) as \(\theta _{a,b}^3\), and introduce the polar angles \(\theta _{a,b}^{1}\), \(\theta _{a,b}^{2}\) between \({{\hat{p}}}_{a,b}\) and the \({{\hat{x}}}\), \({{\hat{y}}}\) axes, respectively. Then, for \(i,j=1,2,3\) we have the distributions [17]

$$\begin{aligned} \frac{1}{\sigma }\frac{d\sigma }{d\cos \theta _a^i d\cos \theta _b^j} = \frac{1}{4} \left( 1 + \alpha _a \alpha _b C_{ij} \cos \theta _a^i \cos \theta _b^j \right) \,. \end{aligned}$$
(24)

Hence, \(C_{ij}\) can be extracted by a fit to the above distributions, properly corrected for detector effects. Alternatively, one can write forward-backward asymmetries

$$\begin{aligned} A_{ij}= & {} \frac{N(\cos \theta _a^i \cos \theta _b^j> 0) - N(\cos \theta _a^i \cos \theta _b^j< 0)}{N(\cos \theta _a^i \cos \theta _b^js > 0) + N(\cos \theta _a^i \cos \theta _b^j < 0)} \nonumber \\= & {} \frac{1}{4} \alpha _a \alpha _b C_{ij} \,, \end{aligned}$$
(25)

with \(N(\cdot )\) standing for the number of events. These asymmetries allow to extract the coefficients, assuming the Standard Model value for \(\alpha _a \alpha _b\).

3.1 Observables for entanglement

A sufficient set of conditions for entanglement is given in (8). Depending on the sign of the sums in the absolute value, the quantities to test are

$$\begin{aligned}&C_{11} + C_{22} - C_{33}> 1 \,, \nonumber \\&-C_{11} - C_{22} - C_{33}> 1 \,, \nonumber \\&C_{11} - C_{22} + C_{33}> 1 \,, \nonumber \\&-C_{11} + C_{22} + C_{33} > 1 \,. \end{aligned}$$
(26)

In the second line we can identify the sum \(C_{11} + C_{22} + C_{33}\). There is a well-known observable that can measure this sum [17, 22,23,24]; namely, the one-dimensional kinematical distribution in terms of the cosine of the ‘opening angle’ \(\cos \theta _{ab} \equiv {{\hat{p}}}_a \cdot {{\hat{p}}}_b\), with \({{\hat{p}}}_a, {{\hat{p}}}_b\) defined in (20),

$$\begin{aligned} \frac{1}{\sigma } \frac{d\sigma }{d\cos \theta _{ab}} = \frac{1}{2} \left( 1 + \alpha _a \alpha _b D \cos \theta _{ab} \right) \,, \end{aligned}$$
(27)

where the D coefficient is related to the trace of the C matrix by [23]Footnote 3

$$\begin{aligned} D = \frac{1}{3} (C_{11} + C_{22} + C_{33}) \,. \end{aligned}$$
(28)

This coefficient can be experimentally measured either by a fit to the \(\cos \theta _{ab}\) distribution, or by a forward-backward asymmetry in \(\cos \theta _{ab}\). The rest of equations in (26) involve two coefficients with the same sign and the third with opposite sign. It is easy to design an observable for these sums. The differential distribution (23) can be written in compact form as

$$\begin{aligned} \frac{1}{\sigma } \frac{d\sigma }{d\Omega _a d\Omega _b} = \frac{1}{(4\pi )^2} \left[ 1 + \alpha _a \alpha _b {{\hat{p}}}_a^T C {{\hat{p}}}_b \right] \,. \end{aligned}$$
(29)

Let us focus for definiteness on the sum in the first line of (26), \(C_{11} + C_{22} - C_{33}\). By introducing a reflection in the \((x_1,x_2)\) plane of the top anti-quark rest frame, \(P_3 = \text {diag}(1,1,-1)\), the angular-dependent term in (29) can be rewritten as

$$\begin{aligned} {{\hat{p}}}_a^T C {{\hat{p}}}_b = {{\hat{p}}}_a^T C P_3^2 {{\hat{p}}}_b = {{\hat{p}}}_a^T (C P_3) (P_3 {{\hat{p}}}_b) \,, \end{aligned}$$
(30)

with

$$\begin{aligned}&(P_3 {{\hat{p}}}_b) = (\sin \theta _b \cos \varphi _b,\sin \theta _b \sin \varphi _b,-\cos \theta _b )^T \,. \end{aligned}$$
(31)

Therefore, the kinematical distribution of the angle \(\theta _{ab}'\), defined by \(\cos \theta _{ij}' = {{\hat{p}}}_a \cdot P_3{{\hat{p}}}_b\),

$$\begin{aligned} \frac{1}{\sigma } \frac{d\sigma }{d\cos \theta '_{ab}} = \frac{1}{2} \left( 1 + \alpha _a \alpha _b D_3 \cos \theta '_{ab} \right) \,, \end{aligned}$$
(32)

measures the trace of the matrix \((C P_3)\), that is,

$$\begin{aligned} D_3 = \frac{1}{3} (C_{11} + C_{22} - C_{33}) \,. \end{aligned}$$
(33)

This dependence is confirmed by directly calculating \(\langle \cos \theta '_{ab} \rangle \) using (23). Likewise as for D, the \(D_3\) coefficient can be measured either with a fit to the \(\cos \theta '_{ab}\) distribution or by a forward-backward asymmetry in \(\cos \theta '_{ab}\).

3.2 Observables for CHSH inequalities

We now want to design observables that can directly measure sums and differences \(C_{ii} \pm C_{jj}\) involved in the relations for CHSH violation (17) or (19). Let us define the azimuthal angle differences

$$\begin{aligned}&\varphi _+ = \frac{1}{2}(\varphi _a + \varphi _b) \,, \quad \varphi _- = \frac{1}{2} (\varphi _a - \varphi _b ) \,. \end{aligned}$$
(34)

Changing variables in (23) to these angles, we arrive at

$$\begin{aligned}&\frac{1}{\sigma } \frac{d\sigma }{d\cos \theta _a d\cos \theta _b d\varphi _+ d\varphi _-} \nonumber \\&\quad =\frac{2}{(4\pi )^2} \left\{ 1 + \alpha _a \alpha _b \sin \theta _a \sin \theta _b \left[ \frac{C_{11} + C_{22}}{2} \cos 2 \varphi _- \right. \right. \nonumber \\&\qquad \left. \left. + \frac{C_{11} - C_{22}}{2} \cos 2 \varphi _+ + \frac{C_{12} + C_{21}}{2} \sin 2\varphi _+ \right. \right. \nonumber \\&\qquad \left. + \frac{C_{21}-C_{12}}{2} \sin 2\varphi _- \right] + \alpha _a \alpha _b C_{33} \cos \theta _a \cos \theta _b \nonumber \\&\qquad + \alpha _a \alpha _b \sin \theta _a \cos \theta _b \left[ C_{13} \cos (\varphi _+ + \varphi _-) + C_{23} \sin (\varphi _+ + \varphi _-) \right] \nonumber \\&\qquad + \alpha _a \alpha _b \cos \theta _a \sin \theta _b \left[ C_{31} \cos (\varphi _+ - \varphi _-) + C_{32} \sin (\varphi _+ - \varphi _-) \right] \,. \nonumber \\ \end{aligned}$$
(35)

Since we are only interested in the sums in the first line of (35), we can integrate over \(\cos \theta _{a,b}\) to obtain

$$\begin{aligned}&\frac{1}{\sigma } \frac{d\sigma }{d\varphi _+ d\varphi _-} = \frac{1}{2\pi ^2} + \frac{\alpha _a \alpha _b}{32} \nonumber \\&\quad \times \left[ \frac{C_{11} + C_{22}}{2} \cos 2 \varphi _- + \frac{C_{11} - C_{22}}{2} \cos 2 \varphi _+ \right. \nonumber \\&\quad \left. + \frac{C_{12} + C_{21}}{2} \sin 2 \varphi _+ + \frac{C_{21}-C_{12}}{2} \sin 2 \varphi _- \right] \,. \end{aligned}$$
(36)

Integration over \(\varphi _+\) and \(\varphi _-\) using the integration measure \(g_+ = \text {sign} \cos 2 \varphi _-\), yields the asymmetry

$$\begin{aligned} A_+ = \frac{1}{\sigma } \int \frac{d\sigma }{d\varphi _+ d\varphi _-} g_+ \, d\varphi _+ d\varphi _- = \frac{\pi }{16} \alpha _a \alpha _b (C_{11} + C_{22}) \,, \end{aligned}$$
(37)

whereas if we use the integration measure \(g_- = \text {sign} \cos 2 \varphi _+\) we obtain the asymmetry

$$\begin{aligned} A_- = \frac{1}{\sigma } \int \frac{d\sigma }{d\varphi _+ d\varphi _-} g_- \, d\varphi _+ d\varphi _- = \frac{\pi }{16} \alpha _a \alpha _b (C_{11} - C_{22}) \,. \end{aligned}$$
(38)

Experimentally, these asymmetries are measured by simply counting events with \(\cos (\varphi _a \mp \varphi _b) > 0\) and \(\cos (\varphi _a \mp \varphi _b) < 0\), that is,

$$\begin{aligned} A_\pm = \frac{N(\cos (\varphi _a \mp \varphi _b)> 0) - N(\cos (\varphi _a \mp \varphi _b)< 0)}{N(\cos (\varphi _a \mp \varphi _b) > 0) + N(\cos (\varphi _a \mp \varphi _b) < 0)} \,. \end{aligned}$$
(39)

4 Numerical results

We use the Monte Carlo generator MadGraph_aMC@NLO [25] to obtain our predictions for \(t {\bar{t}}\) production at the leading order in QCD, using NNPDF 3.0  [26] PDFs with factorisation and renormalisation scales equal to the average transverse mass, \(Q = 1/2 [(m_t^2 + p_{T t}^2)^{1/2} + (m_t^2 + p_{T {\bar{t}}}^2)^{1/2} ]\), with \(p_T\) the transverse momentum in the usual notation. The dilepton decay channel \(t {\bar{t}} \rightarrow \ell ^+ \nu b \ell ^- \nu b\) is selected, with \(\ell = e,\mu \). In order to study the dependence on the \(t{\bar{t}}\) invariant mass \(m_{t{\bar{t}}}\) and the top scattering angle in the center of mass frame \(\theta _\text {CM}\), we generate samples of \(t {\bar{t}}\) events in 50 GeV slices of \(m_{t{\bar{t}}}\) between [300, 350] and [750, 800] GeV, and 100 GeV slices between [0.8, 0.9] and [1.2, 1.3] TeV, totaling \(3.4 \times 10^7\) events. Three additional samples of \(5 \times 10^6\) events each are generated, with an upper cut \(m_{t{\bar{t}}}\le 360\) GeV, with a lower cut \(m_{t{\bar{t}}}\ge 1\) TeV and without cuts on \(m_{t{\bar{t}}}\).

In our Monte Carlo calculations we work at the parton level and do not set any cut on the transverse momenta of the visible particles (charged leptons and b-quark jets) nor missing energy, in order to obtain predictions for the full phase space. An experimental analysis involves some loose lower cuts on the transverse momenta of the visible particles, and some upper cuts on their rapidities, typically \(|\eta | \le 2.5\). A veto is also placed on events where the two leptons have the same flavour and invariant mass \(m_{\ell \ell } \sim M_Z\), in order to suppress the Drell-Yan background. In addition, the momenta of the top quark and anti-quark have to be reconstructed from those of the observed particles and the missing energy. Therefore, the detector-level observables and/or distributions measured have to be unfolded to take into account acceptance, detector and reconstruction effects, and recover their parton-level values. This is a well-established procedure (see for example Refs. [30, 31]) and, because we are only interested on the improvement of the statistical sensitivity, we do not address these issues. Instead, we use the parton-level information (momenta of all the particles, including top quarks) to determine the theoretical predictions of the observables considered.

In this work we mainly use as reference system to express the \(t\bar{t}\) density matrix (4) the helicity basis, with vectors \(({{\hat{r}}},{{\hat{n}}},{{\hat{k}}})\) defined as

  • K-axis (helicity): \({{\hat{k}}}\) is a normalised vector in the direction of the top quark three-momentum in the \(t {\bar{t}}\) rest frame.

  • R-axis: \({{\hat{r}}}\) is in the production plane and defined as \({{\hat{r}}} = \mathrm {sign}(y_p) ({{\hat{p}}}_p - y_p {{\hat{k}}})/r_p\), with \({{\hat{p}}}_p = (0,0,1)\) the direction of one proton in the laboratory frame, \(y_p = {{\hat{k}}} \cdot {{\hat{p}}}_p\), \(r_p = (1-y_p^2)^{1/2}\). The definition for \({{\hat{r}}}\) is the same if we use the direction of the other proton \(- {{\hat{p}}}_p\).

  • N-axis: \({{\hat{n}}} = {{\hat{k}}} \times {{\hat{r}}}\) is orthogonal to the production plane and can also be written as \({{\hat{n}}} = \mathrm {sign}(y_p) ({{\hat{p}}}_p \times {{\hat{k}}})/r_p\), which again is independent of the proton choice.

We point out that, unlike other works [7, 17] we use the same basis for the top quark and anti-quark. For the test of the CHSH inequalities at threshold we use a fixed beamline basis \(({{\hat{x}}},{{\hat{y}}},{{\hat{z}}})\) with \({{\hat{x}}} = (1,0,0)\), \({{\hat{y}}} = (0,1,0)\), \({{\hat{z}}} = (0,0,1)\).

In order to motivate the choice of observables in the following subsections, we present in Fig. 1 the dependence on \(m_{t{\bar{t}}}\) and \(\theta _\text {CM}\) of the diagonal spin correlation coefficients \(C_{kk}\), \(C_{rr}\), \(C_{nn}\) in the helicity basis, as well as the largest off-diagonal coefficient \(C_{kr} = C_{rk}\). The integrated values of the spin correlation coefficients are \(C_{kk} = -0.346\), \(C_{rr} = -0.021\), \(C_{nn} = -0.334\), \(C_{kr} = 0.109\), close to the NLO values [17]. Note that \(C_{nn} \le 0\) across all the range studied, while for \(C_{kk}\) and \(C_{rr}\) there can be large correlations of either sign. In particular, although in average \(C_{rr}\) is the smallest correlation, when the values in specific regions are considered, \(C_{kr}\) is by far the smallest one. All the figures are approximately symmetric in \(\cos \theta _\text {CM}\) but we prefer to keep the \([-1,1]\) range of variation of this variable in order to have a visual estimation of the statistical uncertainty of our Monte Carlo results. We note that for simplicity, throughout this work we calculate the individual spin correlation coefficients from forward-backward asymmetries of the type (25). In the next subsections we study in more detail how the use of an upper cut on \(\beta \) improves the observability of quantum correlations and the violation of the CHSH inequalities.

Fig. 1
figure 1

Dependence on \(m_{t{\bar{t}}}\) and \(\theta _\text {CM}\) of the spin correlation coefficients \(C_{kk}\), \(C_{rr}\), \(C_{nn}\) and \(C_{kr}\), at the parton level without kinematical cuts

Full size image

4.1 Observation of entanglement

In view that in the helicity basis \(C_{nn} \le 0\) and \(C_{kk},\ C_{rr}\) have typically the same sign, the most useful entanglement condition among Eqs. (26) is

$$\begin{aligned} E \equiv |C_{kk} + C_{rr}| - C_{nn} -1 > 0 \,. \end{aligned}$$
(40)

For brevity, we label the entanglement indicator in the l.h.s. of the inequality (40) as E. Near threshold both \(C_{kk}\) and \(C_{rr}\) are negative (see Fig. 1), therefore \(E = -C_{kk} - C_{rr} - C_{nn} - 1 = - 3D -1\), c.f. (28). For boosted central tops \(C_{kk},C_{rr} > 0\), and \(E = C_{kk} + C_{rr} - C_{nn} - 1 = 3 D_3 -1\), as follows from (33) by choosing the third axis in the \({{\hat{n}}}\) direction.

As discussed below Eq. (1), an upper cut on \(\beta \) potentially enhances the entanglement near threshold. In Fig. 2, left panel, we plot E (evaluated from the individual values of \(C_{ii}\)) as a function of \(m_{t{\bar{t}}}\) and \(\theta _\text {CM}\) without applying any kinematical cut, whereas in the right panel we require \(\beta \le 0.8\). The enhancement is notable near threshold, while the requirement on \(\beta \) has little effect at the boosted, central region.

Fig. 2
figure 2

Dependence of the entanglement indicator E in (40) on \(m_{t{\bar{t}}}\) and \(\theta _\text {CM}\)

Full size image

Of course, in order to optimise the sensitivity to entanglement one has to consider the uncertainty in the measurement in each case. In this work we do not consider systematic uncertainties, which cannot be addressed without a full detector simulation, a complete reconstruction of the \(t {\bar{t}}\) kinematics, and unfolding to parton-level. Instead, we just focus on statistical uncertainties. These basically depend on the size of the signal sample after event selection. In this work we assume an event selection and reconstruction efficiency of 0.12, the average value found in Ref. [8]. For comparison, using a typical event selection and a \(t {\bar{t}}\) kinematical reconstruction, an efficiency of 0.17 was found in Ref. [27] using Pythia [28] for hadronisation and showering and Delphes [29] for a fast simulation of the detector. This efficiency is used to obtain the number of events, given the \(t {\bar{t}} \rightarrow \ell ^+ \nu b \ell ^- {\bar{\nu }} {\bar{b}}\) cross section in the phase space region considered and the luminosity.

We devise three sets of simple kinematical cuts to enhance the observability of E near threshold and at the boosted regime; in the former case, with and without the use of \(\beta \), to illustrate the improvement. Since statistical uncertainties are proportional to \(1/\sqrt{\sigma }\), we use the figure of merit

$$\begin{aligned} S_E = E \times \sqrt{\sigma }\end{aligned}$$
(41)

to guide the selection of the kinematical cuts, collected in Table 1, together with the tree-level cross section after these cuts. The last column corresponds to the number of events with a luminosity \(L = 139~\text {fb}^{-1}\) collected at the LHC Run 2, assuming a K factor of 1.8 to normalise the total cross section to next-to-next-to-leading order [11] and using a selection and reconstruction efficiency of 0.12. For reference, we collect the value of spin correlation coefficients with this kinematical selection in Table 2. For the threshold analysis the beamline basis is equivalent to the helicity basis, because the relevant observable, \(D=\mathrm{Tr}\ C\) is the same. For the boosted analysis, where the relevant observable is \(D_3\), the helicity basis is superior.

Table 1 Kinematical cuts on \(m_{t{\bar{t}}}\) (in GeV), \(\cos \theta _\text {CM}\) and \(\beta \) used to optimise the figure of merit \(S_E\). The fourth column gives the tree-level cross section with the corresponding cuts, and the fifth column the expected number of events after reconstruction (see the text for details)
Full size table
Table 2 Parton-level values of the spin correlation coefficients in the helicity and beamline bases with the kinematical selection in Table 1. In the helicity basis \(C_{kr}=C_{rk}\), and in the beamline basis \(C_{xx}=C_{yy}\). The rest of coefficients are below 0.01
Full size table

For each set of cuts we estimate the statistical uncertainty in the measurement of E by performing \(n=200\) pseudo-experiments. In each pseudo-experiment we select a random set of N \(t {\bar{t}}\) events (with N in the last column of Table 1) and calculate E, either from the individual measurements of \(C_{ii}\), or with a direct measurement using D or \(D_3\).Footnote 4 The mean and standard deviation of the pseudo-experiments are presented in Table 3. The standard deviation obtained from the pseudo-experiments is a good estimation of the statistical uncertainty that would be present in such dataset. For the ‘individual’ measurements the uncertainties can also be estimated by simple error propagation. Because the statistical uncertainty in asymmetries is \(1/\sqrt{N}\) (provided the asymmetries are small, as is our case), the C coefficients have an uncertainty of \(4/\sqrt{N}\), see (25), and summing in quadrature their uncertainties results in \(4 \sqrt{3/N}\). For \(N = 108{,}000\), 83, 000, 9400 in Table 1, this yields \(\pm 0.021\), \(\pm 0.024\) and \(\pm 0.071\), in quite good agreement with the uncertainties in the first line of Table 3.

Table 3 Values of the entanglement indicator E in (40) obtained from 200 pseudo-experiments with \(L = 139~\text {fb}^{-1}\) and the kinematical cuts in Table 1. The row labeled as ‘individual presents results from individual determinations of \(C_{kk}\), \(C_{rr}\) and \(C_{nn}\). The row labeled as ‘direct’ corresponds to results obtained measuring either D (at threshold) or \(D_3\) (in the boosted regime)
Full size table

At threshold, the use of \(\beta \) and the direct determination improve the statistical significance by a factor of 1.27, from \(E = 0.560 \pm 0.020\) to \(E=0.678 \pm 0.019\). This is quite remarkable, because it would amount to an increase in luminosity by a factor of 1.6. However, for this measurement the statistics are already large, and it is very likely that the uncertainty will be dominated by systematics, so this improvement may not have a great impact in the total uncertainty. In the boosted regime there is an improvement by a factor of 1.23, equivalent to an increase in luminosity by a factor of 1.5, and brings statistical uncertainties below the 10% level. Provided systematic uncertainties are at the same level, the \(5\sigma \) observation of \(t {\bar{t}}\) entanglement in the boosted regime seems quite feasible. In summary, from Table 3 the two ingredients that can improve the observability of the entanglement are manifest:

  • The application of an upper cut on \(\beta \) (only for the threshold analysis), which reduces the \(q {\bar{q}}\) fraction, enhancing the entanglement and thus the central value of E.

  • The direct measurement of E, using either the D observable (at threshold) or \(D_3\) (in the boosted regime), instead of individually measuring \(C_{kk}\), \(C_{rr}\) and \(C_{nn}\). This direct measurement reduces the statistical uncertainty.

4.2 Observation of CHSH violation

Near threshold, the beamline basis is slightly better to observe a violation of the CHSH inequalities, using as indicator the quantity

$$\begin{aligned} B_1 \equiv |C_{xx} + C_{yy}| - \sqrt{2} > 0 \,. \end{aligned}$$
(42)

(For results in the helicity basis see Appendix A.) Its dependence on \(m_{t{\bar{t}}}\) and \(\cos \theta _\text {CM}\) is shown in Fig. 3, requiring \(\beta \le 0.8\).Footnote 5 This quantity can be determined by a single measurement (given the fact that \(C_{xx} = C_{yy}\) by the symmetry around the beam axis) as proposed in Ref. [9]. However, as we will see in the following, such determination suffers from large statistical uncertainties. The direct determination of the sum \(C_{xx}+C_{yy}\) by using an azimuthal asymmetry of the type \(A_+\) in (37) is much more precise.

In the boosted regime the helicity basis is more convenient, and the most useful indicators, by order of importance, are the quantities

$$\begin{aligned}&B_2 \equiv |C_{rr}-C_{nn}| - \sqrt{2}> 0 \,, \nonumber \\&B_3 \equiv |C_{kk} + C_{rr}| - \sqrt{2} > 0 \,, \end{aligned}$$
(43)

since \(C_{rr}\) and \(C_{nn}\) are the two correlation coefficients with largest modulus and they have opposite sign (see Fig. 1). Their dependence on \(m_{t{\bar{t}}}\) and \(\cos \theta _\text {CM}\) is presented in Fig. 4. We restrict our analysis to \(B_2\). The direct determination of \(C_{rr}-C_{nn}\) is obtained from an azimuthal asymmetry of the type \(A_-\).

Fig. 3
figure 3

Dependence of the indicator \(B_1\) in (42) on \(m_{t{\bar{t}}}\) and \(\theta _\text {CM}\)

Full size image
Fig. 4
figure 4

Dependence of the indicators \(B_{2,3}\) in (43) on \(m_{t{\bar{t}}}\) and \(\theta _\text {CM}\)

Full size image

We use the figure of merit

$$\begin{aligned} S_B = B_{1,2} \times \sqrt{\sigma } \end{aligned}$$
(44)

to devise a simple set of kinematical cuts, either using \(\beta \) or not in the case of the threshold analysis, in order to optimise the statistical significance of \(B_{1,2}\). The sets of cuts are collected in Table 4. For each set of cuts we give the tree-level cross section, as well as the number of expected events with a luminosity of 300 fb\(^{-1}\), assuming a K factor of 1.8 to normalise the total cross section to next-to-next-to-leading order [11] and using a selection and reconstruction efficiency of 0.12. The value of spin correlation coefficients with these kinematical cuts are presented in Table 5. Clearly, for the threshold analysis the beamline basis is better, while the opposite happens for the boosted analysis. Note also that the off-diagonal spin correlation coefficients are quite small; therefore, the estimators \(B_{2,3}\) are very close to the optimal ones. For each set of cuts we estimate the statistical uncertainty in the

Table 4 Kinematical cuts on \(m_{t{\bar{t}}}\) (in GeV), \(\cos \theta _\text {CM}\) and \(\beta \) used to optimise the figure of merit \(S_B\). The fourth column gives the tree-level cross section with the corresponding cuts, and the fifth column the expected number of events after reconstruction (see the text for details)
Full size table
Table 5 Parton-level values of the spin correlation coefficients in the helicity and beamline bases with the kinematical selection in Table 4. In the helicity basis \(C_{kr}=C_{rk}\), and in the beamline basis \(C_{xx}=C_{yy}\). The rest of coefficients are below 0.01
Full size table

measurement of \(B_{1,2}\) by performing \(n=1000\) pseudo-experiments, and present the results for luminosities of 300 and 3000 fb\(^{-1}\) in Tables 6 and 7, respectively. The rows labeled as ‘individual’ give the results obtained from the measurement of individual correlation coefficients (only one of them in the threshold analysis, as proposed in Ref. [9]), while the rows labeled as ‘direct’ give the results from the measurement of azimuthal asymmetries. From error propagation, one estimates for the ‘individual’ measurements an statistical uncertainty of \(4 \sqrt{2/N}\), which yields \(\pm 0.040\), \(\pm 0.052\) and \(\pm 0.15\), respectively, for \(N=19{,}600\), 11, 700 and 1500 in Table 4. The uncertainties quoted in Table 6 are slightly larger, but consistent with these estimations.

Table 6 Values of the indicators \(B_{1,2}\) in (42), (43) obtained from 1000 pseudo-experiments with with \(L = 300~\text {fb}^{-1}\) and the kinematical cuts in Table 4. The row labeled as ‘individual gives the results from individual determinations of spin correlation coefficients. The row labeled as ‘direct’ corresponds to results obtained measuring azimuthal asymmetries (see the text for details)
Full size table
Table 7 The same as Table 6, for \(L = 3000~\text {fb}^{-1}\)
Full size table

Near threshold, we observe a great improvement of the statistical significance, e.g. from \(0.4\sigma \) to \(2.7\sigma \) with 300 fb\(^{-1}\), by the combination of a \(\beta \) cut and the dedicated observable \(A_+\). With 3000 fb\(^{-1}\), the statistical uncertainty of \(B_1\) is at the 10% level. Therefore, provided the systematic uncertainties are at the same level, a \(5\sigma \) observation of the violation of the CHSH inequalities seems feasible. At the boosted regime the statistical uncertainty in \(B_2\) is 17% for 3000 fb\(^{-1}\), thereby allowing for a \(5\sigma \) observation of \(B_2 > 0\).

5 Discussion

In this work we have investigated novel approaches to improve the observability of entanglement and CHSH violation in top pair production at the LHC. The first one is the increase of the gg fraction by a simple upper cut on the \(t {\bar{t}}\) velocity \(\beta \), which enhances the entanglement near threshold. The second one is the use of dedicated observables that directly extract the relevant combinations of spin correlation coefficients \(C_{ij}\) from data, thereby reducing the statistical uncertainty in the measurements.

For simplicity, we have not attempted the optimisation of the kinematical cuts in the \((m_{t{\bar{t}}},\cos \theta _\text {CM})\) plane, nor in the \((m_{t{\bar{t}}},\cos \theta _\text {CM},\beta )\) volume, to achieve the highest possible statistical significance in each case. Instead, we have applied simple rectangular cuts on these variables. Further optimisations of the event selection are quite possible, but out of the scope of this work.

As we restrict ourselves to considering statistical uncertainties, our study is performed at the parton level, without detector simulation nor unfolding from detector-level quantities. In particular, the unfolding has been shown to be accurate enough to obtain precise measurements of spin correlation observables [30, 31] and template methods similar to the one in Ref. [27] could also be exploited. Backgrounds in the dilepton channel have also been shown to be small [30, 31], and have been ignored in our analysis. We have used a fixed CM energy of 13 TeV, even if in Run 3 of the LHC it will be raised to 13.6 TeV, thereby providing slightly larger samples than the ones assumed here.

Fig. 5
figure 5

Dependence of the indicators \(B_{4,5}\) in (45) on \(m_{t{\bar{t}}}\) and \(\theta _\text {CM}\)

Full size image
Table 8 Comparison of entanglement and CHSH violation indicators with Ref. [8]
Full size table

Statistical uncertainties are mainly driven by the number of events obtained after suitable kinematical cuts. We have used an overall efficiency for a typical event selection and \(t {\bar{t}}\) reconstruction consistent with the one found in other works [8]. Possible departures from this reference value would modify the size of all our event samples by the same amount. As the goal of our work is to investigate the improvement brought by the use of \(\beta \) and dedicated observables, our conclusions are quite robust even if the selection and reconstruction efficiency has some departure from the reference value used here.

The methods introduced here to enhance the sensitivity are especially effective when statistical uncertainties are important, namely

  1. (i)

    For the observation of entanglement in the boosted regime;

  2. (ii)

    For the observation of CHSH violation both at threshold and in the boosted regime.

In the boosted region, a cut on \(\beta \) does not provide any advantage, so the central value (both for entanglement and CHSH violation) remains the same, but the statistical uncertainty decreases thanks to the dedicated observables. We find a moderate improvement in the significance for the observation of both phenomena. This strengthens in particular the conclusion of Ref. [8] in the sense that entanglement can be measured with Run 2 data.

On the other hand, the greatest improvement occurs for the measurement of the CHSH violation near threshold. With \(L = 300~\text {fb}^{-1}\), we find that the observable proposed in Ref. [9] only allows for \(0.4\sigma \) statistical significance, while the asymmetry introduced here, together with the use of \(\beta \), enhances the sensitivity to \(2.7\sigma \).

In this work we have not addressed systematic uncertainties, nor it is possible without a full detector simulation. Although these uncertainties will be important, we believe that our proposal for optimisation will be quite useful for future analyses. The application of an upper cut on \(\beta \) selects the events that are more central, where the detector resolution is better, and is not expected to degrade the systematic uncertainty, but on the contrary, it may improve systematics. Concerning the use of dedicated observables, there is no a priori reason why the systematic uncertainties should be larger for them. One would even expect that uncertainties may be smaller. For example, the determination of \(C_{kk}+C_{rr}-C_{nn}\) from its individual terms involves a precise reconstruction of the three axes \(({{\hat{k}}},{{\hat{r}}},{{\hat{n}}})\) (in addition to the lepton momenta) while the determination from \(D_3\) is only sensitive to the reconstruction of the \(({{\hat{k}}}, {{\hat{r}}})\) plane.

In conclusion, the methods outlined here may be quite useful for entanglement measurements in the boosted regime, where already with Run 2 data a measurement seems quite feasible. For CHSH violation more data is required, and \(3 \sigma \) evidence would already be feasible with Run 3.