1 Introduction

The main purpose of experimental high-energy nuclear physics is to investigate the properties of the Quark-Gluon Plasma (QGP). In the “normal” matter, constituents of the visible Universe (partons, i.e., quarks and gluons) are confined within hadrons. In contrast, the QGP is a matter in local thermal equilibrium with quark and gluon degrees of freedom. The partons, not hadrons, define the properties of the QGP. Such a state of matter existed in the early Universe, microseconds after the Big Bang and we can create it for a short while in heavy-ion collisions with high enough energy density. Experiments at the SPS (Super Proton Synchrotron), RHIC (Relativistic Heavy Ion Collider), and the LHC (Large Hadron Collider) demonstrated that the QGP has unique properties [1]. Among others, it behaves like (almost) perfect and the most vortical fluid known so far [2].

The study of partonic matter in a laboratory is a difficult task, since a small droplet of the QGP matter produced in high-energy nuclear collisions only exists for a short time (of the order of 10 fm). Then it cools down and subsequently partons form hadrons which are registered by experiment. Thus, the effects of hadronic phase always accompany the QGP signals.

During the past 25 years, physicists developed a variety of approaches to access the properties of the QGP, for example, studies of the modification of the energy spectra of jets and heavy quarks in heavy-ion collisions in comparison with proton-proton interactions, or analysis of the momentum anisotropy of particles in the final state of the collisions. Here we focus on using charmonium and bottomonium states (quarkonium for short) as a probe of the QGP, and more precisely, on “calibration” of such a probe.

The idea of using the production of a \(J/\psi \)  meson (a bound state of c and \(\overline{c}\) quarks) to study the properties of the QGP was proposed by Matsui and Satz [3]. They demonstrated that the binding potential of c and \(\overline{c}\) quarks will be screened in the partonic matter, which would cause a suppression of \(J/\psi \) meson production (per nucleon-nucleon collision) in heavy-ion reactions with respect to yield in nucleon-nucleon interactions. The same argument applies to other members of the charmonium and bottomonium families [4], however the suppression of a given state depends on the energy density (hence temperature) of the partonic matter. Thus, simultaneous measurement of the production of \(J/\psi \), \(\psi (2s)\), \(\varUpsilon (1S), \varUpsilon (2S),\varUpsilon (3S)\) and other quarkonium states could provide information about thermodynamic properties of the QGP within this paradigm.

This idea of using quarkonium to probe the QGP properties became popular because of its elegance, but the reality of heavy-ion collisions is more complicated. Firstly, there are other possible ways of charmonium and bottomonium interaction with the partonic matter (see the review [5] and the references therein) and there are dynamical effects that one needs to take into account. Specifically, the production rates of \(c\overline{c}\) and \(b\overline{b}\) pairs depend on the parton distribution in a heavy nucleus (which is an initial-state effect) and a bound state (\(c\overline{c}\) or \(b\overline{b}\)) could be destroyed by passing through the “cold” matter of a nucleus present in a collision (this effect is called nuclear absorption). Finally, charmonium or bottomonium could be destroyed in the last, hadronic phase of the reaction.

The production of charmonium and bottomonium states has been extensively studied experimentally in the lastdecades (see for example  [6,7,8,9,10,11,12,13,14,15]), to quantify both effects from the hot partonic and “cold” matter. A plethora of data is available; yet, no theoretical model can describe all results. One source of the problem is the entanglement of processes that affect quarkonium production, which need to be included in such a model. Specifically, the data collected both at the RHIC [16] and at the LHC [13] energies suggest that the interaction of quarkonium with hadrons in the final state is an important factor and it requires attention. We propose a novel experimental approach that provides information about the elastic and inelastic (destructive) interactions of charmonium and bottomonium with hadrons in the final stage of the collision. With the results of such study, we will be one step closer to understanding the quarkonium interaction within the QGP.

We propose to study the correlations at low relative momentum (so-called femtoscopic correlations) of quarkonium with hadrons in proton-proton collisions. The correlations are sensitive to space-time properties of the particle emission region and interactions in the final state; and the existing formalism allows for the calculation of the interaction parameters (see for example [17]). Thus, one can measure the cross section for elastic and inelastic interactions of quarkonium in the hadronic phase. We focus on the \(J/\psi \)-hadron case because \(J/\psi \) is produced copiously in high-energy collisions and its measurement is straightforward, but the reasoning applies to other quarkonium states.

In this paper, in Sect. 2, we briefly review models for \(J/\psi \)-hadron interactions to present the context of our study. Then in Sect. 3, we discuss the theoretical basics of the femtoscopic correlations. We introduce the Lednicky-Lyuboshitz analytical model that links an experimental correlation function to final-state strong interaction parameters, and we outline how to calculate the elastic and inelastic cross section from these parameters. In Sect. 4, we perform a feasibility study for the measurement of the \(J/\psi \)-hadron femtoscopic function and evaluation of the cross section for \(J/\psi \)-hadron interactions for two experiments: STAR at RHIC and LHCb at the LHC, using the data they already collected. The effect of the feed-down from higher charmonium states, non-femtoscopic background and the results are also presented in this section. Then, we discuss the prospects for such measurements in the near future in Sect. 5. Finally, we conclude the paper in Sect. 6.

2 Models of \(J/\psi \) interaction with hadrons in high energy nuclear collisions

We outlined in the previous section that the interaction of charmonium with hadrons in the final state of nuclear collisions (specifically heavy-ion collisions) is an important factor in the interpretation of experiment studies of the QGP. The elastic scattering can change the momentum distribution of \(J/\psi \), thus it mimics the energy loss in the nuclear matter, while the inelastic (destructive) interaction leads to a suppression of the observed yields, thus it resembles the destruction of the quarkonium in the partonic phase.

Those final-state interactions are extremely difficult to quantify because they are convoluted with other effects. In the standard approach, the yields in proton-nucleus (p+A) collisions where the effect occurs are compared to a baseline measurement in proton-proton reactions, and then the results should be checked for compatibility with the phenomenological model (see for example [14, 18, 19]). In the p+A collisions, there are already initial state effects and interactions with nuclear matter, which obscure the final observations.

The comover interaction model [20] is an example of a theoretical calculation, where quarkonium-hadron interactions play an important role. In this model, the probability of inelastic quarkonium-hadron interactions increases as the hadron density increases. In this model, interaction with the medium is quantified by a cross section \(\sigma _{\mathrm {abs}}^{\mathrm {comover}}\) for the breakup of \(J/\psi \) with comoving matter (regardless whether it consists of partons or hadrons). In general, the \(\sigma _{\mathrm {abs}}\) is an external parameter, which has to be fixed, either in comparison with data or in theoretical calculations.

There is a handful of theoretical calculations of the \(\sigma _{\mathrm {abs}}\) parameter, and they vary significantly. For the extended discussion of the models for quarkonium dissociation in hadronic matter, we redirect the reader to the review [21], here we provide a few examples. A calculation based on the meson exchange model in a chiral Lagrangian calculation [22, 23] yields the values of absorption cross section \(0.8{-}3\,\mathrm {mb}\) for \(J/\psi \)-\(\pi \) interaction and \(0.1{-}1\,\mathrm {mb}\) for \(J/\psi \)-\(\rho \). Another approach based on the extended Nambu Jona-Lasinio model [24] yield values of \(0.1{-}1\,\mathrm {mb}\) for \(J/\psi \)-\(\pi \) absorption. These results and conjectures about the corresponding effects in heavy-ion reactions [21] were never verified experimentally. Moreover, the cross section for the interaction of higher charmonium states with hadrons is expected to be much larger [25], which is in agreement with the data [16].

In this paper, we present a method for a direct measurement of these cross sections using quarkonium-hadron femtoscopy. Such a study will provide a good opportunity to test the calculations of charmonium-hadron interactions and improve models for quarkonium production in heavy-ion collisions.

3 \(J/\psi \)-hadron femtoscopy and formalism for extracting the interaction parameters

To measure the space-time characteristics of the particle-emitting region in hadron collisions, the approach of using the correlation of two identical bosons was introduced almost six decades ago [26]. Since then, the method of momentum correlation was refined in the seminal works of Kopylov and Podgoretsky [27,28,29], and extended to non-identical particle pairs (please see [30] for a comprehensive review). The correlations at low relative momentum are also sensitive to Coulomb and strong interactions between particles in a pair [31, 32], thus they provide rich information about collisions.

The femtoscopic correlation function for two particles in general is defined as:

$$\begin{aligned} C(\mathbf{p_{a}},\mathbf{p_{b}}) = \frac{P_{2}(\mathbf{p_{a}},\mathbf{p_{b}})}{P_{1}(\mathbf{p_{a}})P_1(\mathbf{p_{b}})} \end{aligned}$$
(1)

where \(P_{1}\) is the probability of observing a particle with a given momentum \(\mathbf{p_{a}}\) and \(P_{2}\) is the conditional probability of a particle with momentum \(\mathbf{p_{b}}\) being observed if a particle of momentum \(\mathbf{p_{a}}\) is also observed.

According to [30] the Eq. 1 can be rewritten as

$$\begin{aligned} C(k^{*}) = \int d^{3}r^{*} S(r^{*})|\varPsi {(\mathbf {r^{*}},k^{*})}|^{2} \end{aligned}$$
(2)

where \(k^{*}\) is the momentum of a particle in the pair center-of-mass (c.m.) system, and \(S(r^{*})\) is the source function. The \(S(r^{*})\) is a distribution of the relative distance \(r^{*}\) of particles in the pair c.m. system and it contains all space-time information about the emission source. The relative wave function of the particle pair beyond the range of the potential is denoted by \(\varPsi {(\mathbf {r^{*}},k^{*})}\) has the form

$$\begin{aligned} \varPsi (\mathbf {r^{*}},k^{*}) \doteq e^{i\mathbf {k^{*}} \cdot \mathbf {r^{*}}}+ \frac{f^{S}(k^{*})}{r^{*}} e^{-ik^{*} \cdot r^{*}}, \end{aligned}$$
(3)

which represents the stationary solution of the scattering problem.

The correlation function is considered as a square of the wave function \(\varPsi \) averaged over the relative distance vector \({r}^*\) of the emitters in the pair c.m. system and over the pair total spin. It is the asymptotic form of a superposition of the plane and outgoing spherical waves.

The form of \(\varPsi (\mathbf {r^{*}},k^{*})\) is estimated by taking into account the outside range of the strong interaction potential, and the s-wave part of the scattered wave. This allows the internal structure of the strong interaction potential to be neglected and be assumed that it is spherically symmetric. In this case, only the magnitude of the potential is relevant. Then s-wave scattering amplitude in the effective range approximation  [33] at considerably small \(k^*\) values can be written as:

$$\begin{aligned} f^{S}(k^{*}) = \left( \frac{1}{f^{S}_{0}} + \frac{1}{2} d^{S}_{0} k^{*2} - i k^{*}\right) ^{-1}, \end{aligned}$$
(4)

where \(f^{S}_{0}\) is the scattering length and \(d^{S}_{0}\) is the effective radius for a given total spin of 1 (triplet state) or 0 (singlet state). With the assumption that particles are emitted unpolarized, the fraction \(\rho \) of pairs with a given spin are \(\rho _0=1/4\) for pairs in the singlet state and \(\rho _1=3/4\) for the triplet state.

The Lednicky–Lyuboshitz (L–L) analytical model [31] connects the correlation function with final-state strong interaction parameters. It assumes \(\mathbf {r^{*}}\) has Gaussian distribution as follows:

$$\begin{aligned} d^{3}N/d^{3}r^{*} \sim e^{- \mathbf { r^{*} }^{2} /4r_{0}^{2}}, \end{aligned}$$
(5)

where \(r_{0}\) can be considered as the effective radius of the source, then the correlation function can be calculated analytically:

$$\begin{aligned} C(k^*)&= 1 + \sum _S \rho _S\left[ \frac{1}{2}\left| \frac{f^S(k^*)}{r_0}\right| ^2\left( 1-\frac{d_0^S}{2\sqrt{\pi }r_0}\right) \right. \nonumber \\&\quad \left. +\frac{2{\text {Re}}(f^S)(k^{*})}{\sqrt{\pi }r_0}F_1(Qr_0)-\frac{{\text {Im}} (f^S(k^*))}{r_0}F_2(Qr_0)\right] , \end{aligned}$$
(6)

where \(Q=2k^{*}\), \(F_1(z) = \int _0^z dx e^{x^2 - z^2}/z\) and \(F_2(z) = (1-e^{-z^2})/z\). From Eq. 6 one can see that this model relates the observed two-particle correlation function to the source size and the s-wave scattering amplitude.

3.1 Cross section of elastic interactions with the presence of inelastic interactions

Both elastic and inelastic cross sections for \(J/\psi \)-hadron interaction can be calculated from the scattering amplitudes. The scattering amplitudes can in turn be extracted from the experimental data by fitting the \(J/\psi \)-hadron femtoscopic correlation function with the formula in Eq. 6. In the partial wave expansion, the elastic scattering cross section \(\sigma _{el}\) [34] can be calculated with:

$$\begin{aligned} \begin{aligned} \sigma _{el} = 4\pi \sum _{l}(2l+1)|f_{l}|^2 {\mathop {=}\limits ^{l=0}} 4\pi |f^{S}(k^{*})|^2 \end{aligned} \end{aligned}$$
(7)

Where \(f_{l}\) are the scattering amplitudes for each partial wave l, while \(f^{S}(k^{*})\) is the s-wave scattering amplitude from Eq. 4 in the s-wave approximation (\(l=0\)).

If a partial wave is absorbed, which means inelastic scattering is present, then the corresponding S-matrix element \(S_{l}\) satisfies the \(|S_{l}|^{2}<1\) relation. This matrix element can be expressed in terms of \(f_{l}\).

$$\begin{aligned} \begin{aligned} S_{l} = 1+2ik^{*}f_{l} \end{aligned} \end{aligned}$$
(8)

Then the Eq. 8 can be plugged into the formula for inelastic scattering cross section \(\sigma _{inel}\) [34]:

$$\begin{aligned} \sigma _{inel}&= \frac{\pi }{k^{*2}} \sum _{l}(2l+1)(1-|S_{l}|^2)\nonumber \\&= 4\pi \sum _{l}(2l+1) \left( \frac{{\text {Im}}(f_{l})}{k^{*}}-|f_{l}|^{2} \right) \end{aligned}$$
(9)

Again, in the s-wave approximation, this gives:

$$\begin{aligned} \begin{aligned} \sigma _{inel} {\mathop {=}\limits ^{l=0}} 4\pi \left( \frac{{\text {Im}}(f^{S})}{k^{*}}-|f^{S}(k^{*})|^{2} \right) \end{aligned} \end{aligned}$$
(10)

Finally, the total scattering cross section is a sum of both elastic and inelastic cross sections:

$$\begin{aligned} \begin{aligned} \sigma _{tot} = \sigma _{inel}+\sigma _{el} = \frac{4\pi }{k^{*}}{\text {Im}}(f^{S}(k^{*})) \end{aligned} \end{aligned}$$
(11)

It has to be noted that in the case of singlet and triplet states, each cross section component is a sum of cross sections weighted by the spin fraction \(\rho _{i}\) for the corresponding state as:

$$\begin{aligned} \begin{aligned} \sigma = \sum _{i}\rho _{i}\sigma _{i} \end{aligned} \end{aligned}$$
(12)

We tested these formulas for the parameters obtained in the case of \(p-\Lambda \) interactions in the Effective Field Theory approach [35] as shown in Fig. 1. The calculations describe the data reasonably well, however there are small discrepancies as \(k^{*}\) increases. These discrepancies are due to the limitations of the s-wave approximation which is valid mostly at low-\(k^{*}\).

Fig. 1
figure 1

Cross sections vs. \(k^{*}\) calculated in the s-wave approximation using parameters from  [35], compared to the experimental data  [36] for \(p-\Lambda \) interactions

It is worth to mention that the requirement on the matrix element \(|S_{l}|^{2}<1\) for inelastic scattering leads to a constrain on the imaginary part of the scattering length. The \({\text {Im}} f_{0}\) must be positive.

When \({\text {Im}} f_{0} = 0\,\mathrm {fm}\), the inelastic cross section vanishes. Figure  2 shows examples of cross section calculations for different values of \({\text {Re}} f_{0}\), \({\text {Im}} f_{0}\), and \({\text {Re}} d_{0}\).

Fig. 2
figure 2

The sensitivity of elastic and inelastic cross section for \(J/\psi \)-hadron interaction to the parameters of interaction: the real and imaginary part of scattering length

3.2 Range of applicability of the method

Two factors define the range of applicability of the Lednicky-Lyuboshitz model. The first one is the \(k^{*}\) range (\(k^{*} < 300\) MeV/c), where femtoscopic correlations are observed. The other is the applicability of approximations in the model, and Fig. 1 helps to address this question. Figure 1 shows experimental data for proton-\(\Lambda \) scattering cross section compared to our calculations done with the L–L model (see details in the next section). The model describes the data for \(k^{*} < 0.25\) GeV/c.

In this study, we used the Lednicky–Lyuboshitz analytical model because we can control the input parameters and evaluate the precision of their extraction in an experiment. There is another tool, “Correlation Analysis Tool using the Schrödinger equation” (CATS) [37], which provides an exact solution for any source and interaction potential. One could apply CATS to extract interaction parameters and use them to calculate the cross sections.

Another question is the energy range of the interaction that one can probe with femtoscopy. For known momenta of a hadron and quarkonium, the center-of-mass (c.m.) energy of the interaction is defined by the range of relative momentum \(k^*\), where the femtoscopic correlations are observed. In our simulation, we consider \(J/\psi \)-hadron pairs with \(k^* < 0.3\) GeV/c and the c.m. energy is \(4.3{-}4.6\) GeV. Thus, femtoscopy covers a narrow range of energy of the interaction. For instance, it does not address the case when a hadron with high momentum scatters of quarkonium with low momentum. Given there is only a handful of relevant experimental data on the quarkonium-hadron interactions [38, 39], one needs to rely on the models for the estimates of c.m. energy dependence of the quarkonium-hadron elastic and inelastic cross section. We argue that the measurement we introduce, even though at a relatively narrow energy range, will help to constrain the parameters of theoretical calculations and improve the models.

We propose to measure the femtoscopic correlation with charged hadrons, which are easily detected in modern high-energy experiments, but this method provides input for the interaction of quarkonium with neutral hadrons as well. The strong interaction has a property approximate flavor symmetry. Thus, there is no significant difference between the strong interaction of quarkonium with charged and neutral hadrons within a given isospin triplet, for example \(\pi ^{\pm }\) and \(\pi ^0\). Interactions with other neutral mesons (\(\rho , \omega \)) could also play a role, as discussed in [22], but their yields are significantly lower compared to pions and kaons. Therefore, we defer such a study to future work. In principle, femtoscopy allows one to study the interactions of quarkonium with any hadron if one can measure it in an experiment with sufficiently high statistics and low background.

4 Feasibility study for \(J/\psi \)-hadron femtoscopy at the STAR and the LHCb experiments

In this section, we investigate the feasibility of measurements of \(J/\psi \)-hadron femtoscopic correlation function, hence the measurements of the \(J/\psi \) breakup cross section for the studies by the STAR and the LHCb experiments in pp collisions at the center-of-mass energy of 500 GeV and 8 TeV respectively. We consider that the integrated luminosity available at LHCb is \(L_{int}=2082\) \(\mathrm{pb}^{-1}\) from 2012 data taking period [40], and STAR collected data with \(L_{int} = 400\) \(\mathrm{pb}^{-1}\) in 2017 data [41]. In addition, we review the case of STAR collecting f \(L_{int}= 2.2\) \(\mathrm{fb}^{-1}\) which is foreseen for 2023 data taking campaign at  \(\sqrt{s} = 510 \) GeV

4.1 Simulation setup

For this study, the simulation samples were generated with Pythia 8.2 configured with the parameters within the LHCb and STAR experiments acceptance as following:

LHCb setup: The event samples of pp collisions with the energy of 8 TeV, hadron \(p_{T}>\) 0.5 GeV/c, and hadron pseudorapidity 2 \(<\eta<\) 5 were considered for the LHCb acceptance. We selected \(J/\psi \) within the rapidity range of \(2< y^{\mathrm {J}/\psi }< 5\) for the reconstruction via \(J/\psi \rightarrow \mu ^+\mu ^-\) decay channel. We assumed that the hadron and \(J/\psi \) reconstruction efficiency have approximately constant values of 0.96 and 0.25, respectively, which is inspired by the reported performance of the LHCb experiment [42, 43].

STAR setup: The event samples of pp collisions with the energy of \(\sqrt{s} = 500 \) GeV, hadron \(p_{T}>\) 0.2 GeV/c, and hadron pseudorapidity \(|\eta |<\) 1 were considered. We analyzed \(J/\psi \) produced at mid-rapidity: with \(|y^{\mathrm {J}/\psi }|< 0.4\) for \(J/\psi \rightarrow \mu ^+\mu ^-\) and \(|y^{\mathrm {J}/\psi }|< 1\) for the \(J/\psi \rightarrow e^+e^-\) decay channel [44]. The hadron and \(J/\psi \) reconstruction efficiency are taken from  [44, 45] and they are applied as a function of transverse momentum.

From the simulated samples, the \(k^{*}\) distribution for the given efficiency and acceptance of each experiment were modeled. These distributions include the non-femtoscopic background. The sources of non-femtoscopic correlations are any resonances that decay to the same particles, which in our case are mostly various B mesons decay to \(J/\psi \)-hadron pairs.

The \(k^*\) distributions modeled for LHCb-like and STAR-like experiments are shown in Figs. 3 and 4 respectively, with and without applying the efficiencies in the top plots. We observed, that there is no significant difference between the shape of \(k^*\) distributions for \(J/\psi \rightarrow \mu ^+\mu ^-\) and \(J/\psi \rightarrow e^+e^-\) reconstruction at STAR. The bottom panels in Figs. 3 and 4 show the ratio of \(J/\psi \)-hadron pairs from the same parent to all pairs, which indicates that at low \(k^*\) the non-femtoscopic background (\(J/\psi \)-hadron pairs from resonance decays) is negligible.

We calculated the expected number of \(J/\psi \) mesons for the feasibility studies by taking the \(J/\psi \) counts reported by the LHCb and the STAR experiments for \(L_{int} = 18.4\) \(\mathrm{pb}^{-1}\) and \(L_{int} = 22.1\) \(\mathrm{pb}^{-1}\), respectively, and scaling them up to the expected integrated luminosity.

Fig. 3
figure 3

Top panel: The distribution of \(k^*\) for the LHCb acceptance with and without applying the efficiencies. Bottom panel: the ratio of \(J/\psi \)-hadron from the same parent (SP) to all pairs

Fig. 4
figure 4

Top panel: The distribution of \(k^*\) for the STAR acceptance for \(J/\psi \rightarrow e^+e^-\) reconstruction with and without applying the efficiencies. Bottom panel: the ratio of \(J/\psi \)-hadron from the same parent (SP) to all pairs

The number of \(J/\psi \)-hadron pairs expected for the femtoscopic measurement is calculated as following:

$$\begin{aligned} \langle N_{J/\psi -h}\rangle = N_{J/\psi } \times \langle N_{h}\rangle \end{aligned}$$
(13)

where \(N_{J/\psi }\) and \(\langle N_{h}\rangle \) are the \(J/\psi \) raw yield and the estimated mean number of charged hadrons observed in the events that contain \(J/\psi \). We used Pythia to calculate these values and we obtained \(\langle N_{h}\rangle = 5.31 \pm 0.01\) for LHCb-like experiment and \(\langle N_{h}\rangle = 4.83 \pm 0.01\) for the STAR acceptance. In this approach we assumed that at most one \(J/\psi \) is observed in an event. Finally, since we use the range \(k^*<\) 0.4 GeV/c, we applied a correction factor to calculate the \(\langle N_{J/\psi -h}\rangle \) usable for the femtoscopy.

Table 1 shows the estimated number of \(J/\psi \) and \(J/\psi \)-hadron pairs for each of the considered cases. These expected number of \(J/\psi \)-hadron pairs and \(k^{*}\) distribution are used for producing the pseudo-experimental femtoscopic correlation function \(C(k^{*})\). For STAR, we limited our feasibility study to the \(J/\psi \rightarrow e^+e^-\) channel because the \(J/\psi \) yields are significantly higher than for \(J/\psi \rightarrow \mu ^+\mu ^-\).

Table 1 The expected number of \(J/\psi \)-hadron pairs \(\langle N_{J/\psi -h} \rangle \) for LHCb-like  [40, 46] and STAR-like  [41, 44] experiments for the data samples collected by LHCb in 2012 data taking period; and by STAR in 2017, and the foreseen run in 2023 [41]. To obtain the \(N_{J/\psi }\) for each experiment, we considered the measured raw \(J/\psi \) yield in data set with a given integrated luminosity, and scaled the yields up to the total available integrated luminosity of the corresponding experiment

We model the \(C(k^{*})\) according to the L-L analytical model using several sets of parameters listed in Table 2. The choice of given values is motivated by experimental results for femtoscopic measurements [47,48,49]. We varied the effective radius and the scattering length of the potential within \(d_{0}\epsilon [0,1\ \mathrm{fm}]\) and \(f_{0}\epsilon [0,2\ \mathrm{fm}]\), to assess a precision of determination of these interaction parameters for different amplitudes and shapes of the correlation function.

To estimate the statistical precision expected from a measurement of interaction parameters via femtoscopy, pseudo-experimental correlation functions were generated for the LHCb-like and STAR-like experiments for a given set of input parameters. We sampled the \(k^{*}\) distribution at \(k^{*}< 0.4\) GeV/c and created a distribution with weights defined by \(C(k^{*})\) with a given set of input parameters from Table 2. The number of samples is given by the expected yield of \(J/\psi \)-hadron pairs. Figures 5 and 6 show the pseudo-experimental \(C(k^{*})\) with statistical uncertainties for LHCb- and STAR-like experiments.

Finally, we fit the correlation functions with the L-L model to extract the scattering length and the effective range. This way we evaluate the statistical uncertainties expected for these parameters in a given experiment, and we can assess the feasibility of such a study. Since the fit is challenging, we made a few simplifications. First, we assume that one will not be able to differentiate between singlet and triplet states, so we assume they are the same. Then, in the fit we fixed \({\text {Im}}(d_0) = 0\). Finally, we constrain \(r_0 = 1.25\) fm, which means that the average emission distance is the same as observed in the study of hadron-hadron femtoscopic correlations [47,48,49].

Table 2 Parameters of the L-L model which were used for generating femtoscopic correlation functions. For each parameter set we assume \(r_{0}=1.25\) fm and \({\text {Im}}(d_{0}^{S})=0\)
Fig. 5
figure 5

The pseudo-experimental correlation function for the LHCb-like experiment the plot a for the first three sets and plot b for the second three sets of parameters from Table 2, and their fit

The source width \(r_0\) changes with energy, which will modify slightly the correlation signal. In the actual experimental study, the value of \(r_0\) needs be adjusted for a given reaction. If statistics allows, \(r_0\) can be treated as a free parameter in the fit, as it was done in [47]. If not, then the uncertainty on the assumed value of \(r_0\) would be a source of systematic uncertainty. The values of \(r_0\) are known with good precision for different pairs and collision systems [48, 50], therefore we expect such uncertainty to be small. Besides, the assumption of the Gaussian shape of the \(\mathbf {r^{*}}\) distribution could be a source of systematic uncertainty since other models are considered in the literature.

Fig. 6
figure 6

The pseudo-experimental correlation function for the STAR-like experiment the plot a for the first three sets and plot b for the second three sets of parameters from Table 2, and their fit

Table 3 Interaction parameters extracted from the fit of the L-L model to femtoscopic correlation functions simulated for LHCb-like and STAR-like experiments. The parameters \(r_{0}\) and \({\text {Im}}(d_{0}^{S})\) are fixed to 1.25 fm and 0 respectively

4.2 Effect of the feed-down from higher states of charmonium

The experimental evidence of charmonium production on the \(J/\psi \), \(\psi (2s)\) and \(\chi _{c}\) cross section  [51] indicates that \(\chi _{cJ}\) and \(\psi (2s)\) feed-down fraction to \(J/\psi \) production rates by considering all the correlation and uncertainties are \((25.3 \pm 1.8)\%\) and \((7.5 \pm 0.3)\%\) respectively. The analysis [51] shows that \((67.2 \pm 1.9)\%\) of the prompt \(J/\psi \) yield is due to the directly produced meson. According to the 2020 Review of Particle Physics  [52], the \(\chi _{cJ}\) to \(J/\psi \) happens via radiative decay. Although \(\psi (2s)\) decays to \(J/\psi \) + \(\pi ^{+}\pi ^{-}\) and \(J/\psi \) + \(\pi ^{0}\pi ^{0}\) has the fraction around 52\(\%\), for \(J/\psi \)+\(\eta \) around 3\(\%\) and for \(J/\psi \)+\(\pi ^{0}\) around 0.0012\(\%\), in the studies on the feed-down fraction of \(\psi (2s)\)  [51] for lower \(p_{T}\) spectrum of the mother particle, it has been measured around 7.5\(\%\).

In general, radiative decays complicate the interpretation of the measurements, but do not generate a background in the proposed study. However, \(\psi (2s)\) feed-down should be taken into consideration as a possible source of non-femtoscopic background.

4.3 Non-femtoscopic background

The source of non-femtoscopic correlations, that is the background in our study, are the resonances which decay to \(J/\psi \) + hadron pairs. Such candidates are B-mesons (the branching ratio of decay to \(J/\psi \) + anything \((1.094\pm 0.032)\%\)) and the excited state of \(J/\psi \) such as \(\psi (2s)\). We use our Pythia simulations to estimate this effect. Figures 3 and 4 (bottom plots) show that the low \(k^{*}\) region is not significantly affected by resonance decays. The feed down from B mesons is visible in the high-\(k^{*}\) range, outside of the region of our interests.

4.4 Results

Fig. 7
figure 7

Cross sections vs. \(k^{*}\) calculated with the parameters obtained form the fit for LHCb-like experiment the set 3 (a) and set 6 (b). The uncertainties on the plots include statistical uncertainty from the fit

Figures 5 and 6 show the simulated correlation functions for STAR-like and LHCb-like experiments for \(L_{\text {i}nt} = 2.2 \, \mathrm{fb}^{-1}\) and \(L_{\text {i}nt} = 2.1 \, \mathrm{fb}^{-1}\), respectively; together with fits of the L–L model. Table 3 shows the scattering length and the effective range which we extracted from the fits in our feasibility studies, including the case for a STAR-like detector for \(\sqrt{s} = 500 \) GeV and \(L_{\text {i}nt} = 400 \, \mathrm{pb}^{-1}\).

All the results in the Table 3 are consistent with the input values. The precision registered for STAR-like detector and the data collected in 2017 (\(L_{\text {i}nt} = 400 \, \mathrm{pb}^{-1}\)) is not sufficient for a fruitful study. However, we expect that such a measurement will yield useful results for data taking campaign planned at RHIC at 2023. With the expected \(L_{\text {i}nt} = 2.2 \, \mathrm{fb}^{-1}\), the relative uncertainties on scattering length are about \(10{-}20\%\). In the case of the LHCb-like detector, the precision of the obtained scattering length is better than 1% for almost all considered cases. In general, the stronger the correlation effect, the better precision of the fit results.

We use the obtained parameters to calculate the cross section for \(J/\psi \)-hadron interactions. Figure 7 shows examples of elastic, inelastic, and total cross section as a function of \(k^{*}\). It demonstrates that the LHCb experiment can measure with a good precision the elastic and inelastic (break-up) \(J/\psi \)-hadron cross sections using the approach we propose and the data it has already collected.

5 Prospects for \(J/\psi \)-hadron femtoscopic measurements

Besides the data sets that have already been studied in this paper, other data sets with high enough \(J/\psi \) yield also can be considered for the study of \(J/\psi \)-hadron femtoscopic correlations. In the case of the LHCb experiment, one can add the data recorded at \(\sqrt{s} = 13 \) TeV with \(L_{int}\) of 1.4 \(\mathrm{fb}^{-1}\) with \(J/\psi \) yield of 2 million [53] to our study and enhance the statistics. For the future runs of the LHCb experiment, the \(L_{int}\) at the end of Run 3 and Run 4 [54] will be up to 23 \(\mathrm{fb}^{-1}\) and 50 \(\mathrm{fb}^{-1}\) respectively. And in the year 2030, the upgrade of the LHCb detector [55] will allow it to run at higher luminosity up to 300 \(\mathrm{fb}^{-1}\). It is worth noting that the number of quarkonium states per unit of integrated luminosity in Run 2 was higher by a factor of 5 both due to the higher cross-section at 13 TeV and to improvements in the event selection during data taking. Starting from Run 3, pp collision rate at LHCb will increase by a factor of 5 as well [56, 57]. As for the CMS experiment, the expected \(L_{int}\) at the end of Run 3 is up to 300 \(\mathrm{fb}^{-1}\)  [58]. The expected high-quality data samples should allow for more differential studies and measurement of femtoscopic correlations of \(J/\psi \) with identified hadrons (\(J/\psi \)-\(\pi ^\pm \), \(J/\psi \)-proton etc.) For similar correlation studies for \(\Upsilon \)-hadron pairs, for the same data set as we used in this paper for LHCb experiment  [46] within the \(L_{int}=2082\) \(\mathrm{pb}^{-1}\) the expected number of \(\Upsilon \) is around 1.8 million. For Run 3 and Run 4, this number can be scaled to 18 and 36 million respectively, which is more than the number of \(J/\psi \) in STAR for future data campaign in 2023. Therefore, \(\Upsilon \)-hadron correlation studies will be feasible within the LHCb experiment. Moreover, the CMS experiment is expected to register around 1.7 million of \(\Upsilon \) in the LHC Run 3 [59], providing another opportunity for \(\Upsilon \)-hadron femtoscopy.

6 Conclusion

We proposed an experimental method to study the elastic and inelastic interaction of charmonium and bottomonium with hadrons. Quantitative understanding of these processes is important for the correct interpretation of quarkonium production measurements in heavy-ion collisions and for using quarkonium to probe the properties of hot and dense nuclear matter. The proposed approach is straightforward and the experiments employed a similar strategy to study final-state interactions with success.

The method uses the femtoscopic correlation function and the Lednicky-Lyuboshitz analytical model to extract the scattering length and the effective range of the quarkonium-hadron interaction at low relative momentum. We demonstrated that such a measurement is already feasible at the LHCb experiment with pp data at \(\sqrt{s} = 8 \) TeV collected in 2012, and it is within a reach of the STAR experiment in 2023, when a data sample of pp collisions at \(\sqrt{s} = 510 \) GeV with luminosity of 2.2 \(\mathrm{fb}^{-1}\) is planned.

Our feasibility study showed that LHCb can already measure both elastic and inelastic (break-up) cross sections in a hadronic matter as a function of relative momentum \(k^{*}\). Such a study is more viable for LHCb future data taking campaigns when the expected luminosity will be enhanced.