1 Introduction

The understanding of Particle Physics has been challenged and improved by the recent experimental results obtained by the LHC, the Pierre Auger and IceCube Neutrino Observatories [1]. In particular, in recent years, IceCube measured the astrophysical and atmospheric neutrinos fluxes at high energies [2,3,4] and different collaborations from the LHC performed several analyses of the heavy meson production at high energies and forward rapidities [5,6,7,8]. Such distinct sets of data are intrinsically related, since the description of the heavy meson production at the LHC and higher center of mass energies is fundamental to make predictions of the prompt neutrino flux [9], which is expected to dominate the atmospheric \(\nu \) flux for large neutrino energies [10, 10]. An important question, which motivates the present study, is whether the current and future IceCube data can shed light on charm production at the LHC and vice-versa and in particular on the intrinsic charm in the nucleon.

Fig. 1
figure 1

Representation of the ingredients needed to estimate the prompt neutrino flux at the detector level

In order to derive realistic predictions of the prompt atmospheric neutrino flux at the detector level we should have theoretical control of the description of several ingredients (see Fig. 1): the incident cosmic flux, the charm production, its hadronization, the decay of the heavy hadrons, the propagation of the associated particles through the atmosphere and the neutrino interaction (see e.g. Refs. [12,13,14,15,16,17,18,19,20,21,22,23]). As demonstrated in our previous study [9], to address the production of high-energy neutrinos (\(E_{\nu }>\) 10\(^5\) GeV), it is fundamental to precisely describe the charmed meson production at very high energies and large forward rapidities. This aspect motivated the development of new and/or more precise approaches to describe the perturbative and nonperturbative regimes of the Quantum Chromodynamics (QCD) needed to describe the charmed meson production in a kinematical range beyond that reached in hadronic collisions at the LHC. For this new kinematical range, some topics are theme of intense debate: (a) the presence (or not) of intrinsic heavy quarks in the hadronic wave function [24,25,26,27], characterized by a large value of the longitudinal momentum fraction of beam nucleon momentum; (b) the validity of the collinear factorization at high energies [28,29,30,31], since it disregards the transverse momentum of the incident particles; and (c) the presence (or not) of nonlinear (saturation) effects on the description of the QCD dynamics at high energies [32,33,34,35,36], which are expected to contribute at high energies due to the high partonic density predicted by linear DGLAP or BFKL evolution equationsFootnote 1; (d) the impact of subleading fragmentation of light partons on heavy meson production at high energies and very forward rapidities and its consequences for prompt neutrino flux [23, 37]. Such questions naturally arise due to the fact that in the calculation of the prompt neutrino flux at high energies, the main contribution for the charm production cross section comes from partons with very small (large) values of x in the hadrons that constitute the atmosphere (incident cosmic ray flux). Recently, two of us have presented in Ref. [38] a comparative study of the charm production at large rapidities considering the collinear, hybrid and \(k_T\)-factorization approaches taking into account the presence of intrinsic charm in the proton wave function with parton distributions that are solutions of linear and nonlinear evolution equations. One of the goals of this paper is to extend the analysis performed in Ref. [38] and derive associated prompt neutrino fluxes at high energies. In particular, we shall estimate the impact of the intrinsic charm-initiated subprocess and/or saturation effects on the predictions for the prompt neutrino flux. Another more ambitious goal is to verify whether the recent IceCube data for the prompt \(\nu _{\mu }\) flux allow to derive an upper bound for the probability of finding a charm quark–antiquark pair in the proton wave function, which is one of the main uncertainties in the modelling of the intrinsic charm. A similar goal was also present in the analyses performed in Refs. [16, 18]. However, our study differs from these previous analyses in several aspects. Our predictions for the \(x_F\) distributions will be derived using a framework that successfully describes the LHC data, with the main input being the parton distribution functions which were derived using the world data. In these previous studies, the \(x_F\) distribution was fitted using the old data for the D and \(\Lambda _c\) production, with the normalization being a parameter free. Moreover, the energy dependence of the intrinsic charm contribution was assumed to follow the inelastic cross section, which is dictated by soft processes. In contrast, in our approach, such contribution is calculated perturbatively, which implies a steeper energy dependence. Finally, our predictions are calculated using a unified approach for the \(gg \rightarrow c {\bar{c}}\) and \(gc \rightarrow gc\) mechanisms, which we believe to be more realistic to treat the charm production at central and forward rapidities.

The paper is organized as follows. In the next section a brief review of the formalism needed to estimate the prompt \(\nu _{\mu }\) flux is presented. In particular, we discuss the Z-moment method [39], the hybrid approach for production of \(c/{{\bar{c}}}\) quarks/antiquarks and the main inputs and underlying assumptions of our calculations. In Sect. 3, we shall present our predictions for the Feynman \(x_F\) distribution and for the prompt flux considering different charm production mechanisms and different models for the unintegrated gluon distribution. Moreover, the prompt flux is estimated assuming different amounts for the probability of finding an intrinsic charm component in the nucleon and the predictions are compared with the recent IceCube data. Finally, in Sect. 4 we shall summarize our main results and formulate conclusions.

Fig. 2
figure 2

A schematic illustration of the IceCube experiment

2 Formalism

A schematic illustration of the IceCube experiment is shown in Fig. 2. Neutrinos are detected through the Cherenkov light emitted by secondary particles produced in neutrino-nucleon interactions in or around the detector. Although primarily designed for the detection of high-energy neutrinos from astrophysical sources, denoted cosmic neutrino in Fig. 2, IceCube can also be used for investigating the atmospheric neutrino spectrum.Footnote 2 The atmospheric neutrinos are produced in cosmic-ray interactions with nuclei in Earth’s atmosphere [10, 11]. The low-energy neutrinos (\(E_{\nu }\lesssim 10^5\) GeV) arise from the decay of light mesons (pions and kaons) and the associated flux is denoted as the conventional atmospheric neutrino flux [41]. On the other hand, for larger energies, it is expected that the prompt atmospheric neutrino flux associated with the decay of hadrons containing heavy flavour quarks/antiquarks become important [39]. One has that the flux of conventional atmospheric neutrinos is a function of the zenith angle, since horizontally travelling mesons have a much higher probability to decay before losing energy in collisions, which implies a harder conventional neutrino spectrum of horizontal events compared to vertical events. In contrast, heavy mesons decay before interacting and follow the initial spectrum of cosmic rays more closely, being almost independent of the zenith angle in the neutrino energy range probed by the IceCube (see e.g. Ref. [42]). As discussed in the Introduction, the calculation of the prompt atmospheric neutrino flux at the detector level depends on the description of the production and decay of the heavy hadrons as well as the propagation of the associated particles through the atmosphere (see Fig. 1). Following our previous studies [9, 23], we will estimate the expected prompt neutrino flux in the detector \(\phi _{\nu }\) using the Z-moment method [39], which implies that \(\phi _{\nu }\) can be estimated using the geometric interpolation formula

$$\begin{aligned} \phi _{\nu } = \sum _H \frac{\phi _{\nu }^{H,low} \cdot \phi _{\nu }^{H,high}}{\phi _{\nu }^{H,low} + \phi _{\nu }^{H,high}} \,\,. \end{aligned}$$
(1)

where \(H = D^0, D^+, D_s^+\), \(\Lambda _c\) for charmed hadrons and \({\phi _{\nu }^{H,low}}\) and \({\phi _{\nu }^{H,high}}\) are solutions of a set of coupled cascade equations for the nucleons, heavy mesons and leptons (and their antiparticles) fluxes in the low- and high-energy ranges, respectively. They can be expressed in terms of the nucleon-to-hadron (\(Z_{NH}\)), nucleon-to-nucleon (\(Z_{NN}\)), hadron-to-hadron (\(Z_{HH}\)) and hadron-to-neutrino (\(Z_{H\nu }\)) Z-moments, as follows [39]

$$\begin{aligned} \phi _{\nu }^{H,low}= & {} \frac{Z_{NH}(E) \, Z_{H\nu }(E)}{1 - Z_{NN}(E)} \phi _N (E,0) \,, \end{aligned}$$
(2)
$$\begin{aligned} \phi _{\nu }^{H,high}= & {} \frac{Z_{NH}(E) \, Z_{H\nu }(E)}{1 - Z_{NN}(E)}\frac{\ln (\Lambda _H/\Lambda _N)}{1 - \Lambda _N/\Lambda _H}\nonumber \\&\times \frac{m_H c h_0}{E \tau _H} f(\theta ) \, \phi _N (E,0) \,, \end{aligned}$$
(3)

where \(\phi _N(E,0)\) is the primary flux of nucleons in the atmosphere, \(m_H\) is the decaying particle’s mass, \(\tau _H\) is the proper lifetime of the hadron, \(h_0 = 6.4\) km, \(f(\theta ) \approx 1/\cos \theta \) for \(\theta < 60^o\), and the effective interaction lengths \(\Lambda _i\) are given by \(\Lambda _i = \lambda _i/(1 - Z_{ii})\), with \(\lambda _i\) being the associated interaction length (\(i = N,H\)). For \(Z_{H\nu }\), our treatment of the semileptonic decay of D-hadrons follows closely Ref. [17]. In particular, we assume the analytical decay distributions \(H \rightarrow \mu \nu _{\mu }X\) obtained in Ref. [40] and use the decay branching ratios reported in the most recent PDG [1]. For a detailed discussion of the cascade equations, see e.g. Refs. [12, 39]. Assuming that the incident flux can be represented by protons (\(N = p\)), the charmed hadron Z-moments are given by

$$\begin{aligned}&Z_{pH} (E)\nonumber \\&\quad \approx \int _0^1 \frac{dx_F}{x_F} \frac{\phi _p(E/x_F)}{\phi _p(E)} \frac{1}{\sigma _{pA}(E)} \frac{d\sigma _{pA \rightarrow H}(E/x_F)}{dx_F} \,\,, \end{aligned}$$
(4)

where E is the energy of the produced particle (charmed meson), \(x_F\) is the Feynman variable, \(\sigma _{pA}\) is the inelastic proton-Air cross section and \(d\sigma /dx_F\) is the differential cross section for the charmed meson production. Following previous studies [12,13,14,15,16,17,18,19,20,21,22,23], we will assume that \(A = 14\), i.e. we will take the \(^{14}N\) nucleus as the most representative element in the composition of the atmosphere. For this value of the atomic mass number, it is a reasonable approximation to assume that \(\sigma _{pA \rightarrow charm} \approx A \times \sigma _{pp \rightarrow charm}\). Surely a more refined analysis of these two aspects is possible but would shadow our discussion of the selected issues. For \(\sigma _{pA}\) we will assume the prediction presented in Ref. [43] (for a more detailed discussion see Ref. [44]).

The transition from quarks to hadrons in our calculations is done within the independent parton fragmentation picture (see e.g. Ref. [57]). It is done assuming that the hadron pseudorapidity is equal to parton pseudorapidity and only momenta of hadrons are reduced compared to the parent partons. In such an approximation the charmed meson \(x_F\)-distributions at large \(x_F\) can be obtained from the charm quark/antiquark \(x^{c}_{F}\)-distributions as:

$$\begin{aligned} \frac{d \sigma _{pp \rightarrow H} (x_{F})}{d x_{F}} = \int _{x_{F}}^{1} \frac{dz}{z} \frac{d \sigma _{pp \rightarrow charm} (x^{c}_{F})}{d x^{c}_{F}} D_{c\rightarrow H}(z), \end{aligned}$$
(5)

where \(x^{c}_{F} = x_{F}/z\) and \(D_{c\rightarrow H}(z)\) is the relevant fragmentation function (FF). Here, in the numerical calculations we take the traditional Peterson FF [58] with \(\varepsilon = 0.05\). The resulting meson distributions are further normalized by the proper fragmentation probabilities. As in Ref. [9], we assume that \(f_{D^0} = 0.565\), \(f_{D^+} = 0.246\), \(f_{D_s^+} = 0.080\) and \(f_{\Lambda _c} = 0.094\). We have checked numerically that in the case of forward meson production, our predictions of the fragmentation model with the \(x_{F}\) being the scaling variable are fully compatible with other possible prescriptions within this approach, including the three-momentum p, energy E or light-cone momentum \(p^{+}=(E+p)\) (see a discussion in Ref. [57]).

Fig. 3
figure 3

A sketch of the a \(gg^* \rightarrow c{\bar{c}}\) and b \(cg^* \rightarrow cg\) production mechanisms in pp-interactions within the hybrid model

As discussed in Ref. [38], the cross section for the charm production at large forward rapidities, which is the region of interest for estimating the prompt \(\nu _{\mu }\) flux [9], can be expressed as follows

$$\begin{aligned}&d \sigma _{pp \rightarrow charm} \simeq d \sigma _{pp \rightarrow charm}(gg \rightarrow c {\bar{c}}) \nonumber \\&\quad + d \sigma _{pp \rightarrow charm}(cg \rightarrow cg) \,\,, \end{aligned}$$
(6)

where the first and second terms represent the contributions associated with the \(gg \rightarrow c {\bar{c}}\) and \( cg \rightarrow cg \) mechanisms, with the corresponding expressions depending on the factorization scheme assumed in the calculations. In Ref. [38], a detailed comparison between the collinear, hybrid and \(k_T\)-factorization approaches was performed and it was demonstrated that the contribution associated with the \(q {\bar{q}} \rightarrow c {\bar{c}}\) channel is negligible. In what follows, we will focus on the hybrid factorization model, which is based on the studies performed also in Refs. [28,29,30,31]. Such a choice is motivated by: (a) the theoretical expectation that the collinear approach, largely used in previous calculations of \(\phi _{\nu }\), breaks down at very small-xFootnote 3 [29, 31]; and that (b) the \(k_T\)-factorization approach reduces to the hybrid model in the dilute-dense regime, which is the case in the charm production at very forward rapidities, where we are probing large (small) values of x in the projectile (target). In this approach, the differential cross sections for \(gg^* \rightarrow c{\bar{c}}\) and \(cg^* \rightarrow cg\) mechanisms, sketched in Fig. 3, are given by

$$\begin{aligned}&d \sigma _{pp \rightarrow charm}(gg \rightarrow c {\bar{c}}) \nonumber \\&\quad = \int dx_1 \int \frac{dx_2}{x_2} \int d^2k_t \, g(x_1,\mu ^2) \, {{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2) \, d{\hat{\sigma }}_{gg^* \rightarrow c{\bar{c}}} \nonumber \\ \end{aligned}$$
(7)

and

$$\begin{aligned}&d \sigma _{pp \rightarrow charm}(cg \rightarrow cg) \nonumber \\&\quad = \int dx_1 \int \frac{dx_2}{x_2} \int d^2k_t \, c(x_1,\mu ^2) \, {{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2) \, d{\hat{\sigma }}_{cg^* \rightarrow cg} \,\,, \nonumber \\ \end{aligned}$$
(8)

where \(g(x_1,\mu ^2)\) and \(c(x_1,\mu ^2)\) are the collinear PDFs in the projectile, \({{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2)\) is the unintegrated gluon distribution (gluon uPDF) of the proton (nucleon) target, \(\mu ^2\) is the factorization scale of the hard process and the subprocesses cross sections are calculated assuming that the small-x gluon is off mass shell and are obtained from a gauge invariant tree-level off-shell amplitude. In our calculations \(c(x_1,\mu ^2)\), similarly \({{\bar{c}}}(x_1,\mu ^2)\), contain the intrinsic charm component. In the numerical calculations below the intrinsic charm PDFs are taken at the initial scale \(\mu = \mu _{0}\) for a given PDF, so the perturbative charm contribution is intentionally not taken into account when discussing IC contributions. In our analysis using the hybrid model we also have estimated the contribution associated to subprocesses initiated by light quarks, which could become important at very forward rapidities. In particular, we have calculated the Feynman \(x_F\) distribution associated to the \(q g^* \rightarrow q c {\bar{c}}\) mechanism, where \(q = u, \, d, \, s\).  In Fig. 4a we present predictions of the hybrid model for the Feynman-\(x_F\) distribution considering the different mechanisms for the charm production obtained assuming the KMR uPDF. The intrinsic charm contribution \(cg^* \rightarrow cg\) is obtained for the \(p_{t0} = 1.5\) GeV (dotted line), 2.0 GeV (solid line) and 2.5 GeV (dot-dot-dashed line) and for the probability of finding an intrinsic charm in the proton wave function equal to 1.0% (for more details see below). Our results indicate that the contribution of the \(q g^* \rightarrow q c {\bar{c}}\) mechanism is negligible in comparison to the other contributions discussed in the present paper.

Fig. 4
figure 4

a Predictions of the hybrid model for the Feynman \(x_F\) distribution considering the different mechanisms for the charm production; b Comparison between the predictions of the hybrid model for the \(gg \rightarrow c{\bar{c}}\) mechanism and those derived using the FONLL collinear approach

As emphasized in Ref. [38], the hybrid model, already at leading-order, takes into account radiative higher-order corrections associated with extra hard emissions that are resummed by the gluon uPDF. Such result is demonstrated in Fig. 4b, where we present a comparison between the predictions of the hybrid model and those derived using the FONLL [59, 60] collinear approach. The LO collinear predictions are presented for comparison.  Here the hybrid model results are obtained for the CT14nnloIC PDF set [61] while the collinear LO and FONLL predictions for the CT14(n)lo PDF [62]. One has that the hybrid model predictions, derived taking into account the \(gg \rightarrow c{\bar{c}}\) mechanism, are similar to the FONLL one, which were obtained assuming the collinear factorization approach and summing the contributions initiated by gluons and light quarks.

Considering the \(cg^* \rightarrow cg\) mechanism one has to deal with the massless partons (minijets) in the final state. The relevant formalism with massive partons is not yet available. Working with minijets (jets with transverse momentum of the order of a few GeV) requires a phenomenologically motivated regularization of the cross sections. We follow here the known minijet model [63] as adopted in  PYTHIA 8, where a special suppression factor is introduced at the cross section level [64]. The form factor

$$\begin{aligned} F(p_t) = \frac{p_t^2}{ p_{t0}^2 + p_t^2 } \; \end{aligned}$$
(9)

is applied for each of the outgoing massless partons with transverse momentum \(p_t\). It depends on a free parameter \(p_{t0}\), which will be fixed here using experimental data for the D meson production in \(p + p\) and \(p+^4He\) collisions at \(\sqrt{s} = 38.7\) GeV and 86 GeV, respectively. This parameter also enters as an argument of the strong coupling constant \(\alpha _{S}(p_{t0}^2+\mu _{R}^{2})\). This suppression factor was originally proposed to remove singularity of minijet cross sections in the collinear approach at leading-order. In the hybrid model (or in the \(k_{T}\)-factorization) the leading-order cross sections are finite as long as \(k_{t}> 0\), where \(k_{t}\) is the transverse momentum of the incident off-shell parton. However, as it was shown in Ref. [65], the internal \(k_{t}\) cannot give a minijet suppression consistent with the minijet model and related regularization seems to be necessary even in this framework.

The predictions for the charm production strongly depend on the modelling of the partonic content of the proton [38]. In particular, the contribution of the charm-initiated process is directly associated with the description of the extrinsic and intrinsic components of the charm content in the proton (for a recent review see, e.g. Ref. [45]). Differently from the extrinsic charm quarks/antiquarks that are generated perturbatively by gluon splitting, the intrinsic one have multiple connections to the valence quarks of the proton and thus is sensitive to its nonperturbative structure [24,25,26,27]. The presence of an intrinsic component implies a large enhancement of the charm distribution at large x (\(> 0.1\)) in comparison to the extrinsic charm prediction. Moreover, due to the momentum sum rule, the gluon distribution is also modified by the inclusion of intrinsic charm. In recent years, the presence of an intrinsic charm (IC) component have been included in the initial conditions of the global parton analysis [46, 61], the resulting IC distributions that are compatible with the world experimental data. However, its existence and degree of relevance for various phenomenological applications are still a subject of intense debate [47, 48], mainly associated with the amount of intrinsic charm in the proton wave function, which is directly related to the magnitude of the probability to find an intrinsic charm or anticharm (\(P_{ic}\)) in the nucleon.

In our analysis we will consider the collinear PDFs given by the CT14nnloIC parametrization [61] from a global analysis assuming that the x-dependence of the intrinsic charm component is described by the BHPS model [24]. In this model the proton light cone wave function has higher Fock states, one of them being \(|q q q c {\overline{c}}>\). The cross sections will be initially estimated in the next section using the set obtained for \(P_{ic} = 1\%\) and, for comparison, the results for the case without IC will also be presented. Another important ingredient is the modelling of \({{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2)\), which depends on the treatment of the QCD dynamics for the unintegrated gluon distribution at small-x. Currently, there are several models in the literature, some of them have been reviewed in Ref. [38]. In our analysis we shall consider three different models: two based on the solutions of linear evolution equations, which disregard nonlinear (saturation effects) and one being the solution of the Balitsky–Kovchegov equation [49, 50], which takes into account these effects in the small-x region. In particular, we will use the uPDF derived using the Kimber–Martin–Ryskin (KMR) prescription [51], which assumes that the transverse momentum of the partons along the evolution ladder is strongly ordered up to the final evolution step. In the last step this assumption breaks down and the incoming parton that enters into the hard interaction  possesses a large transverse momentum (\(k_t \approx \mu \)). Such prescription allow us to express \({{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2)\) in terms of Sudakov form factor, which resums all the virtual contributions from the scale \(k_t\) to the scale \(\mu \), and a collinear gluon PDF, which satisfies the DGLAP evolution equations. For this model, we will estimate the uPDF using as input the CT14nnlo parametrization (with and without IC) [61] and the associated predictions will be denoted as KMR hereafter. Some time ago we showed that in the case of charm production at the LHC, the KMR uPDF leads to a reasonable description of the experimental data for D-meson and \(D{\bar{D}}\)-pair production [52]. As also discussed in Refs. [53, 54], the KMR model effectively includes extra emission of hard partons (gluons) from the uPDF that corresponds to higher-order contributions and leads therefore to results well consistent with collinear NLO approach. In order to investigate the impact of new dynamical effects – beyond those included in the DGLAP equation – that are expected to be present in the small-x regime, we will also estimate the charm cross section using as input the uPDF’s obtained in Ref. [55] as a solution of the Balitsky–Kovchegov equation [49, 50] modified to include the sub-leading corrections in \(\ln (1/x)\) which are given by a kinematical constraint, DGLAP \(P_{gg}\) splitting function and the running of the strong coupling (for a detailed derivation see Ref. [56]). Such an approach includes the corrections associated with the BFKL equation, in an unified way with the DGLAP one, as well the nonlinear term, which takes into account unitarity corrections. In Ref. [55] the authors performed a fit to the combined HERA data and provided the solutions with and without the inclusion of the nonlinear term. In the next section, we will use these solutions as input in our calculations and the corresponding predictions will be denoted KS nonlinear and KS linear, respectively. For a comparison between predictions for the KMR, KS linear and KS nonlinear \({{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2)\) we refer the interested reader to Fig. 7 in Ref. [38].

3 Results

Fig. 5
figure 5

Predictions of the hybrid model for a the Feynman \(x_F\) and b the transverse momentum distributions of charm particles produced in pp collisions at \(\sqrt{s} = 39\) GeV. Presented here data are from Ref. [67]

In what follows we will present our predictions for the prompt atmospheric neutrino flux derived using the Z-moment method. The effective hadronic interaction lengths \(\Lambda _i\) and the \(Z_{pp}\), \(Z_{HH}\) and \(Z_{H\nu }\)-moments will be estimated following Ref. [13]. On the other hand, the \(Z_{pH}\)-moment will be calculated using as input the \(x_F\)-distribution for the charm production derived in the hybrid approach with the ingredients discussed in the previous section. Moreover, the prompt \(\nu _{\mu }\) flux will be evaluated considering the description of the primary spectrum proposed by Gaisser in Ref. [66], denoted as H3a spectrum, which assumes that it is given by a composition of 3 populations and 5 representative nuclei, with the set of parameters determined by a global fit of the cosmic ray data.

Fig. 6
figure 6

Predictions of the hybrid model for a the Feynman \(x_F\)-distributions for charm particles and b the prompt neutrino flux (rescaled by \(E_{\nu }^3\)), calculated using the KMR model for the uPDF. The predictions with and without the presence of an intrinsic charm (here \(p_{t0} = 2\) GeV was used) are presented separately. The H3a parametrization of the cosmic ray flux is used in this calculation

As discussed in the previous section, the predictions for the \(cg \rightarrow cg\) mechanism are calculated assuming a phenomenologically motivated suppression of the minijet cross section that depend on the \(p_{t0}\) free parameter (see Eq. (9) and Ref. [38] for more details). In order to constrain this parameter, we will initially consider the LEBC-MPC data [67] for the D meson production in pp collisions at \(\sqrt{s} = 39\) GeV. In Fig. 5 we present our predictions for the \(x_F\) and \(p_T\) distributions of the charm meson, obtained using the CT14nnloIC parametrization for \(P_{ic} = 1\%\) in the calculation of the \(cg \rightarrow cg\) mechanism. Here we have used the JH2013set2 CCFM model [68] for the off-shell gluon uPDF that can be safely used for the kinematic regime relevant for the rather low energy fixed-target experiment (see a discussion in Ref. [70]). The results for the \(x_F\) distribution indicate that in our approach the inclusion of the \(cg^* \rightarrow cg\) mechanism is needed in order to describe the data. Moreover, the \(p_T\) distribution is also well described. Both results point out that a value of \(p_{t0} = 2.0\) GeV is a good choice for the cutoff, which can be considered conservative, since smaller values imply a larger amount for the contribution of the \(cg \rightarrow cg\) mechanism. Such a choice is also justified by a recent analysis performed in Ref. [69], where a comprehensive study of the impact of an intrinsic charm component on the D meson production in pHe fixed-target collisions at the LHCb was performed. The results presented in Ref. [69] indicate that the LHCb data can be well described assuming \(p_{t0} = 2.0\) GeV for a probability of 1% of finding a charm quark–antiquark pair in the proton wave function. It is important to emphasize that, in principle, the \(p_{t0}\) parameter can depend on energy. However, results derived in Ref. [71] indicate that the energy dependence is rather mild and that its value for \(\sqrt{s} \le 100\) TeV is smaller than 2.4 GeV. Therefore, we believe that our choice for \(p_{t0}\) is a good first approximation.

In Fig. 6a, we present our predictions for the Feynman \(x_F\) distribution of charm particles produced in pp collisions at the atmosphere, considering an incident proton with an energy of \(E_p = 10^8\) GeV and the KMR model for the uPDF. Similar conclusions are derived using the KS linear and KS nonlinear uPDFs. In the following calculations we will assume that \(m_c = 1.5\) GeV and that the factorization (\(\mu _F\)) and renormalization (\(\mu _R\)) scales are given by the transverse mass, i.e. \(\mu _F = \mu _R = m_T = \sqrt{p_T^2 + m_c^2}\), where \(p_T^2 = (p_{1T}^2 + p_{2T}^2)/2\). The uncertainty present in our predictions due to these choices is discussed in the Appendix. We present separately in the figure the contribution associated with the \(cg \rightarrow cg\) mechanism and the sum of the two mechanisms, denoted by “cg” and “gg + cg”, respectively. Moreover, we compare the IC predictions, obtained using the CT14nnloIC parametrization for \(P_{ic} = 1\%\), with those obtained disregarding the presence of the intrinsic component (denoted No IC hereafter). One has that for small \(x_F (\equiv x_1 - x_2)\), the charm production is dominated by the \(gg \rightarrow c {\bar{c}}\) mechanism, which is expected since for \(x_F \approx 0\) and high energies both longitudinal momentum fractions \(x_i\) are very small and the proton structure is dominated by gluons. For the No IC case, the contribution of the \(cg \rightarrow cg\) mechanism is smaller than the gluon fusion one for all values of \(x_F\). In contrast, when intrinsic charm is included, the behavior of the distribution in the intermediate \(x_F\) range (\(0.06 \le x_F \le 0.6\)) is strongly modified. Such a behaviour is expected, since for this kinematical range, the charm production depends on the description of the partonic content of the incident proton at large values of the Bjorken x variable. As discussed in the previous section, the main impact of the presence of an intrinsic charm is that the charm distribution is enhanced at large x (\(> 0.1\)), becoming larger than the gluon distribution. As a consequence, the presence of an intrinsic charm implies that the Feynman \(x_F\)-distribution for large \(x_F\) is dominated by the \(cg \rightarrow cg\) mechanism. The impact on the predictions for the prompt neutrino flux is presented in Fig. 6b. As expected from the analysis performed in Ref. [9], where we find that the dominant contribution to the neutrino flux comes typically from \(x_F\) in the region \(0.2<x_F<0.5\), one has that the flux is enhanced by one order of magnitude when intrinsic charm is present. In agreement with the results presented in Fig. 6a, the contribution of the \(cg \rightarrow cg\) mechanism is negligible for the No IC case. However, it becomes dominant in the IC case, with the normalization of the prompt flux dependent on the amount of IC present in the projectile proton, as demonstrated in Fig. 7, where we compare the prediction derived assuming \(P_{ic} = 1\%\), which is the assumption present in the CT14nnloIC parametrization, with the results obtained assuming different values for this probability in the calculation of the \(x_F\) distribution for the \(cg \rightarrow cg\) mechanism. As expected from Eqs. (1), (4) and (8), our results indicate that \(\phi _{\nu }\) is linearly dependent on \(P_{ic}\) and, therefore, a precise determination of the prompt neutrino flux can be used, in principle, to constrain the amount of IC in the proton (see below).

Fig. 7
figure 7

Predictions of the hybrid model for the the prompt neutrino flux (rescaled by \(E_{\nu }^3\)), calculated using the KMR model for the uPDF. The IC contribution was obtained with \(p_{t0} = 2\) GeV and assuming different values for the probability to find an intrinsic charm. The H3a prametrization of the cosmic ray flux is used in this calculation

Fig. 8
figure 8

Predictions of the hybrid model for the a Feynman \(x_F\)-distributions for charm particles and b the prompt neutrino flux (rescaled by \(E_{\nu }^3\)), derived assuming different models for the uPDF. The H3a prametrization of the cosmic ray flux is used in this calculation

The charm production at large \(x_F\) is also dependent on the small-x content of the target proton, which is dominated by gluons. The dependence of our results on the model assumed to describe the unintegrated gluon distribution is analyzed in Fig. 8, where we present the predictions for the \(x_F\) distribution and for the prompt neutrino flux derived assuming the KMR, KS linear and KS nonlinear models as input in our calculations. For this analysis, we only present the sum of the two mechanisms for charm production and the IC predictions. One has that KMR and KS linear predictions for the \(x_F\) distribution are similar, with the KMR one being slightly larger for small \(x_F\). On the other hand, the KS nonlinear is a factor \(\approx 3\) smaller for \(x_F = 0.2\). Such a result demonstrates that the inclusion of the BFKL effects in modelling \({{{\mathcal {F}}}}_{g^*}\) has a small effect on the behaviour of the distribution for large \(x_F\). In contrast, the inclusion of the nonlinear (saturation) effects strongly modifies the magnitude of the distribution. A similar conclusion is derived from the analysis of Fig. 8b, where we present our predictions for the prompt neutrino flux. One important aspect is that the saturation effects imply a suppression of the flux in the kinematical range probed by the IceCube (\(E_{\nu } \lesssim 10^7\) GeV). One has that the saturation effects decrease the prompt flux at all energies considered and its impact increases with the neutrino energy. We have verified that it is 30% larger for \(E_{\nu } = 10^8\) GeV in comparison to \(E_{\nu } = 10^5\) GeV. Such results are expected from the analysis performed in Ref. [9], which demonstrated that for \(E_{\nu } \gtrsim 10^5\) GeV, the prompt flux receives a large contribution of partons with \(x \lesssim 10^{ -5}\), i.e. from the region sensitive to saturation effects.  However, here a crucial role plays the saturation scale \(Q_s\). Its naive estimation is \(Q_{s}^2 = Q_0^2 (x_0/x)^{\lambda }\), where \(Q_0^2 \approx \) 1 \(\hbox {GeV}^2\), \(\lambda \approx \) 0.3 and \(x_0 = 10^{-4}\). Thus we could enter the saturation region only for very small x. For \(x = 10^{-6}\) we get \(Q^2_{s} \approx \) 4 \(\hbox {GeV}^2\), \(x = 10^{-7}\) we get \(Q^2_{s} \approx \) 8 \(\hbox {GeV}^2\) and for \(x = 10^{-8}\) we get \(Q^2_{s} \approx \) 16 \(\hbox {GeV}^2\). The typical x values probed in high-energy neutrino production at IceCube are \( 10^{-7} \lesssim x \lesssim 10^{-5}\) [9]. The smallest scale value in considered processes is \(Q^2 = \mu ^2 = m_c^2\). However, a typical scale is rather \(\mu ^2 \approx 8-9\) \(\hbox {GeV}^2\) (see also [9]). So the final effect should not be extremely large for charm production at IceCube.

Fig. 9
figure 9

Comparison between our predictions and the experimental IceCube data [2] for the atmospheric \(\nu _{\mu }\) flux for a KMR and b KS nonlinear uPDFs. The IC contribution was obtained for \(p_{t0} = 2\) GeV as discussed in the main text. The H3a prametrization of the cosmic ray flux is used in this calculation. The shaded band represents the results from Ref. [3] for the astrophysical neutrino flux

Our results indicate that the presence of the intrinsic charm implies enhancement of the prompt \(\nu _{\mu }\) flux, while the saturation effects suppress it for high energies. Another important aspect is that the impact of the \(cg \rightarrow cg\) mechanism depends on the magnitude of \(P_{ic}\). One important question is whether the current or future experimental IceCube data can be used to probe the presence of these effects and constrain the probability to find an IC on the proton structure, i.e. whether those data could help to improve our understanding of the strong interactions theory. In recent years the IceCube Collaboration measured the energy spectrum of atmospheric neutrino flux with larger precision in an extended energy range [2, 3] and more data are expected in the forthcoming years [72, 73]. Such measurements are a challenge due to steeper falling behaviour expected for the atmospheric flux in comparison to that associated with astrophysical neutrinos. Different methods have been proposed to disentangle these two contributions with promising results (see e.g. Ref. [72]). Therefore, the posed question is valid, relevant and timely.

The IceCube apparatus can measure directions of neutrinos/antineutrinos [10, 11]. The IceCube experimental data discussed below is selected taking into account only such \(\nu _{\mu }\) neutrinos that passed through the Earth (see Fig. 2). In Fig. 9 we present our results for the atmospheric \(\nu _{\mu }\) flux, scaled by a factor \(E_{\nu }^2\), which is the sum of the conventional and prompt contributions. The predictions were obtained considering different models for the uPDFs and distinct values for \(P_{ic}\) in the calculation of the prompt contribution. Moreover, for the conventional atmospheric neutrino flux we assume the result derived in Ref. [41]. The resulting predictions are compared with the IceCube data obtained in Ref. [2] for the zenith-averaged flux of atmospheric neutrinos. For completeness, the results from Ref. [3] for the astrophysical neutrino flux are represented by the grey band. As our goal here is to investigate the impact of the \(cg \rightarrow cg\) mechanism and the possibility of establishing an upper bound on \(P_{ic}\) using the IceCube data, in what follows we will only present the predictions derived for a given set of choices for the charm quark mass, parton PDFs, renormalization and factorization scales. The impact of these choices on the predictions is discussed in the Appendix. We are aware that the quoted numbers below depend on these choices, but we believe that it is important to present the estimates derived in our approach. One has that the prompt contribution enhances the flux at large neutrino energies, with the enhancement being strongly dependent on the magnitude of the \(cg \rightarrow cg\) mechanism and the uPDF considered as input in the calculations. If this mechanism is disregarded, the results represented by “Conv. + gg” in the figures indicate that the impact of the prompt flux is small in the current kinematical range probed by IceCube. In particular, it is negligible when the saturation effects are taken into account (see Fig. 9b). On the other hand, the inclusion of the \(cg \rightarrow cg\) mechanism implies a large enhancement of the prompt flux at large \(E_{\nu }\), with the associated magnitude being strongly dependent on the value of \(P_{ic}\). Our results for the KMR uPDF, presented in Fig. 9a, indicate that a value of \(P_{ic}\) larger than \(1.5\%\) implies a prediction for neutrino flux that overestimate the IceCube data at high energies. We have verified that a similar result is obtained for the KS linear uPDF (not shown explicitly). Therefore, the results derived assuming that the QCD dynamics is described by linear evolution equations, which disregard the saturation effects, indicate that in order to describe the current IceCube data we should have \(P_{ic} \lesssim 1.5\%\).Footnote 4 Surely, future data can be more restrictive in the acceptable range of values for \(P_{ic}\). In contrast, the results presented in Fig. 9b suggest the presence of saturation effects with \(P_{ic} = 1.5\%\) is not discarded by the current IceCube data. In particular, it has been verified that the data is still described for \(P_{ic} \lesssim 2.0\%\) when the KS Nonlinear uPDF is used in the calculations.  It is important to emphasize that the values of the \(P_{ic}\) probabilities suggested above for the linear and nonlinear cases can be increased or decreased by \(\approx 50\%\) if the \(p_{t0}\) parameter in the numerical calculations of the \(cg^* \rightarrow cg\) cross section is assumed to be 2.5 GeV or 1.5 GeV, respectively (see left panel of Fig. 4).

One has that the IceCube data can be described assuming different assumptions for the QCD dynamics at small-x and for the amount of IC. In reality, currently, two acceptable interpretations of our results are possible regarding the impact of the \(cg \rightarrow cg\) mechanism: (a) the QCD dynamics is described by a linear evolution equation and the amount of IC in the proton wave function is similar to that predicted by the CT14nnloIC parameterization; or (b) the amount of IC is larger than that described by the CT14nnloIC parameterization and the saturation effects are needed to describe the charm production at very forward rapidities. One has that if the amount of IC is constrained in hadronic colliders, the IceCube data for the atmospheric neutrino flux can be considered as a probe of the QCD dynamics at high energies. Inversely, if the saturation effects are probed in hadronic colliders, the IceCube data can be used to constrain the amount of the IC. Such results demonstrate synergy between IceCube and the LHC, and strongly motivate new experimental and theoretical analyses in the future.

4 Summary

One of the main goals of the IceCube observatory is the study of astrophysical neutrinos. In order to separate the associated component, it is fundamental to have theoretical control of the background related to the atmospheric neutrino flux, where the neutrinos are generated from the decay of particles produced in high energy interactions of the Cosmic Rays with the atmosphere. In particular, the contribution of the prompt neutrino flux is still a theme of intense debate, since its magnitude for the IceCube Observatory and future neutrino telescopes depends on our knowledge about the QCD dynamics at high energies and on the large-x \(c {{\bar{c}}}\) partonic content of the nucleon (proton or neutron). In this paper, we have investigated the impact of the intrinsic charm component in the hadron wave function, which carries a large fraction of the hadron momentum, and from saturation effects, associated with nonlinear corrections in the QCD evolution, in the prompt neutrino flux. Our results has indicated that the inclusion of the \(cg \rightarrow cg\) mechanism has a strong effect on the prompt neutrino flux. In particular, when the IC component is larger than 0.2%, such a mechanism determines the flux at high energies, with the normalization dependent on the value assumed for the probability to find the IC in the proton wave function. Furthermore, we find that the saturation effects suppress the prompt flux in the kinematical range probed by the IceCube. The comparison of our predictions with the current IceCube experimental data has indicated that for a linear QCD dynamics, \(P_{ic}\) can be of the order of \(1.5\%\), i.e. slightly larger than the value assumed by the CT14nnlo parametrization.  Depending on the value of renormalization and factorization scales, such value can be slightly larger. As we have discussed this value depends also on \(p_{t0}\) parameter. Then the upper limit on \(P_{i,c}\) must be increased to about 2%. A somewhat larger value of about 2.0% is  also acceptable when nonlinear effects are included in the description of the QCD dynamics. These results indicate that the current IceCube data restrict the upper limit for the IC probability to be of the order of 2.0%.Footnote 5 As the predictions for the charm production at high energies and forward rapidities are still strongly dependent on the choices for the charm mass, factorization and renormalization scales, the \(c {{\bar{c}}}\) probability is difficult to extract. However, the conclusion that the IceCube data establish an upper bound on \(P_{ic}\) seems possible, in agreement with the conclusions derived in Refs. [16, 18, 22]. Moreover, our results indicate that in order to disentangle the two possibilities for the amount of IC, it is mandatory to have a better theoretical and experimental control of the prompt neutrino flux at IceCube and of the charm production at the LHC. Such a result strongly motivates the analysis of other processes that allow us to probe the presence of the intrinsic charm and constrain the description of the QCD dynamics at high energies. One of such alternatives is the analysis of the high energy behaviour of the neutrino energy distributions for \(\nu _{\mu }\) passing through FASER\(\nu \) [74]  or SND@LHC [75] taking into account both effects, which is expected to be dominated by neutrinos that arise from the decay of charmed mesons. We intend to study such a topic in a forthcoming publication (for preliminary results see Ref. [76]).

Fig. 10
figure 10

a The scale and mass uncertainty bands for the Feynman-\(x_F\) distributions of charm meson produced in pp collisions at the atmosphere, derived using the KMR uPDF. In these calculations the scale \(\mu \) was changed up and down by a factor 2 and charm quark mass was varied around the central value \(m_{c} = 1.5\) GeV by a factor of 0.2 GeV. b Associated predictions for the atmospheric \(\nu _{\mu }\) flux, scaled by a factor \(E_{\nu }^2\)