A schematic illustration of the IceCube experiment is shown in Fig. 2. Neutrinos are detected through the Cherenkov light emitted by secondary particles produced in neutrino-nucleon interactions in or around the detector. Although primarily designed for the detection of high-energy neutrinos from astrophysical sources, denoted cosmic neutrino in Fig. 2, IceCube can also be used for investigating the atmospheric neutrino spectrum.Footnote 2 The atmospheric neutrinos are produced in cosmic-ray interactions with nuclei in Earth’s atmosphere [10, 11]. The low-energy neutrinos (\(E_{\nu }\lesssim 10^5\) GeV) arise from the decay of light mesons (pions and kaons) and the associated flux is denoted as the conventional atmospheric neutrino flux [41]. On the other hand, for larger energies, it is expected that the prompt atmospheric neutrino flux associated with the decay of hadrons containing heavy flavour quarks/antiquarks become important [39]. One has that the flux of conventional atmospheric neutrinos is a function of the zenith angle, since horizontally travelling mesons have a much higher probability to decay before losing energy in collisions, which implies a harder conventional neutrino spectrum of horizontal events compared to vertical events. In contrast, heavy mesons decay before interacting and follow the initial spectrum of cosmic rays more closely, being almost independent of the zenith angle in the neutrino energy range probed by the IceCube (see e.g. Ref. [42]). As discussed in the Introduction, the calculation of the prompt atmospheric neutrino flux at the detector level depends on the description of the production and decay of the heavy hadrons as well as the propagation of the associated particles through the atmosphere (see Fig. 1). Following our previous studies [9, 23], we will estimate the expected prompt neutrino flux in the detector \(\phi _{\nu }\) using the Z-moment method [39], which implies that \(\phi _{\nu }\) can be estimated using the geometric interpolation formula
$$\begin{aligned} \phi _{\nu } = \sum _H \frac{\phi _{\nu }^{H,low} \cdot \phi _{\nu }^{H,high}}{\phi _{\nu }^{H,low} + \phi _{\nu }^{H,high}} \,\,. \end{aligned}$$
(1)
where \(H = D^0, D^+, D_s^+\), \(\Lambda _c\) for charmed hadrons and \({\phi _{\nu }^{H,low}}\) and \({\phi _{\nu }^{H,high}}\) are solutions of a set of coupled cascade equations for the nucleons, heavy mesons and leptons (and their antiparticles) fluxes in the low- and high-energy ranges, respectively. They can be expressed in terms of the nucleon-to-hadron (\(Z_{NH}\)), nucleon-to-nucleon (\(Z_{NN}\)), hadron-to-hadron (\(Z_{HH}\)) and hadron-to-neutrino (\(Z_{H\nu }\)) Z-moments, as follows [39]
$$\begin{aligned} \phi _{\nu }^{H,low}= & {} \frac{Z_{NH}(E) \, Z_{H\nu }(E)}{1 - Z_{NN}(E)} \phi _N (E,0) \,, \end{aligned}$$
(2)
$$\begin{aligned} \phi _{\nu }^{H,high}= & {} \frac{Z_{NH}(E) \, Z_{H\nu }(E)}{1 - Z_{NN}(E)}\frac{\ln (\Lambda _H/\Lambda _N)}{1 - \Lambda _N/\Lambda _H}\nonumber \\&\times \frac{m_H c h_0}{E \tau _H} f(\theta ) \, \phi _N (E,0) \,, \end{aligned}$$
(3)
where \(\phi _N(E,0)\) is the primary flux of nucleons in the atmosphere, \(m_H\) is the decaying particle’s mass, \(\tau _H\) is the proper lifetime of the hadron, \(h_0 = 6.4\) km, \(f(\theta ) \approx 1/\cos \theta \) for \(\theta < 60^o\), and the effective interaction lengths \(\Lambda _i\) are given by \(\Lambda _i = \lambda _i/(1 - Z_{ii})\), with \(\lambda _i\) being the associated interaction length (\(i = N,H\)). For \(Z_{H\nu }\), our treatment of the semileptonic decay of D-hadrons follows closely Ref. [17]. In particular, we assume the analytical decay distributions \(H \rightarrow \mu \nu _{\mu }X\) obtained in Ref. [40] and use the decay branching ratios reported in the most recent PDG [1]. For a detailed discussion of the cascade equations, see e.g. Refs. [12, 39]. Assuming that the incident flux can be represented by protons (\(N = p\)), the charmed hadron Z-moments are given by
$$\begin{aligned}&Z_{pH} (E)\nonumber \\&\quad \approx \int _0^1 \frac{dx_F}{x_F} \frac{\phi _p(E/x_F)}{\phi _p(E)} \frac{1}{\sigma _{pA}(E)} \frac{d\sigma _{pA \rightarrow H}(E/x_F)}{dx_F} \,\,, \end{aligned}$$
(4)
where E is the energy of the produced particle (charmed meson), \(x_F\) is the Feynman variable, \(\sigma _{pA}\) is the inelastic proton-Air cross section and \(d\sigma /dx_F\) is the differential cross section for the charmed meson production. Following previous studies [12,13,14,15,16,17,18,19,20,21,22,23], we will assume that \(A = 14\), i.e. we will take the \(^{14}N\) nucleus as the most representative element in the composition of the atmosphere. For this value of the atomic mass number, it is a reasonable approximation to assume that \(\sigma _{pA \rightarrow charm} \approx A \times \sigma _{pp \rightarrow charm}\). Surely a more refined analysis of these two aspects is possible but would shadow our discussion of the selected issues. For \(\sigma _{pA}\) we will assume the prediction presented in Ref. [43] (for a more detailed discussion see Ref. [44]).
The transition from quarks to hadrons in our calculations is done within the independent parton fragmentation picture (see e.g. Ref. [57]). It is done assuming that the hadron pseudorapidity is equal to parton pseudorapidity and only momenta of hadrons are reduced compared to the parent partons. In such an approximation the charmed meson \(x_F\)-distributions at large \(x_F\) can be obtained from the charm quark/antiquark \(x^{c}_{F}\)-distributions as:
$$\begin{aligned} \frac{d \sigma _{pp \rightarrow H} (x_{F})}{d x_{F}} = \int _{x_{F}}^{1} \frac{dz}{z} \frac{d \sigma _{pp \rightarrow charm} (x^{c}_{F})}{d x^{c}_{F}} D_{c\rightarrow H}(z), \end{aligned}$$
(5)
where \(x^{c}_{F} = x_{F}/z\) and \(D_{c\rightarrow H}(z)\) is the relevant fragmentation function (FF). Here, in the numerical calculations we take the traditional Peterson FF [58] with \(\varepsilon = 0.05\). The resulting meson distributions are further normalized by the proper fragmentation probabilities. As in Ref. [9], we assume that \(f_{D^0} = 0.565\), \(f_{D^+} = 0.246\), \(f_{D_s^+} = 0.080\) and \(f_{\Lambda _c} = 0.094\). We have checked numerically that in the case of forward meson production, our predictions of the fragmentation model with the \(x_{F}\) being the scaling variable are fully compatible with other possible prescriptions within this approach, including the three-momentum p, energy E or light-cone momentum \(p^{+}=(E+p)\) (see a discussion in Ref. [57]).
As discussed in Ref. [38], the cross section for the charm production at large forward rapidities, which is the region of interest for estimating the prompt \(\nu _{\mu }\) flux [9], can be expressed as follows
$$\begin{aligned}&d \sigma _{pp \rightarrow charm} \simeq d \sigma _{pp \rightarrow charm}(gg \rightarrow c {\bar{c}}) \nonumber \\&\quad + d \sigma _{pp \rightarrow charm}(cg \rightarrow cg) \,\,, \end{aligned}$$
(6)
where the first and second terms represent the contributions associated with the \(gg \rightarrow c {\bar{c}}\) and \( cg \rightarrow cg \) mechanisms, with the corresponding expressions depending on the factorization scheme assumed in the calculations. In Ref. [38], a detailed comparison between the collinear, hybrid and \(k_T\)-factorization approaches was performed and it was demonstrated that the contribution associated with the \(q {\bar{q}} \rightarrow c {\bar{c}}\) channel is negligible. In what follows, we will focus on the hybrid factorization model, which is based on the studies performed also in Refs. [28,29,30,31]. Such a choice is motivated by: (a) the theoretical expectation that the collinear approach, largely used in previous calculations of \(\phi _{\nu }\), breaks down at very small-xFootnote 3 [29, 31]; and that (b) the \(k_T\)-factorization approach reduces to the hybrid model in the dilute-dense regime, which is the case in the charm production at very forward rapidities, where we are probing large (small) values of x in the projectile (target). In this approach, the differential cross sections for \(gg^* \rightarrow c{\bar{c}}\) and \(cg^* \rightarrow cg\) mechanisms, sketched in Fig. 3, are given by
$$\begin{aligned}&d \sigma _{pp \rightarrow charm}(gg \rightarrow c {\bar{c}}) \nonumber \\&\quad = \int dx_1 \int \frac{dx_2}{x_2} \int d^2k_t \, g(x_1,\mu ^2) \, {{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2) \, d{\hat{\sigma }}_{gg^* \rightarrow c{\bar{c}}} \nonumber \\ \end{aligned}$$
(7)
and
$$\begin{aligned}&d \sigma _{pp \rightarrow charm}(cg \rightarrow cg) \nonumber \\&\quad = \int dx_1 \int \frac{dx_2}{x_2} \int d^2k_t \, c(x_1,\mu ^2) \, {{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2) \, d{\hat{\sigma }}_{cg^* \rightarrow cg} \,\,, \nonumber \\ \end{aligned}$$
(8)
where \(g(x_1,\mu ^2)\) and \(c(x_1,\mu ^2)\) are the collinear PDFs in the projectile, \({{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2)\) is the unintegrated gluon distribution (gluon uPDF) of the proton (nucleon) target, \(\mu ^2\) is the factorization scale of the hard process and the subprocesses cross sections are calculated assuming that the small-x gluon is off mass shell and are obtained from a gauge invariant tree-level off-shell amplitude. In our calculations \(c(x_1,\mu ^2)\), similarly \({{\bar{c}}}(x_1,\mu ^2)\), contain the intrinsic charm component. In the numerical calculations below the intrinsic charm PDFs are taken at the initial scale \(\mu = \mu _{0}\) for a given PDF, so the perturbative charm contribution is intentionally not taken into account when discussing IC contributions. In our analysis using the hybrid model we also have estimated the contribution associated to subprocesses initiated by light quarks, which could become important at very forward rapidities. In particular, we have calculated the Feynman \(x_F\) distribution associated to the \(q g^* \rightarrow q c {\bar{c}}\) mechanism, where \(q = u, \, d, \, s\). In Fig. 4a we present predictions of the hybrid model for the Feynman-\(x_F\) distribution considering the different mechanisms for the charm production obtained assuming the KMR uPDF. The intrinsic charm contribution \(cg^* \rightarrow cg\) is obtained for the \(p_{t0} = 1.5\) GeV (dotted line), 2.0 GeV (solid line) and 2.5 GeV (dot-dot-dashed line) and for the probability of finding an intrinsic charm in the proton wave function equal to 1.0% (for more details see below). Our results indicate that the contribution of the \(q g^* \rightarrow q c {\bar{c}}\) mechanism is negligible in comparison to the other contributions discussed in the present paper.
As emphasized in Ref. [38], the hybrid model, already at leading-order, takes into account radiative higher-order corrections associated with extra hard emissions that are resummed by the gluon uPDF. Such result is demonstrated in Fig. 4b, where we present a comparison between the predictions of the hybrid model and those derived using the FONLL [59, 60] collinear approach. The LO collinear predictions are presented for comparison. Here the hybrid model results are obtained for the CT14nnloIC PDF set [61] while the collinear LO and FONLL predictions for the CT14(n)lo PDF [62]. One has that the hybrid model predictions, derived taking into account the \(gg \rightarrow c{\bar{c}}\) mechanism, are similar to the FONLL one, which were obtained assuming the collinear factorization approach and summing the contributions initiated by gluons and light quarks.
Considering the \(cg^* \rightarrow cg\) mechanism one has to deal with the massless partons (minijets) in the final state. The relevant formalism with massive partons is not yet available. Working with minijets (jets with transverse momentum of the order of a few GeV) requires a phenomenologically motivated regularization of the cross sections. We follow here the known minijet model [63] as adopted in PYTHIA 8, where a special suppression factor is introduced at the cross section level [64]. The form factor
$$\begin{aligned} F(p_t) = \frac{p_t^2}{ p_{t0}^2 + p_t^2 } \; \end{aligned}$$
(9)
is applied for each of the outgoing massless partons with transverse momentum \(p_t\). It depends on a free parameter \(p_{t0}\), which will be fixed here using experimental data for the D meson production in \(p + p\) and \(p+^4He\) collisions at \(\sqrt{s} = 38.7\) GeV and 86 GeV, respectively. This parameter also enters as an argument of the strong coupling constant \(\alpha _{S}(p_{t0}^2+\mu _{R}^{2})\). This suppression factor was originally proposed to remove singularity of minijet cross sections in the collinear approach at leading-order. In the hybrid model (or in the \(k_{T}\)-factorization) the leading-order cross sections are finite as long as \(k_{t}> 0\), where \(k_{t}\) is the transverse momentum of the incident off-shell parton. However, as it was shown in Ref. [65], the internal \(k_{t}\) cannot give a minijet suppression consistent with the minijet model and related regularization seems to be necessary even in this framework.
The predictions for the charm production strongly depend on the modelling of the partonic content of the proton [38]. In particular, the contribution of the charm-initiated process is directly associated with the description of the extrinsic and intrinsic components of the charm content in the proton (for a recent review see, e.g. Ref. [45]). Differently from the extrinsic charm quarks/antiquarks that are generated perturbatively by gluon splitting, the intrinsic one have multiple connections to the valence quarks of the proton and thus is sensitive to its nonperturbative structure [24,25,26,27]. The presence of an intrinsic component implies a large enhancement of the charm distribution at large x (\(> 0.1\)) in comparison to the extrinsic charm prediction. Moreover, due to the momentum sum rule, the gluon distribution is also modified by the inclusion of intrinsic charm. In recent years, the presence of an intrinsic charm (IC) component have been included in the initial conditions of the global parton analysis [46, 61], the resulting IC distributions that are compatible with the world experimental data. However, its existence and degree of relevance for various phenomenological applications are still a subject of intense debate [47, 48], mainly associated with the amount of intrinsic charm in the proton wave function, which is directly related to the magnitude of the probability to find an intrinsic charm or anticharm (\(P_{ic}\)) in the nucleon.
In our analysis we will consider the collinear PDFs given by the CT14nnloIC parametrization [61] from a global analysis assuming that the x-dependence of the intrinsic charm component is described by the BHPS model [24]. In this model the proton light cone wave function has higher Fock states, one of them being \(|q q q c {\overline{c}}>\). The cross sections will be initially estimated in the next section using the set obtained for \(P_{ic} = 1\%\) and, for comparison, the results for the case without IC will also be presented. Another important ingredient is the modelling of \({{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2)\), which depends on the treatment of the QCD dynamics for the unintegrated gluon distribution at small-x. Currently, there are several models in the literature, some of them have been reviewed in Ref. [38]. In our analysis we shall consider three different models: two based on the solutions of linear evolution equations, which disregard nonlinear (saturation effects) and one being the solution of the Balitsky–Kovchegov equation [49, 50], which takes into account these effects in the small-x region. In particular, we will use the uPDF derived using the Kimber–Martin–Ryskin (KMR) prescription [51], which assumes that the transverse momentum of the partons along the evolution ladder is strongly ordered up to the final evolution step. In the last step this assumption breaks down and the incoming parton that enters into the hard interaction possesses a large transverse momentum (\(k_t \approx \mu \)). Such prescription allow us to express \({{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2)\) in terms of Sudakov form factor, which resums all the virtual contributions from the scale \(k_t\) to the scale \(\mu \), and a collinear gluon PDF, which satisfies the DGLAP evolution equations. For this model, we will estimate the uPDF using as input the CT14nnlo parametrization (with and without IC) [61] and the associated predictions will be denoted as KMR hereafter. Some time ago we showed that in the case of charm production at the LHC, the KMR uPDF leads to a reasonable description of the experimental data for D-meson and \(D{\bar{D}}\)-pair production [52]. As also discussed in Refs. [53, 54], the KMR model effectively includes extra emission of hard partons (gluons) from the uPDF that corresponds to higher-order contributions and leads therefore to results well consistent with collinear NLO approach. In order to investigate the impact of new dynamical effects – beyond those included in the DGLAP equation – that are expected to be present in the small-x regime, we will also estimate the charm cross section using as input the uPDF’s obtained in Ref. [55] as a solution of the Balitsky–Kovchegov equation [49, 50] modified to include the sub-leading corrections in \(\ln (1/x)\) which are given by a kinematical constraint, DGLAP \(P_{gg}\) splitting function and the running of the strong coupling (for a detailed derivation see Ref. [56]). Such an approach includes the corrections associated with the BFKL equation, in an unified way with the DGLAP one, as well the nonlinear term, which takes into account unitarity corrections. In Ref. [55] the authors performed a fit to the combined HERA data and provided the solutions with and without the inclusion of the nonlinear term. In the next section, we will use these solutions as input in our calculations and the corresponding predictions will be denoted KS nonlinear and KS linear, respectively. For a comparison between predictions for the KMR, KS linear and KS nonlinear \({{{\mathcal {F}}}}_{g^*} (x_2, k_t^2, \mu ^2)\) we refer the interested reader to Fig. 7 in Ref. [38].