Enhancing sensitivity to non-standard neutrino interactions at INO combining muon and hadron information

Abstract

In this paper, we explore the impact of flavor violating neutral current non-standard interaction (NSI) parameter $\varepsilon _{\mu \tau }$ in the oscillation of atmospheric neutrinos and antineutrinos separately using the 50 kt magnetized ICAL detector at INO. We find that due to non-zero $\varepsilon _{\mu \tau }$, $\nu _\mu \rightarrow \nu _\mu$ and ${\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu$ transition probabilities get modified substantially at higher energies and longer baselines, where vacuum oscillation dominates. We demonstrate for the first time that by adding the hadron energy information along with the muon energy and muon direction in each event, the sensitivity of ICAL to the NSI parameter $\varepsilon _{\mu \tau }$ can be enhanced significantly. The most optimistic bound on $\varepsilon _{\mu \tau }$ that we obtain is $-\,\,0.01< \varepsilon _{\mu \tau } < 0.01$ at 90$\%$ C.L. using 500 kt$\cdot$yr exposure and considering $E_\mu$, $\cos \theta _\mu$, and $E'_{\mathrm{had}}$ as observables in their ranges of [1, 21] GeV, [−  1, 1], and [0, 25] GeV, respectively. We discuss for the first time the importance of the charge identification capability of the ICAL detector to have better constraints on $\varepsilon _{\mu \tau }$. We also study the impact of non-zero $\varepsilon _{\mu \tau }$ on mass hierarchy determination and precision measurement of oscillation parameters.

Introduction

The observation of neutrino oscillation proves that the neutrinos are massive and mix with each other, which is the first direct evidence for physics beyond the Standard Model  [1]. Currently, the three-flavor lepton mixing is confirmed and neutrino oscillation has entered the era of precision [2,3,4]. To explain the tiny masses of neutrino and large leptonic mixing angles, the extensions of the Standard Model allow some interactions which are not possible in the Standard Model. These interactions are termed as non-standard interactions (NSI’s). The presence of NSI’s in Nature can have subdominant effect on the oscillation of neutrino and antineutrino. Therefore, the phenomenological consequences of NSI’s in three-flavor mixing using neutrino oscillation experiments are interesting and widely studied by many authors in Refs.  [5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60].

In this paper, we study the impact of neutral current (NC) non-standard interactions of neutrino which may arise when atmospheric neutrinos travel long distances inside the Earth. While NC NSI’s affect neutrinos during their propagation, there are charged-current NSI’s which may modify the neutrino fluxes at the production stage and interaction cross-section at the detection level. In this work, we only focus on the NC NSI’s, and do not consider NSI’s at production and detection level. In most of the cases, NSI’s come into the picture as a low-energy manifestation of high-energy theory involving new heavy states. For a detailed discussion on this topic, see the reviews  [40, 54, 56, 61]. Therefore, at low energies, a neutral current NSI can be described by a four-fermion dimension-six operator  [62],

$$\begin{aligned} {{\mathcal {L}}}_{\mathrm{{NC-NSI}}} = -2\sqrt{2} \, G_F \,\varepsilon ^{C\,f}_{\alpha \beta } \, ({\bar{\nu }}_{\alpha }\gamma ^\rho P_L\nu _{\beta }) \, ({{\bar{f}}} \gamma _\rho P_C f), \end{aligned}$$

(1)

where $G_{F}$ is the Fermi coupling constant, $\varepsilon ^{C\,f}_{\alpha \beta }$ is the dimensionless parameter which represents the strength of NSI relative to $G_{F}$, and $\nu _\alpha$ and $\nu _\beta$ are the neutrino fields of flavor $\alpha$ and $\beta$ respectively. Here, f denotes the matter fermions, electron (e), up-quark (u), and down-quark (d). Here, $P_L = (1 - \gamma _5)/2$, $P_R = (1 + \gamma _5)/2$, and the subscript $C = L,\,R$ expresses the chirality of the ff current. Due to the hermiticity of the interaction, we have $\varepsilon ^{C\,f}_{\alpha \beta } = \left( \varepsilon ^{C\,f}_{\beta \alpha }\right) ^*$.

The NSI’s of neutrino with matter fermions can give rise to the additional matter induced potentials apart from the standard MSW potential due to the W-mediated interactions in matter between electron neutrino and ambient electron, which takes the form $\sqrt{2} G_{F} N_{e}$ ($\equiv V_{CC}$). The total relative strength of the matter induced potential generated by the NC NSI’s of neutrinos with all the matter fermions ($\nu _\alpha + f \rightarrow \nu _\beta + f$) can be written in the following fashion,

$$\begin{aligned} \varepsilon _{\alpha \beta } = \sum _{f=e,u,d} \frac{V_f}{V_{CC}}\left( \varepsilon ^{L\,f}_{\alpha \beta }+ \varepsilon ^{R\,f}_{\alpha \beta }\right) , \end{aligned}$$

(2)

where, $V_f$ = $\sqrt{2}$ $G_{F}\, N_{f}$, $f=e, u, d$. The quantity $N_f$ denotes the number density of matter fermion f in the medium. For antineutrino, $V_f \rightarrow -V_f$ and $V_{CC} \rightarrow -V_{CC}$. In general, the total matter induced potential in presence of all the possible NC non-standard interactions of neutrino with matter fermions can be written as

$$\begin{aligned} H_{mat} = \sqrt{2} G_{F} N_{e} \left( \begin{matrix} 1+\varepsilon _{ee}&{} \varepsilon _{e\mu } &{} \varepsilon _{e\tau }\\ \varepsilon _{e\mu }^* &{} \varepsilon _{\mu \mu } &{} \varepsilon _{\mu \tau }\\ \varepsilon _{e\tau }^* &{} \varepsilon _{\mu \tau }^* &{} \varepsilon _{\tau \tau } \end{matrix}\right) . \end{aligned}$$

(3)

In the present study, we focus our investigation to flavor violating NSI parameter $\varepsilon _{\mu \tau }$, that is, we only allow $\varepsilon _{\mu \tau }$ to be non-zero in our analysis, and assume all other NSI parameters to be zero. We also consider $\varepsilon _{\mu \tau }$ to be real entertaining both of its negative and positive values. Since the atmospheric neutrino oscillation is mainly governed by $\nu _\mu \rightarrow \nu _\tau$ transition, it is expected that NSI parameter $\varepsilon _{\mu \tau }$ would have significant impact on this oscillation channel, which in turn can modify $\nu _\mu \rightarrow \nu _\mu$ survival probability by a considerable amount. We can study this effect by observing the atmospheric neutrinos at the proposed 50 kt magnetized ICAL detector. If we will not see any significant deviation from the standard $\mu ^-$ and $\mu ^+$ event spectra at ICAL, we can use this fact to place tight constraints on NSI parameter $\varepsilon _{\mu \tau }$. This is the main theme of our present study.

This article is organized in the following fashion. In Sect. 2, we briefly review the existing bounds on NSI parameter $\varepsilon _{\mu \tau }$ from various neutrino oscillation experiments. We discuss the possible modification in oscillation probabilities of neutrino and antineutrino due to non-zero $\varepsilon _{\mu \tau }$ in Sect. 3. In Sect. 4, we present the expected total $\mu ^-$ and $\mu ^+$ events and their distributions as a function of reconstructed $E_\mu$ and $\cos \theta _\mu$ for the following three cases: (1) $\varepsilon _{\mu \tau } = 0$ (SM), (2) $\varepsilon _{\mu \tau } = 0.05$, and (3) $\varepsilon _{\mu \tau } = -\,\,0.05$ using 500 kt$\cdot$yr exposure of the ICAL detector. In Sect. 5, we discuss the numerical procedure and various binning schemes that we use in our analysis. We present all the results of our study in Sect. 6 where we show the following: (a) The possible improvement in the sensitivity of the ICAL detector in constraining $\varepsilon _{\mu \tau }$ due to the inclusion of events with $E_\mu$ in range of 11–21 GeV in addition to the events that belong to the $E_\mu$ in range of 1–11 GeV. (b) How much the limit on $\varepsilon _{\mu \tau }$ can be improved by considering the information on reconstructed hadron energy ($E'_{\mathrm{had}}$) as an additional observable along with reconstructed variables $E_\mu$ and $\cos \theta _\mu$ on an event-by-event basis. (c) We study the impact of non-maximal $\theta _{23}$ while deriving limits on $\varepsilon _{\mu \tau }$ at ICAL. (d) We show the advantage of having charge identification (CID) capability in the ICAL detector in placing competitive constraint on $\varepsilon _{\mu \tau }$. (e) We present the expected limits on $\varepsilon _{\mu \tau }$ considering different exposures of the 50 kt ICAL detector. (f) We also explore the possible impact of non-zero $\varepsilon _{\mu \tau }$ in determining the mass hierarchy and in the precision measurement of atmospheric oscillation parameters. We provide a summary of this study in Sect. 7.

Existing limits on NSI parameter $\varepsilon _{\mu \tau }$

There are existing constraints on the NSI parameter $\varepsilon _{\mu \tau }$ from various neutrino oscillation experiments. The Super-Kamiokande collaboration performed an analysis of the atmospheric neutrino data collected during its phase-I and -II run assuming only NSI’s with d-quarks  [63]. The following bounds at $90\%$ C.L. are obtained:

$$\begin{aligned} |\varepsilon | = |\varepsilon ^d_{\mu \tau }|< {0.011} \,, |\varepsilon '| = |\varepsilon ^{d}_{\tau \tau } - \varepsilon ^{d}_{\mu \mu }| < 0.049. \end{aligned}$$

(4)

Since $N_d = N_u = 3N_e$ for an electrically neutral and isoscalar Earth matter, the above constraints as obtained in Ref. [63] are actually on the NSI parameters $\varepsilon _{\alpha \beta }/3$. Therefore, the above constraints at $90\%$ C.L. can be interpreted as

$$\begin{aligned} |\varepsilon _{\mu \tau }|< {0.033} \,, |\varepsilon _{\tau \tau } - \varepsilon _{\mu \mu }| < 0.147. \end{aligned}$$

(5)

Recently, the authors in Ref.  [64] considered the possibility of NSI’s in $\mu$-$\tau$ sector in the one-year high-energy through-going muon data of IceCube. In their analysis, they included various systematic uncertainties on both the atmospheric neutrino flux and detector properties, which they incorporated via several nuisance parameters. They obtained the following limits

$$\begin{aligned}&-6.0\times 10^{-3}\,<\,\varepsilon _{\mu \tau }\nonumber \\&\quad <\,5.4\times 10^{-3} \, \mathrm{at \,\,\, 90\%\, credible\,interval\,(C.I.)}. \end{aligned}$$

(6)

The IceCube-DeepCore collaboration also searched for NSI’s involving $\varepsilon _{\mu \tau }$  [65]. Using their three years of atmospheric muon neutrino disappearance data, they placed the following constraint at $90\%$ confidence level

$$\begin{aligned} -6.7\times 10^{-3}\,<\,\varepsilon _{\mu \tau }\,<\,8.1\times 10^{-3}. \end{aligned}$$

(7)

A preliminary analysis to constrain the NSI parameters in context of the ICAL detector was performed in Ref.  [55]. Using an exposure of 500 kt$\cdot$yr and considering only muon momentum ($E_\mu , \cos \theta _\mu$) as observable, the authors in Ref.  [55] obtained the following bound

$$\begin{aligned}&-0.015\,(-0.027)< \varepsilon _{\mu \tau }\nonumber \\&\quad < 0.015\,(0.027) \mathrm{at \,\,\, 90\,\,} (3\sigma )\,\, \mathrm{C.L.\,\, with \,\, NH}. \end{aligned}$$

(8)

In the present study, we estimate new constraints on $\varepsilon _{\mu \tau }$ considering the reconstructed hadron energy ($E'_{\mathrm{had}}$) as an additional observable along with the reconstructed $E_\mu$ and $\cos \theta _\mu$ on an event-by-event basis at the ICAL detector.

$\nu _\mu \rightarrow \nu _\mu$ transition with non-zero $\varepsilon _{\mu \tau }$

This section is devoted to explore the effect of non-zero $\varepsilon _{\mu \tau }$ in the oscillation of atmospheric neutrino and antineutrino propagating long distances through the Earth matter. For this, we numerically estimate the three-flavor oscillation probabilities including NSI parameter $\varepsilon _{\mu \tau }$ and using the PREM profile  [66] for the Earth matter density. The NSI parameter $\varepsilon _{\mu \tau }$ modifies the evolution of neutrino in matter, which in the flavor basis takes the following form,

$$\begin{aligned} \mathrm{i} \frac{\mathrm{d}}{\mathrm{dt}} \left( \begin{matrix} \nu _e(t) \\ \nu _\mu (t) \\ \nu _\tau (t) \end{matrix} \right)= & {} \frac{1}{2E} \, \left[ U \left( \begin{array}{ccc} 0 &{} 0 &{} 0 \\ 0 &{} \Delta m^2_{21} &{} 0 \\ 0 &{} 0 &{} \Delta m^2_{31} \end{array} \right) U^\dag \right. \nonumber \\&\left. + 2\sqrt{2} G_{F} N_{e} E \, \left( \begin{matrix} 1 &{} 0 &{} 0 \\ 0 &{} 0 &{} \varepsilon _{\mu \tau }\\ 0 &{} \varepsilon _{\mu \tau } &{} 0 \end{matrix}\right) \right] \left( \begin{matrix} \nu _e \\ \nu _\mu \\ \nu _\tau \end{matrix} \right) ,\nonumber \\ \end{aligned}$$

(9)

where $\varepsilon _{\mu \tau }$ is real in our analysis.

Fig. 1

The oscillograms for $\nu _\mu \rightarrow \nu _\mu$ (${\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu$) channel in $E_\nu$, $\cos \theta _{\nu }$ plane are shown in top (bottom) panels for three different scenarios: (1) $\varepsilon _{\mu \tau } = -\,\,0.05$ (left panel), (2) $\varepsilon _{\mu \tau }=0.0$ (the SM case, middle panel), and (3) $\varepsilon _{\mu \tau } = 0.05$ (right panel). Here, in all the panels, we assume NH

In upper panels of Fig. 1, we present the oscillograms for $\nu _\mu$ survival channel in the plane of $\cos \theta _\nu$ vs. $E_\nu$ considering NH. Here, we draw the oscillograms for three different cases: (1) $\varepsilon _{\mu \tau } = -\,\,0.05$ (left panel), (2) $\varepsilon _{\mu \tau }=0.0$ (the SM case, middle panel), and (3) $\varepsilon _{\mu \tau } = 0.05$ (right panel). The lower panel depicts the same but for ${\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu$ oscillation channel. To prepare Fig. 1, we take the following benchmark values of vacuum oscillation parameters in three-flavor framework: $\sin ^2\theta _{23} =0.5$, $\sin ^2 2\theta _{13} =0.1$, $\sin ^2 \theta _{12} =0.3$, $\Delta m^2_{21} = 7.5 \times 10^{-5}$ $\hbox {eV}^2$, $\Delta m_{\mathrm{eff}}^{2} = 2.4 \times 10^{-3}$ $\hbox {eV}^2$, and $\delta _{\text {CP}} =0^\circ$. We estimate the value of $\Delta m^2_{31}$ from $\Delta m_{\mathrm{eff}}^{2}$¹where $\Delta m_{\text {eff}}^{2}$ has the same magnitude for NH and IH with positive and negative signs respectively. It is evident from the upper panels of Fig. 1 that in the presence of negative (see left panel) and positive (see right panel) non-zero values of $\varepsilon _{\mu \tau }$, $\nu _\mu$ survival probabilities get modified substantially at higher energies and longer baselines, where vacuum oscillation dominates. We observe the similar changes in case of ${\bar{\nu }}_\mu$ oscillation probabilities as well (see lower panels).

Another broad feature which has been emerging from Fig. 1 is that the $\nu _\mu \rightarrow \nu _\mu$ oscillation probabilities with positive (negative) $\varepsilon _{\mu \tau }$ as shown in upper right (left) panel are similar to that of ${\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu$ transition probabilities with negative (positive) $\varepsilon _{\mu \tau }$ as can be seen from lower left (right) panel. We can understand this behaviour with the help of following approximate analytical expression of $\nu _\mu \rightarrow \nu _\mu$ transition probability. Assuming $\Delta m^2_{21} L/4E \rightarrow 0$ and $\theta _{13} = 0$, $\nu _\mu \rightarrow \nu _\mu$ oscillation channel in the presence of non-zero $\varepsilon _{\mu \tau }$ and under the constant matter density approximation takes the form  [11, 21]

$$\begin{aligned} P_{\nu _\mu \rightarrow \nu _\mu } = 1 - \sin ^2 2\theta _{\mathrm{eff}} \, \sin ^2\left[ \xi \,\frac{\Delta m^2_{31} L}{4 E} \right] , \end{aligned}$$

(11)

where

$$\begin{aligned}&\sin ^2 2\theta _{\mathrm{eff}} = \frac{|\sin 2\theta _{23} \pm 2 \eta _{\mu \tau }|^2}{\xi ^2}\,, \end{aligned}$$

(12)

$$\begin{aligned}&\xi = \sqrt{|\sin 2\theta _{23} \pm 2 \eta _{\mu \tau }|^2 + \cos ^2 2\theta _{23}}\,, \end{aligned}$$

(13)

and

$$\begin{aligned} \eta _{\mu \tau } = \frac{ 2 E \,V_{CC} \,\varepsilon _{\mu \tau }}{\Delta m^2_{31}}. \end{aligned}$$

(14)

In Eqs. 12 and 13, positive and negative signs in front of $\eta _{\mu \tau }$ are associated with the normal and inverted hierarchy respectively. In case of maximal mixing ($\theta _{23} = 45^{\circ }$), Eq. 11 boils down to the following simple expression  [69]

$$\begin{aligned} P_{\nu _\mu \rightarrow \nu _\mu } = \cos ^2 \left[ L\left( \frac{\Delta m^2_{31}}{4E} + \varepsilon _{\mu \tau } V_{CC} \right) \right] \,. \end{aligned}$$

(15)

Since for antineutrino, $V_{CC} \rightarrow -V_{CC}$, following Eq. 15, we can write

$$\begin{aligned} P_{\nu _\mu \rightarrow \nu _\mu } (-\varepsilon _{\mu \tau }) = P_{{\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu } (\varepsilon _{\mu \tau })\, \end{aligned}$$

(16)

and

$$\begin{aligned} P_{\nu _\mu \rightarrow \nu _\mu } (\varepsilon _{\mu \tau }) = P_{{\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu } (-\varepsilon _{\mu \tau }). \end{aligned}$$

(17)

Equations 16 and 17 explain the broad features in Fig. 1 that we mention above.

Fig. 2

The upper left panel shows the difference in $\nu _\mu \rightarrow \nu _\mu$ oscillation channel between the SM case ($\varepsilon _{\mu \tau }= 0$) and $\varepsilon _{\mu \tau }= -\,\,0.05$. In the top right panel, the difference is due to the SM case and $\varepsilon _{\mu \tau }= 0.05$. The lower panels are for ${\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu$ oscillation channel. Here, in all the panels, we assume NH

To have a better look at the changes induced by non-zero $\varepsilon _{\mu \tau }$ as compared to the SM case, we give Fig. 2 where we plot the difference in $\nu _\mu \rightarrow \nu _\mu$ survival channel considering the cases $\varepsilon _{\mu \tau } = 0$ (the SM case) and $\varepsilon _{\mu \tau } = -\,\,0.05$ (see top left panel). In top right panel, we present the same for the cases of $\varepsilon _{\mu \tau } = 0$ (the SM case) and $\varepsilon _{\mu \tau } = 0.05$. The lower panels are for antineutrinos. In all the panels, we see a visible difference in $\nu _\mu$ survival channel due to the presence of non-zero $\varepsilon _{\mu \tau }$ as compared to the SM case ($\varepsilon _{\mu \tau } = 0.0$) at higher baselines with $\cos \theta _\nu$ in the range $-\,\,1$ to $-\,\,0.8$. This range of $\cos \theta _\nu$ corresponds to the baseline in the range $\sim$ 12,700 to 10,000 km where neutrino and antineutrino mostly travel through inner and outer part of the Earth’s core² and have access to large Earth matter effect. Also, we see a trend that the impact of NSI’s is large at higher energies where the three-flavor oscillations are suppressed because the oscillation lengths ($L^{\mathrm{osc}} = \frac{4 \pi E}{ \Delta m^2_{ij} }$) are large at higher energies.

To have a quantitative estimate of the difference in $\nu _\mu \rightarrow \nu _\mu$ oscillation channel due to non-zero $\varepsilon _{\mu \tau }$ as compared to $\varepsilon _{\mu \tau } =0$ case, we use Eq. 15 and obtain the following expression

$$\begin{aligned} P^{\mathrm{diff}}_{\mu \mu }\equiv & {} P^{\mathrm{SM}}_{\mu \mu } - P^{\varepsilon _{\mu \tau }}_{\mu \mu }\nonumber \\= & {} \sin \left( V_{CC}\,\varepsilon _{\mu \tau } \,L \right) \,\sin \left[ L\left( \frac{\Delta m^2_{31}}{2E} + V_{CC}\,\varepsilon _{\mu \tau }\right) \right] .\nonumber \\ \end{aligned}$$

(18)

Equation18 clearly suggests that the impact of NSI is proportional to both NSI induced matter potential ($V_{CC} \varepsilon _{\mu \tau }$) and baseline (L). We see in upper left and lower right panels of Fig. 2 that around $E\sim 18$ GeV and $\cos \theta _\nu \sim -\, 0.9$, $P^{\mathrm{diff}}_{\mu \mu }$ approaches to zero suggesting that the impact of NSI is negligible. We observe the opposite behaviour in upper right and lower left panels, where around $E\sim 18$ GeV and $\cos \theta _\nu \sim -\,0.9$, $P^{\mathrm{diff}}_{\mu \mu }$ attains a quite large value of $-\, 0.8$ suggesting that the influence of NSI is significant there. Here, $\cos \theta _\nu = -\,0.9$ corresponds to $L = 11500$ km for which the line-averaged constant Earth matter density according to the PREM  [66] profile is 6.8 g/$\hbox {cm}^3$. Therefore, the standard line-averaged Earth matter potential³ for 11,00 km baseline is $V_{CC}\sim 2.6\times 10^{-13}$ eV. Considering $\varepsilon _{\mu \tau } = 0.05$, we get

$$\begin{aligned} V_{CC} \,\varepsilon _{\mu \tau } \,L= & {} 2.6\times 10^{-13} \,\mathrm{eV} \,\times 0.05 \times 11,500 \nonumber \\&\times \,\,5.06 \times 10^9 (\mathrm{eV})^{-1}\sim 0.76. \end{aligned}$$

(20)

For $E = 18$ GeV, $L =11,500$ km, and $\Delta m^2_{31} = 2.36\times 10^{-3}$ $\hbox {eV}^2$ (this mass-squared difference is obtained from Eq. 10 using benchmark values of oscillation parameters),

$$\begin{aligned} \Delta m^2_{31} L/2E = 3.8. \end{aligned}$$

(21)

Thus, for $E = 18$ GeV and $L = 11,500$ km, from Eq. 18, we obtain

$$\begin{aligned} P^{\mathrm{diff}}_{\mu \mu }(\varepsilon _{\mu \tau } = -\,0.05)= & {} P^\mathrm{diff}_{{\bar{\mu }}{\bar{\mu }}} (\varepsilon _{\mu \tau } = 0.05)\nonumber \\= & {} \sin (3.8-0.76 ) \, \sin (0.76 ) \sim 0.06\, \nonumber \\ \end{aligned}$$

(22)

and

$$\begin{aligned}&P^{\mathrm{diff}}_{\mu \mu }(\varepsilon _{\mu \tau } = 0.05) = P^\mathrm{diff}_{{\bar{\mu }}{\bar{\mu }}} (\varepsilon _{\mu \tau } = -\,0.05)\nonumber \\&\quad = \sin (3.8 + 0.76 ) \,\sin (0.76 ) \sim -\,0.7. \end{aligned}$$

(23)

Equations 22 and 23 confirm the observations regarding $P^{\mathrm{diff}}_{\mu \mu }$ and $P^{\mathrm{diff}}_{{\bar{\mu }}{\bar{\mu }}}$ in Fig. 2 that we mention above. We know that due to its CID capability, ICAL has an edge to resolve the issue of neutrino mass hierarchy (sign of $\Delta m^2_{31}$) by observing the Earth matter effect in $\mu ^-$ and $\mu ^+$ events separately  [70]. Similarly, the four panels in Fig. 2 suggest that the CID capability of ICAL can provide useful information to determine the sign of NSI parameter $\varepsilon _{\mu \tau }$ for a particular choice of mass hierarchy.

Expected events at ICAL with non-zero $\varepsilon _{\mu \tau }$

The Monte Carlo based neutrino event generator NUANCE [71] is used to simulate the CC interactions of $\nu _\mu$ and ${\bar{\nu }}_\mu$ in the ICAL detector. To generate events in NUANCE, we give a simple geometry of the ICAL detector with 150 alternate layers of iron and glass plates in each module. We have three such modules to account for the 50 kt ICAL detector. As far as the neutrino flux is concerned in generating the neutrino events in the present study, we use the flux as predicted at Kamioka⁴  [74]. To reduce the statistical fluctuation, we generate the unoscillated CC neutrino and antineutrino events considering a very high exposure of 1000 years and 50 kt ICAL. Then, we implement various oscillation probabilities using the reweighting algorithm. Next, we fold the oscillated events with detector response for muon and hadron as described in Ref.  [75, 76]. In the present study, we assume that the ICAL particle reconstruction algorithms can separate the hits due to the hadron shower from the hits originating from a muon track with 100$\%$ efficiency. It means that whenever a muon is reconstructed, we consider all the other hits to be part of the hadronic shower in order to calibrate the hadron energy. It also implies that the neutrino event reconstruction efficiency is same as the muon reconstruction efficiency. Finally, the reconstructed $\mu ^-$ and $\mu ^+$ events are scaled down to the exposure of 10 years for 50 kt ICAL. Now, we present the expected $\mu ^-$ and $\mu ^+$ events for 500 kt$\cdot$yr exposure of the ICAL detector assuming the SM case ($\varepsilon _{\mu \tau } = 0$) and $\varepsilon _{\mu \tau } = \pm \, 0.05$. To estimate these event rates, we use the values of oscillation parameters as considered in Sect. 3 to draw the oscillograms.

Total event rates

First, we address the following question: can we see the signature of non-zero $\varepsilon _{\mu \tau }$ in the total number of $\mu ^-$ and $\mu ^+$ events which will be collected at the ICAL detector over 10 years of running? To have an answer of this question, we estimate the total number of events for the following three cases: (1) $\varepsilon _{\mu \tau } = 0.05$, (2) $\varepsilon _{\mu \tau } =0$ (the SM case), and (3) $\varepsilon _{\mu \tau } = -\,0.05$. We present these numbers in Table 1 with NH and using 500 kt$\cdot$yr exposure of the ICAL detector integrating over entire ranges of $E_\mu$, $\cos \theta _\mu$, and $E'_{\mathrm{had}}$ that we consider in our analysis. As far as the binning schemes are concerned, we use the low-energy (LE) and high-energy (HE) binning schemes⁵, and for both these binning schemes, we take the entire range of $\cos \theta _\mu$ spanning over − 1 to 1. The energy ranges for reconstructed $E_\mu$ and $E'_\mathrm{had}$ are different in LE and HE binning schemes. For LE binning scheme, $E_\mu \in$ [1, 11] GeV and $E'_{\mathrm{had}} \in$ [0, 15] GeV. In case of HE binning scheme, $E_\mu \in$ [1, 21] GeV, and $E'_{\mathrm{had}} \in$ [0, 25] GeV. When we increase the reconstructed muon energy from 11 to 21 GeV and reconstructed hadron energy from 15 to 25 GeV, the number of $\mu ^-$ and $\mu ^+$ events get increased by 300 and 150 respectively for 500 kt$\cdot$yr exposure of the ICAL detector. Apart from showing the total $\mu ^-$ event rates in Table 1, we also present the estimates of individual events coming from $\nu _\mu \rightarrow \nu _\mu$ ($P_{\mu \mu }$) disappearance channel and $\nu _e\rightarrow \nu _\mu$ ($P_{e\mu }$) appearance channel. Also, for $\mu ^+$ events, we separately show the contributions originating from ${\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu$ ($P_{{\bar{\mu }}{\bar{\mu }}}$) disappearance and ${\bar{\nu }}_e\rightarrow {\bar{\nu }}_\mu$ ($P_{\bar{e}{\bar{\mu }}}$) appearance channels. Here, we see that only $\sim 2 \%$ of the total $\mu ^-$ events at the ICAL detector come via the appearance channel. Note that the differences in the total number of $\mu ^-$ and $\mu ^+$ events between the SM case ($\varepsilon _{\mu \tau } = 0$) and non-zero $\varepsilon _{\mu \tau }$ of $\pm 0.05$ are not significant. But, later while presenting our final results, we see that the ICAL detector can place competitive constraints on $\varepsilon _{\mu \tau }$ by exploiting the useful information contained in the spectral distributions of $\mu ^-$ and $\mu ^+$ events as a function of reconstructed observables $E_\mu$, $\cos \theta _\mu$, and $E'_{\mathrm{had}}$. To establish this claim, now, we show how the expected $\mu ^-$ and $\mu ^+$ event spectra get modified in the presence of non-zero $\varepsilon _{\mu \tau }$ in terms of reconstructed $E_\mu$ and $\cos \theta _\mu$ while integrating over entire range of $E'_{\mathrm{{had}}}$.

Table 1 Expected number of $\mu ^-$ and $\mu ^+$ events for 500 kt$\cdot$yr exposure of the ICAL detector considering low-energy (LE) and high-energy (HE) binning schemes. We present the event rates for the following three cases: (1) $\varepsilon _{\mu \tau } = 0.05$, (2) $\varepsilon _{\mu \tau } =0$ (the SM case), and (3) $\varepsilon _{\mu \tau } = -\,\,0.05$. Apart from showing the total $\mu ^-$ event rates, we also give the estimates of individual event rates coming from $\nu _\mu \rightarrow \nu _\mu$ ($P_{\mu \mu }$) disappearance channel and $\nu _e\rightarrow \nu _\mu$ ($P_{e\mu }$) appearance channel. For $\mu ^+$ events also, we separately show the contributions from ${\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu$ ($P_{{\bar{\mu }}{\bar{\mu }}}$) disappearance channel and ${\bar{\nu }}_e\rightarrow {\bar{\nu }}_\mu$ ($P_{{{\bar{e}}}{\bar{\mu }}}$) appearance channel. Here, we consider NH and assume the benchmark values of the oscillation parameters as mentioned in Sect. 4

Fig. 3

The distributions of $\mu ^-$ (upper panels) and $\mu ^+$ (lower panels) events for three different $E_\mu$ ranges: [3, 4] GeV in left panel, [5, 11] GeV in middle panel, and [11, 21] GeV in right panel. In each panel, we consider three different cases: (1) $\varepsilon _{\mu \tau } = 0.05$ (blue line), (2) $\varepsilon _{\mu \tau } =0$ (the SM case, black line), and (3) $\varepsilon _{\mu \tau } = -\,\,0.05$ (red line). Here, we sum over $E'_{\mathrm{had}}$ in its entire range of 0–25 GeV and show the results for 500 kt$\cdot$yr exposure and assuming NH

Event spectra

In Fig. 3, we show the distributions of $\mu ^-$ (upper panels) and $\mu ^+$ (lower panels) events as a function of reconstructed $\cos \theta _\mu$ for three different ranges of $E_\mu$. The ranges of $E_\mu$ that we consider in left, middle, and right panels are [3, 4] GeV,  [5, 11] GeV, and [11, 21] GeV respectively. Here, we integrate over $E'_{\mathrm{{had}}}$ in its entire range of 0–25 GeV. In each panel, we compare the event spectra for three different cases: (1) $\varepsilon _{\mu \tau } = 0.05$ (blue line), (2) $\varepsilon _{\mu \tau } =0$ (the SM case, black line), and (3) $\varepsilon _{\mu \tau } = -\,\,0.05$ (red line). Before we discuss the impact of non-zero $\varepsilon _{\mu \tau }$, we would like to mention few general features that are emerging from various panels in Fig. 3. For both $\mu ^-$ (upper panels) and $\mu ^+$ (lower panels), the number of events get reduced as we go to higher energies. It happens because of $\sim E_\nu ^{-2.7}$ dependence of the atmospheric neutrino flux. Though the neutrino fluxes are higher along the horizontal direction ($\cos \theta _\mu$ around 0) as compared to the other directions (for detailed discussion, see Ref.  [73]), but, due to the poor reconstruction efficiency of the ICAL detector along the horizontal direction  [75], we see a suppression in $\mu ^-$ and $\mu ^+$ event rates around $\cos \theta _\mu \in [-\,0.1,\,0]$ irrespective of the choices of $E_\mu$ ranges. Important to note that as we proceed towards higher $E_\mu$, the relative differences in $\mu ^-$ and $\mu ^+$ event rates between the SM case ($\varepsilon _{\mu \tau } = 0$) and non-zero $\varepsilon _{\mu \tau }$ ($\pm 0.05$) get increased for a wide range of $\cos \theta _\mu$. We see similar features in Fig. 2 in Sect. 3, where we show the differences in $\nu _\mu \rightarrow \nu _\mu$ oscillograms due to the SM case ($\varepsilon _{\mu \tau } = 0$) and non-zero $\varepsilon _{\mu \tau }$ ($\pm 0.05$). We show the improvement in the sensitivity to constrain $\varepsilon _{\mu \tau }$ due to high energy events in Sect. 6.1. Next, we discuss the numerical technique and analysis procedure which we adopt to obtain the final results.

Simulation method

Binning scheme in $\left( E_{\mu }, \cos \theta _{\mu }, E'_{\mathrm{had}}\right)$ plane

In the present study, we produce all the results with low-energy (LE) and high-energy (HE) binning schemes. Table 2 shows the detailed information about the LE binning scheme for the three reconstructed observables $E_{\mu }$, $\cos \theta _{\mu }$, and $E'_{\mathrm{had}}$. Table 3 portrays the same for the HE binning scheme. In case of LE binning scheme, the range of $E_{\mu }$ is [1, 11] GeV with total 10 bins each having a width of 1 GeV. In case of HE binning scheme, we extend the range of $E_{\mu }$ up to 21 GeV by adding two additional bins in the range of 11 to 21 GeV, where each bin has a width of 5 GeV. As far as reconstructed $E'_{\mathrm{had}}$ is concerned, in case of LE (HE) binning scheme, the considered range is 0–15 GeV (0–25 GeV). We can see from Table 2 and Table 3 that the first three bins for $E'_{\mathrm{had}}$ are same for both the binning schemes, whereas the last bin extend from 4 to 15 GeV (4 to 25 GeV) for LE (HE) binning scheme. For both these binning schemes, we consider the entire range of $\cos \theta _{\mu }$ from − 1 to 1. For upward going events, that is $\cos \theta _{\mu }\in \,$ [- 1, 0], we consider 10 uniform bins each having width of 0.1. For downward going events, that is $\cos \theta _{\mu }\in \,$ [0, 1], we consider 5 uniform bins each having width of 0.2. Important to note that the downward going events do not have enough path length to oscillate, but, these events play an important role to increase the overall statistics and to minimize the effect of normalization uncertainties in atmospheric neutrino fluxes. Here, we would like to mention that we have not optimized these binning schemes to obtain the best sensitivities, but we ensure that there are sufficient statistics in most of the bins.

Table 2 The low-energy (LE) binning scheme adopted for the reconstructed observables $E_\mu$, $\cos \theta _\mu$, and $E'_{\mathrm{had}}$ for each muon polarity. The last column shows the total number of bins taken for each observable

Table 3 The high-energy (HE) binning scheme considered for the reconstructed observables $E_\mu$, $\cos \theta _\mu$, and $E'_{\mathrm{had}}$ for each muon polarity. The last column shows the total number of bins taken for each observable

Numerical analysis

In our analysis, the $\chi ^2$ function gives us the median sensitivity of the experiment in the frequentist approach  [77]. We use the following Poissonian $\chi ^2_{-}$ for $\mu ^-$ events in our statistical analysis considering $E_\mu$, $\cos \theta _\mu$, and $E'_{\mathrm{had}}$ as observables (the so-called “3D” analysis as considered in  [78]):

$$\begin{aligned}&\chi ^2_{-}(\mathrm{3D})\,=\,\min _{\zeta _l}\,\sum ^{N_{E'_\mathrm{had}}}_{i=1} \sum ^{N_{E_\mu }}_{j=1} \sum ^{N_{{\cos \theta _\mu }}}_{k=1} \,2\nonumber \\&\quad \times \bigg [ N^\mathrm{theory}_{ijk}\,- \, N^{\mathrm{data}}_{ijk}\, - \, N^\mathrm{data}_{ijk}\,\, {\mathrm{ln}}\,\bigg (\frac{N^{\mathrm{theory}}_{ijk}}{N^\mathrm{data}_{ijk}} \bigg )\bigg ]\,+\,\sum ^5_{l=1} \zeta ^2_{l},\nonumber \\ \end{aligned}$$

(24)

with

$$\begin{aligned} N^{\mathrm{theory}}_{ijk}\,=\,N^0_{ijk}\left( \,1\,+\,\sum ^5_{l=1} \pi ^l_{ijk}\zeta _{l}\,\right) . \end{aligned}$$

(25)

In the above equations, $N^{\mathrm{data}}_{ijk}$ and $N^\mathrm{theory}_{ijk}$ denote the observed and expected number of $\mu ^-$ events in a given [$E_\mu$, $\cos \theta _\mu$, $E'_{\mathrm{had}}$] bin. In case of LE (HE) binning scheme, $N_{E_\mu }$ = 10 (12), $N_{\cos \theta _\mu } = 15$, and $N_{E'_{\mathrm{had}}} = 4$. In Eq. 25, $N^0_{ijk}$ represents the number of expected events without systematic uncertainties. Following Ref.  [79], we consider five systematic errors in our analysis: 20$\%$ flux normalization error, 10$\%$ error in cross-section, 5$\%$ tilt error, 5$\%$ zenith angle error, and 5$\%$ overall systematics. We incorporate these systematic uncertainties in our simulation using the well known “pull” method  [21, 80, 81]. In Eqs. 24 and 25, the quantities $\zeta _{l}$ denote the “pulls” due to the systematic uncertainties.

When we produce results with only $E_\mu$ and $\cos \theta _\mu$ as observables and do not use the information on hadron energy $E'_\mathrm{had}$ (the so-called “2D” analysis as considered in Ref.  [79]), the Poissonian $\chi ^2_{-}$ for $\mu ^-$ events takes the form

$$\begin{aligned}&\chi ^2_{-}(\mathrm{2D})\,=\,\min _{\zeta _l}\,\sum ^{N_{E_\mu }}_{j=1} \sum ^{N_{{\cos \theta _\mu }}}_{k=1} \,2\nonumber \\&\quad \times \bigg [ N^{\mathrm{theory}}_{jk}\,- \, N^{\mathrm{data}}_{jk}\, - \, N^{\mathrm{data}}_{jk}\,\, {\mathrm{ln}}\,\bigg (\frac{N^{\mathrm{theory}}_{jk}}{N^{\mathrm{data}}_{jk}} \bigg )\bigg ]\,+\,\sum ^5_{l=1} \zeta ^2_{l},\nonumber \\ \end{aligned}$$

(26)

with

$$\begin{aligned} N^{\mathrm{theory}}_{jk}\,=\,N^0_{jk}\left( \,1\,+\,\sum ^5_{l=1} \pi ^l_{jk}\zeta _{l}\,\right) . \end{aligned}$$

(27)

In Eq. 26, $N^{\mathrm{data}}_{jk}$ and $N^\mathrm{theory}_{jk}$ indicate the observed and expected number of $\mu ^-$ events in a given [$E_\mu$, $\cos \theta _\mu$] bin. In Eq. 27, $N^0_{jk}$ stands for the number of expected events without systematic errors. In case of LE (HE) binning scheme, $N_{E_\mu }$ = 10 (12) and $N_{\cos \theta _\mu } = 15$.

For both the “2D” and “3D” analyses, the $\chi ^2_+$ for $\mu ^+$ events is determined following the same technique described above. We add the individual contributions from $\mu ^-$ and $\mu ^+$ events to estimate the total $\chi ^2$ for both the “2D” and “3D” schemes:

$$\begin{aligned} \chi ^2_{\mathrm{ICAL}} = \chi ^2_-\,+\,\chi ^2_+. \end{aligned}$$

(28)

In our analysis, we simulate the prospective data considering the following benchmark values of the oscillation parameters: $\sin ^2 \theta _{23} = 0.5$, $\,\sin ^2 2\theta _{13}= 0.1$, $\sin ^2 \theta _{12}\, =\, 0.3$, $\Delta m^2_{21}\, =\, 7.5\times 10^{-5}\,\mathrm{eV^2}$, and $|\Delta m^2_{\mathrm{eff}}|\, =\, 2.4\times 10^{-3}\, \mathrm{eV^2}$. To estimate the value of $\Delta m^2_{31}$ from $\Delta m_{\mathrm{eff}}^{2}$, we use the Eq. 10, where $\Delta m_{\text {eff}}^{2}$ has the same magnitude for NH and IH with +ve and -ve signs respectively. In the fit, we first minimize $\chi ^2_{\mathrm{ICAL}}$ (see Eq. 28) with respect to the “pull” variables $\zeta _l$, and then marginalize over the oscillation parameters $\sin ^2\theta _{23}$ in the range [0.36, 0.66], $|\Delta m_{\text {eff}}^{2}|$ in the range $[2.1, 2.6]\times 10^{-3}$ $\hbox {eV}^2$, and over both the choices of mass hierarchy, NH and IH, while keeping $\theta _{12}$, $\Delta m^2_{21}$, $\sin ^2 2\theta _{13}$ fixed at their benchmark values. We consider $\delta _{\mathrm{CP}} = 0^{\circ }$ throughout our analysis.

Results

Expected bounds on NSI parameter $\varepsilon _{\mu \tau }$

We quantify the statistical significance of the analysis to constrain the NSI parameter $\varepsilon _{\mu \tau }$ in the following fashion

$$\begin{aligned} \Delta \chi ^2_{\,\,\mathrm{ICAL-NSI}} = \chi ^2_{\,\,\mathrm{ICAL}}\left( \mathrm{SM} + \varepsilon _{\mu \tau }\right) -\chi ^2_{\,\,\mathrm{ICAL}}\left( \mathrm{SM}\right) . \end{aligned}$$

(29)

Here, $\chi ^2_{\,\,\mathrm{ICAL}} (\mathrm{SM})$ and $\chi ^2_{\,\,\mathrm{ICAL}}\left( \mathrm{SM} + \varepsilon _{\mu \tau }\right)$ are calculated by fitting the prospective data with zero (the SM case) and non-zero value of NSI parameter $\varepsilon _{\mu \tau }$ respectively. In our analysis procedure, statistical fluctuations are suppressed, and therefore, $\chi ^2_{\,\,\mathrm{ICAL}}(\mathrm{SM})\approx 0$.

Fig. 4

Distributions of $\Delta \chi ^2_{\mathrm{ICAL-NSI}}$ (per unit area) from $\mu ^-$ events in reconstructed $\cos \theta _\mu$ and $E_\mu$ plane assuming non-zero $\varepsilon _{\mu \tau }$ in the fit with a strength of 0.05. The top (bottom) panels are for the LE (HE) binning scheme. For a given binning scheme, left and right panels are obtained with [$E_\mu$, $\cos \theta _\mu$] and [$E_\mu$, $\cos \theta _\mu$, $E'_{\mathrm{had}}$] respectively. In all the panels, we use 500 kt$\cdot$yr exposure and assume NH in both data and theory

Let us first identify the regions in $\cos \theta _\mu$ and $E_\mu$ plane which give significant contributions towards $\Delta \chi ^2_{\mathrm{ICAL-NSI}}$. In Fig. 4, we show the distribution⁶ of $\Delta \chi ^2_{-}$ from $\mu ^-$ events in the reconstructed [$\cos \theta _\mu$-$E_\mu$] plane using 500 kt$\cdot$yr exposure of the ICAL detector and assuming NH. In all the panels of Fig. 4, we consider $\varepsilon _{\mu \tau } = 0.05$ in the fit and show the results for the following four different choices of binning schemes and observables: (1) top left panel: [LE, 2D], (2) top right panel: [LE, 3D], (3) bottom left panel: [HE, 2D], (4) bottom right panel: [HE, 3D]. We show the distribution of $\Delta \chi ^2_{+}$ from $\mu ^+$ events in the plane of reconstructed $\cos \theta _\mu$ and $E_\mu$ for these four different cases in Fig. 5 considering $\varepsilon _{\mu \tau } = 0.05$ in the fit. In left panels of Figs. 4 and 5, we show the sensitivity in the plane of reconstructed $\cos \theta _\mu$ and $E_\mu$ for the “2D” analysis, where we do not use any information on hadrons. But, in right panels of these figures, we portray the sensitivity in the plane of reconstructed $\cos \theta _\mu$ and $E_\mu$ for the “3D” case, where the events are further divided into four sub-bins depending on the reconstructed hadron energy for LE (see Table 2) and HE binning schemes (see Table 3).

Fig. 5

Distributions of $\Delta \chi ^2_{\mathrm{ICAL-NSI}}$ (per unit area) from $\mu ^+$ events in reconstructed $\cos \theta _\mu$ and $E_\mu$ plane assuming non-zero $\varepsilon _{\mu \tau }$ in the fit with a strength of 0.05. The top (bottom) panels are for the LE (HE) binning scheme. For a given binning scheme, left and right panels are obtained with [$E_\mu$, $\cos \theta _\mu$] and [$E_\mu$, $\cos \theta _\mu$, $E'_{\mathrm{had}}$] respectively. In all the panels, we use 500 kt$\cdot$yr exposure and assume NH in both data and theory

The common features which are emerging from all the panels in Figs. 4 and 5 are that most of the sensitivity towards the NSI parameter $\varepsilon _{\mu \tau }$ stems from higher energies and longer baselines where the matter effect term 2$\sqrt{2} G_F N_e E$ becomes sizeable. We observe similar trends in Fig 2 where we plot the differences in $\nu _\mu \rightarrow \nu _\mu$ oscillation probabilities for the cases $\varepsilon _{\mu \tau } = 0$ and $\varepsilon _{\mu \tau } = \pm 0.05$. The event spectra as shown in Fig. 3 also confirm this fact. Figs. 4 and 5 clearly demonstrate while going from LE to HE binning scheme that the sensitivity towards the NSI parameter $\varepsilon _{\mu \tau }$ get enhanced due to the increment in the range of $E_\mu$ from 11 to 21 GeV and for extending the fourth $E'_{\mathrm{had}}$ bin from 15 to 25 GeV. We can also observe from these figures that with the addition of hadron energy information, the area in the $E_\mu$-$\cos \theta _\mu$ plane which contributes significantly to $\Delta \chi ^2_{\pm }$ increases, consequently enhancing the net $\Delta \chi ^2_{\pm }$ for both LE and HE binning schemes. Here, we would like to mention that the increase in $\chi ^2_{\pm }$ is not just due to the information contained in $E'_{\mathrm{had}}$, but also due to the valuable information coming from the correlation between $E'_{\mathrm{had}}$ and muon momentum ($E_\mu$, $\cos \theta _\mu$).

Fig. 6

The sensitivity of the ICAL detector to set bounds on the NSI parameter $\varepsilon _{\mu \tau }$ using 500 kt$\cdot$yr exposure and assuming NH. Left (right) panel is with LE (HE) binning scheme. In each panel, the red solid line shows the sensitivity for the “3D” where we consider $E_{\mu }$, $\cos \theta _{\mu }$, and $E'_{\mathrm{had}}$ as observables. The black dashed line in each panel portrays the sensitivity for the “2D” case considering $E_{\mu }$ and $\cos \theta _{\mu }$ as observables. These results are obtained after performing marginalization over $\theta _{23}$, $\Delta m^{2}_{\mathrm{eff}}$, and both choices of mass hierarchy

In Fig. 6, we show the sensitivity of the ICAL detector to constrain $\varepsilon _{\mu \tau }$ using an exposure of 500 kt$\cdot$yr and assuming NH as the true mass hierarchy. We obtain these results after performing marginalization over $\theta _{23}$, $\Delta m^{2}_{\mathrm{eff}}$, and both the choices of mass hierarchy as discussed in Sect. 5.2. In the left (right) panel, the results are shown for the LE (HE) binning scheme. In each panel, the red solid line shows the sensitivity for the “3D” case where we consider $E_{\mu }$, $\cos \theta _{\mu }$, and $E'_{\mathrm{had}}$ as observables. The black dashed line in each panel portrays the sensitivity for the “2D” case considering $E_{\mu }$ and $\cos \theta _{\mu }$ as observables. We see considerable improvement in the sensitivity for both the LE and HE binning schemes when we add $E'_{\mathrm{had}}$ along with $E_\mu$ and $\cos \theta _\mu$ as observables. We see significant gain in the sensitivity when we increase the $E_\mu$ range from 11 to 21 GeV and extend the fourth $E'_{\mathrm{had}}$ bin from 15 to 25 GeV. It is evident from both the panels in Fig. 6 that for the [HE, 3D] case, we obtain the best sensitivity towards the NSI parameter $\varepsilon _{\mu \tau }$, whereas the [LE, 2D] mode gives the most conservative limits.

Table 4 The expected bound on $\varepsilon _{\mu \tau }$ for four different choices of binning schemes and observables at $3\sigma$ and $90\%$ C.L. obtained using 500 kt$\cdot$yr exposure of the ICAL detector. We give results for the both choices of true mass hierarchy. To obtain these constraints, we marginalize over $\theta _{23}$, $\Delta m^{2}_{\mathrm{eff}}$, and both the choices of mass hierarchy in the fit

The 3$\sigma$ (90$\%$) confidence level bounds on the flavor violating NSI parameter $\varepsilon _{\mu \tau }$ obtained using 500 kt$\cdot$yr exposure of the ICAL are listed in Table 4. The results are shown for true NH (3rd column) and true IH (4th column). For the [HE, 3D] case, we expect the best limit of $-\,0.01<\varepsilon _{\mu \tau }< 0.01$ at 90$\%$ C.L. using 500 kt$\cdot$yr exposure of the ICAL detector and irrespective of the choices of true mass hierarchy. For the [LE, 2D] mode, we obtain the most conservative limit of $-\,0.03< \varepsilon _{\mu \tau } < 0.034$ at 90$\%$ confidence level assuming NH as true choice. So far we have considered $\sin ^2 2\theta _{13}= 0.1$ as our benchmark choice both in data and theory. If we consider the current best fit value of $\sin ^2 2\theta _{13} = 0.085$  [2,3,4], we have checked that our results will remain almost unaltered. For an instance, if we consider $\varepsilon _{\mu \tau } = 0.02$ in the fit, then we obtain $\Delta \chi ^2 = 9.49$ assuming $\sin ^2 2\theta _{13} = 0.1$ for the [HE, 3D] mode (see the red curve in the right panel of Fig. 6). Under the same condition, if we take $\sin ^2 2\theta _{13} = 0.085$, then the $\Delta \chi ^2$ changes to 9.38.

Constraints on $\varepsilon _{\mu \tau }$ for non-maximal $\theta _{23}$

Global fits of world neutrino data point towards non-maximal $\theta _{23}$ giving rise to two degenerate solutions: one lies in the lower octant (LO) where $\sin ^2\theta _{23} < 0.5$, and the other belongs to the higher octant (HO) where $\sin ^2\theta _{23} > 0.5$ [2,3,4]. Therefore, it would be quite interesting to see how the expected constraints on the NSI parameter $\varepsilon _{\mu \tau }$ at ICAL would be affected if $\theta _{23}$ turns out to be non-maximal in Nature. To study this, we consider the [HE, 3D] binning scheme for which we obtain the best limit on $\varepsilon _{\mu \tau }$ in the previous section assuming $\sin ^2\theta _{23}$ (true) = 0.5. In Fig. 7, we exhibit the performance of the ICAL detector to constrain $\varepsilon _{\mu \tau }$ by simulating the prospective data with $\sin ^2\theta _{23}$ (true) = 0.4 as a benchmark choice in the LO (see the blue dashed line). We also depict the expected constraints on $\varepsilon _{\mu \tau }$ at ICAL assuming $\sin ^2\theta _{23}$ (true) = 0.6 as a benchmark choice in the HO (see the brown dotted line). We compare these results with $\sin ^2\theta _{23}$ (true) = 0.5 case (see the red solid line). To generate all these results, we consider an exposure of 500 kt$\cdot$yr, assume NH as the true mass hierarchy, and perform marginalization over $\theta _{23}$, $\Delta m^{2}_{\mathrm{eff}}$, and both the choices of mass hierarchy as discussed in Sect. 5.2. We observe from Fig. 7 that the expected limits on $\varepsilon _{\mu \tau }$ for non-maximal $\theta _{23}$ values deteriorate slightly as compared to the maximal mixing choice. Also, we notice that the bounds are similar for the octant-symmetric true values of $\sin ^2\theta _{23}$ in the LO (0.4) and HO (0.6) since the survival probabilities of muon neutrino and antineutrino in the presence of non-zero $\varepsilon _{\mu \tau }$ largely depend on $\sin 2\theta _{23}$ instead of $\sin \theta _{23}$ (see Eqs. 11, 12, and 13 ).

Fig. 7

Expected limits on $\varepsilon _{\mu \tau }$ for three different true values of $\sin ^2\theta _{23}$: 0.4 (blue dashed curve), 0.5 (red solid curve), and 0.6 (brown dotted curve). Here, we consider the [HE, 3D] binning scheme, 500 kt$\cdot$yr exposure, and NH as true hierarchy. We obtain these results after performing marginalization over $\theta _{23}$, $\Delta m^{2}_{\mathrm{eff}}$, and both the choices of mass hierarchy

Advantage of having charge identification capability

Fig. 8

In each panel, the red solid (black dashed) line shows the expected sensitivity on $\varepsilon _{\mu \tau }$ with (without) charge identification capability of ICAL. The left (right) panel is for the 2D: $E_{\mu }$, $\cos \theta _{\mu }$ (3D: $E_{\mu }$, $\cos \theta _{\mu }$, $E'_{\mathrm{had}}$) mode assuming the HE binning scheme. We consider 500 kt$\cdot$yr exposure and NH. Here, we keep all the oscillation parameters fixed in the fit (fixed parameter scenario)

The ICAL detector is expected to have a uniform magnetic field of strength around 1.5 T over the entire detector. It will enable the ICAL detector to identify the $\mu ^-$ and $\mu ^+$ events separately by observing the bending of their tracks in the opposite directions in the presence of the magnetic field. We label this feature of ICAL as the charge identification capability. In Ref.  [75], it has been demonstrated that the ICAL detector will have a very good CID efficiency over a wide range of reconstructed $E_\mu$ and $\cos \theta _\mu$. In this work, we estimate for the first time the gain in the sensitivity that ICAL may have in constraining the NSI parameter $\varepsilon _{\mu \tau }$ due to its CID capability. In each panel of Fig. 8, we show the expected sensitivity of ICAL in constraining $\varepsilon _{\mu \tau }$ with (red solid line) and without (black dashed line) CID capability using 500 kt$\cdot$yr exposure and assuming NH. While preparing these plots, we keep the oscillation parameters fixed in the fit and depict the result for the 2D: $E_{\mu }$, $\cos \theta _{\mu }$ (3D: $E_{\mu }$, $\cos \theta _{\mu }$, $E'_{\mathrm{had}}$) mode in the left (right) panel assuming the HE binning scheme. It is apparent from Fig. 8 that the CID capability of ICAL in distinguishing $\mu ^-$ and $\mu ^+$ events plays an important role to make it sensitive to the NSI parameter $\varepsilon _{\mu \tau }$ like the mass hierarchy measurements  [70, 78]. In the following, we quote the 90$\%$ confidence level limits on $\varepsilon _{\mu \tau }$ that the ICAL detector can place with and without CID capabilities for [HE, 2D] and [HE, 3D] modes.

$\bullet$ [HE, 2D] mode (left panel of Fig. 8):

$$\begin{aligned} \mathrm{with\,\,CID:}\,\,&-\,0.015<\varepsilon _{\mu \tau }< 0.017&\mathrm{at}\,\, 90\%\,\mathrm{C.L.} \,,\nonumber \\ \mathrm{without\,\,CID:}\,\,&-\,0.025<\varepsilon _{\mu \tau }< 0.04&\mathrm{at}\,\, 90\%\,\mathrm{C.L.} \, \end{aligned}$$

(30)

$\bullet$ [HE, 3D] mode (right panel of Fig. 8):

$$\begin{aligned} \mathrm{with\,\,CID:}\,\,&-\,0.01<\varepsilon _{\mu \tau }< 0.011& \mathrm{at}\,\, 90\%\,\mathrm{C.L.} \,,\nonumber \\ \mathrm{without\,\,CID:}\,\,&-\,0.018<\varepsilon _{\mu \tau }< 0.025& \mathrm{at}\, \,90\%\,\mathrm{C.L.}\, \end{aligned}$$

(31)

The limits on $\varepsilon _{\mu \tau }$ mentioned in Eqs. 30 and 31 clearly demonstrate the improvement that the ICAL detector can have in constraining the NSI parameter $\varepsilon _{\mu \tau }$ due its CID capability.

Fig. 9

Upper limits on $\varepsilon _{\mu \tau }$ at $3\sigma$ C.L. as a function of run-time for 50 kt ICAL assuming NH and keeping all the oscillation parameters fixed in the fit. Left (right) panel is for LE (HE) binning scheme. In each panel, the black and red lines depict the results for 2D ($E_\mu$, $\cos \theta _\mu$) and 3D ($E_{\mu }$, $\cos \theta _{\mu }$, $E'_{\mathrm{had}}$) modes respectively

Limits on $\varepsilon _{\mu \tau }$ for various exposures

Figure 9 shows the 3$\sigma$ limit on $\varepsilon _{\mu \tau }$ as a function of run-time⁷ for 50 kt ICAL. The left (right) panel is for LE (HE) binning scheme. In each panel, the black and red lines depict the results for 2D ($E_\mu$, $\cos \theta _\mu$) and 3D ($E_{\mu }$, $\cos \theta _{\mu }$, $E'_{\mathrm{had}}$) modes respectively. In Fig. 9, various sensitivity curves are drawn keeping all the oscillation parameters fixed in the fit and assuming NH. Here, we give the results only for positive values of $\varepsilon _{\mu \tau }$. We have checked that the results look similar if we consider negative values of $\varepsilon _{\mu \tau }$ as well. If we take total 250 kt$\cdot$yr exposure (50 kt ICAL with a run-time of 5 years), the expected bound is $|\varepsilon _{\mu \tau }|\lesssim 0.28$ at $3\sigma$ C.L. assuming NH in [HE, 3D] binning scheme. It suggests that ICAL can place competitive constraints on $\varepsilon _{\mu \tau }$ even for less exposure.

Impact of non-zero $\varepsilon _{\mu \tau }$ on mass hierarchy determination

This section is devoted to study how the flavor violating NSI parameter $\varepsilon _{\mu \tau }$ affects the mass hierarchy measurement which is the prime goal of the ICAL detector. We quantify the performance ICAL to rule out the wrong hierarchy by adopting the following $\chi ^2$ expression:

$$\begin{aligned} \Delta \chi ^2_{\mathrm{ICAL-MH}} = \chi ^2_{\mathrm{ICAL}} (\mathrm{false\,\, MH}) - \chi ^2_{\mathrm{ICAL}} (\mathrm{true \,\, MH}). \end{aligned}$$

(32)

Here, we obtain $\chi ^2_{\mathrm{ICAL}} (\mathrm{true\,\,MH})$ and $\chi ^2_{\mathrm{ICAL}} (\mathrm{false\,\,MH})$ by performing the fit to the prospective data assuming true and false mass hierarchy respectively. Since the statistical fluctuations are suppressed in our analysis, $\chi ^2_{\mathrm{ICAL}} (\mathrm{true\,MH}) \approx 0$. First, we estimate the sensitivity of the ICAL detector to determine the neutrino mass hierarchy by adopting the procedure as outlined in Ref.  [78] for the standard case, which we denote as “$\Delta \chi ^2_{\mathrm{{ICAL-MH}}}$ (SM)” in the third column of Table 5. Next, to estimate the mass hierarchy sensitivity in the presence of non-zero $\varepsilon _{\mu \tau }$, we adopt the following strategy. We generate the data with a given mass hierarchy assuming $\varepsilon _{\mu \tau } = 0$. Then, while fitting the prospective data with the opposite hierarchy, we introduce $\varepsilon _{\mu \tau }$ in the fit and marginalize over it in the range of - 0.1 to 0.1 along with the oscillation parameters $\theta _{23}$ and $\Delta m^2_{\mathrm{eff }}$ in their allowed ranges as mentioned in Sect. 5. We label this result as “$\Delta \chi ^2_{\mathrm{{ICAL-MH}}}$ (SM + $\varepsilon _{\mu \tau }$)” in the fourth column of Table 5. We show our results for various choices of binning schemes and observables assuming both true NH and true IH. We consider 500 kt$\cdot$yr exposure of the ICAL detector. We can see from Table 5 that depending on the choice of true mass hierarchy and the analysis mode, the mass hierarchy sensitivity of ICAL gets reduced by 10–20$\%$ due to the presence of non-zero $\varepsilon _{\mu \tau }$ in the fit.

Table 5 The mass hierarchy sensitivity of the ICAL detector using 500 kt$\cdot$yr exposure. For the “SM” case (third column), we do not consider $\varepsilon _{\mu \tau }$ in data and in fit. For the “SM + $\varepsilon _{\mu \tau }$” case (fourth column), we introduce $\varepsilon _{\mu \tau }$ in the fit and marginalize over it in the range of [- 0.1, 0.1] along with oscillation parameters $\theta _{23}$ and $\Delta m^2_{\mathrm{eff }}$. Last column shows how much the mass hierarchy sensitivity deteriorates in presence of $\varepsilon _{\mu \tau }$ as compared to the SM case. We present our results for various choices of binning schemes and observables assuming both true NH and true IH

Precision measurement of atmospheric parameters with non-zero $\varepsilon _{\mu \tau }$

Fig. 10

90$\%$ C.L. (2 d.o.f.) allowed regions in $\sin ^2\theta _{23}$ - $|\Delta m^2_{32}|$ plane for 500 kt$\cdot$yr exposure of the ICAL detector assuming NH. The brown dot represents the true choices of $\sin ^2\theta _{23}$ and $|\Delta m^2_{32}|$. The solid lines show the results for the “SM” case, where we do not consider $\varepsilon _{\mu \tau }$ in data and in fit. The dashed lines portray the results when we introduce $\varepsilon _{\mu \tau }$ in the fit and marginalize over its $\pm 10\%$ range. For other details, see text

Next, we turn our attention to the precise measurement of atmospheric oscillation parameters $\sin ^2\theta _{23}$ and $|\Delta m^2_{32}|$ using 500 kt$\cdot$yr exposure of the ICAL detector. We quantify this performance indicator using the following expression:

$$\begin{aligned}&\Delta \chi ^2_{\mathrm{ICAL-PM}}\, \big (\sin ^2\theta _{23},\, |\Delta m^2_{32}|\big ) \nonumber \\&\quad = \,\, \chi ^2_{\mathrm{ICAL}} \big (\sin ^2\theta _{23},\, |\Delta m^2_{32}|\big ) \,\, - \,\, \chi ^2_0, \end{aligned}$$

(33)

where $\chi ^2_0$ is the minimum value of $\chi ^2_{\mathrm{ICAL}}$ in the allowed parameter range. Since we suppress the statistical fluctuations, we have $\chi ^2_0 \approx 0$. First, considering $\sin ^2\theta _{23}$ (true) = 0.5 and $|\Delta m^2_{32}|$ (true) = 2.4 $\times$ 10$^{-3}$ $\hbox {eV}^2$, we estimate the allowed regions in $\sin ^2\theta _{23}$ - $|\Delta m^2_{32}|$ (test) plane in the absence of $\varepsilon _{\mu \tau }$ at 90$\%$ C.L. (2 d.o.f.). We show these results for the “SM” case using solid lines in Fig. 10 for various analysis modes. For the [HE, 3D] case, we achieve the best precision for the atmospheric parameters, and for the [LE, 2D] case, we have the most conservative results.

Next, we study the impact of non-zero $\varepsilon _{\mu \tau }$ in the precision measurement of atmospheric parameters in the following fashion. We again generate the prospective data considering the true values of $\sin ^2\theta _{23}$ and $|\Delta m^2_{32}|$ as mentioned above. Then, while estimating the allowed regions in $\sin ^2\theta _{23}$ - $|\Delta m^2_{32}|$ (test) plane, we introduce $\varepsilon _{\mu \tau }$ in the fit and marginalize over it in the range of [- 0.1, 0.1]. We present these results for the “SM + $\varepsilon _{\mu \tau }$” case at $90\%$ C.L. (2 d.o.f.) with the help of dashed lines in Fig. 10 for various analysis modes. We do not see any appreciable change in the contours when we introduce $\varepsilon _{\mu \tau }$ in the fit and vary in its $\pm 10\%$ range. It suggests that the precision measurement of atmospheric oscillation parameters at the ICAL detector is quite robust even if we marginalize over $\varepsilon _{\mu \tau }$ in the fit. Similar results were obtained by the Super-Kamiokande Collaboration in Ref.  [63], where they studied the impact of NSI’s in $\nu _\mu$-$\nu _\tau$ sector using their Phase I and Phase II atmospheric data.

Summary and concluding remarks

In this paper, we explore the possibility of lepton flavor violating neutral current non-standard interactions (NSI’s) of atmospheric neutrino and antineutrino while they travel long distances inside the Earth matter before reaching to the ICAL detector. During the propagation of these neutrinos, we allow an extra interaction vertex with $\nu _\mu$ as the incoming particle and $\nu _\tau$ as the outgoing one and vice versa. With such an interaction vertex, the neutral current non-standard interaction of neutrino with matter fermions gives rise to a new matter potential whose relative strength as compared to the standard matter potential ($V_{CC}$) is denoted by $\varepsilon _{\mu \tau }$.

We exhibit for the first time that by adding the hadron energy information along with the reconstructed muon energy and muon direction in each event, the sensitivity of ICAL to the NSI parameter $\varepsilon _{\mu \tau }$ can be enhanced significantly. We find that considering reconstructed $E_\mu$ (in the range of 1–11 GeV) and $\cos \theta _\mu$ (in the range of - 1 to 1) as observables, the expected limit on $\varepsilon _{\mu \tau }$ at 90$\%$ C.L. is $-\,0.03< \varepsilon _{\mu \tau } <0.03$. If we increase the muon energy range further from 11 to 21 GeV ($E_\mu \in [1, 21]$ GeV) and consider the reconstructed hadron energy ($E'_{\mathrm{had}}$) as an extra observable on top of the four momenta of muon ($E_\mu ,\,\cos \theta _\mu$), we find a significant improvement in the limit which is $-\,0.01< \varepsilon _{\mu \tau } < 0.01$ at 90$\%$ C.L. using 500 kt$\cdot$yr exposure of the ICAL detector. We also demonstrate for the first time that the charge identification capability of the ICAL detector plays an important role to obtain these tight constraints on $\varepsilon _{\mu \tau }$ as mentioned above.

Assuming 1–21 GeV reconstructed muon energy range and considering $E_\mu ,\,\cos \theta _\mu ,\,$and $E'_{\mathrm{had}}$ as observables, we find that the mass hierarchy sensitivity at the ICAL detector deteriorates by $\sim 10\%$ if we introduce the NSI parameter $\varepsilon _{\mu \tau }$ in the fit and marginalize over it in the range of $-\,$0.1 to 0.1 along with other standard oscillation parameters. On the other hand, the precision measurement of atmospheric oscillation parameters at the ICAL detector is quite robust even if we marginalize over the NSI parameter $\varepsilon _{\mu \tau }$ in fit in the range $-\,$0.1 to 0.1.

Before we conclude, we would like to emphasize that though the expected limit on $\varepsilon _{\mu \tau }$ at ICAL is weaker than IceCube and DeepCore bounds, but, these results are complementary to each other. IceCube extracts information on the NSI parameters using neutrinos whose energies are above 300 GeV. At these high energies (which are relevant for IceCube), oscillation probabilities are completely dominated by NSIs and there is hardly any impact of vacuum oscillations. As far as DeepCore is concerned, they use neutrinos in their analysis whose energies are above 10 GeV or so, and at these energies, NSIs start to dominate the flavor transition. The proposed ICAL detector is very efficient in the neutrino energy range of 1–10 GeV and it is going to provide a complementary information on the NSI parameter $\varepsilon _{\mu \tau }$ in this energy range which is not accessible by IceCube and DeepCore. Moreover, at these energies (where ICAL is very effective), both vacuum oscillations and matter driven NSIs affect the flavor transitions. Therefore, it is quite important to probe these NSI parameters at different energies using different detectors, since they provide complementary information on these parameters.

Another important feature of ICAL is that due to the presence of magnetic field, it provides an opportunity to probe the NSI parameters in neutrino (by observing $\mu ^{-}$ events) and antineutrino (by observing $\mu ^{+}$ events) modes separately. Therefore, ICAL provides an unique platform to test some fundamental symmetries of Nature such as Charge-Parity-Time (CPT) symmetry, whose tiny violation may give rise to different mass, mixing, and NSI parameters for neutrinos and antineutrinos. This study may not be possible using any other existing or planned water-, ice-, scintillator-, or argon-based detectors.