## Abstract

In this paper, we explore the impact of flavor violating neutral current non-standard interaction (NSI) parameter \(\varepsilon _{\mu \tau }\) in the oscillation of atmospheric neutrinos and antineutrinos separately using the 50 kt magnetized ICAL detector at INO. We find that due to non-zero \(\varepsilon _{\mu \tau }\), \(\nu _\mu \rightarrow \nu _\mu \) and \({\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu \) transition probabilities get modified substantially at higher energies and longer baselines, where vacuum oscillation dominates. We demonstrate for the first time that by adding the hadron energy information along with the muon energy and muon direction in each event, the sensitivity of ICAL to the NSI parameter \(\varepsilon _{\mu \tau }\) can be enhanced significantly. The most optimistic bound on \(\varepsilon _{\mu \tau }\) that we obtain is \(-\,\,0.01< \varepsilon _{\mu \tau } < 0.01 \) at 90\(\%\) C.L. using 500 kt\(\cdot \)yr exposure and considering \(E_\mu \), \(\cos \theta _\mu \), and \(E'_{\mathrm{had}}\) as observables in their ranges of [1, 21] GeV, [− 1, 1], and [0, 25] GeV, respectively. We discuss for the first time the importance of the charge identification capability of the ICAL detector to have better constraints on \(\varepsilon _{\mu \tau }\). We also study the impact of non-zero \(\varepsilon _{\mu \tau }\) on mass hierarchy determination and precision measurement of oscillation parameters.

## Introduction

The observation of neutrino oscillation proves that the neutrinos are massive and mix with each other, which is the first direct evidence for physics beyond the Standard Model [1]. Currently, the three-flavor lepton mixing is confirmed and neutrino oscillation has entered the era of precision [2,3,4]. To explain the tiny masses of neutrino and large leptonic mixing angles, the extensions of the Standard Model allow some interactions which are not possible in the Standard Model. These interactions are termed as non-standard interactions (NSI’s). The presence of NSI’s in Nature can have subdominant effect on the oscillation of neutrino and antineutrino. Therefore, the phenomenological consequences of NSI’s in three-flavor mixing using neutrino oscillation experiments are interesting and widely studied by many authors in Refs. [5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60].

In this paper, we study the impact of neutral current (NC) non-standard interactions of neutrino which may arise when atmospheric neutrinos travel long distances inside the Earth. While NC NSI’s affect neutrinos during their propagation, there are charged-current NSI’s which may modify the neutrino fluxes at the production stage and interaction cross-section at the detection level. In this work, we only focus on the NC NSI’s, and do not consider NSI’s at production and detection level. In most of the cases, NSI’s come into the picture as a low-energy manifestation of high-energy theory involving new heavy states. For a detailed discussion on this topic, see the reviews [40, 54, 56, 61]. Therefore, at low energies, a neutral current NSI can be described by a four-fermion dimension-six operator [62],

where \(G_{F}\) is the Fermi coupling constant, \(\varepsilon ^{C\,f}_{\alpha \beta }\) is the dimensionless parameter which represents the strength of NSI relative to \(G_{F}\), and \(\nu _\alpha \) and \(\nu _\beta \) are the neutrino fields of flavor \(\alpha \) and \(\beta \) respectively. Here, *f* denotes the matter fermions, electron (*e*), up-quark (*u*), and down-quark (*d*). Here, \(P_L = (1 - \gamma _5)/2\), \(P_R = (1 + \gamma _5)/2\), and the subscript \(C = L,\,R\) expresses the chirality of the *ff* current. Due to the hermiticity of the interaction, we have \(\varepsilon ^{C\,f}_{\alpha \beta } = \left( \varepsilon ^{C\,f}_{\beta \alpha }\right) ^*\).

The NSI’s of neutrino with matter fermions can give rise to the additional matter induced potentials apart from the standard MSW potential due to the *W*-mediated interactions in matter between electron neutrino and ambient electron, which takes the form \(\sqrt{2} G_{F} N_{e}\) (\(\equiv V_{CC}\)). The total relative strength of the matter induced potential generated by the NC NSI’s of neutrinos with all the matter fermions (\(\nu _\alpha + f \rightarrow \nu _\beta + f \)) can be written in the following fashion,

where, \(V_f\) = \(\sqrt{2}\) \(G_{F}\, N_{f}\), \(f=e, u, d\). The quantity \(N_f\) denotes the number density of matter fermion *f* in the medium. For antineutrino, \(V_f \rightarrow -V_f\) and \(V_{CC} \rightarrow -V_{CC}\). In general, the total matter induced potential in presence of all the possible NC non-standard interactions of neutrino with matter fermions can be written as

In the present study, we focus our investigation to flavor violating NSI parameter \(\varepsilon _{\mu \tau }\), that is, we only allow \(\varepsilon _{\mu \tau }\) to be non-zero in our analysis, and assume all other NSI parameters to be zero. We also consider \(\varepsilon _{\mu \tau }\) to be real entertaining both of its negative and positive values. Since the atmospheric neutrino oscillation is mainly governed by \(\nu _\mu \rightarrow \nu _\tau \) transition, it is expected that NSI parameter \(\varepsilon _{\mu \tau }\) would have significant impact on this oscillation channel, which in turn can modify \(\nu _\mu \rightarrow \nu _\mu \) survival probability by a considerable amount. We can study this effect by observing the atmospheric neutrinos at the proposed 50 kt magnetized ICAL detector. If we will not see any significant deviation from the standard \(\mu ^-\) and \(\mu ^+\) event spectra at ICAL, we can use this fact to place tight constraints on NSI parameter \(\varepsilon _{\mu \tau }\). This is the main theme of our present study.

This article is organized in the following fashion. In Sect. 2, we briefly review the existing bounds on NSI parameter \(\varepsilon _{\mu \tau }\) from various neutrino oscillation experiments. We discuss the possible modification in oscillation probabilities of neutrino and antineutrino due to non-zero \(\varepsilon _{\mu \tau }\) in Sect. 3. In Sect. 4, we present the expected total \(\mu ^-\) and \(\mu ^+\) events and their distributions as a function of reconstructed \(E_\mu \) and \(\cos \theta _\mu \) for the following three cases: (1) \(\varepsilon _{\mu \tau } = 0\) (SM), (2) \(\varepsilon _{\mu \tau } = 0.05\), and (3) \(\varepsilon _{\mu \tau } = -\,\,0.05\) using 500 kt\(\cdot \)yr exposure of the ICAL detector. In Sect. 5, we discuss the numerical procedure and various binning schemes that we use in our analysis. We present all the results of our study in Sect. 6 where we show the following: (a) The possible improvement in the sensitivity of the ICAL detector in constraining \(\varepsilon _{\mu \tau }\) due to the inclusion of events with \(E_\mu \) in range of 11–21 GeV in addition to the events that belong to the \(E_\mu \) in range of 1–11 GeV. (b) How much the limit on \(\varepsilon _{\mu \tau }\) can be improved by considering the information on reconstructed hadron energy (\(E'_{\mathrm{had}}\)) as an additional observable along with reconstructed variables \(E_\mu \) and \(\cos \theta _\mu \) on an event-by-event basis. (c) We study the impact of non-maximal \(\theta _{23}\) while deriving limits on \(\varepsilon _{\mu \tau }\) at ICAL. (d) We show the advantage of having charge identification (CID) capability in the ICAL detector in placing competitive constraint on \(\varepsilon _{\mu \tau }\). (e) We present the expected limits on \(\varepsilon _{\mu \tau }\) considering different exposures of the 50 kt ICAL detector. (f) We also explore the possible impact of non-zero \(\varepsilon _{\mu \tau }\) in determining the mass hierarchy and in the precision measurement of atmospheric oscillation parameters. We provide a summary of this study in Sect. 7.

## Existing limits on NSI parameter \(\varepsilon _{\mu \tau }\)

There are existing constraints on the NSI parameter \(\varepsilon _{\mu \tau }\) from various neutrino oscillation experiments. The Super-Kamiokande collaboration performed an analysis of the atmospheric neutrino data collected during its phase-I and -II run assuming only NSI’s with *d*-quarks [63]. The following bounds at \(90\%\) C.L. are obtained:

Since \(N_d = N_u = 3N_e\) for an electrically neutral and isoscalar Earth matter, the above constraints as obtained in Ref. [63] are actually on the NSI parameters \(\varepsilon _{\alpha \beta }/3\). Therefore, the above constraints at \(90\%\) C.L. can be interpreted as

Recently, the authors in Ref. [64] considered the possibility of NSI’s in \(\mu \)-\(\tau \) sector in the one-year high-energy through-going muon data of IceCube. In their analysis, they included various systematic uncertainties on both the atmospheric neutrino flux and detector properties, which they incorporated via several nuisance parameters. They obtained the following limits

The IceCube-DeepCore collaboration also searched for NSI’s involving \(\varepsilon _{\mu \tau }\) [65]. Using their three years of atmospheric muon neutrino disappearance data, they placed the following constraint at \(90\%\) confidence level

A preliminary analysis to constrain the NSI parameters in context of the ICAL detector was performed in Ref. [55]. Using an exposure of 500 kt\(\cdot \)yr and considering only muon momentum (\(E_\mu , \cos \theta _\mu \)) as observable, the authors in Ref. [55] obtained the following bound

In the present study, we estimate new constraints on \(\varepsilon _{\mu \tau }\) considering the reconstructed hadron energy (\(E'_{\mathrm{had}}\)) as an additional observable along with the reconstructed \(E_\mu \) and \(\cos \theta _\mu \) on an event-by-event basis at the ICAL detector.

## \(\nu _\mu \rightarrow \nu _\mu \) transition with non-zero \(\varepsilon _{\mu \tau }\)

This section is devoted to explore the effect of non-zero \(\varepsilon _{\mu \tau }\) in the oscillation of atmospheric neutrino and antineutrino propagating long distances through the Earth matter. For this, we numerically estimate the three-flavor oscillation probabilities including NSI parameter \(\varepsilon _{\mu \tau }\) and using the PREM profile [66] for the Earth matter density. The NSI parameter \(\varepsilon _{\mu \tau }\) modifies the evolution of neutrino in matter, which in the flavor basis takes the following form,

where \(\varepsilon _{\mu \tau }\) is real in our analysis.

In upper panels of Fig. 1, we present the oscillograms for \(\nu _\mu \) survival channel in the plane of \(\cos \theta _\nu \) vs. \(E_\nu \) considering NH. Here, we draw the oscillograms for three different cases: (1) \(\varepsilon _{\mu \tau } = -\,\,0.05\) (left panel), (2) \(\varepsilon _{\mu \tau }=0.0 \) (the SM case, middle panel), and (3) \(\varepsilon _{\mu \tau } = 0.05\) (right panel). The lower panel depicts the same but for \({\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu \) oscillation channel. To prepare Fig. 1, we take the following benchmark values of vacuum oscillation parameters in three-flavor framework: \(\sin ^2\theta _{23} =0.5\), \(\sin ^2 2\theta _{13} =0.1\), \(\sin ^2 \theta _{12} =0.3\), \(\Delta m^2_{21} = 7.5 \times 10^{-5}\) \(\hbox {eV}^2\), \(\Delta m_{\mathrm{eff}}^{2} = 2.4 \times 10^{-3}\) \(\hbox {eV}^2\), and \(\delta _{\text {CP}} =0^\circ \). We estimate the value of \(\Delta m^2_{31}\) from \(\Delta m_{\mathrm{eff}}^{2}\)^{Footnote 1}where \(\Delta m_{\text {eff}}^{2}\) has the same magnitude for NH and IH with positive and negative signs respectively. It is evident from the upper panels of Fig. 1 that in the presence of negative (see left panel) and positive (see right panel) non-zero values of \(\varepsilon _{\mu \tau }\), \(\nu _\mu \) survival probabilities get modified substantially at higher energies and longer baselines, where vacuum oscillation dominates. We observe the similar changes in case of \({\bar{\nu }}_\mu \) oscillation probabilities as well (see lower panels).

Another broad feature which has been emerging from Fig. 1 is that the \(\nu _\mu \rightarrow \nu _\mu \) oscillation probabilities with positive (negative) \(\varepsilon _{\mu \tau }\) as shown in upper right (left) panel are similar to that of \({\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu \) transition probabilities with negative (positive) \(\varepsilon _{\mu \tau }\) as can be seen from lower left (right) panel. We can understand this behaviour with the help of following approximate analytical expression of \(\nu _\mu \rightarrow \nu _\mu \) transition probability. Assuming \(\Delta m^2_{21} L/4E \rightarrow 0\) and \(\theta _{13} = 0\), \(\nu _\mu \rightarrow \nu _\mu \) oscillation channel in the presence of non-zero \(\varepsilon _{\mu \tau }\) and under the constant matter density approximation takes the form [11, 21]

where

and

In Eqs. 12 and 13, positive and negative signs in front of \(\eta _{\mu \tau }\) are associated with the normal and inverted hierarchy respectively. In case of maximal mixing (\(\theta _{23} = 45^{\circ }\)), Eq. 11 boils down to the following simple expression [69]

Since for antineutrino, \(V_{CC} \rightarrow -V_{CC}\), following Eq. 15, we can write

and

Equations 16 and 17 explain the broad features in Fig. 1 that we mention above.

To have a better look at the changes induced by non-zero \(\varepsilon _{\mu \tau }\) as compared to the SM case, we give Fig. 2 where we plot the difference in \(\nu _\mu \rightarrow \nu _\mu \) survival channel considering the cases \(\varepsilon _{\mu \tau } = 0\) (the SM case) and \(\varepsilon _{\mu \tau } = -\,\,0.05\) (see top left panel). In top right panel, we present the same for the cases of \(\varepsilon _{\mu \tau } = 0\) (the SM case) and \(\varepsilon _{\mu \tau } = 0.05\). The lower panels are for antineutrinos. In all the panels, we see a visible difference in \(\nu _\mu \) survival channel due to the presence of non-zero \(\varepsilon _{\mu \tau }\) as compared to the SM case (\(\varepsilon _{\mu \tau } = 0.0\)) at higher baselines with \(\cos \theta _\nu \) in the range \(-\,\,1\) to \(-\,\,0.8\). This range of \(\cos \theta _\nu \) corresponds to the baseline in the range \(\sim \) 12,700 to 10,000 km where neutrino and antineutrino mostly travel through inner and outer part of the Earth’s core^{Footnote 2} and have access to large Earth matter effect. Also, we see a trend that the impact of NSI’s is large at higher energies where the three-flavor oscillations are suppressed because the oscillation lengths (\(L^{\mathrm{osc}} = \frac{4 \pi E}{ \Delta m^2_{ij} } \)) are large at higher energies.

To have a quantitative estimate of the difference in \(\nu _\mu \rightarrow \nu _\mu \) oscillation channel due to non-zero \(\varepsilon _{\mu \tau }\) as compared to \(\varepsilon _{\mu \tau } =0\) case, we use Eq. 15 and obtain the following expression

Equation18 clearly suggests that the impact of NSI is proportional to both NSI induced matter potential (\(V_{CC} \varepsilon _{\mu \tau }\)) and baseline (*L*). We see in upper left and lower right panels of Fig. 2 that around \(E\sim 18\) GeV and \(\cos \theta _\nu \sim -\, 0.9\), \( P^{\mathrm{diff}}_{\mu \mu }\) approaches to zero suggesting that the impact of NSI is negligible. We observe the opposite behaviour in upper right and lower left panels, where around \(E\sim 18\) GeV and \(\cos \theta _\nu \sim -\,0.9\), \( P^{\mathrm{diff}}_{\mu \mu }\) attains a quite large value of \(-\, 0.8\) suggesting that the influence of NSI is significant there. Here, \(\cos \theta _\nu = -\,0.9\) corresponds to \(L = 11500\) km for which the line-averaged constant Earth matter density according to the PREM [66] profile is 6.8 g/\(\hbox {cm}^3\). Therefore, the standard line-averaged Earth matter potential^{Footnote 3} for 11,00 km baseline is \(V_{CC}\sim 2.6\times 10^{-13}\) eV. Considering \(\varepsilon _{\mu \tau } = 0.05\), we get

For \(E = 18\) GeV, \(L =11,500\) km, and \(\Delta m^2_{31} = 2.36\times 10^{-3}\) \(\hbox {eV}^2\) (this mass-squared difference is obtained from Eq. 10 using benchmark values of oscillation parameters),

Thus, for \(E = 18\) GeV and \(L = 11,500\) km, from Eq. 18, we obtain

and

Equations 22 and 23 confirm the observations regarding \(P^{\mathrm{diff}}_{\mu \mu }\) and \(P^{\mathrm{diff}}_{{\bar{\mu }}{\bar{\mu }}}\) in Fig. 2 that we mention above. We know that due to its CID capability, ICAL has an edge to resolve the issue of neutrino mass hierarchy (sign of \(\Delta m^2_{31}\)) by observing the Earth matter effect in \(\mu ^-\) and \(\mu ^+\) events separately [70]. Similarly, the four panels in Fig. 2 suggest that the CID capability of ICAL can provide useful information to determine the sign of NSI parameter \(\varepsilon _{\mu \tau }\) for a particular choice of mass hierarchy.

## Expected events at ICAL with non-zero \(\varepsilon _{\mu \tau }\)

The Monte Carlo based neutrino event generator NUANCE [71] is used to simulate the CC interactions of \(\nu _\mu \) and \({\bar{\nu }}_\mu \) in the ICAL detector. To generate events in NUANCE, we give a simple geometry of the ICAL detector with 150 alternate layers of iron and glass plates in each module. We have three such modules to account for the 50 kt ICAL detector. As far as the neutrino flux is concerned in generating the neutrino events in the present study, we use the flux as predicted at Kamioka^{Footnote 4} [74]. To reduce the statistical fluctuation, we generate the unoscillated CC neutrino and antineutrino events considering a very high exposure of 1000 years and 50 kt ICAL. Then, we implement various oscillation probabilities using the reweighting algorithm. Next, we fold the oscillated events with detector response for muon and hadron as described in Ref. [75, 76]. In the present study, we assume that the ICAL particle reconstruction algorithms can separate the hits due to the hadron shower from the hits originating from a muon track with 100\(\%\) efficiency. It means that whenever a muon is reconstructed, we consider all the other hits to be part of the hadronic shower in order to calibrate the hadron energy. It also implies that the neutrino event reconstruction efficiency is same as the muon reconstruction efficiency. Finally, the reconstructed \(\mu ^-\) and \(\mu ^+\) events are scaled down to the exposure of 10 years for 50 kt ICAL. Now, we present the expected \(\mu ^-\) and \(\mu ^+\) events for 500 kt\(\cdot \)yr exposure of the ICAL detector assuming the SM case (\(\varepsilon _{\mu \tau } = 0\)) and \(\varepsilon _{\mu \tau } = \pm \, 0.05\). To estimate these event rates, we use the values of oscillation parameters as considered in Sect. 3 to draw the oscillograms.

### Total event rates

First, we address the following question: can we see the signature of non-zero \(\varepsilon _{\mu \tau }\) in the total number of \(\mu ^-\) and \(\mu ^+\) events which will be collected at the ICAL detector over 10 years of running? To have an answer of this question, we estimate the total number of events for the following three cases: (1) \(\varepsilon _{\mu \tau } = 0.05\), (2) \(\varepsilon _{\mu \tau } =0\) (the SM case), and (3) \(\varepsilon _{\mu \tau } = -\,0.05\). We present these numbers in Table 1 with NH and using 500 kt\(\cdot \)yr exposure of the ICAL detector integrating over entire ranges of \(E_\mu \), \(\cos \theta _\mu \), and \(E'_{\mathrm{had}}\) that we consider in our analysis. As far as the binning schemes are concerned, we use the low-energy (LE) and high-energy (HE) binning schemes^{Footnote 5}, and for both these binning schemes, we take the entire range of \(\cos \theta _\mu \) spanning over − 1 to 1. The energy ranges for reconstructed \(E_\mu \) and \(E'_\mathrm{had}\) are different in LE and HE binning schemes. For LE binning scheme, \(E_\mu \in \) [1, 11] GeV and \(E'_{\mathrm{had}} \in \) [0, 15] GeV. In case of HE binning scheme, \(E_\mu \in \) [1, 21] GeV, and \(E'_{\mathrm{had}} \in \) [0, 25] GeV. When we increase the reconstructed muon energy from 11 to 21 GeV and reconstructed hadron energy from 15 to 25 GeV, the number of \(\mu ^-\) and \(\mu ^+\) events get increased by 300 and 150 respectively for 500 kt\(\cdot \)yr exposure of the ICAL detector. Apart from showing the total \(\mu ^-\) event rates in Table 1, we also present the estimates of individual events coming from \(\nu _\mu \rightarrow \nu _\mu \) (\(P_{\mu \mu }\)) disappearance channel and \(\nu _e\rightarrow \nu _\mu \) (\(P_{e\mu }\)) appearance channel. Also, for \(\mu ^+\) events, we separately show the contributions originating from \({\bar{\nu }}_\mu \rightarrow {\bar{\nu }}_\mu \) (\(P_{{\bar{\mu }}{\bar{\mu }}}\)) disappearance and \({\bar{\nu }}_e\rightarrow {\bar{\nu }}_\mu \) (\(P_{\bar{e}{\bar{\mu }}}\)) appearance channels. Here, we see that only \(\sim 2 \%\) of the total \(\mu ^-\) events at the ICAL detector come via the appearance channel. Note that the differences in the total number of \(\mu ^-\) and \(\mu ^+\) events between the SM case (\(\varepsilon _{\mu \tau } = 0\)) and non-zero \(\varepsilon _{\mu \tau }\) of \(\pm 0.05\) are not significant. But, later while presenting our final results, we see that the ICAL detector can place competitive constraints on \(\varepsilon _{\mu \tau }\) by exploiting the useful information contained in the spectral distributions of \(\mu ^-\) and \(\mu ^+\) events as a function of reconstructed observables \(E_\mu \), \(\cos \theta _\mu \), and \(E'_{\mathrm{had}}\). To establish this claim, now, we show how the expected \(\mu ^-\) and \(\mu ^+\) event spectra get modified in the presence of non-zero \(\varepsilon _{\mu \tau }\) in terms of reconstructed \(E_\mu \) and \(\cos \theta _\mu \) while integrating over entire range of \(E'_{\mathrm{{had}}}\).

### Event spectra

In Fig. 3, we show the distributions of \(\mu ^-\) (upper panels) and \(\mu ^+\) (lower panels) events as a function of reconstructed \(\cos \theta _\mu \) for three different ranges of \(E_\mu \). The ranges of \(E_\mu \) that we consider in left, middle, and right panels are [3, 4] GeV, [5, 11] GeV, and [11, 21] GeV respectively. Here, we integrate over \(E'_{\mathrm{{had}}}\) in its entire range of 0–25 GeV. In each panel, we compare the event spectra for three different cases: (1) \(\varepsilon _{\mu \tau } = 0.05\) (blue line), (2) \(\varepsilon _{\mu \tau } =0\) (the SM case, black line), and (3) \(\varepsilon _{\mu \tau } = -\,\,0.05\) (red line). Before we discuss the impact of non-zero \(\varepsilon _{\mu \tau }\), we would like to mention few general features that are emerging from various panels in Fig. 3. For both \(\mu ^-\) (upper panels) and \(\mu ^+\) (lower panels), the number of events get reduced as we go to higher energies. It happens because of \(\sim E_\nu ^{-2.7}\) dependence of the atmospheric neutrino flux. Though the neutrino fluxes are higher along the horizontal direction (\(\cos \theta _\mu \) around 0) as compared to the other directions (for detailed discussion, see Ref. [73]), but, due to the poor reconstruction efficiency of the ICAL detector along the horizontal direction [75], we see a suppression in \(\mu ^-\) and \(\mu ^+\) event rates around \(\cos \theta _\mu \in [-\,0.1,\,0]\) irrespective of the choices of \(E_\mu \) ranges. Important to note that as we proceed towards higher \(E_\mu \), the relative differences in \(\mu ^-\) and \(\mu ^+\) event rates between the SM case (\(\varepsilon _{\mu \tau } = 0\)) and non-zero \(\varepsilon _{\mu \tau }\) (\(\pm 0.05\)) get increased for a wide range of \(\cos \theta _\mu \). We see similar features in Fig. 2 in Sect. 3, where we show the differences in \(\nu _\mu \rightarrow \nu _\mu \) oscillograms due to the SM case (\(\varepsilon _{\mu \tau } = 0\)) and non-zero \(\varepsilon _{\mu \tau }\) (\(\pm 0.05\)). We show the improvement in the sensitivity to constrain \(\varepsilon _{\mu \tau }\) due to high energy events in Sect. 6.1. Next, we discuss the numerical technique and analysis procedure which we adopt to obtain the final results.

## Simulation method

### Binning scheme in \(\left( E_{\mu }, \cos \theta _{\mu }, E'_{\mathrm{had}}\right) \) plane

In the present study, we produce all the results with low-energy (LE) and high-energy (HE) binning schemes. Table 2 shows the detailed information about the LE binning scheme for the three reconstructed observables \(E_{\mu }\), \(\cos \theta _{\mu }\), and \(E'_{\mathrm{had}}\). Table 3 portrays the same for the HE binning scheme. In case of LE binning scheme, the range of \(E_{\mu }\) is [1, 11] GeV with total 10 bins each having a width of 1 GeV. In case of HE binning scheme, we extend the range of \(E_{\mu }\) up to 21 GeV by adding two additional bins in the range of 11 to 21 GeV, where each bin has a width of 5 GeV. As far as reconstructed \(E'_{\mathrm{had}}\) is concerned, in case of LE (HE) binning scheme, the considered range is 0–15 GeV (0–25 GeV). We can see from Table 2 and Table 3 that the first three bins for \(E'_{\mathrm{had}}\) are same for both the binning schemes, whereas the last bin extend from 4 to 15 GeV (4 to 25 GeV) for LE (HE) binning scheme. For both these binning schemes, we consider the entire range of \(\cos \theta _{\mu }\) from − 1 to 1. For upward going events, that is \(\cos \theta _{\mu }\in \,\) [- 1, 0], we consider 10 uniform bins each having width of 0.1. For downward going events, that is \(\cos \theta _{\mu }\in \,\) [0, 1], we consider 5 uniform bins each having width of 0.2. Important to note that the downward going events do not have enough path length to oscillate, but, these events play an important role to increase the overall statistics and to minimize the effect of normalization uncertainties in atmospheric neutrino fluxes. Here, we would like to mention that we have not optimized these binning schemes to obtain the best sensitivities, but we ensure that there are sufficient statistics in most of the bins.

### Numerical analysis

In our analysis, the \(\chi ^2\) function gives us the median sensitivity of the experiment in the frequentist approach [77]. We use the following Poissonian \(\chi ^2_{-}\) for \(\mu ^-\) events in our statistical analysis considering \(E_\mu \), \(\cos \theta _\mu \), and \(E'_{\mathrm{had}}\) as observables (the so-called “3D” analysis as considered in [78]):

with

In the above equations, \(N^{\mathrm{data}}_{ijk}\) and \(N^\mathrm{theory}_{ijk}\) denote the observed and expected number of \(\mu ^-\) events in a given [\(E_\mu \), \(\cos \theta _\mu \), \(E'_{\mathrm{had}}\)] bin. In case of LE (HE) binning scheme, \(N_{E_\mu }\) = 10 (12), \(N_{\cos \theta _\mu } = 15\), and \(N_{E'_{\mathrm{had}}} = 4\). In Eq. 25, \(N^0_{ijk}\) represents the number of expected events without systematic uncertainties. Following Ref. [79], we consider five systematic errors in our analysis: 20\(\%\) flux normalization error, 10\(\%\) error in cross-section, 5\(\%\) tilt error, 5\(\%\) zenith angle error, and 5\(\%\) overall systematics. We incorporate these systematic uncertainties in our simulation using the well known “pull” method [21, 80, 81]. In Eqs. 24 and 25, the quantities \(\zeta _{l}\) denote the “pulls” due to the systematic uncertainties.

When we produce results with only \(E_\mu \) and \(\cos \theta _\mu \) as observables and do not use the information on hadron energy \(E'_\mathrm{had}\) (the so-called “2D” analysis as considered in Ref. [79]), the Poissonian \(\chi ^2_{-}\) for \(\mu ^-\) events takes the form

with

In Eq. 26, \(N^{\mathrm{data}}_{jk}\) and \(N^\mathrm{theory}_{jk}\) indicate the observed and expected number of \(\mu ^-\) events in a given [\(E_\mu \), \(\cos \theta _\mu \)] bin. In Eq. 27, \(N^0_{jk}\) stands for the number of expected events without systematic errors. In case of LE (HE) binning scheme, \(N_{E_\mu }\) = 10 (12) and \(N_{\cos \theta _\mu } = 15\).

For both the “2D” and “3D” analyses, the \(\chi ^2_+\) for \(\mu ^+\) events is determined following the same technique described above. We add the individual contributions from \(\mu ^-\) and \(\mu ^+\) events to estimate the total \(\chi ^2\) for both the “2D” and “3D” schemes:

In our analysis, we simulate the prospective data considering the following benchmark values of the oscillation parameters: \(\sin ^2 \theta _{23} = 0.5\), \(\,\sin ^2 2\theta _{13}= 0.1\), \(\sin ^2 \theta _{12}\, =\, 0.3\), \(\Delta m^2_{21}\, =\, 7.5\times 10^{-5}\,\mathrm{eV^2}\), and \(|\Delta m^2_{\mathrm{eff}}|\, =\, 2.4\times 10^{-3}\, \mathrm{eV^2}\). To estimate the value of \(\Delta m^2_{31}\) from \(\Delta m_{\mathrm{eff}}^{2}\), we use the Eq. 10, where \(\Delta m_{\text {eff}}^{2}\) has the same magnitude for NH and IH with +ve and -ve signs respectively. In the fit, we first minimize \(\chi ^2_{\mathrm{ICAL}}\) (see Eq. 28) with respect to the “pull” variables \(\zeta _l\), and then marginalize over the oscillation parameters \(\sin ^2\theta _{23}\) in the range [0.36, 0.66], \(|\Delta m_{\text {eff}}^{2}|\) in the range \([2.1, 2.6]\times 10^{-3}\) \(\hbox {eV}^2\), and over both the choices of mass hierarchy, NH and IH, while keeping \(\theta _{12}\), \(\Delta m^2_{21}\), \(\sin ^2 2\theta _{13}\) fixed at their benchmark values. We consider \(\delta _{\mathrm{CP}} = 0^{\circ }\) throughout our analysis.

## Results

### Expected bounds on NSI parameter \(\varepsilon _{\mu \tau }\)

We quantify the statistical significance of the analysis to constrain the NSI parameter \(\varepsilon _{\mu \tau }\) in the following fashion

Here, \(\chi ^2_{\,\,\mathrm{ICAL}} (\mathrm{SM})\) and \(\chi ^2_{\,\,\mathrm{ICAL}}\left( \mathrm{SM} + \varepsilon _{\mu \tau }\right) \) are calculated by fitting the prospective data with zero (the SM case) and non-zero value of NSI parameter \(\varepsilon _{\mu \tau }\) respectively. In our analysis procedure, statistical fluctuations are suppressed, and therefore, \(\chi ^2_{\,\,\mathrm{ICAL}}(\mathrm{SM})\approx 0\).

Let us first identify the regions in \(\cos \theta _\mu \) and \(E_\mu \) plane which give significant contributions towards \(\Delta \chi ^2_{\mathrm{ICAL-NSI}}\). In Fig. 4, we show the distribution^{Footnote 6} of \(\Delta \chi ^2_{-}\) from \(\mu ^-\) events in the reconstructed [\(\cos \theta _\mu \)-\(E_\mu \)] plane using 500 kt\(\cdot \)yr exposure of the ICAL detector and assuming NH. In all the panels of Fig. 4, we consider \(\varepsilon _{\mu \tau } = 0.05\) in the fit and show the results for the following four different choices of binning schemes and observables: (1) top left panel: [LE, 2D], (2) top right panel: [LE, 3D], (3) bottom left panel: [HE, 2D], (4) bottom right panel: [HE, 3D]. We show the distribution of \(\Delta \chi ^2_{+}\) from \(\mu ^+\) events in the plane of reconstructed \(\cos \theta _\mu \) and \(E_\mu \) for these four different cases in Fig. 5 considering \(\varepsilon _{\mu \tau } = 0.05\) in the fit. In left panels of Figs. 4 and 5, we show the sensitivity in the plane of reconstructed \(\cos \theta _\mu \) and \(E_\mu \) for the “2D” analysis, where we do not use any information on hadrons. But, in right panels of these figures, we portray the sensitivity in the plane of reconstructed \(\cos \theta _\mu \) and \(E_\mu \) for the “3D” case, where the events are further divided into four sub-bins depending on the reconstructed hadron energy for LE (see Table 2) and HE binning schemes (see Table 3).

The common features which are emerging from all the panels in Figs. 4 and 5 are that most of the sensitivity towards the NSI parameter \(\varepsilon _{\mu \tau }\) stems from higher energies and longer baselines where the matter effect term 2\(\sqrt{2} G_F N_e E\) becomes sizeable. We observe similar trends in Fig 2 where we plot the differences in \(\nu _\mu \rightarrow \nu _\mu \) oscillation probabilities for the cases \(\varepsilon _{\mu \tau } = 0\) and \(\varepsilon _{\mu \tau } = \pm 0.05\). The event spectra as shown in Fig. 3 also confirm this fact. Figs. 4 and 5 clearly demonstrate while going from LE to HE binning scheme that the sensitivity towards the NSI parameter \(\varepsilon _{\mu \tau }\) get enhanced due to the increment in the range of \(E_\mu \) from 11 to 21 GeV and for extending the fourth \(E'_{\mathrm{had}}\) bin from 15 to 25 GeV. We can also observe from these figures that with the addition of hadron energy information, the area in the \(E_\mu \)-\(\cos \theta _\mu \) plane which contributes significantly to \(\Delta \chi ^2_{\pm }\) increases, consequently enhancing the net \(\Delta \chi ^2_{\pm }\) for both LE and HE binning schemes. Here, we would like to mention that the increase in \(\chi ^2_{\pm }\) is not just due to the information contained in \(E'_{\mathrm{had}}\), but also due to the valuable information coming from the correlation between \(E'_{\mathrm{had}}\) and muon momentum (\(E_\mu \), \(\cos \theta _\mu \)).

In Fig. 6, we show the sensitivity of the ICAL detector to constrain \(\varepsilon _{\mu \tau }\) using an exposure of 500 kt\(\cdot \)yr and assuming NH as the true mass hierarchy. We obtain these results after performing marginalization over \(\theta _{23}\), \(\Delta m^{2}_{\mathrm{eff}}\), and both the choices of mass hierarchy as discussed in Sect. 5.2. In the left (right) panel, the results are shown for the LE (HE) binning scheme. In each panel, the red solid line shows the sensitivity for the “3D” case where we consider \(E_{\mu }\), \(\cos \theta _{\mu }\), and \(E'_{\mathrm{had}}\) as observables. The black dashed line in each panel portrays the sensitivity for the “2D” case considering \(E_{\mu }\) and \(\cos \theta _{\mu }\) as observables. We see considerable improvement in the sensitivity for both the LE and HE binning schemes when we add \(E'_{\mathrm{had}}\) along with \(E_\mu \) and \(\cos \theta _\mu \) as observables. We see significant gain in the sensitivity when we increase the \(E_\mu \) range from 11 to 21 GeV and extend the fourth \(E'_{\mathrm{had}}\) bin from 15 to 25 GeV. It is evident from both the panels in Fig. 6 that for the [HE, 3D] case, we obtain the best sensitivity towards the NSI parameter \(\varepsilon _{\mu \tau }\), whereas the [LE, 2D] mode gives the most conservative limits.

The 3\(\sigma \) (90\(\%\)) confidence level bounds on the flavor violating NSI parameter \(\varepsilon _{\mu \tau }\) obtained using 500 kt\(\cdot \)yr exposure of the ICAL are listed in Table 4. The results are shown for true NH (3rd column) and true IH (4th column). For the [HE, 3D] case, we expect the best limit of \(-\,0.01<\varepsilon _{\mu \tau }< 0.01\) at 90\(\%\) C.L. using 500 kt\(\cdot \)yr exposure of the ICAL detector and irrespective of the choices of true mass hierarchy. For the [LE, 2D] mode, we obtain the most conservative limit of \(-\,0.03< \varepsilon _{\mu \tau } < 0.034\) at 90\(\%\) confidence level assuming NH as true choice. So far we have considered \(\sin ^2 2\theta _{13}= 0.1\) as our benchmark choice both in data and theory. If we consider the current best fit value of \(\sin ^2 2\theta _{13} = 0.085\) [2,3,4], we have checked that our results will remain almost unaltered. For an instance, if we consider \(\varepsilon _{\mu \tau } = 0.02\) in the fit, then we obtain \(\Delta \chi ^2 = 9.49\) assuming \(\sin ^2 2\theta _{13} = 0.1\) for the [HE, 3D] mode (see the red curve in the right panel of Fig. 6). Under the same condition, if we take \(\sin ^2 2\theta _{13} = 0.085\), then the \(\Delta \chi ^2\) changes to 9.38.

### Constraints on \(\varepsilon _{\mu \tau }\) for non-maximal \(\theta _{23}\)

Global fits of world neutrino data point towards non-maximal \(\theta _{23}\) giving rise to two degenerate solutions: one lies in the lower octant (LO) where \(\sin ^2\theta _{23} < 0.5\), and the other belongs to the higher octant (HO) where \(\sin ^2\theta _{23} > 0.5\) [2,3,4]. Therefore, it would be quite interesting to see how the expected constraints on the NSI parameter \(\varepsilon _{\mu \tau }\) at ICAL would be affected if \(\theta _{23}\) turns out to be non-maximal in Nature. To study this, we consider the [HE, 3D] binning scheme for which we obtain the best limit on \(\varepsilon _{\mu \tau }\) in the previous section assuming \(\sin ^2\theta _{23}\) (true) = 0.5. In Fig. 7, we exhibit the performance of the ICAL detector to constrain \(\varepsilon _{\mu \tau }\) by simulating the prospective data with \(\sin ^2\theta _{23}\) (true) = 0.4 as a benchmark choice in the LO (see the blue dashed line). We also depict the expected constraints on \(\varepsilon _{\mu \tau }\) at ICAL assuming \(\sin ^2\theta _{23}\) (true) = 0.6 as a benchmark choice in the HO (see the brown dotted line). We compare these results with \(\sin ^2\theta _{23}\) (true) = 0.5 case (see the red solid line). To generate all these results, we consider an exposure of 500 kt\(\cdot \)yr, assume NH as the true mass hierarchy, and perform marginalization over \(\theta _{23}\), \(\Delta m^{2}_{\mathrm{eff}}\), and both the choices of mass hierarchy as discussed in Sect. 5.2. We observe from Fig. 7 that the expected limits on \(\varepsilon _{\mu \tau }\) for non-maximal \(\theta _{23}\) values deteriorate slightly as compared to the maximal mixing choice. Also, we notice that the bounds are similar for the octant-symmetric true values of \(\sin ^2\theta _{23}\) in the LO (0.4) and HO (0.6) since the survival probabilities of muon neutrino and antineutrino in the presence of non-zero \(\varepsilon _{\mu \tau }\) largely depend on \(\sin 2\theta _{23}\) instead of \(\sin \theta _{23}\) (see Eqs. 11, 12, and 13 ).

### Advantage of having charge identification capability

The ICAL detector is expected to have a uniform magnetic field of strength around 1.5 T over the entire detector. It will enable the ICAL detector to identify the \(\mu ^-\) and \(\mu ^+\) events separately by observing the bending of their tracks in the opposite directions in the presence of the magnetic field. We label this feature of ICAL as the charge identification capability. In Ref. [75], it has been demonstrated that the ICAL detector will have a very good CID efficiency over a wide range of reconstructed \(E_\mu \) and \(\cos \theta _\mu \). In this work, we estimate for the first time the gain in the sensitivity that ICAL may have in constraining the NSI parameter \(\varepsilon _{\mu \tau }\) due to its CID capability. In each panel of Fig. 8, we show the expected sensitivity of ICAL in constraining \(\varepsilon _{\mu \tau }\) with (red solid line) and without (black dashed line) CID capability using 500 kt\(\cdot \)yr exposure and assuming NH. While preparing these plots, we keep the oscillation parameters fixed in the fit and depict the result for the 2D: \(E_{\mu }\), \(\cos \theta _{\mu }\) (3D: \(E_{\mu }\), \(\cos \theta _{\mu }\), \(E'_{\mathrm{had}}\)) mode in the left (right) panel assuming the HE binning scheme. It is apparent from Fig. 8 that the CID capability of ICAL in distinguishing \(\mu ^-\) and \(\mu ^+\) events plays an important role to make it sensitive to the NSI parameter \(\varepsilon _{\mu \tau }\) like the mass hierarchy measurements [70, 78]. In the following, we quote the 90\(\%\) confidence level limits on \(\varepsilon _{\mu \tau }\) that the ICAL detector can place with and without CID capabilities for [HE, 2D] and [HE, 3D] modes.

\(\bullet \) [HE, 2D] mode (left panel of Fig. 8):

\(\bullet \) [HE, 3D] mode (right panel of Fig. 8):

The limits on \(\varepsilon _{\mu \tau }\) mentioned in Eqs. 30 and 31 clearly demonstrate the improvement that the ICAL detector can have in constraining the NSI parameter \(\varepsilon _{\mu \tau }\) due its CID capability.

### Limits on \(\varepsilon _{\mu \tau }\) for various exposures

Figure 9 shows the 3\(\sigma \) limit on \(\varepsilon _{\mu \tau }\) as a function of run-time^{Footnote 7} for 50 kt ICAL. The left (right) panel is for LE (HE) binning scheme. In each panel, the black and red lines depict the results for 2D (\(E_\mu \), \(\cos \theta _\mu \)) and 3D (\(E_{\mu }\), \(\cos \theta _{\mu }\), \(E'_{\mathrm{had}}\)) modes respectively. In Fig. 9, various sensitivity curves are drawn keeping all the oscillation parameters fixed in the fit and assuming NH. Here, we give the results only for positive values of \(\varepsilon _{\mu \tau }\). We have checked that the results look similar if we consider negative values of \(\varepsilon _{\mu \tau }\) as well. If we take total 250 kt\(\cdot \)yr exposure (50 kt ICAL with a run-time of 5 years), the expected bound is \(|\varepsilon _{\mu \tau }|\lesssim 0.28\) at \(3\sigma \) C.L. assuming NH in [HE, 3D] binning scheme. It suggests that ICAL can place competitive constraints on \(\varepsilon _{\mu \tau }\) even for less exposure.

### Impact of non-zero \(\varepsilon _{\mu \tau }\) on mass hierarchy determination

This section is devoted to study how the flavor violating NSI parameter \(\varepsilon _{\mu \tau }\) affects the mass hierarchy measurement which is the prime goal of the ICAL detector. We quantify the performance ICAL to rule out the wrong hierarchy by adopting the following \(\chi ^2\) expression:

Here, we obtain \(\chi ^2_{\mathrm{ICAL}} (\mathrm{true\,\,MH})\) and \(\chi ^2_{\mathrm{ICAL}} (\mathrm{false\,\,MH})\) by performing the fit to the prospective data assuming true and false mass hierarchy respectively. Since the statistical fluctuations are suppressed in our analysis, \(\chi ^2_{\mathrm{ICAL}} (\mathrm{true\,MH}) \approx 0\). First, we estimate the sensitivity of the ICAL detector to determine the neutrino mass hierarchy by adopting the procedure as outlined in Ref. [78] for the standard case, which we denote as “\(\Delta \chi ^2_{\mathrm{{ICAL-MH}}}\) (SM)” in the third column of Table 5. Next, to estimate the mass hierarchy sensitivity in the presence of non-zero \(\varepsilon _{\mu \tau }\), we adopt the following strategy. We generate the data with a given mass hierarchy assuming \(\varepsilon _{\mu \tau } = 0\). Then, while fitting the prospective data with the opposite hierarchy, we introduce \(\varepsilon _{\mu \tau }\) in the fit and marginalize over it in the range of - 0.1 to 0.1 along with the oscillation parameters \(\theta _{23}\) and \(\Delta m^2_{\mathrm{eff }}\) in their allowed ranges as mentioned in Sect. 5. We label this result as “\(\Delta \chi ^2_{\mathrm{{ICAL-MH}}}\) (SM + \(\varepsilon _{\mu \tau }\))” in the fourth column of Table 5. We show our results for various choices of binning schemes and observables assuming both true NH and true IH. We consider 500 kt\(\cdot \)yr exposure of the ICAL detector. We can see from Table 5 that depending on the choice of true mass hierarchy and the analysis mode, the mass hierarchy sensitivity of ICAL gets reduced by 10–20\(\%\) due to the presence of non-zero \(\varepsilon _{\mu \tau }\) in the fit.

### Precision measurement of atmospheric parameters with non-zero \(\varepsilon _{\mu \tau }\)

Next, we turn our attention to the precise measurement of atmospheric oscillation parameters \(\sin ^2\theta _{23}\) and \(|\Delta m^2_{32}|\) using 500 kt\(\cdot \)yr exposure of the ICAL detector. We quantify this performance indicator using the following expression:

where \(\chi ^2_0\) is the minimum value of \(\chi ^2_{\mathrm{ICAL}}\) in the allowed parameter range. Since we suppress the statistical fluctuations, we have \(\chi ^2_0 \approx 0\). First, considering \(\sin ^2\theta _{23}\) (true) = 0.5 and \(|\Delta m^2_{32}|\) (true) = 2.4 \(\times \) 10\(^{-3}\) \(\hbox {eV}^2\), we estimate the allowed regions in \(\sin ^2\theta _{23}\) - \(|\Delta m^2_{32}|\) (test) plane in the absence of \(\varepsilon _{\mu \tau }\) at 90\(\%\) C.L. (2 d.o.f.). We show these results for the “SM” case using solid lines in Fig. 10 for various analysis modes. For the [HE, 3D] case, we achieve the best precision for the atmospheric parameters, and for the [LE, 2D] case, we have the most conservative results.

Next, we study the impact of non-zero \(\varepsilon _{\mu \tau }\) in the precision measurement of atmospheric parameters in the following fashion. We again generate the prospective data considering the true values of \(\sin ^2\theta _{23}\) and \(|\Delta m^2_{32}|\) as mentioned above. Then, while estimating the allowed regions in \(\sin ^2\theta _{23}\) - \(|\Delta m^2_{32}|\) (test) plane, we introduce \(\varepsilon _{\mu \tau }\) in the fit and marginalize over it in the range of [- 0.1, 0.1]. We present these results for the “SM + \(\varepsilon _{\mu \tau }\)” case at \(90\%\) C.L. (2 d.o.f.) with the help of dashed lines in Fig. 10 for various analysis modes. We do not see any appreciable change in the contours when we introduce \(\varepsilon _{\mu \tau }\) in the fit and vary in its \(\pm 10\%\) range. It suggests that the precision measurement of atmospheric oscillation parameters at the ICAL detector is quite robust even if we marginalize over \(\varepsilon _{\mu \tau }\) in the fit. Similar results were obtained by the Super-Kamiokande Collaboration in Ref. [63], where they studied the impact of NSI’s in \(\nu _\mu \)-\(\nu _\tau \) sector using their Phase I and Phase II atmospheric data.

## Summary and concluding remarks

In this paper, we explore the possibility of lepton flavor violating neutral current non-standard interactions (NSI’s) of atmospheric neutrino and antineutrino while they travel long distances inside the Earth matter before reaching to the ICAL detector. During the propagation of these neutrinos, we allow an extra interaction vertex with \(\nu _\mu \) as the incoming particle and \(\nu _\tau \) as the outgoing one and vice versa. With such an interaction vertex, the neutral current non-standard interaction of neutrino with matter fermions gives rise to a new matter potential whose relative strength as compared to the standard matter potential (\(V_{CC}\)) is denoted by \(\varepsilon _{\mu \tau }\).

We exhibit for the first time that by adding the hadron energy information along with the reconstructed muon energy and muon direction in each event, the sensitivity of ICAL to the NSI parameter \(\varepsilon _{\mu \tau }\) can be enhanced significantly. We find that considering reconstructed \(E_\mu \) (in the range of 1–11 GeV) and \(\cos \theta _\mu \) (in the range of - 1 to 1) as observables, the expected limit on \(\varepsilon _{\mu \tau }\) at 90\(\%\) C.L. is \(-\,0.03< \varepsilon _{\mu \tau } <0.03\). If we increase the muon energy range further from 11 to 21 GeV (\(E_\mu \in [1, 21]\) GeV) and consider the reconstructed hadron energy (\(E'_{\mathrm{had}}\)) as an extra observable on top of the four momenta of muon (\(E_\mu ,\,\cos \theta _\mu \)), we find a significant improvement in the limit which is \(-\,0.01< \varepsilon _{\mu \tau } < 0.01 \) at 90\(\%\) C.L. using 500 kt\(\cdot \)yr exposure of the ICAL detector. We also demonstrate for the first time that the charge identification capability of the ICAL detector plays an important role to obtain these tight constraints on \(\varepsilon _{\mu \tau }\) as mentioned above.

Assuming 1–21 GeV reconstructed muon energy range and considering \(E_\mu ,\,\cos \theta _\mu ,\,\)and \(E'_{\mathrm{had}}\) as observables, we find that the mass hierarchy sensitivity at the ICAL detector deteriorates by \(\sim 10\%\) if we introduce the NSI parameter \(\varepsilon _{\mu \tau }\) in the fit and marginalize over it in the range of \(-\,\)0.1 to 0.1 along with other standard oscillation parameters. On the other hand, the precision measurement of atmospheric oscillation parameters at the ICAL detector is quite robust even if we marginalize over the NSI parameter \(\varepsilon _{\mu \tau }\) in fit in the range \(-\,\)0.1 to 0.1.

Before we conclude, we would like to emphasize that though the expected limit on \(\varepsilon _{\mu \tau }\) at ICAL is weaker than IceCube and DeepCore bounds, but, these results are complementary to each other. IceCube extracts information on the NSI parameters using neutrinos whose energies are above 300 GeV. At these high energies (which are relevant for IceCube), oscillation probabilities are completely dominated by NSIs and there is hardly any impact of vacuum oscillations. As far as DeepCore is concerned, they use neutrinos in their analysis whose energies are above 10 GeV or so, and at these energies, NSIs start to dominate the flavor transition. The proposed ICAL detector is very efficient in the neutrino energy range of 1–10 GeV and it is going to provide a complementary information on the NSI parameter \(\varepsilon _{\mu \tau }\) in this energy range which is not accessible by IceCube and DeepCore. Moreover, at these energies (where ICAL is very effective), both vacuum oscillations and matter driven NSIs affect the flavor transitions. Therefore, it is quite important to probe these NSI parameters at different energies using different detectors, since they provide complementary information on these parameters.

Another important feature of ICAL is that due to the presence of magnetic field, it provides an opportunity to probe the NSI parameters in neutrino (by observing \(\mu ^{-}\) events) and antineutrino (by observing \(\mu ^{+}\) events) modes separately. Therefore, ICAL provides an unique platform to test some fundamental symmetries of Nature such as Charge-Parity-Time (CPT) symmetry, whose tiny violation may give rise to different mass, mixing, and NSI parameters for neutrinos and antineutrinos. This study may not be possible using any other existing or planned water-, ice-, scintillator-, or argon-based detectors.

## Data Availability Statement

This manuscript has no associated data or the data will not be deposited. [Authors’ comment: This is a sensitivity study of a future experiment and we have generated mock data to study the performance of the experiment. We have not used any external dataset in our study.]

## Notes

The effective mass splitting is related to \(\Delta m^{2}_{31}\) as follows [67, 68]

$$\begin{aligned} \Delta m_{\text {eff}}^{2} = \Delta m^{2}_{31}\,-\Delta m^{2}_{21}\left( \cos ^{2}\theta _{12}\, -\,\cos \delta _{\mathrm{CP}}\, \sin \theta _{13}\,\sin 2\theta _{12}\,\tan \theta _{23}\right) . \nonumber \\ \end{aligned}$$(10)According to a simplified version of the PREM profile [66], the inner core has a radius of \(\sim \) 1220 km with an average density of 13 g/\(\hbox {cm}^3\). For outer core, \(R_{\mathrm{min}} \simeq 1220\) km and \(R_{\mathrm{max}} \simeq \) 3480 km with an average density of 11.3 g/\(\hbox {cm}^3\). Note that in our analysis, we consider the detailed version of the PREM.

The standard neutrino matter potential due to the

*W*-mediated interactions with the ambient electrons can be written as a function of matter density \(\rho \) as follows:where \(Y_e\) \(\left( \frac{N_e}{N_p + N_n}\right) \) is the relative number density. For electrically, neutral and isoscalar medium, \(N_e = N_p\) \(= N_n\), and therefore, \(Y_e =0.5\).

Preliminary calculation of the expected fluxes at the INO site has been performed in Ref. [72, 73]. The visible differences between the neutrino fluxes at the Kamioka and INO sites appear at lower energies. The main reason behind this is that the horizontal components of the geo-magnetic field are different at the Kamioka (30 \(\mu \)T) and INO (40 \(\mu \)T) locations. We plan to use these new fluxes estimated for the INO site (see Ref. [73]) in future studies.

For a detailed description of the two binning schemes that we consider in our analysis, see Sect. 5.1.

In Fig. 4, we do not consider the constant contributions in \(\chi ^2\) coming from the term which involves five pull parameters \(\zeta ^2_l\) in Eq. 24 and Eq. 26. Also, we do not marginalize over the oscillation parameters in the fit to produce these figures. We adopt the same strategy for Fig. 5 as well. Note that we show our final results considering full pull contributions and marginalizing over the oscillation parameters in the fit as mentioned in previous section.

Note that while varying run-time in Fig. 9, we always consider the same LE and HE binning schemes as given in Tables 2 and 3 respectively. For less exposure (small run-time), we may not have sufficient statistics in most of the bins. One needs to consider larger bin widths to tackle this issue which in turn may effect the sensitivity results. We have plans to address this issue in our future study.

## References

**Particle Data Group**Collaboration, C. Patrignani et al., Review of particle physics. Chin. Phys. C**40**, 10,100001 (2016)F. Capozzi, E. Di Valentino, E. Lisi, A. Marrone, A. Melchiorri, A. Palazzo, Global constraints on absolute neutrino masses and their ordering. Phys. Rev. D

**95**(9), 096014 (2017). arXiv:1703.04471I. Esteban, M.C. Gonzalez-Garcia, A. Hernandez-Cabezudo, M. Maltoni, T. Schwetz, Global analysis of three-flavour neutrino oscillations: synergies and tensions in the determination of \(\theta _23, \delta _CP\), and the mass ordering. arXiv:1811.05487 (2018)

P.F. de Salas, D.V. Forero, C.A. Ternes, M. Tortola, J.W.F. Valle, Status of neutrino oscillations 2018: 3 hint for normal mass ordering and improved CP sensitivity. Phys. Lett. B

**782**, 633–640 (2018). arXiv:1708.01186M.C. Gonzalez-Garcia, M.M. Guzzo, P.I. Krastev, H. Nunokawa, O.L.G. Peres, V. Pleitez, J.W.F. Valle, R.Z. Funchal, Atmospheric neutrino observations and flavor changing interactions. Phys. Rev. Lett.

**82**, 3202–3205 (1999). arXiv:hep-ph/9809531S. Bergmann, A. Kagan, Z—induced FCNCs and their effects on neutrino oscillations. Nucl. Phys. B

**538**, 368–386 (1999). arXiv:hep-ph/9803305S. Bergmann, Y. Grossman, E. Nardi, Neutrino propagation in matter with general interactions. Phys. Rev. D

**60**, 093008 (1999). arXiv:hep-ph/9903517P. Lipari, M. Lusignoli, On exotic solutions of the atmospheric neutrino problem. Phys. Rev. D

**60**, 013003 (1999). arXiv:hep-ph/9901350M.M. Guzzo, H. Nunokawa, P.C. de Holanda, O.L.G. Peres, On the massless ’just-so’ solution to the solar neutrino problem. Phys. Rev. D

**64**, 097301 (2001). arXiv:hep-ph/0012089M.C. Gonzalez-Garcia, Y. Grossman, A. Gusso, Y. Nir, New CP violation in neutrino oscillations. Phys. Rev. D

**64**, 096006 (2001). arXiv:hep-ph/0105159A.M. Gago, M.M. Guzzo, H. Nunokawa, W.J.C. Teves, R.Z. Funchal, Probing flavor changing neutrino interactions using neutrino beams from a muon storage ring. Phys. Rev. D

**64**, 073003 (2001). arXiv:hep-ph/0105196Z. Berezhiani, A. Rossi, Limits on the nonstandard interactions of neutrinos from e+ e- colliders. Phys. Lett. B

**535**, 207–218 (2002). arXiv:hep-ph/0111137P. Huber, J.W.F. Valle, Nonstandard interactions: atmospheric versus neutrino factory experiments. Phys. Lett. B

**523**, 151–160 (2001). arXiv:hep-ph/0108193N. Fornengo, M. Maltoni, R. Tomas, J.W.F. Valle, Probing neutrino nonstandard interactions with atmospheric neutrino data. Phys. Rev. D

**65**, 013010 (2002). arXiv:hep-ph/0108043P. Huber, T. Schwetz, J.W.F. Valle, Confusing nonstandard neutrino interactions with oscillations at a neutrino factory. Phys. Rev. D

**66**, 013006 (2002). arXiv:hep-ph/0202048A.M. Gago, M.M. Guzzo, P.C. de Holanda, H. Nunokawa, O.L.G. Peres, V. Pleitez, R.Z. Funchal, Global analysis of the postSNO solar neutrino data for standard and nonstandard oscillation mechanisms. Phys. Rev.

**D65**, 073012 (2002). arXiv:hep-ph/0112060Z. Berezhiani, R.S. Raghavan, A. Rossi, Probing nonstandard couplings of neutrinos at the Borexino detector. Nucl. Phys. B

**638**, 62–80 (2002). arXiv:hep-ph/0111138T. Ota, J. Sato, N.-A. Yamashita, Oscillation enhanced search for new interaction with neutrinos. Phys. Rev. D

**65**, 093015 (2002). arXiv:hep-ph/0112329S. Davidson, C. Pena-Garay, N. Rius, A. Santamaria, Present and future bounds on nonstandard neutrino interactions. JHEP

**03**, 011 (2003). arXiv:hep-ph/0302093A. Friedland, C. Lunardini, M. Maltoni, Atmospheric neutrinos as probes of neutrino-matter interactions. Phys. Rev. D

**70**, 111301 (2004). arXiv:hep-ph/0408264M.C. Gonzalez-Garcia, M. Maltoni, Atmospheric neutrino oscillations and new physics. Phys. Rev. D

**70**, 033010 (2004). arXiv:hep-ph/0404085M.M. Guzzo, P.C. de Holanda, O.L.G. Peres, Effects of nonstandard neutrino interactions on MSW-LMA solution to the solar neutrino problems. Phys. Lett. B

**591**, 1–6 (2004). arXiv:hep-ph/0403134A. Friedland, C. Lunardini, A Test of tau neutrino interactions with atmospheric neutrinos and K2K. Phys. Rev. D

**72**, 053009 (2005). arXiv:hep-ph/0506143N. Kitazawa, H. Sugiyama, O. Yasuda, Will MINOS see new physics? arXiv:hep-ph/0606013 (2018)

G. Mangano, G. Miele, S. Pastor, T. Pinto, O. Pisanti, P.D. Serpico, Effects of non-standard neutrino-electron interactions on relic neutrino decoupling. Nucl. Phys. B

**756**, 100–116 (2006). arXiv:hep-ph/0607267M. Blennow, T. Ohlsson, W. Winter, Non-standard Hamiltonian effects on neutrino oscillations. Eur. Phys. J. C

**49**, 1023–1039 (2007). arXiv:hep-ph/0508175A. Esteban-Pretel, R. Tomas, J.W.F. Valle, Probing non-standard neutrino interactions with supernova neutrinos. Phys. Rev. D

**76**, 053001 (2007). arXiv:0704.0032J. Barranco, O.G. Miranda, T.I. Rashba, Low energy neutrino experiments sensitivity to physics beyond the Standard Model. Phys. Rev. D

**76**, 073008 (2007). arXiv:hep-ph/0702175N.C. Ribeiro, H. Minakata, H. Nunokawa, S. Uchinami, R. Zukanovich-Funchal, Probing non-standard neutrino interactions with neutrino factories. JHEP

**12**, 002 (2007). arXiv:0709.1980J. Kopp, M. Lindner, T. Ota, Discovery reach for non-standard interactions in a neutrino factory. Phys. Rev. D

**76**, 013001 (2007). arXiv:hep-ph/0702269M. Blennow, D. Meloni, T. Ohlsson, F. Terranova, M. Westerberg, Non-standard interactions using the OPERA experiment. Eur. Phys. J. C

**56**, 529–536 (2008). arXiv:0804.2744A. Esteban-Pretel, J.W.F. Valle, P. Huber, Can OPERA help in constraining neutrino non-standard interactions? Phys. Lett. B

**668**, 197–201 (2008). arXiv:0803.1790J. Barranco, O.G. Miranda, C.A. Moura, J.W.F. Valle, Constraining non-standard neutrino-electron interactions. Phys. Rev. D

**77**, 093014 (2008). arXiv:0711.0698J. Kopp, M. Lindner, T. Ota, J. Sato, Non-standard neutrino interactions in reactor and superbeam experiments. Phys. Rev. D

**77**, 013007 (2008). arXiv:0708.0152M. Blennow, T. Ohlsson, J. Skrotzki, Effects of non-standard interactions in the MINOS experiment. Phys. Lett. B

**660**, 522–528 (2008). arXiv:hep-ph/0702059T. Ohlsson, H. Zhang, Non-standard interaction effects at reactor neutrino experiments. Phys. Lett. B

**671**, 99–104 (2009). arXiv:0809.4835M.B. Gavela, D. Hernandez, T. Ota, W. Winter, Large gauge invariant non-standard neutrino interactions. Phys. Rev. D

**79**, 013007 (2009). arXiv:0809.3451A. Bolanos, O.G. Miranda, A. Palazzo, M.A. Tortola, J.W.F. Valle, Probing non-standard neutrino-electron interactions with solar and reactor neutrinos. Phys. Rev. D

**79**, 113012 (2009). arXiv:0812.4417S. Antusch, J.P. Baumann, E. Fernandez-Martinez, Non-standard neutrino interactions with matter from physics beyond the standard model. Nucl. Phys. B

**810**, 369–388 (2009). arXiv:0807.1003C. Biggio, M. Blennow, E. Fernandez-Martinez, General bounds on non-standard neutrino interactions. JHEP

**08**, 090 (2009). arXiv:0907.0097F.J. Escrihuela, O.G. Miranda, M.A. Tortola, J.W.F. Valle, Constraining nonstandard neutrino-quark interactions with solar, reactor and accelerator data. Phys. Rev. D

**80**, 105009 (2009). arXiv:0907.2630. [Erratum: Phys. Rev. D80,129908(2009)]A.M. Gago, H. Minakata, H. Nunokawa, S. Uchinami, R. Zukanovich Funchal, Resolving CP violation by standard and nonstandard interactions and parameter degeneracy in neutrino oscillations. JHEP

**01**, 049 (2010). arXiv:0904.3360C. Biggio, M. Blennow, E. Fernandez-Martinez, Loop bounds on non-standard neutrino interactions. JHEP

**03**, 139 (2009). arXiv:0902.0607H. Oki, O. Yasuda, Sensitivity of the T2KK experiment to the non-standard interaction in propagation. Phys. Rev. D

**82**, 073009 (2010). arXiv:1003.5554J. Kopp, P.A.N. Machado, S.J. Parke, Interpretation of MINOS data in terms of non-standard neutrino interactions. Phys. Rev. D

**82**, 113002 (2010). arXiv:1009.0014D.V. Forero, M.M. Guzzo, Constraining nonstandard neutrino interactions with electrons. Phys. Rev. D

**84**, 013002 (2011)F.J. Escrihuela, M. Tortola, J.W.F. Valle, O.G. Miranda, Global constraints on muon-neutrino non-standard interactions. Phys. Rev. D

**83**, 093002 (2011). arXiv:1103.1366M. Gonzalez-Garcia, M. Maltoni, J. Salvado, Testing matter effects in propagation of atmospheric and long-baseline neutrinos. JHEP

**1105**, 075 (2011). arXiv:1103.4365P. Coloma, A. Donini, J. Lopez-Pavon, H. Minakata, Non-standard interactions at a neutrino factory: correlations and CP violation. JHEP

**08**, 036 (2011). arXiv:1105.5936R. Adhikari, S. Chakraborty, A. Dasgupta, S. Roy, Non-standard interaction in neutrino oscillations and recent Daya Bay, T2K experiments. Phys. Rev. D

**86**, 073010 (2012). arXiv:1201.3047S.K. Agarwalla, F. Lombardi, T. Takeuchi, Constraining non-standard interactions of the neutrino with borexino. JHEP

**12**, 079 (2012). arXiv:1207.3492M.C. Gonzalez-Garcia, M. Maltoni, Determination of matter potential from global analysis of neutrino oscillation data. JHEP

**09**, 152 (2013). arXiv:1307.3092T. Ohlsson, H. Zhang, S. Zhou, Effects of nonstandard neutrino interactions at PINGU. Phys. Rev. D

**88**(1), 013001 (2013). arXiv:1303.6130T. Ohlsson, Status of non-standard neutrino interactions. Rept. Prog. Phys.

**76**, 044201 (2013). arXiv:1209.2710S. Choubey, A. Ghosh, T. Ohlsson, D. Tiwari, Neutrino physics with non-standard interactions at INO. JHEP

**12**, 126 (2015). arXiv:1507.02211Y. Farzan, M. Tortola, Neutrino oscillations and non-standard interactions. Front. Phys.

**6**, 10 (2018). arXiv:1710.09360D. Meloni, Exploring new physics from \(\nu _\tau \) events in OPERA. Phys. Lett. B

**792**, 199–204 (2019). arXiv:1903.08933A. De Gouva, K.J. Kelly, G.V. Stenico, P. Pasquini, Physics with beam Tau-Neutrino appearance at DUNE. Phys. Rev. D

**100**(1), 016004 (2019). arXiv:1904.07265Borexino Collaboration, S.K. Agarwalla et al.,

*Constraints on non-standard neutrino interactions from borexino phase-II*. arXiv:1905.03512 (2019)A. Ghoshal, A. Giarnetti, D. Meloni, On the Role of the \(\nu _\tau \) appearance in DUNE in constraining standard neutrino physics and beyond. arXiv:1906.06212 (2019)

O.G. Miranda, H. Nunokawa, Non standard neutrino interactions: current status and future prospects. New J. Phys.

**17**(9), 095002 (2015). arXiv:1505.06254L. Wolfenstein, Neutrino oscillations in matter. Phys. Rev. D

**17**, 2369–2374 (1978)Super-Kamiokande Collaboration, G. Mitsuka et al., Study of non-standard neutrino interactions with atmospheric neutrino data in super-kamiokande I and II. Phys. Rev. D

**84**, 113008, (2011). arXiv:1109.1889J. Salvado, O. Mena, S. Palomares-Ruiz, N. Rius, Non-standard interactions with high-energy atmospheric neutrinos at IceCube. JHEP

**01**, 141 (2017). arXiv:1609.03450IceCube Collaboration, M. Aartsen et al., Search for nonstandard neutrino interactions with IceCube DeepCore. Phys. Rev. D

**97**(7), 072009 (2018). arXiv:1709.07079A. Dziewonski, D. Anderson, Preliminary reference earth model. Phys. Earth Planet. Interiors

**25**, 297–356 (1981)H. Nunokawa, S.J. Parke, R.Z. Funchal, Another possible way to determine the neutrino mass hierarchy. Phys. Rev. D

**72**, 013009 (2005). arXiv:hep-ph/0503283A. de Gouvea, J. Jenkins, B. Kayser, Neutrino mass hierarchy, vacuum oscillations, and vanishing |U(e3)|. Phys. Rev. D

**71**, 113009 (2005). arXiv:hep-ph/0503079D. Hollander, I. Mocioiu, Sterile neutrinos at LBNE. arXiv:1408.1749 (2019)

ICAL Collaboration, S. Ahmed et al., Physics potential of the ICAL detector at the India-based Neutrino Observatory (INO). Pramana

**88**(5), 79 (2017). arXiv:1505.07380D. Casper, The Nuance neutrino physics simulation, and the future. Nucl. Phys. Proc. Suppl.

**112**, 161–170 (2002). arXiv:hep-ph/0208030M. Sajjad Athar, M. Honda, T. Kajita, K. Kasahara, S. Midorikawa, Atmospheric neutrino flux at INO, South Pole and Pyhásalmi. Phys. Lett. B

**718**, 1375–1380 (2013). arXiv:1210.5154M. Honda, M.S. Athar, T. Kajita, K. Kasahara, S. Midorikawa, Atmospheric neutrino flux calculation using the NRLMSISE-00 atmospheric model. Phys. Rev. D

**92**(2), 023004 (2015). arXiv:1502.03916M. Honda, T. Kajita, K. Kasahara, S. Midorikawa, Improvement of low energy atmospheric neutrino flux calculation using the JAM nuclear interaction model. Phys. Rev. D

**83**, 123001 (2011). arXiv:1102.2688A. Chatterjee, K. Meghna, K. Rawat, T. Thakore, V. Bhatnagar et al., A simulations study of the muon response of the iron calorimeter detector at the india-based neutrino observatory. JINST

**9**, P07001 (2014). arXiv:1405.7243M.M. Devi, A. Ghosh, D. Kaur, L.S. Mohan, S. Choubey et al., Hadron energy response of the iron calorimeter detector at the India-based neutrino observatory. JINST

**8**, P11003 (2013). arXiv:1304.5115M. Blennow, P. Coloma, P. Huber, T. Schwetz, Quantifying the sensitivity of oscillation experiments to the neutrino mass ordering. JHEP

**1403**, 028 (2014). arXiv:1311.1822M.M. Devi, T. Thakore, S.K. Agarwalla, A. Dighe, Enhancing sensitivity to neutrino parameters at INO combining muon and hadron information. JHEP

**1410**, 189 (2014). arXiv:1406.3689A. Ghosh, T. Thakore, S. Choubey, Determining the neutrino mass hierarchy with INO, T2K. NOvA and reactor experiments. JHEP

**1304**, 009 (2013). arXiv:1212.1305P. Huber, M. Lindner, W. Winter, Superbeams versus neutrino factories. Nucl. Phys. B

**645**, 3–48 (2002). arXiv:hep-ph/0204352G. Fogli, E. Lisi, A. Marrone, D. Montanino, A. Palazzo, Getting the most from the statistical analysis of solar neutrino oscillations. Phys. Rev. D

**66**, 053010 (2002). arXiv:hep-ph/0206162

## Acknowledgements

The INO-ICAL Collaboration is exploring various physics potentials of the proposed ICAL detector and this work is a part of that ongoing effort. This work would not have been possible without the contribution of several members of the Collaboration. We would like to thank A. Dighe, A.M. Srivastava, P. Agrawal, S. Goswami, D. Indumathi, S. Choubey, S. Uma Sankar for their useful comments on our work. We thank N. Mondal, A. Dighe, and S. Goswami for helping us during the INO Internal Review Process. A.K. acknowledges the support from the Department of Atomic Energy (DAE), Government of India. S.K.A. acknowledges the support from DST/INSPIRE Research Grant [IFA-PH-12], Department of Science and Technology, India and the Young Scientist Project [INSA/SP/YSP/144/2017/1578] from the Indian National Science Academy. T. T. acknowledges support from the Ministerio de Economíay Competitividad (MINECO): Plan Estatal de Investigación (ref. FPA2015- 65150-C3-1-P, MINECO/FEDER), Severo Ochoa Centre of Excellence and MultiDark Consolider (MINECO), and Prometeo program (Generalitat Valenciana), Spain. We take the help of SAMKHYA: High Performance Computing Facility at Institute of Physics, Bhubaneswar to perform the numerical simulations.

## Author information

### Authors and Affiliations

### Corresponding author

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Funded by SCOAP^{3}

## About this article

### Cite this article

Khatun, A., Chatterjee, S.S., Thakore, T. *et al.* Enhancing sensitivity to non-standard neutrino interactions at INO combining muon and hadron information.
*Eur. Phys. J. C* **80**, 533 (2020). https://doi.org/10.1140/epjc/s10052-020-8097-1

Received:

Accepted:

Published:

DOI: https://doi.org/10.1140/epjc/s10052-020-8097-1