1 Introduction

In the Standard Model (SM), neutrinos are massless fermions which interact only via the weak interaction through the exchange of \(W^\pm \) or \(Z^0\) bosons. A series of experiments dedicated to neutrinos [1,2,3,4,5,6,7,8,9] have proved the existence of neutrino flavour oscillations, which implies that neutrinos are massive. The neutrino flavour states produced along with charged leptons are linear superpositions of the mass eigenstates. Due to the difference in phase between the wave packets of each of the mass eigenstates, neutrino oscillations occur [10, 11].

In the case of three neutrino flavours, the mixing is described by a \(3\times 3\) unitary matrix called the PMNS (Pontecorvo–Maki–Nakagawa–Sakata) matrix [10, 11], where the oscillations are governed by the following parameters: two mass-square difference terms \((\varDelta m^{2}_{21}, \varDelta m^{2}_{32};~\varDelta m^2_{ij}\equiv m_i^2-m_j^2;~i,j=1,2,3;~i\ne j)\), mixing angles (\(\theta _{12}, \theta _{13}, \theta _{23}\)) and one \((\delta )\) or three \((\delta , \alpha _{21}, \alpha _{31})~CP\) violating phases, depending on whether neutrinos are Dirac or Majorana [12, 13] particles, respectively. The present information on these parameters can be found in Ref. [14]. However, the sign of \(\varDelta m^{2}_{32}\) and whether \(\theta _{23}\) results in maximal mixing \(\theta _{23}=45^{\circ }\) or is in the upper (lower) quadrant \(\theta _{23}>45^{\circ }\) \((\theta _{23}<45^{\circ })\), is yet to be determined.

The mixing angle \(\theta _{13}\) is found to be non-zero from reactor [6, 8, 9] and accelerator [15, 16] neutrino oscillation experiments. The relatively large value of \(\theta _{13}\cong 8.6^{\circ }\) has intensified the search for CP violation effects in neutrino oscillations, and also the determination of the sign of \(\varDelta m^{2}_{32}\) via matter effects [17, 18]. Matter plays an important role in enhancing the effect of \(\sin \theta _{13}\) via resonance, which is sensitive to the sign of \(\varDelta m^{2}_{32}\) and is different for neutrinos and antineutrinos [19, 20]. Determination of the sign of \(\varDelta m^{2}_{32}\) would help us understand the correct mass hierarchy (MH) of the neutrinos, i.e., whether the MH is \((m_1<m_2<m_3)\) normal (NH) or \((m_3<m_1<m_2)\) inverted (IH) hierarchy. A series of experiments with complementary approaches have been proposed using accelerator, reactor and atmospheric neutrinos to determine the MH [21, 22]. The intermediate and long baseline, off-axis accelerator neutrino experiments T2K [5] and NO\(\nu \)A [23, 24], search for the appearance of \(\nu _e\) in an intense beam of \(\nu _\mu \), wherein the appearance probability depends on the MH of the neutrino states. Recent results from NO\(\nu \)A [25] have disfavoured the IH by more than \(93\%\) C.L. in the lower octant for all values of CP phase \(\delta \), and it is in good agreement with the T2K result [26] giving a near maximal value of \(\theta _{23}\) and a weak preference of NH. Liquid scintillator detectors proposed at RENO-50 [8] and JUNO [27] could unravel the MH using reactor neutrinos. Atmospheric neutrino experiments using water or ice Cherenkov detectors, such as Hyper-K [28, 29], MEMPHYS [30], ORCA and PINGU [31, 32], make use of different cross-sections and different \(\nu \) and \({\bar{\nu }}\) fluxes to study the MH.

The proposed magnetized Iron Calorimeter (ICAL), to be built at the India-based Neutrino Observatory (INO) [33], will study interactions involving primarily atmospheric muon neutrinos and anti-neutrinos. It will consist of three identical modules, each with dimension \(16~\mathrm {m} \times 16~\mathrm {m} \times 14.5 ~\mathrm {m}\) (length \(\times \) width \(\times \) height) placed in a line and separated by a small gap of \(20~\mathrm {cm}\). Each module will consist of 151 layers of 5.6 cm thick iron plates interleaved with 4 cm air gap containing the active detector elements, glass Resistive Plate Chambers (RPCs). This huge size of \(48~\mathrm {m} \times 16~\mathrm {m} \times 14.5 ~\mathrm {m}\) magnetized detector, with a mass of 50 kton, provides the target nuclei to achieve a statistically significant number of neutrino interactions within a reasonable time frame.

One of the main goals of INO is to study the MH via earth matter effects, and to determine the octant of \(\theta _{23}\). ICAL is designed to have very good muon detection efficiency of greater than \(85\%\) for muons greater than \(2\,\mathrm {GeV}\) (with zenith angle \(\cos \theta _z\ge 0.4\)), combined with excellent angular resolution. The ICAL will prominently observe the matter effects on the upward going muon neutrinos, where it uses the magnetic field to distinguish between \(\nu _{\mu }\) and \({\bar{\nu }}_{\mu }\) events by discriminating the charge on the final state muon. The probability of oscillations are different for \(\nu _{\mu }\) and \({\bar{\nu }}_{\mu }\) in the presence of matter, and depends on the MH (sign of \(\varDelta m^2_{32}\)). The seen relative difference between normal and inverted hierarchy is \(3.9\%~(-5\%)\) for \(\nu _{\mu }\) \(({\bar{\nu }}_\mu )\) for \(\cos \theta _z>0.2\), assuming NH to be the true hierarchy (where \(\cos \theta _{z} = +1\) is the up-going direction). Hence the ICAL could study the MH by observing earth-matter effects independently on \(\nu \) and \({\bar{\nu }}\).

This paper shows the precision reach of ICAL in the \(\sin ^2\theta _{23}-|\varDelta m^{2}_{32}|\) plane for a 5 year run of ICAL. Event-by-event reconstruction and fluctuations arising from low event statistics, using an analysis technique that will be suitable to be employed on the actual data. In the previous studies [33], a lookup table was obtained for the reconstruction efficiencies and resolutions from the studies based on single muons of fixed energy and direction. Later the generated data is folded with the detector efficiencies and smeared by the resolution functions within the lookup table. Hence, the previous methods used parameterizations of the efficiency and resolution that do not reflect the tails of these distributions. As well as those methods used very large sample sizes to negate the effect of low statistics. We also apply a few event selection criteria, as presented in Ref. [34], but within the framework of low event statistics and present its effect on the outcome of this analysis. This paper also compares the results from simulated unfluctuated data, by simulating data sets corresponding to 5-years of data.

This paper is organized as follows. In Sect. 2, we outline the methodology describing the event detection in the INO detector and the software framework used to simulate and reconstruct the events in the detector. In Sect. 3, we describe the event generation and discuss the fluctuations in the data. We describe how the events are reconstructed as well as the event selection criteria applied to obtain a sample of events. We also describe the oscillation analysis including the Earth-matter effects and discuss how data collected by the ICAL are sensitive to the MH. We also describe the \(\chi ^2\) analysis and the binning scheme used, and also discuss the types of systematic uncertainties used in this analysis.

In Sect. 4, we present the results of our simulated analysis, showing the reach of the ICAL for atmospheric oscillation parameters \(\theta _{23}\) and \(\varDelta m^{2}_{32}\). Event selection reduces the event statistics, hence we also present the results with and without event selection to see its effect. We also discuss the effect of fluctuations on the precision measurements and show the different possible outcomes resulting from the low event statistics. We also discuss the results on the MH of the neutrinos and the effects of fluctuations in determining it. Finally, in Sect. 5 we present the summary of our results and conclusions.

2 Methodology

The NUANCE [35] neutrino event generator, along with the Honda neutrino flux [36] at the Kamioka site, is used to generate neutrino interactions within the ICAL detector. The proposed ICAL geometry, containing mainly iron and glass components of the detector, is given as input to NUANCE. It generates secondary particles from interactions with these materials, and calculates event rates integrated over the weighted flux and cross sections of all charged-current (CC) and neutral-current (NC) interactions, at each neutrino energy and angle. The output from NUANCE contains vertex and timing information, as well as energy and momentum of all initial and final state particles in each event.

In the ICAL, atmospheric neutrinos will interact with an iron nucleus, undergoing NC and CC interactions. The main CC interactions taking place in the detector are quasi-elastic (QE) and resonance (RS) at low energies and deep-inelastic scattering (DIS) at higher energies. All neutrinos interacting via CC interaction produce an associated lepton. The DIS events produce a number of hadrons along with the lepton, while RS interactions produce at most one hadron. In the QE process, no hadrons are produced and the final-state lepton takes away most of the energy of the incident neutrino.

A C++ code developed by the collaboration using the GEANT4-based [37] simulation toolkit, containing the full ICAL detector geometry, magnetic field map and RPC characteristics, is used to propagate the secondary particles. The output from GEANT4 contains the information on energy loss and momentum of the particles all along its path, which are then digitized and reconstructed to obtain the event observables. The detailed information of which are discussed in Sect. 3.2. This paper is based only on the CC neutrino events, where we consider the muon information alone. The information on the energy and direction of the muons is used to study the sensitivity to atmospheric neutrino oscillation parameters \(\theta _{23}\) and \(\varDelta m^{2}_{32}\) at INO-ICAL and resolving the MH. Including hadron information is beyond the scope of this paper.

3 Analysis procedure

The first step in the procedure is event generation. The generated events are reconstructed in the GEANT4-simulated ICAL and oscillations applied event-by-event after event selection. The oscillated events are binned and used in the \(\chi ^2\) analysis to determine the oscillation parameters. Each of these procedures are described in detail in the subsections below.

3.1 Event generation

In this analysis, NUANCE data for an exposure of 50 kton \(\times \) 1000 years is generated, out of which sub-samples corresponding to 5 years of data are used as the experimentally simulated sample and the remaining 995 years of data are used to construct probability distribution functions (PDF) that are used in the \(\chi ^2\) fit. Hence the data are uncorrelated with the PDFs that are used to fit the data. This paper is based only on the CC neutrino events with energies less than \(50~\mathrm {GeV}\) which corresponds to \(98.6\%\) of the sample. The idealized case, where the NUANCE data is folded with detector efficiencies and smeared by the resolution functions obtained from GEANT-based studies of single muons with fixed direction and energy, has been presented previously [38]. In the earlier analysis, although the data was analysed for an exposure of 5 or 10 years, it was scaled down from the 1000 year sample. Hence, the reconstructed central value was always practically the same as the input value. Here we examine in detail the more realistic case, where the data size and central value are both subject to fluctuations.

3.2 Event reconstruction

The generated NUANCE data is simulated in GEANT4-based detector environment. The energy deposited due to the energy loss of the charged particle in the RPCs are converted to signals, where they are detected by the mutually orthogonal copper strips (along x and y directions parallel to the global detector coordinates described in Sect. 3.3) on the RPCs. Hence the measured data is digitized to form (xz) or (yz) and time t of the signal, referred to as hits. Here the z position is given by the layer number of the RPC. A recursive optimal state estimator – the Kalman Filter [39, 40], uses the local geometry and magnetic field information to fit the muon hits, where the muons passing through a minimum of three layers are fit to form the track. The direction and the momentum of the muon is obtained from the best fit values of the track. Also the timing information from the RPCs, with a resolution of approximately \(1~\mathrm {ns}\) enables the distinction between upward and downward going particles. More details can be found in Ref. [40].

Hence for the first time we have done this analysis performing event-by-event reconstruction, where each event is simulated through the detector and reconstructed to obtain the observables. Therefore the tails of the resolution functions, which have been approximated by single Gaussians and Vavilov [41] functions in the previous studies [33], are also taken in to account in this analysis.

The \(\mu ^\pm \) leave one or two hits per layer on average, forming a well-defined track, whereas the hadrons leave several hits per layer forming a shower of hits. Rarely (less than 1% of the time) a pion may also leave a well-defined track in the ICAL and may be misidentified as a muon. In this case the longest track is identified as the muon. The iron plates will be magnetised to produce a field upto \(1.5~\mathrm {T}\) and this will be used in the ICAL to probe the charge and momentum of the muon. The direction and the curvature of the muon trajectory, as it propagates through the magnetized detector, gives its charge and momentum, respectively. Figure 1 shows the zenith angle \((\theta _z)\) distribution before (true) and after reconstruction. Note that in the current analysis \(\cos \theta _{z} = +1\) is the up-going direction.

Fig. 1
figure 1

Comparison of true (dashed black) and reconstructed (solid orange) zenith angle (\(\cos \theta _z\)) distribution for muons, averaged over energy and azimuthal angle for an exposure of \(50~ \mathrm {kton} \times 5~\mathrm {years}\) of ICAL.

The energy of hadrons is obtained by calibrating the number of hits not associated with the muon track, in the event [42]. The incident neutrino energy (\(E_{\nu }\)) can be reconstructed from the energies of the muons and hadrons produced in the detector. The poor energy resolution of hadrons [42] affects the reconstruction of the incident neutrino. Hence for ICAL physics analysis hadron and muon energies are used separately, without losing the good energy and angular resolution of muons [33, 38, 43, 44].

3.3 Event selection

The reconstruction of muons is badly affected by the non-uniform magnetic field and dead spaces such as coil slots and support structures. Also the horizontal events which pass through very few layers giving very few hits are reconstructed poorly. Partially contained events, where the \(\mu ^{\pm }\) leaves the detector volume, are typically harder to reconstruct, as most of the time they leave a short track within the detector. To remove these badly reconstructed events and obtain a better reconstructed sample of data, we investigated applying selection cuts as used in the previous Monte Carlo (MC) studies [34].

Events with \(\chi ^2/\mathrm {ndf}<10\) are used in the analysis, where the \(\chi ^2\) is the chi-square estimate of the fit for the track obtained from the Kalman filter, and ndf is the number of degrees of freedom. Here \(\mathrm {ndf}=2 N_{\mathrm {hits}}-5\), where the Kalman filter fits five parameters to form the track and \(N_{\mathrm {hits}}\) are the number of hits associated with the track, with each hit having two degrees of freedom as they are either (xz) or (yz) in coordinates. The badly reconstructed horizontal events are removed by applying a cut of \(|\cos \theta _z|\ge 0.35\). Also, to keep a check on the events leaving from top and bottom of the detector, a cut on the z-position of the event vertex is applied. Events with vertices lying below \(z=6~\mathrm {m}\) and above \(z=-6~\mathrm {m}\) are the ones selected from up-going and down-going events respectively.

The entire ICAL detector was divided into three regions, depending on the magnitude of the magnetic field. Considering three modules of size \(16~\mathrm {m}\times 16~\mathrm {m}\times 14.4~\mathrm {m}\) each and choosing an origin at the centre of the central module, the ICAL will have conventionally \(24~\mathrm {m}\), \(8~\mathrm {m}\) and \(7.2~\mathrm {m}\) on either side of the origin along x, y and z directions respectively. The region \(|x|\le 20~\mathrm {m}\), \(|y|\le 4~\mathrm {m}\), with z unconstrained is defined to be the central region. Here the magnetic field is highest and uniform in magnitude (with \(\approx 12\%\) coefficient of variation) despite the fact that the direction of the magnetic field would flip along y in the regions \(|x|< 4~\mathrm {m}\), \(4~\mathrm {m}\le |x|< 12~\mathrm {m}\) and \(12~\mathrm {m}\le |x|< 20~\mathrm {m}\). In contrast, the region \(|y|> 4~\mathrm {m}\), termed peripheral region, has maximally varying magnetic field in both magnitude (with \(\approx 28\%\) coefficient of variation) and direction. Finally the third region \(|x|> 20~\mathrm {m}\) and \(|y|\le 4~\mathrm {m}\), termed the side region, has a magnetic field smaller by \(\approx 11\%\) and opposite in direction to the central region. The side region has better uniform magnetic field among the three regions (with less than \(5\%\) coefficient of variation).

All the events with the interaction vertices in the central region, and with \(N_{\mathrm {hits}}>0\), are selected as they are either contained within the detector or can form a reasonable length of track to identify the direction and momentum. The rest of the events in the peripheral and side regions are classified into partially (PC) and fully (FC) contained events according to the end position of the track. If the track end lies within \(|x|\le 23~\mathrm {m}\) and \(|y|\le 7.5~\mathrm {m}\) and \(z\le 7~\mathrm {m}\), then the event is classified as FC and is selected. The remaining events are classified as PC, and a selection criterion of \(N_{\mathrm {hits}}>15\) is applied on all such PC events. Table 1 summarise the selection criterion used in this analysis.

Table 1 Lists the region wise and the overall (common) selection criterion used in this current analysis
Fig. 2
figure 2

a (From top to bottom) Zenith angle resolution, energy resolution, reconstruction efficiency and charge identification efficiency, for the events with (WS) (open circles) and without (WOS) (solid circles) event selection, as a function of true muon energy \(E_{\mu }^\mathrm{true}\). The statistical error corresponds to \(50~\mathrm {kton}\times 100~\mathrm {years}\) reconstructed data sample. b Comparison of reconstructed zenith angle (\(\cos \theta _z\)) for WOS (dashed orange) and WS (solid blue), averaged over energy and azimuthal angle for an exposure of \(50~ \mathrm {kton} \times 5~\mathrm {years}\) of ICAL

The response of ICAL can be quantified in terms of reconstruction efficiency (\(\epsilon _{\mathrm {rec}}\)), the relative charge ID (CID) efficiency (\(\epsilon _{\mathrm {cid}}\)), the energy \((\sigma _{E_{\mu }})\) and angular resolutions \((\sigma _{\cos \theta _z})\). The Fig. 2a shows the comparison of these quantities with (WS) and without (WOS) event selection, as a function of true muon energy \(E_{\mu }^{\mathrm {true}}\). The reconstruction efficiency increases with increase in muon energy, as the energetic muon pass through large number of layers. At higher energies the reconstruction efficiencies become almost a constant. The resolution improves considerably at low and high energies with the selection of events. An overall improvement of \(23\%\) and \(19\%\) is noticed in energy and zenith angle resolution of muons (averaged over muon energy, zenith and azimuthal angles) after the event selection. The CID efficiency, i.e., the fraction of events identified with correct muon charge among the total reconstructed events, shows \(\sim 6\%\) to \(10\%\) improvement at all muon energies, after the event selection.

The previous studies without the event-by-event reconstruction obtained an angular resolution less than \(1^{\circ }\) for near horizontal events (\(\cos \theta _z=-0.35\)) with \(E_{\mu }>5~\mathrm {GeV}\) [38], whereas it deteriorates to less than \(8^{\circ }\) after realistic reconstruction. Similarly the CID efficiency was calculated to be above \(95\%\) for all the events, whereas it deteriorates in between \(60\%\) to \(80\%\) after event-by-event reconstruction. The resolutions and efficiencies in the low energy \((E_{\mu }<5~\mathrm {GeV})\) are relatively worse, whereas the near vertical events show a good agreement with the previous results.

The reconstruction efficiency decreases with event selection as expected, where approximately \(40\%\) of reconstructed events are lost with event selection. Figure 2b, shows the reconstructed \(\cos \theta _z\) distribution and compares the reconstructed events with (WS) and without selection (WOS) criterion. The dip at \(\cos \theta _{z}=0\) results from the difficulty to reconstruct horizontal events. This paper also studies the effect of the applied event selection on the sensitivity of oscillation parameters in ICAL. Hence, the parameter sensitivity is studied with and without applying any selection criterion.

Table 2 Assumed values of oscillation parameters [46] used to construct the pseudo-data

3.4 Applying oscillations

The muon signal in the ICAL will have contributions from the component of the \(\nu _{e}\) flux \((\varPhi _{\nu _e})\) that has oscillated to \(\nu _{\mu }\) and the component from \(\nu _{\mu }\) flux \((\varPhi _{\nu _{\mu }})\) that has survived. As the 5-year pseudo-data would have negligible contribution from \(\varPhi _{\nu _e}\) compared to \(\varPhi _{\nu _{\mu }}\), we have only used the \(\varPhi _{\nu _{\mu }}\) events in our analysis. Hence, neglecting oscillations from \(\varPhi _{\nu _e}\), the total number of events appearing in the detector for an exposure time T is obtained from

$$\begin{aligned} \frac{d^2N}{dE_{\nu } \ d\cos \theta _{z}}=T \times N_D \times \sigma _{\nu _{\mu }} \left[ P_{\mu \mu }\frac{d^2\varPhi _{\nu _{\mu }}}{dE_{\nu } \ d\cos \theta _{z}}\right] , \end{aligned}$$
(1)

where \(N_D\) is the number of targets in the detector and \(P_{\mu \mu }\) is the \(\nu _\mu \) survival probability. Oscillation probabilities are calculated by numerically evolving the neutrino flavor eigenstates [19] using the equation,

$$\begin{aligned} i\frac{d}{dt}\left[ \nu _{\alpha }\right] =\frac{1}{2E}\left( UM^2U^{\dagger }+{\mathcal {A}}\right) \left[ \nu _{\alpha }\right] , \end{aligned}$$
(2)

where \(\left[ \nu _{\alpha }\right] \) denotes the vector of flavor eigenstates, \(\nu _{\alpha }\), \(\alpha = e, \mu , \tau \), U is the PMNS mixing matrix, and \(M^2\) is the mass squared matrix. Here, \({\mathcal {A}}\) is the diagonal matrix, diag(A, 0, 0), with matter term A given by

$$\begin{aligned} \begin{aligned} A =&~\pm 2\sqrt{2}G_Fn_eE \\ =&~\pm 7.63 \times 10^{-5}~\rho E~\mathrm {eV^2}, \end{aligned} \end{aligned}$$
(3)

where the sign is positive for \(\nu \) and negative for \({\bar{\nu }}\). Here, \(G_F\) is the Fermi coupling constant, E is the neutrino energy in \(\mathrm {GeV}\), and \(n_e\) is the electron number density which is related to the matter density \(\rho \) in \(\mathrm {gcm^{-3}}\). The density profile of the Earth’s matter, given by Preliminary Reference Earth Model (PREM) [45], is used to calculate oscillation probabilities for \(\nu \) and \({\bar{\nu }}\). The difference in sign of A for \(\nu \) and \({\bar{\nu }}\) leads to differing oscillation probabilities, which in turn are sensitive to the sign of \(\varDelta m^2_{32}\). The ICAL has an advantage because it can differentiate between \(\nu \) and \({\bar{\nu }}\) events and observe the matter effects separately.

Fig. 3
figure 3

\(\cos \theta _z\) distributions with and without oscillations for a \(\mu ^+\) obtained from \({\bar{\nu }}_{\mu }\) events with NH, b \(\mu ^-\) obtained from \(\nu _{\mu }\) events with NH, c \(\mu ^+\) obtained from \({\bar{\nu }}_{\mu }\) events with IH and d \(\mu ^-\) obtained from \(\nu _{\mu }\) events with IH. The distributions are also compared with (WS) and without (WOS) selection criterion

Oscillations are applied on the 5-year data sample using the accept or reject method. First, the survival probability \(P_{\mu \mu }\) is calculated for each \(\nu \) or \({\bar{\nu }}\) with a given energy and direction. To decide whether an un-oscillated \(\nu _{\mu }\) survives oscillations to be detected as \(\nu _{\mu }\), a uniform random number r is generated between 0 and 1. If \(P_{\mu \mu }>r\), the event is accepted to have survived the oscillations. Otherwise it is considered to have oscillated into another flavor and is rejected. Here, we have used the true values of the oscillation parameters, from Ref. [46]; see Table 2. The zenith angle distribution of muons before and after applying oscillations via the accept or reject method is shown in Fig. 3, where the effect of change in sign of \(\varDelta m^2_{32}\) (MH) is shown for \({\bar{\nu }}_{\mu }\) (Fig. 3a, c) and \(\nu _{\mu }\) (Fig. 3b, d) events. It also compares the zenith angle distributions with (WS) and without (WOS) event selection. Note that the oscillation signatures are different in \({\bar{\nu }}_{\mu }\) and \(\nu _{\mu }\) events for normal and inverted hierarchies. This difference is solely due to the matter effects, where it depends on the sign of \(\varDelta m^2_{32}\) as we have assumed no CP violation. (It has been clearly established that CC \(\mu \) events in the ICAL are insensitive to \(\delta _{\mathrm{cp}}\) [33].) Hence \(\nu _{\mu }\) events are separated from \({\bar{\nu }}_{\mu }\) events while binning, to have maximum sensitivity to the MH.

3.5 Binning scheme

During reconstruction, the positively charged particles are tagged with positive momentum and the negatively charged particles are tagged with negative momentum by convention. The reconstructed muons of positive and negative charges are binned separately in \(Q_{\mu }E_{\mu }\) and \(\cos \theta _z\) bins after applying oscillations, where \(Q_{\mu }=\pm 1\) for \(\mu ^+/\mu ^-\). The events with negative and positive \(Q_{\mu }E_{\mu }\) indicate those identified as \(\nu _{\mu }\) and \({\bar{\nu }}_{\mu }\) events respectively. The atmospheric neutrino flux falls rapidly at higher energies. Hence, wider bins were chosen in those energy regions to ensure adequate statistics. Also, within the frame work of low event statistics, increasing the number of high energy bins is not feasible due to the limited statistics. Table 3 summarises the binning scheme used in the current analysis.

Fig. 4
figure 4

PDF’s for \({\bar{\nu }}_\mu \) and \(\nu _\mu \) are shown for a binning in \(\cos \theta _z\), and b binning in \(E_{\mu }\). The \({\bar{\nu }}_{\mu }~(\nu _{\mu })\) entries for \(Q_{\mu }E_{\mu }<0~(>0)\) indicate the charge mis-id content

The effect of finer binning is studied previously [38] for energies less than \(11~\mathrm {GeV}\), and is known to marginally improve the precision in both \(\sin ^2\theta _{23}\) and \(|\varDelta m^2_{32}|\). Also, increasing the range of energies beyond \(11~\mathrm {GeV}\) is known to improve the result [47]. Increasing the number of high energy bins can improve the precision. The optimization of bin widths at higher energies will be a part of the future work, and the current analysis will focus on the effects of fluctuations arising from the low event statistics.

Table 3 The binning scheme for the reconstructed observables \(\cos \theta _z\) and \(E_{\mu }\)

3.6 The \(\chi ^2\) analysis and systematics

The pull approach [48] is used in defining the \(\chi ^2\) such that systematic uncertainties are incorporated. The pull approach is equivalent to the covariance approach, but is computationally much faster. After binning the oscillated events, the 5 year simulated data set is fit by defining the following \(\chi ^2\) [38, 49]:

$$\begin{aligned} \chi ^2= & {} \min _{\{\xi _k\}}\sum _{i=1}^{n_{\cos \theta _{z}}}\sum _{j=1}^{n_{E_{\mu }}}2\left[ \phantom {\left. -N_{ij}^{\mathrm {data}}\ln \left( \frac{N_{ij}^{\mathrm {pdf}}}{N_{ij}^{\mathrm {data}}}\right) \right] } \left( N_{ij}^{\mathrm {pdf}}-N_{ij}^{\mathrm {data}}\right) \right. \nonumber \\&\left. -N_{ij}^{\mathrm {data}}\ln \left( \frac{N_{ij}^{\mathrm {pdf}}}{N_{ij}^{\mathrm {data}}}\right) \right] + \sum _{k=1}^{2}\xi _k^2, \end{aligned}$$
(4)

where,

$$\begin{aligned} N_{ij}^{\mathrm {pdf}}= R\left[ f T_{ij}^{{\bar{\nu }}} +(1-f) T_{ij}^{\nu }\right] \left[ 1+\sum _{k=1}^{2}\pi _{ij}^k\xi _k\right] . \end{aligned}$$
(5)

Here, \(N_{ij}^{\mathrm {data}}\) and \(N_{ij}^{\mathrm {pdf}}\) are the observed and the expected number of muon events respectively, in a given (\( \cos \theta _{z}^i,E_{\mu }^j\)) bin, while \(n_{\cos \theta _{z}}\) and \(n_{E_{\mu }}\) are the total number of \(\cos \theta _{z}\) and \(E_{\mu }\) bins respectively. \(N_{ij}^{\mathrm {data}}\) is calculated for true values of oscillation parameters, summarised in Table 2, whereas \(N_{ij}^{\mathrm {pdf}}\) is obtained by combining \(\nu _\mu \) and \({\bar{\nu }}_\mu \) PDFs as in Eq. 5, where \(T_{ij}^{\nu }\) and \(T_{ij}^{{\bar{\nu }}}\) are the \(\nu \) and \({\bar{\nu }}\) PDFs respectively normalized from 995 year sample, with f as a free parameter describing the relative fraction of \({\bar{\nu }}_\mu \) and \(\nu _{\mu }\) in the sample, with R being the normalization factor in the fit which scales the PDF to 5 years. The \({\bar{\nu }}_\mu \) and \(\nu _\mu \) PDFs with (WS) and without (WOS) selection criterion are shown in Fig. 4a, b.

Fig. 5
figure 5

\(\varDelta \chi ^2\) as a function of a \(\sin ^2\theta _{23}\), for an input value of \(\sin ^2\theta _{23}(\hbox {true}) = 0.5\) and b \(\varDelta m^{2}_{32}\), for an input value of \(\varDelta m^{2}_{32}(\hbox {true})= 2.32\times 10^{-3}~\mathrm {eV^2}\). The dashed (orange) and solid (blue) line shows the fit with (WS) and (WOS) event selection for a fluctuated data set

The systematic errors and the theoretical uncertainties are parametrized in terms of variables \(\left\{ \xi _k \right\} \) called pulls. The value \(\xi _k = 0\) corresponds to the expected value, and the variation, \(\xi _k = \pm 1\) corresponds to a one standard deviation for each source of systematics resulting in an uncertainty of \(\pi _{ij}^k\) for the \(k^{\mathrm {th}}\) source.

In this analysis we have considered two systematic uncertainties, a 5% uncertainty on the zenith angle dependence of the flux and another 5% on the energy dependent tilt error [49], parametrized by \(\xi _{\mathrm {zenith}}^{\mathrm {flux}}\) and \(\xi _{\mathrm {tilt}}^{\mathrm {flux}}\) respectively. There is no systematic uncertainty related to the flux normalization as R and f are the fit parameters which fixes the overall and relative flux normalizations. To calculate the energy tilt error i.e., the possible deviation of the energy dependence of the atmospheric fluxes from the power law, we use the standard procedure as given in, for example Ref. [49], and define:

$$\begin{aligned} \varPhi _{\delta }(E)= \varPhi _{0}(E)\left( \frac{E}{E_0}\right) ^{\delta } \approx \varPhi _{0}(E)\left[ 1+\delta \ln \frac{E}{E_0}\right] , \end{aligned}$$
(6)

Neglecting the effect of oscillations, the expected number of events \(\varPhi _{0}(E)\) is calculated for each \((ij)^{th}\) bin. Then we compute \(\varPhi _{\delta }(E)\), where \(\delta \) is the \(1\sigma \) tilt error taken to be 5% and \(E_0 = 2~\mathrm {Gev}\), and find the relative change in flux to obtain the coupling \(\pi _{ij}^{\mathrm{tilt}}\). The coupling \(\pi _{ij}^{\mathrm{zenith}}\) in each bin is calculated in proportion to the zenith angle value of that particular bin. The parameters in the fit are marginalized as given in Table 4, where f and R are always marginalized over the given ranges. The parameters \(\sin ^2\theta _{13}\), \(\sin ^2\theta _{12}\) and \(\varDelta m^2_{21}\) had minimal effect when marginalized hence were kept constant in the fit without any prior constraint.

Table 4 Marginalization of the parameters as used in the fit

4 Parameter determination

The fluctuated pseudo-data set is first fit to determine \(\sin ^2\theta _{23}\), marginalizing over \(|\varDelta m^2_{32}|\), for an input value of \(\sin ^2\theta _{23} = 0.5\). The comparison of \(\varDelta \chi ^2\) with (WS) and without (WOS) event selection is shown as a function of \(\sin ^2\theta _{23}\) in Fig. 5a, where the octant degeneracy in \(\sin ^2\theta _{23}\), which stems from the leading term \(\sin ^22\theta _{23}\) in the oscillation probability, is broken due to the relatively large value of \(\theta _{13}=8.5^{\circ }\). Hence, the asymmetrical curve in \(\sin ^2\theta _{23}\) shows the effect of matter oscillations in breaking octant degeneracy. The significance of the fit, i.e., how far the observed value (best fit value) is away from the parameters true value (input value), is defined as

$$\begin{aligned} \mathrm {significance} = \sqrt{\varDelta \chi ^2_{\mathrm {input}}-\varDelta \chi ^2_{\mathrm {min}}}~, \end{aligned}$$
(7)

where \(\varDelta \chi ^2_{\mathrm {input}}\) and \(\varDelta \chi ^2_{\mathrm {min}}\) are the \(\varDelta \chi ^2\) values at the true and observed values of the parameter respectively. The fit to \(\sin ^2\theta _{23}\) without event selection converges to a value of \(0.586^{+0.060}_{-0.093}\) with a significance of 0.86, i.e., within \(1\sigma \) of the input value, whereas the fit after event selection converges to \(0.676^{+0.063}_{-0.072}\) within \(2\sigma \) of the input value. The fit to \(\sin ^2\theta _{23}\) with event selection shows relatively larger uncertainty at 2 and \(3\sigma \) range.

The comparison of \(\varDelta \chi ^2\) with and without event selection is shown as a function of \(|\varDelta m^2_{32}|\) in Fig. 5b, where the data is fit to determine \(|\varDelta m^2_{32}|\), marginalizing over \(\sin ^2\theta _{23}\), for an input value of \(\varDelta m^2_{32} = 2.32\times 10^{-3}~\mathrm {eV^2}\). The fit without event selection converges to a value of \((2.38^{+0.11}_{-0.39})\times 10^{-3}~\mathrm {eV^2}\), within \(1\sigma \) of the input value with a significance of 0.51, whereas the fit after event selection converges to \((2.184^{+0.23}_{-0.37})\times 10^{-3}~\mathrm {eV^2}\) also within \(1\sigma \) of the input value. The fit to \(|\varDelta m^2_{32}|\) also shows relatively larger uncertainty at 2 and \(3\sigma \) range after applying event selection. The multiple local minimas in \(\varDelta \chi ^2\) function is due to the statistical uncertainty on the PDF, and it is observed to reduce with fits to PDFs constructed from larger MC samples.

Fig. 6
figure 6

Precision reach obtained from the fit to a 5 year fluctuated pseudo-data set in \(\sin ^2\theta _{23}-\varDelta m^2_{32}\) plane. The dashed (orange), dotted (blue) and the broken (black) line shows the coverage area with \(99\%\), \(90\%\) and \(68\%\) CL respectively without (WOS) event selection, whereas the solid (magenta) line shows the coverage area with \(99\%\) CL with (WS) event selection. The input (true) point is given by the green dot and the plus sign signifies the best fit (converged) point

Fig. 7
figure 7

\(\varDelta \chi ^2\) as a function of a \(\sin ^2\theta _{23}\), for an input value of \(\sin ^2\theta _{23}(\hbox {true}) = 0.5\) and b \(\varDelta m^{2}_{32}\), for an input value of \(\varDelta m^{2}_{32}(\hbox {true})= 2.32\times 10^{-3}~\mathrm {eV^2}\). The solid (orange) and dashed (magenta) line shows the fit without (WOS) and with (WS) the event selection for the unfluctuated data set

The parameters \(\sin ^2\theta _{23}\) and \(\varDelta m^2_{32}\) are correlated; Fig. 6 compares the correlated precision reach obtained fitting a 5 year pseudo-data set with and without event selection. The best-fit point to the fit without event selection is obtained within a significance of \(1\sigma \) from the input value at \(\sin ^2\theta _{23}= 0.586^{+0.060}_{-0.093}\) and \(\varDelta m^2_{32} = (2.381^{+0.11}_{-0.39})\times 10^{-3}~\mathrm {eV^2}\), whereas the fit with event selection converges within a significance of \(2\sigma \) at \(\sin ^2\theta _{23}= 0.676^{+0.063}_{-0.072}\) and \(\varDelta m^2_{32} = (2.184^{+0.23}_{-0.37})\times 10^{-3}~\mathrm {eV^2}\). The free parameter f converges to \(0.26\pm 0.01\) and \(0.27\pm 0.01\) before and after applying the selection, where as the parameter R converges to \(5901\pm 79\) and \(3628\pm 62\) respectively within \(2\%\) of uncertainty. Further adding prior constraints on \(\sin ^2\theta _{13}\) and \(\varDelta m^2_{12}\), was observed not to make any difference in the fit results or in terms of the coverage in the \(\sin ^2\theta _{23}-\varDelta m^2_{32}\) plane. The fit with event selection shows larger coverage at \(99\%\) CL. Note that the data sets are fluctuated, hence many different sets were also studied and all showed a larger coverage for the fits with event selection.

To decouple the effect of fluctuations from the effect of event selection, a 1000 year sample is scaled to required 5-year sample to minimize the fluctuations. The Fig. 7a, b shows the comparison of \(\varDelta \chi ^2\) obtained with and without event selection as a function of \(\sin ^2\theta _{23}\) and \(|\varDelta m^2_{32}|\) respectively for the fit without fluctuations. The fits converge near the input value, and shows a relatively worse precision after the event selection as expected. The deterioration of precision in determining \(\sin ^2\theta _{23}\) and \(\varDelta m^2_{32}\) after the event selection is due to larger statistical uncertainty, as the sample size was reduced by \(40\%\). Hence, the rest of the analysis in this paper mainly focus on the fits and the effects of fluctuations without event selection.

Fig. 8
figure 8

Effect of fluctuations on \(\varDelta \chi ^2\) as a function of a \(\sin ^2\theta _{23}\), for an input value of \(\sin ^2\theta _{23}(\hbox {true}) = 0.5\) and b \(\varDelta m^{2}_{32}\), for an input value of \(\varDelta m^{2}_{32}(\hbox {true})= 2.32\times 10^{-3}~\mathrm {eV^2}\) . The solid (orange) curve represents the fit to data without (WOF) fluctuations, and is compared to the fit to three other independent pseudo-data sets with (WF) fluctuations [WF : 1 (dashed, magenta), WF : 2 (dotted, blue) and WF : 3 (broken, black)]

4.1 Effect of fluctuations

Earlier analyses [33] scaled the 1000 year sample to a size corresponding to 5 years to generate the pseudo-data set. The process of scaling nullifies the effect of fluctuations so that the best-fit is always close to the input value. Then the parameter sensitivities are to be understood as the median value when averaged over a large number of randomly generated samples. In order to see this, we generate an unfluctuated 5 year sample by scaling the 1000 year set and a similar analysis is performed as in Sect. 3. The comparison of \(\varDelta \chi ^2\) with (WF) and without (WOF) fluctuations is shown as a function of \(\sin ^2\theta _{23}\) and \(|\varDelta m^2_{32}|\) in Fig. 8a, b respectively. The ideal fit without fluctuations (WOF) converges near the true (input) value i.e., \(0.496^{+0.228}_{-0.096}\) in \(\sin ^2\theta _{23}\) and \((2.32^{+0.43}_{-0.32})\times 10^{-3}~\mathrm {eV^2}\) in \(\varDelta m^2_{32}\). Note that three fits with fluctuations (WF :1, WF : 2 and WF : 3) from three independent fluctuated data sets are used in comparison, and each of them differ in the parameter sensitivities and the best fit values.

The fit to \(\sin ^2\theta _{23}\) for the data without fluctuation gives smaller uncertainty in the lower octant, and the fluctuations in the data leads to the fluctuations in the octant sensitivities (see Fig. 8a). Also the uncertainties in the parameter determination changes along with the significance of convergence, with each independent fluctuated pseudo-data set. The correlated precision is also observed to change with each fluctuated data set, where they show different coverages in \(\sin ^2\theta _{23}-\varDelta m^2_{32}\) plane.

Fig. 9
figure 9

Significance of convergence obtained from sixty different data sets. The solid (blue) and the broken (orange) line represents the significance for the fit to \(\sin ^2\theta _{23}\) and \(\varDelta m^2_{32}\) respectively and the dotted (black) line shows the significance obtained for the simultaneous fit to \(\sin ^2\theta _{23}\) and \(\varDelta m^2_{32}\)

The analysis was repeated for sixty different fluctuated data sets, performing separate (one parameter) and simultaneous (two parameter) fits to determine \(\sin ^2\theta _{23}\) and \(|\varDelta m^{2}_{32}|\). Figure 9 shows the significance of convergence in terms of standard deviation \(\sigma \). Almost \(68\%\) of times, the fit to \(\sin ^2\theta _{23}\) converges within \(1\sigma \) of the input value \(\sin ^2\theta _{23}=0.5\). A similar trend is observed in the fit to \(\varDelta m^2_{32}\), where \(59\%\) of times it converges within \(1\sigma \) of the input value \(\varDelta m^{2}_{32}=2.32\times 10^{-3}~\mathrm {eV^2}\). Also \(95\%\) of the time the fit converges within \(2\sigma \), which evidently shows the Gaussian nature of the fit. The simultaneous fit to \(\sin ^2\theta _{23}\) and \(\varDelta m^{2}_{32}\) also shows a similar behaviour in significance. Figure 10 shows the average coverage area with \(99\%\) CL in \(\sin ^2\theta _{23}-\varDelta m^2_{32}\) plane, obtained by averaging the coverages from simultaneous fit to fifty different pseudo-data sets. The orange band signifies the \(1\sigma \) uncertainty in calculating the average, where the asymmetrical widths from the best fit point of each data set were used. The precision reach for the fit without fluctuations is within \(1\sigma \) of the average coverage area calculated.

Previous studies [33, 47] have obtained a better precision in \(\sin ^2\theta _{23}\) and \(|\varDelta m^{2}_{32}|\), and have quantified the precision on these parameters as

$$\begin{aligned} \mathrm {precision}=\frac{P_{\mathrm {max}}-P_{\mathrm {min}}}{P_{\mathrm {max}}+P_{\mathrm {min}}}, \end{aligned}$$
(8)

where \(P_{\mathrm {max}}\) and \(P_{\mathrm {min}}\) are the maximum and minimum values of the concerned parameter determined at the given C.L. Figure 11 compares the \(\varDelta \chi ^2\) obtained from the previous [47] and the current analysis methods. It is to be noted that we have used the same binning scheme and the same set of NUANCE data in both the analysis methods for comparison, but the previous method incorporates the smearing of resolution functions whereas the current method incorporates event-by-event reconstruction. The precision in \(\sin ^2\theta _{23}\) for the previous method is \(19.4\%\) at \(1\sigma \), whereas for the current method it deteriorates to \(23.8\%\) for 5 year run of the experiment (see Fig 11(a)). The parameter \(|\varDelta m^2_{32}|\) also shows a similar behavior, where the precision deteriorates from \(5.9\%\) to \(12.9\%\) at \(1\sigma \) for the current method (see Fig 11(b)). The drop in precision for the current method is more predominant in \(|\varDelta m^2_{32}|\) and is clearly seen with a \(30\%\) difference in precision at \(3\sigma \).

The noted difference in precision is due to the realistic approach of the event-by-event reconstruction, where the tails of the resolution functions, which were approximated in the previous studies, have been included. In the previous methods, the NUANCE data was folded with the detector efficiencies and were smeared by the resolution functions obtained from GEANT-based studies of single muons with fixed energy and direction [38].

Fig. 10
figure 10

Comparison of unfluctuated precision reach and the average coverage area calculated from 99% CL coverages of 50 different sets, in \(\sin ^2\theta _{23}-\varDelta m^2_{32}\) plane. The dashed (black) and solid (blue) line shows the unfluctuated precision reach and average coverage area respectively with \(99\%\) CL. The orange band shows the \(1\sigma \) uncertainty in the average coverage and the dot (green) shows the input point

Fig. 11
figure 11

\(\varDelta \chi ^2\) as a function of a \(\sin ^2\theta _{23}\), for an input value of \(\sin ^2\theta _{23}(\hbox {true}) = 0.5\) and b \(\varDelta m^{2}_{32}\), for an input value of \(\varDelta m^{2}_{32}(\hbox {true})= 2.32\times 10^{-3}~\mathrm {eV^2}\). The dashed (orange) and solid (blue) line shows the fit obtained from the current and previous methods respectively. Note that the fits are for the same set of NUANCE data using the same binning scheme

4.2 Mass hierarchy determination

The 5 year pseudo-data set is oscillated via accept or reject method assuming NH (IH), which is then fit with true NH (IH) and false IH (NH) PDFs. The parameters in the fit are marginalized as given in Table 4, and the \(\varDelta \chi ^2\) resolution to differentiate the correct hierarchy from the wrong hierarchy is defined as:

$$\begin{aligned} \varDelta \chi ^2_{\mathrm {MH}}=\chi ^2_{\mathrm {false}}-\chi ^2_{\mathrm {true}}~, \end{aligned}$$
(9)

where \(\chi ^2_{\mathrm {true}}\) and \(\chi ^2_{\mathrm {false}}\) are the minimum values of \(\chi ^2\) from the true and false fits respectively. Figure 12a shows the true and false hierarchical fit to \(\sin ^2\theta _{23}\) for a particular pseudo-data set with fluctuations, wherein the resolution of \(\varDelta \chi ^2_{\mathrm {MH}}= 7.2\) rules out the wrong hierarchy with a significance greater than \(2\sigma \) for this set.

Fig. 12
figure 12

a \(\varDelta \chi ^2\) as a function of \(\sin ^2\theta _{23}\) for true and false fit and b distribution of \(\varDelta \chi ^2_{\mathrm {MH}}\) obtained from fit to sixty fluctuated data sets

The procedure was repeated for sixty independent 5-year fluctuated data sets to see the effect of fluctuations on the mass hierarchy significance. Figure 12b shows the distribution of \(\varDelta \chi ^2_{\mathrm {MH}}\) obtained from the fit to sixty different sets. The mean resolution of \(\varDelta \chi ^2_{\mathrm {MH}}=2.9\) rules out the wrong hierarchy with a significance of \(\approx 1.7 \sigma \) for a 5 year run of 50 kton ICAL detector. The large uncertainty in \(\varDelta \chi ^2_{\mathrm {MH}}\) is due to the fluctuations in the data and the negative values signifies the identification of the wrong mass hierarchy. Note that an earlier analysis that excluded the effect of fluctuations [47] gave a value \(\varDelta \chi ^2_{\mathrm {MH}}=2.7\) and our mean value is compatible with this, as expected, given minor differences in the analysis procedures. A 13 year exposure of ICAL will give a \(3\sigma \) separation to differentiate the correct MH. The classical statistical analysis used here is the same as that in the earlier works by the INO collaboration hence allowing a direct comparison. However, using a Bayesian approach [50] could improve the hierarchy sensitivity.

5 Discussion and summary

One of the main aims of the proposed ICAL at INO is to measure the atmospheric neutrino oscillation parameters \(\sin ^2\theta _{23}\) and \(|\varDelta m^{2}_{32}|\), and also to measure the mass hierarchy (MH) of neutrinos. The moderately large value of \(\theta _{13}\) and the ability of the magnetised ICAL to distinguish a neutrino event from an antineutrino event, allows the observation of earth matter effects separately in \(\nu \) and \({\bar{\nu }}\) and helps identify the MH of neutrinos. In this analysis we focus on the precision measurements and the mass hierarchy resolutions that ICAL could attain within a period of 5 years.

Incorporating a realistic analysis procedure, it is for the first time we have applied event-by-event reconstruction and have considered the tails of angular and energy resolution which were approximated by single Gaussians and Vavilov functions in previous such studies [33]. We show that incorporating non-Gaussian resolutions are likely to affect the parameter sensitivities, whereas reduction in precision by 4% and 7% is obtained for \(\sin ^2\theta _{23}\) and \(\varDelta m^2_{32}\) respectively at \(1\sigma \). Also for the first time we study the effect of low event statistics on the precision and MH measurements, by introducing fluctuations in the data. It is also for the first time within the framework of low event statistics, we show the effect of event selection criterion on the parameter sensitivities, and show that we can include all reconstructed muons to get better sensitivity of parameters. Hence within the framework of low event statistics, we show that the fit without any selection criterion (WOS), where we include all the reconstructed events, is the baseline to obtain better constraints on the parameters.

We start by using 5 year fluctuated pseudo-data set for our analysis and apply oscillations via the accept or reject method. The oscillated data is binned in \(E_{\mu }\) and \(\cos \theta _z\) for the \(\chi ^2\) analysis, where we have used the energy and direction information of the muon from the CC \(\nu _\mu \) events only. The constraints on \(\sin ^2\theta _{23}\) and \(\varDelta m^{2}_{32}\) are compared with and without event selection criterion. Statistically we lose \(40\%\) of events after selection, hence we find large uncertainty in parameter determination after applying event selection. We use an ensemble of independent fluctuated data sets to study the effect of low event statistics on the precision measurements of the oscillation parameters. The constraints on \(\sin ^2\theta _{23}\) and \(\varDelta m^{2}_{32}\) are compared with and without fluctuations, and we find a reasonable agreement between the unfluctuated and the average fluctuated precision reach obtained in \(\sin ^2\theta _{23}-\varDelta m^{2}_{32}\) plane.

As far as the mass hierarchy of the neutrinos is concerned, we find a mean resolution of \(\varDelta \chi ^2_{MH}=2.9\) from an ensemble of sixty experiments. This rules out the wrong hierarchy with a significance of \(\approx 1.7\sigma \), consistent with earlier analysis obtained without considering fluctuations. We also find a significant deviation in the mean value of \(\varDelta \chi ^2_{MH}\), and roughly a \(15\%\) probability of obtaining the wrong hierarchy due to the fluctuations in the data. In the near future, combining the \(\mathrm {CP}\) independent measurements at ICAL with the measurements from NO\(\nu \)A, T2K and other reactor experiments, will help to determine the correct MH [22, 51].

This paper presents an analysis procedure which can be used on the real ICAL data where the fluctuations are inbuilt as the PDFs are uncorrelated. However, in this analysis we have only used muon information from CC \(\nu _\mu \) events and we have ignored the small contribution from \(\nu _e\) to \(\nu _{\mu }\) oscillated events, that is known to slightly dilute the sensitivities [19]. The ICAL can also measure the hadron energy via proper calibration of hits, and including the hadron energy information in CC events is expected to improve the sensitivity of the detector towards the oscillation parameters and also improve the MH significance [44]. Note that there are CC \(\nu _e\) as well as neutral current (NC) events in the detector. Separation of \(\nu _{\mu }\) CC events from the others is quite robust for \(E_{\mu }\gtrsim 1 \mathrm {GeV}\) and has been discussed elsewhere [33]. Separation of low energy CC \(\nu _{\mu }\) events from CC \(\nu _e\) and NC events is an ongoing effort of the INO-ICAL collaboration. A combined analysis including all the CC events along with the hadron information will give us the maximum sensitivity the ICAL can attain, and is likely to improve the results presented in this paper.