Effect of non-unitary mixing on the mass hierarchy and CP violation determination at the Protvino to ORCA experiment

Abstract

In this paper, we have estimated the neutrino mass ordering and the CP violation sensitivity of the proposed Protvino to ORCA (P2O) experiment after 6 years of data-taking. Both unitary and non-unitary $3\times 3$ neutrino mass mixing have been considered in the simulations. A forecast analysis deriving possible future constraints on non-unitary parameters at P2O have been performed.

1 Introduction

Neutrino oscillation is parameterised by the mixing of three mass eigen states into three flavour states by the following mixing scheme:

$$\begin{aligned} \left[ {\begin{array}{c} \nu _e \\ \nu _\mu \\ \nu _\tau \end{array} } \right] = U \left[ {\begin{array}{c} \nu _1 \\ \nu _2 \\ \nu _3 \end{array} } \right] , \end{aligned}$$

(1)

where the vector on the left (right) hand side denotes the neutrino flavour (mass) eigen states. U is the $3\times 3$ unitary PMNS mixing matrix parameterised as

$$\begin{aligned}{} & {} U=\nonumber \\{} & {} \left[ \begin{array}{ccc} c_{13}c_{12} &{} s_{12}c_{13} &{} s_{13}e^{-i\delta _{\textrm{CP}}}\\ -s_{12}c_{23}-c_{12}s_{23}s_{13}e^{i\delta _{\textrm{CP}}} &{} c_{12}c_{23}-s_{12}s_{23}s_{13}e^{i\delta _{\textrm{CP}}} &{} s_{23}c_{13}\\ s_{12}s_{23}-s_{13}c_{12}c_{23}e^{i\delta _{\textrm{CP}}} &{} -c_{12}s_{23}-s_{13}c_{23}s_{12}e^{i\delta _{\textrm{CP}}} &{} c_{23}c_{13}\\ \end{array} \right] ,\nonumber \\ \end{aligned}$$

(2)

where $c_{ij}=\cos \theta _{ij}$ and $s_{ij}=\sin \theta _{ij}.$ Thus neutrino oscillation probabilities depend on three mixing angles ${\theta _{12}},$ ${\theta _{13}},$ and ${\theta _{23}},$ one CP violating phase $\delta _{\textrm{CP}},$ and two independent mass squared differences ${\Delta _{21}}=m_{2}^{2}-m_{1}^{2}$ and ${\Delta _{31}}=m_{3}^{2}-m_{1}^{2}.$ $m_i$ is the mass of the mass eigen state $\nu _i.$ Among these parameters, ${\theta _{12}}$ and ${\Delta _{21}}$ have been measured in the solar neutrino experiments [3, 4], $|{\Delta _{31}}|$ and $\sin ^2 2{\theta _{23}}$ have been measured in atmospheric neutrino experiments [5] and long base line accelerator neutrino experiments like MINOS [6]. The reactor neutrino experiments measured ${\theta _{13}}$ [7,8,9]. In Table 1, we have noted down the present global best-fit values of neutrino oscillation parameters. The current unknowns are the sign of ${\Delta _{31}},$ $\delta _{\textrm{CP}}$ and the octant of ${\theta _{23}}.$ Depending on the sign of ${\Delta _{31}},$ two scenarios are possible: a. Normal Hierarchy (NH): $m_3>>m_2>m_1$ and b. Inverted Hierarchy (IH): $m_2>m_1>>m_3.$ The long baseline accelerator neutrino experiments NO$\nu$A [10] and T2K [11] are currently taking data in both neutrino and anti-neutrino mode to observe both muon disappearances and electron appearances in the detector and are expected to measure the unknowns. However, their recent results [12, 13] show tension between the allowed regions in the $\sin ^2 {\theta _{23}}-\delta _{\textrm{CP}}$ plane, with NO$\nu$A excluding T2K best-fit point at $90\%$ confidence level and T2K excluding NO$\nu$A best-fit point at $3\, \sigma .$ If this trend continues with the future data, upcoming long baseline accelerator experiments like DUNE [14] and T2HK [15] will play vital role in resolving the tension.

Table 1 Global best-fit values of neutrino oscillation parameters [1, 2]. $\Delta _{3l}={\Delta _{31}}>0$ $(\Delta _{32}<0)$ for NH (IH)

Apart from the tension in the $\sin ^2 {\theta _{23}}-\delta _{\textrm{CP}}$ plane, the latest data from NO$\nu$A and T2K also fail to make any distinction between the two hierarchies [16]. Therefore, it’s really important for the future experiments to distinguish between the two hierarchies. Atmospheric neutrino experiments with multi-megaton ice/water Cherenkov detectors, like KM3NeT [17], can provide large statistics, however their mass hierarchy sensitivity is limited by the uncertainties in the flux. Careful analysis to minimise these uncertainties using dedicated detector specific simulations have been done and it has been shown that it is possible to achieve $\simeq 3\, \sigma$ hierarchy sensitivity after 3–4 years of data taking [18]. However the CP sensitivity is rather low unless future upgrade is possible [19].

Combining multi megaton water/ice Cherenkov detector with accelerator neutrino source over a very long baseline to explore the hierarchy and CP sensitivity has been considered in literature [20,21,22,23,24,25]. In Ref. [26], the hierarchy and CP sensitivity of a possible future experiment with beam from Fermilab towards the ORCA detector of KM3NeT collaboration was discussed. Reference [27] discussed the physics potential of an experiment with beam from Protvino to Orca (P2O). In 2019, the letter of intent (LOI) of the P2O experiment came out [28]. The fluxes used in the LOI was less optimistic than the ones used in Ref. [27]. Therefore it is important to explore the physics potential of the P2O with the updated beam.

Apart from measuring the standard oscillation parameter, future long baseline accelerator neutrino experiments will explore beyond Standard Model (BSM) physics [29]. Recently, there have been studies where BSM physics has been used to resolve the tension between the NO$\nu$A and T2K. In Refs. [30, 31] non-standard neutral current interaction during propagation has been used to resolve this tension, while in Ref. [32] same has been done with CPT violating Lorentz invariance violation. Reference [33] tried to probe non-unitary mixing with the recent NO$\nu$A and T2K data. They found out that the tension between the two experiments can be reduced with non-unitarity and that both the experiments prefer non-unitary mixing. They also showed that the signature of non-unitarity in these two experiments have grown stronger over the years. However, a recent paper [34] showed that the short baseline experiments NOMAD and NuTeV prefer unitary mixing and put strong constraints on non-unitary parameters. This means that the short baseline experiments data have a tension with the long baseline data from the NO$\nu$A and T2K. Thus it is important to explore the effects of non-unitary mixing with different baselines and energy.

Earlier, effects of non-unitary parameters on the determination of unknown neutrino oscillation parameters in long baseline experiments like NO$\nu$A, T2K, DUNE, T2HK and TNT2K have been discussed in Refs. [35,36,37,38,39]. Reference [40] explored the role of neutral current measurements in measuring the bounds on non-unitary parameters. In this paper, we study the effect of non-unitary mixing on the determination of hierarchy and CP violation in P2O experiment. In Sect. 2, we discuss the detail of the P2O experiment. Neutrino oscillation probabilities with unitary and non-unitary mixing are discussed in Sect. 3. The physics sensitivities to determine hierarchy both with unitary and non-uniray oscillation are talked about in Sect. 4. Section 5 talks about the physics potential to establish CP violation in the P2O experiment with unitary and non-unitary mixing schemes. The future constraints on non-unitary parameters from P2O experiment are discussed in Sect. 6. The conclusions are drawn in Sect. 7.

Fig. 1

Energy distribution of the expected number of beam neutrino events that would be detected by ORCA after 3 years of running for 4 different values of the CP phase for the case of normal (left panel) and inverted (right panel) neutrino mass ordering. $\theta _{23}=45^{\circ }$ is assumed. The x-axis shows the true neutrino energy

2 Experimental details

ORCA (Oscillation Research with Cosmics in the Abyss) is the low energy component of the KM3NeT Consortium [17], housing two detectors in the Mediterranean sea. It is located about 40 km off the coast of Toulon, France, at a depth of $\sim 2450$ m. Upon completion ORCA will house 115 strings with an effective volume of $\sim 5$ MT. The water Cherenkov detector is optimized for the study of atmospheric neutrino oscillations in the energy range of 3–100 GeV with the primary goal to determine the neutrino mass ordering. A $>2\,\sigma$ sensitivity to mass hierarchy is expected after 3 years of full-ORCA data taking [41]. ORCA will provide a complimentary measurement of $\Delta _{32}$ and $\theta _{23},$ measure the $\nu _\tau$ flux normalisation, and probe a variety of new physics scenarios namely, sterile neutrinos, non-standard interactions, neutrino decay and so on. Currently, 18 ORCA lines are live and taking data since an year.

The Protvino accelerator complex, situated at 100 km south of Moscow, hosts a U-70 accelerator. The baseline between Protvino and ORCA is roughly 2595 km. A beam power of 90 kW, corresponding to $0.8\times 10^{20}$ protons on target (POT) per year [28] is expected. However, there is a proposal to upgrade the beam power to 450 kW [28].

Depending on the Cherenkov signatures of the outgoing lepton from the $\nu _{e}$ and $\nu _{\mu }$ CC and NC interactions, two distinct event topologies are observed at the detector: track-like and shower-like events. $\nu _{\mu }$ CC and those $\nu _{\tau }$ CC interactions with muonic $\tau$ decays mostly account for track-like topology. The shower-like topology has events from $\nu _{e}$ CC, $\nu _{\tau }$ CC interactions with non-muonic $\tau$ decays and NC interactions of all flavours. From fig. 1 of Ref. [41], we derived the energy dependent detector efficiency of each interaction by calculating the ratio of effective volume to total volume. In Ref. [27], the analyses have been done assuming an optimistic beam power of 450 kW. In this work, we have taken a conservative approach followed in the P2O LOI [28]. To do so, we have first matched the expected event numbers with the LOI [28]. To calculate event numbers, the GLoBES [42, 43] simulation package has been used. For energy smearing, GLoBES incorporates a Gaussian resolution function:

$$\begin{aligned} R^c (E,E^\prime )=\frac{1}{\sqrt{2\pi }}e^{-\frac{(E-E^\prime )^2}{2\sigma ^2(E)}}, \end{aligned}$$

(3)

where E is the true neutrino energy and $E^\prime$ is the reconstructed neutrino energy. The energy resolution is parameterised by:

$$\begin{aligned} \sigma (E)=\alpha E+\beta \sqrt{E}+\gamma . \end{aligned}$$

(4)

Values of $\alpha =0.25$ (0.35),  $\beta =0,$ and $\gamma =0$ for shower (track) events are used in this work [41].

For the disappearance channels, the signal comes from $\nu _\mu$ CC events, and the backgrounds come from NC and tau events. For appearance channels, the signal comes from $\nu _e$ CC events and the backgrounds come from $\nu _\tau$ CC events, NC events and also the mis-identification. In our analysis, we have modified GLoBES to match the expected event rates given in Ref. [28]. The energy distributions of the expected neutrino events across various channels after 3 years of running in the neutrino mode (corresponding to $2.4\times 10^{20}$ POT) are shown in Fig. 1.

After matching with the event rates of Ref. [28], the energy dependent particle mis-identification factors were calculated from fig. 6 of Ref. [41]. At the end we included the correction due to atmospheric muon background contamination. From fig. 4 of Ref. [41], we can see that for $10\%$ atmospheric muon background contamination, $98\%$ of all neutrinos will be identified correctly. We have performed our analysis in this paper with $10\%$ atmospheric muon background contamination.

3 Oscillation probabilities with unitary and non-unitary mixing

The standard unitary neutrino oscillation probability with matter effect for uniform matter density can be written as [44]

$$\begin{aligned} P_{\mu e}\simeq & {} \sin ^2 2 {\theta _{13}}\sin ^2 {\theta _{23}}\frac{\sin ^2\hat{\Delta }(1-{\hat{A}})}{(1-{\hat{A}})^2}\nonumber \\{} & {} + \alpha \cos {\theta _{13}}\sin 2{\theta _{12}}\sin 2{\theta _{13}}\sin 2{\theta _{23}}\cos (\hat{\Delta }+\delta _{\textrm{CP}})\nonumber \\{} & {} \times \frac{\sin \hat{\Delta }{\hat{A}}}{{\hat{A}}} \frac{\sin \hat{\Delta }(1-{\hat{A}})}{1-{\hat{A}}}, \end{aligned}$$

(5)

where $\alpha =\frac{{\Delta _{21}}}{{\Delta _{31}}},$ $\hat{\Delta }=\frac{{\Delta _{31}}L}{4E}$ and ${\hat{A}}=\frac{A}{{\Delta _{31}}}.$ A is the Wolfenstein matter term [45], given by $A=2\sqrt{2}G_FN_eE,$ where E is the neutrino beam energy and L is the length of the baseline. Anti-neutrino oscillation probability $P_{\bar{\mu } {\bar{e}}}$ can be obtained by changing the sign of A and $\delta _{\textrm{CP}}$ in Eq. 5. The oscillation probability mainly depends on hierarchy (sign of ${\Delta _{31}}),$ octant of ${\theta _{23}}$ and $\delta _{\textrm{CP}},$ and precision in the value of ${\theta _{13}}.$ This dependency leads to eight fold degeneracy [46,47,48,49,50,51,52,53,54,55,56]. Since the Reactor neutrino experiments [7,8,9] have measured ${\theta _{13}}$ precisely and thus broke the eight fold degeneracy into $1+3+3+1$ pattern [57, 58].

The hierarchy-$\delta _{\textrm{CP}}$ degeneracy is important for experiments like NO$\nu$A and T2K, making it impossible to determine the hierarchy when NH (IH) and $0<\delta _{\textrm{CP}}<180^\circ$ $(-180^\circ<\delta _{\textrm{CP}}<0)$ is the true hierarchy-$\delta _{\textrm{CP}}$ combination chosen by nature. We call $0<\delta _{\textrm{CP}}<180^\circ$ upper half plane (UHP), and $-180^\circ<\delta _{\textrm{CP}}<0$ lower half plane (LHP) of $\delta _{\textrm{CP}}.$ In Eq. 5, NH (IH) enhances (suppresses) $P_{\mu e},$ while $\delta _{\textrm{CP}}$ in UHP (LHP) suppresses (enhances) $P_{\mu e}.$ Thus the value of probabilities for NH and $\delta _{\textrm{CP}}$ in UHP, and IH and $\delta _{\textrm{CP}}$ in LHP are close to each other, making these two combinations as the unfavourable hierarchy-$\delta _{\textrm{CP}}$ combinations. On the other hand, for the combinations NH-$\delta _{\textrm{CP}}$ in LHP, and IH-$\delta _{\textrm{CP}}$ in UHP, the values of $P_{\mu e}$ are well separated. These two are the favourable hierarchy-$\delta _{\textrm{CP}}$ combinations. Hierarch-$\delta _{\textrm{CP}}$ degeneracy exists for $P_{\bar{\mu } {\bar{e}}}$ as well. However, this degeneracy can be broken in the case of bi-magic baseline 2540 km, as shown in Ref. [59]. The P2O baseline 2595 km is very close to the bi-magic baseline, and hence the special effects enhancing the hierarchy sensitivity of bi-magic baseline are also present in the case of P2O as well. In Fig. 2, we have shown the oscillation probabilities $P_{\mu e}$ and $P_{\bar{\mu } {\bar{e}}}$ as a function of energy for P2O and for different hierarchy and $\delta _{\textrm{CP}}$ values as mentioned in the plot. The other oscillation parameters have been fixed at values mentioned in Table 1. We have only plotted probabilities pertaining to $\delta _{\textrm{CP}}$ values $-90^\circ$ and $90^\circ .$ All other $\delta _{\textrm{CP}}$ values fall in between. It can be easily noticed that the hierarchy-$\delta _{\textrm{CP}}$ degeneracy gets broken and hierarchy can be determined even for the unfavourable hierarchy-$\delta _{\textrm{CP}}$ combinations. However, for $P_{\mu e},$ the separation between NH-$\delta _{\textrm{CP}}=+90^\circ ,$ and IH-$\delta _{\textrm{CP}}=-90^\circ$ is least compared to other hierarchy-$\delta _{\textrm{CP}}$ combinations. But for $P_{\bar{\mu } {\bar{e}}},$ the separation between these two hierarchy-$\delta _{\textrm{CP}}$ combinations is maximum. Thus, in case of P2O addition have anti-neutrino run will have better hierarchy sensitivity than only neutrino run for NH-$\delta _{\textrm{CP}}$ in UHP, and IH-$\delta _{\textrm{CP}}$ in LHP.

Fig. 2

(Anti-)neutrino appearance probability as a function of energy in the upper (lower) panel assuming NH and IH with standard unitary oscillation parameters given in Table 1 for $\delta _{cp}=-90^{\circ }$ and $\delta _{cp}=+90^{\circ }$

Another degeneracy affecting the hierarchy sensitivity of the long baseline accelerator neutrino experiments is the octant-hierarchy sensitivity [58, 60]. As explained in Refs. [58, 60], $P_{\mu e}$ gets a positive (negative) boost due to both hierarchy and octant when the hierarchy is NH (IH) and the octant of ${\theta _{23}}$ is HO (LO). Therefore, HO-NH and LO-IH are well separated from each other and they are the favourable octant-hierarchy combinations. When the hierarchy is NH (IH), and ${\theta _{23}}$ is in LO (HO), $P_{\mu e}$ gets a positive (negative) boost due to hierarchy and negative (positive) boost due to octant. Therefore LO-NH and HO-IH are two unfavourable octant-hierarchy combinations in the long baseline experiments like NO$\nu$A and T2K. This octant-hierarchy degeneracy can be broken down by anti-neutrino run. However, as we can see from Fig. 3 that the in case of P2O, octant-hierarchy degeneracy is not present even for the neutrino run. This is also because of the magic-baseline property of P2O. But even then, anti-neutrino run provides better separation between the unfavourable octant-hierarchy combinations. To generate the probability plots, we considered $\sin ^2{\theta _{23}}=0.43$ (0.57) for LO (HO).

Fig. 3

$\nu _\mu \rightarrow \nu _e$ probability plots for LO-NH, and HO-IH and for different $\delta _{\textrm{CP}}$ values. The upper (lower) panel is for neutrino (anti-neutrino) probability. We considered $\sin ^2{\theta _{23}}=0.43$ (0.57) for LO (HO)

In the case of non-unitary mixing, the mixing matrix is defined as [61]

$$\begin{aligned} N=N_{NP}U_{3\times 3}= \left[ {\begin{array}{ccc} \alpha _{00} &{} 0 &{} 0 \\ \alpha _{10} &{} \alpha _{11} &{} 0 \\ \alpha _{20} &{} \alpha _{21} &{} \alpha _{22} \end{array} } \right] U_{\textrm{PMNS}}. \end{aligned}$$

(6)

In Eq. 6, the diagonal elements are real, and the off-diagonal elements are complex with the form $\alpha _{ij}=|\alpha _{ij}|e^{i\phi _{ij}},$ where $i,j=0,1,2$ and $i>j.$ The mixing between flavour states and mass eigenstates can be written as

$$\begin{aligned} |\nu _{\beta }\rangle =\sum _{i=1}^{3}N_{\beta i}|\nu _i\rangle , \end{aligned}$$

(7)

where $\beta$ denotes the flavour states and i denotes the mass eigenstates. The evolution of neutrino mass eigenstates during the propagation, can be written as

$$\begin{aligned} i\frac{d}{dt}|\nu _i\rangle =H_{\textrm{vac}}|\nu _i\rangle , \end{aligned}$$

(8)

where $H_{\textrm{vac}}$ is the Hamiltonian in vacuum and it is defined as

$$\begin{aligned} H_{\textrm{vac}}=\frac{1}{2E}\left[ {\begin{array}{ccc} 0 &{} 0 &{} 0 \\ 0 &{} {\Delta _{21}}&{} 0 \\ 0 &{} 0 &{} {\Delta _{31}}\end{array} } \right] . \end{aligned}$$

(9)

The non-unitary neutrino oscillation probability in vacuum can be written as

$$\begin{aligned} P_{\mu e}^{{\textrm{NU}}}({\textrm{vac}})= & {} \sum _{i,j}^{3}N_{\mu i}^{*}N_{ei}N_{\mu j}N_{ej}^{*}\nonumber \\{} & {} -4\sum _{j>i}^{3}Re\left[ N_{\mu j}N_{ej}N_{\mu i}N_{ei}^{*}\right] \sin ^2 \left( \frac{\Delta _{ji}L}{4E}\right) \nonumber \\{} & {} +2\sum _{j>i}^{3}Im\left[ N_{\mu j}^{*}N_{ej}N_{\mu i}N_{ei}\right] \sin \left( \frac{\Delta _{ji}L}{2E}\right) . \nonumber \\ \end{aligned}$$

(10)

Equation 10 can be obtained by replacing $U_{\textrm{PMNS}}$ in the standard probability formula with N. However this change from $U_{\textrm{PMNS}}$ to N modify the oscillation probability completely. If written explicitly neglecting the cubic products of $\alpha _{10},$ $\sin {\theta _{13}},$ and ${\Delta _{21}},$ the oscillation probability takes the form [61]

$$\begin{aligned} P_{\mu e}^{{\textrm{NU}}}({\textrm{vac}})=(\alpha _{00}\alpha _{11})^2P_{\mu e}^{SO}+ \alpha _{00}^2\alpha _{11}|\alpha _{10}|P_{\mu e}^I+\alpha _{00}^2|\alpha _{10}|^2.\nonumber \\ \end{aligned}$$

(11)

$P_{\mu e}^{SO}$ is the standard three flavour unitary neutrino oscillation probability in vacuum and can be written as

$$\begin{aligned} P_{\mu e}^{SO}= & {} 4\cos ^2 {\theta _{12}}\cos ^2 {\theta _{23}}\sin ^2 {\theta _{12}}\sin ^2\left( \frac{{\Delta _{21}}L}{4E}\right) \nonumber \\{} & {} + 4\cos ^2 {\theta _{13}}\sin ^2 {\theta _{13}}\sin ^2 {\theta _{23}}\sin ^2\left( \frac{{\Delta _{31}}L}{4E}\right) \nonumber \\{} & {} + \sin (2{\theta _{12}})\sin {\theta _{13}}\sin (2{\theta _{23}})\sin \left( \frac{{\Delta _{21}}L}{2E}\right) \nonumber \\{} & {} \times \sin \left( \frac{{\Delta _{31}}L}{4E}\right) \cos \left( \frac{{\Delta _{31}}L}{4E}-I_{123}\right) , \end{aligned}$$

(12)

and

$$\begin{aligned} P_{\mu e}^I= & {} -2\left[ \sin (2{\theta _{13}})\sin {\theta _{23}}\sin \left( \frac{{\Delta _{31}}L}{4E}\right) \right. \nonumber \\{} & {} \left. \quad \sin \left( \frac{{\Delta _{31}}L}{4E}+I_{NP}-I_{012}\right) \right] \nonumber \\{} & {} \quad -\cos {\theta _{13}}\cos {\theta _{23}}\sin (2{\theta _{12}})\sin \left( \frac{{\Delta _{21}}L}{2E}\right) \sin (I_{NP}),\nonumber \\ \end{aligned}$$

(13)

where $I_{012}=\phi _{10}-\phi _{20}+\phi _{21},$ and $I_{NP}=\phi _{10}-{\textrm{Arg}}(\alpha _{10}).$ $\phi _{ij}$s are the phases associated with $\alpha _{ij}=|\alpha _{ij}|e^{i \phi _{ij}}.$ It should be noted that non-unitary parameters $\alpha _{00},$ $\alpha _{11},$ and $\alpha _{10}$ have most significant effects on $P_{\mu e}^{{\textrm{NU}}}({\textrm{vac}}).$

Fig. 4

Neutrino and anti-neutrino appearance (top row) and disappearance (bottom row) probability plots assuming NH with standard oscillation parameters and with non-unitary parameters as given in Table 2 for $\phi _{10}=0^{\circ }$ for different true values of $\delta _{cp}$

While, propagating through matters, the neutrinos undergo forward scattering and the neutrino oscillation probability gets modified due to interaction potential between neutrino and matter. In case of non-unitary mixing, the CC and NC interaction Lagrangian is given as

$$\begin{aligned} {\mathcal {L}}=V_{\textrm{CC}}\sum _{i,j}N_{ei}^{*}N_{ej}\bar{\nu }_i\gamma ^0\nu _j+V_{\textrm{NC}}\sum _{\alpha ,i,j}N_{\alpha i}^{*}N_{\alpha j}\bar{\nu }_i\gamma ^0\nu _j,\nonumber \\ \end{aligned}$$

(14)

where $V_{\textrm{CC}}=\sqrt{2}G_FN_e,$ and $V_{\textrm{NC}}=-G_F N_n/\sqrt{2}$ are the potentials for CC and NC interactions respectively. Therefore, the effective Hamiltonian becomes

$$\begin{aligned} H_{\textrm{matter}}^{{\textrm{NU}}}= & {} \frac{1}{2E}\left[ {\begin{array}{ccc} 0 &{} 0 &{} 0 \\ 0 &{} {\Delta _{21}}&{} 0 \\ 0 &{} 0 &{} {\Delta _{31}}\end{array} } \right] \nonumber \\{} & {} +N^\dagger \frac{1}{2E}\left[ {\begin{array}{ccc} V_{\textrm{CC}}+V_{\textrm{NC}} &{} 0 &{} 0 \\ 0 &{} V_{\textrm{NC}} &{} 0 \\ 0 &{} 0 &{} V_{\textrm{NC}} \end{array} } \right] N. \end{aligned}$$

(15)

The non-unitary neutrino oscillation probability after neutrinos travel through a distance L is given as

$$\begin{aligned} P_{\alpha \beta } (E,L)= |\langle \nu _\beta |\nu _\alpha (L)\rangle |^2=\left| \left( Ne^{-iH_{\textrm{matter}}^{\textrm{NU}}L}N^\dagger \right) _{\beta \alpha }\right| ^2.\nonumber \\ \end{aligned}$$

(16)

A detailed description of non-unitary neutrino oscillation probability in presence of matter effect has been discussed in Refs. [33, 62,63,64].

Fig. 5

Same as Fig. 4, except for IH

Probability plots for P2O with the standard and with non-unitary oscillations have been plotted for [$\phi _{10}$ = 0, $+90$ and $-90$ degree] for different values of $\delta _{cP}$ under Normal hierarchy and Inverted hierarchy for neutrinos and anti-neutrinos separately. Standard oscillation parameter values used for probability plots are given in Table 1. $\delta _{\textrm{CP}}$ values of $-180^\circ ,\, -90^\circ ,\, 0,\, 90^\circ ,\, 180^\circ$ have been used. The non-unitary parameter values have been taken to be the $3\,\sigma$ bound obtained from global analyses of neutrino oscillation for 6 degrees of freedom [65]. Please note that apart from $\alpha _{00},$ $\alpha _{10},$ and $\alpha _{11},$ all other non-unitary parameters have been fixed at their unitary values. This is because $\alpha _{00},$ $\alpha _{10},$ and $\alpha _{11}$ have maximum effects on the oscillation probability, and in this paper, we will concentrate on the effects of these three parameters on the physics potential of P2O. We have used the software GLoBES [42, 43] to calculate both unitary and non-unitary oscillation probabilities. For non-unitary oscillation probabilities, GLoBES has been modified aptly to include non-unitary mixing. Figures 4, 5 and  6 shows the neutrino and anti neutrino probability plots corresponding to $\phi _{10}=0,\, +90^\circ \ and\ -90^\circ$ respectively.

Fig. 6

Neutrino and anti neutrino appearance (top row) and disappearance (bottom row) probability plots assuming NH with standard oscillation parameters and with non-unitary parameters as given in Table 2 for $\phi _{10}=90^{\circ }$ for different true values of $\delta _{cp}$

Fig. 7

Same as Fig. 6, except for IH

Fig. 8

Neutrino and anti neutrino appearance (top row) and disappearance (bottom row) probability plots assuming NH with standard oscillation parameters and with non-unitary parameters as given in Table 2 for $\phi _{10}=-90^{\circ }$ for different true values of $\delta _{cp}$

Fig. 9

Same as Fig. 8, except for IH

From Figs. 4 and 5, it is evident that when $\phi _{10}=0,$ P2O has good discrimination potential between unitary and non-unitary mixing at the peak energy in both appearance and disappearance channels, for both neutrino and anti-neutrino mode and for all the values of $\delta _{\textrm{CP}}$ and both the hierarchies. When $\phi _{10}=90^\circ$ $(-90^\circ ),$ P2O cannot discriminate between unitary and non-unitary mixing for $\delta _{\textrm{CP}}=-90^\circ$ $(90^\circ )$ in the appearance channel for both neutrino and anti-neutrino modes, when NH is the true hierarchy. However, for all other $\delta _{\textrm{CP}}$ values, the discrimination potential between the two mixing schemes is quite good for both the channels, modes and hierarchies. The oscillation probabilities for $\delta _{\textrm{CP}}=-180^\circ ,$ and $180^\circ$ are exactly same even in the case of non-unitary mixing. These features have been illustrated in Figs. 6, 7, 8 and 9.

4 Mass hierarchy sensitivity

In this section, we will discuss the mass hierarchy sensitivity of the P2O experiment for different neutrino and anti-neutrino run times. From now on, x years of neutrino run and y years of anti-neutrino run will be defined as $x+y$ years.

4.1 Mass hierarchy sensitivity with standard unitary oscillation

First we have considered that NH is our true hierarchy and IH is test hierarchy. For the true data set, values of $\delta _{cp}$ has been varied in the range $[-180^\circ :180^\circ ].$ Other standard unitary oscillation parameter values have been fixed at the values given in Refs. [1, 2]. These values have been mentioned in Table 1. We have assumed unitary mixing for both true and test event rates. In the test data set ${\Delta _{31}},$ and $\sin ^2\theta _{23}$ have been varied in their possible $3\, \sigma$ range given in Refs. [1, 2], $\delta _{\textrm{CP}}$ has been varied in its complete range $[-180^\circ :180^\circ ].$ The $\chi ^2$ between true and test event rates have been calculated using GLoBES [42, 43]. Automatic bin based energy smearing for generated test events has been implemented in the same way as described in the GLoBES manual [42, 43] (Table 2).

We used $5\%$ normalisation and $3\%$ energy calibration systematics uncertainty for e and $\mu$ like events [28]. Implementing systematics uncertainty has been discussed in details in GLoBES manual [42, 43]. The details of priors on standard oscillation parameters and systematic uncertainties have been presented in Table 3.

Table 2 The non-unitary parameter values used for probability plots. These are the $3\,\sigma$ bound obtained from global analyses of neutrino oscillation for 6 degrees of freedom [65]

Table 3 Systematic uncertainties and priors on the standard oscillation parameters used in the analysis

For simulations, $\chi ^2$ and $\Delta \chi ^2$ are always equivalent. $\chi ^{2}$ has been calculated as a function of true values of $\delta _{cp}$ by marginalising it over the test parameter values. The same procedure has been repeated with the considerations that IH is true and NH is test hierarchy. In this study, both disappearance and appearance channels have been considered and combined $\chi ^{2}$ for $\nu$ and $\bar{\nu }$ has been shown. MH sensitivity plots have been generated assuming $10\%$ atmospheric muon contamination in P2O detector and are shown in Fig. 10.

From Fig. 10, it is obvious that for $10\%$ atmospheric muon contamination, changes in neutrino and anti-neutrino run time for a total run of 6 years have significant effect on hierarchy sensitivity when NH is the true hierarchy and $6+0$ years run has the maximum hierarchy sensitivity. For IH, however the sensitivity changes by very small amount due to the change in runtime, and $3+3$ years run have the maximum hierarchy sensitivity. This is because the true value of ${\theta _{23}}$ is in HO, and as we have already shown in Sect. 3 with Fig. 3 that the octant-hierarchy combination HO-IH has the maximum separation from another combination LO-NH for $P_{\bar{\mu } {\bar{e}}}.$ Hence, addition of anti-neutrino run improves the hierarchy sensitivity for IH. Another important feature is that for NH $\delta _{\textrm{CP}}=-90^\circ$ has the maximum hierarchy sensitivity for NH, where as $\delta _{\textrm{CP}}=90^\circ$ has the same for IH. This can be explained with the help of Fig. 2, as we can see from there that for both neutrino and anti-neutrino, the maximum separation between two hierarchies at the probability level come from NH-$\delta _{\textrm{CP}}=-90^\circ$ and IH-$\delta _{\textrm{CP}}=90^\circ .$ It can also be noted that wrong hierarchy can be ruled out for all the $\delta _{\textrm{CP}}$ values and all the run times at more than $9\, \sigma$ $(8\, \sigma )$ confidence level (C.L.), when NH (IH) is the true hierarchy. For NH (IH) maximum hierarchy sensitivity of $11\, \sigma$ $(9\, \sigma )$ can be reached for $\delta _{\textrm{CP}}=-90^{\circ }$ $(90^{\circ }).$

Fig. 10

MH sensitivity as a function of true values of $\delta _{cp}$ assuming $10\%$ atmospheric muon contamination and for different run times for 6 years. True hierarchy has been assumed to be normal (inverted) for the upper (lower) panel

4.2 Mass hierarchy sensitivity with non-unitary oscillation

In this subsection, the true event rates have been generated with unitary mixing as before. However, here we have assumed non-unitary mixing to calculate the test event rates. We have considered the effect of one non-unitary parameter at a time. For test values of non-unitary parameters, $\alpha _{00}$ and $\alpha _{11}$ have been varied in the range [0.8 : 1.0],  $\alpha _{10}$ has been varied in the range [0 : 1.0] and $\phi _{10}$ has been kept fixed at 0. As before, test values of standard oscillation parameters $\sin ^2\theta _{23},$ $|{\Delta _{31}}|$ have been varied within their 3$\sigma$ range and rest of the other oscillation parameters are set at their standard values. Priors on $\theta _{23},$ $|{\Delta _{31}}|$ have been applied at final $\chi ^{2}$ calculation. We have assumed only $10\%$ atmospheric muon contamination. We have plotted $\sigma$ =$\sqrt{(}\chi ^{2})$ for sensitivity estimation. We have also shown the effect of $\alpha _{00},$ $\alpha _{11},$ $\alpha _{10}$ variation on MH sensitivity for various run time years. The effects of individual non-unitary parameters have been shown for different run times in Fig. 11 (12) when NH (IH) is the true hierarchy.

Fig. 11

Effect of non unitary parameters on MH sensitivity for (P2O) as a function of true values of $\delta _{cp}$ assuming $10\%$ atmospheric muon contamination for various run time years keeping NH as the true and IH as the test hierarchy

From Fig. 11, we can see that when NH is the true hierarchy, in case of $\alpha _{00},$ and $\alpha _{11},$ mass hierarchy can be established in P2O at more than $7\,\sigma$ C.L. after 6 years. For these two parameters, neutrino run has better hierarchy sensitivity than anti-neutrino run. With 6 years of only neutrino-run, hierarchy can be established at $9\,\sigma$ $(11\,\sigma )$ for $\delta _{\textrm{CP}}=90^\circ$ $(-90^\circ ).$ However, for $|\alpha _{10}|,$ mass hierarchy can be established at only $4\,\sigma$ C.L. after 6 years of only neutrino run. Anti-neutrino run can improve the sensitivity to $8\,\sigma .$

Fig. 12

Effect of non unitary parameters on MH sensitivity for P2O as a function of true values of $\delta _{cp}$ assuming $10\%$ atmospheric muon contamination for various run time years keeping IH as the true and NH as the test hierarchy

In case of IH being the true hierarchy (Fig. 12), in case of $\alpha _{00},$ hierarchy can be determined at $5\, \sigma ,$ after 6 years of only neutrino run. Anti-neutrino run can improve the sensitivity up to $6-7\,\sigma .$ However, for $\alpha _{11},$ the hierarchy can be established at $\sim 7\,\sigma$ after 6 years of neutrino, and anti-neutrino run can improve it up to $8\,\sigma .$ In case of $|\alpha _{10}|,$ after 6 years of neutrino run, hierarchy can be established at $2\,\sigma$ C.L. Anti-neutrino run can improve the sensitivity up to $6\,\sigma .$

5 Establishing CP violation

In this section, we discuss about the CP violation sensitivity, i.e. the sensitivity to rule out CP conserving $\delta _{\textrm{CP}}$ values, of P2O, when the data are analysed with both unitary and non-unitary mixing hypothesis. To do so, the true $\delta _{\textrm{CP}}$ values have been varied in the complete range. Other standard unitary oscillation parameter values have been fixed at the values given in Table 2.

5.1 CP violation sensitivity with standard unitary oscillations

Fig. 13

CPV sensitivity as a function of true values of $\delta _{cp}$ assuming $10\%$ atmospheric muon background contamination

Fig. 14

CP violation sensitivity including non unitary parameters for (P2O)as a function of true values of $\delta _{cp}$ assuming $10\%$ atmospheric muon contamination for various run time years keeping NH is true hierarchy

Fig. 15

CP violation sensitivity including non unitary parameters for (P2O)as a function of true values of $\delta _{cp}$ assuming $10\%$ muon misidentification for various run time years keeping IH is true hierarchy

At first, the test event rates and the $\chi ^2$ between true and test event rates have been calculated assuming unitary mixing. The test $\delta _{\textrm{CP}}$ values are 0,  and $180^\circ .$ Other test parameter values of the unitary oscillation parameters have been varied in the range described in Sect. 4.1. Both NH and IH have been considered as test hierarchy. The $\chi ^2$ values have been marginalised over test parameters and in Fig. 13, the CP violation sensitivity has been shown as a function of true $\delta _{\textrm{CP}}$ values for $10\%$ atmospheric muon contamination. For NH true, the maximum CP violation effect can be seen for $6+0$ years run. However, for IH true, increasing anti-neutrino run improves the CP violation sensitivity with maximum sensitivity achieved at 3 years of neutrino and anti-neutrino run each. For NH (IH), Maximum CP violation sensitivity of $1.5\, \sigma$ $(1\, \sigma )$can be achieved for $\delta _{\textrm{CP}}=\pm 90^\circ .$

Fig. 16

$\sigma =\sqrt{\Delta \chi ^2}$ as a function of individual non-unitary parameters for NH being the true and test hierarchies and for different run times

5.2 CP violation sensitivity with non-unitary oscillation

Now the test event rates have been calculated assuming non-unitary $3\times 3$ neutrino mixing. Test $\delta _{\textrm{CP}}$ values are $-180^\circ ,$ 0,  and $180^\circ .$ Other standard oscillation parameter ranges are same as described in Sect. 4.1. We have observed the effect of one non-unitary parameter at a time on the CP violation sensitivity. The non-unitary parameters have been varied in the range described in Sect. 4.2. Both NH and IH have been considered as the true hierarchy. The $\chi ^2$ values have been marginalised over test parameter values. Effects of individual non-unitary parameters on the CP violation sensitivity has been shown in Figs. 14 and 15 for NH and IH being the true hierarchy respectively. Muon mis-identification factor is $10\%.$

From Fig. 14, we can see that anti-neutrino run worsens CP violation sensitivity at P2O in the case of non-unitary mixing when NH is the true hierarchy. In cases of $\alpha _{11},$ and $|\alpha _{10}|,$ CP violation can be established at more than $1.5\, \sigma$ C.L. for true $\delta _{\textrm{CP}}=\pm 90^\circ$ after 6 years of neutrino run.For $|\alpha _{10}|,$ CP violation can be established at more than $1.5\, \sigma$ $(0.7\,\sigma )$ C.L. for true $\delta _{\textrm{CP}}=-90^\circ$ $(90^\circ )$ after 6 years of neutrino run.

In case of IH being the true hierarchy, CP violation can be established only at $0.7\, \sigma$ for $\alpha _{00},$ $|\alpha _{10}|,$ and $\alpha _{11}$ after 6 years of neutrino run when $\delta _{\textrm{CP}}=\pm 90^\circ .$ Anti-neutrino run cam improve this sensitivity up to $1\,\sigma$ at $\delta _{\textrm{CP}}=\pm 90^\circ .$

6 Constraints on non-unitary parameters

In this section, we have tried to derive the constraints on the non-unitary parameters that P2O can put on them in the future. To do so, we calculated the true event rates with the oscillation parameter values fixed at their present global best-fit values [1, 2]. We varied the test values of $\delta _{\textrm{CP}}$ in the range $[-180^\circ :180^\circ ]$ and $\sin ^2 {\theta _{23}}$ in its $3\, \sigma$ range. The non-unitary parameter, on which we want to derive the constraint was varied, and other non-unitary parameters were fixed at their unitary values. $\phi _{10}$ has been fixed at 0. In Fig. 16 we have shown $\Delta \chi ^2$ as a function of the non-unitary parameters for NH being both the true and test hierarchy. We can see that P2O can provide constraint at $90\%$ $(3\, \sigma )$ confidence level for $\alpha _{00}> 0.92$ (0.86), $\alpha _{11}>0.97$ (0.95), and $|\alpha _{10}|<0.091$ (0.18) after 6 years of only neutrino run when both true and test hierarchies are NH. Anti-neutrino run worsens the constraints. In Fig. 17, we have shown the similar results for IH being the true and test hierarchy. It is obvious that in case of IH, anti-neutrino runs improve the sensitivity to non-unitary parameters. It can be seen that the constraint at $90\%$ $(3\,\sigma )$ confidence level after $3+3$ years are $\alpha _{00}> 0.8$ (out of range), $\alpha _{11}>0.98$ (0.96), and $|\alpha _{10}|<0.12$ (out of range). These constraints are less stringent than the present constraints from the global analysis [65], but better than the constraints put on by the present NO$\nu$A and T2K experimental data [33]. These results are also listed in Table 4.

7 Conclusion

We found out that in case of unitary mixing, the hierarchy-$\delta _{\textrm{CP}},$ and octant-hierarchy degeneracies present at experiments like NO$\nu$A and T2K breaks down for P2O. Now, the wrong hierarchy can be ruled out at more that $9\, \sigma$ $(8\, \sigma )$ C.L. after 6 years for all the $\delta _{\textrm{CP}}$ values when NH (IH) is the true hierarchy and the atmospheric muon contamination is $10\%.$ For NH, only neutrino run has the best hierarchy sensitivity among all the possible neutrino and anti-neutrino runtime combinations. For IH, addition of anti-neutrino run does not have any significant effect on hierarchy sensitivity, but equal neutrino and anti-neutrino runs of 3 years each have the best hierarchy sensitivity. For NH (IH), maximum hierarchy sensitivity of $11\,\sigma$ $(9\,\sigma )$ can be reached at $\delta _{\textrm{CP}}=-90^\circ$ $(90^\circ ).$

CP violation can be established after 6 years of only neutrino run at $1.5\, \sigma$ $(0.7\, \sigma )$ for $\delta _{\textrm{CP}}=\pm 90^\circ$ for NH (IH) being the true hierarchy when the atmospheric muon mis-identification factor is $10\%.$ Addition of anti-neutrino run worsens (improves) the CP violation sensitivity for NH (IH) being the true hierarchy. After equal neutrino and anti-neutrino run of 3 years each, CP violation sensitivity at $\delta _{\textrm{CP}}=\pm 90^\circ$ can be improved to $1\,\sigma$ for the true hierarchy being IH.

We have calculated the hierarchy and CP violation sensitivity assuming $10\%$ muon contamination in case of non-unitary mixing by observing effect of one parameter at a time. When NH is the true hierarchy, the wrong hierarchy can be excluded at more than $9\, \sigma$ $(11\,\sigma )$ at $\delta _{\textrm{CP}}=90^\circ$ $(-90^\circ )$ C.L. after 6 years of only neutrino run for both $\alpha _{00},$ and $\alpha _{11}.$ Neutrino run has better hierarchy sensitivity for these two parameters. In these two cases, non-unitary mixing does not have any significant effect on hierarchy sensitivity. For $|\alpha _{10}|,$ the hierarchy sensitivity reduces for non-unitary mixing. For this case, hierarchy can be established at $\sim 4\, \sigma$ after 6 years of only neutrino run when NH is the true hierarchy.Anti-neutrino run can improve it up to $8\,\sigma .$ When IH is the true hierarchy, the wrong hierarchy can be excluded at $5\,\sigma$ (more than $7\,\sigma )$ after 6 years of only neutrino run for $\alpha _{00}$ $(\alpha _{11}).$ The hierarchy sensitivity for non-unitary mixing gets reduced (remains same) in case of $\alpha _{00}$ $(\alpha _{11}).$ Anti-neutrino run can improve the sensitivity up to $6-7\,\sigma$ $(8\,\sigma )$ for $\alpha _{00}$ $(\alpha _{11}).$ In case of $|\alpha _{10}|,$ hierarchy sensitivity gets reduced for non-unitary mixing. Hierarchy can be established only at $2\, \sigma$ after only 6 years of neutrino run when IH is true. It can be improved up to $6\, \sigma$ after anti-neutrino run.

Fig. 17

$\sigma =\sqrt{\Delta \chi ^2}$ as a function of individual non-unitary parameters for IH being the true and test hierarchies and for different run times

For non-unitary mixing, anti-neutrino run reduces the CP violation sensitivity when NH is the true hierarchy. In cases of $\alpha _{11},$ and $|\alpha _{10}|,$ CP violation can be established at more than $1.5\,\sigma$ for $\delta _{\textrm{CP}}=\pm 90^\circ$ after 6 years of only neutrino run. For $\alpha _{00},$ CP violation can be established at more than $1.5\, \sigma$ $(0.7\, \sigma )$ for $\delta _{\textrm{CP}}=-90^\circ$ $(90^\circ )$ after only neutrino runs for 6 years. When IH is the true hierarchy, anti-neutrino run can improve CP violation sensitivity slightly. For all three non-unitary parameters: $\alpha _{00},$ $\alpha _{11},$ and $|\alpha _{10}|,$ CP violation can be established at $\sim 0.7\, \sigma$ for $\delta _{\textrm{CP}}=\pm 90^\circ .$ In case of $\alpha _{00},$ anti-neutrino run can (can not) be improved up to $1\,\sigma$ when $\delta _{\textrm{CP}}=90^\circ$ $(-90^\circ ).$ In cases of $\alpha _{11},$ and $|\alpha _{10}|,$ CP violation can be improved by anti-neutrino run up to $1\,\sigma$ for $\delta _{\textrm{CP}}=\pm 90^\circ$ when IH is the true hierarchy.

Table 4 P2O bounds on the magnitudes of the non-unitarity parameters at $90\%$ and $3\sigma$ with NH and IH as true and test hierarchy and comparative results from global fit analysis [65]

We have also calculated the constraints that can be put on $\alpha _{00},$ $|\alpha _{11}|,$ and $|\alpha _{10}|$ by the P2O in the future. We found out that the constrained at $90\%$ $(3\, \sigma )$ confidence level can be achieved for $\alpha _{00}> 0.92$ (0.86), $\alpha _{11}>0.97$ (0.95), and $|\alpha _{10}|<0.091$ (0.18) after 6 years each of neutrino run when both true and test hierarchies are NH. For IH, the future possible constraints at $90\%$ $(3\,\sigma )$ confidence level are $\alpha _{00}> 0.8$ (out of range), $\alpha _{11}>0.98$ (0.96), and $|\alpha _{10}|<0.12$ (out of range). after 3 years of neutrino and 3 years of anti-neutrino run.