Probing the mixing between sterile and tau neutrinos in the SHiP experiment

: We study the expected sensitivity to the mixing between sterile and tau neutrinos directly from the tau neutrino disappearance in the high-energy fixed target experiment. Here, the beam energy is large enough to produce tau neutrinos at the target with large luminosity. During their propagation to the detector, the tau neutrino may oscillate into sterile neutrino. By examining the energy spectrum of the observed tau neutrino events, we can probe the mixing between sterile and tau neutrinos directly. In this paper, we consider Scattering and Neutrino Detector (SND) at SHiP experiment as a showcase, which uses 400 GeV protons from SPS at CERN, and expect to observe 6,300 tau and anti-tau neutrinos from the 2 × 10 20 POT for 5 years operation. Assuming the uncertainty of 10%, we find the sensitivity | U τ 4 | 2 ∼ 0 . 08 (90% CL) for ∆ m 2 41 ∼ 500 eV 2 with 10% signal-to-background ratio. We also consider a far SND at the end of the SHiP hidden sector detector, in which case the sensitivity would be enhanced to | U τ 4 | 2 ∼ 0 . 02.


Introduction
One of the simplest extensions to explain the neutrino mass is the introduction of additional right-handed neutrinos in the standard model [1][2][3][4][5].These additional neutrinos should be free from the weak interaction [6], and thus they are called sterile.The sterile neutrinos may also explain the anomalies in the short baseline neutrino oscillation experiments [7,8], and solve the cosmological problems of dark matter and baryon asymmetry [9].
The sterile neutrinos do not have weak interaction, however, they can communicate with visible sector through the mixing with active neutrinos at low energy.The resulting oscillation has been the main subject to search and constrain the sterile neutrinos [10,11].The mixing of sterile neutrino with electron and muon neutrino is constrained with a bound of |U e4 | 2 ≲ 0.04 (90% CL) for ∆m 2 41 ∼ 10 −2 eV 2 and |U µ4 | 2 ≲ 10 −2 for 10 −2 eV 2 ≲ ∆m 2 41 ≲ 1 eV 2 [12].
In this paper, to overcome such insensitive nature of |U τ 4 | 2 , we propose a new approach to constrain the mixing between sterile and tau neutrinos focusing on the SHiP experiment.
SHiP experiment is a facility that has been proposed at the SPS beam dump to search for hidden sectors beyond the SM [21][22][23][24][25]. Considering SND at SHiP as a short baseline, we can search for the evidence of tau neutrino disappearance during the propagation.Thanks to the high potential to detect O(10 4 ) tau neutrino Charged-Current(CC) events, it is possible to probe the mixing between sterile and tau neutrinos directly by measuring the tau neutrino energy spectrum.
The rest of this paper is organised as follows.In Sec. 2, we review the neutrino oscillation with sterile neutrino, and in Sec. 3 we summarise the SHiP experiment and show the expected energy spectrum of tau neutrino events.In Sec. 4, we describe the statistical method we employed and presents the expected sensitivity to the mixing between sterile neutrinos and tau neutrinos.In Sec. 5, we propose a new SND with long baseline that is placed after Hidden Sector Decay Spectrometer and estimate the enhancement of sensitivities due to the new SND.Lastly, we make a conclusion in Sec. 6.

Neutrino Oscillation with Sterile Neutrino
The weak eigenstates of neutrino |ν α ⟩ is the linear combination of the mass eigenstates |ν i ⟩ determined by the mixing matrix U, In '3+1' model with single sterile neutrino as well as 3 active neutrinos in SM, the matrix becomes 4 × 4 matrix, which can be written as

.2)
This mixing matrix can be parameterized with six rotation angles and three CP phases.The rotation angles include three additional mixing angles between the sterile neutrino and each active neutrino, θ 14 , θ 24 and θ 34 as well as the usual three mixing angles between active neutrinos θ 12 , θ 23 and θ 13 .After travelling a distance L, the state of the neutrino evolves as and the probability for the transition is given by (2.4) Especially, for the short baseline experiment where the mixing between active neutrinos can be ignored, the probability of the relativistic neutrino can be approximated as where ∆m2 41 ≡ m 2 4 − m 2 1 is the mass squared difference between the sterile neutrino and the lightest neutrino, L is the baseline distance, and E ν is the energy of the neutrino.
For tau neutrino disappearance experiments, the survival probability becomes where |U τ 4 | 2 = sin θ 2 34 , ignoring other mixing angles.In Fig. 1, we show the survival probability of tau neutrinos, P τ τ , for a mixing parameter of |U τ 4 | 2 = 0.1 with a neutrino detector at a distance L = 30 m from the target.The different colors show the probability for different sterile neutrino masses: green, blue, and red line for ∆m 2 41 = 600, 2400, and 10 4 eV 2 , respectively.The dashed gray line represents P τ τ = 1, for the case without mixing between tau neutrinos and sterile neutrinos.

Tau Neutrino Spectrum at the SHiP Experiment
In the SHiP experiment, the high energy protons of 400 GeV from SPS can generate all three flavors of neutrino, when the protons are stopped at tungsten target.The neutrinos propagate through the shielding to the SND located 30 m away from the proton target.Three different flavors of neutrino can be observed using CC interaction at the emulsion detector in the SND.
While the electron and muon neutrinos are primarily produced from kaon and pion decays at the target, the tau neutrinos are mainly from the decay of the charm meson, D s → τ ν τ , and subsequent tau decays.The subsequent interaction of secondary particles, which we call cascade interaction, also can produce tau neutrinos 1 .The number of tau neutrinos produced at the target can be estimated using Monte Carlo(MC) simulation of the experiment, and it could be around 10 16 for 5 years operation [25].
The differential number of events of tau neutrinos with CC interaction at SND, with respect to the traveling distance and the energy of tau neutrino can be obtained by where n W is the number density of tungsten atom in tungsten layers with total thickness L W in SND, which length is L SND .ϕ ντ (ϕ ντ ) is the expected number of tau (anti) neutrinos passed through SND per E ν , σ νA (σ νA ) is the cross-section of (anti) neutrino-nucleus interaction, and ϵ eff is the detection efficiency of tau neutrino.Here, the survival probability P τ τ (E ν , l) depends on the neutrino energy and the distance from the proton target.Instead of full MC simulation, we adopt the event rate of tau neutrino CC interaction on 3 flavor model from Ref. [28], which is the most conservative result to the best of our knowledge.From here, we can write dN dEν with sterile neutrino oscillation as where L 0 is the distance between the target and SND.Since the design of SND on Ref. [28] is similar with the design on the technical proposal of SHiP experiment at 2015 [21], we adopt ϵ eff from the technical proposal at 2015, while L W = 100 cm, L SND = 3 m and L 0 = 30 m, following recent designs of SND at SHiP experiment [24,25].Therefore, after 5 years operation with 2 × 10 20 POT, about 6,300 tau neutrino events are expected to be observed in the 3 flavor model.
To consider the energy response of SND in the reconstructed neutrino energy E rec from the true energy E ν , we used the reconstructed energy spectrum as where the Gaussian response function f (E rec , E ν ) is given by The energy classification from the hadron contribution determines σ, which we used as 20% of E ν , i.e. σ = 0.2E ν , following the result from the technical proposal of SND@LHC.[29] Note that, apart from the energy response, the baseline uncertainty also exists.This uncertainty comes from the varying interaction lengths between the target and the protons from SPS.However, this uncertainty is relatively minor compared to the length of SND, and can be ignored in our analysis.
In Fig. 2, we show the number of tau neutrino + tau anti neutrino events per energy at SND with respect to E rec , considering the sterile neutrino oscillation for three mass squared differences, i.e. ∆m 2 41 = 600, 2400, and 10 4 eV 2 with green, blue, and red colors, respectively.We can see the depletion of the probability depending on the energy of the neutrino, due to the oscillation into sterile neutrino.

Sensitivity to the Mixing between Sterile and Tau Neutrinos
In this section, we show the expected sensitivity to '3+1' model using the tau neutrino disappearance at SHiP experiment.To find the expected sensitivity, we assumed that the result from SHiP experiment is in agreement with the standard three neutrino model and found the Confidence Level (CL) of the virtual result at each point of '3+1' model parameters.To find the CL, we used two different methods: 1) Wilks' theorem [30], and 2) the profiled Feldman-Cousins (FC) method [31,32].
We use the well-known statistic ∆χ 2 which quantifies the fitness of the data with the corresponding model, defined as where χ 2 a ≡ −2 max a log L, which maximises the likelihood function L for any set of parameters a.For a likelihood function from SHiP experiment, we use Poisson probability distributions Pois(O i |µ i ), and a likelihood function for the nuisance parameters from previous experiments L(λ), which confines their possible range with uncertainties of SHiP experiment.The combined likelihood function of a data O for model parameters θ and nuisance parameters λ is written as where O i and µ i are the observed and expected number of ν τ CC events at a i-th bin.The number of events at i-th bin µ i ≡ s i + b i is the sum of signal s i and background b i , which is evaluated for given set of parameters θ and λ.
The expected number of signal at i-th bin is calculated as where the nuisance parameters A, α i ∈ λ represent overall and shape uncertainties for signal, which are assumed to have a mean value of zero, while E i,low and E i,high are the lower and upper energy limit of i-th energy bin.Overall and shape uncertainties reflect the uncertainties from parton distributions, factorization and renormalization scale factor, intrinsic transverse momentum [22,28,33], and other possible experimental uncertainties.In this paper, we assume the likelihood function of nuisance parameters as similar as a normal distribution.For simplicity, all nuisance parameters are independent of each other and share the same variance σ2 norm during the analysis, even though the likelihood function may include non-zero covariance terms between nuisance parameters.Therefore, we rewrite L(λ) as For the variance of nuisance parameters, we choose σ norm = 10% or σ norm = 20% in later figures.These uncertainties can be refined with the results from the DsTau Project in the future [34].
To consider the background, we use the tau neutrino spectrum divided by signal-tobackground ratio R s/b in 3 flavor model 2 .Similar with A and α i , we included a overall uncertainty B ∈ λ and shape uncertainties β i ∈ λ in the background.Therefore, the ) under the assumption of θ 14 = θ 24 = 0, for different number of background R s/b = 10 (left) and 1 (right).Blue solid (dashed) lines are the constraint of 90% CL using the profiled FC method (Wilks' theorem), with uncertainty σ norm = 10% (thick lines) or 20% (thin lines).The vertical dashed lines show the existing constraints from IceCube-DeepCore [18] (black) and Super-Kamiokande [17] (grey), respectively.To check the validity of Wilks' theorem, CDFs of ∆χ 2 at orange and green stars on the left figure are drawn in Fig. 5.
expected number of the background at i-bin is written as Here, the background is independent of sterile neutrino parameters, and R s/b is chosen as 10 or 1 in later figures.A logarithmic scale is used for our bins, beginning at 10 GeV and each bin ends at 1.5 times its starting energy, with a total of 7 bins.In Fig. 3, we show the number of expected events per bin (green) after 5 years operation with the mixing |U τ 4 | 2 = 0.1, assuming 10% background (R s/b = 10 with grey color) for ∆m 2 41 = 600 eV 2 (left) and 2400 eV 2 (right).For a comparison, the blue plot shows the most probable pseudo-data for the 3-neutrino model.
To find a CL, as a first method, we use Wilks' theorem [30], which points out that under certain conditions, the probability distribution of ∆χ 2 follows a χ 2 distribution with the same number of degrees of freedom as θ, which is 2 in this study since θ = (∆m 2 41 , |U τ 4 | 2 ).However, in neutrino oscillation models, the probability distribution of ∆χ 2 does not necessarily follows a χ 2 distribution.Therefore, as a second method, we utilized the profiled FC method [32] which is a variant of FC method [31] for a case with nuisance parameters.After choosing O as most-probable data in 3 flavor model, we generated 10 4 number of pseudo-data by MC simulation on each point θ on the map of model parameters, assuming λ = argmax λ L(θ, λ|O).By calculating ∆χ 2 of each set of pseudo-data, we find its probability distribution which is denoted as f (x).
From the probability distribution of ∆χ 2 , the CL of the data O is defined as follows: where ∆χ 2 (O) is ∆χ 2 value of the data O.Here, the definition of a CL indicates the possibility that other possible outcome from the same experiment would give a better fit, more than the data O.The range of integration in Eq. (4.6) starts at x = 0, as ∆χ 2 must be non-negative according to the definition.In Fig. 4, we show the expected sensitivity after 5 year observation at SHiP on the plane of (|U τ 4 | 2 , ∆m 2 41 ) under the assumption of θ 14 = θ 24 = 0, for different number of background R s/b = 10 (left) and 1 (right).Uncertainties of nuisance parameters are chosen as σ norm = 10% (thick lines) and 20% (thin lines), while solid (dashed) lines are the constraint of 90% CL using the profiled FC method (Wilks' theorem).Wilks' theorem is applied by drawing a contour of ∆χ 2 = 4.61, which corresponds to first 90% cut of χ 2 distribution with two degrees of freedom, while profiled FC method is applied through drawing a contour of 90% CL using Eq.(4.6).The vertical dashed lines show the existing constraints from IceCube-DeepCore [18] and Super-Kamiokande [17], respectively.
From the left window in Fig. 4, we can find the sensitivity using profiled FC method could be |U τ 4 | 2 ∼ 0.08 (0.1) for ∆m 2 41 ∼ 500 eV 2 with uncertainty 10% (20%), respectively, after 5 year operation of SHiP experiment with 10% background.For ∆m 2 41 ≳ 500 eV 2 , to the NSND only case, both methods gives similar sensitivity near ∆m 2 41 ∼ 10 3 eV 2 , and it merely depends on σ norm , meaning that the uncertainty from nuisance parameters can be relaxed.If R F/N = 100% and R s/b = 10 (1), the combined sensitivity from NSND+FSND can be |U τ 4 | 2 ∼ 0.02 (0.03) near ∆m 2 41 ∼ 500 eV 2 .In Fig. 8, we show the CDFs of ∆χ 2 at orange and green stars on the upper-left window in Fig. 7.As can be seen from this figure, the CDFs from both method are consistent very well for the orange point, which explains the similar sensitivity with both methods in Fig. 7 for ∆m 2 41 ≲ 5000 eV 2 .For larger ∆m 2 41 , sensitivities from both methods start to mismatch, as can be seen with the CDFs for the point of the green star.

Conclusion
In this paper, we considered tau neutrino disappearance for the first time in the high energy fixed target experiment of SHiP, where several thousand tau neutrinos are expected to be observed after 5 years operation.This gives good environment for the tau neutrino oscillation experiment.
We studied sterile neutrino oscillation in '3+1' model and derived possible sensitivity to the mixing and mass difference using the energy spectrum of the tau (anti)neutrinos.With NSND only, the sensitivity would be |U τ 4 | 2 ∼ 0.08 (0.1) near ∆m 2 41 ∼ 500 eV 2 , for σ norm = 10% (20%) and R s/b = 10 using profiled FC method.We find that this is slightly weaker than the sensitivity assuming the Wilks' theorem is applied.
To enhance the sensitivity, we suggested additional FSND which can be put after Hidden Sector Decay Spectrometer.It can play a role of far detector and its spectrum can be directly compared to that of NSND.From Fig. 7, NSND+FSND gives a robust sensitivity that does not depend on the choice of statistical method.Also, the constraints barely change with the choice of σ norm .If R F/N = 100%, the combined data gives the sensitivity |U τ 4 | 2 ∼ 0.02 (0.03) near ∆m 2 41 ∼ 500 eV 2 with R s/b = 10 (1) .

Figure 1 .
Figure 1.The survival probability of tau neutrinos, P τ τ , for a mixing parameter of |U τ 4 | 2 = 0.1 with a neutrino detector at a distance L = 30 m from the target.The different colors show the probability for different sterile neutrino masses: green, blue, and red line for ∆m 2 41 = 600, 2400, 10 4 eV 2 , respectively.The dashed gray line represents P τ τ = 1, for the case without mixing between tau neutrino and sterile neutrino.

Figure 2 .
Figure 2. Differential number of distinguished ν τ CC events on SND with respect to the reconstructed energy E rec , if a mixing parameter |U τ 4 | 2 = 0.1.Different scenarios are depicted: the green line for ∆m 2 41 = 600 eV 2 , the blue line for ∆m 2 41 = 2400 eV 2 , and the dashed red line for ∆m 2 41 = 10 4 eV 2 .The dashed gray line represents the case with P τ τ = 1, indicating no mixing between tau neutrino and sterile neutrino.

1 Figure 4 .
Figure 4.The expected sensitivity after 5 year observation at SHiP on the plane of (|U τ 4 | 2 , ∆m 2 41) under the assumption of θ 14 = θ 24 = 0, for different number of background R s/b = 10 (left) and 1 (right).Blue solid (dashed) lines are the constraint of 90% CL using the profiled FC method (Wilks' theorem), with uncertainty σ norm = 10% (thick lines) or 20% (thin lines).The vertical dashed lines show the existing constraints from IceCube-DeepCore[18] (black) and Super-Kamiokande[17] (grey), respectively.To check the validity of Wilks' theorem, CDFs of ∆χ 2 at orange and green stars on the left figure are drawn in Fig.5.

Figure 7 . 2 RFigure 8 .
Figure 7.The same as Fig.4but with the combined analysis with both NSND and FSND for different R F/N values, 10% (purple) and 100% (red) as well as NSND only (blue).In the upper (lower) windows σ norm = 10% (20%), and in the left (right) windows R s/b = 10 (1) is used.Like Fig.4, solid (dashed) lines indicates that the profiled FC method (Wilks' theorem) is used to find the CL.