Constraining active-sterile neutrino mixing via lepton flavor violating decays of mesons

As a hypothetic particle, a sterile neutrino can exist in types of new physics models. In this work by investigating flavor violating semileptonic and leptonic decays of K,D, and B mesons induced by the GeV scale sterile neutrino, we explore its allowed regions in parameter space. Analytically, we derive branching fractions of semileptonic decay Mh+→Ml+ℓ1+ℓ2−\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$M^{+}_{h}\to M_{l}^{+}\ell _{1}^{+}\ell _{2}^{-}$\end{document} and leptonic decay M0→ℓ1−ℓ2+\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$M^{0}\to \ell _{1}^{-}\ell _{2}^{+}$\end{document}, including the contribution from a new box diagram contribution due to the massive sterile neutrino. Imposing its mass ranges, we numerically analyze the active-sterile neutrino mixings in corresponding regions. We find that when the sterile neutrino mass is located in between pion and kaon mass, K+→π+e±μ∓ gives the strongest constraint while B+→π+e±μ∓ provides the dominated constraint when its mass in between of kaon and B meson. If the sterile neutrino is even heavier than Bmesons, the LHCb Bs0→μ±e∓\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$B^{0}_{s}\to \mu ^{\pm } e^{\mp }$\end{document} results gives the strongest constraint.


Introduction
Now it has been well established that at least two active neutrinos are massive with tiny masses. The origin of neutrino mass is still an open question. Sorts of ideas have been proposed to solve this fundamental question, including seesaw mechanism [1] and radiative correction mechanism (for example, [2,3] and for a recent review see [4]). General speaking, new particles out of SM particle spectrum will appear associated with neutrino mass models. As a hypothetic particle, though does not participate weak interaction, sterile neutrino is unavoidable in some neutrino mass models beyond SM. For example, in Type I seesaw mechanism the heavy right-handed neutrino singlet contributing the tiny mass of left-handed neutrino is absent from SU (2) interaction and hence appears as a sterile neutrino.
The prediction of sterile neutrino mass in theory is model dependent thus is not unique. In the view of exper-*Correspondence: fanrongxu@jnu.edu.cn Department of Physics and Siyuan Laboratory, Jinan University, Guangzhou 510632, People's Republic of China iment, there are some hints to indicate the existence of sterile neutrino as well as its mass. One type of experiment is neutrino oscillation. In 2001, the LSND experiment searchedν μ →ν e oscillations, suggesting that neutrino oscillations occur in the 0.2 < m 2 < 10 eV 2 range [5]. Later, the MiniBooNE experiment indicated a twoneutrino oscillation,ν μ →ν e , occurred in the 0.01 < m 2 < 1.0 eV 2 range [6]. An updated global fit [7], taking into account recent progress, gives m 2 41 ≈ 1.7eV 2 (best-fit), 1.3eV 2 (at 2σ ), 2.4eV 2 (at 3σ ). Hence, if its mass is located at eV, sterile neutrino effect can be unfolded by oscillation experiments and related theoretical studies about oscillations involving a light sterile neutrino can be found in [8][9][10][11][12] . On the other hand, sterile neutrino mass can be even heavier. The operation of LHC provides an opportunity to search TeV scale heavy sterile neutrino [13][14][15]. The IceCube Neutrino Observatory, which locates in Antarctic, gives an unique vision to observe PeV neutrino [17,18]. In between eV and TeV-PeV, the GeV sterile neutrino could appear in weak decays of bound states of heavy quarks. Thus the sterile neutrino, with its undermined mass varied from eV to TeV-PeV, provides a port to connect New Physics beyond SM.
Hadron physics provides a special platform for searching new physics, taking the discovery of four-quark matter as an example [19], though with the challenging nonperturpative effects. In recent years the semileptonic decays B 0 → K * + − and B 0 → π + − have been studied extensively both theoretically and experimentally. Though the expectation of new physics in forwardbackward asymmetry of lepton pairs in B 0 → K * + − has already faded away, NP hope still holds at the so-called observable P 5 [20,21]. Similar situation happened in the leptonic decays of B 0 and B 0 s . The SM-like B(B s → μ + μ − ) and perhaps new physics allowed B(B d → μ + μ − ) give strong constraints to theories beyond SM [22,23], but the windows to NP is not closed. It is known that both types of decays are FCNC process, giving a chance to put NP particles in the loop. Then, it is nature to consider the possibility of a sterile neutrino in the loop. In fact there have been continuous efforts to study GeV scale sterile neutrino indirectly via some certain semileptonic and leptonic decays of B, D, and K mesons. In the semileptonic decay processes, if its mass is in between the meson masses of initial and final states, the sterile neutrino can be onshell produced [24]. A popular consideration is to take sterile neutrino as the Majorana neutrino; thus, lepton number violating decays are induced [25][26][27][28][29][30][31][32][33][34]. The idea to make use of leptonic decay with neutrino final state, in which sterile neutrino is involved at tree level, is also proposed [35,36]. In above works, the final state leptons, though with lepton number violation, are mostly with same flavors; thus, only single PMNS matrix is relevant. The lepton flavor violating decays from mesons, on the other hand, is related to two PMNS matrix elements, thus could give complementary information to corresponding lepton number violating decays.
The early quest for lepton flavor violating processes can be traced back to the leptonic cecay K L → e ± μ ∓ in 1998 [41]. So far, K L → e ± μ ∓ still gives a very strong constraint to NP models. The latest experiment for leptonic decay is carried out in LHCb by searching B 0 → e ± μ ∓ giving the upper limit 1.3 × 10 −9 [49]. As for the semileptonic decays of K, D, and B mesons, most of them are still results from BaBar [43] and it is hoped that LHCb can bring new limit in the near future. A detailed summary for related experiments is given in Table 3. In this paper, we will analysis both leptonic and semileptonic decays from K, D, and B mesons induced by sterile neutrino. By combing all the currently related experiments, we will give the constraints to relevant PMNS matrix elements. This paper is organized as follows. In Section 2, we will give a brief introduction to the model related to heavy sterile neutrino. Then we will derive the exact formulas for leptonic decay and semileptonic decays with heavy ster-ile neutrino contribution separately. In Section 3, we will perform a numerical study and give the allowed coherent parameter space. Discussion and conclusion will be made in Section 4.

Working frame
In this section, a brief introduction of heavy sterile neutrino is given firstly. Then, we derive the required analytical formulas in semileptonic and leptonic decays separately.

Model setting
As introduced in Section 1, here, we are only interested in the GeV scale sterile neutrino. Due to its heavy mass, in flavor space, the sterile neutrino will decouple from other three active neutrinos in the oscillation processes. With the appearance of a sterile neutrino and without involving the details of a concrete model, the mass mixing can always written via a non-unitary mixing matrix, which characterizes the rotation between mass eigenstate and flavor eigenstate in vacuum. A direct consequent for the non-unitary mixing is zero-distance effect [37], the oscillation could happen even without propagate few distance. Such effect has been pointed out to be detected in oscillation experiment by a near detector, which will be discussed in a separate work. In the following context, we will focus on the mass of ν 4 and the mixing elements U eN,μN,τ N . And hereafter we adopt the notation N to denote sterile neutrino for the purpose of emphasize.

Semileptonic decay
The sterile neutrino, if its mass is in between with the initial heavy meson and final mass meson, can be produced on-shell and then decay shortly. As for the heavy meson, we are especially interested in those charged ones. The reason for such a choice is due to the fact that tree-level annihilation diagram not only gives dominated contribution to the decay of heavy meson, but also provides a chance to produce sterile neutrino from W boson sourced from quark annihilation, see Fig. 1. The branching fraction for the three-body decay can be simplified to the multiplication of two-body decays in Breit-Wigner approximation, in which M h (M l ) denotes heavy (light) meson, and 1 , 2 represent charged lepton (e, μ, τ ) with different flavors.
which relies on two unknown parameters, the PMNS matrix element U 1 N and the mass of N. Especially the heavy meson-dependent function in which ξ 1 is a particular CKM matrix element corresponding to the mother particle, and the definition of the auxiliary function is given as λ(x, y, z) = x 2 + y 2 + z 2 − 2(xy+xz+yz). The two parameters, characterizing the initial and final state particles, are defined as For the cascade decay of sterile neutrino, one can calculate its decay width, Similar to the decay of M h , besides the PMNS matrix U 2 N , the width depends on the final state dependent function Y, given with another CKM matrix element ξ 2 which is determined by M l . The branching fractions of cascade decays of sterile neutrino rely on its lifetime, which is unknown at current stage. For the illustration purpose, here, we adopt the working scenario in which we neglect the W decays into leptons, giving In the denominator, the summation is performed only to the first two generations for both lepton and quark sector.
As for the function Y, the value for its first parameter M q should be chosen as M q = π + (K + ).
One should keep in mind that Eq. (2) gives a general description of this type process, which actually contains many modes when different initial and final states are chosen. In Table 1, we have summarized explicitly corresponding parameters for such modes.

Leptonic decay
In SM, the leptonic decays with non-conserving flavors induced by active neutrino is negligible. Hence, this type of modes can be made use of searching new physics. If the processes can be observed in experiment, it will be definitely a signal for desired new physics. As an illustration, we will consider such a process induced by heavy sterile neutrino.
To calculate the amplitude of usual final states with same flavor, both penguin diagram and box diagram contribution should be included. In this work, we only focus on the flavor-changing leptons in the final states, thus only box diagram contributes since Z-penguin only provides flavor-conserving final leptons. The initial neutral meson M 0 , could be B 0 , D 0 and K L and the intermediate particle N is off-shell (Fig. 2). Generically we have the branching fraction for the pure leptonic decays of pure neutral meson as with z i ≡  Table 2. The reliance on both quark and lepton can be classified into two functions in addition to PMNS matrix (U 1 N , U 2 N ) and CKM matrix (ξ ). The dependence on initial and final states is absorbed into the function Z(M 0 , m 1 , m 2 ), giving    Fig. 3, reveals the new physics information encoded in M 0 decay. [38], and recovers B 0 when vanishing the second variable y. A more qualitative analysis for A(x, y) will be given in next section. To clarify the relevant modes described by Eq. (8) as well as the exact contained parameters, in Table 2, explicit parameters involved in the corresponding channels are summarized.

It can be checked that function A(x, y) is an extension of standard loop function
Note, here, the decay constants in Table 2 correspond to neutral mesons while those in Table 1 stand for the charged pseudoscalar mesons.

Experiment status
There have been about 20 years history for the search for flavor violating decays. We summarize all the relevant experiments in Table 3 as the input of our numerical study.

Property of A(x,y)
The branching fractions of leptonic decays largely relies on the A(x, y), hence before numerical studies of phenomenology it is necessary to explore the features of this function. In Fig. 3, we plot the dependence its behaviors Table 2 The detailed parameters for semileptonic decay respect to sterile neutrino mass. Typic features of A(x, y) are shown below.
• A singularity appears at m N = m W , and more close to W mass, more enhanced the function value is. This is understandable as current working frame is not UV-completed. • Even if the neutrino is massless, the value of loop function approaches A ∼ −0.2. The appearance of loop function effect due to sterile neutrino, combing non-vanishing lepton mixing, causes LFV decays. • There is a particular choice that A = 0. Take B decay as an example, the internal heavy quark loop comes • When sterile neutrino mass is larger than electroweak scale, the behavior is asymptotic stable, giving a value smaller than SM. Since we are only interested in GeV scale sterile neutrino, such a range will not be involved in this paper.

The combining analysis
Now with the prepared necessary analytical formulas in above, we will present our numerical study in the following section. The semileptonic decay happens, in our working frame, is due to the on-shell production of sterile neutrino, which actually requires the sterile neutrino mass in between of the initial heavy meson and the final light meson. However, the effect of off-shell sterile neutrino can play a role in leptonic decays. Thus, whatever mass of sterile neutrino is, the contribution from leptonic decays cannot be negligeble. In other words, if the mass is not located in between initial and final mesons, only the leptonic decay experiments give constraints to corresponding mixing matrix, we call this scenario D. In addition to scenario D, in the mass region m π < m N < m B , we classify the mass range into three other different scenarios, named as scenarios A, B, and C.
• Scenario A: m π < m N < m K If sterile neutrino mass located at this region, the semileptonic decays induced by the on-shell sterile neutrino contains B + → π + μ ± e ∓ , B + → π + τ ± μ ∓ , B + → π + τ ± e ∓ , D + → π + μ ± e ∓ and K + → π + μ ± e ∓ . In principle, all the leptonic decays from K L , D 0 and B 0 , B 0 s , including K L → μ ± e ∓ , D 0 → μ ± e ∓ and B 0 (s) → μ ± e ∓ , should also be taken into account. However, from the numerical analysis, all the parameter spaces are fully allowed by these leptonic decays, which is too weak to give an efficient constraint. Thus, only these semileptonic ones provide some effective information.
Since the decays into τ is kinematically forbidden, here, we only consider the modes decaying into μ, e final states. we compare the decays from three different parent particle and find K + decay provides the most stringent constraint and show it in Fig. 4, while B + and D + decays give a much wide allowed region and hence are neglected there. It is easily to see the product of U eN and U μN is strictly constrained to O(10 −5 ), but a further restriction to U αN requires other input experiment, which will be discussed in a separate work.
• Scenario B: m K < m N < m D As pointed in above context, leptonic decays always appear. For the semileptonic decays in this case, only B + and D + decays while K + is forbidden otherwise Fig. 4 The allowed parameter space in scenario A, in which we have taken m N = 0.3 GeV as an illustration the mother particle is lighter than its daughter particle. The explicit modes which are incorporated into our numerical simulations are: As the first step, let us focus on e, μ final states. First, by comparing various B + decay modes with different final states, one can find the constraint to PMNS matrix from B + → π + e ± μ ∓ dominates the corresponding ones from B + → K + e ± μ ∓ , as shown in Fig. 5. Second, for the allowed region extracted from D + decays, D + → π + e + μ − is much stronger than D + → K + e ± μ ∓ . Looking at the same π + final states, the numerical analysis tells that B + decay gives the strongest restriction, which actually gives an upper limit for the product of U eN and U μN , O(10 −2 ). As for the individual matrix elements, one has to resort to other way. It is noted that so far no more stringent constraint can be obtained from leptonic decay. And the constraint of the correlation of PMNS matrix V τ N and V eN , V μN is still too weak from τ , μ, or τ , e final states, which is also neglected here.
In addition to leptonic decays, only semileptonic decays from B + can appear in this situation, which actually gives more stringent constraints. We still stick to the μ, e final states with the same reason as previous scenarios. Though sterile neutrino mass is taken 4 GeV, the numerical simulation leads to the same conclusion as scenario B. Thus we will not show its corresponding plot here. • Scenario D: m N < m π or m N > m B In this scenario, semileptonic decays are forbidden and only leptonic decays happen. If the sterile neutrino mass is lighter than the lightest meson π, the mass dependent function A is close to the SM situation giving a small amplitude (module to PMNS matrix element), then the further constraint to PMNS matrix from experiment measurement is weak. Such behavior has been checked and we will not show in graphs here. It is interesting to explore the mass range larger than B mesons. We take Fig. 6 as an illustration, with sterile neutrino mass m N = 70 GeV. Among all the 4 charged neutral meson decays, the parameter space from D 0 and B 0 decays are still fully filled thus not marked in the figure. The experiment upper limit for K L and B 0 s indeed touch the restriction to U eN − U μN parameter space. As shown in Fig. 6, the recent LHCb experiment B 0 s → e ± μ ∓ now catch up with the classical BNL experiment on K L → e ± μ ∓ .
In a summary, the allowed parameter of PMNS matrix is sterile neutrino mass dependent. When the mass is lighter than m π , parameter space does not receive a constraint from current meson decay experiments. If the mass is located between m π and m K , the semileptonic decay K + → π + e + μ − provides the most stringent constraint, U μN U eN ∼ O(10 −5 ). When its mass is in between kaon mass and B meson, BaBar experiment B + → π + e ± μ ∓ in fact dominate the constraint, giving U μN U eN ∼ O(10 −2 ). If the sterile neutrino is heavier than B meson, leptonic decay B 0 s → e ± μ ∓ provides the strongest constraint.

Conclusion
In this work, we consider various semileptonic and leptonic decays of neutral mesons induced by a heavy sterile neutrino, which can in turn constrain parameter space of the unknown PMNS matrix elements. Especially we calculated the loop function of a box diagram contributing to leptonic decays. Making use of the two types of decays  The allowed parameter space in scenario D, in which we have taken m N = 70 GeV as an illustration from different parent particle, we find the allowed range of parameter space of sterile neutrino is mass dependent. If sterile neutrino is lighter than pion mass, these meson decays have null restriction. When sterile neutrino mass is located in between pion and kaon mass, K + → π + e ± μ ∓ gives the strongest constraint while B + → π + e ± μ ∓ provides the dominated constraint when sterile neutrino mass in between of kaon and B meson. If sterile neutrino is even heavier than B mesons, the measurement performed at LHCb B 0 s → μ ± e ∓ gives the strongest constraint. It should be noted that so far we can only extract the restriction information to parameter space from the decays with e, μ final states while the decays with a τ in final state is not incorporated. From the analysis, we provide a global constraint for |V μN V eN | in different m N mass region, however, the magnitude of an individual PMNS matrix element cannot determined in this work and will be discussed in a separate work.

A summary of input parameters
The neutral and charged scalar mesons are involved in two types of decays, respectively. Here, in Table 4, we summarize the typical values adopted in the analysis. We do not distinguish the decay constants between the corresponding neutral and charged mesons in the practical numerical evaluation.