Angular and polarization observables for Majorana-mediated B decays with effective interactions

Abstract

We probe the effective field theory extending the Standard Model with a sterile neutrino in B meson decays at B factories and lepton colliders, using angular and polarization observables. We put bounds on different effective operators characterized by their distinct Dirac–Lorentz structure, and probe the N-mediated B decays sensitivity to these interactions. We define a forward–backward asymmetry $A_\mathrm{FB}^{\ell \gamma }$ between the muon and photon directions for the $B \rightarrow \mu \nu \gamma$ decay, which allows us to separate the SM contribution from the effective lepton number conserving and violating processes, mediated by a near on-shell N. Using the most stringent constraints on the effective parameter space from Belle and BaBar we find that a measurement of the final polarization $P_{\tau }$ in the rare $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$ decays can help us infer the scalar or vector interaction content in the N production or decay vertices. We find that the B meson decays are more sensitive to scalar operators.

1 Introduction

Besides the remarkable performance of the standard model (SM) of particle physics in describing nature, neutrino oscillations are currently the most compelling experimental evidence of the need to extend the SM in order to include mechanisms for neutrino mass generation. In recent years, the LHC experiments have put stringent constraints on the existence of new physics involving colored states, but still the possible extensions of the electroweak (EW) sector are far less restricted. A variety of new physics scenarios may be hidden at energies well above the EW scale, and their study is being consistently tackled by the use of the standard model effective field theory (SMEFT) [1, 2].

But also new physics might involve weakly coupled fields at the EW scale. Many models leading to neutrino masses predict the existence of sterile right-handed neutrinos with Majorana masses, as the Type I [3,4,5,6,7] and also the linear and inverse seesaw mechanisms. These possibly not-that-heavy degrees of freedom can be described by an EFT including them: standard model effective field theory framework extended with right-handed neutrinos (SMNEFT) [8,9,10,11], and here we concentrate on a simplified scenario with only one right-handed neutrino added [8]. The phenomenology of models extending the Type I seesaw renormalizable Lagrangian with effective interactions of higher dimension for the right-handed neutrinos are being studied [12, 13], and recently complemented with the implications of the Minimal Flavor Violation ansatz on the new heavy neutrino interactions [14, 15].

The simplified scenario considering only one sterile state N has started to get attention in connection to the novel dimension five Higgs-neutrino interactions [16] and the heavy neutrino magnetic moment dipole portal [17]. In particular, some studies have set constraints on the effective operators in this simplified scenario using existing LHC searches [18] and recent work explored the matching between the off-shell EW-scale operator basis and a low-energy on-shell basis [19, 20].

On the other hand, approaches like the so-called neutrino non-standard interactions (NSI) and general neutrino interactions (GNI) are incorporating right-handed neutrinos to the SMEFT [21, 22] considering their Majorana and/or Dirac nature. Also effective neutrino long-range interactions are being studied [23] in an EFT approach.

The presence of a Majorana mass term for the right-handed neutrinos is the source of lepton number violation (LNV) in these models. While the phenomenology of LNV has been thoroughly studied in the past in the context of seesaw scenarios, for recent reviews see [24, 25], lepton number violating and conserving (LNC) interference effects in final states with light neutrinos are usually discarded. Here we assess the chances of disentangling possible contributions from effective operators with distinct Dirac–Lorentz structure to B leptonic decays where the final neutrinos do not allow for the identification of LNV or LNC interactions.

In this article we focus on the simplified scenario with only one heavy Majorana neutrino N, and we neglect the effect of the renormalizable Yukawa term $N L \phi$ giving the heavy–active neutrino mixings $U_{l N}$, given that it is strongly constrained not only by the naive seesaw relation $U_\mathrm{lN}^2 \sim m_{\nu }/M_N \sim 10^{-14}$–$10^{-10}$ required to account for the light $\nu$ masses [25, 26], but also by the experimental constraints on a toy-like model in which the SM is extended with a massive Majorana neutral fermion, assumed to have non-negligible mixings with the active states, without making any hypothesis on the neutrino mass generation mechanism [27, 28]. Such a minimal SM extension leads to contributions to LNV observables which are already close, or even in conflict, with current data from meson and tau decays, for masses $M_N$ below 10 GeV (see [27, 29] and the references therein).

Our group has studied the Majorana N decays [30, 31] and phenomenology mostly in colliders for masses $m_N$ above the EW-scale [32,33,34,35,36] focusing on LNV processes, and in the order 1–10 GeV scale, where its decay is dominated by the $N\rightarrow \nu \gamma$ channel [37], which was also considered in [38].

The pure leptonic and radiative decays of pseudo-scalar mesons mediated by a sterile N in this effective scenario were initially studied in [39], where the authors only took into account vectorial operators contributions. The study of N-mediated lepton number violation in rare B meson decays has been pursued, for example, in [26, 27, 40,41,42,43,44,45,46,47,48] and the references therein. Also, the tension with the SM values of the ratios of branching fractions R(D) and $R(D^*)$ in semileptonic B decays [49], measured at Belle, BaBar and LHCb, has led to proposals involving sterile neutrinos as solutions, in seesaw scenarios [50] and also including EFTs with right-handed neutrinos [51], with sterile N interactions mediated by leptoquarks [52] or a $W^{\prime }$ $SU(2)_L$ singlet vector boson [53, 54].

In this work we go on studying the effect on the B meson decays of the presence of a Majorana N with effective interactions. In a previous paper [55] we studied the constraints imposed by the bound on the $B^{-}\rightarrow \mu ^{\pm }\mu ^{\mp }\pi ^{+}$ decay by LHCb [56] and in the radiative leptonic $B\rightarrow \mu \nu \gamma$ decay by Belle [57] on the effective operators. Here we propose to use final tau polarization and angular observables to disentangle the possible contributions of a Majorana neutrino N with effective interactions to the $B^{-} \rightarrow \tau ^{-} \nu$, the radiative $B\rightarrow \mu \nu \gamma$ and the LNV $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$ decays in future experiments.

The paper is organized as follows. In Sect. 1.1 we introduce the effective Lagrangian formalism for the Majorana N, discuss the existing bounds on the couplings weighting the different effective operators contributions Sect. 1.2, and review the experimental prospects for B meson leptonic decays Sect. 1.3. In Sect. 2 we study the B tauonic decay. The radiative $B\rightarrow \ell \nu \gamma$ decay is assessed in Sect. 3 and the LNV $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$ in Sect. 4. We summarize our results in Sect. 5.

1.1 Effective interactions formalism

We extend the SM Lagrangian including only one right-handed neutrino $N_R$ with a Majorana mass term, giving a relatively light massive state N as an observable degree of freedom. The new physics effects are parameterized by a set of effective operators ${\mathcal {O}}_{\mathcal {J}}$ constructed with the SM and the Majorana neutrino fields and satisfying the SU(2)$_L \otimes U(1)_Y$ gauge symmetry [8, 10, 58].

The effect of these operators is suppressed by inverse powers of the new physics scale $\Lambda$. The total Lagrangian is organized as follows:

$$\begin{aligned} {\mathcal {L}}={\mathcal {L}}_\mathrm{SM}+\sum _{n=5}^{\infty } \frac{1}{\Lambda ^{n-4}}\sum _{{\mathcal {J}}} \alpha _{{\mathcal {J}}} {\mathcal {O}}_{{\mathcal {J}}}^{(n)} \end{aligned}$$

(1)

where n is the mass dimension of the operator ${\mathcal {O}}_{{\mathcal {J}}}^{(n)}$.¹

Table 1 Basis of dimension-6 baryon (and lepton) number conserving operators with a right-handed neutrino N [8, 10]. Here $l_i$, $u'_i$, $d'_i$ and $L_i$, $Q'_i$ denote, for the family labeled i, the right-handed SU(2) singlet and the left-handed SU(2) doublets, respectively (collectively $f_i$). The field $\phi$ is the scalar doublet, $B_{\mu \nu }$ and $W_{\mu \nu }^I$ represent the $U(1)_{Y}$ and SU(2)$_{L}$ field strengths, respectively. Also $\sigma ^{\mu \nu }$ is the Dirac tensor, $\gamma ^{\mu }$ are the Dirac matrices, and $\epsilon =i\sigma ^{2}$ is the antisymmetric symbol in two dimensions. Types V, S and T stand for scalar, vectorial and tensorial (one-loop level generated) structures

The dimension-5 operators were studied in detail in [9]. These include the Weinberg operator ${\mathcal {O}}_{W}\sim (\bar{L}\tilde{\phi })(\phi ^{\dagger }L^{c})$ [59] which contributes to the light neutrino masses, ${\mathcal {O}}_{N\phi }\sim (\bar{N}N^{c})(\phi ^{\dagger } \phi )$ which gives Majorana masses and couplings of the heavy neutrinos to the Higgs (its LHC phenomenology has been studied in [12,13,14]), and the operator ${\mathcal {O}}^{(5)}_{NB}\sim (\bar{N}\sigma _{\mu \nu }N^{c}) B^{\mu \nu }$ inducing magnetic moments for the heavy neutrinos, which is identically zero if we include just one sterile neutrino N in the theory.

In the following, as the dimension-5 operators do not contribute to the processes studied—discarding the heavy–light neutrino mixings—we will only consider the contributions of the dimension-6 operators, following the treatment presented in [8, 10], and shown in Table 1.

The effective operators above can be classified by their Dirac–Lorentz structure into scalar, vectorial and tensorial. The couplings of the tensorial operators are naturally suppressed by a loop factor $1/(16\pi ^2)$, as they are generated at one-loop level in the UV complete theory [8, 60].

In this paper we will consider the B decays $B\rightarrow \tau N$ in Sect. 2, $B^- \rightarrow \mu ^- \nu \gamma$ in Sect. 3 and $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$ in Sect. 4, mediated by a Majorana neutrino N. We can thus take into account the following effective Lagrangian terms involved in those processes:

$$\begin{aligned} {\mathcal {L}}^\mathrm{tree}= & {} {\mathcal {L}}_\mathrm{SM} + \frac{1}{\Lambda ^2} \sum _{i,j}\Big \{ -\alpha ^{(i)}_W \frac{ ~v ~m_W}{\sqrt{2}}\, \overline{l_i} \gamma ^{\nu } P_R N \, W^{-}_{\mu }\nonumber \\&+ \alpha ^{(i)}_{V_0} \, \overline{u'_j} \gamma ^{\nu } P_R d'_j \, \, \overline{l_i} \gamma _{\nu } P_R N \nonumber \\&+ \alpha ^{(i)}_{S_1}\, \, \overline{u'_j} P_L d'_j \, \, \overline{l_i} P_R N - \alpha ^{(i)}_{S_2}\, \overline{u'_j} P_R d'_j \, \, \overline{l_i} P_R N \nonumber \\&+ \alpha ^{(i)}_{S_3}\, \overline{u'_j} P_R N \, \, \overline{l_i} P_R d'_j + \text{ h.c. } \Big \} \end{aligned}$$

(2)

and

$$\begin{aligned} {\mathcal {L}}^\mathrm{1-loop}= & {} -i \frac{\sqrt{2} v}{\Lambda ^2} (\alpha _\mathrm{NB}^{(i)}c_W \nonumber \\&+ \alpha _\mathrm{NW}^{(i)}s_W) (P^{(A)}_{\mu } ~{\bar{\nu }}_{L,i} \sigma ^{\mu \nu }N_R~ A_{\nu }). \end{aligned}$$

(3)

The one-loop generated effective Lagrangian contributes to the $N\rightarrow \nu \gamma$ decay. Here $-P^{(A)}$ is the 4-momentum of the outgoing photon and $s_W$ and $c_W$ are the sine and cosine of the Weinberg mixing angle.

The tree-level Lagrangian in (2) includes a vectorial interaction stemming from the operator ${\mathcal {O}}^{(i)}_{LN\phi }$ as the reader can read from Table 1. This structure, multiplying the coupling $\alpha ^{(i)}_W$ in (2) is the low-energy reduction of this dimension-6 current–current operator, and should not be confused with a gauge coupling. This interaction term will be considered in Sect. 1.2 in order to exploit existing bounds on heavy N interactions.

As we would like to disentangle the kind of new physics contributing to the Majorana neutrino interactions, for the numerical analysis, explained in Sect. 1.2, we will consider different benchmark scenarios for the effective couplings, where we switch on/off the operators with distinct Dirac–Lorentz structure: vectorial, scalar and the tensorial one-loop generated operators. These are defined in Table 2.

In the effective four-fermion terms in (2) the quark fields are flavor eigenstates with family $j=1,2,3$. In order to find the contribution of the effective Lagrangian to the $B^{-}$ decays we are studying, we must write it in terms of the massive quark fields. Thus, we consider that the contributions of the dimension-6 effective operators to the Yukawa Lagrangian are suppressed by the new physics scale with a factor $\frac{1}{\Lambda ^2}$, and neglect them, so that the matrices that diagonalize the quark mass matrices are the same as in the pure SM.

Table 2 Effective operators benchmark sets

Writing with a prime symbol the flavor fields, we take the matrices $U_{R}, ~U_{L}, D_{R}$ and $D_{L}$ to diagonalize the SM quark mass matrix in the Yukawa Lagrangian. Thus the left- and right-handed quark flavor fields (subscript j) are written in terms of the massive fields (subscript $\beta$):

$$\begin{aligned} u^{'}_{(R,L) j}= & {} U_{(R,L)}^{j,\beta } u_{(R,L) \beta } \;, \; \; \; \; \; \overline{u^{'}_{(R,L)j}}=\overline{u_{(R,L)\beta }} (U_{(R,L)}^{j, \beta })^{\dagger } \nonumber \\ d^{'}_{(R,L) j}= & {} D_{(R,L)}^{j,\beta } d_{(R,L) \beta } \;, \; \; \; \; \; \overline{d^{'}_{(R,L)j}}=\overline{d_{(R,L)\beta }} (D_{(R,L)}^{j, \beta })^{\dagger }.\nonumber \\ \end{aligned}$$

(4)

With this notation, the SM $V_{CKM}$ mixing matrix corresponds to the term $V^{\beta \beta '} = \sum _{j=1}^{3} (U_{L}^{j \beta })^{\dagger }D_L^{j \beta '}$ appearing in the charged SM current $J^{\mu }_{CC}= \overline{u_{\beta }} ~V^{\beta \beta '} \gamma ^{\mu } P_L d_{\beta '}$.

Thus in the mass basis the tree-level generated Lagrangian in (2) can be written in terms of the massive quarks as

$$\begin{aligned} {\mathcal {L}}^\mathrm{tree}= & {} {\mathcal {L}}_{SM} + \frac{1}{\Lambda ^2} \left\{ -\sum _{i} \alpha ^{(i)}_W \frac{ ~v ~m_W}{\sqrt{2}}\, \overline{l_i} \gamma ^{\nu } P_R N \, W^{-}_{\nu }\right. \nonumber \\&+ \sum _{i,j} \left( \alpha ^{(i)}_{V_0}~ U_{R}^{\beta j~*} ~D_{R}^{j \beta '} \, {\overline{u}}_{\beta } \gamma ^{\nu } P_R d_{\beta '} \, \, \overline{l_i} \gamma _{\nu } P_R N \right. \nonumber \\&\left. + \alpha ^{(i)}_{S_1}~ U_{R}^{\beta j~*} ~D_{L}^{j \beta '} \, \, {\overline{u}}_{\beta } P_L d_{\beta '} \, \, \overline{l_i} P_R N \right. \nonumber \\&\left. - \alpha ^{(i)}_{S_2}~ U_{L}^{\beta j~*} D_{R}^{j \beta '} \, {\overline{u}}_{\beta } P_R d_{\beta '} \, \, \overline{l_i} P_R N\right. \nonumber \\&\left. \left. + \alpha ^{(i)}_{S_3}~ U_{L}^{\beta j~*} D_{R}^{j \beta '} \,\, {\overline{u}}_{\beta } P_R N \, \, \overline{l_i} P_R d_{\beta '} \right) +\text{ h.c. } \right\} . \end{aligned}$$

(5)

For the sake of simplicity, we will rename the new quark mixing matrix element products as

$$\begin{aligned} Y^{\beta \beta '}_{RR}\equiv & {} \sum _{j} U_{R}^{\beta j*} ~D_{R}^{j \beta '}, \quad Y^{\beta \beta '}_{RL} \equiv \sum _{j} U_{R}^{\beta j*} ~D_{L}^{j \beta '}, \nonumber \\ Y^{\beta \beta '}_{LR}\equiv & {} \sum _{j} U_{L}^{\beta j*} D_{R}^{j \beta '}. \end{aligned}$$

(6)

These new quark flavor-mixing matrices combinations $Y^{\beta \beta '}$ are unknown, and in principle their entries may be found by independent measurements, as is done in the case of the SM $V_{CKM}$ matrix. In this occasion we will make an ansatz and consider that all the $Y^{\beta \beta '}$ values in (6) shall be of the order of the SM $V^{\beta \beta '}_{CKM}$ value, taking it as a measure of the strength of the coupling between the respective quarks: $Y^{\beta \beta '}_{RR} = Y^{\beta \beta '}_{RL} = Y^{\beta \beta '}_{LR} \approx V_{CKM}^{\beta \beta '}$. This will be done in the numerical calculations, while we leave the explicit expressions in our analytical equations.²

1.2 Numerical bounds on the effective couplings

Besides depending on the quark mixing matrices $Y^{\beta \beta '}$ values, all the quantities we calculate of course depend on the numerical values of the effective couplings $\alpha _{{\mathcal {J}}}^{(i)}$, which weight the contribution of each operator ${\mathcal {O}}_{{\mathcal {J}}}^{(i)}$ in Table 1 to the Lagrangian (1). Their numerical value can be constrained considering the current experimental bounds on the active–heavy neutrino mixing parameters $U_{\ell N}$ in low scale minimal seesaw models appearing in the charged (V–A) interactions N L W when neutrino mixing is taken into account [26]. Inspired by this interaction we consider the combination

$$\begin{aligned} U^2=(\alpha v^2 /(2 \Lambda ^2))^2 \end{aligned}$$

(7)

which is derived from the Lagrangian term coming from the operator ${\mathcal {O}}^{(i)}_{LN\phi }$ in Table 1 and allows a direct comparison with the mixing angles in the Type I seesaw scenarios [26]. Updated reviews of the existing bounds on the Type I seesaw mixings $U_{\ell N}$ can be found in Refs. [40, 61].

Some of the operators involving the first fermion family (with indices $i=1$) are strongly constrained by the neutrinoless double beta decay bounds, currently obtained by the KamLAND-Zen collaboration [62]. Following the treatment already made in [31], the values of the $0\nu \beta \beta$-decay constrained couplings $\alpha ^{(1)}_W,\,\alpha ^{(1)}_{V_0},\,\alpha ^{(1)}_{S_{1,2,3}}$ are taken as equal to the bound $\alpha ^b_{0\nu \beta \beta }=3.2\times 10^{-2}\left( \frac{m_N}{100\, \mathrm{GeV}} \right) ^{1/2}$ for $\Lambda =1$ TeV. These operators, although not directly involved in the processes we consider in this work, appear as contributions to the $\Gamma _N$ total width. The numerical treatment of the second and third fermion family effective couplings $\alpha _{{\mathcal {J}}}^{(i)}$ for $i=2,3$ (involved in the processes studied in this work) is explained below.

As explained above, for the numerical analysis we will consider different benchmark scenarios for the effective couplings, where we switch on/off the operators with distinct Dirac–Lorentz structure: vectorial, scalar and the tensorial one-loop generated operators. If we call (V, S, T) the factors multiplying the vectorial, scalar and tensorial one-loop generated operators, respectively, we can define six sets, presented in Table 2. Throughout the paper we will take all the vectorial couplings numerical values equal to a generic $\alpha _{V}^{(i)}$, the scalar couplings as $\alpha _{S}^{(i)}$ and the tensorial ones as $\alpha _{T}^{(i)}$ for $i=2,3$. In the different sections of the paper, we will take complementary approaches to assigning these values, that will be explained in a timely manner. However, we will use the sets defined in Table 2 to see which operators are taken into account in each case.

For the calculation of the total decay width of the Majorana neutrino $\Gamma _{N}$ we include all the kinematically allowed channels for $m_N$ in the range 1  GeV$< m_N < 5$ GeV. The details of the calculation are described in our Ref. [31]. The sets 1, 2 and 3 in Table 2 take into account the contributions of the tensorial one-loop-generated effective couplings in (3) to the N decay width. In particular these sets allow for the existence of the $N\rightarrow \nu \gamma$ decay channel. As we found in [31], this channel gives the dominant contribution to the N decay width for the low mass $m_N$ range considered in this work. The sets 4, 5 and 6 discard this contribution. The total $\Gamma _N$ width is around three orders of magnitude higher in the sets 1, 2 and 3 than in the sets 4, 5 and 6 [55].

1.3 B meson leptonic and semileptonic decays

Many ongoing and future experiments will copiously produce B mesons, enabling the study of its leptonic and semileptonic decays. The number of B mesons is expected to be over $10^{10}$ at Belle II, over $10^{13}$ at the 300 fb$^{-1}$ luminosity LHCb upgrade, and a similar number is expected at SHiP. The estimated number of B mesons to decay in the MATHUSLA detector volume is over $10^{14}$, and the expected number of B meson pairs to be produced at the Z peak at the FCC-ee is above $10^{11}$ [40].

Measuring pure leptonic B decays is challenging. The $B \rightarrow \tau \nu$ predicted branching ratio is of order $10^{-5}$ in the SM. For muonic and electronic B decays it is much smaller due to the $m^2_{\ell }$ factor coming from helicity suppression. The spinless B meson—like the pion—prefers to decay to the heaviest possible charged lepton because balancing the spins of the outgoing leptons requires them to have the same handedness, and the neutrino forces its charged partner into the unfavored helicity. The SM decay $B^{-} \rightarrow \tau ^{-} {\bar{\nu }}_{\tau }$ (or its conjugate) has not yet been measured by one single experiment with $5\sigma$ significance, and in Sect. 2 we use the value of the combined measurement [63] to put bounds on the effective couplings $\alpha _{{\mathcal {J}}}$ involved in the $B \rightarrow \tau N$ decay. However, the prospects are for Belle II to achieve a measurement with approximately 2 ab$^{-1}$ (assuming the SM branching ratio) [64]. This would be a first step towards disentangling the interference with possible new physics, as $B \rightarrow \tau N$ decays, for instance with the aid of the final tau lepton polarization information.

The final taus are the only fermions whose polarization is accessible by means of the energy and angular distribution of its decay products. These measurements rely on the dependence of kinematic distributions of the observed tau decay products on the helicity of the parent tau. Tau lepton polarization measurements at Belle have focused on the $R_{(D^*)}$ anomaly [65, 66]. In Sect. 4 we find that measuring the final tau polarization in eventual LNV $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$ decays with $\ell ^-_{1 ~\text {and/or}~ 2}=\tau ^{-}$ could help to discern between the contributions of different types of effective operators. The Belle II prospects for measuring SM rare semi-tauonic modes as $B^{+}\rightarrow \tau ^{+} \tau ^{-} \pi ^{+}$ are discouraging, because it is very difficult to reconstruct both final taus, and the lepton flavor violating $B^{+}\rightarrow \tau ^{+} \mu ^{-} \pi ^{+}$ decays with final taus and muons are expected to be easier to measure [64]. One can hope that the LNV semi-tauonic decays could be discovered in experiments with more B meson events, as MATHUSLA and SHiP [40]. However, these would not allow for the reconstruction of the taus polarization, leaving us with the eventual measurements at the FCC-ee [67, 68] to test our proposal.

B factories as Belle II produce B mesons at the center of mass (CM) energy of the $\Upsilon (4S)$ resonance, which decays in 50% to $B^+ B^-$ pairs. The decay of the $\Upsilon (4S)$ produces B mesons in pairs: if one (the tag $B_{tag}$) is reconstructed, the rest of the event must be a B meson (the signal $B_{sig}$). In hadronic B tags all the charged and neutral particles are identified, and used to reconstruct the tag-side B meson. The efficiency of this method is low—of order $10^{-3}$—but a very pure sample of B mesons is obtained. In semileptonic B tags a charmed meson is reconstructed together with a high momentum lepton. The efficiency of this method is higher, of order $10^{-2}$, though the obtained sample is not so pure. Inclusive tagging combines the four-momenta of all particles in the rest of the event of the signal-side B candidate. The achieved tagging efficiency is usually one order of magnitude above the hadronic and semileptonic tagging approaches [64].

The SM radiative leptonic B decays have been extensively studied in the literature [69,70,71,72,73,74], as they are a means of probing the strong and weak SM interactions in a heavy meson system. While the measurement of pure leptonic B decays is very difficult due to helicity suppression and the fact of having only one detected final-state particle, the radiative modes can be larger as they escape helicity suppression, and are easier to reconstruct because of the extra real final photon. In Sect. 4 we use the bounds imposed on the effective couplings by the last Belle measurements on the $B \rightarrow \mu \nu \gamma$ partial branching fraction [57]. Now, as the signal yields for the $B \rightarrow \mu \nu \gamma$ decay are expected to be three times higher in Belle II compared to Belle [64], in Sect. 3 we explore the possibility to, if the signal is found, use angular information from the hard photon and the muon flight directions to study the interference of N mediation in this decay. We are particularly interested in the inclusive tagging technique mentioned above, which allows for the reconstruction of the $B_{sig}$ meson flight direction, and the use of its rest frame, without a high loss in efficiency, as has been recently been done in [75]. This technique could be used for implementing the angular forward–backward asymmetry $A_{FB}^{\ell \gamma }$ between the final charged lepton and photon flight directions we propose in Sect. 3.1.

2 B tauonic decay: final tau polarization

In this section we aim to study the possibility to measure the effects of the presence of a Majorana neutrino N, considering the final tau polarization in the SM decay $B^{-} \rightarrow \tau ^{-} {\bar{\nu }}_{\tau }$ and the effective $B \rightarrow \tau N$ decay. The latter mode can leave the same final tau plus missing energy signal as the SM process when the N escapes the detector before decaying into observable particles.

The current value of the branching fraction $Br(B \rightarrow \tau \nu )= (1.09 \pm 0.24)\times 10^{-4}$ [63], which averages the latest measurements by Belle [76, 77] and BaBar [78, 79] allows us to impose bounds on the effective couplings $\alpha _{{\mathcal {J}}}$ involved in the $B \rightarrow \tau N$ decay.

The expression for the effective decay width $\Gamma _{B \rightarrow \tau N}$ is given below. The details of the calculation³ can be found in our Ref. [55]. The result is

$$\begin{aligned}&\Gamma _{B \rightarrow \tau N}\nonumber \\&\quad =\frac{1}{16\pi m_B}\left( \frac{f_B m_B^2}{2 \Lambda ^2} \right) ^2 \left\{ |A_V|^2 \left[ (1+x_{\tau }^{2}-x_N^{2})\right. \right. \nonumber \\&\qquad \left. \left. \times (1-x_{\tau }^{2}+x_N^{2})-(1-x_{\tau }^{2}-x_N^{2})\right] \right. \nonumber \\&\qquad + \left. |A_S|^2\frac{(1-x_{\tau }^{2}-x_N^{2})}{(x_u+x_b)^2} \right. \nonumber \\&\qquad \left. + (A^*_S A_V+A^*_V A_S) \frac{x_{\tau }}{(x_u+x_b)}(1-x_{\tau }^{2}+x_N^{2}) \right\} \nonumber \\&\qquad \times ((1-x_{\tau }^{2}+x_N^{2})^2-4 x_N^{2})^{1/2}, \end{aligned}$$

(8)

where $x_{\tau }= m_{\tau }/m_B , \;\; x_N=m_{N}/m_B, \;\; x_{u} = m_{u}/m_B, \;\; x_{b}=m_{b}/m_B$,

$$\begin{aligned} A_V= & {} \left( \alpha ^{(3)}_{V_0} Y^{ub}_{RR} + \alpha _W^{(3)} V^{ub} \right) \;\text {and}\;\; \\ A_S= & {} \left( \alpha ^{(3)}_{S_1} Y^{ub}_{RL} + (\alpha ^{(3)}_{S_2} + \frac{1}{2} \alpha ^{(3)}_{S_3}) Y^{ub}_{LR} \right) . \end{aligned}$$

Here $f_B$ is the B meson decay constant.

The effective couplings in $A_{V,S}$—as the subscript indicates—correspond to vectorial and scalar interactions. We find that, due to the pseudo-scalar nature of the B meson, the scalar operators contribution to the $B\rightarrow \tau N$ decay width is enhanced with respect to the vectorial contribution. This leads to the presence of the quark masses $m_u$ and $m_b$ in the denominator of the scalar term in (8). In turn, this will enable us to put more stringent bounds on the scalar operators contributions to this process [55].

In order to find the allowed values for the effective couplings in this context, we compare the experimental value $Br^\mathrm{exp}(B \rightarrow \tau \nu )= (1.09 \pm 0.24)\times 10^{-4}$ to the theoretical prediction Br$^\mathrm{th}= (\Gamma ^{SM}_{B \rightarrow \tau \nu }+ \Gamma _{B \rightarrow \tau N})/\Gamma ^\mathrm{total}_{B}$ subject to the constraint $\tau _{N}> 10^{3}$ ps on the N lifetime, enforcing it to escape undetected [56].

The SM result for the branching ratio is

$$\begin{aligned} \mathrm{Br}^\mathrm{SM}_{B \rightarrow \tau {\bar{\nu }}_{\tau }} = \frac{\Gamma ^\mathrm{SM}}{\Gamma _B}= \frac{G^2_F ~ m_B ~ m^{2}_{\tau } }{8 \pi \Gamma _B} \left( 1- \frac{m^{2}_{\tau }}{m^{2}_{B}}\right) ^{2} f^2_B ~|V^{ub}|^{2}.\nonumber \\ \end{aligned}$$

(9)

The uncertainty in this calculation comes from the B meson decay constant $f_B$, the CKM mixing $V^{ub}$, the total width $\Gamma _B$ and the values of the masses. Summing in quadrature the uncertainties in the SM ($\Delta Br^\mathrm{SM}$) and experimental ($\Delta Br^\mathrm{exp}=0.24\times 10^{-4}$) values, we find the region in the $(m_N, \alpha )$ plane to be consistent with the conditions

$$\begin{aligned}&\mathrm{Br}^\mathrm{exp}-\sqrt{(\Delta \mathrm{Br}^\mathrm{SM})^2+ (\Delta \mathrm{Br}^\mathrm{exp})^2} \nonumber \\&\quad \le \mathrm{Br}^\mathrm{th} \le \mathrm{Br}^\mathrm{exp}+ \sqrt{(\Delta \mathrm{Br}^\mathrm{SM})^2+ (\Delta \mathrm{Br}^\mathrm{exp})^2} \end{aligned}$$

(10)

and $\tau _{N}> 10^{3}$ ps.

In the numerical calculation we consider all the values of the effective couplings to be equal: $\alpha _V^{(i)}=\alpha _S^{(i)}=\alpha _T^{(i)}= \alpha$, and we let them be on/off depending on the benchmark set defined in Table 2 considered. As we need the Majorana N to decay outside the detector, the sets 1, 2 and 3 do not give a contribution, as the N decays very fast if the radiative $N\rightarrow \nu \gamma$ channel is available. Thus we are left with sets 4, 5 and 6.

The allowed regions are shown in Fig. 1, for sets 4 and 6. The left panel shows the results for set 4, which includes all the scalar and vector operators, while the right panel shows the result for set 6, where only scalar interactions are active. As we mentioned above, the constraints for the vectorial operators in set 5 are loose as compared to the scalar operators, and we do not show them here. The dashed border (light gray) is obtained when we consider that only the third lepton family (tau) can have effective interactions: this is implemented numerically taking $\alpha ^{(1,2)}_{S}=\alpha ^{(1,2)}_{V}=0$, with only $\alpha ^{(3)}_{S}=\alpha ^{(3)}_{V}= \alpha \ne 0$. The dotted border (dark gray), includes the three fermion families in both sets. As can be seen in the plot, this condition reduces the allowed region, because the N lifetime is shortened when it can decay to the light charged leptons, which is not allowed if only the third family interactions are included.

Fig. 1

Tau polarization $P_{\tau }$ contours (solid black), and allowed regions in the $(m_N,\alpha )$ plane (dotted and dashed) for $B\rightarrow \tau \nu$ decay

Fig. 2

Schematic representation for the effective contribution to the decay $B^-\rightarrow \ell ^-\nu \gamma$

We now turn to calculate the polarized $B \rightarrow \tau \nu$ and $B \rightarrow \tau N$ decays. We will consider the final tau polarization observable $P_{\tau }$ in terms of the decay widths $\Gamma (h)$, with $h=\pm 1$ for the positive and negative tau helicity states. Thus we write

$$\begin{aligned} P_{\tau }=\frac{\Gamma (+1)-\Gamma (-1)}{\Gamma (+1)+\Gamma (-1)}. \end{aligned}$$

(11)

We will also assume that tau decays preserve CP invariance and therefore the distributions for $h=\pm$ anti-taus $\tau ^{+}$ follow those of $h=\mp$ taus $\tau ^{-}$.

The SM calculation result, summarized in Appendix A.1, is $P_{\tau }^\mathrm{SM}=1$. The expression for the polarized decay width $\Gamma (h)_{B\rightarrow \tau N}$ is

$$\begin{aligned} \Gamma (h)_{B\rightarrow \tau N}= & {} \frac{1}{32\pi m_B}\left( \frac{f_B m_B^2}{2 \Lambda ^2} \right) ^2 \left\{ |A_V|^2 \left[ x_{\tau }^{2}+x_N^{2}\right. \right. \nonumber \\&\left. \left. -(x_{\tau }^{2}-x_N^{2})(x_{\tau }^{2}-x_N^{2}+ h R)\right] \right. \nonumber \\&+ \left. |A_S|^2\frac{1}{(x_u+x_b)^2} (1-x_{\tau }^{2}-x_N^{2}-h R)\right. \nonumber \\&\left. + (A^*_S A_V+A^*_V A_S) \frac{x_{\tau }}{(x_u+x_b)}(1\right. \nonumber \\&\left. -x_{\tau }^{2}+x_N^{2}-h R) \right\} \nonumber \\&\times ((1-x_{\tau }^{2}+x_N^{2})^2-4 x_N^{2})^{1/2}, \end{aligned}$$

(12)

where $R=[(1-x_N^2)^2-2(1+x_N^2)x_{\tau }^2+x_{\tau }^4]^{1/2}$ and we use the notation in (8). We find that the pure scalar contribution in the second term is divided by the squared sum of the quark mass quotients $x_u$ and $x_b$. This enhances its contribution (and also the contribution of the second mixed term) compared to the pure vectorial contribution of the first term.

The contour curves in the $(m_N, \alpha )$ plane that give values of $P_{\tau }=0.8, ~0.85, ~0.9, ~0.95$ are shown in Fig. 1 for sets 4 and 6 in solid black lines. While set 4 takes into account both the vectorial and the scalar operators contribution to the to the polarized $B\rightarrow \tau N$ decay, set 5 only considers the vectorial contributions, and set 6 the scalars. As we saw from Eq. (12), the departure from the SM value of the polarization ($P^\mathrm{SM}_{\tau }=1$) is expected to be dominated by the scalar contribution present in sets 4 and 6.

3 B radiative decay: lepton–photon angular asymmetry

In this section we investigate the sensitivity of the forward–backward asymmetry between the flight directions of the final lepton and photon in $B\rightarrow \ell \nu \gamma$ to the different kinds of—vectorial or scalar—effective interactions.

For the case of the B leptonic radiative decay, we consider the lepton number conserving (LNC) and violating (LNV) contributions to $B^{-}\rightarrow \ell ^{-} {\bar{\nu }} \gamma$ and $B^{-}\rightarrow \ell ^{-} \nu \gamma$, as in the experiment we cannot distinguish between neutrinos and anti-neutrinos in the final state.

In the case of a specific final lepton flavor, taking $\ell =\mu$, we consider the amplitude in the SM, which conserves lepton number: $B^{-} \rightarrow \mu ^{-} {\bar{\nu }}_{\mu } \gamma$ (${\mathcal {M}}^\mathrm{SM}_{{\bar{\nu }}_{\mu }}$) and the one mediated by the Majorana neutrino represented in Fig. 2. This effective process gives a LNC contribution to the same final state (${\mathcal {M}}^{N}_{{\bar{\nu }}_{\mu }}$) together with LNC contributions to final states with the other two anti-neutrino flavors (${\mathcal {M}}^{N}_{{\bar{\nu }}_{(e,\tau )}}$) and the LNV $B^{-} \rightarrow \mu ^{-} \nu _{j} \gamma$ (${\mathcal {M}}^{N}_{LNV}$) with final neutrinos. The first two must be added coherently⁴ in order to calculate the squared matrix element, while the last must be added incoherently. Thus we write

$$\begin{aligned} |{\mathcal {M}}_{B^{-}\rightarrow \ell ^{-} \nu ({\bar{\nu }}) \gamma }|^2= & {} | {\mathcal {M}}^\mathrm{SM}_{{\bar{\nu }}_{\mu }} + {\mathcal {M}}^{N}_{{\bar{\nu }}_{\mu }}|^2 \nonumber \\&+ | {\mathcal {M}}^{N}_{{\bar{\nu }}_{(e,\tau )}}|^2 + | {\mathcal {M}}^{N}_{{LNV}}|^2. \end{aligned}$$

(13)

The SM amplitude is written following the treatment in [69]:

$$\begin{aligned} {\mathcal {M}}^\mathrm{SM}= & {} e G_F V_{ub} \sqrt{2} \left( \overline{u_{\ell }}(p_{\ell }\right) \gamma ^{\alpha } v_{\nu }(p_{\nu }))~ \varepsilon ^{* \delta }(k) \nonumber \\&\times \left[ f_V ~\epsilon _{\alpha \delta \rho \omega }~ v^{\rho } k^{\omega } -i f_A ~(g_{\alpha \delta }~ v.k - v_{\alpha } k_{\delta })\right] , \end{aligned}$$

(14)

with the B meson momentum $q^{\rho }= m_B v^{\rho }$, $p_{\ell }$ and $p_{\nu }$ the final lepton and anti-neutrino momenta, and k the photon momentum. The constants $f_V$ and $f_A$ are the vector and axial-vector form factors in [69, 73], e is the electron charge and $G_F$ is the Fermi constant.

The LNC Majorana contribution to the amplitude is

(15)

and the LNV part is

(16)

Here $p_{N}$ is the intermediate Majorana momentum and $P_{N}(p_{N}^{2})= 1/(p_{N}^{2}-m_N^2)$. The coefficients are $~{\mathbf {C}}=\frac{v \sqrt{2}}{\Lambda ^4}(\alpha _{NB}^{(j)} c_W + \alpha _{NW}^{(j)} s_W)$,

$$\begin{aligned} C_{V}= & {} \left( \alpha _{V_0}^{(i)} Y^{ub}_{RR} + \alpha _{W}^{(i)} V^{ub}\right) \;\text {and}\;\; \\ C^{B}_{S}= & {} \left( \alpha _{S_1}^{(i)} Y^{ub}_{RL}+ (\alpha _{S_2}^{(i)}+ \frac{1}{2} \alpha _{S_3}^{(i)}) Y^{ub}_{LR} \right) \frac{m_B^2}{m_{u}+m_{b}}. \end{aligned}$$

The indices (j) in ${\mathbf {C}}$ correspond to the final (anti-)neutrino flavor, and (i) in $C_V$ and $C^{B}_{S}$ to the final charged lepton flavor. A sum over the corresponding indices $j=1,2,3$ in (16) is understood. The details of the calculation can be found in Appendix C in Ref. [55].

With the amplitudes above, we perform a phase space Monte Carlo integration, and we use it to calculate a forward–backward asymmetry between the final charged lepton and photon flight directions in the B rest frame, which can be used as an observable to disentangle the effective Majorana from the SM contribution to the leptonic radiative B decay.

3.1 Forward–backward lepton–photon asymmetry

The forward–backward asymmetry $A_\mathrm{FB}^{\ell \gamma }$ between the final charged lepton and photon flight directions is defined as

$$\begin{aligned} A_\mathrm{FB}^{\ell \gamma }= \frac{N_{+}-N_{-}}{N_{+}+N_{-}}, \end{aligned}$$

(17)

where $N_{\pm }$ is the number of events with a positive (negative) value of $\cos (\theta )$, the angle between the final lepton and the photon flight directions in the B meson rest frame.

In Fig. 3 we show our results for the value of the $A_\mathrm{FB}^{\ell \gamma }$ asymmetry for the SM process $B^{-}\rightarrow \mu ^{-} {\bar{\nu }}_{\mu } \gamma$ considering a final muon, for each photon energy bin $E_{\gamma }$ (black dots with error bars). The SM asymmetry value is negative in all the allowed photon energy range. The photon has maximum energy when the muon and neutrino have parallel momenta, opposed to $\vec {p_{\gamma }}$. In this case, the asymmetry value is strictly $A_\mathrm{FB}^{\ell \gamma }=-1$, as can be seen in the plot. In the massless muon limit, the energy allowed for the photon in the B rest frame is

$$\begin{aligned} E_{\gamma }= \frac{m_B}{2}+ \frac{E_{\mu }E_{\nu }}{m_B}\left( \cos (\varphi )-1\right) \end{aligned}$$

(18)

where $\varphi$ is the angle between the muon and neutrino momenta $\vec {p_{\mu }}$ and $\vec {p_{\nu }}$. Thus the photon has maximum energy when the muon and neutrino have the same flight directions in the B rest frame, meaning the photon and muon momenta are back-to-back.

The error bars $\delta _i$ for the SM calculation in each photon energy bin are obtained by adding in quadrature the statistical and theoretical uncertainties in the numbers of events $N_{\pm }$. The leading theoretical error comes from the uncertainty in the values of the form factors $f_A$ and $f_V$ in (14), which are taken from [73].

Fig. 3

$A_\mathrm{FB}^{\ell \gamma }$ asymmetry for the SM process $B^{-}\rightarrow \mu ^{-} {\bar{\nu }}_{\mu } \gamma$, for each photon energy bin $E_{i}$

Fig. 4

$B^-\rightarrow \mu ^- \nu \gamma$ kinematics in the B rest frame

3.1.1 Sensitivity to effective couplings

The numerical value of the effective contribution to the forward–backward asymmetry $A_\mathrm{FB}^{\ell \gamma }$ depends on the Majorana neutrino mass $m_N$ and the effective couplings $\alpha _{{\mathcal {J}}}$.

If the number of events in Belle II allows us to detect the $B^{-}\rightarrow \mu ^{-} \nu ({{\bar{\nu }}}) \gamma$ process, and measure the forward–backward lepton–photon asymmetry in (17), we would like to see which values of the effective couplings $\alpha _{{\mathcal {J}}}$ would be compatible (or not) with the SM prediction.

As we saw previously, the effective radiative decay can only occur when the tensorial couplings $\alpha _{T}$ are included in the calculation. However, the vector and scalar contributions to the numerical value of the asymmetry can change, predicting different results. For the sake of concreteness, we will consider a scenario where the tensorial couplings take a numerical value as $\alpha _{T}= \frac{|\alpha _{V}|+|\alpha _S|}{2}$ to avoid unnatural cancelations, and let the numerical values of $\alpha _V$ and $\alpha _S$ change freely. This means that when both $\alpha _V=\alpha _S=0$ there is no effective contribution to $A_\mathrm{FB}^{\ell \gamma }$.

In Fig. 3a we show the result for the sum of the effective and SM contributions $A^\mathrm{Tot}_\mathrm{FB}$ to the asymmetry for four different $m_N$ values taking into account only scalar contributions, fixing $\alpha _S=1$ and $\alpha _V=0$. Figure 3b shows the result for pure vectorial contributions, fixing $\alpha _S=0$ and $\alpha _V=1$.

The different curves in both panels in Fig. 3 show that the effective contribution to the asymmetry is moderate, except for an interval in the photon energy $E_{\gamma }$ range, where it steps toward less negative values. This means that, for this energy interval, some of the effective events contribute with a positive value of $\cos (\theta )$ to the asymmetry. This interval is different for different $m_N$ values.

The effect has kinematical reasons. In the effective $B \rightarrow \mu N \rightarrow \mu \nu \gamma$ decay, for each intermediate $m_N$ mass, the photon energy $E_{\gamma }$ in the B meson rest frame can take the values shown in Fig. 4a. This is due in part to the cut value $E_\mathrm{cut}=1$ GeV introduced to ensure the validity of the QCD treatment in the calculation [73], and to the minimal allowed energy $E_{\gamma }^\mathrm{min}=E_N (1+ \beta _N)/2$ the photon can have in the B rest frame, according to the energy $E_N$, boost factor $\gamma _N$ and boost velocity $\beta _N$ of the intermediate Majorana neutrino. The maximal energy is given by the formula corresponding to (18) for $m_{\mu } \ne 0$. Thus, the effective contribution to the asymmetry only exists in the region between the curves in Fig. 4a. The minimum value of $E_{\gamma }$ in Fig. 4a for each $m_N$ is the value where the effective contribution starts [55]. This contribution stops to rise the asymmetry value when the scalar product $\vec {p_{\mu }}. \vec {p_{\gamma }}= |\vec {p_{\mu }}| |\vec {p_{\gamma }}| \cos (\theta )$ starts to be negative again:

$$\begin{aligned} \cos (\theta )= \frac{1}{\beta _{N}}\left( \frac{m_N}{2 \gamma _N E_{\gamma }}-1\right) . \end{aligned}$$

(19)

In Fig. 4b, we plot Eq. (19) for the $m_N$ values in Fig. 3 as an aid to visualize the $E_{\gamma }$ value where the sign of $\cos (\theta )$ starts to be negative.

Fig. 5

Sensitivity of $A_\mathrm{FB}^{\ell \gamma }$ to the effective couplings

Let us consider the curves for $m_N=4$ GeV in Fig. 3. The step in both panels starts at $E_{\gamma }= 1.5$ GeV, which is the minimum value we obtain from the curve in Fig. 4a. At this energy $\cos (\theta )$ is positive (it is entering the allowed range $|\cos (\theta )|\le 1$), and it changes sign when $E_{\gamma }=1.95$ GeV, as can be seen in Fig. 4b. Here the step in the curves in Fig. 3 comes to an end. We also note that, when we consider only vectorial operators, in Fig. 3b, the curve for $m_N=4$ GeV stands between the SM value error bars, while the scalar contribution in Fig. 3a increases above them. This can be studied by considering a $\Delta \chi ^2$ distribution to see which values of the effective couplings $\alpha _{{\mathcal {J}}}$ would be compatible (or not) with the SM prediction. We construct a $\Delta \chi ^2$ function by

$$\begin{aligned} \Delta \chi ^2= \sum _{E_i}\frac{ \left( A^\mathrm{SM}_\mathrm{FB}(E_i)- A^\mathrm{Tot}_\mathrm{FB}(E_i, m_N,\alpha _V, \alpha _S)\right) ^2}{\delta _i^2}, \end{aligned}$$

(20)

where $E_i$ are the photon energy bins shown in Fig. 3, and $\delta _i$ are the errors for the SM prediction, and $A^\mathrm{Tot}_\mathrm{FB}$ is the sum of the SM and the effective value for $A_\mathrm{FB}^{\ell \gamma }$ in each photon energy bin. In Fig. 5 we show the contours in the $(\alpha _S,\alpha _V)$ plane corresponding to a coverage probability $(1-\alpha )$ of 68.27% ($1 \sigma$), 90% and 95% from the SM prediction for $A_\mathrm{FB}^{\ell \gamma }$, for different $m_N$ values.

Let us consider the panel for $m_N=4$ GeV in Fig. 5c. The point $\alpha _S=0, \alpha _V=1$ lies in the region inside the $1\sigma$ contour, and thus the effective contribution to $A^\mathrm{Tot}_\mathrm{FB}$ lies between the error bars in Fig. 3b, while the point $\alpha _S=1, \alpha _V=0$ is far outside the outer contour, and the effective contribution can be well distinguished from the pure SM curve in Fig. 3a. The same effect occurs for other $m_N$ values: the asymmetry is more sensitive to the scalar interactions contribution. This can be seen easily by inspection of Eqs. (16) and (15). The scalar contribution term $C^B_S$ is weighted by a factor of order $m_B/(m_u+m_b)$ with respect to the vectorial contribution, and this is again due to the pseudo-scalar nature of the B meson.

Fig. 6

Schematic representation for the effective contribution to the decay $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$

Table 3 Kinematic regions

Fig. 7

Final-state polarization $P_{\tau }$ for the different kinematic regions I, II and III defined in Table 3. For $\lambda =1$ only vectorial interactions are considered, and for $\lambda =0$, only scalar

4 N mediated LNV B decays with final taus

In this section we study the LNV decay $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$, mediated by a Majorana neutrino, as depicted in Fig. 6. In the case where one or two of the final leptons are tau leptons, we would like to see if a measurement of their polarization could give a hint on the kind of new physics involved in the production or decay of the intermediate N. As we will see, the polarization of the final taus in these decays could be used to distinguish the vectorial and scalar operators contributions to the N production and decay vertices, complementing those proposed for a higher $m_N$ scale in [36].

As we want to keep track of the final taus polarization, we perform a calculation of the decay amplitude considering the intermediate N to be near the mass shell, but without recurring to the narrow width approximation. The details of the calculation are summarized in Appendix A.2.

The amplitude for the LNV decay $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$ can be written as

(21)

with

$$\begin{aligned} C_{V}^{(I)}= & {} \left( \alpha _{V_0}^{(i)} Y^{ub}_{RR} + \alpha _{W}^{(i)} V^{ub}\right) \\ C^{B}_{S}= & {} \left( \alpha _{S_1}^{(i)} Y^{ub}_{RL}+ (\alpha _{S_2}^{(i)}+ \frac{1}{2} \alpha _{S_3}^{(i)}) Y^{ub}_{LR} \right) \frac{m_B^2}{m_{u}+m_{b}} \end{aligned}$$

and

$$\begin{aligned} C_{V}^{(II)}= & {} \left( \alpha _{V_0}^{(j)} Y^{ud}_{RR} + \alpha _{W}^{(j)} V^{ud}\right) \\ C^{\pi }_{S}= & {} \left( \alpha _{S_1}^{(j)} Y^{ud}_{RL}+ (\alpha _{S_2}^{(j)}+ \frac{1}{2} \alpha _{S_3}^{(j)}) Y^{ud}_{LR} \right) \frac{m_{\pi }^2}{m_{u}+m_{d}}. \end{aligned}$$

The indices (i) and (j) for the effective couplings correspond to the $\ell _1$ and $\ell _2$ flavors, respectively. In order to keep track of the final leptons polarizations, we use the relations in (A1), which allow us to perform a Monte Carlo simulation and obtain the number of decay events with positively or negatively polarized taus in the final state.

We study the polarization of final taus in the processes depicted in Fig. 6. The different possible processes interchanging the final leptons flavor ($\ell _1=\tau , \mu$ and $\ell _2=\tau , \mu$) depend on the value of the mediator mass $m_N$. The possibilities for the different regions are summarized in Table 3.

In the case of only one final tau, we define the final-state polarization as

$$\begin{aligned} P_{\tau }=\frac{N_{+}-N_{-}}{N_{+}+N_{-}}=\frac{N_{+}-N_{-}}{N}, \end{aligned}$$

(22)

and the two-taus final-state polarization in $B^{-} \rightarrow \tau ^{-} \tau ^{-} \pi ^{+}$ as

$$\begin{aligned} P_{\tau }=\frac{N_{++}+N_{+-}-N_{-+}-N_{--}}{N_{++}+N_{+-}+N_{-+}+N_{--}}. \end{aligned}$$

(23)

Here the number of events ($N_{\pm }$, $N_{{\pm }{\pm } }$) with subscripts $+$ and − correspond, respectively, to the number of $h=+$ and $h=-$ polarization states of each final tau $\ell _1$ or $\ell _2$ in Fig. 6 measured in the experiment, as considered in (A2). The defined final-state polarizations, being quotients, are independent of the total number of B decay events considered.

In our previous paper [55], we studied the bounds that can be set on the different couplings $\alpha _{{\mathcal {J}}}$ in the effective dimension-6 Lagrangian (1) involved in N mediated B decays by exploiting the experimental results existing on the $B^{-} \rightarrow \mu ^{-} \mu ^{-} \pi ^{+}$ [56] and $B^{-} \rightarrow \mu ^{-} \nu \gamma$ [57] processes. Figure 6 in [55] shows different $U^2 (m_N)$ (7) curves for the upper bounds obtained for the $U^2$ combination varying with the $m_N$ mass. The most restrictive one corresponds to a benchmark scenario (set 1) where all the types of operators involved in the effective $B^{-} \rightarrow \mu ^{-} \mu ^{-} \pi ^{+}$ decay process (vectorial, scalar and tensorial) are considered to be present in the Lagrangian. In this numerical calculation we use this curve, $U^2 (m_N)$, to fix the value of the remaining couplings (generically $\alpha _{{\mathcal {J}}}$, not involved in $0\nu \beta \beta$-decay) inverting for the $\alpha (m_N)$ value in (7). While this constraint is obtained from interactions involving the $\ell = \mu , ~i=2$ family, we will use it here to fix also the $\alpha ^{(3)}_{{\mathcal {J}}}$ couplings in (21), for the sake of simplicity.⁵

To appreciate the ability of the final taus polarization to determine the kind of effective operators involved in the studied interaction, we define a parameter $\lambda \in [0,1]$ to measure the proportion of vectorial and scalar operators contributing to the processes. Thus we multiply the vector operators by $\lambda$ and the scalars by $(1-\lambda )$, so that $\lambda =1$ means pure vectorial and $\lambda =0$ means pure scalar interactions.

In our numerical code, given the value of the scalar–vector interaction content $\lambda$, we let the value of the intermediate $m_N$ mass vary in the allowed interval for each region in Table 3. After fixing the numerical values of the effective couplings ($\alpha$) and mixing matrices (Y) in (21), this gives as output an interval of values for the final-state polarization $P_{\tau }$ as defined in Eqs. (22) and (23) for one or two tau processes, consisting of a band in the ($\lambda$, $P_{\tau }$) plane.

In Fig. 7 we show the curves of maximum and minimum values giving the allowed values for the final tau polarization $P_{\tau }$ in (22) depending on the $\lambda$ parameter for Majorana neutrino masses $m_N$ in the kinematic regions I (Fig. 7a), II (Fig. 7b) and III (Fig. 7c) defined in Table 3. For the case of two final taus (23), the curves are shown in Fig. 7d.

In the experiment, such as FCC-ee as we discussed in Sect. 1.3, the first and second leptons can in principle be distinguished, as $\ell _1$ should have the higher momentum. Then, the $m_N$ mass range can be reconstructed with a measurement of the invariant mass $M(\ell _2, \pi )$. This would allow us to distinguish the three regions I, II and III.

If only one tau lepton is found in the final state, we can distinguish between regions I and III by comparing its momentum with the muon momentum. If $p_{\tau } > p_{\mu }$, we should be in region I, and the invariant mass $M(\mu , \pi )$ value should confirm this. In this case, the final tau is produced together with the N in the primary vertex (see Fig. 6). This means in this case we are probing the $C_V^{(I)}$ and $C_S^{B}$ coefficients in (21). As we find in Fig. 7a, these tau leptons are expected to be mostly negatively-polarized if we only consider scalar interactions ($\lambda =0$), and an unpolarized final tau state could be obtained in the case of purely vector interactions ($\lambda =1$). If $p_{\mu } > p_{\tau }$, we should be in region III, which can be checked with the invariant mass $M(\tau , \pi )$. Here, by measuring the tau polarization we access the N decay vertex, and test the $C_V^{(II)}$ and $C_S^{\pi }$ coefficients in (21). Figure 7c shows that finding a negative $P_{\tau }$ would point toward a dominant scalar interaction, while a positive value would signal a vector-dominated N decay.

The case of region II shows that we expect mostly a negative final-state tau polarization, indicating the dominance of the scalar interactions in both the N production and the decay vertices. This is consistent with the fact that the scalar interactions are weighted in both cases by the quark masses in the denominators in (21), which, as we mentioned in the discussion of (8), enhances the scalar operators contribution, due to the pseudo-scalar nature of both the B and the $\pi$ mesons.

5 Summary

The effective field theory extending the standard model with right-handed neutrinos (SMNEFT) parameterizes new high-scale weakly coupled physics in a model independent manner. We consider massive Majorana neutrinos coupled to ordinary matter by dimension-6 effective operators, focusing on a simplified scenario with only one right-handed neutrino added, which provides us with a manageable parameter space to probe. These massive neutrinos would be produced in leptonic and semileptonic decays of B mesons, a theoretically clean system to probe new physics effects in lepton colliders and B factories. We exploit the known bounds on B meson decays to constrain the effective parameter space, and propose to use final taus polarization and angular observables to disentangle vectorial or scalar type effective contributions.

The B meson couples preferably to the scalar operators in the left column of Table 1 in comparison to the vectorial ones. Thus, as we already found in [55], probing its decay to a Majorana neutrino N we can either put more stringent bounds or give more precise predictions on the scalar operators effective couplings.

In Sect. 2 we considered the bounds on the tauonic B decay [76,77,78,79] to constrain the allowed regions in the $(m_N, \alpha )$ plane (Fig. 1), and we find that a measure of the final tau polarization could help to separate the pure SM decay, which gives $P_{\tau }=1$, from the effective $B\rightarrow \tau N$, where the N escapes undetected. This contribution can lower the $P_{\tau }$ value to 0.8 in the allowed region (with $m_N< 3.5$ GeV) for heavy neutrinos coupling only to the third lepton family. In Table 2 we define six different numerical coupling sets which turn on/off the scalar, vector or tensorial interactions. We find that sets 4 and 6, which involve scalar N interactions, are the ones giving the more interesting results.

In Sect. 3 we define a forward–backward asymmetry $A_\mathrm{FB}^{\ell \gamma }$ between the muon and photon directions for the $B \rightarrow \mu \nu \gamma$ decay, which allows us to separate the SM lepton number conserving contribution from the effective lepton number conserving and violating processes, mediated by a near on-shell N. We find that the effective contributions tend to deviate $A_\mathrm{FB}^{\ell \gamma }$ from the SM value in photon energy intervals depending on the mass $m_N$, according to the boost of the N in the B meson rest frame (Fig. 3). This measurement could be done in Belle II with the aid of inclusive tagging techniques, which allow for the reconstruction of the signal B frame [75]. Again, the asymmetry is more sensitive to the scalar interactions, as is found in Fig. 5.

In Sect. 4 we study the final taus polarization in the lepton number violating $B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+$ decay when $\ell ^-_{1 ~\text {and/or}~ 2}=\tau ^{-}$. Using the bounds on the effective couplings obtained in [55] we probe the chances to disentangle different effective operators contributions to these decays. We weight the vectorial and scalar operators by a factor $\lambda \in [0,1]$ with $\lambda = 1$ (purely vectorial) and $\lambda = 0$ (purely scalar) contributions. In Fig. 7 we find that a measurement of the final polarization $P_{\tau }$ can help us infer the scalar or vector content in the N production or decay vertices, provided that the invariant mass $M(\ell _2, \pi )$ can be reconstructed.

The B meson provides a rich environment to search for and constrain possible high-energy physics effects in the case a GeV-mass heavy N is the only new accessible state. This study complements probes of lepton number violating same-sign dilepton signals, Higgs-neutrino interactions and other collider searches to characterize the standard model neutrino effective field theory parameter space.

Appendix A: Polarized final taus in B decays

1.1 A.1 Tau polarization in the SM B tauonic decay

In Sect. 2 we showed the $B \rightarrow \tau {{\bar{\nu }}}_{\tau }$ polarized decay result in the SM. For reference purposes, we summarize the calculation here. The SM amplitude can be written as

$$\begin{aligned} {\mathcal {M}}_{B\rightarrow \tau {{\bar{\nu }}}_{\tau }}= & {} i\frac{g^2}{2}\frac{V^{ub}}{m_W^2} \langle 0|\bar{u}\gamma ^{\beta } P_L b|B^-\rangle \langle \tau ^- {{\bar{\nu }}}_{\tau } |{\bar{\tau }} \gamma _{\beta } P_L \nu |0 \rangle \\= & {} i\frac{g^2}{2}\frac{V^{ub}}{m_W^2} i f_B q^{\beta } (\bar{u}_{\tau }(p) \gamma _{\beta } P_L v_{\nu }(k)), \end{aligned}$$

with $q\ll m_W$ the B meson momentum and $q= p+k$, the tau and anti-neutrino momenta, respectively.

We take the squared amplitude $|{\mathcal {M}}|^{2}$, considering the spin projectors for final leptons with mass m, spin s and momentum p, with $p.s=0$:

(A1)

The leptons spin vectors can be written in terms of their helicity $h=\pm$ as

$$\begin{aligned} s^{\beta }=h \left( \frac{|\vec {p}|}{m},\frac{E}{m} \frac{\vec {p}}{|\vec {p}|} \right) . \end{aligned}$$

(A2)

Thus we have, in the B meson rest frame,

$$\begin{aligned} |{\mathcal {M}}_{B\rightarrow \tau {{\bar{\nu }}}_{\tau }}|^2= G_F^2 f_B^2 {V^{ub}}^2 m_{\tau }^2 (m_B^2-m_{\tau }^2 +2 m_{\tau } ~q.s), \end{aligned}$$

which gives a polarized decay width

$$\begin{aligned} \Gamma _{B\rightarrow \tau {{\bar{\nu }}}_{\tau }}(h)= G_F^2 f_B^2 ~|V^{ub}|^2 m_B^3 x_{\tau }(1-x_{\tau }^2)(1+h), \end{aligned}$$

with $x_{\tau }=m_{\tau }/m_B$, which shows that we only have taus with $h=1$ in the final state, giving a polarization $P_{\tau }=1$ in the SM.

1.2 A.2 Final taus polarization in $B\rightarrow \ell \ell \pi$ decays

In Sect. 4 we presented the decay $B^{-} \rightarrow \ell _1^{-} \ell _2^{-} \pi ^{+}$ mediated by the Majorana neutrino N, where we considered final polarized taus $\ell ^-_{1 ~\text {and/or}~ 2}=\tau ^{-}$.

In the calculation, the amplitude ${\mathcal {M}}^{N}({B^-\rightarrow \ell ^-_{1}\ell ^-_{2} \pi ^+})$ is initially written as ${\mathcal {M}}=\langle \ell _1^{-} \ell _2^{-} \pi ^{+}| \hat{{\mathcal {O}}}|B^{-} \rangle$. The interaction $\hat{{\mathcal {O}}}= \hat{{\mathcal {O}}}_{I} \cdot \hat{{\mathcal {O}}}_{II}$ includes the terms coming from the N production vertex I in Fig. 6, considering the Lagrangian terms in (5) with the up-type quark $\bar{u}_\beta =u$, the down-type quark $d_\beta '=b$, the charged lepton $l_i=\ell _1$ and the N decay vertex II, with $\bar{u}_\beta =u$, $d_\beta '=d$ and $l_i=\ell _2$.

The leptonic and hadronic parts of the final state can be separated and one considers the intermediate N in , taking the charge conjugate of the N decay terms.

The only non-zero hadronic currents for a pseudo-scalar meson M (this includes both the B and the pion) are the pseudo-vector $\langle 0| \bar{u}\gamma ^{\mu }\gamma ^{5} d| M \rangle = i f_M q^{\mu }$ and the pseudo-scalar $\langle 0| \bar{u}\gamma ^{5} d| M \rangle = -i f_M \frac{m_{M}^2}{m_u+m_d}$ currents, where u and d are up-type and down-type quarks, $f_M$ is the meson constant, and q its momentum.

The amplitude is finally written as in (21)

(A3)

Here $p_{1,2}$ are the $\ell _{1,2}$ momenta, $f_B$ and $f_{\pi }$ are the B and $\pi$ mesons constants and

$$\begin{aligned} C_{V}^{(I)}= & {} \left( \alpha _{V_0}^{(i)} Y^{ub}_{RR} + \alpha _{W}^{(i)} V^{ub}\right) ,\\ C^{B}_{S}= & {} \left( \alpha _{S_1}^{(i)} Y^{ub}_{RL}+ (\alpha _{S_2}^{(i)}+ \frac{1}{2} \alpha _{S_3}^{(i)}) Y^{ub}_{LR} \right) \frac{m_B^2}{m_{u}+m_{b}}, \end{aligned}$$

which are the terms corresponding to the hadronic B current in the N production vertex. On the N decay side we have the terms corresponding to the hadronic $\pi$ current,

$$\begin{aligned} C_{V}^{(II)}= & {} \left( \alpha _{V_0}^{(j)} Y^{ud}_{RR} + \alpha _{W}^{(j)} V^{ud}\right) \;\;\;\; \\ C^{\pi }_{S}= & {} \left( \alpha _{S_1}^{(j)} Y^{ud}_{RL}+ (\alpha _{S_2}^{(j)}+ \frac{1}{2} \alpha _{S_3}^{(j)}) Y^{ud}_{LR} \right) \frac{m_{\pi }^2}{m_{u}+m_{d}}. \end{aligned}$$

We then take the squared amplitude and, as we did in A.1, we use Eqs. (A1) and (A2) to calculate the polarized decay width.