Searches for heavy neutrinos at multi-TeV muon collider: a resonant leptogenesis perspective

Abstract

In this work, the Standard Model (SM) is extended with two right-handed (RH) neutrinos and two singlet neutral fermions to yield active neutrino masses via (2,2) inverse see-saw mechanism. We first validate the multi-dimensional model parameter space with neutrino oscillation data, obeying the experimental bounds coming from the lepton flavor violating (LFV) decays: $\mu \rightarrow e \gamma ,~ \tau \rightarrow e \gamma , ~ \tau \rightarrow \mu \gamma$. Besides we also search for the portion of the parameter space which yield the observed baryon asymmetry of the universe via resonant leptogenesis. Further, we pick up a few benchmark points from the aforementioned parameter space with TeV scale heavy neutrinos and perform an exhaustive collider analysis of the final states: $2l + E_T \!\!\!\!\!\!/\;\,$ and $1\,l + 2j + \not \! E_T$ in multi-TeV muon collider.

1 Introduction

While the Discovery of the Higgs boson at the Large Hadron Collider (LHC) [1, 2] completes the particle spectrum of the Standard Model (SM), this spin-zero boson also confirms the mass generation mechanism of the fermions and gauge bosons via spontaneous symmetry breaking. However, an exception of the aforesaid mechanism occurs for the neutrinos owing to the absence of the counterpart of the left-handed neutrinos in SM. On the contrary to this theoretical observation, the flavor oscillations of the neutrinos yield massive active neutrinos with an upper limit of ${\mathcal {O}}$ (0.1 eV) [3] coming from cosmological observations. This tiny neutrino mass can be generated via see-saw mechanism, which requires the extension of SM with additional fermionic or bosonic degrees of freedom. Depending on the nature and representations of the extended sector, various types of see-saw mechanisms like Type-I [4,5,6,7,8], Type-II [9,10,11,12,13,14,15,16,17,18,19,20], Type-III [21,22,23,24,25,26,27], have been studied in the literature. The requirement of tiny active neutrino masses pushes the masses of the additional beyond Standard Model (BSM) fields to higher end and also put a lower bound on the masses of the same. In most of the see-saw mechanisms, the BSM fields are too heavy to be produced and analysed in the present and future collider experiments. The inverse see-saw mechanism [28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47] turns out to be very effective for addressing this problem as it can produce TeV scale heavy neutrinos which is well within the reach of future collider experiments. Heavy neutrinos have already been searched at LHC [48], LHeC and lepton colliders [49] previously. Signatures of heavy neutrinos originated from heavy $Z'$-boson has been analysed in [50].

Another shortcoming of SM causes lack of explanation of the observed baryon asymmetry of the universe. According to the current observation [3], the baryon asymmetry is:

$$\begin{aligned} \eta _B = \frac{n_B - n_{{\bar{B}}}}{n_\gamma } = (6.12 \pm 0.04) \times 10^{-10} . \end{aligned}$$

(1)

Following the Sakharov conditions [51], baryon number, C and CP symmetry as well as thermal equilibrium should be violated to provide an explanation to the dynamic generation of this asymmetry. One of the most popular mechanisms to generate this asymmetry is leptogenesis [52,53,54,55,56,57,58,59,60,61,62,63,64,65,66], where the lepton asymmetry can originate from the out-of-equilibrium decay of heavy neutrinos. This asymmetry is further translated to baryon asymmetry via sphaleron transitions in SM [67,68,69], which are basically $B-L$ conserving but $B+L$ violating processes. One of the minimal models where leptogenesis can be realised by adding heavy right-handed neutrinos (with masses $> 10^{9}$ GeV [70, 71]) to SM, can yield correct baryon asymmetry via Type-I thermal leptogenesis. Thus to probe interesting signatures involving the heavy neutrinos in various colliders, one needs to lower the masses of the heavy neutrinos at least to the TeV scale. A specific framework called inverse see-saw (ISS) serves the aforementioned purpose, where two of the mass eigenstates of the heavy neutrinos are almost mass degenerate, yielding the required baryon asymmetry via resonant leptogenesis [72,73,74,75,76,77,78,79].

In this paper, we consider a minimal inverse see-saw scenario ISS(2,2) [34], where the SM is extended by two generations of right-handed neutrinos and two SM gauge singlet neutral fermions. The phenomenology of the model in light of neutrino oscillation data [80], lepton flavor violating decay [81] and leptogenesis, has been thoroughly studied by us in one of our previous works [82]. Here, we intend to explore the parameter space derived from our previous study from collider perspective and examine the prospect of some interesting signatures at the multi-TeV muon collider [83]. As we mentioned earlier, this particular framework provides with TeV scale heavy neutrinos which can produce the baryon asymmetry of the universe in the correct ballpark, as well as can be probed at present and future colliders. Here we aim to probe the signal involving the pair production of active neutrinos along with the heavy neutrinos (masses ranging from 3.28 TeV to 10.12 TeV), followed by the subsequent decay of the heavy neutrinos to $W^\pm$ and charged leptons. Further we consider both leptonic and hadronic decay of $W^\pm$, leading to $2l+ E_T \!\!\!\!\!\!/\;\,$ and $1l + 2j + \not \! E_T$ final state respectively. We would have been opted for LHC or ILC to search for the aforementioned signal. In the context of LHC, the signal significance turns out to be low due to the presence of huge SM background and tiny signal cross-section at 14 TeV. The significance might be improved in the proposed 100 TeV collider but still, the SM background cross-section is expected to be very large compare to the signal cross-section. Besides the similar signal involving TeV scale heavy neutrinos at the production level cannot be investigated at ILC, since the maximum achievable center of mass energy (COM) at ILC is 1 TeV.

Now the possibility of detecting TeV scale neutrino can be achieved in a leptonic collider with a sufficiently high COM energy. Recently, there is a growing interest in muon colliders with multi TeV COM energy. This can provide a clean environment with hugely achievable signal cross-section, enhancing the signal significance. For our analyses, we choose multi TeV COM energies ($\sqrt{s} = 6~\textrm{TeV},14$ TeV) at the muon collider with integrated luminosity (${\mathcal {L}}$) varying as [84]:

$$\begin{aligned} {\mathcal {L}} = \left( \frac{\sqrt{s}}{10 ~ \textrm{TeV}}\right) ^2 \times 10^4 ~~ \textrm{fb}^{-1}. \end{aligned}$$

(2)

With this motivation, we analyze the $2l + E_T \!\!\!\!\!\!/\;\,$ and $1l+ 2j$ $+ \not \! E_T$ final states at multi-TeV muon collider with $\sqrt{s} = 6$ TeV and 14 TeV. We shall show how a high significance could be produced by applying suitable cuts on the relevant kinematic variables to suppress the SM backgrounds using traditional cut-based analysis. We shall also compare the performances of the two aforementioned channels at the end of the analysis.

This paper is organized as follows. In Sect. 2, we describe the framework and the mechanism of neutrino mass generation. Further, we study phenomenological constraints in the context of the model in Sect. 3. In Sect. 4, we perform cut-based analysis to analyze the collider signature of $2l + E_T \!\!\!\!\!\!/\;\,$ and $1l + 2j + E_T \!\!\!\!\!\!/\;\,$ channels at multi-TeV muon collider. Finally, we summarize and conclude in Sect. 5.

2 Model

In the present work, we will be operating in the framework of SM, minimally extended with two right-handed neutrinos $N_{{R_1}}, N_{{R_2}}$ and two neutral singlet fermions $S_1, S_2$ yielding tiny neutrino mass and mixing via inverse see-saw mechanism. Thus this model is termed as ISS(2,2) in short, where the two numbers in the first bracket denote the numbers of added heavy right-handed neutrinos and two neutral singlet fermions respectively. Table 1 shows the $SU(3)_C, ~ SU(2)_L, ~ U(1)_Y$ quantum numbers assigned to the bosonic and fermionic fields of the model. Here the hyper-charge Y is calculated using: $Q = T_3 + \frac{Y}{2}$, where $T_3$ and Q are the weak isospin and electric charge of the field respectively. In Table 1, the SM Higgs doublet is denoted by $\phi$. $Q_{L_i}, L_{L_i}$ are the left-handed SM quark and lepton doublets respectively, whereas $u_{R_i}, d_{R_i}, \ell _{R_i}$ are right-handed up-type, down-type quark and lepton singlets respectively with $i=3$.

Table 1 $SU(3)_C, ~ SU(2)_L, ~ U(1)_Y$ quantum number assigned to the particles. i denotes the number of fermionic generations

The Yukawa Lagrangian signifying the inverse see-saw mechanism is:

$$\begin{aligned} -{\mathcal {L}}_y= & {} y_{i \alpha } {\bar{N}}_{R_i} \phi ^\dag \ell _{L_\alpha } + \frac{1}{2} M_{R_{ij}} N_{R_i}^T C^{-1} N_{R_j}\nonumber \\{} & {} + M_{S_{ij}} N_{R_i}^T C^{-1} S_j + \frac{1}{2} \mu _{ij} S_i^T C^{-1} S_j + \mathrm{h.c.} \end{aligned}$$

(3)

where $\alpha$ represents the flavor of leptons. Here $y_{i \alpha }$ denotes the Yukawa coupling matrix with complex entries and C is the charge conjugation operator. The aforementioned Lagrangian contains both lepton number conserving Dirac mass term, as well as lepton number violating Majorana mass terms.

In the flavor basis $(\nu _L^i, N_{R_j}^c, S_k)^T$ (with $i=3,~ j,k=2$), following Eq. (3), the neutrino mass matrix can be written as:

$$\begin{aligned} M_\nu {=}\!\begin{pmatrix} 0 &{}\quad 0 &{}\quad 0 &{}\quad {M_D}_{1,1} &{}\quad {M_D}_{1,2} &{}\quad 0 &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad 0 &{}\quad {M_D}_{2,1} &{}\quad {M_D}_{2,2} &{}\quad 0 &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad 0 &{}\quad {M_D}_{3,1} &{}\quad {M_D}_{3,2} &{}\quad 0 &{}\quad 0 \\ {M_D}_{1,1} &{}\quad {M_D}_{2,1} &{}\quad {M_D}_{3,1} &{}\quad {M_R}_{1,1} &{}\quad {M_R}_{1,2} &{}\quad {M_S}_{1,1} &{}\quad {M_S}_{1,2} \\ {M_D}_{1,2} &{}\quad {M_D}_{2,2} &{}\quad {M_D}_{3,2} &{}\quad {M_R}_{1,2} &{}\quad {M_R}_{2,2} &{}\quad {M_S}_{2,1} &{}\quad {M_S}_{2,2} \\ 0 &{}\quad 0 &{}\quad 0 &{}\quad {M_S}_{1,1} &{}\quad {M_S}_{2,1} &{}\quad \mu _{1,1} &{}\quad \mu _{1,2} \\ 0 &{}\quad 0 &{}\quad 0 &{}\quad {M_S}_{1,2} &{}\quad {M_S}_{2,2} &{}\quad \mu _{1,2} &{}\quad \mu _{2,2} \end{pmatrix} \nonumber \\ \end{aligned}$$

(4)

Here each and every matrix element of $M_\nu$ is considered to be complex to generalise the analysis, and thus the elements can be decomposed into real and imaginary parts as:

$$\begin{aligned} {M_D}_{i,j}= & {} {M_D}_{i,j}^R + i ~{M_D}_{i,j}^I ~, ~~{M_R}_{l,m} = {M_R}_{l,m}^R + i ~{M_R}_{l,m}^I ~, \nonumber \\ {M_S}_{a,b}= & {} {M_S}_{a,b}^R + i ~{M_S}_{a,b}^I ~, ~~ \mu _{p,q } = \mu _{p,q}^R + i ~\mu _{p,q}^I \end{aligned}$$

(5)

We can write down the neutrino mass matrix $M_\nu$ in a compact form as:

$$\begin{aligned} M_\nu = \begin{pmatrix} 0 &{}\quad M_D &{} \quad 0 \\ M_D^T &{}\quad M_R &{} \quad M_S \\ 0 &{}\quad M_S^T &{} \quad \mu \end{pmatrix} \end{aligned}$$

(6)

with $M_D = y_{i\alpha } \frac{v}{\sqrt{2}}$. With three generations of active neutrinos, two generations of $N_{R_j}$ and $S_j$s, $M_\nu$ is $7\times 7$ dimensional. The individual dimensions of $M_D, M_R, M_S$ and $\mu$ are $3 \times 2, ~ 2 \times 2, ~ 2 \times 2, ~ 2 \times 2$ respectively.

Following the mass hierarchy $\mu , M_R \ll M_D \ll M_S$, with the see-saw approximation in the inverse see-saw mechanism, the effective neutrino mass matrix looks like: [85],

$$\begin{aligned} m_\nu= & {} -\begin{pmatrix} M_D&\quad 0 \end{pmatrix} \begin{pmatrix} M_R &{}\quad M_S \\ M_S^T &{}\quad \mu \end{pmatrix}^{-1} \begin{pmatrix} M_D^T \\ 0 \end{pmatrix} . \nonumber \\= & {} -M_D ~ (M_R - M_S~ \mu ^{-1} M_S^T)^{-1} ~ M_D^T . \end{aligned}$$

(7)

Thus one can neglect $M_R$ in Eq. (7) and approximate the active neutrino mass matrix as:

$$\begin{aligned} m_\nu= & {} M_D ~ (M_S^T)^{-1} ~\mu ~M_S^{-1} ~M_D^T. \end{aligned}$$

(8)

From Eq. (8), it is evident that, with $M_R = 0$,¹ double suppression by the mass scale $M_S$ along with small $\mu$, yield tiny active neutrino mass.

Upon diagonalising $m_\nu$ in Eq. (8) the light neutrino masses are generated via the transformation:

$$\begin{aligned} U_{\textrm{PMNS}}^T~ m_\nu ~U_{\textrm{PMNS}} = \textrm{diag}(m_1,m_2,m_3) = \hat{m_\nu } . \end{aligned}$$

(9)

where $m_1, m_2, m_3$ are three light active neutrino masses, $U_{\textrm{PMNS}}$ is the Pontecorvo–Maki–Nakagawa–Sakata matrix (PMNS matrix).²

3 Constraints

In this subsection, we shall briefly describe relevant constraints imposed on the model parameter space. For details of the constraints, we refer to [82].

3.1 Neutrino data fitting

Following Eq. (8), $\mu$ can be expressed in terms of the matrices $m_\nu , M_R, M_D, M_S$ as:

$$\begin{aligned} \mu = M_S^T~ M_D^{-1}~ m_\nu ~(M_D^T)^{-1}~ M_S \end{aligned}$$

(11)

Here $m_\nu$ can be expressed in terms of $U_{\textrm{PMNS}}, \hat{m_\nu }$ from Eq. (9), where the elements of the $U_{\textrm{PMNS}}$ matrix in Eq. (10) are already constrained from neutrino oscillation data [80]. Thus the model parameter space which we consider in the present analysis becomes compatible with the neutrino oscillation data.³ The texture of $M_\nu$ in Eq. (6) yields one massless active neutrino, which is an unavoidable feature of the (2,2) inverse see-saw realising framework. Thus we set the lightest active neutrino mass $m_1$ to be zero, satisfying $(m_1 + m_2 + m_3) \le 0.12$ eV [3, 86].⁴

Fig. 1

Diagrams contributing to the CP-asymmetry $\epsilon _1$ and $\epsilon _2$: a tree-level decay of $\tilde{\Psi _{1}} (\tilde{\Psi _{2}})$, b vertex correction, c self-energy diagram

3.2 Lepton flavor violation

Through the diagonalisation of mass matrix $M_\nu$ in Eq. (6) by a $7 \times 7$ unitary matrix U, one can obtain seven mass eigenstates $\nu _1', \nu _2', \nu _3', \tilde{\Psi _1},\tilde{\Psi _2},\tilde{\Psi _3},\tilde{\Psi _4}$ with masses $m_1,m_2,m_3,M_{\tilde{\Psi _1}},M_{\tilde{\Psi _2}},M_{\tilde{\Psi _3}},M_{\tilde{\Psi _4}}$ respectively. Following the mass hierarchy ($\mu \ll M_D \ll M_S$) in the inverse see-saw framework, the mass difference: $|M_{{\tilde{\Psi }}_{1(3)}}-M_{{\tilde{\Psi }}_{2(4)}}| \sim \mu$. Thus the corresponding pairs having such a tiny mass difference become mass degenerate.

Thus the lepton flavor violating (LFV) decays like $l_i \rightarrow \ l_j \gamma$ obtain additional contributions coming from the heavy neutrinos $\tilde{\Psi _1},\tilde{\Psi _2},\tilde{\Psi _3},\tilde{\Psi _4}$ [82]. Among all LFV decays the most stringent bound comes from $\mu \rightarrow e \gamma$ [81]. As we shall see later, in presence of the TeV scale heavy neutrinos, all these bounds are satisfied automatically.

3.3 Leptogenesis and baryon asymmetry

In this subsection, we briefly discuss the generation of baryon asymmetry via leptogenesis. For an exhaustive description of the mechanism, we refer to reference [82]. Considering the mass degeneracy of the pair of heavy neutrinos, the required CP-asymmetry will be generated through the out of equilibrium decay of the lightest mass degenerate pair (say $\tilde{\Psi _{1}}, \tilde{\Psi _{2}}$) via resonant leptogenesis. The diagrammatic representation of the aforementioned processes can be found in Fig. 1.

Fig. 2

Branching ratios of radiative decay of muon ($\mu \rightarrow e \gamma$) with the mass of the heavy neutrinos (a $M_{\tilde{\Psi _1}}$, b $M_{\tilde{\Psi _3}}$). Color codes are expressed in legends

Assuming $M_R=0$, $M_\nu$ becomes block diagonalisable. While computing CP-asymmetry, the preferred choice of basis is that where the lower block of the $M_\nu$ (4 $\times$ 4 complex symmetric sub-matrix ${\mathcal {M}}$⁵) is diagonal. In this preferred basis, the Lagrangian in Eq. (3) can be rewritten as,

$$\begin{aligned} -{\mathcal {L}}_h = h_{i \alpha } \bar{\tilde{\Psi _i}} \phi ^\dag \ell _{L_\alpha } + \frac{1}{2} M_{\textrm{diag}} \tilde{\Psi _i}^T C^{-1} \tilde{\Psi _i} + \mathrm{h.c.} \end{aligned}$$

(14)

The relations connecting Yukawa couplings in the diagonal mass basis ($h_{i \alpha }$) and the Yukawa couplings in the flavor basis ($y_{i \alpha }$) can be found in [82]. The total CP-asymmetry $\epsilon _j$ in the decay of $\tilde{\Psi _j}$ into $\ell _\alpha \phi ~ (\bar{\ell _\alpha } \phi ^\dag )$ can be calculated by summing over the SM flavor $\beta$ as follows:

$$\begin{aligned} \epsilon _j= & {} \frac{\sum _\beta \left[ \Gamma (\tilde{\Psi _j} \rightarrow \ell _\beta \phi ) - \Gamma (\tilde{\Psi _j} \rightarrow {\bar{\ell }}_\beta \phi ^\dag ) \right] }{\sum _\beta \left[ \Gamma (\tilde{\Psi _j} \rightarrow \ell _\beta \phi ) + \Gamma (\tilde{\Psi _j} \rightarrow {\bar{\ell }}_\beta \phi ^\dag ) \right] } \nonumber \\= & {} \frac{1}{8 \pi } \sum _{i\ne j} \frac{\textrm{Im}[(h h^\dag )_{ji}^2]}{(h h ^\dag )_{jj}} f_{ji} \end{aligned}$$

(15)

Since we are operating in the regime of resonant leptogenesis, the dominant contribution to $f_{ij}$ comes from self energy correction, i.e. $f_{ij} \sim f_{ij}^{\textrm{self}}$, where $f_{ij}^{\textrm{self}} = \frac{(M_i^2 - M_j^2) M_i M_j}{(M_i^2 - M_j^2)^2 + R_{ij}^2}$.⁶

Here we have assumed that $\tilde{\Psi _{1}}, \tilde{\Psi _{2}}$ are almost mass degenerate and lighter than other two states $\tilde{\Psi _{3}}, \tilde{\Psi _{4}}$. Here it is considered that the lightest mass degenerate states $\tilde{\Psi _{1}}, \tilde{\Psi _{2}}$ decay into each other to generate the lepton asymmetry $\epsilon _1$ and $\epsilon _2$ as shown in Fig. 1. To be more specific, Fig. 1a–c represent the diagrams which contribute to the computation of $\epsilon _1$ and $\epsilon _2$. These lepton asymmetries enter into the coupled Bolzmann equations. By solving three coupled Boltzmann equations simultaneously, one can obtain co-moving densities $Y_{\tilde{\Psi _{1}}}, Y_{\tilde{\Psi _{2}}}$ and $Y_{B-L}$ of $\tilde{\Psi _{1}}, \tilde{\Psi _{2}}$ and $B-L$ asymmetry respectively. The co-moving density is denoted as the ratio of actual number density⁷ and the entropy density ${\bar{s}}$ of the universe.⁸ The detailed presentation and discussion of the Boltzmann equations are relegated to Appendix A. The lepton asymmetry is converted into baryon asymmetry $Y_B$ via sphelaron process [87] as:

$$\begin{aligned} Y_{B} = \bigg ( \frac{8 N_{f} + 4 N_{H}}{22 N_{f} + 13 N_{H} } \bigg ) Y_{B-L} (z_{\textrm{sph}}). \end{aligned}$$

(17)

where $Y_{B-L} (z_{\textrm{sph}})$ is the solution of Boltzmann equations at $z = z_{\textrm{sph}} = \frac{M_{\tilde{\Psi _1}}}{T_{\textrm{sph}}}$. Here $N_f$ and $N_H$ denote the number of generations of fermion families and number of Higgs doublets respectively. In our framework, $N_f = 3$ and $N_H =1$. In Eq. (1), we have quoted the observed baryon asymmetry $\eta _B$⁹ at present days.

4 Collider studies

Before proceeding to perform the collider analysis, let us discuss some of the salient features of the model parameter space in light of various constraints described in Sect. 3. In Sect. 2, we already have mentioned that with the inverse see-saw hierarchy, one can ascertain tiny active neutrino masses by setting $M_R = 0$. We have also verified that this assumption hardly alters the results with respect to the scenario with $M_R \ne 0$. Thus we set $M_R =0$ throughout this analysis, which in turn introduces a resemblance with the original inverse see-saw model. In addition, all the entries of $M_\nu$ (except $M_R$) are considered as complex to make the analysis a general one. While fitting the neutrino oscillation data, we adopt normal hierarchy among the light active neutrinos and also set the mass of the lightest active neutrino to be zero ($m_1 = 0$), which arises as an artifact of the present framework. For the collider analysis, we shall restrict ourselves to a particular mass region, where the mass of the lightest heavy neutrino ($M_{\tilde{\Psi _1}} \simeq M_{\tilde{\Psi _2}}$) is less than or almost equal to 10 TeV. Now, the ranges for input parameters for the aforementioned range of $M_{\tilde{\Psi _1}}$ are: $M_{D_{i,j}}^{R,I} \in [10^7~\textrm{eV}: 10^8~\textrm{eV}]$, $M_{S_{i,j}}^{R,I} \in [10^{10}~\textrm{eV}: 10^{13}~\textrm{eV}]$. From Eq. (11) one can solve $\mu$ and express it in terms of the input parameters $M_D, M_S$ ( with $M_R = 0$ ) and neutrino oscillation parameters. In addition, it has been verified that in presence of the TeV scale heavy neutrinos, the most stringent bound coming from the LFV decay $\mu \rightarrow e \gamma$ [81], can be easily evaded for all points in the parameter space. Next by solving the Boltzmann equations (given in Appendix A), one can compute the baryon asymmetry of the universe at each and every point of the parameter space. In Fig. 2a (Fig. 2b) (taken from one of our previous studies [82]), we have plotted Br($\mu \rightarrow e \gamma$) vs. $M_{\tilde{\Psi _1}}~(M_{\tilde{\Psi _3}})$. The blue, red and green points are compatible with neutrino oscillation data only, neutrino oscillation data + LFV decay and neutrino oscillation data + LFV decay + current baryon asymmetry of the universe respectively. Here we have considered a 1$\sigma$ deviation around the central value of observed baryon asymmetry.

Fig. 3

The Feynman diagrams representing the signal process $\mu ^+ \mu ^- \rightarrow 2l+E_T \!\!\!\!\!\!/\;\,$. In these diagrams, $l^{\pm }_{\alpha }$ denote $e, \mu$

Fig. 4

The Feynman diagrams representing the signal process $\mu ^+ \mu ^- \rightarrow 1l+2j+E_T \!\!\!\!\!\!/\;\,$. In these diagrams, $l^{\pm }_{\alpha }$ denote $e, \mu$

Table 2 Benchmark points characterizing the masses of the heavy neutrinos satisfying the observed baryon asymmetry within $1 \sigma$ error bar

In one of our previous studies [82], we have shown that there exists a stringent lower bound of $\sim$ 3.2 TeV on $M_{\tilde{\Psi _1}}$ to satisfy the current baryon asymmetry data. Thus one can anticipate that due to the presence of these TeV scale heavy neutrinos, our present framework becomes very attractive from the collider perspective.

In this paper, we are interested to probe the pair production of the heavy neutrinos ${\tilde{\Psi }}_{1,2,3,4}$) along with SM active neutrinos ($\nu _l$) via s-channel and t-channel processes, followed by subsequent decays of the heavy neutrinos to $W^+$ and an SM lepton ($l^{\pm }$¹⁰). Further decays of unstable $W^+$ into leptonic ($W^{\pm } \rightarrow l^{\pm } \nu _l({\bar{\nu }}_l$) and hadronic ($W^\pm \rightarrow j j$) modes, give rise to $2l + E_T \!\!\!\!\!\!/\;\,$ and $1l + 2j + E_T \!\!\!\!\!\!/\;\,$ final states respectively as shown in Figs. 3a, b and 4a, b respectively. Details of decay chain of the respective processes leading to the final states will be described in the following subsections.

For performing collider analysis of the aforementioned channels, we shall choose eight benchmark points (BP1, BP2, BP3, BP4, BP5, BP6, BP7, BP8) from the model parameter space, which lie well within the correct baryon asymmetry band. Thus the chosen benchmark points are compatible with the experimental data coming from neutrino oscillation, LFV decays and current baryon asymmetry of the universe. These benchmark points are characterised by low, medium and high lightest heavy neutrino masses ($M_{\tilde{\Psi _1}}$), spanning a wide range, from 3.2 TeV to 10.12 TeV. The eight benchmarks are tabulated in Table 2 along with $M_{{\tilde{\Psi }}_{1(2)}}$, $M_{{\tilde{\Psi }}_{3(4)}}$ and corresponding baryon asymmetry yield. To probe the benchmark points, we perform the collider analysis at two different center of mass (COM) energies ($\sqrt{s}$), i.e. $\sqrt{s} = 6$ TeV (for BP1, BP2, BP3), 14 TeV (for BP4, BP5, BP6, BP7, BP8) at the muon collider.

In the next subsections we shall perform back to back collider analyses of the $2l + E_T \!\!\!\!\!\!/\;\,$ and $1l + 2j + E_T \!\!\!\!\!\!/\;\,$ final states, which turn out to be promising at aforementioned COM energies. Owing to small signal cross section and large background, even with high luminosity, this signal turns out to be non-promising at LHC. Besides, this signal involving TeV scale heavy neutrinos in the final state cannot be probed at ILC since the maximum achievable COM energy at ILC is 1 TeV. The leading order (LO) signal and background cross sections for the channels are generated via MG5aMC@NLO [88]. Further decays of the unstable particles are emulated through Pythia8 [89]. The detector effects are included in the analysis by passing both the signal and the backgrounds through Delphes$-$3.5.0 [90]. We use the default muon collider simulation card [91] for this purpose. We shall comment about the jet clustering algorithm later in Sect. 4.2. Following acceptance cuts are used for the aforementioned channels:

$$\begin{aligned} p_{T_j}> & {} 20 ~ \textrm{GeV}, ~~ |\eta _j|< 5.0, \nonumber \\ p_{T_l}> & {} 10 ~ \textrm{GeV}, ~~ |\eta _l| < 2.5, \nonumber \\ \Delta R_{ij}> & {} 0.4, ~ \textrm{for }~ i,j = l, \textrm{jets} \end{aligned}$$

(18)

Here $p_{T_{j(l)}}$ and $|\eta _{j(l)}|$ are the transverse momentum and pseudo-rapidity of jets (leptons) respectively. The separation between the final state i-th and j-th particles can be defined as:

$$\begin{aligned} \Delta R_{ij}=\sqrt{(\eta _{i}-\eta _{j})^2+(\phi _{i}-\phi _{j})^2}, \end{aligned}$$

where $\eta _{i}$ and $\phi _i$ are the pseudo rapidity and azimuthal angle of i-th particle. Since traditional cut-based analysis is enough to yield high significances for all benchmarks, we shall only present the results generated from it. The signal significance has been computed using ${\mathcal {S}} = \sqrt{2\Big [(S + B) \log \Big (\frac{S + B}{B}\Big )- S\Big ]}$ [92], where S(B) denote the number of signal (background) events surviving the cuts applied on relevant kinematic variables.

4.1 $2l + E_T \!\!\!\!\!\!/\;\,$ final state

In muon collider, heavy neutrinos can be produced along with active neutrinos via t-channel $W^\pm$ mediation and s-channel Z mediation. In this analysis, we aim to probe the following signal containing two opposite sign (OS) same/different flavor leptons plus missing energy in the final state:

$$\begin{aligned}{} & {} \mu ^+ \mu ^- \rightarrow \nu _l ~{\tilde{\Psi }}_{1,2,3,4},~~ {\tilde{\Psi }}_{1,2,3,4} \rightarrow W^{\pm }~ l^{{\mp }}; ~ W^{\pm }\nonumber \\{} & {} \qquad \quad \rightarrow l^{\pm } \nu _l({\bar{\nu }}_l); \rightarrow ~ l^+ l^- + \not \! E_T \end{aligned}$$

(19)

Table 3 The LO cross sections for signal and backgrounds for the process $\mu ^{+} \mu ^{-} \rightarrow 2l+E_T \!\!\!\!\!\!/\;\,$ at 6 TeV and 14 TeV

Fig. 5

Normalized distributions of the kinematic variables a $p_{T_{l_1}}$, b $p_{T_{l_2}}$, c $p^{vect}_{T_{l_1,l_2}}$, d $M_{eff}$, e $\Delta R_{l_1 l_2}$, (f) $\Delta \phi _{l_1 l_2}$ (for signal and SM background) for $2l + E_T \!\!\!\!\!\!/\;\,$ channel at 6 TeV muon collider

Fig. 6

Normalized distributions of the kinematic variables a $p_{T_{l_1}}$, b $p_T^{l_1,l_2}$, c $p_{T_{l_1 l_2}}^{vect}$, d $\Delta R_{l_1 l_2}$, e $\Delta \phi _{l_1 l_2}$, f (for signal and SM background) for $2l + E_T \!\!\!\!\!\!/\;\,$ channel at 14 TeV muon collider

Here, the heavy neutrino can decay to $W^{\pm }$ via charged current interaction which is further accompanied by the leptonic decay of $W^{\pm }$. The dominant background for this signal originates from the $\mu ^+ \mu ^- \rightarrow l^+ l^- + \not \! E_T$ final state, which receives dominant contributions from the following sub-processes¹¹:

• $\mu ^{+}\mu ^{-}\rightarrow l^{-} l^{+*} \rightarrow l^{-} W^{+*} {\bar{\nu }}_l \rightarrow l^{-} l^{+} \nu _l {{\bar{\nu }}}_l$,

• $\mu ^{+}\mu ^{-}\rightarrow l^{-} l^{+*} \rightarrow l^{-} Z^{*} l^+ \rightarrow l^{-} l^{+} \nu _l {\bar{\nu }}_l$,

• $\mu ^{+}\mu ^{-}\rightarrow \nu _l^{*} \nu _l\rightarrow \nu _l Z^{*} {\bar{\nu }}_l \rightarrow \nu _l {\bar{\nu }}_l l^+ l^{-}$,

• $\mu ^{+}\mu ^{-}\rightarrow \nu _l^{*} {\bar{\nu }}_l\rightarrow {\bar{\nu }}_l W^{+*} l^- \rightarrow \nu _l {\bar{\nu }}_l l^+ l^{-}$,

• $\mu ^+ \mu ^- \rightarrow W^+ W^-$; $W^+ \rightarrow l^+ ~\nu _l, ~ W^- \rightarrow l^- ~{\bar{\nu }}_l$,

• $\mu ^+ \mu ^- \rightarrow ZZ$; $Z \rightarrow l^+ l^-, ~ Z \rightarrow \nu _l ~{\bar{\nu }}_l$,

• $\mu ^+ \mu ^- \rightarrow W^+ W^- Z$; $W^+ \rightarrow l^+ ~\nu _l, ~ W^- \rightarrow l^- ~{\bar{\nu }}_l, ~ Z \rightarrow \nu _l ~{\bar{\nu }}_l$,

• $\mu ^+ \mu ^- \rightarrow ZZZ$; $Z \rightarrow l^+ l^-, ~ Z \rightarrow \nu _l ~{\bar{\nu }}_l, ~ Z \rightarrow \nu _l ~{\bar{\nu }}_l$.

The LO cross sections of the signals (for all benchmarks) and SM backgrounds are tabulated in Table 3.¹² To reduce the background with respect to signal, we choose a set of cuts on some of the kinematic variables. We denote the cuts on the kinematic variables for benchmarks BP1- BP3 at 6 TeV COM energy as $B_i~(i:1-7)$ and for the rest of the benchmarks (BP4–BP8) at $\sqrt{s}=14$ TeV, the cuts are defined as $C_j~(j:1-7)$. For brevity, and also to avoid repetition, we discuss the cuts for two different COM energies together. All the names of the cuts or numbers within the bracket below will correspond to the analysis at 14 TeV. Following are the depiction of cuts chosen to enhance significance:

• ${B_1~(C_1):}$ We demand two opposite sign, same or different flavored charged leptons l in the final state. As mentioned previously, here l denotes only electron and/or muon.

• ${B_2~(C_2):}$ The normalized distributions of the transverse momentum of the leading lepton ($p_{T_{l_1}}$) for signal¹³ and background are shown in Fig. 5a (Fig. 6a) for COM energy 6 TeV (14 TeV). Here the leading lepton gets larger boost in $p_T$ for the signal, since it is generated directly via pair production. In contrary, for the background, the leading lepton is produced from subsequent decays of pair produced particles. Thus $p_{T_{l_1}}$ for the signal peaks at a larger value with respect to the SM background. Thus putting a large lower cut on the transverse momentum, i.e. $p_{T_{l_1}}> 400$ GeV (200 GeV) for 6 TeV (14 TeV) can be suitable to enhance the signal significance.

• $B_3$: We show the normalized distributions of the transverse momentum of the next to leading lepton ($p_{T_{l_2}}$) for signal and backgrounds in Fig. 5b. As can be seen from the distributions, the SM background peaks around $p_T\sim 0$, because the major fraction of the transverse momentum is taken away by the leading lepton. The signal distribution starts dominating the background distribution after 200 GeV. Therefore, we put a lower cut of $p_{T_{l_2}}>200$ GeV at 6 TeV to maximise the significance.

• $C_3:$ We depict the normalized distributions of scalar sum of the transverse momenta of two leptons in the final state in Fig. 6b. The nature of the distributions of the signal and background can be explained following the descriptions of the distributions of $p_{T_{l_1}}$ and $p_{T_{l_2}}$ in previously defined cuts B2 (C2) and B3. A suitable lower cut: $p_T^{l_1,l_2}>2$ TeV helps magnifying the significance drastically for $\sqrt{s} = 14$ TeV.

• $B_4~(C_4):$ In Fig. 5c (Fig. 6c), we plot the normalised distributions of the magnitude of the vector sum of the transverse momenta of two leptons in the final state for signal and background for $\sqrt{s}$ = 6 TeV (14 TeV). It is defined as:

  $$\begin{aligned} p^{vect}_{T_{l_1,l_2}} {\equiv }\sqrt{(p_{{T x}_{l_1}}{+}p_{{T x}_{l_2}})^2{+}(p_{{T y}_{l_1}}{+}p_{{T y}_{l_2}})^2{+}(p_{{T z}_{l_1}}{+}p_{{T z}_{l_2}})^2} \end{aligned}$$

  Table 4 The cut-flow for signal and backgrounds along with the required integrated luminosity to achieve 5$\sigma$ significance for the $\mu ^+ \mu ^- \rightarrow 2l+E_T \!\!\!\!\!\!/\;\,$ channel for BP1, BP2 and BP3 at 6 TeV Muon collider

  Here, $p_{{T x}_{l_i}}, p_{{T y}_{l_i}}$ and $p_{{T z}_{l_i}}$ are the x, y, z-components of the transverse momentum vector for $i=1,2$ respectively. We choose $p^{vect}_{T_{l_1,l_2}}>600$ GeV (2 TeV) to suppress SM background for $\sqrt{s} = 6$ TeV ( 14 TeV ).

• $B_5:$ We define a kinematic variable $M_{eff}$, i.e. effective mass as the scalar sum of the transverse momenta of the final state leptons and missing transverse energy. The normalized distributions of $M_{eff}$ both for signal and background are shown in Fig. 5d. The distributions mimic the distributions for individual transverse momentum of the final state leptons as expected. Since the signal distributions for almost all benchmarks start overshadowing the SM background distribution for $M_{eff}>2$ TeV, to achieve large significance, we apply a 2 TeV lower cut on $M_{eff}$.

• $B_6~(C_5)$: We portray the normalized distribution of $\Delta R_{l_1 l_2}$ for signals and backgrounds in Fig. 5e (Fig. 6d) at COM energy 6 TeV (14 TeV). For the signal, the distribution of $\Delta R_{l_1 l_2}$ peaks at higher values, which is expected. According to the signal topology in the $2l + E_T \!\!\!\!\!\!/\;\,$ channel, one of the final state charged leptons is produced along with the W-boson from the decay of the heavy neutrino, whereas the second lepton is produced from the decay of the W-boson itself. Thus the two opposite sign same or opposite flavoured leptons do not originate from the same mother particle. Therefore the normalized distribution of $\Delta R_{l_1 l_2}$ peaks at comparatively higher value for the signal benchmarks. While generating the background sample, we have included all possible backgrounds leading to the final state $2l + E_T \!\!\!\!\!\!/\;\,$. Thus the distribution of the aforesaid variable is smeared spanning a wide range. Besides, one of the dominant contributions in $2l + E_T \!\!\!\!\!\!/\;\,$ background comes from ZZ process. This process will mimic the $2l + E_T \!\!\!\!\!\!/\;\,$ final state if one of the Z-bosons decays to two opposite sign same flavoured leptons, while the other Z decays into two neutrinos. In this case the leptons being originated from a single mother particle, they will be collimated enough to keep $\Delta R_{l_1 l_2}$ small. In addition, if we consider the contribution coming from $W^+ W^-$ process to the background, two leptons in the final state originate from two different W-bosons, thus they will be widely separated making $\Delta R_{l_1 l_2}$ large. That is why the distribution of $\Delta R_{l_1 l_2}$ for background does not peak at some particular value, rather it has a spread spanning low as well as high values. It is evident from the distributions of the signal and background, a proper cut of $\Delta R_{l_1 l_2}>3.0(3.0)$ has been chosen to distinguish the signal from the background.

• $B_7~(C_6)$: We compute the magnitude of azimuthal angular separation of two final state leptons as:

  $$\begin{aligned} \Delta \phi _{l_1 l_2}= & {} |\phi _{l_1}-\phi _{l_2}| \text { for }\Delta \phi _{l_1 l_2}<\pi \\ \Delta \phi _{l_1 l_2}= & {} 2\pi -|\phi _{l_1}-\phi _{l_2}|, \text { otherwise} \end{aligned}$$

  The normalized distributions of $\Delta \phi _{l_1 l_2}$ are presented for signal and backgrounds in Fig. 5f (Fig. 6e) for $\sqrt{s} =$ 6 TeV (14 TeV). The nature of the distributions of $\Delta \phi _{l_1 l_2}$ for signal and backgrounds can be explained following the same logic provided to describe the distributions for $\Delta R_{l_1 l_2}$ previously. We choose $\Delta \phi _{l_1 l_2}>3.0~ (3.1)$, for the survival of the signal over background for two aforementioned COM energies.

• $C_7:$ We define the azimuthal angular separation between direction of the leading lepton $l_1$ and missing energy vector as and present the normalized distributions of the same in Fig. 6f for COM energy 14 TeV. To differentiate the signal from the background, most optimal cut on the aforementioned variable is .

Table 5 The cut-flow for signal and backgrounds along with the required integrated luminosity to achieve 5$\sigma$ significance for the $\mu ^+ \mu ^- \rightarrow 2l+E_T \!\!\!\!\!\!/\;\,$ channel for BP4–BP8 at 14 TeV Muon collider

We summarize the results obtained after each cut, i.e. $(B_1- B_7)$ for $\sqrt{s}=6$ TeV and $(C_1- C_7)$ for $\sqrt{s}=14$ TeV in two tables. The cut-flow of the first three benchmarks along with the SM background for COM 6 TeV are presented in Table 4. The number of signal and background events after applying each cut are quoted at integrated luminosity 100 fb$^{-1}$. It is evident from the table that the cuts are very effective to wipe out the background compare to the signal. At $\sqrt{s} = 6$ TeV and ${\mathcal {L}} = 100$ fb$^{-1}$, we are left with only 192 background events. Whereas the signal events remains almost steady after applying the same cuts. In the last column of the table, we compute the required integrated luminosity to achieve the 5$\sigma$ significance. For BP1, 0.17 $\textrm{fb}^{-1}$ luminosity is enough to achieve $5\sigma$ significance while a benchmark with higher heavy neutrino mass 5.11 TeV (BP3) would require 14.19 $\textrm{fb}^{-1}$. This happens owing to the decrease in signal cross section with increasing $M_{{\tilde{\Psi }}_{1}}$.

Next in Table 5, we present the cut flow for benchmarks BP4–BP8 for COM energy 14 TeV with integrated luminosity 1600 $\textrm{fb}^{-1}$. SM background events are reduced to 237 after employing all the chosen cuts. Whereas the number of the signal events surviving after imposing all cuts for BP4, BP5, BP6, BP7, BP8 are 25809, 4698, 1323, 430, and 98 respectively. In addition, we derive the integrated luminosity needed to attain $5\sigma$ significance. For BP4–BP8, 1.56 $\textrm{fb}^{-1}$, 8.94 $\textrm{fb}^{-1}$, 35.66 $\textrm{fb}^{-1}$, 144.05 $\textrm{fb}^{-1}$ and 1380.55 $\textrm{fb}^{-1}$ integrated luminosities are required to achieve 5$\sigma$ discovery. Thus one can conclude that the benchmark with larger cross section and hence with smaller value of $M_{{\tilde{\Psi }}_{1}}$, is more promising than others to probe at the muon collider in light of $2l + E_T \!\!\!\!\!\!/\;\,$ final state.

4.2 $1l + 2j + E_T \!\!\!\!\!\!/\;\,$ final state

In this subsection we shall analyse the aforementioned channel featuring the hadronic decay of W-boson (Fig. 4a, b). We consider corresponding SM backgrounds with the same final state. Before we start analysing the channel, let us make brief comment on the jet clustering algorithm, which we have used here. Here the jet clustering occurs via Valencia (VLC) jet clustering algorithm [93, 94], which is well suited at high energy lepton colliders. Using this algorithm, the events are clustered into two jets with $R = 0.5$ in our case. The dominant background for this signal originates from the $\mu ^+ \mu ^- \rightarrow 1l + 2j + \not \! E_T$ final state, which receives contributions from the following sub-processes:

• $\mu ^{+} \mu ^{-} {\rightarrow } W^{+} W^{-}$; $W^{+} (W^-) {\rightarrow } l^{+}(l^-) ~\nu _\ell (~{\bar{\nu }}_\ell ), ~W^-(W^+)$ ${\rightarrow } jj,$

• $\mu ^+ \mu ^- \rightarrow ZZ$; $Z \rightarrow l^+ l^-, ~ Z \rightarrow jj$, (one of the leptons is missed)

• $\mu ^+ \mu ^- \rightarrow W^+ W^- Z$;

  • 1. $W^+ \rightarrow l^+ ~\nu _l, ~ W^- \rightarrow l^- ~{\bar{\nu }}_l, ~ Z \rightarrow jj$, (one of the leptons is missed)

  • 2. $W^+ \rightarrow l^+ ~\nu _l, ~ W^- \rightarrow jj, ~ Z \rightarrow \nu _l ~{\bar{\nu }}_l$,

  • 3. $W^+ \rightarrow jj, ~ W^- \rightarrow l^- ~{\bar{\nu }}_l, ~ Z \rightarrow \nu _l ~{\bar{\nu }}_l$,

• $\mu ^+ \mu ^- \rightarrow ZZZ$; $Z \rightarrow l^+ l^-, ~ Z \rightarrow jj, ~ Z \rightarrow \nu _l ~{\bar{\nu }}_l$, (one of the leptons is missed)

• $\mu ^+ \mu ^- \rightarrow Z ~ h$;

  • 1. $Z \rightarrow l^+ l^-, ~h \rightarrow jj,$ (one of the leptons is missed)

  • 2. $Z \rightarrow jj, ~h \rightarrow l^+ l^-$. (one of the leptons is missed)

We follow the same procedure as the previous channel for analysing this signal. We shall refrain ourselves from discussing the details of generating the signal and backgrounds to avoid repetition. Similar to the previous analysis, we divide the study in two parts: for the first three benchmarks BP1–BP3, the COM energy of muon collider is taken to be 6 TeV and for the rest of the benchmarks BP4–BP8, the COM energy is assumed to be 14 TeV.

Table 6 The LO cross sections for signal and backgrounds for the process $\mu ^{+} \mu ^{-} \rightarrow 1l + 2j +E_T \!\!\!\!\!\!/\;\,$ at 6 TeV and 14 TeV

Fig. 7

Normalized distributions of the kinematic variables a $p_{T_{j_{1}}}$, b $p_{T_{j_{2}}}$ (for signal and SM background) for $1l + 2j + E_T \!\!\!\!\!\!/\;\,$ channel at 6 TeV muon collider

We compute the leading order cross-sections for signal (for all the benchmarks) and SM backgrounds and presented them in the Table 6. In Table 6, the top half of the table presents LO cross-sections for BP1–BP3 and corresponding backgrounds for $\sqrt{s}=6$ TeV. The lower half contains cross-sections for BP4–BP8 and SM backgrounds at $\sqrt{s}=$ 14 TeV. As can be seen from Table 6, the most significant contribution in backgrounds comes from $W^+ W^{-}$ owing to its high cross-section. With the change of $\sqrt{s}$ from 6 to 14 TeV, the dominant SM background cross-section decreases significantly from 40.92 fb to 9.25 fb. Same feature is observed for $W^+ W^- Z$ background, i.e. cross-section decreases with increase in $\sqrt{s}$. The cross-sections for BP1 and BP4 are quite large compare to the dominant SM background $W^+ W^-$. However for BP3 and BP6–BP8, the signal cross-sections are lower than the cross-section of the dominant background.

We have drawn the normalized distributions of some relevant kinematic variables for both signal and backgrounds. like the previous subsection, following the same logic, we have presented the distributions for BP1,BP3 (BP4,BP6,BP8) and dominant backgrounds $W^+ W^-$ and $W^+ W^- Z$ for 6 TeV (14 TeV). Further, we imposed cuts on some kinematic variables to enhance the signal significance. We choose cuts $D_i$’s (for BP1–BP3 at 6 TeV) and $E_i$s (for BP4–BP8 at 14 TeV) which are described below:

• $D_1~(E_1):$ Particularly we select those events in which the number of jets are $\ge 2$ and there is one charged lepton $l=e$ or $\mu$.

• $D_2:$ We plot normalized distributions of the transverse momentum of the leading jet ($p_{T_{j_{1}}}$) for signal and backgrounds in Fig. 7a. For signal, the mother particle of two jets ($W^+$) is originated from the decay of a heavy TeV scale neutrino. Thus the distributions of $p_{T_{j_1}}$ for BP1,BP3 peak at higher values. Since the distributions of BP1,BP3 start dominating for $p_{T_{j_1}}>800$ GeV, we apply lower cut of 800 GeV on $p_{T_{j_{1}}}$ to maximize the signal significance.

• $D_3:$ Next we depict the normalized distributions of the transverse momentum of the second leading jet ($p_{T_{j_2}}$) in Fig. 7b. Most of the transverse momentum being carried away by the leading jet, the corresponding distribution of $p_{T_{j_2}}$ for the background peaks around zero transverse momentum. The distributions of $p_{T_{j_2}}$ for BP1,BP3 peak at higher values due to the aforementioned reason. Thus a cut: $p_{T_{j_2}}>400$ GeV is chosen which significantly improve the signal significance.

• $E_2:$ The normalized distributions of the pseudo rapidity of leading jet ($\eta _{j_1}$) for signal and backgrounds are shown in Fig. 8b. For signal, the jets originate from the decay of W-boson, which is previously produced from the decay of the heavy neutrino. Thus the distributions of $\eta _{j_1}$ for signals are centrally peaked. A suitable cut $\eta _{j_1}<0.6$ is chosen to wipe out the backgrounds.

Table 7 The cut-flow for signal and backgrounds along with the required integrated luminosity to achieve 5$\sigma$ significance for the $\mu ^+ \mu ^- \rightarrow 1l + 2j +E_T \!\!\!\!\!\!/\;\,$ channel for BP1, BP2 and BP3 at 6 TeV Muon collider

Fig. 8

Normalized distributions of the kinematic variables a $\eta _{j_{1}}$ (for signal and SM background) for $1\,l + 2j + E_T \!\!\!\!\!\!/\;\,$ channel at 14 TeV muon collider

Table 8 The cut-flow for signal and backgrounds along with the required integrated luminosity to achieve 5$\sigma$ significance for the $\mu ^+ \mu ^- \rightarrow 1l + 2j +E_T \!\!\!\!\!\!/\;\,$ channel for BP4–BP8 at 14 TeV Muon collider

We summarize the results of the analysis and present the cut-flow for all the signal benchmarks and backgrounds in Tables 7 and 8 for $\sqrt{s} =$ 6 TeV and 14 TeV respectively. We apply all the cuts mentioned above and the number of events surviving after each cut are quoted in the aforementioned tables. The upper half of Table 7, presents cut-flow for dominant backgrounds $W^+ W^-$ and $W^+ W^- Z$ at integrated luminosity 100 fb$^{-1}$. In the bottom half of the same table, we show the cut-flow of the signal benchmarks BP1–BP3 at $\sqrt{s}=6$ TeV. After incorporating all three cuts ($D_1, D_2, D_3$), only 187 and 103 events survive from $W^{+} W^{-}$ and $W^{+} W^{-}Z$ backgrounds respectively at integrated luminosity 100 fb$^{-1}$. While remaining signal events for BP1, BP2 and BP3 are 27545, 3962 and 541 respectively at the same luminosity. Therefore, it is observed that the combination of cuts have efficiently reduced the SM backgrounds. We also compute the required luminosity to achieve $5\sigma$ significance (${\mathcal {L}}_{5 \sigma }$) and quote them in Table 7. For BP1,BP2 and BP3, 0.09 fb$^{-1}$, 0.68 fb$^{-1}$ and 7.12 fb$^{-1}$ are required respectively to claim discovery.

We summarize the results for benchmarks BP4–BP8 at 14 TeV in the Table 8. The first half of the table show the surviving events of two dominating backgrounds after imposing each of the aforementioned cuts ($E_1-E_2$). The background events for $W^+ W^-$ drops from 2275 to 1301 events and for $W^+ W^- Z$ it reduces from 1426 to 864 events after applying all the cuts at integrated luminosity 1600 fb$^{-1}$. In the bottom half part of the table, we provide the number of signal events for benchmarks BP4–BP8 after applying the same cuts. One can infer that the combination of two cuts $E_1$ and $E_2$ efficiently kills the backgrounds and helps to enhance the signal significance. Finally, in the same table, we provide the luminosity needed to achieve 5$\sigma$ significance, which increases from BP4 to BP8 owing to monotonically decreasing signal cross-sections.

Before concluding this section, let us present a comparative study between two signal channels which have been analysed in this paper. Comparing the integrated luminosities required to achieve $5 \sigma$ significances for the two channels, $1l + 2j + \not \! E_T$ channel turns out to be the most promising channel among the two. For example to probe BP1 (BP4) in $2l + \not \! E_T$ channel, 0.17 fb$^{-1}$ (1.56 fb$^{-1}$) is required at $\sqrt{s} = 6$ TeV (14 TeV). Whereas, for $1l + 2j + \not \! E_T$ channel, the corresponding luminosity is 0.09 fb$^{-1}$ (0.32 fb$^{-1}$) at $\sqrt{s} = 6$ TeV (14 TeV). This holds for other benchmarks too. The signal cross sections (at a given COM energy) are larger in the hadronic final state and corresponding background cross sections are smaller than the same in the leptonic final state. Putting these all together, the ratio of the signal cross section over the background cross section before any selection cut is much larger for the $1l+2j+E_T \!\!\!\!\!\!/\;\,$ channel than the $2l + E_T \!\!\!\!\!\!/\;\,$ channel. Basically this is one of the reasons for which a leptonic collider is preferred over a hadronic one to probe the hadronic final states. Since we cannot produce this signal at ILC, muon collider turns out to be useful to probe such hadronic final states. Thus at muon collider, the second channel containing jets in the final state turns out to be the most promising. It is also expected as a consequence of a clean, jet-free environment of a leptonic collider compare to LHC.

5 Summary and conclusion

In this present paper, we have focused on a framework popularly known as ISS(2,2), i.e. (2,2) inverse see-saw realisation, where the SM is accompanied with two singlet right handed neutrinos and two singlet neutral fermions. From the name of the model itself, it is evident that the neutrino mass generation in this scenario occurs via inverse see-saw mechanism. To make the analysis a robust one, we have considered the most general structure of the $7 \times 7$ neutrino mass matrix $M_\nu$ with complex entries. Further by regulating the input model parameters and assuming normal mass hierarchy between the active neutrinos, we can segregate the portion of the parameter space, which becomes compatible with the neutrino oscillation data. Upon diagonalising the mass matrix $M_\nu$, we are left with three active neutrino and four heavy neutrino mass eigenstates ($\tilde{\Psi _1}, \tilde{\Psi _2}, \tilde{\Psi _3}, \tilde{\Psi _4}$). The interesting feature of these heavy neutrino mass eigenstates is the pairwise mass degeneracy of the same, i.e. $M_{\tilde{\Psi _1}} \simeq M_{\tilde{\Psi _2}}$ and $M_{\tilde{\Psi _3}} \simeq M_{\tilde{\Psi _4}}$. In addition, we have checked that the former parameter space also obeys the bounds coming from the LFV decays including $\mu \rightarrow e \gamma$. Next at each and every point in the parameter space satisfying neutrino oscillation data and the bounds coming from the LFV decays, the baryon asymmetry is computed. Thus we can obtain a subset of the larger parameter space, which is compatible with (i) neutrino oscillation data, (ii) constraints coming from LFV decay and (iii) observed baryon asymmetry data. One can conclude that the baryon asymmetry constraint itself has put a lower bound on the lightest heavy neutrino state i.e. $M_{\tilde{\Psi _1}} > 3.2$ TeV.

Next we aim to analyse some promising collider signatures of the TeV scale heavy neutrinos at multi-TeV muon collider. First we choose some benchmark points from the previously derived parameter space, depending on $M_{\tilde{\Psi _1}}$. For collider study, we intend to analyse those benchmarks which satisfy the observed baryon asymmetry of the universe. Accordingly, we have picked up different benchmarks where $M_{\tilde{\Psi _1}}$ ranges between 3.2 TeV to 10.12 TeV (BP1–BP8). The range of heavy neutrino masses which we are talking about, cannot be produced at ILC. The signal contains pair production of active neutrino and a heavy neutrino, followed by the decay of the heavy neutrino into $W^\pm$ and a charged lepton (e, $\mu$). Further if we consider the leptonic decay of the generated $W^\pm$, it will lead to a di-lepton plus missing energy state. For analysis, we choose $\sqrt{s} = 6$ TeV for BP1, BP2 and BP3 and $\sqrt{s} = 14$ TeV for the rest of the benchmarks (BP4–BP8). We have not considered conventional LHC or ILC for purpose. The cross-section of the same process at LHC is much smaller than what we obtained in muon-collider. In addition, the large LHC backgrounds would be sufficient to kill the signal. The disadvantage of ILC lies in the maximal achievable COM energy (1 TeV). The dominant background for this signal is $2l + E_T \!\!\!\!\!\!/\;\,$, arising from the subprocesses like $W^+ W^-, ~ZZ, ~ W^+ W^- Z, ~ ZZZ$. Here we impose cuts on a few relevant kinematic variables to achieve maximally enhanced signal significances, suppressing the backgrounds with respect to the signals. Among all benchmarks, BP1 (BP4) turns out to be most promising with respect to the analysed signal at $\sqrt{s} =6$ TeV (14 TeV). To achieve 5$\sigma$ significance one needs 0.17 fb$^{-1}$, 1.28 fb$^{-1}$ and 14.19 fb$^{-1}$ integrated luminosity for BP1, BP2, BP3 respectively at COM energy 6 TeV. Besides the integrated luminosities required for 5$\sigma$ significance are 1.56 fb$^{-1}$, 8.94 fb$^{-1}$, 35.66 fb$^{-1}$, 144.05 fb$^{-1}$, 1380.55 fb$^{-1}$ respectively for BP4, BP5, BP6, BP7, BP8 at $\sqrt{s}=14$ TeV. Though this particular channel provides encouraging results at muon collider, for completeness, we also consider the hadronic decay of $W^\pm$ in the signal instead of leptonic decay, leading to $1l + 2j + E_T \!\!\!\!\!\!/\;\,$ signature. Analysing this channel is very challenging due to the presence of large jetty SM backgrounds. In presence of the dominant background $W^+ W^-$ and $W^+ W^- Z$, the end result of analysing this channel is encouraging. Required luminosities to procure 5$\sigma$ significance at $\sqrt{s} =$ 6 TeV (14 TeV) for BP1–BP3 (BP4–BP8) are 0.09 fb$^{-1}$, 0.68 fb$^{-1}$ and 7.12 fb$^{-1}$ (0.32 fb$^{-1}$, 2.0 fb$^{-1}$, 9.11 fb$^{-1}$, 44.59 fb$^{-1}$ and 563.06 fb$^{-1}$) respectively. By looking at the integrated luminosity required to achieve 5$\sigma$ significances for different benchmarks at $\sqrt{s} = 6$ TeV and 14 TeV, one can conclude that $1l+2j+\not \! E_T$ channel is indeed more promising than the previous one.

Fig. 9

Feynman diagram representing a decay of lightest heavy RH-neutrino $\tilde{\Psi _{1}}~(\tilde{\Psi _{2}})$ (contributes to $\gamma _{D}^{(1)}, \gamma _{D}^{(2)}$), b $\Delta L = 2$, s-channel scattering via $\tilde{\Psi _{1}}~(\tilde{\Psi _{2}})$ (contributes to $\gamma _{{\tilde{\Psi }},s}$), c $\Delta L = 1$, s-channel scattering via Higgs (contributes to $\gamma ^{1}_{\phi ,s}, \gamma ^{2}_{\phi ,s}$), d, e $\Delta L = 1$, t-channel scattering via Higgs (contributes to $\gamma ^{1}_{\phi ,t}, \gamma ^{2}_{\phi ,t}$), f $\Delta L = 2$, t-channel scattering via $\tilde{\Psi _{1}}~(\tilde{\Psi _{2}})$ (contributes to $\gamma _{{\tilde{\Psi }},t}$)

Appendix A: Evolution of asymmetry

Evolution of asymmetry can be calculated by solving the following Boltzmann equations: $\tilde{\Psi _{1}}, \tilde{\Psi _{2}}$ and the $(B-L)$ asymmetry can be written as [95],

$$\begin{aligned} \frac{\text {d} Y_{\tilde{\Psi _{1}}}}{\text {d} z}= & {} - \frac{z}{s \,\ H(M_{\tilde{\Psi _1}})} \left[ \left( \frac{Y_{\tilde{\Psi _{1}}}}{Y^{eq}_{\tilde{\Psi _{1}}}} - 1\right) (\gamma _{D}^{(1)}+ 2 \gamma ^{(1)}_{\phi ,s} + 4 \gamma ^{(1)}_{\phi ,t})\right] ,\qquad \end{aligned}$$

(A1)

$$\begin{aligned} \frac{\text {d} Y_{\tilde{\Psi _{2}}}}{\text {d} z}= & {} - \frac{z}{s \,\ H(M_{\tilde{\Psi _1}})} \left[ \left( \frac{Y_{\tilde{\Psi _{2}}}}{Y^{eq}_{\tilde{\Psi _{2}}}} - 1\right) (\gamma _{D}^{(2)} + 2 \gamma ^{(2)}_{\phi ,s} + 4 \gamma ^{(2)}_{\phi ,t})\right] ,\nonumber \\ \end{aligned}$$

(A2)

$$\begin{aligned} \frac{\text {d} Y_{B-L}}{\text {d} z}= & {} - \frac{z}{s \,\ H(M_{\tilde{\Psi _1}})} \left[ \!\sum _{j{=}1}^2 \left\{ \!\frac{1}{2} \frac{Y_{B{-}L}}{Y^{eq}_{l}} {+} \epsilon _j ~\left( \!\frac{Y_{\tilde{\Psi _{j}}}}{Y^{eq}_{\tilde{\Psi _{j}}}} {-} 1\!\right) \right\} \gamma _{D}^{(j)} \right. \nonumber \\{} & {} \left. {+}\!\frac{Y_{B{-}L}}{Y^{eq}_{l}}\left\{ 2 \gamma _{{\tilde{\Psi }},s} {+} 2 \gamma _{{\tilde{\Psi }},t}\right\} {+} \frac{Y_{B-L}}{Y^{eq}_{l}} \sum _{j=1}^2 \left\{ 2 \gamma ^{(j)}_{\phi ,t} + \frac{Y_{\tilde{\Psi _{j}}}}{Y^{eq}_{\tilde{\Psi _{j}}}} \gamma ^{(j)}_{\phi ,s} \right\} \right] , \nonumber \\ \end{aligned}$$

(A3)

where $z = \frac{M_{\tilde{\Psi _1}}}{T}$ and $H(M_{\tilde{\Psi _1}})$ is the Hubble parameter at $T = M_{\tilde{\Psi _1}}$ and $H(T=M_{\tilde{\Psi _1}}) = 1.66 ~g_{eff}^{1/2} \frac{T^{2}}{M_{{\textrm{Pl}}}}|_{T=M_{\tilde{\Psi _1}}}$, $M_{{\textrm{Pl}}} \sim 10^{19}$ GeV being Planck scale. $Y_{\tilde{\Psi _{j}}}^{eq}, Y_l^{eq}$ are the comoving densities at equilibrium. We solve these three equations with initial conditions:

$$\begin{aligned} Y_{\tilde{\Psi _i}}(0) = Y_{\tilde{\Psi _i}}^{eq}, ~\textrm{and}~ Y_{B-L}(0) = 0 . \end{aligned}$$

(A4)

at $T \gg M_{\tilde{\Psi _1}}$.

Contributions to $\gamma ^{'}$s are given by diagrams shown in Fig. 9. The lepton asymmetry is further can be translated into baryon asymmetry. This results in the final baryon number at $T_{\textrm{sph}} \sim 150$ GeV (the freeze-out temperature of the sphelaron process) as [87]:

$$\begin{aligned} Y_{B} = \left( \frac{8 N_{f} + 4 N_{H}}{22 N_{f} + 13 N_{H} } \right) Y_{B-L} (z_{\textrm{sph}}). \end{aligned}$$

(A5)

With $Y_{B-L} (z_{\textrm{sph}})$ as the solution of Boltzmann equations at $z = z_{\textrm{sph}} = \frac{M_{\tilde{\Psi _1}}}{T_{\textrm{sph}}}$.

Here $N_f:$ the number of generations of fermion families and $N_H:$ number of Higgs doublets. For our case, $N_f = 3$ and $N_H =1$. $\gamma _{D}^{(i)}$ can be written as [95]:

$$\begin{aligned} \gamma _{D}^{(i)} = N_{{\tilde{\Psi }}_i}^{eq}~ \frac{K_{1}(\frac{M_{{\tilde{\Psi }}_i}}{T})}{K_{2}(\frac{M_{{\tilde{\Psi }}_i}}{T})}~ \Gamma _{\tilde{\Psi _i}} , \textrm{with}~ i=2 \end{aligned}$$

(A6)

$N_{{\tilde{\Psi }}_i}^{eq}$ being the equilibrium number density of mass eigenstate ${\tilde{\Psi }}_i$. Here $K_1$ and $K_2$ are the first and second modified Bessel functions of second kind respectively and $\Gamma _{{\tilde{\Psi }}_i}$ is the total decay width of ${\tilde{\Psi }}_i$.

Decay width of ${\tilde{\Psi }}_i$ at tree level,

$$\begin{aligned} \Gamma _{{\tilde{\Psi }}_i}:= & {} \Gamma ({\tilde{\Psi }}_i \rightarrow \phi ^{\dagger } + l) + \Gamma ({\tilde{\Psi }}_i \rightarrow \phi + {\bar{l}}) \nonumber \\{} & {} = \frac{\alpha }{\text {sin}^{2}\theta _W} \frac{M_{{\tilde{\Psi }}_i}}{4} \frac{(M^{\dagger }_{D} M_{D})_{ii}}{M^{2}_{W}} \end{aligned}$$

(A7)

with $\alpha , \theta _W$ being the Fine structure constant and the Weinberg angle.

For two body scattering $a + b \rightarrow i + j +\cdots$, $\gamma _{eq}$ can be written as [95],

$$\begin{aligned} \gamma _{eq} = \frac{T}{64 \pi ^{4}} \int _{(M_a+M_b)^2}^{\infty } \, ds ~{\hat{\sigma }}(s) \sqrt{s} \, K_{1}\left( \frac{\sqrt{s}}{T}\right) . \end{aligned}$$

(A8)

here s¹⁴ is the square of center of mass energy and ${\hat{\sigma }}(s)$ is reduced cross section, which can be expressed in terms of actual cross section as [95]:

$$\begin{aligned} {\hat{\sigma }}(s) = \frac{8}{s}\left[ (p_a.p_b)^2 - M_a^2 M_b^2\right] \sigma (s) , \end{aligned}$$

(A9)

with $p_k$ and $M_k$ being three momentum and mass of particle k. Expressions for reduced cross-section (${\hat{\sigma }}$) for all the scattering processes shown in Fig. 9 are given in [82, 95].

For our model the total D.O.F turns out to be:

$$\begin{aligned} g_{eff}~( T > 174 ~\textrm{GeV})= & {} g_{\textrm{boson}}+g_{\textrm{fermion}}= 28\nonumber \\ {}{} & {} + \frac{7}{8} \times 94 = 110.25 \end{aligned}$$

(A10)