Displaced fat-jets and tracks to probe boosted right-handed neutrinos in the $U(1)_{B-L}$ model

Abstract

We investigate the pair-production of Right-Handed Neutrinos (RHNs) via a $B-L$ $Z'$ boson and present the sensitivity studies of the active-sterile neutrino mixing ($|V_{\mu N}|$) at the High-Luminosity run of the LHC (HL-LHC) and a future pp collider (FCC-hh). We focus on RHN states with a mass of $10-70$ GeV which naturally results in displaced vertices for small $|V_{\mu N}|$. Being produced through a mass resonance with $M_{Z'} \ge 1$ TeV, the RHNs are heavily boosted, leading to collimated decay products that give rise to fat-jets. We investigate the detection prospect of dedicated signatures in the inner detector and the muon spectrometer, namely a pair of displaced fat-jets and the associated tracks, respectively. We find that both the HL-LHC and FCC-hh can be sensitive to $|V_{\mu N}| > 10^{-6}$ and $|V_{\mu N}| > 10^{-7}$ with the number of events reaching ${\mathcal {O}}(10)$ and ${\mathcal {O}}(10^3)$, respectively. This allows probing the generation of light neutrino masses through the Seesaw mechanism.

1 Introduction

The discovery of the Higgs boson at the Large Hadron Collider (LHC) experiment has proven that in the Standard Model (SM), masses of the fermions and gauge bosons are generated via the Brout–Englert–Higgs mechanism. However, the origin of light neutrino masses and mixing still remains a key question, which can not be explained by the SM. A number of neutrino oscillation experiments have observed that the solar and atmospheric neutrino mass splittings are $\Delta m^2_{12} \sim 10^{-5}$ $\mathrm{eV}^2$ and $\Delta m^2_{13} \sim 10^{-3}$ $\mathrm{eV}^2$, and the mixing angles are $\theta _{12} \sim 32^\circ$, $\theta _{23} \sim 45^\circ$, and $\theta _{13} \sim 9^\circ$ [1]. There have been a number of beyond-the-Standard-Model (BSM) extensions, which can explain the observed light neutrino masses and mixing and contain SM gauge singlet right-handed neutrinos (RHNs). The $U(1)_{B-L}$ model is such an extension, where the gauge sector has an additional $U(1)_{B-L}$ gauge symmetry [2,3,4,5]. The model contains three right-handed neutrinos $N_i$, which are SM gauge singlets, that however carry non-trivial charges under the $U(1)_{B-L}$. Other than the three RHNs, the model also contains a $Z^\prime$ gauge boson, and a complex scalar S. The light neutrino masses in this model are generated from the lepton number violating (LNV) $d=5$ seesaw operator $LLHH/\Lambda$ [6, 7] via the Type-I seesaw mechanism [8,9,10,11,12].

Being SM gauge singlets, RHNs interact with the SM particles only via their mixing with the active neutrinos, referred to as active-sterile neutrino mixing, which is proportional to $\sqrt{{m_\nu }/{M_N}}$, where $m_{\nu }$ and $M_N$ are the light and heavy neutrino mass scales, respectively. Since $m_\nu <{\mathcal {O}}$ (eV), this mixing is small for RHN mass $M_N\simeq 10-100$ GeV, leading to an suppressed production of RHNs at collider. This limitation can however be evaded in the gauged $B-L$ model, since RHNs can be produced via additional production mechanisms that involve unsuppressed interactions of RHNs with BSM/SM particles [13, 14].

Motivated by this, we study the pair-production of RHNs in the gauged $B-L$ model, primarily via the $B-L$ gauge boson $Z^{\prime }$, $pp\rightarrow Z^\prime \rightarrow NN$. Specifically, we focus on the low RHN mass region, $M_N< M_W$, which gives rise to distinctive signatures. The RHN undergoes three body decays, where we mainly consider $N \rightarrow \mu q q^{\prime }$. Since the decay of N strongly depends on the active-sterile mixing, hence for mixing in agreement with eV light neutrino masses, the RHN is long-lived with the decay vertex considerably displaced from the production vertex. Additionally, due to a large hierarchy between the masses of the $Z^{\prime }$ and N, the decay products of N are collimated, resulting in a displaced fat-jet signature. We hence study distinctive long-lived signatures of N, that can be detected at the High-Luminosity upgrade of the LHC, and the proposed FCC-hh machine. The higher center of mass (c.m.) energy of FCC-hh offers additional advantage. Due to very large c.m. energy, the FCC-hh will be capable of probing much higher $Z^{\prime }$ masses. Production of such heavy $Z^{\prime }$ with subsequent decays to very light N will thus naturally lead to very collimated objects which we are interested in. For previous studies on displaced RHN decay, see [15,16,17,18,19,20,21,22,23,24,25,26,27,28]. A review on potential reach of the LHC to probe long-lived particles has been presented in [29].

In particular, we consider displaced decays of N in two different region of the detectors, a) in the inner detector (ID) and b) the outer region of the detector, occupied by the muon spectrometer (MS). To realise the former, N must have proper decay length ranging between $c\tau \sim$ (1,100) mm, while for the latter, the decay length should be $c\tau \sim$ m. A study of displaced decays in the CMS muon spectrometer has been presented in [30]. We note that the signal description between a) and b) differ widely. For the decay of N in the ID, the signature contains two displaced fat-jets. For N decaying in the MS, absence of tracks in ID and/or energy deposits in the calorimeter hampers reliable identification of charged final states, which are used to make jets. Therefore, a jet description of the final state particles is inadequate in this case, and we instead perform a track-based analysis. We further consider the case where we demand displaced decay of at-least one RHN in the ID/MS. Assuming a background free environment we find that few tens of displaced fat-jet events can be observed at the HL-LHC for c.m.energy $\sqrt{s}=14$ TeV and luminosity ${\mathcal {L}}=3\, \mathrm{ab}^{-1}$. For the FCC-hh that can operate with a much higher c.m. energy $\sqrt{s}=100$ TeV and luminosity ${\mathcal {L}}=30\, \mathrm{ab}^{-1}$, the maximum achievable event numbers increase by an order of magnitude. For the decay of N in the MS, final number of events at the HL-LHC is rather low. This however improves significantly for the FCC-hh.

The discussion proceeds as follows: in Sect. 2, we present a brief review of the model, followed by a discussion of existing constraints in Sect. 3. In Sects. 4, and  5, we discuss the pair-production of RHN and its displaced decays. We present an extensive analysis for the HL-LHC and FCC-hh in Sects. 6 and  7, respectively. Finally, we present a summary in Sect. 8.

2 The $B-L$ gauge model

We consider the gauge $B-L$ model, which in addition to the SM particles also contains three right-handed neutrinos $\nu _R$, a BSM Higgs field S and a BSM gauge boson $B^{\prime }_{\mu }$. The gauge group is $SU(3)_c\times SU(2)_L \times U(1)_Y \times U(1)_{B-L}$, where the scalar and fermionic fields S and ${\nu _R}_i$ have $B-L$ charges $B-L = +2$ and $-1$, respectively. The SM states have the conventional $B-L$ charges. The complex scalar field S acquires a vacuum expectation value (vev) $v_{BL}\ne 0$ and breaks the $B-L$ gauge symmetry. The states ${\nu _R}_i$ contribute to the light neutrino mass generation via the seesaw mechanism. The complete Lagrangian of the model has the form

$$\begin{aligned} {\mathcal {L}}= {\mathcal {L}}_{SM}+{\mathcal {L}}_{B-L}, \end{aligned}$$

(2.1)

where ${\mathcal {L}}_{B-L}$ is the $B-L$ Lagrangian, and ${\mathcal {L}}_{SM}$ is the Lagrangian for the SM sector. The $B-L$ Lagrangian has the form

$$\begin{aligned} \begin{aligned} {\mathcal {L}}_{B-L}&=\left( D_{\mu } {S}\right) ^{\dagger }\left( D^{\mu } {S}\right) -\frac{1}{4} F^\prime _{\mu \nu } F^{\prime \mu \nu }\\&\quad +{i} {{{\bar{\nu }}}_{Ri}} \gamma ^{\mu } D_{\mu } {\nu _R}_{i}-V_{B-L} \left( {\phi }, {S}\right) \\&\quad -\sum _{i=1}^{3} y^{M} {S} \bar{\nu _R}_{i}^{c} {\nu _R}_{i}-\sum _{i, j=1}^{3} y^{\nu }_{ij} {\bar{L}}_{i} {\tilde{\phi }} {\nu _R}_{j}+h . c . \ , \end{aligned} \end{aligned}$$

(2.2)

with

$$\begin{aligned} V_{B-L}\left( \phi , S\right)= & {} \mu _{S}^{2} S^{\star } {S}+\mu _{\phi }^{2} {\phi }^{\dagger } {\phi }+\lambda _{1}\left( {\phi }^{\dagger } {\phi }\right) ^{2}\nonumber \\&+\lambda _{2}\left( {S}^{\star } {S}\right) ^{2}+\lambda _{3}\left( {\phi }^{\dagger } {\phi }\right) \left( {S}^{\star } {S}\right) . \end{aligned}$$

(2.3)

In the above, $D_\mu$ represents the covariant derivative [31],

$$\begin{aligned} D_\mu&= \partial _{\mu } + ig_{s}{\mathcal {T}}_\alpha G_\mu ^\alpha + igT_a W_\mu ^a \nonumber \\&\quad + ig_1 Y B_\mu + i ({\tilde{g}}Y + g^{\prime } Y_{B-L}) B^\prime _\mu , \end{aligned}$$

(2.4)

where $G^\alpha _\mu$, $W^a_\mu$, $B_\mu$ are the SM gauge fields with associated couplings $g_s$, g, $g_1$ and the respective generators are ${\mathcal {T}}_\alpha$, $T_a$, Y, respectively. $B^\prime _\mu$ represents the gauge field for the $U(1)_{B-L}$ gauge symmetry, $g^{\prime }$ represents the respective gauge coupling and the $B-L$ quantum number is denoted by $Y_{B-L}$. We neglect the mixing between $U(1)_{B-L}$ and $U(1)_{Y}$ to simplify the model, i.e. we consider the minimal gauged $B-L$ model. This model is therefore valid in the limiting case of small mixing between $B_\mu$ and $B^\prime _\mu$. The gauge sector of the model also includes the following kinetic term for the gauge field $B^{\prime }_{\mu }$,

$$\begin{aligned} \mathcal{L}^{BL}_{kin} = -\frac{1}{4} F^{\prime \mu \nu } F_{\mu \nu }^\prime , \end{aligned}$$

(2.5)

where $F^\prime _{\mu \nu } = \partial _\mu B^\prime _\nu - \partial _\nu B^\prime _\mu$ is the field strength tensor of the $B-L$ gauge group. The kinetic term for the RHN and the SM fermion fields are

$$\begin{aligned} \mathcal{L} = i\overline{\nu _{Ri}}\gamma _\mu D^\mu \nu _{Ri}+ i\overline{\psi _{i}}\gamma _\mu D^\mu \psi _{i} \end{aligned}$$

(2.6)

where, $\psi$ represents the SM fermion fields. These fields receive an additional term in their covariant derivatives, since they are non-trivially charged under the $B-L$ gauge symmetry, with $Y_{B-L} = 1/3$ and $-1$ for the quark and lepton fields, respectively. In the above a summation over the fermion species and generations is implied.

Due to the additional matter content, the gauged $B-L$ model offers three portals for RHN production.

• Active-sterile neutrino mixing and neutrino mass: In Eq. (2.2), the Yukawa matrix $y^\nu$ represents the Dirac Yukawa coupling and $y^M$ is the Yukawa coupling connecting the RHNs with the complex scalar field S. The RHN mass is generated due to breaking of the $B-L$ symmetry, with the mass matrix given by $M_R = y^M \langle S \rangle$. The light neutrinos mix with the RHNs via the Dirac mass matrix $m_D = y^\nu v/\sqrt{2}$. The complete mass matrix in the $(\nu _L, \nu ^c_R)$ basis has the form

  $$\begin{aligned} \mathcal{M} = \begin{pmatrix} 0 &{} m_D \\ m^T_D &{} M_R \end{pmatrix}, \end{aligned}$$

  (2.7)

  where

  $$\begin{aligned} m_D = \frac{y^\nu }{\sqrt{2}}v, \quad M_R = \sqrt{2} y^M v_{BL}. \end{aligned}$$

  (2.8)

  Here, $v = \langle \phi ^0 \rangle$ and $v_{BL} = \langle S \rangle$ are the vacuum expectation values for electroweak and $B-L$ symmetry breaking, respectively. In the seesaw limit, $M_R \gg m_D$, the light and heavy neutrino masses are

  $$\begin{aligned} m_\nu \sim - m_D M^{-1}_R m^T_D, \quad M_N \sim M_R. \end{aligned}$$

  (2.9)

  The flavour and mass eigenstates of the light and heavy neutrinos are connected as

  $$\begin{aligned} \begin{pmatrix} \nu _L \\ \nu ^c_R \end{pmatrix} = \begin{pmatrix} V_{ll} &{} V_{lN} \\ V_{Nl} &{} V_{NN} \end{pmatrix} \begin{pmatrix} \nu ^m_L \\ {N^c_R} \end{pmatrix}, \end{aligned}$$

  (2.10)

  where $\nu ^m_L$ and $({N^c_R})^T$ represent the left-chiral mass basis for the light and heavy neutrinos, respectively. In our subsequent discussion, we represent the physical Majorana fields for the light and heavy neutrinos via $\nu ^m=\nu ^m_L+(\nu ^m_L)^c$ and $N=N_R+(N_R)^c$, respectively. In the above, we schematically write the 6-dimensional mixing matrix in terms of 3-dimensional blocks. Assuming the charged lepton mass matrix to be diagonal, the sub-block $V_{ll}$ can approximately be considered as the PMNS mixing matrix $U_\text {PMNS}$. The other sub-block $V_{lN}$ represents the mixing between the light and RHN states and is referred as active-sterile mixing. In general, $V_{lN}$ is an arbitrary $3\times 3$ matrix. However, to pursue a collider study on the proposed signature, it is sufficient for us to consider one generation of RHNs mixing only with muon flavour $V_{\mu N}$, which we follow in the subsequent sections. This is the same active-sterile neutrino mixing portal present in the minimal extensions of the SM.

• $Z^{\prime }$ gauge boson mass and gauge coupling $g^\prime$ : Due to the presence of an additional $U(1)_{B-L}$ gauge symmetry, the model contains a BSM gauge boson $B^{\prime }_{\mu }$. We refer to the massive state as $Z^{\prime }$. Similar to the RHNs, the additional neutral gauge boson mass $M_{Z^{\prime }}$ is generated via spontaneous breaking of $B-L$ gauge symmetry. The mass of $Z^{\prime }$ is related to the symmetry breaking scale $v_{BL}$ as

  $$\begin{aligned} M_{Z^{\prime }}=2g^{\prime }v_{BL}, \end{aligned}$$

  (2.11)

  where $g^{\prime }$ is the associated $B-L$ gauge coupling constant. The coupling of this gauge boson to the SM fermions, proportional to $g_{B-L}$ gauge coupling. Production of $Z^{\prime }$ and its subsequent decays to RHN pair offers another portal for RHN production.

• SM Higgs and BSM Higgs: After spontaneous symmetry breaking, the SM Higgs doublet $\phi$ and BSM scalar S are represented as

  $$\begin{aligned} \phi = \begin{pmatrix} 0 \\ \dfrac{v+h_{1}}{\sqrt{2}} \end{pmatrix}, \ \ \ S= \dfrac{v_{BL}+h_{2}}{\sqrt{2}} \end{aligned}$$

  (2.12)

  with the dynamical states $h_1$ and $h_2$. Owing to the non-zero $\lambda _{3}$, $h_1$ and $h_2$ mix with each other which leads to the scalar mass matrix given by

  $$\begin{aligned} {\mathcal {M}}^2_{\text {scalar}} = \begin{pmatrix} 2\lambda _1 v^2 &{} \lambda _{3}\,v_{BL}\,v \\ \lambda _{3}\,v_{BL}\,v &{} 2 \lambda _2 v^2_{BL} \end{pmatrix}. \end{aligned}$$

  (2.13)

  The basis states $h_1$ and $h_2$ can be rotated by a suitable angle $\alpha$ to the new basis states $H_1$ and $H_2$, representing the physical fields,

  $$\begin{aligned} H_{1}&= h_{1} \cos \alpha - h_{2}\ \sin \alpha , \nonumber \\ H_{2}&= h_{1} \sin \alpha + h_{2}\ \cos \alpha . \end{aligned}$$

  (2.14)

  Here $H_1$ is the SM-like Higgs and $H_2$ is the mostly BSM Higgs. The mixing angle between the two states is

  $$\begin{aligned} \tan 2\alpha =\frac{v v_{BL}\lambda _{3}}{v^{2}\lambda _{1}-v_{BL}^{2}\lambda _{2}}, \end{aligned}$$

  (2.15)

  and the mass square eigenvalues of $H_1$ and $H_2$ are given by,

  $$\begin{aligned} M_{H_1,H_2}^2= & {} \lambda _1 v^2+\lambda _{2}v_{BL}^2\nonumber \\&\pm \sqrt{(\lambda _{1} v^2-\lambda _{2}v_{BL}^2)^{2}+(\lambda _{3}vv_{BL})^2}. \end{aligned}$$

  (2.16)

In case of a large scalar mixing angle, the Higgs-mediated pair production of RHNs is possible [18]. In what follows, we consider the scalar mixing to be negligible, for which the SM and BSM Higgs masses have the form, respectively,

$$\begin{aligned} M_{H_1} \sim \sqrt{\lambda _1} v, \, \, \, \,\, M_{H_2} \sim \sqrt{\lambda _2} v_{BL}. \end{aligned}$$

(2.17)

Due to this choice of small mixing, any BSM Higgs production and its decay to two RHNs will be suppressed.

The RHNs have charged current and neutral current interactions with the SM fields, with the scalars $H_1,H_2$ and the gauge boson $Z^{\prime }$. The respective charged current interaction Lagrangian has the form,

$$\begin{aligned} -{\mathcal {L}}_{CC}= & {} \frac{g}{\sqrt{2}} W^-_{\mu }{\bar{\ell }} \gamma ^{\mu } \nu _\ell + \mathrm{H.c.} \ \nonumber \\= & {} \frac{g}{\sqrt{2}} W^-_{\mu }{\bar{\ell }} \gamma ^{\mu } \left( (V_{ll})_{\ell i} P_L {\nu ^m_i}\right. \nonumber \\&\left. + (V_{lN})_{\ell i} P_L N_i \right) + \mathrm{H.c.}, \end{aligned}$$

(2.18)

where $\ell$, i represent generation indices and $P_L$ is the left-chirality projection operator $P_L=\frac{1-\gamma ^5}{2}$. Similarly, the NC interaction is given by

$$\begin{aligned} -{\mathcal {L}}_{NC}&=\frac{g}{2 \cos \theta _w} Z_{\mu } \bar{\nu _{\ell }} \gamma ^\mu \nu _{\ell } + {g^{\prime }} Z^{\prime }_{\mu } {\bar{\nu }}_R \gamma ^\mu \nu _R \nonumber \\&\approx \frac{g}{2 \cos \theta _w} Z_{\mu } \Big [ {{\bar{\nu }}^m}_i \gamma ^{\mu } P_L {\nu ^m_i} \nonumber \\&\quad + \{(V_{ll}^{\dagger } V_{lN})_{ij} {{\bar{\nu }}}_i \gamma ^{\mu } P_L {N}_j + \mathrm{H.C.}\}\Big ] \nonumber \\&\quad + g^{\prime } Z^{\prime }_{\mu } {{\bar{N}}}_i \gamma ^{\mu } P_R {N}_i. \end{aligned}$$

(2.19)

In the above, i, j represent generation indices. The interactions with the SM and BSM Higgs have the form

$$\begin{aligned} {\mathcal {L}}^N_{int}= & {} y^M \cos \alpha \ {\bar{N}} H_2 N+ y^M \sin \alpha \ {\bar{N}} H_1 N\nonumber \\&+\bigg [V_{ll}^{\dagger } V_{lN}\cos \alpha \ \bar{\nu ^m} H_1 N+H.C. \bigg ]. \end{aligned}$$

(2.20)

While the $B-L$ model offers several interesting portals, in the scenario we consider $Z^{\prime } \rightarrow N N$ turns out to be the most interesting channel. We therefore concentrate on this production mode below.

3 Experimental constraints

Fig. 1

Limits in the $M_{Z^{\prime }}$ and $g^{\prime }$ plane derived from ATLAS [32], and CMS [33] searches. The limit from LEP-II [34] is shown for comparison by the red dot-dashed line. The green dashed and pink dashed lines represent the projection for HL-LHC [35] and FCC-hh [36], respectively. The black horizontal dotted line indicates $g^\prime$ such that $\Gamma /M_{Z^{\prime }}=0.6\%$

The mass and coupling of $Z^{\prime }$ and N in the gauged $B-L$ model are significantly constrained from the non-observation of direct and indirect signatures at collider and other searches. In addition, eV light neutrino masses impose additional constraints on the model parameters. Below we briefly summarize the different existing constraints.

• Heavy resonance search: The search for a massive resonance at LHC decaying to di-lepton/di-jet imposes tight limits on the respective production cross-section. The 13 TeV LHC search for a heavy resonance decaying into two leptons $pp\rightarrow Z^\prime \rightarrow \ell ^+\ell ^-$ puts a constraint $M_{Z^{\prime }} > 5.0$ TeV at 90$\%$ CL, assuming a 100$\%$ branching ratio of $Z^{\prime }$ decaying into two leptons. For different branching ratios, the limit relaxes. In Fig. 1, we translate the LHC di-lepton constraint in the $M_{Z^{\prime }} \text{-- } g^{\prime }$, plane. We adopt the following procedure in translating the experimental bound. For the ATLAS search with ${\mathcal {L}}=139\,\mathrm{fb}^{-1}$, we consider ${\sigma ^{\text {th}}(pp\rightarrow Z^\prime \rightarrow \ell ^+\ell ^-)} < \sigma _{obs}$, where the observed limit corresponds to the $95\%$ CL result from [32]. The CMS limit is set on the relative cross section,

  $$\begin{aligned} \sigma ^{\text {rel}}=\frac{{\sigma \times Br}^{\text {obs}}\big |_{Z^{\prime }}}{{\sigma \times Br}^{\text {obs}}\big |_{Z}}\times 1928\, \text {pb}. \end{aligned}$$

  Here, ${\sigma \times Br}^{\text {obs}}\big |_{Z}$ is the observed cross section in the $m_{\ell ^+\ell ^-}=60-120$ GeV window, which can be calculated as $N^{{\mathrm{obs}}}/(\mathrm{Acc} \times \mathrm{Eff} \times {\mathcal {L}})$¹. Therefore, to obtain the $95\%$ CL limit on ${\sigma \times Br}^{\text {obs}}\big |_{Z^{\prime }}$ we fold $\sigma ^{\text {rel}}$ with ${\sigma \times Br}^{\text {obs}}\big |_{Z}/1928$ pb. In Fig. 1, we represent the $ee+\mu \mu$ combined limit from the CMS search (purple line) and the from the ATLAS search (orange line). We also show $\Gamma /M_{Z^{\prime }}=0.6\%$ limit (black dotted line) for the CMS search, which we have used. Alongside, we also show the future sensitivity of HL-LHC with 3000  $\mathrm{fb}^{-1}$ [35] (dashed green line) and projection for FCC-hh (dashed magenta line), recast from [36]. We also note that the former search at LEP-II [34, 37,38,39] for a massive resonance constrains $M_{Z^\prime }$ and $g^\prime$, and thus the $B-L$ breaking scale as $v_{BL} \equiv M_{Z^\prime }/{(2g^\prime )} \ge 3.45$ TeV. This limit (red dot-dashed line) is considerably relaxed w.r.t. the present LHC limit.

• Search for RHN: Other than the constraint on $Z^{\prime }$, the RHN mass and mixings are also constrained, both from neutrino mass measurements as well as direct searches. In the present work, we consider relatively light RHN, 10 GeV $< M_N < M_W$, which are mainly produced from the $Z^{\prime }$ mediated channel.The RHN mass and its mixing with the active neutrinos are tightly constrained from light neutrino mass measurements. As we are working in a Type-I seesaw scenario with $B-L$ gauge symmetry, the light neutrino mass $m_\nu \simeq {m^2_D}/M_R \simeq V^2_{lN}M_N$ where the active-sterile mixing angle $V_{lN} \simeq m_D/M_R$. This fixes the active-sterile mixing,

  $$\begin{aligned} V_{\ell N} \approx 10^{-6}\sqrt{\frac{m_\nu / (0.1~\text {eV})}{M_N / (50~\text {GeV})}}. \end{aligned}$$

  (3.1)

  However, we do not enforce this condition and keep $V_{\ell N}$ as a free parameter. The low mass RHN is further constrained by LHC searches for a heavy neutral lepton [40] via the decay mode $p p \rightarrow W \rightarrow \ell N$ and the decay $N \rightarrow \ell \ell \nu / \ell jj$. The search for a displaced neutral lepton [40, 41] in particular constrains active-sterile mixing as $V_{\mu N} < 10^{-2}$ for RHN mass ${\mathcal {O}}(10)$ GeV. Other searches such as [42,43,44] mainly target heavier masses, where the RHN is not displaced, and hence not relevant for our case.

• LLP searches from SM Higgs decay: There are different exotic decays of the SM Higgs boson which give rise to distinctive signatures. The decay $H_1\rightarrow NN$ followed by the decay of N into $\ell jj/\ell \ell \nu$ final states gives rise to displaced lepton/displaced jet signatures, for suppressed active-sterile mixing. While kinematically allowed, it has a very suppressed branching ratio due to our choice of a small $\alpha$. The partial decay width of this mode has the form

  $$\begin{aligned}&\Gamma (H_1 \rightarrow NN)=\frac{3 M_{H_1} (y^M)^2 \sin ^2\alpha }{8\pi }\left( 1-\frac{4 M_{N}^2}{M_{H_1}^2}\right) ^{3/2}, \nonumber \\&\quad \text {where}\,~~ y^M=\frac{M_N}{\sqrt{2}v_{BL}}, \end{aligned}$$

  (3.2)

  which clearly shows a $\sin \alpha$ suppression. For instance, the branching ratio BR$(H_1\rightarrow NN)$ becomes $0.005\%$ for $\sin \theta = 0.03$, $v_{BL}=4$ TeV and $M_N=50$ GeV. There are different CMS and ATLAS searches [45,46,47,48] to probe exotic decays of the SM Higgs into two LLP states, which can be used to constrain this decay mode. This includes a search for exotic decays of the Higgs boson into LLPs in the inner detector [45,46,47,48], or a search for LLPs decaying in the ATLAS muon spectrometer [49,50,51], and in the CMS end-cap muon detectors [52]. Due to a very suppressed branching ratio, $M_N$ and $V_{\ell N}$ are unconstrained from these searches.

Fig. 2

Feynman diagram representing pair-production of two RHNs via $Z^{\prime }$ mediation and its displaced decays to a $lqq^{\prime }$ final states

Fig. 3

Cross-section $\sigma (pp\rightarrow Z^\prime \rightarrow NN)$ as a function of $M_{Z^\prime }$ at the HL-LHC and FCC-hh, for $M_N=20$ GeV. The black solid (dashed) line represents LHC limits for $g^\prime =0.1\ (3\times 10^{-3})$ as shown in Fig. 1

4 Pair production and decay of RHN at a pp machine

In the light of discussion above, the RHN pair production primarily occurs via the $p p \rightarrow Z^{\prime } \rightarrow N N$ channel. The cross-section depends strongly on the gauge coupling $g^{\prime }$, the mass of $Z^{\prime }$ gauge boson $M_{Z^{\prime }}$, its decay width $\Gamma _{Z^{\prime }}$, as well as the mass of the RHN $M_N$. We show the respective Feynman diagram for this process in Fig. 2.

In Fig. 3, we show the pair production cross-section, $\sigma (pp\rightarrow Z^\prime \rightarrow NN)$ as a function of $M_{Z^\prime }$ at the HL-LHC (red solid and dotted lines) and FCC-hh (blue solid line), respectively. This cross-section is calculated using MadGraph. For illustration, we assume two different values of couplings, $g^\prime =0.1, 3\times 10^{-3}$ and $M_N=20$ GeV. We show the LHC constraints (as shown in Fig. 1) for $g^\prime$ values 0.1 (black solid line) and $3\times 10^{-3}$ (black dotted line), respectively. For instance, for $g^{\prime }=3\times 10^{-3}$, the heavy resonance search in $\ell ^+\ell ^-$ decay channel rules out any value of $M_{Z^{\prime }}\lesssim 0.7$ TeV. For $g^{\prime }=0.1$ the exclusion limit reaches higher value, $M_{Z^{\prime }}\gtrsim 4.5$ TeV. For $M_{Z^{\prime }}>2M_N$ as considered in this analysis, the on-shell production of $Z^{\prime }$ and its decay will dominate the pair-production cross-section $\sigma (p p \rightarrow N N)$. As can be seen from the figure, for the same gauge coupling $g^{\prime }$, the ratio of cross-section ${\mathcal {R}}=\sigma (p p \rightarrow N N)_{ HL-LHC}/ \sigma (p p \rightarrow N N)_{ FCC-hh}$ varies over a wide range. For on-shell $Z^{\prime }$ production with narrow width approximation, $\Gamma _{Z^{\prime }}/M_{Z^{\prime }}\lesssim 10\%$, this can be expressed as,

$$\begin{aligned}&\sigma (p p \rightarrow N N)_{ HL-LHC}/ \sigma (p p \rightarrow N N)_{ FCC-hh}\nonumber \\&\quad \approx \sigma (p p \rightarrow Z^{\prime })_{ 14}/ \sigma (p p \rightarrow Z^{\prime })_{ 100} \end{aligned}$$

(4.1)

With an increase in the c.m. energy from $\sqrt{s}=14$ TeV to 100 TeV, the ratio ${\mathcal {R}}$ increases by order of magnitude, and the increase is larger for higher masses. This occurs as the partonic c.m. energy is larger in a 100 TeV collider compared to the available partonic c.m. energy for 14 TeV.

Fig. 4

Cross-section $\sigma (pp\rightarrow Z^\prime \rightarrow NN)$ as a function of $M_N$ and $M_{Z^\prime }$, for $g^\prime =0.003$ and c.m. energy $\sqrt{s}=14$ TeV (left) and $g^\prime =0.1$ and c.m. energy $\sqrt{s}=100$ TeV (right). The shaded regions are excluded by the LHC searches

We also show the variation of cross-section for HL-LHC and FCC-hh with respect to the variation of both $M_N$ and $M_{Z^{\prime }}$ in Fig. 4. For this figure we consider the same values of gauge coupling that we use for Fig. 3. The shaded vertical region shows the present constraint from the LHC, both in the left and right panel. Comparing the RHN pair production cross-section at LHC (left panel) and FCC-hh (right panel), a few observations can be made. First and foremost, the production cross-section drops sharply for off-shell $Z^{\prime }$. Second, the production cross section is generally much larger at FCC-hh compared to LHC even for heavier $Z^{\prime }$ thanks to the increased c.m. energy. Finally, the production cross section is almost constant as long as $Z^{\prime } \rightarrow N N$ threshold is not reached.

Decay of RHN: The RHN interactions to the SM lepton and W boson are governed by the active-sterile mixing $V_{\ell N}$. It also interacts with a light neutrino and a $Z, H_1$, as well as $Z^{\prime }$ and BSM Higgs $H_2$. For the mass range being considered in this work, $M_Z^{\prime }, M_{H_2} \gg M_{W}, M_Z, M_{H_1} > M_N$, the RHN decays dominantly via off-shell $W,Z,H_1$ states into three SM fermions. The partial decay widths for the $N \rightarrow l q q^{\prime }$, $\nu f {\bar{f}}$ and $\nu \nu \nu$ modes have the following expressions [53].

$$\begin{aligned} \Gamma (N \rightarrow \ell _\alpha ^-u {\bar{d}})= & {} N_c |V^{CKM}_{ud}|^2 |V_{\ell N}|^2\nonumber \\&\frac{ G_F^2 M_{N}^5}{192 \pi ^3} {\mathcal {I}}(x_{u},x_{d},x_{\ell }), \end{aligned}$$

(4.2)

with ${\mathcal {I}}(x_{u},x_{d},x_{\ell })= 12 \int _{(x_{d}+x_{\ell })^2}^{(1-x_{u})^2}\frac{dx}{x} (1+x_{u}^2-x) (x-x_{d}^2-x_{\ell }^2)\lambda ^{\frac{1}{2}}(1,x,x_{u}^2)\lambda ^{\frac{1}{2}}(x,x_{\ell }^2,x_{d}^2)$, $x_{u/d/\ell }=\frac{m_{u/d/\ell }}{M_{N}}$, $\lambda (a,b,c)=a^2+b^2+c^2-2ab-2bc-2ca$, and $N_c=3$ is the color factor. The partial decay width for $N \rightarrow \ell _\alpha ^-\nu _\beta \ell ^+_\beta$ with generation index $\alpha \ne \beta$ have the following expression

$$\begin{aligned} \Gamma (N \rightarrow \ell _\alpha ^-\nu _\beta \ell ^+_\beta )= |V_{\alpha N}|^2\frac{ G_F^2 M_{N}^5}{192 \pi ^3} {\mathcal {I}}(x_{\ell _\alpha },x_{\ell _\beta },x_{\nu _\beta }). \end{aligned}$$

(4.3)

The partial decay width for the other mode with a same $\alpha =\beta$ is given by

$$\begin{aligned}&\Gamma (N \rightarrow \nu _\alpha f {\bar{f}})= N_c |V_{\alpha N}|^2\frac{ G_F^2 M_{N}^5}{192 \pi ^3} \nonumber \\&\qquad \times \Big [ C_1^f\Big ( (1-14x^2-2x^4-12x^6)\sqrt{1-4x^2}\nonumber \\&\qquad +12 x^4(x^4-1)L(x)\Big ) \nonumber \\&\qquad + 4C_2^f\Big ( x^2(2+10x^2-12x^4)\sqrt{1-4x^2}\nonumber \\&\qquad +6 x^4(1-2x^2+2x^4)L(x)\Big )\Big ]. \end{aligned}$$

(4.4)

Here $x=\frac{m_{f}}{M_{N}}$, $L(x)=\ln \Big [ \frac{1-3x^2-(1-x^2)\sqrt{1-4x^2}}{x^2(1+\sqrt{1-4x^2})}\Big ]$. The values of $C_1^f$ and $C_2^f$ are given in [53]. Finally, the decay to the light neutrinos has the partial width

$$\begin{aligned} \Gamma (N \rightarrow \nu _\alpha \nu \nu )= |V_{\alpha N}|^2\frac{ G_F^2 M_{N}^5}{96 \pi ^3}. \end{aligned}$$

(4.5)

In addition to this there can also be two body decays $N \rightarrow l^{\pm } \pi ^{\pm }$, however, this is suppressed in our case, as we consider $M_N > 1$ GeV $\sim$ the scale of non-perturbative QCD. Therefore, we do not consider this channel in our analysis. For $M_N=10-70$ GeV, the dominant decay mode is $N\rightarrow \mu jj$ with a branching ratio BR$(N\rightarrow \mu jj)\simeq 0.5$. Branching ratios for other decay modes are BR$(N\rightarrow \nu jj)\simeq 0.2$, BR$(N\rightarrow \nu _\mu l^+l^-)\simeq 0.10$, BR$(N\rightarrow \mu e \nu _e+ \mu \tau \nu _\tau )\simeq 0.15$ and BR$(N\rightarrow \nu \nu \nu )\simeq 0.05$.

5 RHN decay in inner detector and muon spectrometer

We consider $Z^{\prime }$ mediated RHN pair production with subsequent decays to $l q q^{\prime }$. To be specific, we consider the $N \rightarrow \mu q q^{\prime }$ final state. Signatures with an electron in the final state $N \rightarrow e q q^{\prime }$ can also potentially be explored by similar search strategies. Specifically, we consider two scenarios: (a) when the N decays within the inner detector (ID) of the HL-LHC/FCC-hh, and (b) in the muon spectrometer (MS), which is the farthest part of detector. For proper decay length of the RHN, $c\tau \simeq {\mathcal {O}}(1-100)$ mm, decays occur in the ID and for $c\tau \simeq {\mathcal {O}}(1)$ m in the MS. Decay of RHN in the ID leads to a bunch of tracks in the ID and associated energy deposit in the calorimeter. Whereas, decay in the MS results in tracks in the MS without any associated activity in the ID and calorimeter. Hence the the search strategies for these two different signatures differ widely.Major sources of background for a displaced vertex signal in the ID are decays of long-lived hadrons, interaction of particles with the detector material in the ID and fake vertices (which are originated from the random crossing of tracks). Such backgrounds can be reduced by imposing selection cuts on decay position of LLP, number of tracks associated with the displaced vertex and its mass [50]. For displaced signatures in the MS, the main source of background is QCD jets that punch through the calorimeter into the MS. Such background can be reduced by imposing isolation cuts on the vertex in the MS. Other sources of background are electronic noise, cosmic-ray muons, and machine-induced background, which can be handled by demanding a minimum number of hits associated with the vertex [50].

In order to analyse the model signatures, we simulate the events using the following steps. We use the FeynRules [54, 55] model file and Universal FeynRules Output (UFO) [56] corresponding to Ref. [18], which we use in combination with the Monte Carlo event generator MadGraph5aMC@NLO -v2.6.7 [57] to generate events at the parton level. The FeynRules [54, 55] model file and UFO are publicly available from the FeynRules Model Database at [58]. For every signal sample, we generate 50k signal events with MadGraph5aMC@NLO -v2.6.7, where we use the NN23LO1 PDF set [59]. We set max jet flavour at 5 to take account of the b quark contribution in the PDF. We then pass the generated parton level events on to PYTHIA v8.235 [60] which handles the initial and final state radiation, parton showering, hadronization, and heavy hadron decays. The clustering of the events and jet formation are performed by FastJet v3.2.1 [61]. We consider a Cambridge-Aachen jet algorithm [62] for jet clustering with radius parameter ${\mathcal {R}}=1.0$. We analyse events at the generator level with PYTHIA v8.235.

5.1 Decay probability of N

Fig. 5

Distribution of the decay length of N in the lab frame at the HL-LHC and at FCC-hh for the considered masses of $Z^{\prime }$ and N

For $M_N<M_W$ and for the viable range of active-sterile mixing that satisfies the light neutrino mass constraint, the RHN undergoes displaced decays with a displacement ${c\tau _N} \sim$mm or even longer. N decays via off-shell W, Z, H states with the decay length,²

$$\begin{aligned} c\tau _N\simeq 25 \times \Big ( \frac{10^{-6}}{|V_{\mu N}|}\Big )^2 \times \Big ( \frac{100 ~\mathrm{GeV}}{M_N}\Big )^5~\mathrm{mm} \end{aligned}$$

(5.1)

In Fig. 5, we show the distribution of the decay length of N in the lab frame for $\sqrt{s}=14$ TeV and $\sqrt{s}=100$ TeV for few benchmark values of $M_{Z^{\prime }}$, and $M_N$, which we later on use in the analysis. For this distribution we generate $p p \rightarrow Z^{\prime } \rightarrow N N$ in MadGraph, and to keep the information of decay length, use the time-of-flight option of MadGraph. As can be seen, in the majority of events, decay length of N is more than a mm, thereby giving rise to a decay vertex considerably displaced from the production vertex of N.

As we have two RHNs from the channel $p p \rightarrow Z^{\prime } \rightarrow N N$, either N can decay in different parts of the HL-LHC and FCC-hh detectors. The probability that both RHNs decay within a distance interval $(L_1, L_2)$ from their production vertex is given by

$$\begin{aligned}&{\mathcal {P}}(L_1, L_2, \sqrt{s}, M_N,M_{Z^\prime },\theta ) \nonumber \\&\quad = \!\!\int \! db_1 db_2 \ f(\sqrt{s}, M_N,M_{Z^\prime },b_1,b_2)\nonumber \\&\quad \times \prod _{i=1}^{2}\Big [e^{\frac{-L_1}{b_ic\tau (\theta )}}-e^{\frac{-L_2}{b_ic\tau (\theta )}}\Big ], \end{aligned}$$

(5.2)

where $c\tau$ is proper decay length, $\theta \equiv V_{\mu N}$ is the active-sterile neutrino mixing and $b_{1,2}=(\beta \gamma )_{1,2}$ are the boost factors of the two RHNs, and $f(\sqrt{s}, M_N,M_{Z^\prime },b_1,b_2)$ is the distribution function of the boost.

The probability that one RHN decays within the interval $(L_1, L_2)$ and the other within interval $(L_3, L_4)$ is given by

$$\begin{aligned}&{\mathcal {P}}(L_1, L_2,L_3,L_4,\sqrt{s}, M_N,M_{Z^\prime }, \theta ) \nonumber \\&\quad = 2 \times \int db_1 db_2 \ f(\sqrt{s}, M_N,M_{Z^\prime },b_1,b_2) \nonumber \\&\qquad \times \Big [e^{\frac{-L_1}{b_1c\tau (\theta )}}-e^{\frac{-L_2}{b_1c\tau (\theta )}}\Big ] \Big [e^{\frac{-L_3}{b_2c\tau (\theta )}}-e^{\frac{-L_4}{b_2c\tau (\theta )}}\Big ]. \end{aligned}$$

(5.3)

The decay probability of one RHN decaying within length $L_1-L_2$ then becomes

$$\begin{aligned}&{\mathcal {P}}(L_1, L_2,\sqrt{s}, M_N,M_{Z^\prime },\theta )\nonumber \\&\quad = 2 \times \int db \ f(\sqrt{s}, M_N,M_{Z^\prime },b) \Big [e^{\frac{-L_1}{b c\tau (\theta )}}-e^{\frac{-L_2}{b c\tau (\theta )}}\Big ], \end{aligned}$$

(5.4)

where b is the boost factor of the respective RHN. We use the boost distributions of N from the HepMC file [63], which has been generated by showering the LHE events [64] from MadGraph.

Table 1 Coverage of geometric acceptance of various sub-detectors of the HL-LHC and FCC-hh

Fig. 6

Decay probability of one RHN (left) and two RHN (right) in different parts of the HL-LHC detector. For this figure, we consider $M_N=20$ GeV, $M_{Z^\prime }=1$ TeV. ID, MS, CAL and MET represent the decay of N in the inner detector, muon spectrometer, calorimeter and outside of the detector, respectively. In the right panel, IDMS (IDCAL) represents the joint probability of one N decaying in the ID and the other decaying in the MS (CAL). The gray band indicates the range of $V_{\mu N}$ satisfying light neutrino mass, $m_\nu \simeq [0.01,0.1]$ eV

To estimate the probability of a RHN decay in various parts of the HL-LHC and FCC-hh detectors, we model the detector geometries in a simple fashion. In Table 1, we present the fiducial volumes of the sub-detectors, where the decay of a RHN leads to observable signatures. For the ID we demand RHN decays in between 2 mm and 300 mm from the interaction point. The RHN decays in the ID lead to multi-track displaced vertices. Similarly, decays in the MS create a number of tracks. In the ID, the vertex reconstruction efficiency is typically large [50], while it drops significantly for decays in the MS. Therefore, while on one hand a large detector volume is expected probe small mixing angles, reduced vertex reconstruction efficiency in the outer parts of the detector counter balances this advantage. For this we consider that RHN decays within 4000-7000 mm where muon region of interest³(ROI) trigger efficiency is higher [51]. Hadronic decays of RHN in the outer edge of the ECal or in the HCal leads to a distinct signatures marked by significantly large energy deposit in HCal compared to that in the ECal [65]. We however do not consider such signature in the present analysis and rather restrict it to RHN decays in the ID and in the MS. For the FCC-hh detector, coverage of various sub-detectors used in the analysis, we refer the reader FCC-CDR [66].

In Fig. 6, we show the probability of the RHNs decaying in different modules of the HL-LHC detector. The left panel represents the decay probability of a single N in the detector and in the right panel, we show the probability for both the Ns to undergo displaced decays. The acronyms ID, CAL, MS, MET refer to the decay of RHN inside the inner detector, the calorimeter, the muon spectrometer, and outside of the CMS and ATLAS detector, respectively. From the figure, it is evident that the probability is maximal if the RHN decays inside the ID or outside of the detector, and it is minimal, if the decay happens in the MS . Given that the MS is the farthest part of the detector, this observation can be counter intuitive. However, as we consider decays strictly within the MS i.e. within 4000–7000 mm, the integrated probability distribution within these two end points leads to rather small values. The probability for other configurations, such as decays within the calorimeter, or cross correlated signatures such as simultaneous decays of two RHNs in the ID and calorimeter/MS is in between these two extremes. We show the respective probabilities for the FCC-hh in Fig. 7.

Figures 6 and  7 illustrate useful information about the respective geometric probabilities of the RHN to undergo displaced decays. A naive comparison between the results derived for HL-LHC and FCC-hh shows that while the maximum decay probability of one/two N decaying within the ID is very similar $\simeq 95\%$, the probability for one N decaying in the ID and the other in the MS is somewhat larger at FCC-hh than HL-LHC. A large probability along with a higher c.m. energy and a higher achievable integrated luminosity will result in a significantly large number of events, that can be observed at the FCC-hh. In the subsequent sections, we present a detailed discussion of the observable signal events at HL-LHC and FCC-hh, while taking into account both the kinematic and geometric cut efficiencies.

Fig. 7

As Fig. 6, but showing the decay probabilities for the FCC-hh with $M_N=20$ GeV and $M_{Z^\prime }=8$ TeV

5.2 Signal description for N decaying in the ID

As we are focussing on a heavier $Z^{\prime }$ with mass $M_{Z^{\prime }}\ge 1$ TeV, and lighter N with masses $10-70$ GeV satisfying $M_N <M_W$, the N hence will be boosted, resulting in collimated decay products. As this work is based on a zero-background assumption, we do not consider $M_N<10$ GeV for which the displaced vertex mass will be $<10$ GeV resulting in relatively higher background [67]. In Fig. 8, we show the $\Delta R=\sqrt{(\Delta \eta )^2+(\Delta \phi )^2}$ separation between the muon and the closest quark at the parton level, originating from the decay of a RHN. The $\Delta R$ separation between two closest quarks also shows similar features. As can be seen, in most of the events the muon-quark separation $\Delta R (\mu ,q)\ll 0.4$, where $\Delta R=0.4$ has often been utilised in experimental analyses as the isolation criterion between final states. Instead of abiding by the selection criterion with jet radius ${\mathcal {R}}=0.4$ and standard isolation criterion $\Delta R(\ell , j), \Delta R(j,j) >0.4$, we hence demand a large jet radius ${\mathcal {R}}=1.0$, referred to as fat-jet [68].

As we have pointed out before, for the considered RHN state with mass $M_N<M_W$, the eV scale light neutrino mass demands a very suppressed active-sterile mixing $V_{lN} \sim 10^{-6}$, thereby leading to a macroscopic decay length for N. Therefore, the final state originating from $p p \rightarrow Z^{\prime } \rightarrow N N$ decay would be

• $p p \rightarrow Z^{\prime } \rightarrow N N \rightarrow \mu j j~~\mu j j \rightarrow J^{ dis}_{fat}~~J^{ dis}_{fat}$

In the above $J^{ dis}_{fat}$ represents the fat-jet originating from N decay, which is also displaced. We denote the leading and sub-leading $J^{ dis}_{fat}$ as $j_0$ and $j_1$, respectively. Given the small $\Delta R$ between muon and jets, as well as those between the two jets, along with constructing large ${\mathcal {R}}$ jets, we do not require any lepton isolation criteria. This leads to lepton being part of the considered fat-jet events. Extending further, we also show the distributions of $\Delta R$ after jet clustering. In left panel of Fig. 9, we show the $\Delta R$ separation of the closest $\mu$ from the leading jet, for both $\sqrt{s}=14$ TeV and $\sqrt{s}=100$ TeV. It is evident from the figure that the $\Delta R$ separation between $\mu$ and j is $\Delta R(\mu ,j) \ll 0.4$, which mimics the partonic distribution, shown in Fig. 8.

Fig. 8

$\Delta R$ separation between the $\mu$ and closest q at the partonic level

Since the fat-jet is formed by the decay products of N, which is considerably lighter than $Z^{\prime }$, satisfying $\frac{M_N}{M_{Z^{\prime }}}\ll 1$, the N and hence the $j_0$ and $j_1$ have a high transverse momentum. We show the $p_T$ distribution of $j_0$ and $j_1$ in the right panel of Fig. 9 for HL-LHC and for FCC-hh. As can be seen, for $M_{Z^{\prime }}=1$ TeV and $M_N=20$ GeV for HL-LHC c.m. energy, the distribution peaks around $p_T \sim M_{Z^{\prime }}/2=500$ GeV. For $M_{Z^{\prime }}=8$ TeV and FCC-hh c.m. energy, the peaks appear at a higher value, $p_T \sim M_{Z^{\prime }}/2=4$ TeV. This high transverse momentum of leading and sub-leading jets will be useful in designing the selection cuts.

We also note that the proposed signal can be extended to include ejj final state. This however depends strongly on the active-sterile mixing $V_{eN}$. With $|V_{eN}|\ll |V_{\mu N}|$, the contribution from ejj channel will be small. Here, we adopt a simplistic approach and consider only $|V_{\mu N}|\ne 0$, therefore we do not consider an ejj component in fat-jet description.

Fig. 9

Left: $\Delta R$ separation between the $\mu$ and leading jet. Right: transverse momentum distribution of the leading and sub-leading fat-jet

5.3 Signal description for N decaying in the muon spectrometer (MS)

If the RHN decays within the muon spectrometer, then one can not use the information on the number of tracks in the ID, the $p_T$ of a tracks in the ID, and other variables such as the deposited energy in the calorimeter. Instead of performing an analysis based on observables of a fat-jet, we rather present a track based analysis in the MS. The final state originating from $p p \rightarrow Z^{\prime } \rightarrow N N$ would be considered as

• $p p \rightarrow Z^{\prime } \rightarrow N N \rightarrow \mu q q^{\prime } ~~\mu q q^{\prime } \rightarrow \mu ^{ dis} \mu ^{ dis} + X^{ dis} \rightarrow track_1+track_2+...track_n+ Y$

where $\mu ^{ dis}$ represents a muon, produced from the RHN displaced decays. In the above, X corresponds to all other final state particles, including hadrons, electrons, photons etc. which are generated due to RHN decays as well as due to showering. We represent all the charged particles generated from the decay of RHN as $track_1, track_2,..track_n$ and Y represent any neutral particles in the final state that do not leave any footprint in the MS.

5.4 Analysis strategy

For these above mentioned model signatures, we first perform the analysis for HL-LHC with c.m. energy $\sqrt{s}=14$ TeV considering an integrated luminosity ${\mathcal {L}}=3\, \mathrm{ab}^{-1}$ and later for FCC-hh with $\sqrt{s}=100$ TeV with ${\mathcal {L}}=30\, \mathrm{ab}^{-1}$. To evaluate the number of events, we implement both the geometric and kinematic cuts, the details of which are provided in Sects. 6, and  7. The geometric cuts take into account the probability of RHN to decay within a specified region of HL-LHC/FCC-hh detector. For the geometric cut efficiency, we follow the prescription described in Sect. 5.1, and use Eqs. (5.2),  (5.3), and  (5.4). As kinematic cuts, we use following few variables – $p_T$ of jet, $p_T$ of associated tracks, $\eta$ of jet and tracks. Below, we briefly outline the adopted procedure, which we implement for the analysis:

We implement both kinematic and geometric cuts to evaluate the number of signal events that can be detected. Let g and k denote the geometric and kinematic cuts. We denote the corresponding probabilities for an event to pass g and k by $\epsilon _g$ and $\epsilon _k$, respectively, and $\epsilon _{g \& k}$ is the probability that an event pass both g and k. The probability that an event known to pass the kinematic cuts k, will also pass g is given as (using conditional probability),

$$ \begin{aligned} \epsilon _{g|k}=\epsilon _{g \& k}/\epsilon _{k}. \end{aligned}$$

(5.5)

To evaluate the above, we use the boost distribution of RHN, undergoing displaced decays. In Eqs. (5.2), (5.3), and  (5.4), we consider that $f(\sqrt{s}, M_N,M_{Z^\prime },b_1,b_2)$ is the boost distribution of RHNs from those events that satisfy the kinematic cut. We evaluate the geometric cut-efficiency using the above-mentioned equations and the kinematic cut-efficiency has been calculated in PYTHIA v8.235 [60].

$$\begin{aligned} \epsilon _{g|k}={{\mathcal {P}}}(L_1, L_2, \sqrt{s}, M_N,M_{Z^\prime },\theta ). \end{aligned}$$

(5.6)

Using Eq. (5.5), the final cut-efficiency including both the geometric and kinematic cuts becomes

$$ \begin{aligned} \epsilon _{g \& k} = \epsilon _{g|k} \times \epsilon _{k}={{\mathcal {P}}}(L_1, L_2, \sqrt{s}, M_N,M_{Z^\prime },\theta )\times \epsilon _{k},\nonumber \\ \end{aligned}$$

(5.7)

where the probability ${{\mathcal {P}}}$ in the above has been simulated from the event samples that satisfy the kinematic cuts. The cross-sections after cut for the two signal descriptions are evaluated as

$$\begin{aligned}&\sigma ( p p \rightarrow N N \rightarrow J^{ dis}_{ fat} J^{ dis}_{ fat}) \big |_{after-cut}\nonumber \\&\quad =\sigma _p \times {\mathcal {P}}(L_1, L_2, \sqrt{s}, M_N,M_{Z^\prime },\theta )\times \epsilon _{k}, \end{aligned}$$

(5.8)

and

$$\begin{aligned}&\sigma ( p p \rightarrow N N \rightarrow track_1+..track_n+Y) \big |_{after-cut}\nonumber \\&\quad =\sigma _p \times \mathcal {{P}}(L_1, L_2, \sqrt{s}, M_N,M_{Z^\prime },\theta )\times \epsilon _{k}, \end{aligned}$$

(5.9)

where ${\mathcal {P}}(L_1, L_2, \sqrt{s}, M_N,M_{Z^\prime },\theta )$ and $\epsilon _{k}$ represent the corresponding geometric and kinematic cut-efficiencies for RHN decay, respectively. ${\mathcal {P}}(L_1, L_2, \sqrt{s}, M_N,M_{Z^\prime },\theta )$ is given either Eqs. (5.2) and  (5.3) for both the RHNs decays or by Eq. (5.4) for one RHN decay. In the above, $\sigma _p$ is the partonic cross-section for $p p \rightarrow Z^{\prime } \rightarrow N N \rightarrow \mu j j \mu j j$. We have explicitly checked that the cut-efficiencies following this procedure match with the cut-efficiencies obtained with a full Pythiav8.235 based numerical simulation with a mismatch $< {\mathcal {O}}(5\%)$.

6 Projection for HL-LHC

Fig. 10

Left panel: $p_T$ of tracks associated with leading and sub-leading jets at HL-LHC. Right panel: number of tracks associated to $j_{0,1}$ at HL-LHC

To evaluate the sensitivity for observing displaced RHN signature, which decays in the ID/MS of the HL-LHC detector, we use the kinematic variables:- transverse momentum $p_T$ and pseudo-rapidity $|\eta |$ of jets, and $p_T$, $|\eta |$ of tracks. We first obtain the results demanding displaced decays of two Ns either in the ID or in MS, which give distinctive signatures with low background. This is referred to as 2IDvx or 2MSvx events, respectively. We also show the projection relaxing the tight requirement of exactly two displaced vertices, and analyse an inclusive signature with at least one N undergoing displaced decay referred to as 1IDvx, and 1MSvx events. Note that all the results are calculated under the zero background assumption. For the HL-LHC analysis we consider $M_{Z^\prime }=1$ TeV, $g^\prime =3\times 10^{-3}$.

Fig. 11

Number of 2IDvx (left) and 2MSvx (right) events as a function of the RHN mass $M_N$ and the active-sterile neutrino mixing strength $|V_{\mu N}|$, at the HL-LHC with ${\mathcal {L}} = 3~\text {ab}^{-1}$ for $M_{Z^\prime } = 1$ TeV, $g^\prime = 3\times 10^{-3}$ (solid blue lines and shaded regions). The value following the $\le$ symbol indicates the approximate maximal number of events in each case. The brown dashed line represents the FCC-ee projection [69, 70], and the red shaded region indicates current limits [71, 72]. The grey band illustrates the range of $|V_{\mu N}|$ required to generate light neutrino masses between 0.01 eV and 0.1 eV through the seesaw mechanism, $m_\nu = |V_{\mu N}|^2 M_N$

Fig. 12

As Fig. 11, but showing number of 1IDvx (left) and 1MSvx (right) events

6.1 Decay vertex in the inner-detector (IDvx)

As shown in the right-panel of Fig. 9, the majority of the signal events contain 2 jets with $p_T(j)\ge 500$ GeV. Thus signal events can be selected using a displaced-jet trigger [73] which requires an $H_T\ge 350$ GeV, where $H_T$ is defined as the scalar sum of $p_T$ of all jets satisfying $p_T > 40$ GeV and $| \eta | < 2.5$ in the event. We consider the following sets of cuts for the analysis of two displaced vertices in the ID,

• C1: Both RHNs decay within $ID_{L_1}=2$ mm and $ID_{L_2}=300$ mm.

• C2: Both leading and sub-leading fat-jet $j_{0,1}$ have to satisfy $|\eta (j)| <4.5$ and $p_T(j)\ge 150$ GeV. The strong cut on jet-$p_T$ is motivated by the ATLAS analysis of boosted RHNs [74] and other fat-jet analyses [75, 76].

• C3: Additionally for each of the leading and sub-leading fat-jet, we require $N_{trk} \ge 4$, with $p_T(trk)\ge 1 \ \text {GeV}$, $|\eta (trk)|<2.5$. To satisfy the $p_T$ threshold and $\eta$ acceptance of the ID we demand $p_T(trk)\ge 1 \ \text {GeV}$ and $|\eta (trk)|<2.5$, respectively. These criteria are inspired from the ATLAS search for long-lived particles decaying into displaced hadronic jets in the ID [50]. Fig. 2 of [50] indicates that for majority of background events, displaced vertices are associated with 2-3 tracks. Thus the cut $n_{trk} \ge 4$ is used to reduce the background. These cuts generally have very high acceptance for signal. We demonstrate this by plotting the distribution of track $p_T$ and track multiplicity of leading and subleading fat-jet in Fig. 10 (left and right panels respectively) for the benchmark values of $M_{Z^{\prime }}, M_N = (1 \,\mathrm{TeV, 20\,\mathrm{GeV}})$. It can be seen that the tracks generally have high $p_T$ with the multiplicity peaking at $\sim 7$. For higher $M_N$, the number of tracks increases further (as can be seen from Fig. 2 of ref. [50]).

• C4: Finally we require that the number of jets $n_j\ge 2$ and the above-mentioned cuts are satisfied for both the leading and sub-leading fat-jets.

As we outline in Sect. 5.1, we consider the boost distribution of the two Ns from the events that satisfy the kinematic cuts C2-C4 and evaluate the geometric cut-efficiency as ${\mathcal {P}}(L_1, L_2, \sqrt{s}, M_N,M_{Z^\prime },\theta )$ using Eq. (5.2). In Fig. 11 (left panel), we show the contours for $N_{events} = 3$ and 8 in the $|V_{\mu N}|$ and $M_N$ plane (blue solid lines). We also indicate the approximate maximal number of events (following the $\le$ symbol) for a specific scenario. In this case the maximum 10 2IDvx events can be observed with the luminosity ${\mathcal {L}}=3\, \mathrm{ab}^{-1}$. In the same plot, we also show the projection for FCC-ee in the channel $e^+e^-\rightarrow Z\rightarrow \nu N$ by the (brown dashed line), derived in [69, 70]. The region shaded in red is excluded from CMS and ATLAS searches in both prompt and displaced leptonic decay signatures of RHNs [70,71,72]. We also show the active-sterile neutrino mixing required for light neutrino mass $m_{\nu }$ in between 0.01 and 0.1 eV.

We also estimate the number of events with at least one RHN decaying in the ID and show the result in Fig. 12 (left panel). For this, we consider only the probability of one RHN decay within 2–300 mm using Eq. (5.4) and the kinematic cut-efficiencies as $\epsilon _k \sim 100\%$. We checked explicitly that more than $90\%$ events satisfy the above mentioned kinematic cuts. With the requirement of at least one N decaying in the ID, the probability is slightly larger, which leads to a maximum number of events ${ N_{events}=28}$ that can be obtained with $3\, \mathrm{ab}^{-1}$ of data. Note that demand of one IDvx gives rise to less background reduction compared to 2IDvx requirement.

6.2 Decay vertex in the muon spectrometer (MSvx)

We now turn our attention to even longer lived N decaying to $\mu q q^{\prime }$ final states in the MS. We consider the following sets of selection cuts:

• C1: We demand both the RHN decays to occur within the MS between the length $MS_{L_1}=4000$ mm, and $MS_{L_2}=7000$ mm. The chosen length interval corresponds to the outer edge of the HCal and the middle station of muon chambers where the muon RoI trigger efficiency is higher [51].

• C2: For each of the tracks originating from the RHN decay vertex we impose $p_T(trk)>1$ GeV, $|\eta (trk)| <2.7$. We demand displaced vertex with $N_{trk} \ge 4$ following the ATLAS analysis [51], which selects MSvx with atleast 5 tracklets⁴ for background rejection in MS.

• C3: Finally we select the events containing two decay vertices, where each vertex satisfies the requirements on tracks mentioned in C2. We also demand $\Sigma _{{trk}} P_T(trk)>60$ GeV, which along with the cut $N_{trk} \ge 4$ ensures a significant number of hits in the MS.

We evaluate the geometric cut-efficiencies following Eqs. (5.2) and  (5.3) using the event samples that satisfy selection cuts C2-C3. The kinematic and geometric cuts have been designed based on the ATLAS search [51]. In Fig. 11 (right panel), we show the number of events for the 2MSvx events. We also show the number of events demanding decay of at least one N in the MS in Fig. 12 (right panel), where we consider only the geometric cut-efficiency. The notable difference between the 2IDvx events and 2MSvx events is that the number of events in the later case reduces significantly. This can be understood by referring Fig. 6, where we show the probability of two N decaying in different regions of the HL-LHC detector. As can be seen that the probability of two N decaying in the MS is much smaller ${\mathcal {P}} \sim 1\%$, compared to the probability of two N decaying in the ID, where probability is ${\mathcal {P}} > 99\%$. This order of magnitude difference in the geometric cut-efficiency is primarily responsible⁵ for the reduction in number of 2MSvx signal events. In a background free scenario, the prospect of detection of 1MSvx events is higher at HL-LHC compared to 2MSvx events, as can be seen by comparing Figs. 11 and  12 (right panels). A maximum of $N_{events}=3$ events can be obtained with 3 $\mathrm{ab}^{-1}$ of data.

Finally, we also show the HL-LHC sensitivity when one N decays in the ID while the other one decays in the MS, which we refer to as MSID. The geometric probability for this is similar to the probability of 2MSvx events, as can be seen from Fig. 6. For this category, we apply no selection cuts, and show the resulting event contours in Fig. 16 (left panel). Due to the very small probability we obtain no sensitivity for such decays. As shown in Figs. 11 and  12 (right panels), the sensitivity is very poor for both 2MSvx and 1MSvx scenarios. For this reason we do not consider larger values for $M_{Z^\prime }$ at the HL-LHC. For instance, if $M_{Z^\prime } = 2$ TeV, the maximum allowed coupling is $g^\prime \simeq 2\times 10^{-2}$ (see Fig. 1). In this case, $\sigma (pp\rightarrow Z^\prime \rightarrow NN) = 3.6\times 10^{-2}$ fb, which is less than half of that for $M_{Z^\prime } = 1$ TeV and $g^\prime \simeq 3\times 10^{-3}$. A significant improvement in sensitivity is not expected for larger $M_{Z^\prime }$ at the HL-LHC for which we restrict our analysis to $M_{Z^\prime } = 1$ TeV.

7 Projection for FCC-hh

We now turn our attention to an even higher c.m. energy and consider the FCC-hh at $\sqrt{s} =$100 TeV. Due to the large c.m. energy, FCC-hh will enable production of very heavy $Z^{\prime }$, as discussed in Sect. 3. Correspondingly, we consider $M_{Z^{\prime }} = 8 \ \mathrm{TeV}$, which is beyond the scope of dilepton constraints even with the full FCC-hh luminosity. We consider the same decay channel and signal description as we consider for HL-LHC and present results for $M_{Z^\prime }=8 \ \mathrm{TeV}, \ g^\prime =0.1$. The necessary details about the geometry of the ID and MS that we consider for the analysis have been given in Table 1. We implement a similar set of kinematic cuts as used for the HL-LHC study in Sect. 6. Here we use harder $p_T$ cuts for fat-jets.

Fig. 13

Left: $p_T$ of tracks associated to leading and sub-leading jets, relevant for FCC-hh. Right: Number of tracks associated to $j_{0,1}$ for FCC-hh

Fig. 14

Number of 2IDvx (left) and 2MSvx (right) events as a function of the RHN mass $M_N$ and the active-sterile neutrino mixing strength $|V_{\mu N}|$, at the FCC-hh with ${\mathcal {L}} = 30~\text {ab}^{-1}$ for $M_{Z^\prime } = 8$ TeV, $g^\prime = 0.1$ (solid blue lines and shaded regions). The value following the $\le$ symbol indicates the approximate maximal number of events in each case. The brown dashed line represents the FCC-ee projection [69, 70], and the red shaded region indicates current limits [71, 72]. The grey band illustrates the range of $|V_{\mu N}|$ required to generate light neutrino masses between 0.01 eV and 0.1 eV through the seesaw mechanism, $m_\nu = |V_{\mu N}|^2 M_N$

Fig. 15

As Fig. 14, but showing number of event for 1IDvx (left) and 1MSvx (right)

7.1 Decay vertex in the inner-detector (IDvx)

We consider two RHN decaying in the inner detector of FCC-hh, for which we use the following sets of cuts:

• C1: Both the Ns decay between $ID_{L_1}=25$ mm, and $ID_{L_2}=1550$ mm.

• C2: The leading and sub-leading jet $j_{0,1}$ have to satisfy $|\eta (j)| <4.5$ and $p_T(j)\ge 300$ GeV. A higher cut on the jet $p_T$ will be useful to suppress the SM backgrounds.

• C3: Additionally for both $j_{0,1}$, the number of associated tracks with $p_T\ge 1 \ \text {GeV}$ and $|\eta |<2.5$ should satisfy $N_{trk} \ge 4$. We show the distribution of track-$p_T$ and number of tracks in Fig. 13 for $M_N=20$ GeV.

• C4: Finally we select the events if the number of jets $n_j\ge 2$ and the above-mentioned cuts are satisfied for both $j_{0,1}$ . Additionally, similar to the analysis of the HL-LHC, we also present the results demanding displaced decay of at least one RHN,⁶ for which we consider only the geometric cut efficiency.

In Fig. 14 (left panel), we show the contours for 3 and 800 2IDvx events. We show the event contours corresponding to 1IDvx events in Fig. 15 (left panel). For both of these scenarios, a huge number of events ${ N_{events}} > 1000$ can be observed at FCC-hh with ${\mathcal {L}}=30\, \mathrm{ab}^{-1}$.

7.2 Decay vertex in the muon spectrometer (MSvx)

If the RHN decays to a $\mu q q^{\prime }$ in the MS, we implement the following set of selection cuts:

• C1: We demand both the RHN decays to occur within the MS between $MS_{L_1}=6000$ mm, and $MS_{L_2}=9000$ mm.

• C2: For each of the tracks originating from a RHN decay vertex we impose $p_T(trk)>1$ GeV , $|\eta (trk)| <2.7$ and $N_{trk} \ge 4$. Additionally, $\Sigma _{{trk}} P_T(trk)>60$ GeV. These cuts are similar to the cuts that we use for HL-LHC projection.

• C3: Finally we select the events if two decay vertices and the associated tracks satisfy the above selection criteria.

Fig. 16

As Fig. 14, but showing the number of 1IDvx-1MSvx events at the HL-LHC with ${\mathcal {L}}=3~\text {ab}^{-1}$ for $M_{Z^\prime } = 1$ TeV, $g^\prime = 3\times 10^{-3}$ (left) and at the FCC-hh with ${\mathcal {L}} = 30~\text {ab}^{-1}$ for $M_{Z^\prime } = 8$ TeV, $g^\prime = 0.1$

In Fig. 14 (right panel), we show the event contours for two N decaying inside the MS. We also show the number of events demanding decay of at least one N in the MS in Fig. 15 (right panel), where we take into account only the geometric cut-efficiency. The detection prospects of displaced N decaying in the muon spectrometer is significantly larger for FCC-hh, compared to HL-LHC. This increase in ${ N_{events}}$ occurs due to an increase in the luminosity, and a marginal increase in the cross-section. The kinematic cut-efficiencies are more than $98\%$ for both HL-LHC and FCC-hh MSvx events, and the geometric cut-efficiencies are also somewhat similar (see Eqs. 5.2 and  5.3). Hence, these have little effects in determining increase in the number of events. Finally, in Fig. 16 we also present results for the MSID event category for HL-LHC (left panel) and for FCC-hh (right panel). A large number of events can be observed at FCC-hh, compared to HL-LHC, with a maximum number of events $N_{events}=120$ for FCC-hh.

7.3 Sensitivity reach for $|V_{\mu N}|$

Finally, in Fig. 17, we present the sensitivity reach for $|V_{\mu N}|$ at FCC-hh with different choices of $M_{Z^{\prime }}$ and $g^{\prime }$ for 2IDvx (left panel) and 2MSvx (right panel). The region inside the red solid contour is excluded by the recent ATLAS RHN search [77] using di-lepton displaced vertices. We consider few benchmark points with $M_{Z^{\prime }}=5, 8$ TeV and coupling $g^{\prime }=0.1$, as well as, a relatively higher mass $M_{Z^{\prime }}=10$ TeV with coupling $g^{\prime }=1$. Given the large expected RHN displacements, we assume a background free scenario. Under this assumption, following a Poisson distribution only $N_{event} = 3.09$ are required at 95$\%$ confidence level. We note that, fixing $M_{Z^{\prime }}$, $g^{\prime }$ and $M_N$, a smaller value of the active-sterile mixing $|V_{\mu N}|$ is required to obtain $N_{event} \sim 3.09$ 2MSvx events compared to 2IDvx events. This occurs due to the necessary larger decay length into a MS. In other words, the MS strategy provides a lower mass reach provided $M_{Z^{\prime }}$, $g^{\prime }$ and $|V_{\mu N}|$ is fixed. If we consider the case $M_{Z^{\prime }} = 10$ TeV and $g^{\prime }=1$, for $V_{\mu N} = 10^{-6}$, the 2IDVx exclusion limit is $M_N\simeq 37$ GeV and for 2MsVx it is $M_N\simeq 32$ GeV. For $M_N = 40$ GeV, the 2IDVx exclusion limit is $V_{\mu N} = 9\times 10^{-7}$ and for 2MsVx it is $V_{\mu N} = 9\times 10^{-7}$. Although the sensitivity reach in $|V_{\mu N}|$ for 2MSvx events has a significant overlap with the sensitivity reach for 2IDvx, the former has added benefit as 2MSvx events will serve as a background free distinctive signal of the model.

Fig. 17

$95\%$ CL sensitivity of the FCC-hh with ${\mathcal {L}}=30~\text {ab}^{-1}$ towards the active-sterile neutrino mixing strength $|V_{\mu N}|$, as a function of the RHN mass $M_N$ and for different choices of $M_{Z^{\prime }}$ and $g^{\prime }$ (solid contours as indicated). The left and right panels show the sensitivity using the 2IDvx and 2MSvx signature, respectively. The brown dashed line represents the FCC-ee projection [69, 70], and the red shaded region indicates current limits [71, 72], with the region inside the solid red contour excluded by the recent ATLAS search [77]. The grey band illustrates the range of $|V_{\mu N}|$ required to generate light neutrino masses between 0.01 eV and 0.1 eV through the seesaw mechanism, $m_\nu = |V_{\mu N}|^2 M_N$

8 Conclusion and outlook

The RHN N in the gauged $B-L$ model can be pair-produced at a pp machine via the heavy $Z^{\prime }$ mediated channel, and give distinctive signatures with displaced vertices. The pair-production of N is independent of the active-sterile neutrino mixing and instead depends on the $B-L$ gauge coupling $g^{\prime }$, the mass of $Z^{\prime }$, and the mass of N. Hence, even with an active-sterile mixing which is suppressed due to light neutrino mass constraint, fairly large cross-section $\sigma (pp\rightarrow Z^\prime \rightarrow NN) \sim$ fb and even higher can be achieved. We consider a relatively light N with $M_N<M_W$ for which the constraint from eV light neutrino masses result in a suppressed active-sterile mixing $|V_{\ell N}| \sim 10^{-6}$. For the considered range of $M_N$ and small $|V_{\ell N}|$, the RHN undergoes three body decays with the decay vertex considerably displaced from its production vertex. We find that, N undergoes a large displacement $\ell \sim$ mm up to m if $M_N$ varies in between 10-70 GeV. Additionally, due to $M_N \ll M_{Z^{\prime }}$, the produced N is significantly boosted, resulting in collimated decay products.

We analyse the displaced decay signatures of such a light N that can be probed at the high-luminosity run of the LHC (HL-LHC) with c.m. energy $\sqrt{s}=14$ TeV and the future pp machine FCC-hh that can operate with c.m. energy $\sqrt{s}=100$ TeV. Specifically, we consider two scenarios, which are N decays (a) within the inner-detector (ID) of the HL-LHC and FCC-hh detectors, and (b) within the first few layers of the muon spectrometer (MS). For the former, N should have proper decay length $c\tau \sim {\mathcal {O}}(1-100)$ mm, while for the latter, the decay length should be $c\tau \sim$ m. We emphasize that the signal description between (a) and (b) differ widely. For a detail analysis of the signature, we specifically consider $N \rightarrow \mu q q^{\prime }$ decay mode and if the decay of N occurs in the ID, the collimated decay products of N result in a displaced fat-jet. For this case, we thus analyse the model signature with one or two displaced fat-jets. We find that ${\mathcal {O}}(10)$ displaced fat-jet events can be observed at HL-LHC with 3 $\mathrm{ab}^{-1}$ of luminosity, and for FCC-hh, a significantly larger number of events ${\mathcal {O}}(1000)$ can be observed with ${\mathcal {L}}=30\, \mathrm{ab}^{-1}$.

If the RHN decays in the MS, the fat-jet description can not be used, since the energy deposit in the calorimeter, informations about the track in the ID etc are missing, which are being used for the formation of a jet. In this case, we instead perform an analysis that relies on the properties of tracks in the MS. We apply selection cuts on the number of tracks from the RHN decay ($N_{trk} \ge 4$), $p_T(trk)>1$ GeV, $|\eta (trk)| <2.7$, and summation of $p_T$ of the tracks associated with N ($\Sigma _{{trk}} P_T(trk)>60$ GeV). We find that for HL-LHC, even after using the full integrated luminosity, the sensitivity reach is significantly low. For FCC-hh, it improves by an order of magnitude, as a large number of events ${\mathcal {O}}(100)$ can be observed with the full integrated luminosity. We also extend the analysis for the case, when one of the N decays in the ID and the other N decays in the MS, for which results are similar to (b).

Our proposed signature can potentially be explored for other RHN decay modes, including $N \rightarrow e q q^{\prime }$ as well as $N \rightarrow \nu q q^{\prime }$ final states. The final state with an electron depends on active sterile mixing $|V_{eN}|$, and hence choosing a large $|V_{eN}|$ compared to $|V_{\mu N}|$ will make the contribution from $N \rightarrow e q q^{\prime }$ large. The $N \rightarrow \nu q q^{\prime }$ decay instead depends on all possible $|V_{\ell N}|$. The final state in addition to displaced fat-jet or charged tracks, will also carry missing transverse energy. By applying a veto on MET will hence reduce the contamination from $\nu q q^{\prime }$ decay mode. A detail analysis of the model signature including both $e q q^{\prime }$ and $\nu q q^{\prime }$ decays will be presented elsewhere.