Probing the decoupled seesaw scalar in rare Higgs boson decay

The Higgs boson can mix with a singlet scalar that dynamically generates the Majorana mass of the right-handed neutrino NR. We show that even a tiny mixing between the Higgs boson and a ‘decoupled’ singlet scalar allows for Higgs-mediated pair production of NR without significant mixings between the active neutrinos and NR, and thus testable at colliders via a characteristic signal of two same-sign same-flavor lepton pairs, plus missing energy. We demonstrate that this search channel is mostly background- free in pp-collision and can be a highly sensitive probe of the Higgs-singlet mixing at the current and future pp colliders. Such channel provides a clean signal to discover the singlet scalar and explore the origin of neutrino masses.


JHEP02(2020)101
of a Majorana heavy neutrino. N R characteristic and collider-testable rare Higgs decay channels are very important precision searches, especially as so far collider searches show no robust sign of BSM mediators.
Generally, a singlet scalar that dynamically generates the Majorana mass of the N R can mix with the Higgs boson, which allows the Higgs boson's decay mode of h → N N without requiring sizeable left-right neutrino mixings. Such a singlet extension of the SM is often motivated for the dynamical N R mass generation, and it potentially helps understanding several important phenomena: the baryon number asymmetry via a strong first order electroweak phase transition [11,12], the abundance of dark matter [13,14] and the issue of vacuum stability [15][16][17], etc. In this study we advocate that such new singlet scalar could be discovered at pp colliders by searching for the fully leptonic rare Higgs decay channel h → N N → 4l + / E T . For minimal parameter dependence we start with a small-mixing scenario of the BSM scalar sector in a Type I seesaw setup where the L-R mixing is typically small, Φ denotes the scalar field(s) that yield the 125 GeV Higgs boson, and S for a singlet scalar field that couples to the right-hand singlet neutrino N R . Here we leave out the specific form of the potentials and only assume 'small-mixing', which is a very subleading mixing term 1 2 λ|Φ| 2 S 2 , so that V (Φ) and V (S) minimize independently. Φ and S develop their own vacuum expectation values (vev): Φ = (v Φ + φ)/ √ 2 to give rise to electroweak symmetry breaking and generate a Higgs-like boson φ, and S = v S + s to yield a necessary Majorana mass m N R = 2y N v S . The small but non-zero mixing term also portals N R into φ decay, yielding our search signal. For simplicity here we neglected the |H| 2 S term which has a dimensional coupling, by assuming its coupling is subdominant to λv Φ .
The λ term develops a φ − s mass mixing term λv Φ v S · φs after potential minimization, where the scalar mass-square matrix can be parametrized as, (1.1) With 'small mixing' such that λv Φ v S m 2 φ , m 2 s , this matrix is dominated by the diagonal terms and the mass eigenstate {h 1,2 } is a rotation by a small angle α from the gauge eigenstates {φ,s}, Without loss of generality h 1 identifies with the SM Higgs boson and contains an s-component of the magnitude  [18,19] if S is completely decoupled (y N → 0). For small mixings we would take 'decoupling' Φ, S conditions: The latter simplifies to λv 2 S ≤ O(10 −2 )m 2 h 1 if we consider an h 2 comparable to or heavier than the Higgs boson.
A finite y N leads to the h 1 decay partial width for each N species, where m h 1 =125 GeV, m N = 2y N v S and the factor 1/2 is due to N being Majorana. Given the rather small SM Higgs boson's decay width Γ h ∼ 4 MeV and relatively light heavy neutrino mass m N m h 1 /2, the branching ratio of h 1 decay to N N is In both Majorana and Dirac N R cases, the subsequent N N decay can lead to a characteristic two same-sign, same-flavor lepton pair, plus missing transverse energy signal e ± e ± µ ∓ µ ∓ + / E T .

Signal at pp colliders
The Higgs-like h 1 can be copiously produced via gluon fusion during pp collisions with a cross-section σ h 1 = σ pp→h . N N decay then yields a characteristic and remarkably clean signal of two same-sign, same-flavor (SSSF) lepton pairs as illustrated in figure 1. The two lepton pairs should have different lepton flavors and opposite charges (i.e. e ± e ± µ ∓ µ ∓ and no opposite-sign, same-flavor lepton pair exists). This channel differs from the LNV N N → l ± l ± 4j case that it does not necessarily violate lepton number conservation (LNC) if N couples to more than one lepton flavor, as the final state ν can be either neutrinos or anti-neutrinos. Therefore, this final state can be present either N is Majorana or Dirac. With a Majorana N , the N N system has a higher decay branching to this final state as one N can also decay as its own antiparticle. In such case, for each species of N the branching factor of the signal final state from N N is obtained by summing up the LNV and LNC processes, where Σ 2 ≡ |V eN | 2 + |V µN | 2 + |V τ N | 2 , and i, j = {e, µ} for lepton flavors. BF l i is the W ( * ) → l i ν branching fraction in N decay. Note the N → ll ν requires charged weak gauge boson mediation, which makes BF l i depend on both m N and whether BSM mediators are present in N decay. The maximal BF sig. is reached if N only couples to one lepton flavor, and BF sig. ≤ (BF l ) 2 /2.
Similarly, the h 2 scalar can also be produced via its φ-component at a cross-section , where the latter is the SM Higgs-like boson production cross-section evaluated for a different kinematical mass m h 2 . Since both the h 1 → N N branching and the h 2 's production cross-section scale as sin 2 α, assuming a near-unity branching of h 2 → N N , their contributions are the same order in terms of h − s mixing.
We note that h 2 decay modes depend on the relative size between m h 1 , m h 2 and m N . When m h 2 > 2m h 1 , the h 2 → h 1 h 1 channel opens up, but it is suppressed at small λ. h 2 may also decay to the SM fermion/gauge boson pairs, with an α 2 suppression. Therefore, in case of a kinematically accessible m h 2 > 2m N and a small λ, h 2 → N N dominates h 2 decays and acquire a near-unity branching. In case m h 2 m h 1 and m h 2 < 2m N , however, the relative branching ratios become less transparent as h 2 decays via the φand s-components compete between the α 2 -dependent two-body channels and the |V lN | 2 , m N -dependent multi-body channels.
The total signal cross-section is then is the detector selection efficiency for h 1 , h 2 mediated event samples, which can be separately studied via Monte Carlo simulations. h 2 's event rate may quickly become subdominant with a heavy h 2 mass, and with m h 2 ∼ O(m h 1 ) it can still be the main contributor due to its near-unity decay branching into N N . Note this h 1,2 mediated signal rate bypasses the vertex suppression from ν L − N R mixing, and serves as a probe for low ν L − N R mixing scenarios, e.g. Type-I seesaw models.
For convenience in phenomenology study we consider the simplified case where only the mass of one heavy neutrino N 1 lies below 1 2 m h 1 and N 1 mixes with only one active neutrino -4 -

JHEP02(2020)101
flavor ν µ . In this case only the LNV diagram in figure 1 contributes, and BF sig. reaches its maximum BF sig. = 0.16%. Complication rises when both m N and |V lN | 2 are small the N lifetime can be quite long, which leads to displaced decay at colliders. Our study relies on reconstructing decay kinematics so we require N decays inside the detector. Note that this decay-length issue can be circumvented in BSM models: |V lN | 2 can be enhanced by inverse-seesaw [20] after introducing mixture into an extra singlet fermion; also if N R is charged under BSM interactions, e.g. in extra-U(1) and Left-Right symmetric models, N gains extra partial width that helps make its decay visible but it also suppresses BF l at the same time. The major kinematic difference from a similar large invariant-mass search for doublycharged Higgs [21,22] is that our four leptons are much softer in energy, only adding up to a fraction of the Higgs boson mass in the optimal mass range m N < 1 2 m h 1 . In addition, the same-sign lepton pair must rise from different parent particles in the N N decay, leading to different kinematic topologies. Moreover, our final state contain two neutrinos which act as missing energy.

Background and multivariate analysis
The e ± e ± µ ∓ µ ∓ + / E T signal has very few SM background events at the pp colliders due to the presence of two pairs of SSSF leptons. The irreducible background includes contribution from pp → 4τ, W W Z(Z → 2τ ) channels in which the flavor combinatorics among the four e/µ leptons after τ, W decays can produce the same final state. Because 4τ channel is mainly ZZ → 4τ after all selection cuts, it is related to the di-boson production, and is the leading background at lower invariant masses. The pp → 4W channel is much subdominant due to its higher number of weak interaction vertices. Another potential background may arise from events with jets, usually from decays of hadrons into leptons, misidentified as leptons. However, a reliable fake-lepton background is usually obtained by the "data-driven" approach, which relies on experimental data. A very relevant experiment study [22] which has a similar signal final state shows that the background from doubly faked leptons can be subdominant to di-boson production. Besides, our signal requires two muons while faked leptons are mostly electrons. Thus, it might indicate that the fakelepton contribution is controllable in our case. We include 4τ and W W Z channels as the main background in our pp collision event simulations, and leave the fake-lepton process for further study when data become available.
We quote the measured Higgs boson's single production cross-section via gluon fusion process σ pp→h = 54.7 pb at 14-TeV [24] and 740 pb at 100-TeV [25]. As h 2 is also mostly produced by gluon fusion, we adopt the N 3 LO scalar mass dependence [26] to scale the h 2 production rate in σ h 2 . For data simulations we use MadGraph5 aMC@NLO version 2.6.1 [27] for parton level event generation, Pythia6 [28] for the parton shower and hadronization, and Delphes 3.4.1 [29] for detector simulation.
At the HL-LHC (FCC-hh/SppC), we generate about 150 (165) million 4τ , 10 (15) million W W Z, and 0.1 million signal events for kinematic analyses. Simulated events are analyzed to reject the SM background and maximize the signal significance. First the -5 -
III. observables between the missing energy and lepton(s): III.1) ∆φ between the missing transverse momentum and lepton(s) ∆φ( / E T , µ 1 ), ∆φ( / E T , µ 2 ), ∆φ( / E T , e 1 ), ∆φ( / E T , e 2 ), ∆φ( / E T , µ 1 + µ 2 ), ∆φ( / E T , e 1 + e 2 ), ∆φ( / E T , µ 1 + e), ∆φ( / E T , µ 2 + e), ∆φ( / E T , µ 1 + µ 2 + e 1 + e 2 ); III.2) the transverse mass M T of the system formed by the missing momentum plus Here µ 1 , µ 2 , e 1 , e 2 denote muons or electrons with the first or second leading transverse momentum, while for (µ 1 + e) or (µ 1 , e) the muon and electron are two leptons from the same N decay. The variables can be ranked according to their importance in the TMVA analysis. ∆φ( / E T , e 1 + e 2 ), ∆φ(µ 1 , µ 2 ), M (µ 1 + µ 2 + e 1 + e 2 ). We note that since the kinematics vary with m N , the rank may also change for different heavy neutrino masses. Figure 5 in the appendix A illustrates the distributions of a few input observables for a m N =20 GeV signal sample (black, filled), and those for the SM background including 4τ (red) and W W Z (blue) processes after the pre-selection cuts. Many observables show clear distinctions between the signal and background. It is worth noting that the signal's transverse mass M T2 distribution develops a sharp end-point around m N , which means that it can be used to determine the mass of the heavy neutrino as well if this new physics scenario is discovered.
The resulting Boosted Decision Trees (BDT) distributions from the MVA demonstrate excellent separation between signal and background, as shown in figure 2, which can reduce the background to a negligible level. Since the kinematics vary with m N , here we show a representative result of h 1 decay with m N = 20 GeV. Table 1 lists the corresponding background event rate estimated after pre-selection and the MVA, where the BDT cut is optimized for the signal significance, given by where N s (N b ) is the number of signal (total background) events after all selection cuts with 3 ab −1 integrated luminosity at the HL-LHC or 20 ab −1 integrated luminosity at the FCC-hh/SppC. Note that table 1 does not include h 2 contribution; h 2 increases the signal but the BDT cut needs to be re-optimized according to its fraction in the signal.  Table 1. Event rates for the signal from h 1 → N N → e ± e ± µ ∓ µ ∓ 2ν with m N = 20 GeV and BF h1→e ± e ± µ ∓ µ ∓ 2ν = 10 −7 and for background processes of 4τ and W W Z. The numbers at the HL-LHC (FCC-hh/SppC) correspond to center-of-mass energy √ s = 14(100) TeV and 3 (20) ab −1 integrated luminosity.

Collider sensitivites
Since the final state kinematics vary with the heavy neutrino mass, We simulate the signal sample and perform the MVA for each M N case from 10 to 60 GeV, at increment of 10 GeV. The corresponding BDT distributions at the HL-LHC with √ s = 14 TeV are shown in figure 6, and distributions at the FCC-hh/SppC with √ s = 100 TeV are shown in figure 7, see appendix B.
As m N increases, the kinematics of the 4τ background becomes similar to that of the signal process N N , which renders its BDT distribution largely overlapping with that of the signal. At each m N , the BDT criterion is optimized for maximal signal significance from its BDT distribution. The selection efficiencies for both N N signal and background processes at the HL-LHC and FCC-hh/SppC for different heavy neutrino masses are presented in table 2. Figure 3 shows the 95% confidence level (C.L.) limits on the combined branching fraction of Higgs boson into our final state when h 2 is heavy and only h 1 is relevant to the N N signal. Here, the 95% C.L. limits correspond to 2-σ signal significance given by the eq. (3.1). The number of events after all selections, N s (N b ), is the product of the production cross-section, luminosity and total selection efficiency for the signal (total background). Note the signal production cross-section is proportional to the Higgs decay branching fraction, cf. eq. (2.2). The m N dependence of the limit is the result of two competing reasons: the four-lepton pre-selection efficiencies increase at larger m N , while the BDT cuts reject background more efficiently at lower m N . When m N = 20 GeV, the combined branching fraction can be probed to as tiny as 4.4 × 10 −7 , 6.7 × 10 −8 , 1.9 × 10 −9 at the 13, 14, 100 TeV pp collision runs, respectively. Since the kinematics are quite similar between 13-TeV and 14-TeV, the 13-TeV limits are derived by adopting the same detector selection efficiencies as those for 14-TeV. σ pp→h = 48.6 pb at 13-TeV [23] is used for signal production cross-section. For background processes, their production cross-sections at 13 TeV are calculated by MadGraph5. Now we interpret these limits in the content of our 'small mixing' scenario. By eq. (1.3) the mixing angle is most sensitive to the mass-square difference between φ and s, which  Figure 3. 95% C.L. limits on the combined branching fraction BF h1→e ± e ± µ ∓ µ ∓ 2ν = BF h1→N N · BF sig. at the 13-TeV LHC, 14-TeV HL-LHC and 100-TeV FCC-hh/SppC with 280 fb −1 , 3 ab −1 , 20 ab −1 luminosities, respectively. For these limits, h 2 is assumed to be heavy and only h 1 decay contributes to the N N signal.
can be approximated the mass-square difference between h 1 and h 2 , off by a correction ∆m 2 Generally, m s can be lighter than v S and it is possible to have h 2 at the same (or lower) mass scale as the SM Higgs, so that |∆m 2 s,φ | ∼ m 2 h even if v S is more massive, at the TeV scale or above. So we adopt a 150 GeV h 2 as a showcase scenario, with the consideration that an m h 2 below the 2m W threshold can avoid a dramatic increment in BF h 2 →W W,ZZ which suppresses the N N branching and complicate our physics picture. In this scenario, both h 2 and h 1 decays contribute significantly. Figure 4 illustrates the 95% C.L. contour limits on λ, v S from various pp collision runs. The width of each filled band is due to varying m N from 10 to 60 GeV, which affects the cut efficiencies. R is the fraction of h 1 contribution in the total N N signal events. h 1 contributes mostly to the signal at low v S , while h 2 can also contribute due to its relatively light mass, and becomes dominant towards v S ∼ TeV. The limits indicate that the 13-TeV LHC with 280 fb −1 data can access the weak-scale v S part of the decoupling region, while the future HL-LHC and FCC-hh/SppC can be sensitive to even lower Φ − S mixing region, where the singlet components in h 1 can be as low as 10 −4 to 10 −6 .
For heavier h 2 , at the TeV scale, h 2 's contribution to the signal diminishes and the mixing sin 2 α is suppressed by the TeV m s . However, since y N = m N /(2v S ), an m N < 1 2 m h1 also significantly suppresses the hN N coupling, leading to m −4 s suppression in the signal rate. Therefore, a large m s would require a sizable λ to yield a detectable signal rate, which could violate the decoupling condition as v S is generally expected to be comparable to or higher than m s . In such cases, the h − s mixing needs to be parametrized under specific models where Φ, S sectors are no longer independent. Note in SU(2) L doublet(s)+singlet scalar models, the joint scalar potential minimization removes λ dependence from the diagonal scalar mass-square terms at the tree-level, indicating eq. (1.5) to be a strong requirement if λ 1, which suppresses the singlet's loop contribution to the Higgs mass.

Conclusion
We show that the fully leptonic Higgs decay channel e ± e ± µ ∓ µ ∓ + / E T is almost SM background-free and is accessible in both the lepton number violating and conserving heavy neutrino N decays. This rare channel can be a high-precision probe for the mixing between the Higgs boson and a singlet scalar boson which dynamically gives the N mass in the |V lN |-suppressed simple seesaw scheme. We consider a generic 'decoupled' scenario where the Higgs and singlet scalar bosons only share a small mixing, and demonstrate that the signal channel is capable of testing such decoupled scenario if the singlet and the Higgs bosons are in the comparable mass range. Comprehensive signal and SM background analyses with machine-learning MVA yield stringent limits on the signal Higgs boson decay branching fraction at the current and future pp colliders. For a benchmark case with the singlet-dominated h 2 at 150 GeV, the 13-TeV LHC, future HL-LHC and FCC-hh/SppC are sensitive to the Higgs-singlet mixing to 10 −3 , 10 −4 and 10 −6 with 280 fb −1 , 3 ab −1 , and 20 ab −1 pp collision luminosities, respectively. In addition, this test does not require a sizable left-right neutrino mixing and it is complimentary to the existing |V lN | 2 -based searches.

A Distributions of representative input observables
In figure 5, we show the distributions of some observables for the signal (black, filled), and those for the SM background including 4τ (red) and W W Z (blue) processes after the pre-selection cuts. These plots illustrate a h 1 decay signal by gluon fusion at the HL-LHC, with m N =20 GeV.    In table 2, we show selection efficiencies for both signal N N and background processes at the HL-LHC and FCC-hh/SppC for different heavy neutrino masses. The total selection efficiency after both pre-selection and BDT criteria is the product of two efficiencies in the same column. The numbers of events for background after all selections can be calculated by multiplying the initial numbers in the table 1 by the total cut efficiency, while the number of events for signal can be calculated from eq. (2.2) by substituting the total cut efficiency for N N as A 10 GeV Table 2. Selection efficiencies for the signal N N and background processes of 4τ and W W Z at the HL-LHC (with center-of-mass energy √ s = 14 TeV) and FCC-hh/SppC (with √ s = 100 TeV) for m N = 10, 20, 30, 40, 50, 60 GeV, respectively. Here h 2 is assumed to be heavy and only h 1 decay contributes to the signal N N .