1 Introduction

The standard model (SM) fields are either \(SU(2)_L\) singlet or doublet although there is no restriction for the existence of a larger multiplet. In fact we can introduce larger \(SU(2)_L\) multiplet fields as exotic field contents which would work to explain the mystryes in the SM such as non-zero neutrino masses and dark matter, and give rich phenomenologies [1,2,3,4,5,6,7,8,9,10,11,12, 20,21,22,23,24,25]. For example, models with a septet scalar with hypercharge \(Y=2\) have been discussed in Refs. [5,6,7, 9, 11] in which \(\rho \)-parameter is preserved to be 1 at tree level and Higgs phenomenologies are addressed. On the other hand, models in Refs. [10] and [12] have also realized the neutrino masses with septet scalar and some other multiplets at tree level and combination of tree and one-loop levels respectively; more models with large \(SU(2)_L\) multiplets realizing neutrino mass are referred to, e.g., Refs. [13,14,15,16,17,18,19]. In such models, tiny neutrino mass can be partially explained by small vacuum expectation value (VEV) of large multiplet scalar fields, where smallness of these VEVs is also required by the \(\rho \)-parameter. Furthermore, large multiplets provide several multiply charged particles which can be produced at collider experiments, which gives interesting phenomenology.

In this paper, we extend the model in Ref. [12] by introducing a quintet scalar field with non-zero VEV. As a result, tree level neutrino mass can be forbidden and by making quartet scalar an inert scalar, neutrino mass is generated at one-loop level. We thus obtain tiny neutrino mass more naturally. Then the neutrino mass matrix is analyzed taking constants from lepton flavor violation (LFV) into account, and we also estimate muon anomalous magnetic moment and muon \(g-2\). Then the collider analysis is done for parameter sets satisfying all the constraints. In addition, collider phenomenology of exotic particles is different from the previous model since quintet scalar is inert in this model while it develops a VEV in previous one. We thus discuss exotic particle production at the large hadron collider (LHC) and show signature of our model taking various decay chains into account. We also project the discovery significance for channels involving 4 b-jets and 1/2 leptons as a function of the luminosity.

This paper is organized as follows. In Sect. 2, we introduce our model, derive some formula for active neutrino mass matrix, and show the typical order of Yukawa couplings and related masses. In Sect. 3, we discuss neutrino mass matrix carrying out numerical analysis and implications to physics at the LHC focusing on the pair production of doubly charged fermion in the multiplet. We discuss and conclude in Sect. 4.

Table 1 Contents of fermion and scalar fields and their charge assignments under \(SU(2)_L\times U(1)_Y\)

2 Model setup

In this section, we review our model, where we add quintet scalar field with zero hypercharge, that is symbolized by \(\Phi _5\), to field contents of previous model Ref. [12]. In the model, several large \(SU(2)_L\) multiplet fields are introduced such as septet scalar \(\Phi _7\), quadruplet scalar \(\Phi _4\) and quintet fermion \(\Sigma _R\) whose hypercharges are 1, 1/2 and 0,respectively. We summarize the new field contents and their charges with the leptons and the SM Higgs field in Table 1. Here we summarize the roles of introduced \(SU(2)_L\) multiplet fields in our scenario of neutrino mass generation. The quintet fermion \(\Sigma _R\) can have Majorana mass term which violates lepton number. The quadruplet scalar \(\Phi _4\) connects \(\Sigma _R\) and the SM lepton doublet L by \(\Phi _4 \bar{L} \Sigma _R\) term. The quintet scalar \(\Phi _5\) is introduced to realize a vacuum with \(\langle \Phi _4 \rangle =0\) making \(\Phi _4\) inert scalar to forbid neutrino mass at tree level. Then septet \(\Phi _7\) is required to generate neutrino mass matrix at one-loop level; the VEV of \(\Phi _7\) is necessary to induce mass generation between real and imaginary components of neutral component in \(\Phi _4\) to realize nonzero one-loop contribution. Therefore, In our extended model, thanks to the existence of \(\Phi _5\), we find quadruplet scalar \(\Phi _4\) can be inert and hence the neutrino mass matrix is induced not at tree level but one-loop level. Note also that the quintet fermion \(\Sigma _R\) is the same one as discussed in Ref. [1] which can be a dark matter candidate without imposing any additional symmetry. However, in our case, the lightest component of \(\Sigma _R\) would decay because of an interaction associated with \(\Phi _4\) and leptons, and thus we do not discuss dark matter in this paper. Here we write these multiplets in terms of their components such as,

$$\begin{aligned} \Phi _7&= \left( \phi ^{4+}, \phi ^{3+}, \phi ^{++}_2, \phi ^{+}_2, \phi ^{0}, \phi ^{-}_1, \phi ^{--}_1 \right) ^T, \nonumber \\ \Phi _5&= \left( \xi ^{++}_2, \xi ^{+}_2, \xi ^{0}, \xi ^{-}_1 , \xi ^{-}_2 \right) ^T, \nonumber \\ \Phi _4&= \left( \varphi ^{++}, \varphi ^{+}_2, \varphi ^{0}, \varphi ^{-}_1 \right) ^T, \nonumber \\ \Sigma _R&= \left[ \Sigma _R^{++}, \Sigma ^{+}_R, \Sigma _R^{0}, \Sigma ^{+c}_L, \Sigma _L^{++c} \right] ^T, \end{aligned}$$
(2.1)

where subscripts for scalar components distinguish different particle with same electric charge.

First of all, we will investigate a hierarchy among vacuum expectation values (VEVs) of the bosons. Here, we assume that only neutral components of H, \(\Phi _5\), and \(\Phi _7\) have nonzero VEVs, which are respectively symbolized by \(v/\sqrt{2}\) and \(v_5/\sqrt{2}\) and \(v_7/\sqrt{2}\). Then, the VEVs are constrained by the \(\rho \) parameter at tree-level, which is given by [26]:

$$\begin{aligned} \rho =\frac{v^2+12 v_5^2 + 22 v_7^2}{v^2 + 4 v_7^2}, \end{aligned}$$
(2.2)

where the experimental value is \(\rho =1.00039\pm 0.00019\) at \(1\sigma \) confidential level. Also we have to satisfy the condition \(v_\mathrm{SM}=\sqrt{v^2+12 v_5^2 + 22 v_7^2} \simeq 246\) GeV which is the VEV of the SM-like Higgs. Requiring these two conditions fixed by \(\rho =1.00058\), we find a upper bound of \(v_5\) and \(v_7\) to be \(\mathcal {O}(1)\) GeV and \(v \simeq \) 246 GeV. It suggests that small VEVs of \(\Phi _5\) and \(\Phi _7\) are required.

Next task is how to realize inert feature of \(\Phi _4\). The scalar potential in the model is given by

$$\begin{aligned} \mathcal {V}&= -\mu _H^2 H^\dagger H + M_4^2 \Phi _4^\dagger \Phi _4 + M_5^2 \Phi _5^\dagger \Phi _5\nonumber \\&\quad + M_7^2 \Phi _7^\dagger \Phi _7 + \mathcal {V}_\mathrm{non-trivial} +\mathcal {V}_4, \end{aligned}$$
(2.3)

where \(\mathcal {V}_4\) is the trivial quartic term. The nontrivial Higgs potential terms under these symmetries are given by

$$\begin{aligned} \mathcal {V}_\mathrm{non-trivial}&= \mu [\Phi _4 \Phi _7^* \Phi _4] + \lambda _0 [H^\dag \Phi _4^* H H]\nonumber \\&\quad + \lambda _1 [H^* H^* \Phi _4 \Phi _4] + \mu _B[H^* \Phi _5^* \Phi _4] \nonumber \\&\quad - \mu _{5} [\Phi _5 \Phi _5 \Phi _5] + \lambda _X [H H \Phi _7^* \Phi _5] +\mathrm{c.c.}, \end{aligned}$$
(2.4)

where the first term contributes to the neutrino mass matrix at one-loop level and an inside bracket is “\([ \ ]\)” the \(SU(2)_L\) indices, which are implicitly contracted so that it makes singlet. Then, the inert condition is given by the tadpole condition for \(\Phi _4\), \(\partial \mathcal {V}/\partial \Phi _4=0\);

$$\begin{aligned} \lambda _0=-\sqrt{3}\frac{\mu _B v_5}{v^2}. \end{aligned}$$
(2.5)
Fig. 1
figure 1

The one loop diagrams generating dimension 5 term of scalar potential

The other VEVs are also obtained by solving conditions \(\partial \mathcal {V}/\partial v_5 = \partial \mathcal {V}/\partial v_7 = 0\). To obtain small VEV of \(\Phi _5\), we include dimension 5 term induced by one loop diagrams in Fig. 1

$$\begin{aligned} \Delta V = \frac{1}{\Lambda } (\Phi _5^* \tilde{H} \tilde{H} H H ) \end{aligned}$$
(2.6)

where \(\tilde{H} = H^* i \sigma _2\) with second Pauli matrix \(\sigma _2\), SU(2) indices are implicitly contracted inside bracket to make the term invariant, and \(1/\Lambda \) factor is obtained from the diagrams as a function of couplings \(\lambda _0\), \(\lambda _1\), \(\mu _B\) and mass of \(\Phi _4\). Here we take \(\Lambda \) as a parameter for simplicity. Then the vacuum conditions are given by

$$\begin{aligned}&M_5^2 v_5 - \frac{\sqrt{3}}{4} \mu _5 v_5^2 - \frac{v^4}{\sqrt{6} \Lambda } \simeq 0, \end{aligned}$$
(2.7)
$$\begin{aligned}&M_7^2 v_7 - \frac{\lambda _X}{2 \sqrt{10}} v^2 v_5 \simeq 0, \end{aligned}$$
(2.8)

where we ignored contributions from quartic terms of the potential, assuming small \(v_{5,7}\) and couplings associated with \((\Phi _{5,7}^\dagger \Phi _{5,7})(H^\dagger H)\) term. In addition, we assume \(\mu _5\) to be negligibly small for simplicity. Thus, from these conditions, VEVs are estimated as

$$\begin{aligned} v_5 \simeq \frac{v^4}{\sqrt{6} \Lambda M_5^2}, \quad v_7 \simeq \frac{\lambda _X}{2 \sqrt{10}} \frac{v_5 v^2}{M_7^2}. \end{aligned}$$
(2.9)

We can obtain \(v_5 \sim 1\) GeV by choosing \(\Lambda \); for example \(\Lambda \sim 5\) TeV for \(M_5 \sim 500 \) GeV which can be obtained by tuning couplings \(\mu _B\) and \(\lambda _1\). Note that if we do not include \(\Delta V\) we need large \(\mu _5\) as \(\mathcal {O}(100)\) TeV to get \(v_5 \sim 1\) GeV for \(M_5 \sim 500\) GeV, and such large trilinear coupling would violate perturbative unitarity. Then choosing \(v_5 \simeq 1.44\) GeV as a reference value, typical size of septet VEV is given by \(v_7 \simeq 0.007\) GeV where we take \(\lambda _X = 0.5\) and \(M_7 = 1\) TeV. We then find that septet VEV tends to be small and the smallness can suppress neutrino mass in addition to loop factor as we see below.

2.1 Yukawa sector

The renormalizable Lagrangian is given by

$$\begin{aligned} -\mathcal {L}_{Y}&= \sum _{\ell =e,\mu ,\tau }y_{\ell } \bar{L}_{L_\ell } H e_{R_\ell } +(y_{\nu })_{ij} [\bar{L}_{L_i} \tilde{\Phi }_4 \Sigma _{R_j} ]\nonumber \\&\quad + (M_{R})_i [\bar{\Sigma }^c_{R_i} \Sigma _{R_i}] - y_{\varphi _i} [\Phi _5 \bar{\Sigma }^c_{R_i} \Sigma _{R_i}] + \mathrm{h.c.}, \end{aligned}$$
(2.10)

where \((i,j)=1-3\), and \(y_\ell \) contributes to the charged-lepton masses in the SM and \(M_R\), \(y_\ell \), and \(y_{\varphi }\) are assumed to be diagonal basis.

2.2 Exotic fermion masses

Here we consider masses of the extra particles in the model. The masses for components in \(\Sigma _R\) are obtained from the last two terms of Eq. (2.42.10) after \(\Phi _5\) developing a VEV. Then we obtain the mass terms

$$\begin{aligned} L_{M_\Sigma }= & {} \left( M_R + \frac{2v_5}{\sqrt{6} }y_\varphi \right) \bar{\Sigma }^{++} \Sigma ^{++} + \left( M_R - \frac{v_5}{\sqrt{6} }y_\varphi \right) \bar{\Sigma }^{+} \Sigma ^{+}\nonumber \\&+ \left( M_R - \frac{v_5}{\sqrt{6} }y_\varphi \right) \bar{\Sigma }^{0c}_R \Sigma ^{0}_R, \end{aligned}$$
(2.11)

where the mass of neutral component is Majorana type. Since quintuplet VEV \(v_5\) cannot be large, the mass differences among the components are at most several GeV.

The masses of the components of scalar multiplet \(\Phi _{4,5,7}\) can obtain separate values from the contribution of non-trivial terms in the potential. Here we consider the mass differences are at most \(\mathcal {O}(100)\) GeV scale and masses for \(\Phi _{4,5,7}\) are dominantly given by \(M_{4,5,7}\).

2.3 Neutrino mass matrix

Let us first decompose the relevant Lagrangian in order to derive the neutrino mass matrix. The neutrino mass matrix is given in terms of \(y_\nu \), and its explicit form is found as

$$\begin{aligned} -\mathcal{L}&\supset \frac{ {(y_\nu )_{ij}}}{\sqrt{2}} \bar{\nu }_{L_i} \Sigma _{R_j}^0 (\varphi _R -i \varphi _I) + M_{R_i} \bar{\Sigma }^{0c}_{R_i}\Sigma ^0_{R_i} +\mathrm{h.c.}, \end{aligned}$$
(2.12)

where the mass difference between the real \(\Phi _4\) component; \(\varphi _R\) and the imaginary one; \(\varphi _I\) is generated by the term \(\mu \) through VEV of \(\Phi _7\). Then the formula of active neutrino mass matrix \(m_\nu \) as shown in Fig. 2 is given by

$$\begin{aligned} (m_{\nu })_{ij}&=\frac{{\mu v_7}}{(4\pi )^2} \sum _{a=1}^3\frac{(y_\nu )_{ia}(y_\nu ^T)_{aj}}{M_{R_a}} F_I(r_{R_a},r_{I_a}), \nonumber \\ F_I(r_1,r_2)&=\frac{r_1 \ln [r_1]-r_2\ln [r_2]+r_1r_2\ln [r_2/r_1]}{(1-r_1)(1-r_2)}, \end{aligned}$$
(2.13)

where we define \(r_{R(I)_a}\equiv (m_{\varphi _{R(I)}}/M_{R_a})^2\) with \(a=1-3\). The mass matrix \(({m}_\nu )_{ij}\) can be generally diagonalized by the Pontecorvo–Maki–Nakagawa-Sakata mixing matrix \(V_\mathrm{MNS}\) (PMNS) [27] as

$$\begin{aligned} ({m}_\nu )_{ab}&=(V_\mathrm{MNS} D_\nu V_\mathrm{MNS}^T)_{ab},\quad D_\nu \equiv (m_{\nu _1},m_{\nu _2},m_{\nu _3}), \end{aligned}$$
(2.14)
$$\begin{aligned} V_\mathrm{MNS}&= \left[ \begin{array}{ccc} {c_{13}}c_{12} &{}c_{13}s_{12} &{} s_{13} e^{-i\delta }\\ -c_{23}s_{12}-s_{23}s_{13}c_{12}e^{i\delta } &{} c_{23}c_{12}-s_{23}s_{13}s_{12}e^{i\delta } &{} s_{23}c_{13}\\ s_{23}s_{12}-c_{23}s_{13}c_{12}e^{i\delta } &{} -s_{23}c_{12}-c_{23}s_{13}s_{12}e^{i\delta } &{} c_{23}c_{13}\\ \end{array} \right] , \end{aligned}$$
(2.15)

where we neglect the Majorana phase as well as Dirac phase \(\delta \) in the numerical analysis for simplicity. The following neutrino oscillation data at 95% confidential level [26] is given as

$$\begin{aligned}&0.2911 \le s_{12}^2 \le 0.3161, \; 0.5262 \le s_{23}^2 \le 0.5485,\nonumber \\&0.0223 \le s_{13}^2 \le 0.0246, \nonumber \\&\ |m_{\nu _3}^2- m_{\nu _2}^2| =(2.44\pm 0.06) \times 10^{-3} \ \mathrm{eV}^2, \nonumber \\&m_{\nu _2}^2- m_{\nu _1}^2 =(7.53\pm 0.18) \times 10^{-5} \ \mathrm{eV}^2. \end{aligned}$$
(2.16)

The observed PMNS matrix can be realized by introducing the following parametrization. Here we parametrize the Yukawa coupling by \(y_\nu \), so called Casas-Ibarra parametrization [28], as follows

$$\begin{aligned} y_\nu&= V_\mathrm{MNS} \sqrt{D_\nu } OR^{-1/2}, \end{aligned}$$
(2.17)
$$\begin{aligned} R_{aa}&\equiv \frac{{\mu v_7}}{(4\pi )^2} \sum _{a=1}^3\frac{1}{M_{R_a}} F_I(r_{R_a},r_{I_a}), \end{aligned}$$
(2.18)

where O is an arbitrary complex orthogonal matrix with three degrees of freedom.

Fig. 2
figure 2

Contribution to neutrino masses at one-loop level

2.4 LFVs and muon \(g-2\)

LFVs at one-loop level arise from \(y_\nu \), and the related Lagrangian is given by

$$\begin{aligned}&-\mathcal{L} \supset (y_\nu )_{ij}\bar{\ell }_{L_i} \left( \frac{1}{\sqrt{2}}\Sigma _{R_j}^0 \varphi _1^- + \frac{1}{2} \Sigma _{1R_j}^+ \varphi ^{--}\right. \nonumber \\&\quad \left. + \frac{\sqrt{3}}{2}\Sigma _{2R_j}^- \varphi ^{0*} + \Sigma _{2R_j}^{--} \varphi _{2}^{+} \right) +\mathrm{h.c.}. \end{aligned}$$
(2.19)

Then the branching ratio is found as

$$\begin{aligned}&\mathrm{BR}(\ell _i\rightarrow \ell _j)\approx \frac{75}{64\pi }\frac{\alpha _{em} C_{ij}}{G_F^2}|y_{\nu _{ja}} y^\dag _{\nu _{ai}}G(\varphi ,\Sigma _a)|^2\nonumber \\&\quad \times \left( 1+\frac{m^2_{\ell _j}}{m^2_{\ell _i}}\right) ^2, \end{aligned}$$
(2.20)
$$\begin{aligned}&G(m_1,m_2)=\int _0^1dx\nonumber \\&\quad \times \int _0^{1-x-y}dy\frac{xy}{(x^2-x)m^2_{\ell _i}+x m_1^2+(1-x) m_2^2}, \end{aligned}$$
(2.21)

where \(C_{21}=1\), \(C_{31}=0.1784\), \(C_{32}=0.1736\), \(\alpha _{em}(m_Z)=1/128.9\), and \(G_F=1.166\times 10^{-5}\) GeV\(^{-2}\). The experimental upper bounds are given by [29,30,31]

$$\begin{aligned}&\mathrm{BR}(\mu \rightarrow e\gamma )\lesssim 4.2\times 10^{-13},\quad \mathrm{BR}(\tau \rightarrow e\gamma )\lesssim 3.3\times 10^{-8},\nonumber \\&\mathrm{BR}(\tau \rightarrow \mu \gamma )\lesssim 4.4\times 10^{-8}, \end{aligned}$$
(2.22)

which will be imposed in our numerical calculation.

Muon \(g-2\) is also induced from the same term and is given by

$$\begin{aligned} \Delta a_\mu = \frac{5m^2_\mu }{(4\pi )^2} \sum _a y_{\nu _{2a}} y^\dag _{\nu _{a2}} G(\varphi ,\Sigma _a) , \end{aligned}$$
(2.23)

while the experimental result implies \(\Delta a_\mu =(26.1\pm 8.0)\times 10^{-10}\) [32].

2.5 Beta functions of g and \(g_Y\)

Here we discuss running of gauge couplings and estimate the effective energy scale by evaluating the Landau poles for g and \(g_Y\) in the presence of new fields with nonzero multiple hypercharges. Each of the new beta function of g and \(g_Y\) for one \(SU(2)_L\) quintet fermion (\(\Sigma _R\)), quartet boson (\(\Phi _4\)), quintet boson \(\Phi _5\), and septet boson(\(\Phi _7\)) with (0, 1/2, 1) hypercharge is given by

$$\begin{aligned}&\Delta b^{\Sigma _R}_g=\frac{20}{3}, \quad \Delta b^{\Phi _4}_g=\frac{5}{3} \ , \quad \Delta b^{\Phi _5}_g=\frac{10}{3} \ ,\quad \Delta b^{\Phi _7}_g=\frac{28}{3} , \end{aligned}$$
(2.24)
$$\begin{aligned}&\Delta b^{\Sigma _R}_Y=0, \quad \Delta b^{\Phi _4}_Y=\frac{3}{5} \, \quad \Delta b^{\Phi _5}_Y=0\ ,\quad \Delta b^{\Phi _7}_Y=\frac{7}{5}. \end{aligned}$$
(2.25)

Then one finds the energy evolution of the gauge coupling g and \(g_Y\) as [33]

$$\begin{aligned} \frac{1}{g^2(\mu )}&=\frac{1}{g^2(m_{in.})}-\frac{b^{SM}_g}{(4\pi )^2}\ln \left[ \frac{\mu ^2}{m_{in.}^2}\right] \nonumber \\&\quad -\theta (\mu -m_{th.}) \frac{\Delta b^{\Sigma _R}_g}{(4\pi )^2}\ln \left[ \frac{\mu ^2}{m_{th.}^2}\right] \nonumber \\&\quad -\theta (\mu -m_{th.}) \frac{\Delta b^{\Phi _4}_g+\Delta b^{\Phi _5}_g+\Delta b^{\Phi _7}_g}{(4\pi )^2}\ln \left[ \frac{\mu ^2}{m_{th.}^2}\right] , \end{aligned}$$
(2.26)
$$\begin{aligned} \frac{1}{g^2_Y(\mu )}&=\frac{1}{g_Y^2(m_{in.})}-\frac{b^{SM}_Y}{(4\pi )^2}\ln \left[ \frac{\mu ^2}{m_{in.}^2}\right] \nonumber \\&\quad -\theta (\mu -m_{th.}) \frac{\Delta b^{\Phi _4}_Y+\Delta b^{\Phi _7}_Y}{(4\pi )^2}\ln \left[ \frac{\mu ^2}{m_{th.}^2}\right] , \end{aligned}$$
(2.27)

where \(\mu \) is a reference energy, \(b^{SM}_Y=41/6\), \(b^{SM}_g=-19/6\), and we assume to be \(m_{in.}(=m_Z)<m_{th.}\), being respectively input electroweak mass scale \(m_{in}\) and threshold masses of exotic fermions and bosons \(m_{th.}\). The resulting flow of \(g_Y(\mu )\) is then given by the Fig. 3 for g for each of \(m_{th.}=(0.5,1,10)\) TeV, where \(g_Y\) is valid up to Planck scale. This figure shows that g is relevant up to the mass scale \(\mu \approx (2,4)\times 10^2\) TeV for \(m_{th.}=(0.5,1)\) TeV and \(\mu \approx 5\times 10^3\) TeV for \(m_{th.}=10\) TeV. Thus our theory does not spoil, as far as we work on around or below the scale of TeV.

Fig. 3
figure 3

The running of g in terms of \(\mu \), depending on \(m_{th.}=(0.5,1, 10)\) TeV

3 Numerical analysis and implications to physics at the LHC

In this section, we perform numerical analysis of neutrino mass matrix taking into account LFV constraints. Then implications to collider physics are discussed adopting benchmark point accommodating with neutrino data and the constraints.

3.1 Numerical analysis for neutrino sector

Here we carry out numerical analysis scanning free parameters and check if we can fit the neutrino data. The free parameters are chosen within the range of

$$\begin{aligned}&M_{R_1} \in [500, 2000] \ \mathrm{GeV}, \ M_{R_{2,3}} \in [M_{R_1}, 5000] \ \mathrm{GeV}, \nonumber \\&m_{\varphi _R} \in [100, 2000] \, \mathrm{GeV}, \nonumber \\&m_{\varphi _I} \in [m_{\varphi _R} - 1, m_{\varphi _R} + 1] \, \mathrm{GeV}, \ \mu \in [30, 40] \, \mathrm{GeV} , \end{aligned}$$
(3.1)

where we take \(v_7 = 0.00748\) GeV as a reference value and we calculate \(y_\nu \) as output using Eq. (2.17) scanning parameters in orthogonal matrix O as \(\mathcal {O}(0.1)\)\(\mathcal {O}(1)\) randomly. We find that the neutrino data can be fitted with \(\mathcal {O}(0.1)\) to \(\mathcal {O}(1)\) Yukawa couplings \((y_\nu )_{ij}\) satisfying constrains from LFV processes. In addition, muon \(g-2\) is found to be maximally \(\sim 4 \times 10^{-11}\) due to constraint from \(\mu \rightarrow e \gamma \). We will choose some benchmark points satisfying all the constraints and provide maximal muon \(g-2\) to consider collider physics in the following subsection.

3.2 Implications to LHC physics

In this subsection, we discuss collider physics regarding the charged particles from large fermion/scalar multiplets in the model. In particular, we focus on particles from quintuplet fermion \(\Sigma _R\) and \(\Phi _4\) which propagate inside the loop diagram in neutrino mass generation where we assume components from \(\Phi _7\) and \(\Phi _5\) are heavier than those particles. The relevant gauge interactions associated with \(\Sigma _R\) can be written by

$$\begin{aligned}&\bar{\Sigma }_R \gamma ^\mu i D_\mu \Sigma _R \supset \bar{\Sigma }^{++} \gamma ^\mu \left( 2 e A_\mu + 2 g c_W Z_\mu \right) \Sigma ^{++}\nonumber \\&\quad + \bar{\Sigma }^{+} \gamma ^\mu \left( e A_\mu + g c_W Z_\mu \right) \Sigma ^{+} \nonumber \\&\quad - \sqrt{2} g \bar{\Sigma }^{++} \gamma ^\mu W_\mu ^+ \Sigma ^{+} - \sqrt{3} g \bar{\Sigma }^{+} \gamma ^\mu W_\mu ^+ \Sigma _R^{0}\nonumber \\&\quad - \frac{\sqrt{5} g}{\sqrt{2}} \bar{\Sigma }^{+} \gamma ^\mu W_\mu ^+ \Sigma ^{0c}_R - \sqrt{2} g \bar{\Sigma }^{+} \gamma ^\mu W_\mu ^- \Sigma ^{++} \nonumber \\&\quad - \sqrt{3} g \bar{\Sigma }^{0}_R \gamma ^\mu W_\mu ^- \Sigma ^{+} - \frac{\sqrt{5} g}{\sqrt{2}} \bar{\Sigma }^{0 c}_R \gamma ^\mu W_\mu ^- \Sigma ^{+}, \end{aligned}$$
(3.2)

where \(s_W(c_W) = \sin \theta _W (\cos \theta _W)\) with the Weinberg angle \(\theta _W\). Also the relevant gauge interactions associated with \(\Phi _4\) can be obtained from following kinetic term

$$\begin{aligned} |D_\mu \Phi _4|^2&= \sum _{m= -\frac{3}{2},-\frac{1}{2},\frac{1}{2},\frac{3}{2}} \biggl | \left[ \partial _\mu - i \left( \frac{1}{2}+m \right) e A_\mu \right. \nonumber \\&\quad \left. - i \frac{g}{c_W} \left( m - \left( \frac{1}{2}+m \right) s_W^2 \right) Z_\mu \right] (\Phi _4)_{m} \nonumber \\&\quad + \frac{ig}{\sqrt{2}} \sqrt{ \left( \frac{3}{2} + m \right) \left( \frac{5}{2} -m \right) } W^+_\mu (\Phi _4)_{m-1} \nonumber \\&\quad + \frac{ig}{\sqrt{2}} \sqrt{ \left( \frac{3}{2} - m \right) \left( \frac{5}{2} +m \right) } W^-_\mu (\Phi _4)_{m+1} \biggr |^2 \end{aligned}$$
(3.3)
$$\begin{aligned}&\supset i \sqrt{\frac{3}{2}} g W^-_\mu (\partial ^\mu \varphi ^-_2 \varphi ^{++} - \partial ^\mu \varphi ^{++} \varphi ^-_2)\nonumber \\&\quad + i \sqrt{2} g W^-_\mu ( \partial ^\mu \varphi ^{0*} \varphi ^+_2 - \partial ^\mu \varphi _2^+ \varphi ^{0*} ) + h.c., \end{aligned}$$
(3.4)
Fig. 4
figure 4

The cross section for pair production process \(p p \rightarrow Z/\gamma \rightarrow \Sigma ^{++} \Sigma ^{--}\) as a function of \(\Sigma ^{\pm \pm }\) mass at 14 TeV and 27 TeV

Table 2 Different benchmark points of this model. \(\lambda _0 \simeq -4.12 \times 10^{-5} \mu _{B}\)/GeV, where we have kept \(\mu _B\) at EW scale, thus keeping \(\lambda _0\) well within perturbative limit

where \((\Phi _4)_m\) indicates the component of \(\Phi _4\) which has the eigenvalue of diagonal SU(2) generator \(T_3\) given by m, and the last line shows the relevant interactions for decay of \(\varphi ^{\pm \pm }\) and \(\varphi ^\pm _2\). In addition, Yukawa coupling associated with \(\Sigma _R\) can be expanded as

$$\begin{aligned}&-\mathcal{L} \supset (y_\nu )_{ij} \left[ \bar{\nu }_{L_i} \left( \frac{1}{\sqrt{2}}\Sigma _{R_j}^0 \varphi ^{0*} + \frac{\sqrt{3}}{2} \Sigma _{R_j}^+ \varphi _1^-\right. \right. \nonumber \\&\quad \left. - \frac{1}{2} \Sigma _{L_j}^{+c} \varphi _{2}^+ + \Sigma _{R_j}^{++} \varphi ^{--} \right) \nonumber \\&\quad + \bar{\ell }_{L_i} \left( \frac{1}{\sqrt{2}}\Sigma _{R_j}^0 \varphi _1^- + \frac{1}{2} \Sigma _{R_j}^+ \varphi ^{--}\right. \nonumber \\&\quad \left. \left. - \frac{\sqrt{3}}{2}\Sigma _{L_j}^{+c} \varphi ^{0*} + \Sigma _{L_j}^{++c} \varphi _{2}^{+} \right) \right] \nonumber \\&\quad + \frac{(y_\varphi )_{ij}}{\sqrt{6}} \left[ \bar{\Sigma }^{0c}_{R_i} \Sigma ^{0}_{R_j} \xi _0 + \bar{\Sigma }^{+c}_{R_i} \Sigma ^{+c}_{L_j} \xi _0\right. \nonumber \\&\quad \left. - 2 \bar{\Sigma }^{++c}_{R_i} \Sigma ^{++c}_{L_j} \xi _0 - \sqrt{\frac{3}{2}} \bar{\Sigma }^{+}_{L_i} \Sigma ^{+c}_{L_j} \xi _1^{++} + \sqrt{6} \bar{\Sigma }^{++c}_{R_i} \Sigma ^{+c}_{L_j} \xi _2^- \right. \nonumber \\&\quad - \sqrt{\frac{3}{2}} \bar{\Sigma }^{+c}_{R_i} \Sigma ^{+}_{R_j} \xi _2^{--} - \bar{\Sigma }^{0c}_{R_i} \Sigma ^{+c}_{L_j} \xi _1^+ +2 \bar{\Sigma }^{0c}_{R_i} \Sigma ^{++c}_{L_j} \xi _2^{++}\nonumber \\&\quad \left. + \bar{\Sigma }^{0c}_{R_i} \Sigma ^{+}_{R_j} \xi _2^- + 2 \bar{\Sigma }^{0c}_{R_i} \Sigma ^{++}_{R_j} \xi _2^{--} \right] +\mathrm{h.c.}, \end{aligned}$$
(3.5)

where these terms are obtained from second and forth term in Eq. (2.42.10).

Here we consider signature of the model focusing on the production of doubly charged fermion in \(\Sigma _R\). The doubly charged fermion pair can be produced via electroweak interaction as

$$\begin{aligned} p p \rightarrow Z/\gamma \rightarrow \Sigma ^{++} \Sigma ^{--}. \end{aligned}$$
(3.6)

In order to compute the cross-sections and generate events at the LHC, we incorporate the model Lagrangian of Eqs. (3.2), (3.4), and (2.12) in FeynRules (v2.3.13) [34, 35]. Using FeynRules, we generate the model file for MadGraph5_aMC@NLO (v2.2.1) [36]. For the cross-sections, we use the NNPDF23LO1 parton distributions [37] with the factorization and renormalization scales at the central \(m_T^2\) scale after \(k_T\)-clustering of the event. We have computed the signal cross section of \(p p \rightarrow Z/\gamma \rightarrow \Sigma ^{++} \Sigma ^{--}\), where \(p = q, \bar{q}, \gamma \). The cross sections are normalised to the 5 flavor scheme. We have shown the production cross section in Fig. 4. The inclusion of the photon PDF increases the signal cross section significantly as the coupling is proportional to the charge of the fermion. Moreover, inclusion of photon PDF is important for the consistency of the calculation as the other PDF’s are determined up to NNLO in QCD. We would like to note that, in view of the above, NNPDF [38, 39], MRST [40] and CTEQ [41] have already included photon PDF into their PDF sets.

The possible decay modes of the fermions are,

$$\begin{aligned} \Sigma ^{\pm \pm }&\rightarrow \ell ^\pm \varphi ^{\pm }_2 \rightarrow \ell ^\pm W^{\pm *} \varphi _R \rightarrow \ell ^\pm W^{\pm *} h h, \nonumber \\ \Sigma ^{\pm \pm }&\rightarrow \nu \varphi ^{\pm \pm } \rightarrow \nu W^{\pm *} \varphi ^+_2 \nonumber \\&\rightarrow \nu W^{\pm *} W^{\pm *} \varphi _R \rightarrow \nu W^{\pm *} W^{\pm *} h h, \end{aligned}$$
(3.7)

where \(\varphi _R \rightarrow hh\) decay is induced by the interaction with coupling \(\lambda _0\). This gives rise to final states comprising of a number of leptons, jets, and missing energy.

Some benchmark points of the model are shown in Table 2 for various values of the parameter space. Note that the values of \((y_\nu )\) satisfies the constrains from LFV’s and provide maximal muon \(g-2\) contribution, as discussed in the previous subsection. It is evident from Table 2 that the branching ratio of \(\Sigma ^{\pm \pm }\) to e, \(\mu \) and \(\tau \) depends on the choice of \(y_\nu \). In Table 2, branching ratio to (\(l=e, \mu \)) and \(\tau \) are given separately. In the collider analysis we focus on states involving \(l=e, \mu \) only and we have assumed a simplified scenario, where BR(\(\Sigma ^{\pm \pm }\rightarrow \ell ^\pm \varphi ^{\pm }_2) =(\Sigma ^{\pm \pm } \rightarrow \nu \varphi ^{\pm \pm })\sim 50\%\) and we assumed it to be same for every lepton family. A detailed analysis involving the parameter space where the decay of \(\Sigma ^{\pm \pm }\) to (\(\tau \varphi ^{\pm \pm }\)) is maximum, will be studied elsewhere.

Once \(\Sigma ^{\pm \pm }\) is produced in pair, the three major channels to observe this signal are (\(l^+W^+ h h\),\(l^-W^- h h\)), (\(W^+W^+ h h\), \(W^-W^- h h\)) + MET and (\(l^+W^+ h h\),\(W^-W^- h h\)) + MET. W can decay either leptonically with BR \((W^\pm \rightarrow l \nu )\) = 0.108 for each lepton or hadronically with BR \((W^\pm \rightarrow q \bar{q})\) = 0.676. The cross section\(\times \) BR in each possible case is given below for two cases, where all W’s decay leptonically or all decay hadronically because that will give the minimum \(\sigma \times BR\) and maximum \(\sigma \times BR\) respectively. There can be many other possible channels with different combinations of leptons and jets with \(\sigma \times BR\) varying between these two numbers. We have kept the mass of \(\Sigma ^{\pm \pm }\) at 1 TeV in the following.

$$\begin{aligned}&\sigma \times BR(l^+W^+ h h, l^-W^- h h)\\&\quad \rightarrow (l^+l^+)(l^-l^-)(hhhh)+MET \sim 0.06\hbox { fb}\\&\quad \rightarrow (l^+l^+)(4j)(hhhh)\sim 0.58\hbox { fb}\\&\sigma \times BR(W^+W^+ h h, W^-W^- h h+\hbox { MET})\\&\quad \rightarrow (l^+l^+)(l^-l^-)(hhhh)+MET \sim 0.003\hbox { fb}\\&\quad \rightarrow (l^+l^+)(8j)(hhhh)+ MET \sim 0.26\hbox { fb}\\&\sigma \times BR(l^+W^+ h h, W^-W^- h h +\hbox { MET})\\&\quad \rightarrow (l^+l^+)(l^-l^-)(l^-l^-)(hhhh)+MET \sim 0.03\hbox { fb}\\&\quad \rightarrow (l^+)(6j)(hhhh)+ MET \sim 0.77\hbox { fb} \end{aligned}$$

If the final state is rich with jets, coming from the decay of W, \(\sigma \times BR\) is higher, but then the QCD background will be dominant. On the other hand, final states with the requirement of 1–4 leptons have negligible background and the channels are comparatively clean. Hence here we focus on the final state \((l^+l^+)(l^-l^-)(hhhh)+MET\) as the cross section is relatively high, compared to the other leptonic channels.

In the final state we require at least 4 b-jets, coming from the Higgs and at least one of the two oppositely charged lepton pairs. We have checked that if we demand \(\ge \)6 b-jets, or exactly two oppositely charged lepton pairs, the signal efficiency decreases significantly. For event reconstruction we have based our analysis on Ref. [42]. Events with b-tagged jet with transverse momentum \(p_T>\) 40 GeV and \(|\eta |<2.5\) are considered. Then we select at least two Higgs boson candidates, each composed of two b-tagged anti-\(k_t\) small-R jets, with invariant masses near \(m_H\). The invariant mass of the two-Higgs-boson-candidate system \(M_{4b}\) is used as the final discriminant which is expected to peak at \(m_{ \varphi _R }\). For the leading and subleading leptons, the transverse momentum selections are \(p_T(l_1)>\) 40 GeV, \(p_T(l_2)>\)20 GeV, if any other lepton is present in the event then the minimum \(p_T\) is required to be 10 GeV. The other selections for the leptons are \(|\eta |<2.5\), \(\Delta R(l,l) > 0.4\), \(\Delta R(l,b) > 0.4\).

In our study we choose three representative points, \(m_{\varphi _R}\) = 600, 700 and 800 GeV, where the mass difference between the other scalars and the fermions of the model are kept well with O(10) GeV to O(100) GeV. Pairing of the b-jets is required to satisfy the following criteria for the angular distance between them, (see Ref. [42]).

$$\begin{aligned}&\frac{360}{m_{4b}}-0.5< \Delta R_{bb}(\mathrm{leading})< \frac{653}{m_{4b}}+0.475 \\&\frac{235}{m_{4b}}< \Delta R_{bb}(\mathrm{subleading}) < \frac{875}{m_{4b}}+0.35 \end{aligned}$$

From the first combination we get the leading Higgs and from the second we get the subleading Higgs candidate. Moreover, the pair that gives \(m_{2b}\) closest to the Higgs mass are considered to be the leading pair of jets. In order to reject multijet events we also choose \(|\Delta \eta (hh)|<1.5\). The distribution of the leading and sub-leading b-jet pairs are shown in Fig. 5. We also show the 4-bjet invariant mass distribution in Fig. 6 for \(m_{\varphi _R}\) = 600, 700 and 800 GeV. If the mass of \(m_{\varphi _R}\ge \) 1 TeV, then boosted jet techniques are required for the analysis which is beyond the scope of this paper.

Fig. 5
figure 5

The invariant mass distribution of leading (left) and subleading (right) b-jet pairs in events of \(p p \rightarrow Z/\gamma \rightarrow \Sigma ^{++} \Sigma ^{--}\) at 14 TeV

Finally we select the events that satisfy,

$$\begin{aligned} (M_{H}-15)\, \hbox {GeV}< & {} M_{2b}< (M_{H}+15)\,\hbox { GeV}\\ (M_{\varphi _R}-200)\,\hbox { GeV }< & {} M_{4b} < (M_{\varphi _R}+200)\,\hbox { GeV}, \end{aligned}$$

where \(M_H\) is the SM Higgs mass. After all the selections as mentioned above, the number of events to expect at 14 TeV LHC at different luminosities are given in Table 3. Note that, we do not include the effect of the running of the coupling constant (see previous section), as the mass of the scalar (\(\varphi _R\))is less than 1 TeV, but for higher masses the running will be important. The number of events is given for both (4b-jets+ 1l) and (4-bjets+\(l^+l^-\)) channels. The number of events in each channel will further improve at a higher center of mass energy, 27 TeV. Hence the luminosity reach as a function of \(\varphi _R\) mass is given in Fig. 7 for 14 TeV and 27 TeV center of mass energy at LHC.

Fig. 6
figure 6

The four b-jet invariant mass distribution in events of \(p p \rightarrow Z/\gamma \rightarrow \Sigma ^{++} \Sigma ^{--}\) at 14 TeV for different masses of \(\varphi _R\)

Table 3 Number of expected events at 150 \(fb^{-1}\), 300 \(fb^{-1}\) and 3000\(fb^{-1}\) at 14 TeV p-p collision in different channels
Fig. 7
figure 7

The required luminosity to observed at least 10 events in 4 b-jets and at least one lepton final state for \(p p \rightarrow Z/\gamma \rightarrow \Sigma ^{++} \Sigma ^{--}\) at 14 TeV and 27 TeV as a function of \(\Sigma ^{\pm \pm }\) mass

4 Conclusions and discussions

In this paper, we have considered an extension of the SM, introducing large \(SU(2)_L\) multiplet fields such as quartet, quintet and septet scalar fields, and Majorana quintet fermions. In our scenario, the quintet and septet scalars have vacuum expectation values which are constrained by the \(\rho \)-parameter, while the quartet scalar filed does not develop a VEV which is realized by assuming relation among the parameters in the potential. Then, the active neutrino masses can be induced by interactions among these multiplets and the neutrinos at one loop level. We have found that the neutrino masses are suppressed by the small VEVs of the septet and a loop factor, explaining the smallness of the neutrino mass with relaxing the Yukawa hierarchies. Carrying out numerical analysis, we find the neutrino data can be accommodated with \(\mathcal {O}(0.1)\) to \(\mathcal {O}(1)\) Yukawa couplings taking extra particle masses at TeV scale.

We have also discussed the collider physics considering production processes of charged particles in the large multiplets especially focusing on doubly charged fermion from quintet. The doubly charged scalar decays into lepton and components of quartet scalar via Yukawa interaction generating neutrino mass. The components of quartet scalar decay via gauge interaction and/or interactions in scalar potential. We then obtain signal of multi Higgs boson plus charged leptons and/or jets with/without missing transverse energy. It has been shown that we can test our model in future LHC experiments estimating number of events imposing specific kinematical cuts. We have shown the discovery potential of this background free channel at current and future luminosities, at LHC.