1 The model

Recently, both ATLAS and CMS have found an excess in the diphoton channel around \(750~\mathrm{GeV}\) with a width possibly \({\lesssim }45~\mathrm{GeV}\) in the LHC run-II data at \(\sqrt{s}=13~\mathrm{TeV}\) [1, 2]. The local significance of the excess is \(3.6\sigma \) at \(747~\mathrm{GeV}\) and \(2.6\sigma \) at \(760~\mathrm{GeV}\), and the global significance is \(2.0\sigma \) and \(1.2\sigma \) for ATLAS [1] and CMS [2], respectively. While CMS previously observed a slight excess \(\sim 2\sigma \) around \(750~\mathrm{GeV}\) at \(\sqrt{s}=8~\mathrm{TeV}\) [3], ATLAS did not go beyond \(600~\mathrm{GeV}\) in the same channel [4]. If the excess persists with accumulation of more data in the near future, it will likely point to new physics beyond the standard model (SM). The excess has caused a burst of discussions on its possible origin in various phenomenological frameworks [5–161].

On the other hand, there is an established piece of evidence for new physics beyond SM, namely the neutrino mass; and dark matter whose gravitational evidence is robust may also originate from new particles and physics that are unknown to us so far. It would be desirable if the newly found excess is related to the same physics of the neutrino mass or dark matter. In this work we try to connect the excess with physics that is responsible for neutrino mass. Since the cross section for the excess is large for such a heavy resonance, it seems natural that the resonance is produced and decays through interactions with some new particles that participate in strong and electromagnetic interactions. In this context the color-octet model of the neutrino mass [162] stands out, in which the octet particles generating radiative neutrino mass would also couple to the resonance field resulting in its strong production and electromagnetic decay.

Since the nature of the resonance, named \(\Phi \) below, such as parity and spin is completely unknown, we treat it in the simplest manner from the point of view of effective field theory. Assuming it is an electroweak singlet of spin zero, its relevant effective interaction with color-octet scalars is

$$\begin{aligned} V(\Phi ,H,S_x)\supset \mu _{rs}\Phi S_r^{a\dagger }S_s^a, \end{aligned}$$
(1)

where the octet fields \(S_r^a\) have the same electroweak quantum numbers as the SM Higgs field H with a referring to color and r enumerating \(N_s\ge 1\) such scalars. We note two points in passing. First, the interaction entering radiative neutrino mass is proportional to \(S_r^{a\dagger }HS_s^{a\dagger }H\) while the coupling in Eq. (1) does not contribute. Second, the interaction may originally be a combination of trilinear and quartic terms when the field develops an expectation value. We are aware that a large \(\mu \) could be potentially dangerous for vacuum stability, but a simple estimation based on the boundedness from below indicates that for TeV-scale octet scalars and appropriate values of scalar couplings, there is no threat on the vacuum stability as long as \(\mu \) does not far exceed TeV scale. For renormalization-group improved analyses of the issue, see Ref. [163] in the presence of color-octet scalars, Ref. [159] with color-octet scalars coupled to a singlet scalar \(\Phi \), and Refs. [160, 161] with the singlet \(\Phi \) being a scalar or pseudoscalar that couples to a vector-like fermion. The octet fermions carry no hypercharge and can be a singlet, \(\rho _x^a\), or a triplet \(\chi _x^a\), of \(SU(2)_L\), where x enumerates \(N_f\) species of fermions. The Yukawa coupling for radiative neutrino mass is proportional to \(\bar{F}_{L}\tilde{S}^a_r\rho _x^a\) or \(\bar{F}_L\chi _x^a\tilde{S}_r^a\), where \(F_L\) is the SM left-handed lepton doublet, \(\tilde{S}=i\sigma _2S^*\), and \(\chi \) is written in a \(2\times 2\) matrix form. The relevant Yukawa coupling for our purpose here is simply

$$\begin{aligned} \mathcal{L}_\rho&\supset&-y_{xy}\Phi \overline{\rho _x^a}\rho _y^a, \end{aligned}$$
(2)

or

$$\begin{aligned} \mathcal{L}_\chi&\supset&-y_{xy}\Phi \text {tr}\overline{\chi _x^a}\chi _y^a. \end{aligned}$$
(3)

For simplicity we will assume a diagonal and universal Yukawa coupling in numerical analysis. The neutral singlet scalar \(\Phi \) in general mixes with the SM Higgs boson. While the mixing can be made small in a complete theory by arranging, for instance, a relatively small quartic coupling to the SM Higgs field, it must be phenomenologically small in order not to break the constraints established for the SM Higgs boson; see for instance Ref. [159]. We will therefore neglect the mixing in our work.

2 The diphoton excess and LHC constraints

The rich phenomenology for color-octet particles at LHC has been extensively studied in the literature [164–169]. We mention some of it relevant to our study here. The Yukawa couplings of S to quarks (q) must be small to avoid constraints from flavor physics and single production of neutral color-octet scalars at LHC [170–172]. Consequently, the effective \(\Phi q\bar{q}\) coupling induced by the S loop can be ignored. The direct search for pair production of S in the \(Zgb\bar{b}\) (with g denoting gluon) final state by CMS has excluded \(m_S<625~\mathrm{GeV}\) at 95 % CL [173], while the CMS search for four jets [174] and ATLAS search for four tops [175] have excluded \(m_S\lesssim 830~\mathrm{GeV}\) at 95 % CL. Concerning color-octet fermions, the pair production of gluinos has been searched for at LHC in the context of simplified supersymmetric models, with their masses excluded up to 1.1–\(1.2~\mathrm{TeV}\) at 8 TeV LHC [176, 177]. Very recently, the analysis of 13 TeV LHC data has significantly extended the lower mass bound up to 1.6–\(1.8~\mathrm{TeV}\) [178, 179]. Considering these direct search constraints, we will illustrate our numerical result by assuming \(m_{\chi ,\rho }=2~\mathrm{TeV}\) and \(m_S=1,~1.5~\mathrm{TeV}\) throughout this paper.

Currently, there exist a series of observed upper limits related to various decay final states of the singlet \(\Phi \), which must be respected in our analysis. To be specific, for \(m_\Phi =750~\mathrm{GeV}\), the searches for jj, VV, hh, \(t\bar{t}\), and \(\ell ^{+}\ell ^{-}\) channels at LHC \(8~\mathrm{TeV}\) have set the bounds on the cross sections: \(\sigma _{jj}<1.78~\mathrm{pb}\) [180, 181],Footnote 1 \(\sigma _{hh}<45~\mathrm{fb}\) [182–184], \(\sigma _{WW}<60~\mathrm{fb}\) [185], \(\sigma _{ZZ}<12~\mathrm{fb}\) [186], \(\sigma _{Z\gamma }<6~\mathrm{fb}\) [187], \(\sigma _{t\bar{t}}<300~\mathrm{fb}\) [188, 189], and \(\sigma _{\ell ^{+}\ell ^{-}}<1.3~\mathrm{fb}\) [190]. Among them, the constraints from the hh, WW, ZZ, and \(t\bar{t}\) channels can be safely ignored due to negligible mixing between the SM Higgs h and \(\Phi \). As we will see later, the bounds from the \(Z\gamma \) and dilepton channels can also easily be evaded.

Fig. 1
figure 1

Cross sections \(\sigma _{gg}\) (red curves) and \(\sigma _{\gamma \gamma }\) (green) are shown as a function of \(\mu \) for the singlet fermion case with \(m_\rho =2~\mathrm{TeV}\), \(m_S=1~\mathrm{TeV}\). The left and right panels correspond to \((N_f,N_s)=(2,1),~(1,2)\) respectively, and the solid (dashed) curve assumes \(y=0.2\) (0.5)

Full size image

We now consider the diphoton signature at LHC. The excess observed by ATLAS and CMS implies a theoretical diphoton rate roughly in the range \(\sigma _{\gamma \gamma }\sim 3\)–\(10~\mathrm{fb}\) depending on the narrow or wide width assumption of \(\Phi \) that was obtained in Refs. [157, 158] by taking into account efficiency and acceptance; see for instance, the footnote 1 in Ref. [158]. From the interactions shown in Eqs. (1) and (2) or (3), we obtain the decay widths of \(\Phi \) to the most relevant channels:

$$\begin{aligned} \Gamma _{\Phi \rightarrow gg}^\rho= & {} \frac{N_c^2\alpha _s^2m_\Phi ^3}{128\pi ^3}\left| N_f\sum _{\rho }\frac{2y}{m_\rho }A_{1/2}(\tau _\rho )\right. \nonumber \\&\left. +\,N_s\sum _{S^0,S^{\pm }}\frac{\mu }{m_S^2}A_0(\tau _S)\right| ^2, \end{aligned}$$
(4)
$$\begin{aligned} \Gamma _{\Phi \rightarrow \gamma \gamma }^\rho= & {} \frac{(N_c^2-1)^2\alpha ^2m_\Phi ^3}{1024\pi ^3} \left| N_s\sum _{S^{\pm }}\frac{\mu }{m_S^2}A_0(\tau _S)\right| ^2, \end{aligned}$$
(5)
$$\begin{aligned} \Gamma _{\Phi \rightarrow Z\gamma }^\rho= & {} \frac{(N_c^2-1)^2\alpha ^2m_\Phi ^3}{512\pi ^3}\left( 1-\frac{m_Z^2}{m_\Phi ^2}\right) ^3\nonumber \\&\times \left( \frac{1-2s_W^2}{c_Ws_W}\right) ^2\left| N_s\sum _{S^{\pm }} \frac{\mu }{m_S^2}A_0(\tau _S^{-1},\eta _S^{-1})\right| ^2,\qquad \end{aligned}$$
(6)

for the case of singlet fermions \(\rho \), and

$$\begin{aligned} \Gamma _{\Phi \rightarrow gg}^\chi= & {} \frac{N_c^2\alpha _s^2m_\Phi ^3}{128\pi ^3} \left| N_f\sum _{\chi ^0,\chi ^{\pm }}\frac{2 y}{m_\chi } A_{1/2}(\tau _\chi )\right. \nonumber \\&\left. +\,N_s\sum _{S^0,S^{\pm }}\frac{\mu }{m_S^2}A_0(\tau _S)\right| ^2,\end{aligned}$$
(7)
$$\begin{aligned} \Gamma _{\Phi \rightarrow \gamma \gamma }^\chi= & {} \frac{(N_c^2-1)^2\alpha ^2m_\Phi ^3}{1024\pi ^3} \left| N_f\sum _{\chi ^{\pm }}\frac{2 y}{m_\chi }A_{1/2}(\tau _\chi )\right. \nonumber \\&\left. +\,N_s\sum _{S^{\pm }}\frac{\mu }{m_S^2}A_0(\tau _S)\right| ^2,\end{aligned}$$
(8)
$$\begin{aligned} \Gamma _{\Phi \rightarrow Z\gamma }^\chi= & {} \frac{(N_c^2-1)^2\alpha ^2m_\Phi ^3}{512\pi ^3}\nonumber \\&\left( 1-\frac{m_Z^2}{m_\Phi ^2}\right) ^3 \left( \frac{1-2s_W^2}{c_W s_W}\right) ^2 \nonumber \\&\times \,\left| N_f\sum _{\chi ^{\pm }}\frac{4 y}{m_\chi }A_{1/2}(\tau _\chi ^{-1},\eta _\chi ^{-1})\right. \nonumber \\&\left. -N_s\sum _{S^{\pm }}\frac{\mu }{m_S^2}A_0(\tau _S^{-1},\eta _S^{-1})\right| ^2, \end{aligned}$$
(9)

for the case of triplet fermions \(\chi \), where \(N_c=3\) is the number of colors and \(\alpha \) (\(\alpha _s\)) is the QED (QCD) fine structure constant. To reduce the number of parameters we will assume in the numerical analysis that the \(N_s\) (\(N_f\)) species of octet scalars (fermions) are degenerate. The loop functions \(A_{0,1/2}\) of \(\tau _i=m_\Phi ^2/(4m_i^2)\) and of \(\tau _i\) and \(\eta _i=m_Z^2/(4m_i^2)\) are well known, and they can be found, for instance, in Ref. [191, 192]. Note that the decay width for the diphoton channel distinguishes between the singlet and triplet cases since the electrically neutral singlet \(\rho \) does not enter. In the singlet case, the branching ratio for diphoton may receive considerable enhancement when there is a strong cancellation in the digluon channel between the octet fermions and scalars, which occurs for \(y\mu <0\). In contrast, for the triplet case, the branching ratio for diphoton tends to a constant when the neutral and charged particles are almost degenerate.

In the narrow width approximation, the diphoton cross section from the \(\Phi \) resonance is estimated as

$$\begin{aligned} \sigma _{\gamma \gamma } = \frac{\Gamma _{\Phi \rightarrow gg}}{\Gamma _{H\rightarrow gg}^\mathrm{SM}} \times \sigma ^\mathrm{SM}(H)\times \text{ BR }(\Phi \rightarrow \gamma \gamma ), \end{aligned}$$
(10)

where \(\Gamma _{H\rightarrow gg}^\mathrm{SM}\) denotes the digluon decay width of a SM-like Higgs boson H scaled up to \(m_H=750~\mathrm{GeV}\), and \(\sigma ^\mathrm{SM}(H)=735~\mathrm{fb}\) is its production cross section at \(13~\mathrm{TeV}\) LHC with NNLO accuracy [193, 194]. For \(\Gamma _{H\rightarrow gg}^\mathrm{SM}\), we employed the full one-loop result in Refs. [191, 192] with a top quark mass \(m_t=172~\mathrm{GeV}\) and neglected the bottom quark contribution. The value of \(\sigma ^\mathrm{SM}(H)\) at 13 TeV was obtained by rescaling its value (\(156.8~\mathrm{fb}\)) listed in Ref. [193] for a mass of \(750~\mathrm{GeV}\) at 8 TeV LHC by the gg parton luminosity ratio (4.69) from 8 TeV to 13 TeV [194]. In our scenario, the constraint in the \(Z\gamma \) channel is trivially fulfilled since the decay width for \(\Phi \rightarrow Z\gamma \) is always smaller than that for \(\Phi \rightarrow \gamma \gamma \) in both the singlet and the triplet cases. The dilepton decays of \(\Phi \) are induced at one loop through the trilinear coupling in Eq. (1) and the Yukawa coupling \(y^{\prime }\bar{F}_{L}\tilde{S}\rho \) or \(y^{\prime }\bar{F}_{L}\chi \tilde{S}\). The same Yukawa coupling appears also in the anomalous magnetic moments and electromagnetic transitions of the charged leptons, and is thus severely constrained by current experimental bounds. Without requiring a special flavor structure, these bounds impose a universal limit \(|y^{\prime }|\lesssim 0.01\). This implies that the dilepton decay widths of \(\Phi \) are generally much smaller than that of diphoton. We therefore only have to consider the dijet constraint from \(\Phi \rightarrow gg\). In our numerical analysis, the upper limit on \(\sigma _{jj}\) at \(13~\mathrm{TeV}\) is obtained by rescaling the limits at \(8~\mathrm{TeV}\) through their gg parton luminosity ratio, which yields \(\sigma _{jj}<8.35~\mathrm{pb}\). We will restrict ourselves to the minimal choice that can induce two nonzero neutrino masses, namely, with the numbers of octet species being \((N_f,N_s)=(2,1)\) or \((N_f,N_s)=(1,2)\).

Fig. 2
figure 2

Allowed parameter space in the y–\(\mu \) plane for the singlet fermion case with \(m_\rho =2~\mathrm{TeV}\) and \(m_S=1~\mathrm{TeV}\) (yellow) or \(m_S=1.5~\mathrm{TeV}\) (green). The left and right panels correspond to \((N_f,N_s)=(2,1),~(1,2)\), respectively

Full size image

We first investigate the singlet fermion case. In Fig. 1, we plot the diphoton (\(\sigma _{\gamma \gamma }\)) and dijet (\(\sigma _{gg}\)) cross sections as a function of the trilinear coupling \(\mu \) for the two minimal choices of the neutrino mass generation. The vertical bands represent the allowed region of \(\mu \) by the required diphoton excess and the upper dijet limit, with the blue (pink) band corresponding to the Yukawa coupling \(y=0.2\) (0.5). In addition, the allowed parameter space in the y–\(\mu \) plane is presented in Fig. 2. From these two figures, we learn some important features:

  • The diphoton cross section required by the excess can be reached for \(\mathcal {O}(0.1)\) Yukawa coupling and \(\mathcal {O}(\mathrm{TeV})\) trilinear coupling \(\mu \).

  • For \(\mu \) positive (negative), the main constraint comes from \(\sigma _{gg}\) (\(\sigma _{\gamma \gamma }\)). This arises from the interference effect in the gg channel which is constructive for \(\mu >0\) and destructive for \(\mu <0\) (recalling that y is always assumed to be positive), while the \(\gamma \gamma \) channel has no such interference as it is contributed only by the octet scalars.

  • For a smaller Yukawa coupling \(y=0.2\), \(\mu \) is allowed to be either positive or negative. On the contrary, for a larger Yukawa coupling, only negative \(\mu \) survives. This also results from the interference effect, and implies that the Yukawa coupling cannot be too large for a positive \(\mu \).

  • As shown in Fig. 2, the allowed regions for the diphoton excess are not so sensitive to y, because y enters in \(\sigma _{\gamma \gamma }\) only indirectly through the gg channel.

  • The allowed values of \(|\mu |\) are smaller in the case of \((N_f,N_s)=(1,2)\) than \((N_f,N_s)=(2,1)\), simply because more charged particles contribute to the diphoton process in the former case.

  • Since the loop functions are more suppressed by heavier octet particles, a larger \(|\mu |\) is demanded to fulfill the observed diphoton excess.

Fig. 3
figure 3

Same as Fig. 1, but for the triplet fermion case

Full size image
Fig. 4
figure 4

Same as Fig. 2, but for the triplet fermion case

Full size image

Now we turn to the triplet fermion case. The crucial difference from the singlet case is that now the charged members of \(\chi \) also contribute to the diphoton channel. From Eq. (7), one can roughly estimate that \(\mathrm{BR}(\Phi \rightarrow \gamma \gamma )\) is a constant, since

$$\begin{aligned} \frac{\Gamma _{\Phi \rightarrow \gamma \gamma }^\chi }{\Gamma _{\Phi \rightarrow gg}^\chi }= \frac{(N_c^2-1)^2\alpha ^2}{32N_c^2\alpha _s^2}=\frac{2\alpha ^2}{9\alpha _s^2}\approx 10^{-3}. \end{aligned}$$
(11)

Therefore, to explain the diphoton excess with \(\sigma _{\gamma \gamma }\sim 3\)–\(10~\mathrm{fb}\), we just need \(\sigma _{gg}\sim 3\)–\(10~\mathrm{pb}\). The cross sections \(\sigma _{\gamma \gamma ,gg}\) versus the trilinear coupling \(\mu \) are presented in Fig. 3, while Fig. 4 shows the allowed parameter space in the y–\(\mu \) plane. We summarize the properties as follows:

  • The diphoton excess can also be explained naturally with \(\mathcal {O}(0.1)\) Yukawa coupling and \(\mathcal {O}(\mathrm{TeV})\) trilinear coupling \(\mu \) in the case of triplet fermions \(\chi \).

  • The lower and upper bound on \(\mu \) comes, respectively, from \(\sigma _{\gamma \gamma }\) and \(\sigma _{gg}\), since \(\mathrm{BR}(\Phi \rightarrow \gamma \gamma )\) is a constant in this case. This behavior is different from the singlet \(\rho \) case.

  • It is clear that \(\mu <0\) must be satisfied with a bigger \(y=0.75\) in the case of \((N_f,N_s)=(2,1)\). Meanwhile for the case of \((N_f,N_s)=(1,2)\), \(\mu >0\) is still allowed at \(y=0.75\).

  • The simple relation between \(\Gamma _{\Phi \rightarrow \gamma \gamma }^\chi \) and \(\Gamma _{\Phi \rightarrow gg}^\chi \) implies that the allowed bands in the y–\(\mu \) plane present a linear correlation. This is clearly shown in Fig. 4. Moreover, a larger \(m_S\) leads to an increase of the slope, and for the same \(m_S\) the slope for the \((N_f,N_s)=(2,1)\) scenario is steeper than that of \((N_f,N_s)=(1,2)\).

3 Conclusion

We have interpreted the \(750~\mathrm{GeV}\) diphoton excess in the framework of color-octet neutrino mass model. In addition to a singlet scalar \(\Phi \) that plays the role of the \(750~\mathrm{GeV}\) resonance, the model introduces \(N_s\) species of color-octet, electroweak doublet scalars S and \(N_f\) color-octet fermions that can be an electroweak singlet \(\rho \) or triplet \(\chi \). The diphoton signal results from the production of \(\Phi \) via gluon fusion through color-octet messengers and its subsequent decay into the diphoton through interactions with charged color-octet particles. We find that for the triplet case \(\mathrm{BR}(\Phi \rightarrow \gamma \gamma )\) is a constant of about 0.1 %, while for the singlet case \(\mathrm{BR}(\Phi \rightarrow \gamma \gamma )\) can be varied. With the \(\mathcal {O}(0.1)\) Yukawa coupling y and the \(\mathcal {O}(\mathrm{TeV})\) trilinear coupling \(\mu \), both cases can explain the diphoton excess naturally without conflict with the current experimental limits. Meanwhile, the interference effect between the scalar and fermion octet particles plays a crucial role. The distinct features and allowed parameter space are discussed in detail for both cases, as well as for the two minimal scenarios of neutrino mass generation with \((N_f,N_s)=(2,1)\) or (1, 2).