Radiative neutrino mass model with degenerate right-handed neutrinos

Abstract

The radiative neutrino mass model can relate neutrino masses and dark matter at a TeV scale. If we apply this model to thermal leptogenesis, we need to consider resonant leptogenesis at that scale. It requires both finely degenerate masses for the right-handed neutrinos and a tiny neutrino Yukawa coupling. We propose an extension of the model with a U(1) gauge symmetry, in which these conditions are shown to be simultaneously realized through a TeV scale symmetry breaking. Moreover, this extension can bring about a small quartic scalar coupling between the Higgs doublet scalar and an inert doublet scalar which characterizes the radiative neutrino mass generation. It also is the origin of the $Z_2$ symmetry which guarantees the stability of dark matter. Several assumptions which are independently supposed in the original model are closely connected through this extension.

1 Introduction

ATLAS and CMS groups in the LHC experiment have reported the discovery of the Higgs-like particle [1, 2]. All the standard model contents seem to have been found by now. However, the standard model has serious problems from experimental and observational view points. Although the existence of neutrino masses and dark matter has been confirmed through various experiments and observations [3–14], it cannot be explained in the standard model. The standard model cannot give a framework for the generation of baryon number asymmetry in the Universe, either [15–17]. These facts now cause serious tension between the standard model and Nature so that they motivate us to consider its extension.

The radiative neutrino mass model proposed in [18] is a simple and interesting extension of the standard model which could be an explanation. In several previous articles [19–36], we have studied these problems in this model and its extensions. They suggest that these problems could be explained in a consistent way, simultaneously. Unfortunately, however, we could not justify several assumptions and the parameter tuning adopted in these explanations. For example, if we consider thermal leptogenesis in this model, both finely degenerate right-handed neutrino masses and a small Yukawa coupling for the lightest right-handed neutrino are required in order to make possible sufficient generation of lepton number asymmetry through the out-of-equilibrium decay of the lightest right-handed neutrino. In this work, we have just assumed them independently in a way consistent with other phenomenological issues.

In this paper, we consider an extension of the model which makes it possible to realize these required conditions simultaneously in the evolution of the Universe. We suppose a new symmetry breaking at a scale of O(1) TeV for this purpose. After this symmetry breaking, a small mass difference is induced between two lighter right-handed neutrinos, although they have an equal mass originally. At the same time, a Yukawa coupling of the lightest right-handed neutrino becomes much smaller than that of the heavier one. To realize this scenario, we introduce a low energy U(1) gauge symmetry to the model. We show that (i) both the almost degenerate right-handed neutrino masses and a tiny neutrino Yukawa coupling, which are indispensable for TeV scale resonant leptogenesis [37–40], are brought about after the breaking of this symmetry. Moreover, we find that this extension can also explain important key features required in the original Ma model, that is, (ii) a small quartic coupling between the Higgs doublet scalar and an inert doublet scalar which plays a crucial role in the neutrino mass generation, and (iii) the origin of the $Z_2$ symmetry which guarantees the stability of dark matter.

The remaining part of this paper is organized as follows. After introducing an extended model in the next section, we discuss features in the scalar sector and also the right-handed neutrino mass degeneracy. Baryon number asymmetry generated through the thermal leptogenesis is studied taking account of these. In Sect. 3, we study the dark matter relic abundance and other cosmological aspects of the model. Finally, in Sect. 4 we give a brief summary of the main results of the paper.

2 An extended model

2.1 U(1) gauge symmetry at a TeV scale

The original Ma model is a simple extension of the standard model which can relate neutrino masses and dark matter [18]. In this model, only an inert doublet scalar $\eta$ and right-handed neutrinos $N_i$ are added to the standard model. Although ingredients of the standard model are assigned an even parity of the imposed $Z_2$ symmetry, new fields are assumed to have odd parity. This feature forbids tree-level neutrino mass generation and guarantees the stability of dark matter.

We extend this model with a $U(1)$ $_X$ gauge symmetry, a singlet scalar S, and also additional right-handed neutrinos $\tilde{N}_i$ whose number is equal to the one of $N_i$. The $U(1)$ $_X$ charge is assigned each new ingredient as $Q_X(S)=2$, $Q_X(\eta )=-1$, $Q_X(N_i)=1$, and $Q_X(\tilde{N}_i)=-1$. Normalization for the $U(1)$ $_X$ charge and coupling is fixed through a covariant derivative, which is defined as $D_\mu =\partial _\mu -ig\frac{\tau ^a}{2}W_\mu ^a-ig_Y\frac{Y}{2}B_\mu -ig_X\frac{Q_X}{2}X_\mu$. Since the standard model fields are assumed to have no charge for this $U(1)$ $_X$, it is obvious that the $U(1)_X$ is anomaly free. If this symmetry is assumed to break down due to a vacuum expectation value $\langle S\rangle$, the model has a remnant exact symmetry $Z_2$ after this breaking. Since only $\eta$, $N_i$, and $\tilde{N}_i$ have odd parity, the lightest one of them is stable and can be dark matter. We assume that dark matter is the lightest neutral component of $\eta$ in this study.

The relevant part of the Lagrangian for these new ingredients of the model is summarized as

$$\begin{aligned} \begin{aligned} -\mathcal{L}_N=&\,h_{\alpha i} \bar{N_i}\eta ^\dagger \ell _{\alpha } +f_{\alpha i}\frac{S^\dagger }{M_*}\bar{\tilde{N_i}} \eta ^\dagger \ell _{\alpha }+M_iN_i\tilde{N}_i\\&+\frac{y_i}{2}S^\dagger N_iN_i+ \frac{\tilde{y}_i}{2}S\tilde{N_i}\tilde{N_i} + \mathrm{h.c.}, \\ V=&\,\lambda _1(\phi ^\dagger \phi )^2+\lambda _2(\eta ^\dagger \eta )^2 +\lambda _3(\phi ^\dagger \phi )(\eta ^\dagger \eta )\\&+ \lambda _4(\eta ^\dagger \phi )(\phi ^\dagger \eta ) +\frac{\lambda _5^\prime }{2} \Bigg [\frac{S}{M_*}(\phi ^\dagger \eta )^2 + \mathrm{h.c.}\Bigg ] \\&+ \lambda _6(S^\dagger S)(\phi ^\dagger \phi ) + \lambda _7(S^\dagger S) (\eta ^\dagger \eta ) +\kappa (S^\dagger S)^2 \\&+ m_\phi ^2\phi ^\dagger \phi +m_\eta ^2\eta ^\dagger \eta +m_S^2S^\dagger S, \end{aligned} \end{aligned}$$

(1)

where $\ell _{\alpha }$ is a left-handed doublet lepton and $\phi$ is an ordinary doublet Higgs scalar. $M_*$ is a cut-off scale of this model. The bare masses $M_i$ and $m_\eta$ in Eq. (1) are assumed to be real and of O(1) TeV. The couplings $h_{\alpha i}$ and $f_{\alpha i}$ in the neutrino sector are considered to be written by using the basis in which the Yukawa coupling matrix of charged leptons is diagonal. As easily found in Eq. (1), if the singlet S has a vacuum expectation value, the coupling $\lambda _5$ in the original Ma model and neutrino Yukawa couplings $\tilde{h}_{\alpha i}$ for $\tilde{N}_i$ are determined as [23, 24]

$$\begin{aligned} \lambda _5=\lambda _5^\prime \frac{\langle S\rangle }{M_*}, \quad \tilde{h}_{\alpha i}=f_{\alpha i}\frac{\langle S^\dagger \rangle }{M_*}, \end{aligned}$$

(2)

where it may be natural to consider that both $\lambda _5^\prime$ and $f_{\alpha i}$ are of O(1). The magnitude of $\lambda _5$ is crucial for the neutrino mass determination in the model. We note that it can be small enough if $|\langle S\rangle |\ll M_*$ is satisfied. Scales assumed for $|\langle S\rangle |$ and $M_*$ in the present study are discussed below.

2.2 Scalar sector

First, we discuss the scalar sector of the model. We express the scalar fields by using a unitary gauge,

$$\begin{aligned} \phi ^T= & {} \left( 0, \langle \phi \rangle +\frac{h}{\sqrt{2}}\right) , \quad \eta ^T=\left( \eta ^+, \frac{1}{\sqrt{2}}(\eta _R+i\eta _I)\right) , \nonumber \\ S= & {} \langle S\rangle + \frac{\sigma }{\sqrt{2}}, \end{aligned}$$

(3)

where both vacuum expectation values $\langle \phi \rangle$ and $\langle S\rangle$ are assumed to be real and positive. In this vacuum, the new Abelian gauge boson $X_\mu$ gets a mass $m_X^2=2g_X^2\langle S\rangle ^2$. The scalar potential V in Eq. (1) can be represented by using Eq. (3) as

$$\begin{aligned} V= & {} \frac{1}{2}( 4\lambda _1\langle \phi \rangle ^2h^2+ 4\kappa \langle S\rangle ^2\sigma ^2 + 4\lambda _6\langle \phi \rangle \langle S\rangle h\sigma )\nonumber \\&+\,\frac{1}{2}M_{\eta _R}^2\eta _R^2+ \frac{1}{2}M_{\eta _I}^2\eta _I^2 + M_{\eta _c}^2\eta ^+\eta ^- \nonumber \\&+\,\frac{1}{4}\left[ \sqrt{\lambda _1}h^2-\sqrt{\lambda _2} (2\eta ^+\eta ^-+\eta _R^2+\eta _I^2)-\sqrt{\kappa }\sigma ^2\right] ^2\nonumber \\&+\,\frac{1}{4}\left[ \left\{ 2(\lambda _3+2\sqrt{\lambda _1\lambda _2})\eta ^+\eta ^- \right. \right. \nonumber \\&+\,\left. \left. (\lambda _++2\sqrt{\lambda _1\lambda _2})\eta _R^2+ (\lambda _-+2\sqrt{\lambda _1\lambda _2})\eta _I^2 \right. \right. \nonumber \\&+\,\left. \left. (\lambda _6+2\sqrt{\lambda _1\kappa })\sigma ^2\right\} h^2\right. \nonumber \\&+\,\left. (\lambda _7-2\sqrt{\lambda _2\kappa })(2\eta _+\eta _-+\eta _R^2 +\eta _I^2)\sigma ^2\right] \nonumber \\&+\,\sqrt{2}\lambda _1\langle \phi \rangle h^3 +\sqrt{2}\kappa \langle S\rangle \sigma ^3 \nonumber \\&+\,{\sqrt{2}}(\lambda _3 \langle \phi \rangle h + \lambda _7\langle S\rangle \sigma )\eta _+\eta _- \nonumber \\&+\,\frac{1}{\sqrt{2}}\left( \lambda _+\langle \phi \rangle h +\lambda _7\langle S\rangle \sigma \right) \eta _R^2 \nonumber \\&+\,\frac{1}{\sqrt{2}}\left( \lambda _-\langle \phi \rangle h +\lambda _7\langle S\rangle \sigma \right) \eta _I^2 +\frac{\lambda _6}{\sqrt{2}}(\langle \phi \rangle h\sigma ^2 +\langle S\rangle \sigma h^2) \nonumber \\&+\, \frac{\lambda _5^\prime }{4\sqrt{2}M_*}\sigma h^2(\eta _R^2-\eta _I^2), \end{aligned}$$

(4)

where we use the definition $\lambda _\pm =\lambda _3+\lambda _4\pm \lambda _5$ and

$$\begin{aligned}&M_{\eta _c}^2=m_\eta ^2+\lambda _7\langle S\rangle ^2 + \lambda _3\langle \phi \rangle ^2, \nonumber \\&\quad M_{\eta _{R(I)}}^2=m_\eta ^2+ \lambda _7\langle S\rangle ^2 + \lambda _{+(-)}\langle \phi \rangle ^2. \end{aligned}$$

(5)

The difference between these masses is estimated to be

$$\begin{aligned}&\frac{M_{\eta _I}-M_{\eta _R}}{M_{\eta _R}}\simeq \frac{\lambda _5\langle \phi \rangle ^2}{M_{\eta _R}^2} \equiv \frac{\delta }{M_{\eta _R}}, \nonumber \\&\quad \frac{M_{\eta _c}-M_{\eta _R}}{M_{\eta _R}}\simeq \frac{(\lambda _4+\lambda _5)\langle \phi \rangle ^2}{2M_{\eta _R}^2}, \end{aligned}$$

(6)

which could be a good approximation as long as $m_\eta ^2+\lambda _7\langle S\rangle ^2 \gg \langle \phi \rangle ^2$ is satisfied. A large value of $m_\eta ^2+\lambda _7\langle S\rangle ^2$ is favored from the analysis of the T parameter in precise measurements of the electroweak interaction [47–55]. We assume such a situation in the present study.

Quartic scalar couplings in the potential V are constrained by several conditions. The stability of the assumed vacuum requires

$$\begin{aligned}&\lambda _1, \lambda _2, \kappa >0; \quad \lambda _3,\lambda _\pm > -2\sqrt{\lambda _1\lambda _2}; \quad \lambda _6>-2\sqrt{\lambda _1\kappa }; \nonumber \\&\lambda _7>-2\sqrt{\lambda _2\kappa }. \end{aligned}$$

(7)

These can be easily read off from the expression of the scalar potential V given in Eq. (4).¹ Perturbativity of the model imposes that these quartic couplings should be smaller than $4\pi$.² Moreover, if we assume that $\eta _R$ is the lightest one among the fields with odd parity of the remnant $Z_2$, Eq. (5) shows that the following conditions should be satisfied:

$$\begin{aligned} \lambda _4+\lambda _5<0, \quad \lambda _5<0; \quad M_{\eta _R} <\mathrm{min}(M_{\pm i}), \end{aligned}$$

(8)

where $M_{\pm i}$ are the mass eigenvalues for $N_i$ and $\tilde{N}_i$, which are discussed in detail later. Using the value of $\lambda _1$ predicted by the Higgs mass observed at LHC experiments [1, 2] and the conditions given in Eqs. (7) and (8), we can roughly estimate the allowed range of $\lambda _{3,4}$ as

$$\begin{aligned} -2.5<\lambda _3<4\pi , \quad -4\pi <\lambda _4 <0, \end{aligned}$$

(9)

for sufficiently small values of $|\lambda _5|$.

The potential minimum in Eq. (4) is obtained as

$$\begin{aligned} \langle \phi \rangle ^2=\frac{\lambda _6m_S^2-2\kappa m_\phi ^2}{4\lambda _1\kappa -\lambda _6^2}, \quad \langle S\rangle ^2=\frac{\lambda _6m_\phi ^2-2\lambda _1 m_S^2}{4\lambda _1\kappa -\lambda _6^2}. \end{aligned}$$

(10)

Since the new gauge boson does not couple with the standard model fields, both cases $\langle S\rangle ^2 \gg \langle \phi \rangle ^2$ and $\langle S\rangle ^2 \ll \langle \phi \rangle ^2$ could be phenomenologically allowed. However, if we apply this model to the leptogenesis, $\langle S\rangle ^2 \gg \langle \phi \rangle ^2$ should be satisfied as discussed later. Such a vacuum can be realized for a sufficiently small $|\lambda _6|$ satisfying $4\lambda _1\kappa \gg \lambda _6^2$ and negative values of $m_S^2$ and $m_\phi ^2$ satisfying $|m_S^2|\gg |m_\phi ^2|$. In this case, both vacuum expectation values are approximately expressed as $\langle \phi \rangle ^2\simeq -\frac{m_\phi ^2}{2\lambda _1}$ and $\langle S\rangle ^2\simeq -\frac{m_S^2}{2\kappa }$. If the contribution of $\langle S\rangle$ to the $\eta$ mass is of the same order as that of $\langle \phi \rangle$, $|\lambda _7|$ should be much smaller than $|\lambda _{3,4}|$ as found from Eq. (5).

Since h and $\sigma$ defined in Eq. (3) have mass mixing as found from the first line in Eq. (4), the mass eigenstates $\tilde{h}$ and $\tilde{\sigma }$ are a mixture of these. They are found to enable us to write

$$\begin{aligned} \tilde{h}\simeq h-\frac{\lambda _6\langle \phi \rangle }{2\kappa \langle S\rangle }\sigma , \quad \tilde{\sigma }=\sigma +\frac{\lambda _6\langle \phi \rangle }{2\kappa \langle S\rangle }h. \end{aligned}$$

(11)

However, since $\langle S\rangle ^2\gg \langle \phi \rangle ^2$ is assumed and $|\lambda _6|< \sqrt{\kappa }$ is expected, the mass eigenstates could be almost equal to h and $\sigma$. In this case, the mass eigenvalues are approximately expressed as

$$\begin{aligned} m_{\tilde{h}}^2 = \left( 4\lambda _1 -\frac{\lambda _6^2}{\kappa }\right) \langle \phi \rangle ^2, \quad M_{\tilde{\sigma }}^2\simeq 4\kappa \langle S\rangle ^2. \end{aligned}$$

(12)

These should have positive values for the stability of the considered vacuum. It requires $4\lambda _1\kappa > \lambda _6^2$, which is consistent with the above discussion.

The value of $\lambda _1$ might be estimated by using $m_{\tilde{h}}\simeq 125$ GeV. If we apply it to the tree-level formula in Eq. (12), we have

$$\begin{aligned} \lambda _1-\frac{\lambda _6^2}{4\kappa }\sim 0.13. \end{aligned}$$

(13)

This result suggests that $\lambda _1$ could have a somewhat larger value than the corresponding quartic coupling in the standard model. However, this effect is expected to be small since the assumed vacuum requires $4\lambda _1\kappa \gg \lambda _6^2$. On the other hand, the model has the additional scalar couplings $\lambda _3$ and $\lambda _4$, which are known to improve the potential stability [56, 57]. Thus, the constraint from the potential stability against the radiative correction in the present model could be milder than that of the standard model.

If we impose the requirement that $\tilde{\sigma }$ is heavier than the Higgs scalar, $\kappa$ satisfies $\kappa ~{^>_\sim }~10^{-3}\left( \frac{2~\mathrm{TeV}}{\langle S\rangle }\right) ^2$ and $\lambda _6$ could take a small value so as to be consistent with the condition $|\lambda _6|<2\sqrt{\lambda _1\kappa }$. If the above condition for $\kappa$ is not satisfied, $\tilde{\sigma }$ can be lighter than $\tilde{h}$ so as to realize $m_{\tilde{h}}>2 M_{\tilde{\sigma }}$. In that case, the coupling $\lambda _6$ satisfies $|\lambda _6|~{^<_\sim }~ 10^{-2} \left( \frac{\lambda _1}{0.13}\right) ^{\frac{1}{2}} \left( \frac{2~\mathrm{TeV}}{\langle S\rangle }\right)$ and the interaction in the last line of Eq. (4) induces the invisible decay $\tilde{h}\rightarrow 2\tilde{\sigma }$. The decay width can be estimated as

$$\begin{aligned} \Gamma (\tilde{h}\rightarrow 2\tilde{\sigma })= \frac{\lambda _6^2|\langle \phi \rangle |^2}{16\pi m_{\tilde{h}}}\sqrt{1-4\frac{M_{\tilde{\sigma }}^2}{m_{\tilde{h}}^2}}. \end{aligned}$$

(14)

The branching ratio of this invisible decay should be less than $19\,\%$ of the Higgs total width $\sim$ $4$ MeV [58]. This constrains the value of $\lambda _6$ as $|\lambda _6|<0.0126$ [59], which could be consistent with the vacuum condition discussed above. Here, we note that both $\kappa$ and $\lambda _6$ take small values for the light $\tilde{\sigma }$. In that case, $\tilde{\sigma }$ could have non-negligible cosmological effects. We will come back to this point later.

2.3 Degenerate right-handed neutrinos

Next, we discuss the neutrino sector. If the thermal leptogenesis at TeV scales is supposed to be the origin of baryon number asymmetry in the Universe, the mass degeneracy among right-handed neutrinos is indispensable, at least in certain parameter regions [31, 32]. In the present model, spontaneous breaking of a new Abelian gauge symmetry due to a vacuum expectation value of S could make the singlet fermions $N_i$ and $\tilde{N}_i$ behave as pseudo-Dirac fermions. In fact, if $|y_i\langle S^\dagger \rangle |, |\tilde{y}_i\langle S\rangle |\ll M_i$ is satisfied, their masses are almost degenerate.³

The mass matrix of the singlet fermions is expressed as

$$\begin{aligned} \frac{1}{2}(N_i, \tilde{N_i})\left( \begin{array}{cc} |y_i|e^{i\gamma _i}\langle S^\dagger \rangle &{} M_i \\ M_i &{} |\tilde{y_i}|e^{i\tilde{\gamma }_i} \langle S\rangle \\ \end{array}\right) \left( \begin{array}{c} N_i \\ \tilde{N_i}\\ \end{array}\right) +\mathrm{h.c.}, \end{aligned}$$

(15)

where $M_i$ and $\langle S\rangle$ can be taken to be positive generally. The mass eigenvalues $M_{\pm i}$ are derived as

$$\begin{aligned} \begin{aligned} M_{+i}\simeq&\, M_i\sin 2\theta _i+(|y_i|\cos (\gamma _i-\xi _i)\cos ^2\theta _i\\&+\,|\tilde{y}_i|\cos (\tilde{\gamma }_i+\xi _i)\sin ^2\theta _i) \langle S\rangle , \\ M_{-i}\simeq&\, M_i\sin 2\theta _i -(|y_i|\cos (\gamma _i-\xi _i)\sin ^2\theta _i \\&+\,|\tilde{y}_i|\cos (\tilde{\gamma }_i+\xi _i)\cos ^2\theta _i) \langle S\rangle , \end{aligned} \end{aligned}$$

(16)

and the corresponding mass eigenstates $\mathcal{N}_{\pm i}$ are found to enable us to write

$$\begin{aligned}&\mathcal{N}_{+i}=e^{-i\frac{\xi _i}{2}} (N_i\cos \theta _i + \tilde{N_i}e^{-i\xi _i}\sin \theta _i), \nonumber \\&\mathcal{N}_{-i}=ie^{-i\frac{\xi _i}{2}} (-N_i\sin \theta _i + \tilde{N_i}e^{-i\xi _i}\cos \theta _i), \end{aligned}$$

(17)

respectively. Here, the phase $\xi _i$ is fixed by the parameters in the mass matrix as

$$\begin{aligned} \tan \xi _i=\frac{|y_i|\sin \gamma _i-|\tilde{y}_i|\sin \tilde{\gamma }_i}{|y_i|\cos \gamma _i+|\tilde{y}_i|\cos \tilde{\gamma }_i}, \end{aligned}$$

(18)

and the mixing angle $\theta _i$ is given by using this $\xi _i$ as

$$\begin{aligned} \tan 2\theta _i=\frac{M_i}{\langle S\rangle } \frac{2}{|y_i|\cos (\gamma _i-\xi _i) -|\tilde{y_i}|\cos (\tilde{\gamma }_i+\xi _i)}. \end{aligned}$$

(19)

The difference of the mass eigenvalues given in Eq. (16) is expressed by using these, thus:

$$\begin{aligned} \Delta _i\equiv \frac{M_{+i}-M_{-i}}{M_{-i}}\simeq \frac{\langle S\rangle }{M_i} \frac{|y_i|\cos (\gamma _i-\xi _i)+|\tilde{y}_i|\cos (\tilde{\gamma }_i +\xi _i)}{\sin 2\theta _i}. \end{aligned}$$

(20)

From these formulas, we find that $\theta _i$ could be approximated as $\frac{\pi }{4}$ and also the right-handed neutrino masses might be finely degenerate at a period where the sphaleron interaction is in thermal equilibrium, simultaneously. The condition required for this is that both $|y_i|\langle S\rangle$ and $|\tilde{y}_i|\langle S\rangle$ are much smaller than $M_i$ which is assumed to be of O(1) TeV. This implies that the resonant leptogenesis could occur for a value of $\langle S\rangle$ which is larger than the weak scale as long as both $|y_i|$ and $|\tilde{y}_i|$ are sufficiently small.

The neutrino Yukawa couplings and other relevant interactions of the right-handed neutrinos in Eq. (1) can be written by using the mass eigenstates $\mathcal{N}_{\pm i}$, thus

$$\begin{aligned}&\sum _{i=1,2}\Bigg [e^{-i\frac{\xi _i}{2}} \Big (h_{\alpha i}\cos \theta _i +\tilde{h}_{\alpha i}e^{-i\xi _i}\sin \theta _i\Big ) \bar{\mathcal{N}}_{+i}\eta ^\dagger \ell _\alpha \nonumber \\&-ie^{-i\frac{\xi _i}{2}}\Big (h_{\alpha i}\sin \theta _i -\tilde{h}_{\alpha i}e^{-i\xi _i}\cos \theta _i\Big )\bar{\mathcal{N}}_{-i}\eta ^\dagger \ell _\alpha \nonumber \\&+\,\frac{1}{2\sqrt{2}}\Bigg \{ \left( |y_i|e^{i(\gamma _i+\xi _i)}\cos ^2\theta _i+|\tilde{y}_i| e^{i(\tilde{\gamma }_i+ 3\xi _i)}\sin ^2\theta _i\right) \tilde{\sigma }\mathcal{N}_{+i}^2 \nonumber \\&-\left( |y_i|e^{i(\gamma _i+\xi _i)}\sin ^2\theta _i+|\tilde{y}_i| e^{i(\tilde{\gamma }_i +3\xi _i)}\cos ^2\theta _i\right) \tilde{\sigma }\mathcal{N}_{-i}^2 \nonumber \\&+i\sin 2\theta _i\left( |y_i|e^{i(\gamma _i+\xi _i)}-|\tilde{y}_i| e^{i(\tilde{\gamma }_i +3\xi _i)}\right) \tilde{\sigma }\mathcal{N}_{+i}\mathcal{N}_{-i} \Bigg \} \nonumber \\&+ ig_X\sin 2\theta _i~X_\mu (\bar{\mathcal{N}}_{+i} \gamma ^\mu \mathcal{N}_{-i}) + \mathrm{h.c.}~ \Bigg ]. \end{aligned}$$

(21)

If $h_{\alpha i}=\tilde{h}_{\alpha i}$ is satisfied,⁴ the flavor structure of the model becomes very simple. In that case, the neutrino Yukawa couplings can be rewritten as

$$\begin{aligned} g_{\alpha i}^{(+)}\equiv & {} e^{-i\frac{\xi _i}{2}} h_{\alpha i}(\cos \theta _i +e^{-i\xi _i}\sin \theta _i) \nonumber \\= & {} h_{\alpha i}\left( 1+\cos \xi _i\sin 2\theta _i\right) ^{\frac{1}{2}} e^{i(\delta _{+i}-\frac{\xi _i}{2})}, \nonumber \\ g_{\alpha i}^{(-)}\equiv & {} -ie^{-i\frac{\xi _i}{2}}h_{\alpha i} (\sin \theta _i- e^{-i\xi _i}\cos \theta _i) \nonumber \\= & {} h_{\alpha i}\left( 1-\cos \xi _i\sin 2\theta _i\right) ^{\frac{1}{2}} e^{i(\delta _{-i}-\frac{\xi _i}{2})}, \end{aligned}$$

(22)

where we suppose $h_{\alpha i}$ to be real, for simplicity. The phases $\delta _{\pm i}$ are defined as

$$\begin{aligned} \tan \delta _{+ i}=\frac{-\sin \xi _i\tan \theta _i}{1+\cos \xi _i\tan \theta _i}, \quad \cot \delta _{-i}=\frac{\sin \xi _i}{\cos \xi _i-\tan \theta _i}. \end{aligned}$$

(23)

We use these simplified neutrino Yukawa couplings in the following discussion.

The neutrino mass is induced through one-loop diagrams which have $\mathcal{N}_{+i}$ or $\mathcal{N}_{-i}$ in an internal fermion line as in the original model. The mass formula is given by

$$\begin{aligned} \mathcal{M}_{\alpha \beta } =\sum _{i}\sum _{s=\pm }|g_{\alpha i}^{(s)}g_{\beta i}^{(s)}\lambda _5| e^{i(2\delta _{si}-\xi _i)}\Lambda (M_{s i}), \end{aligned}$$

(24)

where $\Lambda (M_{\pm i})$ is defined as

$$\begin{aligned} \Lambda (M_{\pm i})=\frac{\langle \phi \rangle ^2}{8\pi ^2} \frac{M_{\pm i}}{M_\eta ^2-M_{\pm i}^2} \left( 1+\frac{M_{\pm i}^2}{M_\eta ^2-M_{\pm i}^2} \ln \frac{M_{\pm i}^2}{M_\eta ^2}\right) . \end{aligned}$$

(25)

$M_\eta$ is an averaged value of the mass eigenvalues of $\eta _R$ and $\eta _I$. If the model has two sets of $(N_i, \tilde{N}_i)$ at least, neutrino mass eigenvalues suitable for the explanation of the neutrino oscillation data could be derived.⁵ We consider a model with two sets of $(N_i, \tilde{N}_i)$ in the following.

Since the scale $\Lambda (M_{\pm i})$ is estimated as $\Lambda (M_{\pm i})=O(10^9)$ eV for $\eta$ and $\mathcal{N}_{\pm i}$ whose masses are in the TeV range, Eq. (24) suggests that the atmospheric neutrino data require the relevant neutrino Yukawa couplings to satisfy

$$\begin{aligned} \sum _i |g_{\alpha i}^{(\pm )}g_{\beta i}^{(\pm )}\lambda _5| =O(10^{-11}). \end{aligned}$$

(26)

On the other hand, if $\mathcal{N}_{-1}$ is identified with the lightest right-handed neutrino, its decay should occur in out-of-thermal equilibrium for successful leptogenesis. This condition could impose strong constraints on various interactions of $\mathcal{N}_{-1}$. They can be roughly estimated by imposing both reaction rates of the decay of $\mathcal{N}_{-1}$ and its scattering with other particles to be smaller than the Hubble parameter. The most important process is the $\mathcal{N}_{-1}$ decay. If the neutrino Yukawa couplings of $\mathcal{N}_{-1}$ satisfy

$$\begin{aligned} \left( \sum _\alpha \left| g_{\alpha 1}^{(-)}\right| ^2\right) ^{\frac{1}{2}} \le 10^{-8}, \end{aligned}$$

(27)

it does not reach equilibrium at the temperature $T~{^>_\sim }~100$ GeV.

The condition (27) shows that $\mathcal{N}_{-1}$ causes a negligible contribution to the neutrino mass generation, which is found from Eqs. (24) and (26). On the other hand, if $\mathcal{N}_{+1}$ is supposed to cause a main contribution to the neutrino mass generation, the condition (26) shows that its Yukawa couplings should satisfy

$$\begin{aligned} |g_{\alpha 1}^{(+)}|^2= O\left( \frac{10^{-11}}{|\lambda _5|}\right) \quad (\alpha =e,\mu ,\tau ). \end{aligned}$$

(28)

Equation (22) suggests that the original neutrino Yukawa couplings $|h_{\alpha 1}|$ do not need to be extremely small for the simultaneous realization of the conditions (27) and (28), as long as $\cos \xi _1\sin 2\theta _1\simeq 1$ is satisfied to a good accuracy and also $|\lambda _5|$ takes a small value of $O(10^{-4})$. Other nonzero neutrino mass eigenvalues could be determined through the second pair $(N_2,\tilde{N}_2)$. Since the relevant Yukawa couplings $h_{\alpha 2}$ are not constrained by the leptogenesis, we can derive neutrino masses and mixing favorable for the explanation of the neutrino oscillation data through Eq. (24) independently. If only one of $\mathcal{N}_{\pm 2}$ contributes to the neutrino mass generation as in the $(N_1, \tilde{N}_1)$ sector, one of three neutrino mass eigenvalues is expected to be negligibly small as in the model studied in [31, 32].

2.4 Resonant leptogenesis

In this framework, we consider resonant leptogenesis [37–44]. The dominant contribution to the CP asymmetry $\varepsilon$ in the $\mathcal{N}_{-1}$ decay comes from the resonance appearing in the one-loop self-energy diagram. In that case, $\varepsilon$ is known to enable us to express it as [41–44]

$$\begin{aligned} \varepsilon= & {} \frac{\mathrm{Im} \left( \sum _\alpha g_{\alpha 1}^{(+)*} g_{\alpha 1}^{(-)}\right) ^2}{\left( \sum _\alpha g_{\alpha 1}^{(-)*} g_{\alpha 1}^{(-)}\right) \left( \sum _\alpha g_{\alpha 1}^{(+)*}g_{\alpha 1}^{(+)}\right) }~ \frac{2\Delta _1{\tilde{\Gamma }_{\mathcal{N}_{+1}}}}{4\Delta _1^2+\tilde{\Gamma }_{\mathcal{N}_{+1}}^2} \nonumber \\= & {} \frac{\cos 2\theta _1\sin 2\xi _1}{1-\sin ^22\theta _1\cos ^2\xi _1} \frac{2\Delta _1{\tilde{\Gamma }_{\mathcal{N}_{+1}}}}{4\Delta _1^2+\tilde{\Gamma }_{\mathcal{N}_{+1}}^2}, \end{aligned}$$

(29)

where we use the expression of the neutrino Yukawa couplings $|g_{\alpha 1}^{\pm }|$ given in Eq. (22). The mass degeneracy $\Delta _1$ is defined in Eq. (20) and $\tilde{\Gamma }_{\mathcal{N}_{+1}}= \frac{\sum _\alpha \left| g_{\alpha 1}^{(+)}\right| ^2}{8\pi }\left( 1-\frac{M_\eta ^2}{M_{+1}^2}\right) ^2$. If we assume $\langle S\rangle =M_1$ for simplicity, the right-handed neutrino sector $(N_1,\tilde{N}_1)$ has five free parameters. Using these, we study the relation between the CP asymmetry and the structure of right-handed neutrino sector.

Fig. 1

CP asymmetry as a function of $\gamma _1$ for typical values of $(|y_1|, |h_{\alpha 1}|)$. In each case, these parameters are fixed as A$(10^{-5}, 4\times 10^{-4})$, B$(10^{-5}, 5\times 10^{-4})$, C$(2\times 10^{-5}, 4\times 10^{-4})$, and D$(2\times 10^{-5}, 5\times 10^{-4})$. Other relevant parameters are taken to be $\tilde{\gamma }_1=0.1$, $\tilde{y}_1=10^{-8}$, $M_1=\langle S\rangle =2$ TeV and $M_\eta =1$ TeV

Table 1 Derived values of the quantities relevant to the leptogenesis for each case given in Fig. 1. These are estimated at $\gamma _1\sim 9 \times 10^{-5}$ and $4\times 10^{-5}$ for the cases A, B and C, D, respectively

In Fig. 1, we plot the CP asymmetry $\varepsilon$ as a function of $\gamma _1$ for four typical sets of $(|y_1|, |h_{\alpha 1}|)$. Other parameters are fixed at the values given in the caption of Fig. 1. We find that $\varepsilon$ changes the sign from minus to plus at $\gamma _1\sim 10^{-4}$ and $5\times 10^{-5}$ for the cases A, B and C, D, respectively. Its absolute value is enhanced largely around these values of $\gamma _1$. If we note that $|g_{\alpha 1}^{(-)}|\le O(10^{-8})$ is required for the out-of-equilibrium decay of $\mathcal{N}_{-1}$, we find that $|\xi _1|$ should take a very small value such as $O(10^{-4})$ for $|h_{\alpha 1}|=O(10^{-4})$. As found from Eq. (18), such a small $|\xi _1|$ could be easily realized for hierarchical $|y_1|$ and $|\tilde{y}_1|$ by fixing the values of $\gamma _1$ and $\tilde{\gamma }_1$ appropriately. In these examples, such hierarchical values are assumed for $|y_1|$ and $|\tilde{y}_1|$. We also note that the same parameter set could induce the degenerate right-handed neutrino masses as found from Eq. (20). This feature makes it for the model possible to satisfy the minimum conditions for the success of resonant leptogenesis. Although we have to introduce a tiny coupling $|\tilde{y}_1|$ in this scenario, the important quantities for the leptogenesis are closely related each other. The model can bring about their favorable values simultaneously based on the common parameters. In fact, for the parameters used in Fig. 1, the desirable values of the relevant quantities to the leptogenesis can be obtained. We present their values derived from these parameters in Table 1. These results show that $|g_{\alpha 1}^{(-)}|$ takes small values which satisfy the condition (28) at the points where the CP asymmetry $|\varepsilon |$ has large values. The mass degeneracy $\Delta _1=O(10^{-5})$ between the right-handed neutrinos $\mathcal{N}_{\pm 1}$ is also realized at this region. This level of degeneracy has been shown to be sufficient for the leptogenesis in the radiative neutrino mass model in the previous study [31, 32]. Although the smallness of $|\tilde{y}_1|$ should be explained by considering some complete model in the high energy region, it is beyond the scope of the present study and we do not go further in this direction here.

The baryon number asymmetry generated through the decay of $\mathcal{N}_{-1}$ can be fixed by estimating the generated lepton number asymmetry through solving the Boltzmann equations numerically for both the $\mathcal{N}_{-1}$ number density $n_{\mathcal{N}_{-1}}$ and the lepton number asymmetry $n_L({\equiv } n_\ell -n_{\bar{\ell }})$. We introduce these number densities in the co-moving volume as $Y_{\mathcal{N}_{-1}}=\frac{n_{\mathcal{N}_{-1}}}{s}$ and $Y_L=\frac{n_L}{s}$ by using the entropy density s. The Boltzmann equations for these are written as

$$\begin{aligned} \begin{aligned} \frac{\mathrm{d}Y_{\mathcal{N}_{-1}}}{\mathrm{d}z}=&\,-\frac{z}{sH(M_{-1})} \left( \frac{Y_{\mathcal{N}_{-1}}}{Y_{\mathcal{N}_{-1}}^\mathrm{eq}}-1\right) \\&\times \{ \gamma ^D_{\mathcal{N}_{-1}}+ \gamma _{{\mathcal{N}_{-1}} {\tilde{\sigma }}}^{S}+ \gamma _{{\mathcal{N}_{-1}}{X}}^{S}\}, \\ \frac{\mathrm{d}Y_L}{\mathrm{d}z}=&\,\frac{z}{sH(M_{-1})}\left\{ \varepsilon \left( \frac{Y_{\mathcal{N}_{-1}}}{Y_{\mathcal{N}_{-1}}^\mathrm{eq}}-1\right) \gamma ^D_{\mathcal{N}_{-1}}-\frac{2Y_L}{Y_\ell ^\mathrm{eq}}\right. \\&\times \left. \left( \frac{\gamma ^D_{\mathcal{N}_{+1}}}{4} +\gamma _{\mathcal{N}_{+1}}^{(2)} +\gamma _{\mathcal{N}_{+1}}^{(13)}\right) \right\} , \end{aligned} \end{aligned}$$

(30)

where $z=\frac{M_{-1}}{T}$ and $H(M_{-1})=1.66g_*^{1/2}\frac{M_{-1}^2}{m_\mathrm{pl}}$. The equilibrium values for these are expressed as $Y_{\mathcal{N}_{-1}}^\mathrm{eq}(z)=\frac{45}{2\pi ^4g_*}z^2K_2(z)$ and $Y_\ell ^\mathrm{eq}\simeq \frac{81}{\pi ^4g_*}$, where $K_2(z)$ is the modified Bessel function of the second kind. Since the Yukawa couplings of $\mathcal{N}_{+1}$ are large enough, it is expected to be in thermal equilibrium throughout the relevant period. In these equations, we take into account the important reactions which could keep $\mathcal{N}_{-1}$ in the equilibrium and wash out the generated lepton number asymmetry. The former ones include the 2–2 scatterings of $\mathcal{N}_{-1}$ with $\tilde{\sigma }$ and $X_\mu$, whose reaction densities are represented by $\gamma ^S_{\mathcal{N}_{-1}\tilde{\sigma }}$ and $\gamma ^S_{\mathcal{N}_{-1}X}$ in Eq. (30). These could be effective if $\tilde{\sigma }$ and $X_\mu$ are light enough. Other reaction densities in Eq. (30) can be found in the appendix of [31, 32].

Fig. 2

In the upper panels, solutions of the Boltzmann equations are plotted as a function of z for the case A shown in Table 1. In the lower panels, relevant reaction rates $\Gamma /H$ are plotted as a function of z for the same parameters used in the corresponding upper panels. Reaction rates of the $\mathcal{N}_{-1}$ decay, the $\mathcal{N}_{+1}$ inverse decay, and the lepton number violating $\mathcal{N}_{+1}$ scatterings are represented by $\Gamma _{N_{-1}}^D$, $\Gamma _{N_{+1}}^{ID}$ and $\Gamma _{N_{+1}}^{(2)}$, $\Gamma _{N_{+1}}^{(13)}$, respectively. The masses of $\tilde{\sigma }$ and $X_\mu$ are set as $(M_{\tilde{\sigma }},m_X)=(200,300)$, (60, 100), and $(200,10^{-3})$ in GeV units from left to right, respectively

In Fig. 2, the solutions of these equations and the reaction rates $\Gamma$ of the relevant processes are plotted as functions of z for the case A in Table 1. In these panels, the masses of $\tilde{\sigma }$ and $X_\mu$ are fixed to be $(M_{\tilde{\sigma }}, m_X)=(200,300)$, (60, 100), and $(200, 10^{-3})$ in GeV units, respectively. As the initial condition for $Y_{\mathcal{N}_{-1}}$ in the Boltzmann equations we use its equilibrium value, since both $N_1$ and $\tilde{N}_1$ are expected to be in thermal equilibrium. Since we adopt this initial condition, its deviation $\Delta _{\mathcal{N}_{-1}}$ from the equilibrium value does not change sign as found in the upper panels of this figure. After $\langle S\rangle$ becomes nonzero, the mass eigenstate $\mathcal{N}_{-1}$ leaves the equilibrium because of its small Yukawa coupling $g_{\alpha 1}^{(-)}$. Thus, it could be crucial in the estimation of the lepton number asymmetry at what value of z we introduce the effect of nonzero $\langle S\rangle$ in the equations. As a simple approximation, we introduce its effect as a step function at $z_0$. In order to check the validity of this analysis, we change the value of $z_0$ in the range $0.3<z_0<1$ to examine the $z_0$ dependence of the final results. Since their difference stays at most in a few 10 % range without showing a serious $z_0$ dependence, the present treatment can be considered to give reliable results.

In the lower panels, which plot the behavior of the reaction rates, we find that the inverse decay of $\mathcal{N}_{+1}$ plays a dominant role for the wash-out of the generated lepton number asymmetry among various processes. Although the $\mathcal{N}_{+1}$ mass is almost degenerate with the mass of $\mathcal{N}_{-1}$, its Yukawa coupling $g_{\alpha 1}^{(+)}$ is not so small as to decouple at an earlier period. This is an expected feature in the resonant leptogenesis generally. The rapid increase of the lepton number asymmetry shown in the $z>10$ region can be understood from the large decrease of $\Gamma _{N_{+1}}^{ID}$ there. The scatterings of $\mathcal{N}_{-1}$ with $\tilde{\sigma }$ and $X_\mu$ cannot be effective in keeping $\mathcal{N}_{-1}$ in thermal equilibrium even if $\tilde{\sigma }$ and $X_\mu$ are light enough. Since $\langle S\rangle$ is supposed to be rather large, the assumed masses for $\tilde{\sigma }$ and $X_\mu$ are obtained only for the small couplings $\kappa$ and $g_X$. This is considered to be the cause of these results.

Table 2 Baryon number asymmetry $Y_B$ predicted for the parameter sets given in Table 1. $M_{\tilde{\sigma }}$ and $m_X$ are given in GeV units

The baryon number asymmetry $Y_B(\equiv \frac{n_B}{s})$ is expressed by using the solution $Y_L$ of the Boltzmann equations, thus

$$\begin{aligned} Y_B=-\frac{8}{23}Y_L(z_\mathrm{EW}), \end{aligned}$$

(31)

where $z_\mathrm{EW}$ is related to the sphaleron decoupling temperature $T_\mathrm{EW}$ by $z_\mathrm{EW}=\frac{M_{-1}}{T_\mathrm{EW}}$. The baryon number asymmetry predicted for the parameters given in Table 1 is listed in Table 2 for several values of $(M_{\tilde{\sigma }},m_X)$. These results show that the model could generate the sufficient baryon number asymmetry compared with $8.1\times 10^{-11}<Y_B< 9.2\times 10^{-11}~(95\,\%\, \mathrm{CL})$ required from the observation [45] as long as the relevant parameters take suitable values.⁶ We note that the light $\tilde{\sigma }$ which can contribute to the invisible decay of the Higgs particle $\tilde{h}$ is also allowed from the view point of the generation of baryon number asymmetry.

The condition (26) imposed by the neutrino oscillation data requires $|\lambda _5|=O(10^{-4})$ for the above numerical results. As we will see in the next section, it is consistent with the constraint derived from the dark matter direct search. The values of $\lambda _5$ and $\tilde{h}_{\alpha 1}$ used in the above study are found to be realized through Eq. (2) for the cut-off scale such as $M_*=O(10^4)$ TeV, since we assume $\langle S\rangle =M_1$ here. Even if we do not assume this relation and $\langle S\rangle$ is supposed to have a larger value, a similar result is expected to be obtained for a larger value of $M_*$ and smaller values of $|y_i|$ and $|\tilde{y}_i|$.

3 Physics in dark sector

3.1 Relic abundance and detection of dark matter

It is well known that there are three possible mass ranges for an inert doublet dark matter to realize the required relic abundance [47–55]. We are considering the high mass possibility here.⁷ The $\eta _R$ relic abundance can be estimated along the same lines as the original model [31, 32, 55]. However, we have to take into account that the thermally averaged (co)annihilation cross section $\langle \sigma _\mathrm{eff}v\rangle$ has additional contributions from the processes which have $X_\mu$ or $\tilde{\sigma }$ in the final states or intermediate states in the present model. Moreover, for the inert doublet dark matter $\eta _R$, the direct search imposes severe constraints on the scalar couplings $\lambda _i$.

First, we consider the constraint induced through inelastic scattering of $\eta _R$ with a nucleus. Since the masses of $\eta _R$ and $\eta _I$ are almost degenerate for the small values of $|\lambda _5|$ as found from Eq. (6), this inelastic scattering of $\eta _R$ mediated by the $Z^0$ exchange brings about substantial effects to the direct search experiments [62–65]. The interaction of $\eta _{R}$ relevant to this process is given by

$$\begin{aligned} \mathcal{L}=\frac{g}{2\cos \theta _W}Z^\mu \left( \eta _R\partial _\mu \eta _I - \eta _I\partial _\mu \eta _R\right) . \end{aligned}$$

(32)

Inelastic $\eta _R$-nucleus scattering can occur for $\eta _R$ whose velocity is larger than the minimum value [66] given by

$$\begin{aligned} v_\mathrm{min}=\frac{1}{\sqrt{2m_NE_R}}\left( \frac{m_NE_R}{\mu _N} +\delta \right) , \end{aligned}$$

(33)

where $\delta$ is the mass difference between $\eta _R$ and $\eta _I$ defined in Eq. (6). $E_R$ is the nucleus recoil energy. The mass of the target nucleus and the reduced mass of the nucleus-$\eta _R$ system are represented by $m_N$ and $\mu _N$. The mass difference $\delta$ is constrained by the fact that no dark matter signal has been found in the direct search yet [67–71]. This condition might be estimated as $\delta ~{^>_\sim }~150$ keV [64, 65]. Since $\delta$ is related to $\lambda _5$ through Eq. (6), the condition on $\delta$ constrains the value of $|\lambda _5|$ to satisfy [31, 32]

$$\begin{aligned} |\lambda _5|\simeq \frac{M_{\eta _R}\delta }{\langle \phi \rangle ^2} ~{^>_\sim }~5.0\times 10^{-6} \left( \frac{M_{\eta _R}}{1~\mathrm{TeV}}\right) \left( \frac{\delta }{150~\mathrm{keV}}\right) . \end{aligned}$$

(34)

Since $\tilde{\lambda }_5=O(1)$ is expected, Eq. (2) suggests that $\langle S\rangle ~{^>_\sim }~5\times 10^{-6}M_*$ should be satisfied.

The present results from a dark matter direct search also impose a constraint on the values of the scalar couplings $\lambda _{3,4}$ and $\lambda _6$. The $\eta _R$–nucleus elastic scattering is induced through the exchange of $\tilde{h}$ and $\tilde{\sigma }$. The corresponding cross section for $\eta _R$–nucleon scattering at zero momentum transfer can be calculated to be

$$\begin{aligned} \sigma _n^0=\frac{f^{(n)2}m_n^4\lambda _+^2}{8\pi M_{\eta _R}^2m_{\tilde{h}}^4} \left( 1 +\frac{\lambda _6^2}{4\kappa \lambda _1}\right) ^2, \end{aligned}$$

(35)

where $m_n$ is a nucleon mass and $f^{(n)}\simeq 0.3$. The second term in the parentheses comes from the $\tilde{\sigma }$ exchange. If we apply the present direct search constraint $\sigma _n^0<1\times 10^{-44}~\mathrm{cm}^2$ for $M_{\eta _R}=O(1)$ TeV [69], we find that the scalar couplings $\lambda _{3,4}$ should satisfy

$$\begin{aligned} \lambda _+\left( 1+\frac{\lambda _6^2}{4\kappa \lambda _1}\right) <1.5\left( \frac{M_{\eta _R}}{1~\mathrm{TeV}}\right) , \end{aligned}$$

(36)

where $\lambda _+\simeq \lambda _3+\lambda _4$. Since the potential stability requires $\lambda _6^2<4\kappa \lambda _1$ as seen before, the $\tilde{\sigma }$ exchange contribution to the $\eta _R$–nucleon scattering can be generally neglected except for the case where $\lambda _6^2$ takes the same value as regards the order, $4\kappa \lambda _1$.

Fig. 3

Relic abundance of $\eta _R$ in the case of the existence of new interactions. It is plotted as a function of $\lambda _4$ for typical sets of $(|\lambda _7|,\lambda _3)$. In the left and right panels, $\lambda _3$ is assumed to be negative and positive, respectively. A horizontal dashed line stands for the observed value $\Omega _{\eta _R}h^2=0.12$ [13, 14]. In this plot, $g_X=0.1g_Y$ and $\lambda _5=-10^{-4}$ are assumed

We now proceed to the estimation of the $\eta _R$ relic abundance taking account of the conditions discussed above. We use the notation $(\eta _1,\eta _2,\eta _3,\eta _4)=(\eta _R,\eta _I,\eta _+,\eta _-)$ for convenience here. The dominant parts of the effective (co)annihilation cross section including the new contributions are calculated to be

$$\begin{aligned}&\langle \sigma _\mathrm{eff}v\rangle \nonumber \\&\simeq \frac{1}{128\pi M_{\eta _1}^2} \left( \frac{g_2^4(1+2\cos ^4\theta _W)}{\cos ^4\theta _W} + \frac{2g_2^2g_X^2}{\cos ^2\theta _W} + g_X^4\right) \nonumber \\&\quad \times \left( N_{11}+N_{22}+ 2N_{34}\right) \nonumber \\&\quad +\frac{1}{32\pi M_{\eta _1}^2} \left( \frac{g_2^4\sin ^2\theta _W}{\cos ^2\theta _W} + g_2^2g_X^2\right) \left( N_{13}+N_{14}+N_{23}+ N_{24}\right) \nonumber \\&\quad +\frac{1}{64\pi M_{\eta _1}^2} [\{\lambda _+^2+\lambda _-^2+2(\lambda _3^2+\lambda _7^2)\} (N_{11}+N_{22}) \nonumber \\&\quad +(\lambda _+-\lambda _-)^2(N_{33}+N_{44}+ N_{12}) +\{(\lambda _+ +\lambda _-)^2 +4\lambda _3^2+ 2\lambda _7^2 \}N_{34} \nonumber \\&\quad +\{(\lambda _+-\lambda _3)^2+ (\lambda _- -\lambda _3)^2\} (N_{13}+N_{14}+N_{23}+N_{24})], \end{aligned}$$

(37)

where $g_X$ is assumed to be much smaller than $g_Y$ and then $X_\mu$ is sufficiently lighter than the dark matter $\eta _R$. $N_{ij}$ is defined by using $g_\mathrm{eff}=\sum _i \frac{n^\mathrm{eq}_i}{n_1^\mathrm{eq}}$,

$$\begin{aligned} N_{ij}&\equiv \frac{1}{g_\mathrm{eff}^2} \frac{n_i^\mathrm{eq}}{n_1^\mathrm{eq}}\frac{n_j^\mathrm{eq}}{n_1^\mathrm{eq}} =\frac{1}{g_\mathrm{eff}^2} \left( \frac{M_{\eta _i}M_{\eta _j}}{M_{\eta _1}^2}\right) ^{\frac{3}{2}}\nonumber \\&\quad \times \exp \left[ -\frac{M_{\eta _i}+M_{\eta _j}-2M_{\eta _1}}{T}\right] , \end{aligned}$$

(38)

where $n_i$ is the $\eta _i$ number density and $n_i^\mathrm{eq}$ is its equilibrium value. In order to estimate the relic abundance of $\eta _R$, we use the well-known analytic formula instead of solving the Boltzmann equation numerically. The formula is given by [72, 73]

$$\begin{aligned} \Omega _{\eta _1}h^2\simeq \frac{1.07\times 10^9~\mathrm{GeV}^{-1}}{J(x_F) g_*^{\frac{1}{2}}m_\mathrm{pl}}, \end{aligned}$$

(39)

where $g_*$ is for the relativistic degrees of freedom. The freeze-out temperature $T_F(\equiv \frac{M_{\eta _1}}{x_F})$ and $J(x_F)$ are defined as

$$\begin{aligned}&x_F=\ln \frac{0.038~ m_\mathrm{pl}~ g_\mathrm{eff}~ M_\mathrm{\eta _1}\langle \sigma _\mathrm{eff}v\rangle }{(g_*x_F)^{\frac{1}{2}}}, \nonumber \\&J(x_F)=\int ^\infty _{x_F} \frac{\langle \sigma _\mathrm{eff}v\rangle }{x^2}\mathrm{d}x. \end{aligned}$$

(40)

In Fig. 3 we show the predicted relic abundance of $\eta _R$ when the new interactions are taken into account. It is plotted as a function of $\lambda _4$ by assuming typical values of $(|\lambda _7|,\lambda _3)$. To plot this figure, we assume a small value for $g_X$, such as $0.1g_Y$, and we fix the value of $m_\eta ^2+\lambda _7\langle S\rangle ^2$ at 1 $\mathrm{TeV}^2$ for $\langle S\rangle =2$ TeV. Thus, the mass of $X_\mu$ is comparable to the one of the weak bosons and $\lambda _7$ is confined to $|\lambda _7| < 0.25$. The figure shows that the above cross section can explain the required dark matter relic abundance for a wide range of values of $\lambda _{3,4}$. Since the additional (co)annihilation decay processes can generate substantial contributions for a larger $|\lambda _7|$ in this extended model, $|\lambda _3|$ and $|\lambda _4|$ could take much smaller values in comparison with the values required in the original model [31, 32]. From the view point of dark matter search, however, the small $|\lambda _7|$ may be promising as suggested through Eq. (35). Since larger values of $|\lambda _{3,4}|$ are required by the relic abundance in this case, the $\eta _R$ dark matter could be found in the Xenon1T direct search as discussed in [31, 32]. On the other hand, it might be difficult to detect even in the Xenon1T experiment in the case of a large $|\lambda _7|$.

3.2 Cosmological signal

In this model, the main phenomenological difference from the original Ma model is the existence of the neutral scalar $\tilde{\sigma }$ and the neutral gauge boson $X_\mu$.⁸ They have no direct interaction with the contents of the standard model except for the one caused by the $\lambda _6S^\dagger S\phi ^\dagger \phi$ term. If $\tilde{\sigma }$ is light enough, it induces the Higgs invisible decay through this term as discussed already. Even in that case, if $\lambda _6$ satisfies the required condition, the model is consistent with the present data obtained from collider experiments. Moreover, we find no substantial constraint on the masses of $\tilde{\sigma }$ and $X_\mu$ from the study of the baryon number asymmetry in the previous section at least for the assumed value of $\langle S\rangle$. On the other hand, these new particles could bring about some crucial influence to the thermal history of the Universe depending on their masses.

First of all, we consider the case where $X_\mu$ is heavier than $\tilde{\sigma }$ and then $g_X^2>2\kappa$ is satisfied. The new gauge boson $X_\mu$ couples only with $\tilde{\sigma }$, $\eta$, $N_i$, and $\tilde{N}_i$. Since the latter three are considered to be much heavier than $X_\mu$, $X_\mu$ can decay only to $\gamma \tilde{\sigma }$ and $\ell _\alpha \bar{\ell }_\beta$ through one-loop diagrams with $\eta$ or $N_i$ and $\tilde{N}_i$ in the internal lines. If we take into account that the neutrino Yukawa couplings $h_{\alpha i}$ and $\tilde{h}_{\alpha i}$ should be of $O(10^{-4})$, we find that the dominant contribution to the $X_\mu$ decay comes from the $X_\mu \rightarrow \gamma \tilde{\sigma }$ process. Its decay width can be estimated as

$$\begin{aligned} \Gamma _X\simeq \frac{\alpha _\mathrm{em}\mathcal{F}^2}{288(4\pi )^4} \frac{m_X^5}{M_{\eta _c}^4} \left( 1-\frac{M_{\tilde{\sigma }}^2}{m_X^2}\right) ^3, \end{aligned}$$

(41)

where $\mathcal{F}=\lambda _7-\frac{\lambda _3\lambda _6}{2\lambda _1}$. If we impose the requirement that $\Gamma _X ~{^>_\sim }~H$ is satisfied at the temperature where both the freeze-out of the neutron-to-proton ratio and the neutrino decoupling are completed, $\mathcal{F}$ is found to have a lower bound,

$$\begin{aligned}&|\mathcal{F}|~{^>_\sim }~ 10^{-8} \left( \frac{M_{\eta _c}}{1~\mathrm{TeV}}\right) ^2 \left( \frac{300~\mathrm{GeV}}{m_X}\right) ^{\frac{5}{2}}\nonumber \\&\times \left( \frac{T}{1~\mathrm{MeV}}\right) \left( 1-\frac{M_{\tilde{\sigma }}^2}{m_X^2}\right) ^{-\frac{3}{2}}. \end{aligned}$$

(42)

Using the constraint on $\lambda _{1,6}$ obtained from the Higgs sector phenomenology and the constraint on $\lambda _{3,7}$ required by the dark matter abundance, $|\mathcal{F}|$ is found to take a large value of O(0.1). This suggests that $\Gamma _X>H$ could be satisfied at the period where the photon temperature is about 1 MeV even for $m_X~{^>_\sim }~O(1)$ GeV.

Although the decay product $\tilde{\sigma }$ does not have direct interactions with the standard model contents, it can decay to them through loop effects. Such decay products could affect the cosmological thermal history depending on the time when $\Gamma _{\tilde{\sigma }}\simeq H$ is realized. Since the neutrino Yukawa couplings should be of $O(10^{-4})$, the $\tilde{\sigma }$ decay is dominated by a two photon final state. It is induced through the one-loop diagram with a charged $\eta$ in the internal line and the decay width can be estimated as

$$\begin{aligned} \Gamma _{\tilde{\sigma }}\simeq \frac{\alpha _\mathrm{em}^2\mathcal{F}^2}{9216 \pi ^3g_X^2}\frac{M_{\tilde{\sigma }}^3m_X^2}{M_{\eta _c}^4}. \end{aligned}$$

(43)

If $g_X<0.88\kappa ^{\frac{3}{10}}$ is satisfied for $g_X^2 > 2\kappa$, $\Gamma _{\tilde{\sigma }}$ is larger than $\Gamma _X$. In such a case, $\tilde{\sigma }$ is expected to decay instantaneously after the $X_\mu$ decay yields it. Since Eq. (42) shows that this $\tilde{\sigma }$ decay occurs at $T>1$ MeV, no cosmological effect is expected.

In the other case, $g_X>0.88\kappa ^{\frac{3}{10}}$, the decay of $\tilde{\sigma }$ occurs with a delay from its production time. If we use the condition $\Gamma _{\tilde{\sigma }} \sim H$ to make a rough estimation of the temperature where the $\tilde{\sigma }$ decay comes in the thermal equilibrium, we have

$$\begin{aligned} T\sim & {} 54 g_*^{-1/4}\left( \frac{|\mathcal{F}|}{10^{-7}}\right) \left( \frac{1~\mathrm{TeV}}{M_{\eta _c}}\right) ^2 \left( \frac{m_X}{300~\mathrm{GeV}}\right) ^{\frac{3}{2}}\nonumber \\&\times \left( \frac{M_{\tilde{\sigma }}}{m_X}\right) ^{\frac{3}{2}} \left( \frac{\langle S\rangle }{2~\mathrm{TeV}}\right) ~\mathrm{MeV}. \end{aligned}$$

(44)

From this result, we find that the $\tilde{\sigma }$ decay could occur before the neutrino decoupling as long as both $|\mathcal{F}|$ and $m_X$ take suitable values for a supposed $M_{\tilde{\sigma }}$. In this case, this decay process does not affect the neutrino effective number in the Universe. For example, the light $X_\mu$ such as $m_X=O(1)$ GeV does not affect it for $|\mathcal{F}|>O(10^{-4})$ as long as $10^{-4}m_X<M_{\tilde{\sigma }}<m_X$ is satisfied.

On the other hand, $\ell _\alpha \bar{\ell }_\beta$ could also be a dominant decay mode of $X_\mu$ for smaller values of $|\mathcal{F}|$ such as $|\mathcal{F}|~{^<_\sim }~ 10^{-7} \frac{g_X}{g_Y} \left( \frac{\bar{h}}{10^{-4}}\right) ^2$. Here, we recall that the averaged value $\bar{h}$ of the relevant neutrino Yukawa couplings $h_{\alpha i}$ is required to be of $O(10^{-4})$ to explain both the neutrino oscillation data and the baryon number asymmetry in the Universe. Such small values of $|\mathcal{F}|$ could be also consistent with the dark matter abundance as long as $\lambda _3$ or $\lambda _4$ is of O(1) and both $|\lambda _6|$ and $|\lambda _7|$ are small enough. In such a case, this decay process could be in thermal equilibrium still after the neutrino decoupling. The neutrinos produced here could contribute to the effective neutrino number as the non-thermal neutrino components. Although this possibility may be interesting from a cosmological view point, a detailed analysis is beyond the scope of this paper.

Finally, we study the case where $X_\mu$ is extremely light and then $\tilde{\sigma }$ is heavier than $X_\mu$. In such a case, the $X_\mu$ decay could cause a cosmological problem generally since its decay mode is limited. The cosmological indication could largely change without affecting other results of the model obtained in the previous part. As an interesting example, we address the situation $m_X<2m_e$ where the gauge coupling $g_X$ becomes unnaturally small.⁹ There, $X_\mu$ can decay only to neutrino–antineutrino pairs through one-loop diagrams. These non-thermally produced neutrinos affect the present effective neutrino number. Its deviation from the standard value $N_\mathrm{eff}=3.046$ may be estimated [75].

The non-thermal neutrinos make the effective neutrino number shift from the standard value by

$$\begin{aligned} \Delta N_\mathrm{eff}(T)=\frac{120}{7\pi ^2}\left( \frac{11}{4}\right) ^{\frac{4}{3}} \frac{\rho _\nu ^\mathrm{nth}(T)}{T^4}, \end{aligned}$$

(45)

where $\rho _\nu ^\mathrm{nth}(T)$ is the energy density of non-thermally produced neutrinos at the photon temperature T. This energy density in the co-moving volume $R^3$ evolves following the differential equation

$$\begin{aligned} \frac{\mathrm{d}(\rho _\nu ^\mathrm{nth}R^3)}{\mathrm{d}t}=\Gamma _X(\rho _XR^3) -H(\rho _\nu ^\mathrm{nth}R^3). \end{aligned}$$

(46)

Assuming radiation domination through this evolution, we can find the solution

$$\begin{aligned} \rho _\nu ^\mathrm{nth}R^3=m_XN_X^f\frac{1}{\sqrt{\Gamma _Xt}}\xi (t), \end{aligned}$$

(47)

where $\xi (t)$ is defined as $\xi (t)=\mathrm{erf}(\sqrt{\Gamma _Xt})-\sqrt{\Gamma _Xt}~e^{-\Gamma _Xt}$ and it is reduced to $\frac{\sqrt{\pi }}{2}$ in the limit $\Gamma _Xt\gg 1$. $N_X^f$ stands for the $X_\mu$ number in the co-moving volume $R^3$ at the freeze-out time of $X_\mu$. Since it could be identified with the freeze-out time of $\eta _R$, $X_\mu$ is relativistic there and then $\frac{N_X^f}{R^3}=\frac{\zeta (3)}{\pi ^2}\mathrm{g}_XT^3$ is satisfied. Using these, we finally obtain the deviation of the effective neutrino number due to the non-thermally produced neutrinos:

$$\begin{aligned} \Delta {N_{{\text {eff}}}}= & {} \frac{{60\sqrt{2} \zeta (3)}}{{7{\pi ^{\frac{7}{2}}}}}{\left( {\frac{{11}}{4}} \right) ^{\frac{4}{3}}}{\left( {\frac{{8{\pi ^3}}}{{90}}} \right) ^{\frac{1}{4}}}{\text {g}}_R^{\frac{1}{4}}{{\text {g}}_X}{m_X}\sqrt{\frac{1}{{{\Gamma _X}{m_{{\text {pl}}}}}}} \nonumber \\\simeq & {} 0.39{{\text {g}}_X}\left( {\frac{{{m_X}}}{{{\text {MeV}}}}} \right) {\left( {\frac{{{{10}^{ - 20}}{\text {MeV}}}}{{{\Gamma _X}}}} \right) ^{\frac{1}{2}}}, \end{aligned}$$

(48)

where $\mathrm{g}_R$ is for the present degrees of freedom of radiation and it can be approximated by the value of the standard model. This result suggests that the decay width of $X_\mu$ should be $\Gamma _X~{^>_\sim }~10^{-20}$ MeV for $X_\mu \rightarrow \nu _\alpha \bar{\nu }_\beta$ or $X_\mu \rightarrow \nu _\alpha \nu _\beta$ in order to satisfy the present observational results [13, 14]. However, since the dominant contribution comes from the latter one, which is induced through a one-loop diagram with the small neutrino Yukawa couplings of $O(10^{-4})$ and also $\lambda _5$ of $O(10^{-4})$, the decay width is much smaller than the required value. It means that the neutrinos produced non-thermally through the decay of $X_\mu$ give a too large contribution to $\Delta N_\mathrm{eff}$. Thus, the model with $m_X<2m_e$ seems to be ruled out by the observed effective neutrino number. If we introduce the kinetic term mixing for $X_\mu$ and $B_\mu$, this problem might be evaded even in such a case. This point is briefly discussed in the appendix.

In the present model, the new $U(1)$ $_X$ symmetry is assumed to be local. Even if this symmetry is supposed to be global, the scenario works well in the same way. However, the reasoning for the pairwise introduction of $N_i$ and $\tilde{N}_i$ is lost in the global U(1) case. The difference between them is whether the massless Nambu–Goldstone boson appears after the breaking of $U(1)$ $_X$ symmetry or not. This boson behaves as dark radiation and changes the effective neutrino number in the Universe just in the same way as discussed in [59].

4 Conclusion

We have considered an extension of the radiative neutrino mass model proposed by Ma with a low energy U(1) gauge symmetry. If we assume a cut-off scale of the model at $O(10^{4})$ TeV and the breaking of this U(1) at a rather low energy scale such as O(1) TeV, several assumptions adopted in the original model to explain the neutrino masses, the dark matter abundance, and the baryon number asymmetry in the Universe could be closely related.

We have shown that the breaking of this U(1) symmetry could give a common background for these assumptions. Both the mass degeneracy among the right-handed neutrinos required for the resonant decay of the lightest right-handed neutrino and its small neutrino Yukawa coupling required for the out-of-equilibrium decay could be explained by the same reasoning through this extension. The $Z_2$ symmetry, which forbids the tree-level neutrino mass generation and guarantees the dark matter stability, has the same origin as the smallness of the quartic coupling constant between the Higgs doublet scalar and the inert doublet scalar, which is an important feature of the model to explain the small neutrino masses. It is useful to recall that these are independent assumptions in the original Ma model. We have also discussed some cosmological issues of the model which appear to be related to this extension. The effective neutrino number could be an interesting subject in this model.

It is interesting that we can have an economical model which could explain the three big problems in the standard model through a simple extension of the Ma model with a low energy U(1) symmetry. A detailed study of the model might give us a clue to the construction of a complete framework beyond the standard model. We will present further results obtained from a quantitative analysis of the related problems in the model elsewhere.

Appendix

In this appendix, we consider cosmological issues in the case with a very light $X_\mu$, where the resonant leptogenesis occurs successfully as discussed in the text. In order to avoid the late time decay of $X_\mu$, we might introduce kinetic term mixing between the gauge fields $\hat{B}_\mu$ and $\hat{X}_\mu$ for the gauge groups $U(1)$ $_Y$ and $U(1)$ $_X$.¹⁰ The kinetic term mixing between them may be given by

$$\begin{aligned} -\frac{1}{4}\hat{F}_{\mu \nu }\hat{F}^{\mu \nu }-\frac{1}{4}\hat{G}_{\mu \nu } \hat{G}^{\mu \nu }-\frac{\sin \chi }{2}\hat{F}_{\mu \nu }\hat{G}^{\mu \nu }, \end{aligned}$$

(49)

where $\hat{F}_{\mu \nu }$ and $\hat{G}_{\mu \nu }$ are the field strengths of $\hat{B}_\mu$ and $\hat{X}_\mu$, respectively. We can diagonalize these terms by taking the canonically normalized basis $B_\mu$ and $X_\mu$ as

$$\begin{aligned} \left( \begin{array}{c} \hat{B}_\mu \\ \hat{X}_\mu \end{array}\right) = \left( \begin{array}{cc} 1 &{} -\tan \chi \\ 0 &{} \displaystyle {\frac{1}{\cos \chi }}\\ \end{array}\right) \left( \begin{array}{c} B_\mu \\ X_\mu \\ \end{array}\right) . \end{aligned}$$

(50)

The modified $U(1)$ $_X$ charge with this new basis is given by

$$\begin{aligned} Q_X=\frac{\hat{Q}_X}{\cos \chi }+\frac{g_Y}{g_X}Y\tan \chi , \end{aligned}$$

(51)

where the $U(1)$ $_Y$ charge and both the coupling constants $g_Y$ and $g_X$ are defined as the ones in the no mixing case. This suggests that the standard model contents with $Y\not =0$ could couple with $X_\mu$ as long as the kinetic term mixing exists. As a result, the analysis of the direct search and the relic abundance of dark matter should be modified. In this case, the following new interaction should be added to Eq. (32):

$$\begin{aligned} \frac{g_X}{2}\left( \frac{1}{\cos \chi }+\frac{g_Y}{2g_X}\right) X^\mu \left( \eta _R\partial _\mu \eta _I - \eta _I\partial _\mu \eta _R\right) . \end{aligned}$$

(52)

If the kinetic term mixing exists, inelastic scattering of $\eta _R$ can also be brought about by the $X_\mu$ exchange. Since both $\eta _R$–nucleon scattering cross sections $\sigma _n^0(X_\mu )$ and $\sigma _n^0(Z_\mu )$, which are mediated by the $X_\mu$ and $Z_\mu$ exchange at zero momentum transfer, can be related each other as

$$\begin{aligned} \sigma _n^0(X_\mu )\simeq \left( \frac{m_Z^2}{m_X^2}\tan \chi \right) ^2\sigma _n^0(Z_\mu ), \end{aligned}$$

(53)

the present experimental results require that the kinetic term mixing should satisfy

$$\begin{aligned} \tan \chi ~{^<_\sim }~\frac{m_X^2}{m_Z^2}. \end{aligned}$$

(54)

This shows that the kinetic term mixing should be sufficiently small for $m_X~{^<_\sim }~ O(1)$ GeV. New non-negligible (co)annihilation modes of $\eta _R$ to the standard model contents could also appear, depending on the magnitude of the kinetic term mixing $\sin \chi$. However, the constraint (54) suggests that the $\eta _R$ relic abundance could not be affected by the process mediated through the $X_\mu$ exchange. In the study of the $\eta _R$ relic abundance, even if we introduce the kinetic term mixing, we can neglect the effect of it as long as the condition (54) is satisfied. Thus, the results obtained in this paper do not change.

As another interesting phenomenon caused by the kinetic term mixing, we consider the $X_\mu$ direct decay to the lighter fermions in the standard model through tree diagrams. Its decay width could be estimated as

$$\begin{aligned} \Gamma _X(f\bar{f})=\sum _f\frac{g_Y^2}{16\pi m_X} \left( \frac{Y_f}{2}\right) ^2\tan ^2\chi . \end{aligned}$$

(55)

If we impose that $\Gamma _X~{^>_\sim }~H$ is satisfied before the neutrino decoupling, we find that the kinetic term mixing should satisfy

$$\begin{aligned} \tan \chi ~{^>_\sim }~10^{-11}\left( \frac{1~\mathrm{GeV}}{m_X}\right) ^{\frac{1}{2}} \left( \frac{T}{1~\mathrm{MeV}}\right) . \end{aligned}$$

(56)

This shows that a sufficiently small kinetic term mixing is enough to bring about the $X_\mu$ decay to the standard model fermions before the neutrino decoupling. As long as the very small kinetic term mixing exists, the model can overcome the cosmological difficulty for the effective neutrino number in both cases with $m_X>M_{\tilde{\sigma }}$ and $m_X<M_{\tilde{\sigma }}$. Especially, if the kinetic term mixing takes a suitable value in the case $m_X< 1$ MeV, the deviation of the effective neutrino number $N_\mathrm{eff}=3.62\pm 0.25$, which is suggested through the combined analysis of the data from Planck and the $H_0$ measurement from the Hubble Space Telescope [13, 14], might be explained.