$Z'$ portal dark matter in $B-L$ scotogenic Dirac model

Abstract

In this paper, we perform a detail analysis on the phenomenology of $Z'$ portal scalar and Dirac fermion dark matter in $B-L$ scotogenic Dirac model. Unconventional $B-L$ charge Q is assigned to the right-handed neutrino $\nu _R$ in order to realise scotogenic Dirac neutrino mass at one-loop level, where three typical value $Q=-\,\frac{1}{4},-\,4,\frac{3}{2}$ are chosen to illustrate. Observational properties involving dilepton signature at LHC, relativistic degrees of freedom $N_\text {eff}$, dark matter relic density, direct and indirect detections are comprehensively studied. Combined results of these observables for the benchmark scenarios imply that the resonance region $M_\text {DM}\sim M_{Z'}/2$ is the viable parameter space. Focusing on the resonance region, a scanning for TeV-scale dark matter is also performed to obtain current allowed and future prospective parameter space.

1 Introduction

Despite the unambiguous confirmation of non-zero neutrino masses and mixings, the nature of neutrinos, which could be Majorana or Dirac, is still an open question. It is usually argued that tiny neutrino masses may naturally originate from the effective $\Delta L=2$ lepton number violating (LNV) operators [1], so the Majorana scenario seems more promising. However, the neutrinoless double beta decay ($0\nu 2\beta$) experiments, aimed to search for $\Delta L=2$ LNV signatures, have been performed without any positive result so far and the possibility that neutrinos are Dirac particles can not be excluded. In the standard model (SM) along with three copies of $\nu _{R}$, Dirac neutrinos can acquire their masses via direct Yukawa coupling $y_{\nu }\overline{L}H\nu _{R}$. In order to generate sub-eV neutrino masses, the coupling constants $y_{\nu }$ have to be fine tuned to $10^{-12}$ order. Consequently, if neutrinos are Dirac particles, certain new physics beyond SM should exist to explain the extremely small Yukawa coupling constants in a natural way. Motivated by these considerations, interest in Dirac neutrino masses has been revived recently and several models [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20] are proposed at tree or loop level.

On the other hand, the connections between LNV effects and the nature of neutrinos are often overlooked. For example, Dirac neutrinos are compatible with standard baryogenesis processes [21, 22], where the non-perturbative effect breaks the baryon (B) and lepton (L) number each by three units while $B-L$ number remains conserved. As is well known, the simplest anomaly-free $B-L$ extension of SM includes three copies of $\nu _{R}$ having $-1$ charge under $U(1)_{B-L}$ symmetry. Then the Majorana neutrino will not arise if $U(1)_{B-L}$ symmetry never breaks down by two units, i.e., $\Delta (B-L)\ne 2$, which explains why neutrinos are Dirac fermions. In Refs. [23, 24], the quartic LNV operators i.e., $\Delta (B-L)= 4$ processes are considered as new sources for leptogenesis even if neutrinos are Dirac particles.

However, the simple $B-L$ extension of SM model can not explain the extremely small Yukawa coupling constant in the presence of $y_{\nu }\overline{L}H\nu _{R}$ term. It is reasonable to ask if gauged $U(1)_{B-L}$ symmetry can play a key role in the generation of naturally small Dirac neutrino masses. Recently, a class of $B-L$ scotogenic models are proposed by us in Ref. [11] where the Dirac neutrino masses are generated at loop level. The Greek “scotogenic” means darkness. The original model, firstly propose by Ma [25], is to accommodate two important missing pieces of SM: no-zero Majorana neutrino mass and dark matter (DM) in a unified framework. The main idea is based on the assumption that the DM candidates, whose the stability is protected by an ad hoc $Z_{2}$ or $Z_{3}$ [26] symmetry, can serve as intermediate messengers propagating inside the loop diagram in neutrino mass generation. In the scotogenic Dirac model, the $B-L$ quantum number of $\nu _{R}$ are appropriately assigned so that the $y_{\nu }\overline{L}H\nu _{R}$ interaction and Majorana term $\overline{\nu }_{R}\nu _{R}^{c}$ are forbidden. Some new Dirac fermions are introduced to be intermediate particles carrying $B-L$ quantum numbers. Since all new particles are SM singlets, we only need to consider the $[U(1)_{B-L}]\times [Gravity]^{2}$ and $[U(1)_{B-L}]^{3}$ anomaly free conditions. Then the effective $\overline{\nu _{L}}\nu _{R}$ operator is induced after the spontaneous breaking of $U(1)_{B-L}$ at high scale. Moreover, the discrete $Z_{2}$ or $Z_{3}$ symmetry could appear as a remnant symmetry of gauged $U(1)_{B-L}$ symmetry, leading to DM candidates.

In previous studies [11], we focus on scalar interactions of DM in $B-L$ scotogenic Dirac model. As complementary, this paper will concentrate on gauge interactions of DM. Early researches on various $Z'$ portal DM can be found in Refs. [27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45]. As for Majorana fermion DM in conventional $B-L$ scotogenic Majorana model for neutrino masses [46], no observable signatures are expected at incoming direct or indirect detection experiments except for LHC dilepton signature of $Z'$. In contrast, the $Z'$ portal scalar DM in $B-L$ scotogenic Dirac model could be tested at direct detection experiments, and Dirac fermion DM could be further probed by indirect detection. Therefore, if the ongoing DM experiments observe some positive signatures, the nature of DM might be revealed.

Moreover, the existence of right-handed neutrino $\nu _R$ will also contribute to effective number of relativistic degrees of freedom $N_\text {eff}$ for neutrinos via $Z'$ portal, which can also be used to verify this model. Note that although the well studied simplified $Z'$ portal DM approach is constructive, there might be some different predictions for a complete model, especially when a new observable as $N_\text {eff}$ in this model is involved. So in this paper, we perform a comprehensive analysis on the phenomenology of $Z'$ portal dark matter in $B-L$ scotogenic Dirac model in aspect of collider signatures, dark radiation, relic density, direct and indirect detection. Our paper differs from previous simplified or concrete models in the following aspects:

• A new constraint from $N_\text {eff}$ is considered. Under certain circumstances (e.g. Fig. 13c), this could set a more tight constraint than direct detection.

• Some exotic $B-L$ charge assigned in simplified model can be explained by the requirement of generating Dirac neutrino mass in the $B-L$ scotogenic Dirac model.

• Our model for fermion DM has both vector and axial-vector coupling, while previous simplified models only consider vector coupling.

• The indirect detection limits are considered in our paper. As will shown latter, the indirect detection experiments are powerful to probe the resonance region $M_\text {DM}\lesssim M_{Z'}/2$.

The rest of the paper is organised as following. In Sect. 2, we briefly review the minimal gauged $U(1)_{B-L}$ scotogenic Dirac model. Collider signatures from LHC are discussed in Sect. 3. The exclusion limits are derived from the latest dilepton signature search. Dark radiation observable $N_\text {eff}$ and corresponding limits are then considered in Sect. 4. The DM phenomenology in aspect of relic density, direct detection and indirect detection are detail studied in Sect. 5 for some benchmark scenarios. In Sect. 6, we first present the combined results for certain benchmark scenarios. And then based on the combined results, we perform a scan over the resonance region $M_\text {DM}\simeq M_{Z'}/2$. Finally, the conclusion is devoted to Sect. 7.

Table 1 Particle contents and corresponding charge assignment

2 The model

The gauged $U(1)_{B-L}$ scotogenic Dirac neutrino mass models are systematically studied in Ref. [11]. In this paper, we concentrate on the minimal $U(1)_{B-L}$ scotogenic Dirac model, i.e., model $A_1$ in Ref. [11]. The relevant particle contents and corresponding charge assignment are explicitly shown in Table 1. In addition to three generations of right-handed neutrino $\nu _R$ with $B-L$ charge Q, this model further employs three generations of Dirac fermion F with $B-L$ charge $+1$ for the left-handed component and $-Q$ for the right-handed component. In the scalar sector, this model introduces one scalar doublet $\eta$, two scalar singlets $\chi$ and $\sigma$ with $B-L$ charge $Q-1$, $Q-1$ and $Q+1$ respectively. Owing to newly introduced chiral fermions being singlets under the SM gauge group, one only need to check the $[U(1)_{B-L}]\times [Gravity]^2$ and $[U(1)_{B-L}]^3$ anomaly free conditions [47]

$$\begin{aligned} -3-3Q~~-3(-Q)~~+3\times 1= & {} 0, \end{aligned}$$

(1)

$$\begin{aligned} -3-3Q^3-3(-Q)^3+3\times 1= & {} 0. \end{aligned}$$

(2)

Obviously, the anomalies has been canceled for per generation fermions, since $F_{L,R}$ has opposite $B-L$ charge comparing with $\nu _{L,R}$. Therefore, Q can be regarded as a free parameters apart from the following exceptions:

• To forbid the SM direct Yukawa coupling term $\bar{\nu }_L\nu _R\overline{\phi ^0}$, the condition $Q\ne -1$ should be imposed.

• To forbid Majorana mass terms $(m_{R})\overline{\nu _{R}^C}\nu _{R}$, $\sigma \overline{\nu _{R}^C}\nu _{R}$ and $\sigma ^{*}\overline{\nu _{R}^C}\nu _{R}$, the conditions $Q\ne 0, -1/3$ and 1 should also be satisfied respectively.

• To avoid tree-level neutrino mass, linear terms as $\sigma ^{k}\chi$ and $(\sigma ^{*})^{k}\chi$($k=1,2,3$) must be forbidden, which then requires $Q\ne 0, -1/3,-1/2,-2$ and $-3$.

• Similarly, forbidding the $(\Phi ^{\dag }\eta )\sigma ^{k}$ and $(\Phi ^{\dag }\eta )(\sigma ^{*})^{k} (k=1,2)$ terms leads to $Q\ne 0, -1/3,-3$.

Once an appropriate Q is assigned, the residual $Z_{2}$ symmetry appears in Eqs. (3), (4) and (6), under which the parity is odd for inert particles ($\eta ,\chi ,F_{L/R}$) and even for all other particles. In the following, we consider $Q=-\frac{1}{4},-4$ and $\frac{3}{2}$ for illustration.

The relevant new gauge interactions are dictated by

$$\begin{aligned}&{\mathcal {L}}_{Z'} \supset -\frac{1}{4} F'^{\mu \nu } F'_{\mu \nu } \nonumber \\&\quad + (D^\mu \eta )^\dag (D_\mu \eta ) + |D^\mu \chi |^2 + |D^\mu \sigma |^2 \nonumber \\&\quad + \frac{g'}{3} \sum _{i=1}^6( \bar{q}_i\gamma ^\mu q_i) Z'_\mu + g' \sum _{i=1}^3(- \bar{\ell }_i \gamma ^\mu \ell _i - \bar{\nu }_{Li}\gamma ^\mu \nu _{Li} \nonumber \\&\quad + Q \bar{\nu }_{Ri}\gamma ^\mu \nu _{Ri} + \bar{F}_{Li}\gamma ^\mu F_{Li} - Q \bar{F}_{Ri}\gamma ^\mu F_{Ri}) Z'_\mu , \end{aligned}$$

(3)

where $D_\mu =\partial _\mu -ig'Q_{BL}Z'_\mu$. The complete scalar potential is given by [11]

$$\begin{aligned} V= & {} -\mu _\Phi ^2 \Phi ^\dag \Phi + \mu _\eta ^2 \eta ^\dag \eta + \mu _\chi ^2 \chi ^*\chi - \mu _\sigma ^2 \sigma ^*\sigma \nonumber \\&+\lambda _\Phi ^2 (\Phi ^\dag \Phi )^2+ \lambda _\eta (\eta ^\dag \eta )^2 + \lambda _\chi (\chi ^*\chi )^2 + \lambda _\sigma (\sigma ^*\sigma )^2 \nonumber \\&+ \lambda _{\Phi \eta } (\Phi ^\dag \Phi ) (\eta ^\dag \eta ) + \lambda '_{\Phi \eta } (\eta ^\dag \Phi ) (\Phi ^\dag \eta ) \nonumber \\&+ \lambda _{\Phi \chi } (\Phi ^\dag \Phi ) (\chi ^*\chi ) + \lambda _{\Phi \sigma } (\Phi ^\dag \Phi ) (\sigma ^*\sigma ) \nonumber \\&+ \lambda _{\eta \chi } (\eta ^\dag \eta ) (\chi ^*\chi ) + \lambda _{\eta \sigma } (\eta ^\dag \eta ) (\sigma ^*\sigma ) \nonumber \\&+ \lambda _{\chi \sigma } (\chi ^*\chi ) (\sigma ^*\sigma ) + \left[ \mu (\Phi ^{\dagger }\eta )\chi ^{*} + \text {h.c.}\right] , \end{aligned}$$

(4)

where $\mu ^2_X(X=\Phi ,\eta ,\chi ,\sigma )$ are all taken to be positive, and $\mu$ is also positive by proper re-phasing $\eta$ and $\chi$. Due to the lack of $(\Phi ^\dag \eta )^2$ term, masses of $\eta _R^0$ and $\eta _I^0$ are degenerate as a complex scalar $\eta ^0$. Because $\eta ^0$ has direct coupling with Z, only the singlet $\chi$ is suitable for scalar DM. To concentrate on $Z'$-portal DM in this paper, the Higgs-portal $\lambda _{\Phi \chi }$, $\lambda _{\Phi \sigma }$ and $\lambda _{\chi \sigma }$ terms are assumed to be tiny.

Fig. 1

One-loop generation of Dirac neutrino mass

After spontaneous symmetry breaking, the scalar $\Phi$ and $\sigma$ are denoted as

$$\begin{aligned} \Phi =\left( \begin{array}{c} 0\\ \frac{v_{\phi }+\phi ^0}{\sqrt{2}} \end{array}\right) , \quad \sigma = \frac{v_{\sigma }+ \varphi ^0}{\sqrt{2}}, \end{aligned}$$

(5)

in unitarity gauge. Here, $v_\phi =246~\mathrm{GeV}$ is the electroweak scale. Then the $\lambda _{\Phi \sigma }$ term induces mixing between $\phi ^0$ and $\varphi ^0$ with mixing angle $\alpha$, resulting to mass eigenstates $h^0$ and $H^0$ [48]. In this paper, we regard $h^0$ as the $125~\mathrm{GeV}$ Higgs boson discovered at LHC [49, 50], and $H^0$ is a heavier scalar singlet [51]. Meanwhile, the $Z_2$-odd scalar $\chi$ and $\eta ^0$ mix into $H_1^0$ and $H_2^0$ with mixing angle $\beta$. In this paper, we assume $M_{H_1^0}<M_{H_2^0}$ and $\beta \ll 1$, so $H_1^0$ is dominant by $\chi$-component and is regarded as scalar DM candidate. Note that $\beta \lesssim 0.01$ is favored by DM direct detection experiments [52]. Besides, the $Z_2$-odd charged scalar $H_2^\pm (=\eta ^\pm )$ do not mix with other particles.

The Yukawa interactions accounting for radiative Dirac neutrino mass are given as [11]

$$\begin{aligned} {\mathcal {L}}_Y = y_{1}\overline{L}F_{R}i\tau _{2}\eta ^{*}+ y_{2}\overline{\nu _{R}}F_{L}\chi + y_F\overline{F_{L}}F_{R}\sigma +\text {h.c.} \end{aligned}$$

(6)

Hence, VEV of $\sigma$ will induce Dirac fermion mass $M_F= y_F v_\sigma /\sqrt{2}$, the lightest of which, i.e., $F_1$ serves as fermion DM candidate. As shown in Fig. 1, the above Yukawa interactions will induce Dirac neutrino mass at one-loop level. The neutrino mass matrix can be expressed as

$$\begin{aligned} m_{\nu }^{ij}= & {} \frac{\sin 2\beta }{32\pi ^2}\sum _k y_1^{ik} y_2^{jk*} M_{F_k} \nonumber \\&\left[ \frac{M_{H_2^0}^2}{M_{H_2^0}^2-M_{F_k}^2}\ln \left( \frac{M_{H_2^0}^2}{M_{F_k}^2}\right) \right. \nonumber \\&\left. -\frac{M_{H_1^0}^2}{M_{H_1^0}^2-M_{F_k}^2}\ln \left( \frac{M_{H_1^0}^2}{M_{F_k}^2}\right) \right] . \end{aligned}$$

(7)

Under constraints from lepton flavor violation [11] and DM direct detection, $\beta \sim 10^{-4}$, $y_{1,2}\sim 0.01$ with masses of $Z_2$-odd particles at TeV-scale are required to accommodate neutrino mass around $0.1~\mathrm{eV}$.

In the following, we concentrate on the phenomenology of $Z'$ portal DM. In this case, there are only four free parameters: the new gauge coupling $g'$, the new gauge boson mass $M_{Z'}$, the DM mass $M_\text {DM}$ and $\nu _R$’s $B-L$ charge Q. Before investigate the DM phenomenon, we consider possible constraints from collider signature in Sect. 3 and dark radiation in Sect. 4.

Table 2 Branching ratio of $Z'$ for different value of Q

Fig. 2

Cross section of the dilepton signature $pp\rightarrow Z'\rightarrow \ell ^+\ell ^-$ at $13~\mathrm{TeV}$ LHC. a $Q=-\frac{1}{4}$, $g'=0.1,0.2,0.5$. b $g'=0.2$, $Q=-\frac{1}{4}, -4, \frac{3}{2}$. The dashed line is the upper limit from Ref. [60]

3 Collider signature

Spontaneous breaking of the $U(1)_{B-L}$ gauge symmetry by the VEV of $\sigma$ induces the mass term of the $U(1)_{B-L}$ gauge boson as $M_{Z'}=Q_\sigma g' v_\sigma =(Q+1)g' v_\sigma$. The precise measurement of four-fermion interactions at LEP requires [53]

$$\begin{aligned} \frac{M_{Z'}}{g'}\gtrsim 7~\mathrm{TeV}. \end{aligned}$$

(8)

In the limit that masses of SM fermions f($f\equiv q, l, \nu$) are small compared with the $Z^{\prime }$ mass, the decay width of $Z^{\prime }$ into fermion pair $f\overline{f}$ is given by

$$\begin{aligned} \Gamma (Z^{\prime }\rightarrow f\overline{f})=\frac{g'^{2}M_{Z^{\prime }}}{24\pi }N_C^{f}(Q_{fL}^{2}+Q_{fR}^{2}) \end{aligned}$$

(9)

where $N_C^f$ is the number of colours of the fermion f, i.e., $N_C^{l,\nu }=1$, $N_C^{q}=3$. Then the branch ratios of $Z^{\prime }$ decay into each final states take the ratios as

$$\begin{aligned}&\text {BR}(Z^{\prime }\rightarrow q\overline{q}):\text {BR}(Z^{\prime }\rightarrow l^{+}l^{-}):\text {BR}(Z^{\prime }\rightarrow \nu \overline{\nu }) \nonumber \\&\quad =4:6:3(1+Q^{2}). \end{aligned}$$

(10)

The Q-value has a great impact on the decay properties of $Z'$. According to Eq. (10), the dominant decay mode of $Z'$ is $Z'\rightarrow \ell ^+\ell ^-$ for $|Q|<1$, and $Z'\rightarrow \nu \bar{\nu }$ for $|Q|>1$. A viable pathway to verify the Q-value is via the measurement of the ratio $\text {BR}(Z'\rightarrow \nu \bar{\nu })/\text {BR}(Z'\rightarrow \mu ^+\mu ^-)=3(1+Q^2)/2$. In Table 2, we depict the explicit value of branching ratios of $Z'$ for $Q=-\frac{1}{4}$, $-4$ and $\frac{3}{2}$ respectively. For $Q=-\frac{1}{4}$, the dominant decay mode $Z'\rightarrow \ell ^+\ell ^-$ has a branching ratio of 0.45. In comparison, the branching ratio of this dilepton mode diminishes to 0.10 when $Q=-4$, and the invisible decay mode $Z'\rightarrow \nu \bar{\nu }$ with a branching ratio 0.83 becomes the dominant one.

The most promising signature of $U(1)_{B-L}$ gauge boson at LHC is the dilepton signature $pp\rightarrow Z' \rightarrow \ell ^+\ell ^-(\ell =e,\mu )$ [54, 55]. Searches for the new gauge boson $Z'$ in the dilepton final states have been performed by ATLAS [56, 57] and CMS [58, 59]. Due to no observation of any excess, the most stringent upper limit on the cross-section times branching ratio ($\sigma \text {BR}$) has been set by ATLAS using $36.1~\mathrm{fb}^{-1}$ data at $\sqrt{s}=13~\mathrm{TeV}$ LHC [60]. Here, this upper limit will be interpreted in the the frame work of $B-L$ scotogenic Dirac model. See Refs. [61,62,63,64,65,66,67,68,69] for some earlier studies.

In carrying out the theoretical cross section of the dilepton signature, we implement the scotogenic Dirac model into FeynRules [70] and employ MadGraph5_aMC@NLO [71] with NNPDF [72] parton distribution function. As suggested by Ref. [62], a factor of $k=1.20$ is also multiplied in order to include the QCD corrections. The results are shown in Fig. 2. Clearly, for a fixed Q-value, a larger $g'$ leads to a larger cross section (shown in Fig. 2a). Meanwhile, for a fixed $g'$-value, a larger |Q| leads to a smaller cross section (shown in Fig. 2b), mainly due to the suppression of $\text {BR}(Z'\rightarrow \ell ^+\ell ^-)$.

Fig. 3

LHC exclusion limits in the $g'-M_{Z'}$ for $Q=-\frac{1}{4},-4,\frac{3}{2}$ in the $U(1)_{B-L}$ scotogenic Dirac model. The dashed line corresponds to the LEP limit in Eq. (8)

By comparing the theoretical prediction and experimental limit, one can easily obtain the exclusion limit in the $g'-M_{Z'}$ plane, which is depicted in Fig. 3 for $Q=-\frac{1}{4},-4,\frac{3}{2}$ respectively. Basically speaking, the exclusion limit for $Q=-4$ are the weakest, while exclusion limits are similar for $Q=-\frac{1}{4}$ and $Q=\frac{3}{2}$. Taking $Q=-\frac{1}{4}$ for an instance, the lower limit of $Z'$ mass is about $4~\mathrm{TeV}$ with $\mathcal {O}(1)$ $g'$-value, and this lower limit could down to about $3~\mathrm{TeV}$ with $g'\sim 0.1$. In the lower mass region $M_{Z'}\lesssim 3~\mathrm{TeV}$, the limit from LHC dilepton signature is more stringent than LEP. But for $M_{Z'}\gtrsim 4~\mathrm{TeV}$, the LEP limit becomes severer than the LHC limit. It is noteworthy that if no evidence for the dilepton signature of $Z'$ is found, the future operation of high-luminosity (HL) and high-energy (HE) LHC will significantly restrict the parameter space, e.g. for $g'=0.1$, the 13 TeV HL-LHC (27 TeV HE-LHC) with $3000~\mathrm{fb}^{-1}$ data is able to exclude $M_{Z'}\lesssim 3.85(6.55)~\mathrm{TeV}$ [73].

4 Dark radiation

In this $B-L$ scotogenic Dirac model, the light right-handed neutrinos $\nu _R$ serve as a new form of dark radiation in the expansion of the universe, which will affect the effective number of relativistic degrees of freedom $N_\text {eff}$ for neutrinos. Here, we use the combined Planck TT + lowP + BAO data, $N_\text {eff}=3.15\,\pm \,0.23$ [74]. One naively may worry that the existence of three generation of $\nu _R$ is in conflict with observed data, since the relativistic degrees of freedom for neutrinos are doubled. Actually, due to different gauge interactions, $\nu _R$ could decouple earlier than $\nu _L$, leading to a suppression of contribution to $N_\text {eff}$ from $\nu _R$. The corresponding theoretical contribution of $\nu _R$ to $\Delta N_\text {eff}=N_\text {eff}-N_\text {eff}^\text {SM}$ is given by [75, 76]

$$\begin{aligned} \Delta N_{\text {eff}} = 3 \left( \frac{g(T^{\nu _L}_\text {dec})}{g(T^{\nu _R}_\text {dec})}\right) ^{4/3}, \end{aligned}$$

(11)

where $N_{\text {eff}}^\text {SM}=3.046$ comes from the contribution of SM left-handed neutrinos [77, 78], and g(T) is the effective number of degrees of freedom at temperature T [79]. The decoupling temperature of left-handed neutrinos $\nu _L$ is $T^{\nu _L}_\text {dec}\sim 3~\mathrm{MeV}$, thus $g(T^{\nu _L}_\text {dec})=43/4$ corresponds to three $\nu _L$, $e^\pm$ and photon [80]. In the gauged $U(1)_{B-L}$ scotogenic Dirac model, $\nu _R$ are in equilibrium with SM fermions via the new gauge boson $Z'$, therefore the interaction rate of $\nu _R$ is calculated as [79]

$$\begin{aligned} \Gamma _{\nu _R}(T)= & {} n_{\nu _R}(T)\sum _f \langle \sigma _f(\nu _R \bar{\nu }_R\rightarrow f\bar{f})v \rangle \nonumber \\= & {} \sum _f \frac{g_{\nu _R}^2}{n_{\nu _R}}\int \frac{d^3 p }{(2\pi )^3} \frac{d^3 q }{(2\pi )^3} f_{\nu _R}(p)f_{\nu _R}(q) \sigma _f(s) v,\nonumber \\ \end{aligned}$$

(12)

where $s=2pq(1-\cos \theta )$, $v=(1-\cos \theta )$ with $\theta$ the relative angle of the colliding right-handed neutrinos. $f_{\nu _R}(p)$ is the Fermi-Dirac distribution function $f_{\nu _R}(k)=1/(e^{k/T}+1)$ and the number density of the right-handed neutrinos $n_{\nu _R}$ with spin number $g_{\nu _R}=2$ is given by

$$\begin{aligned} n_{\nu _R}(T)=g_{\nu _R}\int \frac{d^3k}{(2\pi )^3}f_{\nu _R}(k). \end{aligned}$$

(13)

In the limit $M^2_{Z'}\gg s$, the annihilation cross section in Eq. (12) is given by

$$\begin{aligned} \sigma _f(s)\simeq \frac{ N_C^f (Q_{BL}^f)^2 Q^2 s}{12\pi }\left( \frac{g'}{M_{Z'}}\right) ^4. \end{aligned}$$

(14)

Including the contribution of three right-handed neutrinos, the Hubble parameter is now derived by

$$\begin{aligned} H(T)=\sqrt{\frac{4\pi ^3G_N(g(T)+\frac{21}{4})}{45}}\,T^2. \end{aligned}$$

(15)

Fig. 4

$\Delta N_{\text {eff}}$ as a function of $M_{Z'}$. a $Q=-\frac{1}{4}$, $g'=0.1,0.2,0.5$. b $g'=0.2$, $Q=-\frac{1}{4}, -4, \frac{3}{2}$. The shaded pink region is allowed by current experiments [74]

The right-handed neutrinos decouple when $\Gamma _{\nu _R}(T^{\nu _R}_\text {dec})=H(T^{\nu _R}_\text {dec})$. Solving this condition numerically, one can obtain $T^{\nu _R}_\text {dec}$, and then $\Delta N_\text {eff}$ via Eq. (11). The results of $\Delta N_\text {eff}$ as a function of $M_{Z'}$ are exhibited in Fig. 4. According to Eq. (14), for a larger $g'$, |Q| or a smaller $M_{Z'}$, the corresponding annihilation cross section is larger, resulting in a smaller decoupling temperature $T^{\nu _R}_\text {dec}$ [79]. Therefore, $g(T^{\nu _R}_\text {dec})$ is smaller and then $\Delta N_\text {eff}$ is larger. Such changes are clearly shown in Fig. 4. We notice that for TeV-scale $Z'$, the minimum value of $\Delta N_\text {eff}$ is about 0.2, corresponding to a plateau around $g(T_\text {dec}^{\nu _R})\sim 86$ when $3~\mathrm{GeV}\lesssim T_\text {dec}^{\nu _R}\lesssim 10~\mathrm{GeV}$, and are hopefully within the reach of future Euclid experiment [81].

Fig. 5

$\Delta N_\text {eff}$ exclusion limit in the $g'-M_{Z'}$ for $Q=-\frac{1}{4},-4,\frac{3}{2}$. The dashed line corresponds to the LEP limit in Eq. (8)

By requiring $\Delta N_\text {eff}=0.564$, one can further derive the exclusion limit in the $g'-M_{Z'}$ plane, which is shown in Fig. 5. Depending on |Q|, the exclusion limits are approximately $M_{Z'}/g'\lesssim 3.5, 9.5, 21~\mathrm{TeV}$ for $Q=-\frac{1}{4},\frac{3}{2},-4$ respectively. In contrast to LHC limits, a larger |Q| will lead to a more stringent limit from $\Delta N_\text {eff}$. And for $Q=\frac{3}{2}$ and $-4$, the exclusion limits have already exceeded LEP limit.

5 Dark matter

In this section, we investigate the phenomenology of $Z'$-portal DM, which is determined by relevant interactions presented in Eq. (3). As stated in Sect. 2, we consider $H_1^0$ and $F_1$ as scalar and fermion DM candidate respectively. DM observables, such as the DM relic density, the DM-nucleon cross section and the thermally averaged annihilation cross section $\langle \sigma v \rangle$, are calculated with the help of micrOMEGAs [82, 83] for a more precise results. In order to illustrate the dependence of DM observables on certain variables, we first carry out the results of some benchmark scenarios in this section. The combined analysis of different observables and a scan over corresponding parameter space will be performed in Sect. 6.

5.1 Relic density

Here, we consider the conventional WIMP DM freeze-out scenario. As long as the temperature T is high enough, the DM candidate is in thermal equilibrium with the primordial thermal bath via the new gauge interaction. And after the DM interaction rate is smaller than the expansion rate of the Universe, it is decoupled. The DM number density n is then calculated by solving the following Boltzmann equation [84]

$$\begin{aligned} \frac{d n}{dt} + 3 H n= -\langle \sigma v \rangle (n^2-n_\text {eq}^2), \end{aligned}$$

(16)

where H is the Hubble parameter, and $n_\text {eq}$ is the thermal equilibrium value. In the following numerical results, we use micrOMEGAs [82, 83] to solve the above Boltzmann equation and determine the DM relic density. Approximately, one can estimate the DM relic density by

$$\begin{aligned} \Omega h^2\approx \frac{3\times 10^{-27} \text {cm}^3\text {s}^{-1}}{\langle \sigma v\rangle }, \end{aligned}$$

(17)

which will be taken for qualitative illustration, since it is more intuitional. Moreover, the DM relic density measured by Planck is $\Omega h^2=0.1199\pm 0.0027$ [74].

In the $B-L$ scotogenic Dirac model, the DM candidate $H_1^0$ or $F_1$ can annihilate into $Z_2$-even fermion pairs via the exchange of $Z'$ in the s-channel. Neglecting masses of the final state fermions, the corresponding thermal averaged annihilation cross sections are given by [85]

$$\begin{aligned}&\langle \sigma v\rangle _{H_1^0 H_1^{0*}\rightarrow f\bar{f}}\simeq \frac{2 g'^4 N_C^f (V_f^2+A_f^2) Q^2_{H_1^0} M_{H_1^0}^2}{3\pi [(4M_{H_1^0}^2-M_{Z'}^2)^2+M_{Z'}^2\Gamma _{Z'}^2]} v^2,\nonumber \\ \end{aligned}$$

(18)

$$\begin{aligned}&\langle \sigma v\rangle _{F_1\bar{F}_1\rightarrow f\bar{f}}\simeq \frac{2g'^4 N_C^f(V_f^2+A_f^2) V_{F_1}^2 M_{F_1}^2}{\pi [(4M_{F_1}^2-M_{Z'}^2)^2+M_{Z'}^2\Gamma _{Z'}^2]}, \end{aligned}$$

(19)

for scalar DM $H_1^0$ and fermion DM $F_1$ respectively. Here, $V_q=\frac{1}{3}$, $A_q=0$, $V_\ell =-1$, $A_\ell =0$, $V_\nu =-V_{F_1}=\frac{Q-1}{2}$, $A_\nu =-A_{F_1}=\frac{Q+1}{2}$, can be extracted from Eq. (3). In the mean time, since the scalar DM $H_1^0$ is dominant by the $\chi$-component, we set $Q_{H_1^0}\simeq Q_\chi =Q-1$. Clearly, the annihilation cross sections for scalar DM is velocity suppressed. If DM is heavier than $Z'$, then DM pair can also annihilate into $Z'$ pair, leading to annihilation cross sections as [85]

$$\begin{aligned}&\langle \sigma v\rangle _{H_1^0 H_1^{0*}\rightarrow Z'Z'} \simeq \frac{g'^4 Q_{H_1^0}^4 M_{H_1^0}^2}{16\pi M_{Z'}^4}\left( 1-\frac{M_{Z'}^2}{M_{H_1^0}^2}\right) ^{\frac{1}{2}} \nonumber \\&\times \left( 1-\frac{M_{Z'}^2}{2M_{H_1^0}^2}\right) ^{-2} \nonumber \\&\times \left[ 4\left( 1-\frac{M_{Z'}^2}{M_{H_1^0}^2}\right) ^2 -\left( 4-4\frac{M_{Z'}^2}{M_{H_1^0}^2}-3\frac{M_{Z'}^4}{M_{H_1^0}^4}\right) \right. \nonumber \\&\quad \left. \left( 1-\frac{M_{Z'}^2}{2M_{H_1^0}^2}\right) ^2\right] , \end{aligned}$$

(20)

$$\begin{aligned}&\langle \sigma v\rangle _{F_1\bar{F}_1\rightarrow Z'Z'} \simeq \frac{g'^4}{4\pi M_{F_1}^2 M_{Z'}^2}\left( 1-\frac{M_{Z'}^2}{M_{F_1}^2}\right) ^{\frac{3}{2}} \nonumber \\&\times \left( 1-\frac{M_{Z'}^2}{2M_{F_1}^2}\right) ^{-2} \nonumber \\&\times \left[ (V_{F_1}^4+A_{F_1}^4)M_{Z'}^2+2V_{F_1}^2A_{F_1}^2(4M_{F_1}^2-3M_{Z'}^2)\right] . \end{aligned}$$

(21)

Therefore, free parameters involved in DM relic density are gauge coupling $g'$, $B-L$ charge Q, gauge boson mass $M_{Z'}$, and DM mass $M_{H_1^0,F_1}$, impacts of which are discussed in the following.

Fig. 6

Relic density as a function of scalar DM mass $M_{H_1^0}$. a $Q=-\frac{1}{4}$, $M_{Z'}=2~\mathrm{TeV}$, $g'=0.1,0.2,0.5$. b $g'=0.2$, $M_{Z'}=2~\mathrm{TeV}$, $Q=-\frac{1}{4},-4,\frac{3}{2}$. (c) $Q=-\frac{1}{4}$, $g'=0.2$, $M_{Z'}=2,4,6~\mathrm{TeV}$. The cyan band corresponds to Planck measured result $\Omega h^2=0.1199\pm 0.0027$ [74]

First, we consider the relic density of scalar DM $H_1^0$, which is shown in Fig. 6. Because of resonant production of $Z'$ in the s-channel, all the sub-figures exhibit sharp dips around $M_{H_1^0}\sim M_{Z'}/2$ in the relic density curves, which is a clear behaviour of Eq. (18). And the resonance dips are broadened with increasing gauge $g'$, absolute value of $H_1^0$ charge $|Q_{H_1^0}|=|Q-1|$ and gauge boson mass $M_{Z'}$. In addition, around $M_{H_1^0}\sim M_{Z'}$, one also sees a decrease of relic density as increase of DM mass. Because, when $M_{H_1^0}>M_{Z'}$, the velocity independent annihilation process $H_1^0 H_1^{0*}\rightarrow Z'Z'$ (see Eq. 20) is kinematically allowed and dominates over the velocity dependent process $H_1^0 H_1^{0*}\rightarrow Z'^* \rightarrow f\bar{f}$ (see Eq. 18).

More specifically, in Fig. 6a, we fix $Q=-\frac{1}{4}$, $M_{Z'}=2~\mathrm{TeV}$, and vary $g'=0.1,0.2,0.5$. It is obvious that a larger $g'$ leads to a larger annihilation cross section, thus a smaller relic density. By requiring $\Omega h^2\simeq 0.12$ to satisfy observed value, one actually finds two viable DM mass $M_{H_1^0}$, one below $M_{Z'}/2$ and the other above $M_{Z'}/2$. In Fig. 6b, $g'=0.2$, $M_{Z'}=2~\mathrm{TeV}$ are fixed and $Q=-\frac{1}{4},-4,\frac{3}{2}$ is varied. From Eqs. (18) and (20), one is aware that the annihilation cross sections are positively correlated with $|Q_{H_1^0}|=|Q-1|$. So $Q=\frac{3}{2}$ is equivalent to $Q_{H_1^0}=\frac{1}{2}$, corresponding to smallest $|Q_{H_1^0}|$ and largest relic density. Notably, for $Q=-4$, i.e., $Q_{H_1^0}=-5$, $g'Q_{H_1^0}=-1$, the cross section of annihilation process $H_1^0 H_1^{0*}\rightarrow Z'Z'$ is sufficient large to reach $\Omega h^2\simeq 0.12$. Figure 6c depicts the impact of gauge boson mass $M_{Z'}$ when fixing $Q=-\frac{1}{4}$ and $g'=0.2$. Although the resonance dips exist for all kinds of $M_{Z'}$, the minimum value of relic density increases as $M_{Z'}$ increases. And eventually for $M_{Z'}=6~\mathrm{TeV}$, the minimum value of relic density is larger than the observed value $\sim 0.12$.

Fig. 7

Relic density as a function of fermion DM mass $M_{F_1}$. a $Q=-\frac{1}{4}$, $M_{Z'}=2~\mathrm{TeV}$, $g'=0.1,0.2,0.5$. b $g'=0.1$, $M_{Z'}=2~\mathrm{TeV}$, $Q=-\frac{1}{4},-4,\frac{3}{2}$. c $Q=-\frac{1}{4}$, $g'=0.2$, $M_{Z'}=2,4,6~\mathrm{TeV}$. The cyan band corresponds to Planck measured result $\Omega h^2=0.1199\pm 0.0027$ [74]

Then, we move on to fermion DM. The results are shown in Fig. 7. Similar to scalar DM, resonance dips appear around $M_{F_1}\sim M_{Z'}/2$. But comparing with scalar DM, the relic density of fermion DM is easier to reach $\Omega h^2\sim 0.12$ with same values of free parameters, because the s-channel process $F_1\bar{F}_1\rightarrow Z'^*\rightarrow f\bar{f}$ is not velocity suppressed (see Eqs. 18, 19). In the limit of $M_{F_1}^2\gg M_{Z'}^2$, one approximately has

$$\begin{aligned}&\langle \sigma v\rangle _{F_1\bar{F}_1\rightarrow f\bar{f}}\sim \frac{g'^4 N_C^f(V_f^2+A_f^2) V_{F_1}^2}{8\pi M_{F_1}^2} \ll \langle \sigma v\rangle _{F_1\bar{F}_1\rightarrow Z'Z'} \nonumber \\&\quad \sim \frac{2g'^4 V_{F_1}^2 A_{F_1}^2}{\pi M_{Z'}^2}. \end{aligned}$$

(22)

So with increasing $M_{F_1}$, the process $F_1\bar{F}_1\rightarrow Z'Z'$ gradually becomes more important. Besides, the annihilation cross section, thus the resulting relic density, tends to a constant when $M_{F_1}$ increases.

The impacts of gauge coupling $g'$, $\nu _R$’s $B-L$ charge Q and gauge boson mass $M_{Z'}$ are shown in Fig. 7a–c respectively. Again, effects of changing $g'$ and $M_{Z'}$ are similar as scalar DM, but not Q. When $M_{F_1}<M_{Z'}$, Eq. (19) indicates that the vector coupling $V_{F_1}$ is the dominant contribution. So $Q=-\frac{1}{4}$ with $|V_{F_1}|=\frac{5}{8}$ corresponds to larger cross section and smaller relic density than $Q=\frac{3}{2}$ with $|V_{F_1}|=\frac{1}{4}$. But when $M_{F_1}>M_{Z'}$, one has to take into account the contribution of axial-vector coupling $A_{F_1}$ according to Eq. (21). In this way, $Q=\frac{3}{2}$ leads to a larger annihilation cross section and a smaller relic density than $Q=-\frac{1}{4}$.

5.2 Direct detection

Through t-channel exchange of $Z'$, the new gauge interactions in Eq. (3) would induce spin-independent scattering of DM on nucleon. Currently, various direct detection experiments [86,87,88] have already set stringent bounds on the spin-independent scattering cross section. For $M_{\text {DM}}\lesssim 100~\mathrm{GeV}$, the most stringent limit is provided by XENON1T2017 [87], meanwhile PandaX2017 [88] sets the most stringent limit for $M_{\text {DM}}\gtrsim 100~\mathrm{GeV}$. Moreover, we also incorporate the projected limit by XENON1T(2t$\cdot$y) [89] to illustrate future direct detection limit. Typically, for $M_\text {DM}\sim 1~\mathrm{TeV}$, PandaX2017 [88] requires $\sigma ^\text {SI}\lesssim 10^{-9}$ pb, and XENON1T(2t$\cdot$y) will push this limit down to about $10^{-10}$ pb [89].

Fig. 8

Spin-independent cross section of scalar DM $H_1^0$ scattering on nucleon. Left: as a function of $M_{Z'}$ for $M_{H_1^0}=1~\mathrm{TeV}$, $Q=-\frac{1}{4},-4,\frac{3}{2}$. Right: as a function of $M_{H_1^0}$ for $M_{Z'}=2,4,6~\mathrm{TeV}$, $Q=-\frac{1}{4}$. Here, $g'$ is determined by yielding correct relic density $\Omega h^2\simeq 0.12$. Experimental bounds come from XENON1T2017 (green) [87], PandaX2017 (pink) [88], and XENON1T(2t$\cdot$y) (brown) [89]

The spin-independent scattering cross section is given by [90]

$$\begin{aligned} \sigma ^\text {SI}\approx & {} \frac{\mu _N^2}{\pi }\frac{g'^4}{M_{Z'}^4} Q^2_\text {DM} =1.24\times 10^{-4}\left( \frac{\mu _N}{1~\mathrm{GeV}}\right) ^2 \nonumber \\&\times \left( \frac{1~\mathrm{TeV}}{M_{Z'}}\right) ^4 g'^4 Q_\text {DM}^2~(\mathrm{pb}), \end{aligned}$$

(23)

where $\mu _N=M_N M_\text {DM}/(M_N+M_\text {DM})$ is the reduced mass, $M_N=(M_n+M_p)/2=939~\mathrm{MeV}$ is the averaged nucleon mass, $Q_\text {DM}=Q_{H_1^0}=Q-1$ for scalar DM $H_1^0$, and $Q_\text {DM}=V_{F_1}=(1-Q)/2$ for fermion DM $F_1$. Provided $g'\sim 0.1$, $M_{Z'}\sim 1$, $Q_\text {DM}\sim 1$ and $M_\text {DM}\sim 1~\mathrm{TeV}$, then $\sigma ^\text {SI}\sim 10^{-8}~\mathrm{pb}$ is obtained, which is an order of magnitude larger than PandaX2017 limits [88]. Because the spin-independent cross section is proportional to $Q_\text {DM}^2 g'^4/M_{Z'}^4$, the results of other free parameter sets can be easily derived by a simple rescaling. So in order to escape the direct detection limits, one naively expects that smaller $Q_\text {DM}$, smaller $g'$ and larger $M_{Z'}$ is required. But one also has to keep in mind that such choice of free parameters usually leads to a larger relic density as discussed in Sect. 5.1. Hence, for a more comprehensive analysis, we further require the free parameter set yielding correct relic density.

Again, we consider scalar DM first. In Fig. 8, we display the predicted spin-independent cross section $\sigma ^\text {SI}$. Left panel of Fig. 8 shows $\sigma ^\text {SI}$ as a function of $M_{Z'}$, while we fix $M_{H_1^0}=1~\mathrm{TeV}$ and vary $Q=-\frac{1}{4},-4,\frac{3}{2}$. Similar as relic density, resonance dips also exist when $M_{H_1^0}\sim M_{Z'}/2$. Out of this resonance region, current direct detection experiments XENON1T2017 [87] and PandaX2017 [88] have already excluded the corresponding parameter space, which indicates that the annihilation channel $H_1^0 H_1^{0*}\rightarrow Z'Z'$ is ruled out. Considering the impact of $B-L$ charge $Q_{H_1^0}=Q-1$, one can see that a smaller $|Q_{H_1^0}|$ actually leads to larger DM-nucleon cross section when imposing relic density conditions. For $Q=\frac{3}{2}(|Q_{H_1^0}|=\frac{1}{2})$, it has been fully excluded by PandaX2017 [88]. For $Q=-\frac{1}{4}(|Q_{H_1^0}|=\frac{5}{4})$, although the resonance region can escape current PandaX2017 limit [88], it is within the reach of future XENON1T(2$\cdot$y) [89]. For $Q=-4(|Q_{H_1^0}|=5)$, it can even avoid future XENON1T(2$\cdot$y) limit [89] in the resonance region. Right panel of Fig. 8 depicts $\sigma ^\text {SI}$ as a function of $M_{H_1^0}$, while we fix $Q=-\frac{1}{4}$ and vary $M_{Z'}=2,4,6~\mathrm{TeV}$. Basically, the larger $M_{Z'}$ is, the easier it will be to escape direct detection bounds, due to the bounds is less stringent with increasing DM mass. For $M_{Z'}=4,6~\mathrm{TeV}$, except for the region quite close to the resonance region $M_{H_1^0}\simeq M_{Z'}/2$, XENON1T(2$\cdot$y) [89] could probe most of the parameter space.

Fig. 9

Same as Fig. 8, but for fermion DM $F_1$

Then, we come to fermion DM $F_1$. The results are shown in Fig. 9. Similar as scalar DM, only the resonance region can escape direct detection limits. But in this region, the resulting $\sigma ^\text {SI}$ is much smaller, so that both varying $Q=-\frac{1}{4},-4,\frac{3}{2}$ and varying $M_{Z'}=2,4,6~\mathrm{TeV}$ could easily avoid current and even future limits. Since only the vector coupling $V_{F_1}=(1-Q)/2$ is involved in relic density (see Eq. 18) and spin-independent DM-nucleon scattering (see Eq. 23), $Q=\frac{3}{2}(|V_{F_1}|=\frac{1}{4})$ corresponds to the smallest $B-L$ charge. Hence, the largest $g'$ is required to reproduce correct relic density, leading to largest scattering cross section.

In summary, only the resonance region $M_\text {DM}\sim M_{Z'}/2$ can satisfy direct detection limits by requiring the free parameter set yielding correct relic density for both scalar and fermion DM. Moreover, in the resonance region, we find that a larger effective absolute value of $B-L$ charge, i.e., $|Q_{H_1^0}|=|Q-1|$ for scalar DM and $|Q_{F_1}|=|V_{F_1}|=|(1-Q)/2|$ for fermion DM, actually leads to a smaller spin-independent cross section. And comparing with scalar DM, fermion DM is easier to escape direct detection limits.

5.3 Indirect detection

According to the results of direct detection in Sect. 5.2, the so-called secluded DM scenario $M_\text {DM}>M_{Z'}$ [91, 92] has already been excluded. So in the $B-L$ scotogenic Dirac model, DM pair can only annihilate into SM fermion pair through the gauge boson $Z'$ in the s-channel nowadays. Then, this leads to high energy gamma-rays, which is detectable at Fermi-LAT [93], H.E.S.S. [94, 95], and the forthcoming CTA [96, 97]. Provided DM 100% annihilating into $b\bar{b}$ or $\tau ^+\tau ^-$ with an annihilation cross section of $3\times 10^{-26}~\text {cm}^3\text {s}^{-1}$, Fermi-LAT has excluded DM mass below $100~\mathrm{GeV}$ [93]. As for TeV-scale DM, H.E.S.S. has excluded $500~\mathrm{GeV}\lesssim M_\text {DM}\lesssim 2~\mathrm{TeV}$ when DM solely annihilates into $\tau ^+\tau ^-$ final states also assuming an annihilation cross section of $3\times 10^{-26}~\text {cm}^3\text {s}^{-1}$ [94, 95]. To illustrate prospect of indirect detection, we also take into account the upcoming CTA experiment [96] and considering the most optimistic limits in Ref. [97], which in principle could probe $M_\text {DM}\lesssim 10~\mathrm{TeV}$ for a 100% branching ratio into $\tau ^+\tau ^-$ and an annihilation cross section of $3\times 10^{-26}~\text {cm}^3\text {s}^{-1}$. Since for TeV-scale DM, the $\tau ^+\tau ^-$ channel provides the most stringent limit, we will take this channel to illustrate for simplicity. A more appropriate pathway is taking into all annihilation channels [32, 103].

Fig. 10

Annihilation cross section for scalar DM $H_1^0$ into $\tau ^+\tau ^-$ final state. Left: as a function of $M_{Z'}$ for $M_{H_1^0}=1~\mathrm{TeV}$, $Q=-\frac{1}{4},-4,\frac{3}{2}$. Right: as a function of $M_{H_1^0}$ for $M_{Z'}=2,4,6~\mathrm{TeV}$, $Q=-\frac{1}{4}$. Here, $g'$ is determined by yielding correct relic density $\Omega h^2\simeq 0.12$. Experimental bounds are derived from Fermi-LAT (green) [93], H.E.S.S. (pink) [94], and CTA (brown) [97]

The differential photon flux in any angular direction can be obtained by

$$\begin{aligned} \frac{d \Phi _\gamma (\Delta \Omega )}{d E}(E_\gamma )= \frac{1}{4\pi } \frac{\langle \sigma v\rangle }{2 M_\text {DM}^2}\sum _f \text {BR}_f \frac{d N_\gamma ^f}{d E_\gamma }\cdot J_{ann}, \end{aligned}$$

(24)

where $\hbox {BR}_f$ is the branching ratio of $Z'$ into fermion pair f, $d N_\gamma ^f/d E_\gamma$ is the differential $\gamma$-spectrum from fermion pair f, $J_{ann}$ is the astrophysical J-factor. And the annihilation cross sections are presented in Eq. (18) for scalar DM and Eq. (19) for fermion DM, respectively. Because the annihilation cross section today is velocity suppressed for scalar DM, the corresponding indirect detection signatures are usually neglected [98]. As for fermion DM, the annihilation cross section is not suppressed, hence indirect detection signatures are in principle promising. Furthermore, since only the resonance region $M_\text {DM}\sim M_{Z'}/2$ is viable under constraints from relic density and direct detection, the naive estimation of DM annihilation cross section with Eq. (17), i.e., $\langle \sigma v \rangle \simeq 3\times 10^{-26}~\text {cm}^3\text {s}^{-1}$, is invalid [99, 100]. In this case, the annihilation cross section could be enhanced via the Breit-Wigner mechanisms [101, 102], so as a consequence the indirect detection limits are strengthened.

We start with scalar DM as well. The annihilation cross sections for the $\tau ^+\tau ^-$ channel are shown in Fig. 10. It is obvious that even with Breit-Wigner enhancement, the annihilation cross sections are still several orders of magnitudes lower than future CTA limits. In left panel of Fig. 10, we show the impact of $B-L$ charge Q. Off the resonance region, one has a similar cross section for $Q=-\frac{1}{4}$ and $Q=\frac{3}{2}$, while the cross section is much smaller for $Q=-4$ due to sizable suppression of $\tau ^+\tau ^-$ branching ratio. When the gauge boson $Z'$ is almost on-shell, the cross section could reach about $2\times 10^{-30}\text {cm}^3\text {s}^{-1}$, $1\times 10^{-30}\text {cm}^3\text {s}^{-1}$ and $1\times 10^{-31}\text {cm}^3\text {s}^{-1}$ for $Q=-\frac{1}{4}$, $Q=-4$ and $Q=\frac{3}{2}$ respectively. In right panel of Fig. 10, we show $\langle \sigma v\rangle _{\tau ^+\tau ^-}$ as a function of $M_{H_1^0}$ by fixing $Q=-\frac{1}{4}$ and varying $M_{Z'}=2,4,6~\mathrm{TeV}$. Here, we also impose $|M_{H_1^0}-M_{Z'}/2|\lesssim 500~\mathrm{GeV}$ to show corresponding resonance region. Although the cross sections are similar when $M_{H_1^0}$ far away from $M_{Z'}$, we find that as $M_{Z'}$ increases, the maximum(minimum) value decreases(increases), and the resonance width becomes larger.

Fig. 11

Same as Fig. 10, but for fermion DM $F_1$

Now, we consider fermion DM. The results are shown in Fig. 11. In spite of the cross sections are much larger than scalar DM, the effects of varying $B-L$ charge Q and $M_{Z'}$ are similar. In left panel of Fig. 11, we show the benchmark scenario $M_{F_1}=1~\mathrm{TeV}$ with $Q=-\frac{1}{4},-4,\frac{3}{2}$. Close to $M_{H_1^0}=M_{Z'}/2$, the maximum values of $\langle \sigma v \rangle _{\tau ^+\tau ^-}$ for $Q=-\frac{1}{4},-4$ are enhanced to the order of $10^{-24}\text {cm}^3\text {s}^{-1}$, which has already exceed Fermi-LAT limit. The maximum value for $Q=\frac{3}{2}$ is one order of magnitude smaller. Although this is lower than Fermi-LAT limit, it is still larger than H.E.S.S. limit. Future CTA limit will push the limit down to about $5\times 10^{-27}\text {cm}^3\text {s}^{-1}$, hence large part of the resonance regions are detectable. Moreover, when $M_{F_1}$ is slightly lighter than $M_{Z'}/2$, the cross sections dramatically diminish to $3\times 10^{-29}\text {cm}^3\text {s}^{-1}$, $8\times 10^{-30}\text {cm}^3\text {s}^{-1}$,$6\times 10^{-31}\text {cm}^3\text {s}^{-1}$ for $Q=\frac{3}{2},-\frac{1}{4},-4$ respectively. Therefore, this region can easily escape indirect detection limit. Right panel of Fig. 11 shows $\langle \sigma v \rangle _{\tau ^+\tau ^-}$ as a function of $M_{H_1^0}$, while we fix $Q=-\frac{1}{4}$ and vary $M_{Z'}=2,4,6~\mathrm{TeV}$. One can see that only $M_{Z'}=2~\mathrm{TeV}$ could exceed Fermi-LAT limit, since as $M_{H_1^0}(M_{Z'})$ getting bigger, the Fermi-LAT limit is less stringent while the enhancement effect is weaker. With clearly enhanced cross section, the resonance regions are all within the reach of H.E.S.S. and CTA.

In a nutshell, constraints from indirect detection can be neglected for scalar DM, even taking into account the Breit-Wigner enhancement effect. Meanwhile, for fermion DM, the indirect detection experiments are sensitive to the region close to $M_{F_1}\gtrsim M_{Z'}/2$, and the region $M_{F_1}\lesssim M_{Z'}/2$ can avoid indirect detection limit.

6 Results

In previous Sects. 3–5, we perform a separated analysis on certain observables. Now, in this section, we first combine the separated analysis result from collider signature, dark radiation, DM relic density, direct and indirect detection for some benchmark scenarios. Based on the benchmark results, a scanning of the resonance region $M_\text {DM}\simeq M_{Z'}/2$ is then carried out.

6.1 Combined results

Some general comments before we present the combined results. First, the LEP precise measurement, dark radiation observable $\Delta N_\text {eff}$ and direct detection all set limits on $M_{Z'}/g'$, but there dependence on $B-L$ charge Q is different. The benchmark scenarios $Q=-\frac{1}{4},-4,\frac{3}{2}$ with $M_\text {DM}=1~\mathrm{TeV}$ will show this. Second, as already shown in Fig. 3, the LHC@13TeV dilepton signature is sensitive to $M_{Z'}\lesssim 4~\mathrm{TeV}$. So for DM heavier than 2 TeV, there will be no constraint from LHC@13TeV. We choose one benchmark scenario $M_\text {DM}=3~\mathrm{TeV}$ with $Q=-\frac{1}{4}$ to illustrate this. Third, the perturbation condition $|g'\cdot Q_\text {max}|<\sqrt{2\pi }$ is also needed to keep the model perturbative. For $Q=-4$, $|Q_\text {max}|=|Q_\chi |=5$, so $g'<\sqrt{2\pi }/Q_\text {max}\approx 0.5$ is required. For the other two choices $Q=-\frac{1}{4},\frac{3}{2}$, $g'<1$ will satisfy the perturbation condition.

Fig. 12

Combined results in the $g'$-$M_{Z'}$ plane for scalar DM $M_{H_1^0}$. a $M_{H_1^0}=1~\mathrm{TeV}$ with $Q=-\frac{1}{4}$; b $M_{H_1^0}=1~\mathrm{TeV}$ with $Q=-4$; c $M_{H_1^0}=1~\mathrm{TeV}$ with $Q=\frac{3}{2}$; d $M_{H_1^0}=3~\mathrm{TeV}$ with $Q=-\frac{1}{4}$. Experimental bounds are obtained from 13 TeV LHC dilepton signature (blue) [60], LEP precise measurement (cyan) [53], dark radiation observable $\Delta N_\text {eff}$ (red) [74], direct detection limit from PandaX2017 (purple) [88] and future XENON1T(2t$\cdot$y) (gray) [89]. With fixed DM mass and $B-L$ charge Q, the green lines further satisfy relic density condition $\Omega h^2\simeq 0.12$ [74]. To make sure the model is perturbative, we also require $|g'\cdot Q_\text {max}|<\sqrt{2\pi }$ (black)

Choosing four benchmark scenarios, the combined results for scalar DM $H_1^0$ are depicted in Fig. 12. As discussed in Sect. 5.3, no parameter space for scalar DM is within the reach of indirect detection, so corresponding limits are missing in these figures. The benchmark scenario $M_{H_1^0}=1~\mathrm{TeV}$ with $Q=-\frac{1}{4}$ is shown in Fig. 12a. In this scenario, it is clear that nowadays the most stringent constraint is from LHC@13TeV when $M_{Z'}\lesssim 2800~\mathrm{GeV}$ and it has already excluded all parameter space. Therefore, no signature is expected to be observed at ongoing direct detection experiments. The benchmark scenario $M_{H_1^0}=1~\mathrm{TeV}$ with $Q=-4$ is shown in Fig. 12b. In this scenario, constraints from $\Delta N_\text {eff}$ and direct detection are already severer than LHC@13TeV even in the low mass region. And only the resonance region $M_{Z'}\sim 2~\mathrm{TeV}$ could avoid all constraints. The result for $M_{H_1^0}=1~\mathrm{TeV}$ with $Q=\frac{3}{2}$ is shown in Fig. 12c. In this case, although the $\Delta N_\text {eff}$ limit is severer than LEP, it is still weaker than PandaX2017. And the parameter space is again fully excluded by PandaX2017 and LHC@13TeV. The benchmark scenario for heavier DM $M_{H_1^0}=3~\mathrm{TeV}$ with $Q=-\frac{1}{4}$ is shown in Fig. 12d. With less stringent limits from direct detection, now the resonance region can escape PandaX2017 limit, but most region are still in the reach of XENON1T(2t$\cdot$y).

Fig. 13

Same as Fig. 12, but for fermion DM $F_1$. In addition, the pink and brown curves are excluded by H.E.S.S. [94] and future CTA [97] indirect detection experiments

Fig. 14

Scanning results for scalar DM $H_1^0$. a Viable parameter space for relic density in the $g'$-$M_{Z'}$ plane. The solid, dotted and dashed lines (from the bottom up) correspond to LHC@13TeV exclusion limits for $Q=-\frac{1}{4},\frac{3}{2}$ and ${-4}$ respectively. b Spin-independent cross section as a function of $M_{H_1^0}$. c Annihilation cross section into $\tau ^+\tau ^-$ final states as a function of $M_{H_1^0}$. d Allowed parameter space after applying all constraints. The gray points are excluded

Then, the combined results for fermion DM $F_1$ is shown in Fig. 13. Comparing with scalar DM, the indirect detection experiments could also exclude certain parameter space. Instead of deriving the whole exclusion region in the $g'$-$M_{Z'}$ plane, we only get the excluded curves satisfying relic density condition, which can be easily obtained from Fig. 11a. The benchmark scenario $M_{F_1}=1~\mathrm{TeV}$ with $Q=-\frac{1}{4}$ is shown in Fig. 13a. It is amazing that the combined result of LHC@13TeV and H.E.S.S. has already excluded $M_{F_1}\gtrsim M_{Z'}/2$. So neither direct detection nor indirect detection experiment will observe positive signature. And the only way to probe the allowed region $M_{F_1}\lesssim M_{Z'}/2$ is via dilepton signature at ongoing LHC. Similar arguments are also true for benchmark scenario $M_{F_1}=1~\mathrm{TeV}$ with $Q=\frac{3}{2}$ as shown in Fig. 13c. Note in this scenario, the constraint from $\Delta N_\text {eff}$ is more stringent than current direct detection PandaX2017. The result for $M_{F_1}=1~\mathrm{TeV}$ with $Q=-4$ is shown in Fig. 13b. In this scenario, the $M_{F_1}\gtrsim M_{Z'}/2$ region, i.e. $1880~\mathrm{GeV}\lesssim M_{F_1}\lesssim 1995 ~\mathrm{GeV}$, is also viable. And either direct detection or indirect detection experiment is hopefully to have some positive signature. Figure 13d shows the result of $M_{F_1}=3~\mathrm{TeV}$ with $Q=-\frac{1}{4}$. In this scenario, the constraints are relative loose. Hence, one can see that $M_{Z'}$ can be over about 1 TeV away from the resonance nowadays. Again similar as Fig. 13b, no region could be cross checked by direct and indirect detection.

Fig. 15

Same as Fig. 14, but for fermion DM $F_1$

6.2 Scanning results

Based on the combined results of benchmark scenario, one is aware that the resonance region $M_\text {DM}\simeq M_{Z'}/2$ is the viable parameter space to satisfy all experimental constraints. Hence, instead of fully scan the whole parameter space, we focus on the resonance region $M_\text {DM}\simeq M_{Z'}/2$. We fix $Q=-\frac{1}{4},-4$ or $\frac{3}{2}$, and then scan in the following range

$$\begin{aligned}&g'\in [0,1], M_\text {DM}\in [500,5000]~\mathrm{GeV}, M_{Z'}-2M_\text {DM} \nonumber \\&\quad \in [-50,100]~\mathrm{GeV}\end{aligned}$$

(25)

for both scalar and fermion DM. For scalar DM, the direct detection could exclude the samples out of the mass interval $M_{Z'}-2M_{H_1^0}\in [-50,100]~\mathrm{GeV}$. In principle, the mass interval $|M_{Z'}-2M_{F_1}|$ for fermion DM could be $\mathcal {O}(\mathrm{TeV})$, but the indirect detection experiments are only sensitive to the near resonance region. We focus on this region so as well to probe the less considered indirect signatures. During the scan, we require the relic density satisfies Planck result in $3\sigma$ range, i.e., $0.1118<\Omega h^2<0.1280$. For each case, we generate over $5\times 10^4$ samples, and get about $1\times 10^3$ survived samples. Constraints from LHC dilepton signature, direct and indirect detection are further taken into account to acquire the allowed parameter space at present, because these provide the most stringent constraints under certain circumstances according to the combined results.¹

The scanning results for scalar DM $H_1^0$ are shown in Fig. 14. Figure 14a presents the result in the $g'$-$M_{Z'}$ plane. Basically speaking, $g'$ increases as $M_{Z'}$ increases. For $Q=\frac{3}{2}$, $g'$ is larger than 1 when $M_{Z'}$ is larger than about $4~\mathrm{TeV}$, and most samples are already excluded by LHC@13TeV. For $Q=-\frac{1}{4}$, LHC@13TeV could fully exclude $M_{Z'}\lesssim 2~\mathrm{TeV}$, while for $Q=-4$, $M_{Z'}$ downs to about 1 TeV is still possible. Figure 14b shows the predicted spin-independent DM-nucleon scattering cross section as a function of $M_{H_1^0}$. Clearly, the case of $Q=\frac{3}{2}$ is fully excluded by PandaX2017. Although for $Q=-\frac{1}{4}$, most samples could survive under PandaX2017, most samples are within the reach of XENON1T(2t$\cdot$y). And if no signature is observed by XENON1T(2t$\cdot$y), $M_{H_1^0}$ should fall in the range $[1200,4400]~\mathrm{GeV}$. As for $Q=-4$, most samples could escape direct detection limits even in the future. The annihilation cross section into $\tau ^+\tau ^-$ final states is also presented in Fig. 14c for completeness, even though it is several orders of magnitude under future CTA limit. The survived simples after imposing all constraints are depicted in Fig. 14d. For $Q=\frac{3}{2}$, no survived points are obtained, due to tight direct detection constraints. For $Q=-\frac{1}{4}$, one needs $M_{H_1^0}\gtrsim 1.2~\mathrm{TeV}$ to escape all constraints, while for $Q=-4$, sub-TeV $H_1^0$ with small $g'\sim 0.01$ is still viable.

The scanning results for fermion DM $F_1$ are shown in Fig. 15. Viable parameter space for relic density in the $g'$-$M_{Z'}$ plane are shown in Fig. 15a. Comparing with scalar DM, a smaller $g'$ is usually needed for same value of $M_\text {DM}$ and Q. Hence, it is more easier for fermion DM to satisfy the constraints from LHC@13TeV. Especially for $Q=-4$, no sample is excluded by LHC@13TeV. According to direct detection results in Fig. 15b, only small amount of samples with $M_{F_1}\lesssim 1~\mathrm{TeV}$ are excluded. As for indirect detection shown in Fig. 15c, the most promising case is $Q=-\frac{1}{4}$. With few samples excluded by H.E.S.S., about half of the samples are within reach of future CTA experiment for $Q=-\frac{1}{4}$. For the other two cases, a little of the samples could lead to observable indirect signatures even in the future. Samples passing through all constraints are shown in Fig. 15d. It is clear that most samples are survived for all three choices of Q.

7 Conclusions

In this paper, the phenomenology of $Z'$ portal scalar and Dirac fermion DM in $B-L$ scotogenic Dirac model is comprehensively studied. With unconventional $B-L$ charge Q of right-handed neutrino $\nu _R$, the Dirac neutrino mass induced at one-loop level, DM stability and anomaly cancellations are naturally realised. Depending on the mass spectrum, both scalar DM $H_1^0$ with $B-L$ charge $Q_{H_1^0}\simeq Q-1$ and fermion DM $F_1$ with $Q_{F_{1L}}=1$, $Q_{F_{1R}}=-Q$ are allowed. Three typical values $Q=-\frac{1}{4},-4,\frac{3}{2}$ are chosen to illustrate the testability of this model.

Phenomenology aspects are detail examined in collider signature, dark radiation, DM relic density, direct and indirect detection. Generally speaking, constraints from dilepton search at 13 TeV LHC usually place the most stringent exclusion limits in the lower mass region, while direct detection limits becomes the most stringent in the higher mass region. Combining all the constraints leads to the viable parameter space falling in the resonance region $M_\text {DM}\sim M_{Z'}/2$. In addition, indirect detection could also exclude the region very close to the resonance region $M_{F_1}\gtrsim M_{Z'}/2$ for Dirac fermion DM. Notably, direct detection is sensitive to the region away from the resonance region, so next generation direct and indirect detection experiment might hard to both have positive signatures provided Dirac fermion DM. Meanwhile, this model also predict testable $\Delta N_\text {eff}$ for TeV-scale $Z'$, which is an intrinsic character that can be used to distinguish this Dirac model from the Majorona model for neutrino mass.

Scanning on the resonance region, allowed parameter space is then obtained. As for scalar DM, the case of $Q=\frac{3}{2}$ is already fully excluded by PandaX2017, while for $Q=-\frac{1}{4}$, most samples allowed at present are within the reach of future XENON1T(2t$\cdot$y). For $Q=-4$, sub-TeV DM is still viable. Comparing with scalar DM, the Dirac fermion DM is easier to satisfy all constraints. For $Q=-\frac{1}{4}$, it is promising at future indirect detection CTA experiment, meanwhile it is less promising for $Q=-4$ and $\frac{3}{2}$. From the above phenomenology results, we also expect that this model can be verified by determine the parameter Q via precise measurements of $Z'$ branching ratio. Meanwhile, the observation of $\Delta N_\text {eff}$, direct and indirect detection are also possible to confirm the value of Q, and an unconventional Q would strongly support this model.