Inverse seesaw and dark matter in a gauged B − L extension with flavour symmetry

We propose a model which generates neutrino masses by the inverse seesaw mechanism, provides a viable dark matter candidate and explains the muon (g−2) anomaly. The Standard Model (SM) gauge group is extended with a gauged U(1)B−L as well as a gauged U1Lμ−Lτ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \mathrm{U}{(1)}_{{\mathrm{L}}_{\mu }-{\mathrm{L}}_{\tau }} $$\end{document}. While U1Lμ−Lτ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \mathrm{U}{(1)}_{{\mathrm{L}}_{\mu }-{\mathrm{L}}_{\tau }} $$\end{document} is anomaly free, the anomaly introduced by U(1)B−L is cancelled between the six SM singlet fermions introduced for the inverse seesaw mechanism and four additional chiral fermions introduced in this model. After spontaneous symmetry breaking the four chiral fermionic degrees of freedom combine to give two Dirac states. The lightest Dirac fermion becomes stable and hence the dark matter candidate. We focus on the region of the parameter space where the dark matter annihilates to the right-handed neutrinos, relating the dark matter sector with the neutrino sector. The U1Lμ−Lτ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \mathrm{U}{(1)}_{{\mathrm{L}}_{\mu }-{\mathrm{L}}_{\tau }} $$\end{document} gauge symmetry provides a flavour structure to the inverse seesaw framework, successfully explaining the observed neutrino masses and mixings. We study the model parameters in the light of neutrino oscillation data and find correlation between them. Values of some of the model parameters are shown to be mutually exclusive between normal and inverted ordering of the neutrino mass eigenstates. Moreover, the muon (g − 2) anomaly can be explained by the additional contribution arising from U1Lμ−Lτ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \mathrm{U}{(1)}_{{\mathrm{L}}_{\mu }-{\mathrm{L}}_{\tau }} $$\end{document} gauge boson.


Introduction
Even though the Standard Model (SM) of elementary particles has been very successful in describing the nature around us, it is unable to explain all the observed phenomena. The main puzzles which SM can not explain are the presence of neutrino masses and mixings observed in the neutrino oscillation data [1][2][3][4][5][6][7][8][9][10][11], the matter antimatter asymmetry of the universe [12][13][14][15] and the presence of non baryonic matter, also called the dark matter (DM) [16][17][18][19][20][21][22][23]. In this work, we study in detail two of the above mentioned puzzles, viz., the presence of masses and mixings of neutrinos and DM, and propose an extension of the SM that can successfully describe the two observed phenomena. One of the simplest ways to explain the tiny neutrino masses naturally is to extend the SM by an additional U(1) B−L gauge symmetry which dictates the introduction of three RH-neutrinos due to anomaly cancellation. However, the price one pays in this model is that in order to get the correct neutrino masses either the RH neutrinos masses have to be of the order of GUT scale or the corresponding Yukawa couplings have to be kept small giving a very small active-sterile mixing angle ( m D M R ∼ 10 −6 ) [24][25][26][27][28][29]. Hence, it is difficult to test this model at the collider. The other draw-back of the U(1) B−L extension with three RH neutrinos is that it has no prediction for neutrino mixing angle. The first concern regarding testability of the seesaw mechanism can be addressed using the so called inverse seesaw mechanism (ISS) , wherein one can have lower the mass scale of RH neutrinos while JHEP08(2018)062 keeping the Yukawa couplings large. The second concern regarding correct prediction of the neutrino mixing pattern can be addressed by extending the model further by a flavour symmetry. Although, in principle one could add a discrete horizontal symmetry, 1 we choose in this work U(1) Lµ−Lτ gauged flavour symmetry to explain the mixing angles of the light neutrinos [29,[54][55][56][57][58][59]. Moreover, this has an added benefit as the extra neutral gauge boson Z µτ coming from the U(1) Lµ−Lτ gauged flavour symmetry can provide additional contribution to the muon magnetic moment and thus explain the muon (g − 2) data [60][61][62]. Furthermore, the U(1) Lµ−Lτ symmetry reduces the number of free parameters in the neutrino sector and provides a very peculiar structure to the neutrino mass matrices. Hence, we expect sharp correlations among the parameters that satisfy the neutrino oscillation data. We have analysed the model and present results both for the normal hierarchy (NH) where the third neutrino mass eigenstate is taken as the heaviest, as well as the inverted hierarchy (NH) where the third neutrino mass eigenstate is assumed to be the lightest one.
Since we introduce two additional gauge symmetries U(1) B−L and U(1) Lµ−Lτ , we need to choose the U(1) gauge group charges of all fermions in such a way that the chiral anomalies cancel consistently. While U(1) Lµ−Lτ is known to be anomaly free [54][55][56][57], the gauged U(1) B−L extension of the SM is anomalous and hence extra fermionic degrees of freedom are needed to make the theory anomaly free. In our model, we require six additional fermion singlets of SM for the neutrino mass generation via ISS mechanism. In addition, we also need a viable DM candidate. All three requirements are met consistently by introducing three usual RH neutrinos N α s with B − L charge −1, three additional RH neutrinos N α s having opposite B − L charge +1 and four chiral fermions ξ L , η L , χ 1R and χ 2R with fractional B − L and L µ − L τ charges, such as 4 3 , 1 3 , − 2 3 , − 2 3 and 1 3 , 4 3 , 1 3 , 4 3 , respectively. The above charge assignment cancels out all the B − L and L µ − L τ anomalies consistently while allowing all the necessary terms in the Lagrangian for the ISS mechanism. Besides after the spontaneous symmetry breaking, the lightest of the two massive Dirac states created out of the four chiral fermions ξ L , η L , χ 1R and χ 2R is stable in this model and becomes the DM candidate of the universe. The anomaly cancellation by assigning fractional B − L charges to the additional fermions has been proposed in [63]. Thereafter, both thermal as well as non-thermal dark matter phenomenologies in this new B − L model have been studied in [63][64][65] . In this work, we have mainly concentrated on that portion of the parameter space where our DM candidate dominantly annihilates to heavy as well as light neutrinos, which enables us to find a strong correlation between the neutrino mass generation and DM freeze-out mechanism.
Rest of the work is arranged in the following manner. In section 2 we describe the model. In section 3 we present our result for neutrino and dark matter sector. Finally, in section 4 we conclude.

Model
In this section, we will describe the present model briefly. In the present work, we have extended all three sectors of SM, namely the gauge sector, the fermionic sector as well as JHEP08(2018)062 the scalar sector. The additional particle content of the present model is shown in table 1. The extensions in the fermionic sector are necessary for anomaly cancellation while that in the scalar sector is required for spontaneous breaking of the additional symmetries and the mass generation of the extra fermionic fields. We have extended the gauge sector by imposing two additional local U(1) gauge symmetries. Thus, the complete gauge group under which the Lagrangian remains invariant before spontaneous symmetry breaking is The addition of these extra U(1) symmetries introduce new anomalies in the theory. We know that the SM extended by the gauged U(1) Lµ−Lτ symmetry is anomaly free [54][55][56][57]. Here, actually anomaly cancels between second and third generations of leptons. On the other hand, gauged U(1) B−L extension of the SM is anomalous and one thus needs to add extra fermionic degrees of freedom to the particle content of the SM to make the theory anomaly free. The minimal way to cancel B − L anomaly is by adding three RH neutrinos having B − L charge −1. However, in the present work our motivation is to study both inverse seesaw mechanism [66][67][68][69] and WIMP dark matter within a complete model. Therefore, besides the three usual RH neutrinos (N α ) with B − L charge −1, we have introduced three more RH neutrinos (N α ) having opposite B − L charge i.e. +1. As we have already mentioned earlier that gauged L µ − L τ extension of the SM is non-anomalous, hence we have assigned the L µ − L τ charges of RH neutrinos in a such way that their contribution to axial vector anomaly [70,71] and mixed gravitational-gauge anomaly [72,73] cancels among themselves. On the other hand, the B − L anomaly cancellation requires more fermionic states with appropriate B − L charges. We thus add four chiral fermions ξ L , η L , χ 1R and χ 2R with fractional B − L charges 4 3 The L µ − L τ charges of these chiral fermionic states cannot be zero due to the presence of mixed anomaly like U(1) B−L [U(1) Lµ−Lτ ] 2 . We will see below that their charges will be fixed from the anomaly cancellation conditions coming from U(1) 3 and Gravity 2 U(1) Lµ−Lτ , respectively. Let us assume the L µ − L τ charges of ξ L , η L , χ 1R and χ 2R are a, b, c and d. With this charge assignment we will check various anomaly cancellation conditions for both U(1) B−L and U(1) Lµ−Lτ gauge groups.
contributions from RH neutrinos  In this work, we have adopted (a, b, c, d) = (1/3, 4/3, 1/3, 4/3). Furthermore, in addition to the usual SM Higgs doublet, we have also introduced three singlet scalars φ i (i = 1 to 3) with properly chosen U(1) B−L and U(1) Lµ−Lτ charges. Among these scalars, φ 1 and φ 2 are required to generate masses for the chiral fermions (ξ L , η L , χ 1R and χ 2R ) in a gauge invariant manner while the remaining one, φ 3 , is important for writing the interaction terms between N α and N β . The later ones are required for successful implementation of inverse seesaw mechanism within the present scenario. In where L SM is the SM Lagrangian and L N represents the Lagrangian for the extended neutrino sector. The extended scalar sector Lagrangian is denoted by the third and fourth term of the above equation while the dark sector Lagrangian, containing the interaction terms of chiral fermions, is defined by the term L DM . Finally, last two terms are the kinetic terms for the B − L and L µ − L τ gauge bosons in terms of the respective field strength tensor. Below we have discussed in detail about all the parts of the Lagrangian written in eq. (2.9).

Extended scalar sector
The Lagrangian for the extended scalar sector of the present model is given in eq. (2.9). The potential V(φ h , φ 1 , φ 2 , φ 3 ) appearing in eq. (2.9) contains all types of interaction terms among the scalar fields, which are allowed by SU U(1) Lµ−Lτ gauge symmetries. Therefore, the expression of V(φ h , φ 1 , φ 2 , φ 3 ) can be written as As we want the gauge symmetry to be spontaneously broken to SU(3) c × U(1) em i.e., coefficients of all the quadratic terms must be negative, i.e., µ 2 h < 0 and µ 2 i < 0 (for i = 1 to 3). It is now well established that the only spin zero resonance observed in the LHC has properties very similar to the SM Higgs boson. This actually tells us that the mixing between Higgs doublet (φ h ) and the other scalars (φ i ) will be inevitably small. Therefore in the current work, just for the sake of simplicity, we take the mixing angles between the SM-like Higgs boson and the other non-standard scalars as equal to zero, i.e., ρ i = 0 (for i = 1, 2, 3). On the other hand, we need to consider mixing among the remaining three (BSM) scalars φ 1 , φ 2 and φ 3 . Handling the mixing among three scalars simultaneously is a tedious job, hence we will take the vacuum expectation value (VEV) of φ 1 large enough and correspondingly the mass of the neutral component such that it will have negligible effect on the relic density for the mass range we are considering in the current work. Moreover, we particularly focus on the parameter space of the model where DM and heavy neutrinos as well as light neutrinos are one to one related which means that a reasonable portion of DM annihilate to these heavy and light neutrinos. Therefore, we will consider only mixing between φ 2 and φ 3 scalars, i.e., we take λ 23 = 0 while all the other mixing terms we will neglect to focus on the above mentioned parameter space. Although we take all the other quartic mixing terms except λ 23 to be equal to zero, but mixing term will be generated due to the presence of the trilinear couplingμ. Sinceμ is a dimensionful quantity, its magnitude can be of any order. In the current, work we have adopted small value forμ so that the mixing term generated due to the trilinear term can be safely neglected.
After getting VEV, the neutral components of all the scalars take the following form, where j = 1 to 3 and G h represents the massless Goldstone boson which gives mass to the SM neutral gauge boson Z. On the other hand, A j corresponds to the CP odd neutral component of the singlet scalar field φ j . Among them, A 3 and one linear combination of A 1 , A 2 will be massless as those are responsible for the mass generation of the L µ − L τ and B − L gauge bosons, respectively. The mass of other CP odd state (A), which is orthogonal JHEP08(2018)062 to the massless state, is given by , the ratio between the two VEVs v 1 , v 2 and since M 2 A > 0, this implies µ < 0. Moreover, as both the singlet scalars φ 1 , φ 2 have nonzero B − L charges, hence, they both contribute to the mass of U(1) B−L gauge boson Z BL and it has the following form, . (2.13) The remaining singlet scalar φ 3 is the only member in the scalar sector which has nonzero L µ − L τ charge hence the mass of the gauge boson Z µτ appears when φ 3 gets a VEV, i.e., where we have denoted v 3 by v µτ . As we have considered only the mixing between φ 2 and φ 3 , hence the mass matrix with respect to the basis (H 2 , H 3 ) takes the following form, (2.15) By diagonalizing the above mass matrix one can easily obtain mass basis (physical states) from the gauge basis through an orthogonal transformation by the mixing angle β in the following manner, Now, we can write down the quartic couplings related to φ 2 , φ 3 in terms of the masses M h 2 , M h 3 and the mixing angle β and have the following form, (2.17) JHEP08(2018)062

Extended neutrino sector and inverse seesaw
Here we have shown only those terms in the Lagrangian for the neutrino sector which are necessary for the inverse seesaw mechanism. All the terms in the Lagrangian are allowed by both U(1) B−L and U(1) Lµ−Lτ gauge symmetries.
where D X µ represents the covariant derivative for the field X (X = N α , N α , ν α ). The first three terms are the kinetic terms for N α , N α and ν α , while the last term is the Yukawa interaction term (Dirac type) between the SM lepton doublet (L α = (ν α l α ) T ), Higgs doublet and the RH neutrino N α . All the other terms in the above Lagrangian are the interaction terms between N α , N β and the Majorana mass terms of N . The general form of the inverse seesaw Lagrangian is given by, Therefore, from the above Lagrangian one can construct a 9 × 9 mass matrix sandwiched between the basis states (2.20) Comparing eq. (2.19) with the Lagrangian L N , one can easily find the structure of the individual matrices, namely, m D , M N and µ as Although all the elements allowed by the imposed gauge symmetries can be in general complex numbers, however by redefining the phases of the fermionic fields one can check that there can only be three independent complex phases possible. Consequently, we have chosen (2,2), (3,3) elements of M N matrix and (2,3) 2 element of µ matrix as complex numbers.
In the above mass matrix given by eq. (2.20), for simplicity we have neglected the Majorana mass term M R of N α in the (2,2) element of M, although it is allowed by both U(1) B−L and U(1) Lµ−Lτ symmetries. This is because, if we consider M R with the same order of magnitude as M N , then in the limit M N > m D µ (the condition for inverse seesaw mechanism), this term has a negligible effect on the light neutrino mass matrix [67].
After diagonalising the 9 × 9 mass matrix M we get the three light neutrinos and six heavy neutrinos with the following expressions of mass matrices, The physical basis (ν c l , N H , N H ) can be written in terms of the (ν c , N , N ) basis in the following manner [66], In determining a 1 , a 2 and a 3 we have taken the model parameters value as given in the appendix A.
Here m D , M N and µ have a very particular structure due to the (µ − τ ) flavour symmetry. For our convenience, we have defined few new variables which are, where, all the parameters defined above have dimensions of mass. In section 3.2, we will show the allowed regions among the different parameters of the above mentioned mass matrices (eq. (2.21)-(2.23)) after applying the neutrino oscillation data constraints for both NH and IH. Constraints on mixing angles, mass square differences and the sum of all the light neutrinos, which we have followed in determining the allowed parameter space are as follows, • there is a bound on the sum of all three light neutrinos from cosmology which is,  [74].
In the above mass matrices m D , M R and µ, many elements are zero. Therefore, when we will apply the above constraints, the oscillation data will put severe constraints on the parameter values and we will get nice correlations among the parameters which we will see in the result section.

Dark matter sector
As discussed earlier, we need to introduce four chiral fermions (ξ L , η L , χ 1R and χ 2R ) with fractional B − L and L µ − L τ charges (see table 1) to make the present model anomaly free. The Lagrangian for these exotic fermionic states has been denoted by L DM in eq. (2.9).
Since these chiral fermions are singlet under the SM gauge group, hence L DM contains only those terms which are invariant under U(1) B−L × U(1) Lµ−Lτ gauge group: The last term in the above equation shows that in order to write Dirac mass terms for all the chiral fermions, one needs to have at least two scalar fields φ 1 and φ 2 with different B − L charges. This is the main reason why we have introduce more than one scalar field to break the U(1) B−L symmetry spontaneously. After symmetry breaking of U(1) B−L symmetry by the VEVs of φ 1 and φ 2 , the Dirac mass matrix with respect to the basis states (ξ L , η L ) and (χ 1R , χ 2R ) takes the following diagonal form, which is possible due to the assignment of particular L µ − L τ charges to these chiral fermions. (2.28) From the above mass matrix one can easily construct two Dirac-type fermionic states Σ 1 = ξ L χ 1 R and Σ 2 = η L χ 2 R which are the combinations of these exotic chiral fermions and having masses respectively. Between these two Dirac fermions, the lightest one will be stable and can be a dark matter candidate. Therefore, for definiteness, throughout this work we have considered M 1 < M 2 and hence Σ 1 is our dark matter candidate. Moreover, we can also write down the Yukawa couplings γ 1 and κ 1 in terms of masses M 1 , M 2 of the fermions Σ 1 , Σ 2 in the following way, Feynman diagram for the spin independent elastic scattering cross section between dark matter candidate Σ 1 and the nucleon (N ) mediated by Z BL .

Spin independent scattering cross section of Σ 1
In this work, our dark matter candidate Σ 1 can talk to the quark sector only through the exchange of B − L gauge boson Z BL , as we have not considered the mixing between the SM Higgs boson and the other BSM singlet scalars. The coupling between Σ 1 and Z BL has the following form, where, As shown in figure 1, our dark matter candidate scatters off the detector nucleus via a t-channel process mediated by Z BL . The expression for the spin independent elastic scattering cross section for the above process is given by, where µ is the reduced mass of nucleon (N ) and dark matter Σ 1 given as µ = M DM M N M DM +M N . The quantity g N N Z BL is the effective coupling between N and Z BL , which is defined as g N N Z BL N γ µ N Z BLµ and it has the following expression, (2.33) Here, g qqZ BL = g BL 3 represents the coupling between first generation quark and the gauge [77]. Therefore, the effective coupling between nucleon (p or n) and Z BL is (2.34) In section 3.3, we show the variation of σ SI with the mass of dark matter and we compare our results with the latest bounds on σ SI from XENON1T [75] and PandaX-II [76] dark matter direct search experiments.

Muon (g − 2)
The presence of the extra neutral gauge boson Z µτ gives an additional one-loop contribution to the muon (g − 2) shown in figure 2. The contribution coming from the digram in figure 2 to the muon magnetic moment a the µ is given as, where, r = (M Zµτ /m µ ) 2 and M Zµτ is the mass of Z µτ . Further, m µ is the mass of muon µ ± while g µτ is the U(1) Lµ−Lτ gauge coupling. For g µτ = 9 × 10 −4 and M Zµτ = 0.1 GeV, one can get from eq. (3.1) which lies roughly within the 3.2 σ range of the observed discrepancy. In what follows, we keep g µτ and M Zµτ fixed at the above values. Here, we would like to note that the benchmark value we have chosen, is allowed from the neutrino trident production experiments like CHARM-II and CCFR [78][79][80]. As a result, following eq. (2.14), we fix v µτ = 111.11 GeV throughout the analysis. We will see that these parameter values, as we have also argued before, affect other phenomenology such as neutrino masses and mixing angles and DM.

Neutrino parameters
In generating the scatter plots among the different parameters of the neutrino mass matrices (given in the eqs. (  both for NH and IH: After satisfying the neutrino oscillation data as mentioned in the section 2.2, we get the allowed regions for the model parameters which are described below. In figure 3, we have shown the variation of the parameters in the planes Y µ −Y e , Y τ −Y µ , Y τ −V τ e and Y µ −M ee . We can see that the neutrino oscillation data puts constraints on the model parameters, restricting them to take values in the ranges shown in figure 3. More interestingly, we note that in all the plots shown in this figure, the neutrino oscillation JHEP08 (2018)  behave in a similar way for both NH and IH.
In figure 5, we show the allowed regions in the Y ee − V eµ and Y R µτ − Y I µτ model space for NH and IH. The correlation between the parameters for both NH and IH can be seen.
From the above study we can infer that the elements of the Dirac mass matrix m D play an important role in determining the neutrino mass hierarchy for the light neutrinos. . Each point satisfies neutrino oscillation data and here also we find overlapping regions between NH and IH for the elements of µ matrix. One can notice in the planes Y e − Y µ and Y µ − Y τ of figure 3, that these parameters have different values for NH and IH. For other parameters of the M R and µ matrices, there exist overlap regions in different planes between the elements of this two matrices. Therefore, these parameters are less important in determining the neutrino mass hierarchy, unlike the Dirac mass matrix elements as discussed above.

Dark matter
As discussed in the subsection 2.3, among the two neutral fermions Σ 1 and Σ 2 , the lightest one will be the DM candidate. In our analysis we have considered Σ 1 as the DM candidate with mass M DM (= M 1 ). The most important property of any DM candidate, which has been measured precisely by WMAP and Planck, is its relic density. The DM relic density Ωh 2 , which is defined as the ratio of DM mass density to the critical density of the Universe, is related to the DM comoving number density at the present epoch by the following relation [81], where Y (T 0 ) is the value of comoving number density Y at T = T 0 , the present temperature of the Universe. In order to find Y (T 0 ), one has to solve the relevant Boltzmann equation which is given by [82] Here, x = M DM T and σv is the thermally averaged annihilation cross section of DM into various final state particles. Moreover, G N is the Newton's gravitational constant. Further, the quantity g is related to the degrees of freedom g eff and h eff of energy and entropy densities of the Universe and its expression is given in ref. [82]. The 1/2 factor in the R.H.S. of Boltzmann equation is due to non-self-conjugate nature of our DM candidate Σ 1 (Dirac fermion). We have solved the Boltzmann equation numerically using micrOMEGAs [83] package. For that, we have generated the required model files by implementing present model in Feynrules [84]. In figure 6, we show the dominant annihilation channels of Σ 1 , which are mediated by h 2 , h 3 and Z BL respectively.
In the left panel of figure 7, we show the variation of DM relic density with the mass of dark matter M DM for three different values of U(1) B−L gauge coupling. The figure shows two resonance regions corresponding to the h 2 and Z BL masses, respectively. The dependence of the DM relic density on g BL is similar in both the resonance regions. For the Z BL resonance region, the cross-section increases with the increase of g BL and as a result the relic density decreases, as can be seen in the figure. On the other hand, for the resonance region corresponding to the h 2 mediated diagrams, the effect of g BL comes indirectly. We see from eq. (2.13) that with increase of g BL , v 2 decreases. Since the coupling γ 1 in eq. (2.29) depend on the VEV v 2 , the γ 1 coupling increases when g BL increases and consequently the cross-section of the h 2 mediated diagrams increase and the relic density falls. In the right panel of same figure we have shown the variation of relic density for three different values of gauge boson mass M Z BL , shown in the legend. For different value of M Z BL there is a shift of the resonance peak, as expected. Since the mass of h 2 is kept fixed, there is no visible change in the h 2 resonance peaks. However, one can notice that around the h 2 resonance region the DM relic density increases almost linearly with the gauge boson mass. Again, eq. (2.13) shows that v 2 is proportional to gauge boson mass M Z BL , leading to this dependence, as shown by eq. (2.29).

Parameters
Range of the Z BL resonance gradually disappears. The reason can be understood from eq. (2.13).
With the increase of r vev , for a fixed v 1 , the value of v 2 decreases and consequently the γ 1 couplings increase. This leads to greater dominance of the h 2 mediated diagrams and as a result even in the Z BL resonance region it diminishes the gauge boson resonance effect.

Allowed parameter space near h 2 resonance
In the left and right panels of figure 9, the scatter plots in the M DM − M h 2 and M DM − σ SI planes show the points that satisfy the DM relic density constraint. In generating these plots we have varied three parameters as shown in table 2. We see in the left panel a sharp correlation in M DM − M h 2 plane. This is expected because the DM relic density is satisfied near the resonance region. Another thing to note here is that for the lower value of g BL (can be seen from figure 7), the relic density is satisfied in the narrower lower side of the resonance. In the left panel, lower values of g BL are shown by the red points and one can see that the region in M DM − M h 2 is narrower for these lower values of g BL . On the other hand, in the right panel we have shown the variation of spin independent elastic scattering cross section with the DM mass. In this work, DM can scatter elastically with the earth based detectors nuclei only via the exchange of Z BL . Therefore, for the variation of g BL , the corresponding spin independent scattering cross section cross section also changes. Table 3. Parameters varied in generating the scatter plot near the Z BL resonance region.

Allowed parameter space near Z BL resonance
In the left and right panel of figure 10, we show the points in the M DM − M Z BL and M DM − σ SI planes that satisfy the DM relic density near the Z BL resonance region. In generating these plots three parameters have been varied as shown in the  figure 8). In the right panel of the same figure we show the scatter plot in the σ SI − M DM plane that can be detected in the different direct detection experiments [75,76,[85][86][87]. In the same plane we have shown the recent bound from XENON1T experiment [75] and PandaX-II experiment [76]. A large area of the plane is accessible in the future run of the different ongoing direct detection experiments.
In figure 11, we show regions in the M Z BL − g BL and M DM − g BL planes allowed by the DM relic density bound. In the left panel we show the LEP bound [88][89][90] as well as JHEP08(2018)062 the LHC dilepton search bound [91][92][93] on the M Z BL − g BL plane. 3,4 The region between the red line and the blue line is excluded by the LHC data. Still, one can see that a large portion of the area is allowed and can be accessed in the future run of LHC to test the validity of the present model. On the other hand in the right panel we show the allowed region in the M DM − g BL plane. Each point satisfies the DM relic density bound. The color map shows the corresponding value of the SI direct detection cross-section.

Conclusion
In this work we extended the SM by two additional gauge groups U(1) B−L and U(1) Lµ−Lτ . Introducing the U(1) Lµ−Lτ gauge group helps us in two way. Firstly, it provides a solution to the muon (g −2) anomaly due to the presence of the extra gauge boson Z µτ and secondly it provides a peculiar form to the neutrino mass matrix due to flavour symmetry. In this work, we generated the light neutrino masses by the inverse seesaw mechanism. Due to the peculiar form of the neutrino mass matrix we obtained correlation among the allowed model parameters after putting constraints on mass squared differences and mixing angles from the neutrino oscillation data. In particular, we have shown that the parameter values which reproduce neutrino oscillation data for NH and IH are almost non-overlapping for JHEP08(2018)062 some of the model parameters. However, some parameters are seen to have overlapping values for both NH and IH.
We also have studied in detail the DM phenomenology. We have shown the variation of the DM relic density with its mass for different values of the other relevant models parameters. We have mainly focussed on that portion of the parameter space where DM dominantly annihilates to the RH neutrinos. We have kept U(1) Lµ−Lτ gauge boson light in order to explain the muon (g − 2) anomaly. Moreover, the small value of g µτ as required to explain muon (g − 2) anomaly, makes Z µτ insignificant to the cosmic evolution of DM. But the other gauge boson Z BL which has a TeV scale mass, plays an important role in DM relic density as well as its direct detection. Further, we also have two extra Higgs bosons which also play an important role in the freeze-out processes of DM. We have found that in our considered mass range for DM, relic density satisfies the Planck limit (0.1172 ≤ Ωh 2 ≤ 0.1226) mainly around the resonance regions of the mediators h 2 and Z BL respectively. We have explored both the resonance regions separately by varying the relevant parameters. We have shown that near the h 2 resonance region a sharp correlation exists between the DM mass and the mass of the scalar h 2 . One could also expect a similar type of correlation between the DM mass and that of U(1) B−L gauge boson, but due to the dominance of the scalar mediated diagrams (h 2 , h 3 ) for the particular values of the gauge coupling (g BL ) and gauge boson mass (M Z BL ), such correlations are destroyed near the Z BL resonance region.
In this work, the parameter space have been chosen in such a way that the dark sector can talk to the SM particles only via neutral gauge boson Z BL . Therefore, the direct detection of our DM candidate Σ 1 is possible only through the spin independent elastic scattering mediated by the B − L gauge boson Z BL . We have computed the spin independent elastic scattering cross section between DM and nucleon and have compared our results with the latest exclusion limits obtained from XENON1T and PandaX-II experiments. We have found that although some portion of σ SI − M DM plane of our present model is already ruled-out by the present direct detection experiments, there still remains sufficient region which can be tested in the near future by the different ongoing direct detection experiments like XENON1T , PandaX-II and Darwin [87]. Another test of this model would be via detection at the collider. One of the signatures will be Drell Yan dilepton production mediated by the Z BL gauge boson like pp → Z BL → ll. Another interesting search will be di-jet (2j) + missing energy ( E T ) or dilepton (2l) + missing energy ( E T ) by the following processes pp → Z BL Z BL → 2j(2l) + E T . (4.1) Therefore, the viability of the present model can be tested both at direct detection as well as collider experiments in near future. A Determination of a 1 , a 2 and a 3 matrices

JHEP08(2018)062
In studying the DM phenomenology we need to know the value of the a 1,2 and a 3 . Here, we have chosen the value of the parameters of matrices m D , M R and µ which satisfy the neutrino oscillation data for normal hierarchy as given in the section 2. Open Access. This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.