1 Introduction

The significance of physics Beyond the Standard Model (BSM) can be mainly verified by the existence of neutrino mass [1], dark matter (DM) [2, 3] and baryon asymmetry of the Universe (BAU) [4, 5]. The mass of neutrinos to be of the order of sub-eV [6] got its confirmation from the neutrino oscillations [7, 8]. Neutrino oscillation was first suggested by Pontecorvo in the 1960’s. Whereas, its experimental discovery was made by Super-Kamiokande Observatory and Sudbury Neutrino Observatories which was recognised by the 2015 Noble Prize shared by Takaaki Kajita and Arthur B. McDonald. Neutrino physics has come up with some benchmark evidences for developing new physics for elementary particles thereby probing into the evolution of the Universe. The recent Neutrino experiments MINOS [9], RENO [10],T2K [11],Double-Chooz [12] have not only confirmed but also measured the neutrino oscillation parameters more accurately. In case of Type I seesaw [13, 14], the Standard Model is extended with the help of three right handed neutrinos which results in the generation of neutrino masses at sub eV scale whereas the right handed neutrino (RHN) mass is related to the Grand Unified Theory (GUT) scale. Though the heavy RHN can generate leptogenesis [4] and simultaneously the small neutrino mass, a naturalness problem arises [15]. Some possible pathways to overcome the high scale leptogenesis are through resonant leptogenesis [16], ARS (Akhmedov, Rubakov and Smirnov) proposed a mechanism via neutrino oscillation wherein the baryon asymmetry of the Universe could be generated with the help of CP-violating sterile neutrino oscillations [17, 18], or via Higgs decay [19, 20]. Such mechanisms however demand for a degenerate RHN mass, which further is another naturalness problem. Whereas, we do have models like the \(\nu \)2HDM [21] and scotogenic model [22,23,24] which accounts an explanation for these discrepencies by considering hierarchical masses of RHN and thereby producing leptogenesis at TeV scale. In this work, we have taken the \(\nu \)2HDM and studied various phenomenologies persisting in it.

There have been numerous lines of evidence of DM including observations from galaxy rotation curve [25], cosmic microwave background [26], galaxy cluster [27, 28], gravitational lensing [29] , etc. However, the most recent cosmological constraints on DM comes from the Planck satellite [30]. It ascertains 27\(\%\) of the present Universe to constitute of DM, which accounts for five times the abundance of the baryonic matter. Depending upon the internal structure i.e. the constituent particles and symmetries, the history of thermalisation of the dark sector take different paths. This gives rise to different types of DM on the basis of its mass range [31]. We basically have the FIMP, hot and cold type dark matter followed from the above mentioned aspect. Though the lightest of the RHN can be considered to be a sterile DM in keV scale [32, 33], it faces stringent bounds from various experiments [34], thereby leaving a very small viable region for it to be a probable dark matter candidate. So, we introduce another scenario in our work by incorporating a dark sector to the \(\nu \)2HDM. It includes a scalar singlet (\(\eta \)) and a Dirac singlet fermion (\(\xi \)), which are charged under \(Z_{2}\) symmetry [35]. Interestingly, the bounds from X-ray experiment can be ignored as the stability of DM candidate \(\xi \) is maintained by the \(Z_{2}\) symmetry [34]. Due to the fact that we have null results from the Direct detection [36] as well as indirect detection [37], the nature of the DM candidate is taken to be of FIMP type.

The basic idea followed in our work comprises of the realisation of the \(\nu \)2HDM by the virtue of the flavor symmetries \(A_{4}\otimes Z_{4}\). Along with the constituent particles of the generic \(\nu \)2HDM, we have added a singlet scalar and a Dirac singlet fermion to the model. The newly added particles incorporate the dark sector, by which the DM phenomenology can be explained. As already mentioned, the DM considered in our work is of FIMP type. We have analysed leptogenesis in our work with the consideration of a mass hierarchy of the RHN \(M_{N_{1}}\ll M_{N_{2,3}}\). On finding the numerical values of the Yukawa coupling matrix, we further carry out a detailed analysis of leptogenesis as well as neutrinoless double beta decay (\(0\nu \beta \beta \)). Also the branching ratio of the decay \((\mu \rightarrow e\gamma )\) is calculated in order to constrain the Yukawa couplings obtained from the model. We emphasize on a relative study due to the variation in the values of the rotational matrix angles used for parametrization of the Yukawa coupling matrix. This yields variation in decay parameter value \(K_{N_{1}}\), which is used in determining baryon asymmetry of the Universe and also explains the various DM phenomenologies. Thus, we see how the choice of the angles has an impact on neutrino phenomenology and related cosmology carried out in this work.

Table 1 Fields and their respective transformations under the symmetry group of the model
Full size table

We have further categorised the paper into six sections which are as follows. Section 2 includes the basic model and the flavor symmetric model. Phenomenologies such as leptogenesis and dark matter are mentioned in Sects. 3 and 4 respectively. We finally show the numerical analysis and the results obtained in Sect. 5, followed by the conclusion in Sect. 6.

2 Flavor symmetric neutrino two higgs doublet model (\(\nu \)2HDM)

In spite of being the most successful theory of particle physics, Standard model (SM) fails to explain quite a few experimental and theoretical phenomena such as smallness of neutrino mass, dark matter (DM), baryon asymmetry of the Universe (BAU), etc. In the SM, neutrino remains massless, although it can acquire a small Majorana mass through dimension five operator [38, 39].

$$\begin{aligned} \frac{1}{\Lambda }(\nu _{i} \Phi ^{0}-l_{i} \Phi ^{+})(\nu _{j} \Phi ^{0}-l_{j} \Phi ^{+}) \end{aligned}$$
(2.1)

where \(\Lambda \) is the effective large mass scale and H is usual Higss doublet. This operator can be commonly realized in the framework of canonical seesaw [14], where SM is extended with three right handed neutrino (\(N_{i}\)) such that-

$$\begin{aligned} m_{\nu }=\frac{m_{D}^{2}}{m_{N}} \end{aligned}$$
(2.2)

where \(m_{D}=fv\) is the Dirac neutrino mass considering vaccuum expection value (vev) of Higgs to be \(\langle \Phi ^{0}\rangle =v'\) and \(m_{N}=\Lambda f\). Such kind of mechanisms does not have a direct test in the experiment because of the large mass scale of \(m_{N}\).

\(\nu 2\mathrm{HDM}\) is one such framework where Eq. (2.2) can be realised naturally with mass of the right handed neutrino to be in experimental reach (order of 1 TeV) by extending the SM with three right handed neutrino and a scalar doublet (\(\phi \)) having a very small vaccum expectation value (vev) [21]. We have to introduce a \(U (1)_{L}\) global symmetry under which \(L_{\Phi }=0\), \(L_{\phi }=-1\) and \(L_{N}=0\), so that it forbids type-I seesaw interaction term \(\bar{l_{L}}{\tilde{\Phi }}N\). Thus, the additional scalar doublet \(\phi \) couples with right handed neutrino and SM Higgs will couple with quarks and charged lepton. In this model the smallness of Dirac neutrino mass is achieved through this additional scalar doublet. To accommodate dark matter candidate in this model, we introduce a dark sector to the generic \(\nu \)2HDM (discussed above).This newly added dark sector includes a singlet scalar \(\eta \) and a Dirac singlet fermion \(\xi \) [31, 40] which are charged under \(Z_{2}\) symmetry. Considering \(m_{\xi }< m_{\eta }\), we can say that \(\xi \) serves as a probable DM candidate. Particle content of the minimal \(\nu 2\mathrm{HDM}\) with charge assignment under \(SU(3)_{C} \otimes SU(2)_{L} \otimes U(1)_{Y}\) are given below-

$$\begin{aligned} \left[ \begin{array}{cc} \nu _{i}\\ l_{i} \end{array}\right] _L= & {} (1,2,-1/2) ; \left[ \begin{array}{c} l_{R_{i}} \end{array}\right] =(1,1,-1),\nonumber \\ \left[ \begin{array}{c} N_{i} \end{array}\right]= & {} (1,1,0), \left[ \begin{array}{c} \phi \end{array}\right] =(1,2,1/2). \end{aligned}$$
(2.3)

The scalar doublets \(\Phi \) and \(\phi \) in the model can be denoted as:

$$\begin{aligned} \Phi= & {} \left[ \begin{array}{cc} \Phi ^{+}\\ \frac{v'+\Phi ^{0,r}+i\Phi ^{0,i}}{\sqrt{2}} \end{array}\right] ,\nonumber \\ \phi= & {} \left[ \begin{array}{cc} \phi ^{+}\\ \frac{v+\phi ^{0,r}+i\phi ^{0,i}}{\sqrt{2}} \end{array}\right] \end{aligned}$$
(2.4)

Thus, the corresponding Higgs potential will be:

$$\begin{aligned} V= & {} m_{\Phi }^{2}\Phi ^{\dagger }\Phi +m_{\phi }^{2}\phi ^{\dagger }\phi + m_{\eta }^{2}\eta ^{\dagger }\eta + \frac{1}{2}\lambda _{1}(\Phi ^{\dagger }\Phi )^{2}\nonumber \\&+\frac{1}{2} \lambda _{2}(\phi ^{\dagger }\phi )+ \lambda _{3}(\Phi ^{\dagger }\Phi )(\phi ^{\dagger }\phi )\nonumber \\&+\lambda _{4}(\Phi ^{\dagger }\phi )(\phi ^{\dagger }\Phi )-\mu _{12}^{2}(\Phi ^{\dagger }\phi +h.c)\nonumber \\&+\left( \frac{1}{2} \lambda _{5}(\Phi ^{\dagger }\phi )^{2}+h.c\right) \nonumber \\&+\frac{1}{2} \lambda _{6}(\eta ^{\dagger }\eta )^{2}+\lambda _{7}(\eta ^{\dagger }\eta )(\Phi ^{\dagger }\Phi )+\lambda _{8}(\eta ^{\dagger }\eta )(\phi ^{\dagger }\phi ) \end{aligned}$$
(2.5)

where the lepton symmetry is explicitly and softly broken by the \(\mu _{12}\) term. The condition \(\langle \eta \rangle =0\) should be satisfied for the unbroken \(Z_{2}\) symmetry so that it do not have any interaction with the Higgs doublet. Also the mixing angle tan\(\beta \) can be expressed in terms of the vev of SM Higgs boson and the inert Higgs doublet, which is given by \(tan\beta = \frac{v^{'}}{v}\).

Again the minimization condition is given by the equation:

$$\begin{aligned} m_{\Phi }^{2}= & {} \mu _{12}^{2}\frac{v}{v^{'}} - \frac{\lambda _{1}}{2}v'^{2} -\frac{(\lambda _{3}+\lambda _{4}+\lambda _{5})}{2}v^{2} \end{aligned}$$
(2.6)
$$\begin{aligned} m_{\phi }^{2}= & {} \mu _{12}^{2}\frac{v^{'}}{v} - \frac{\lambda _{1}}{2}v^{2} -\frac{(\lambda _{3}+\lambda _{4}+\lambda _{5})}{2}v'^{2} \end{aligned}$$
(2.7)

which helps in expressing the vev of Higgs doublet in terms of parameter present in the Higgs potential.

Radiative corrections to the term \(\mu _{12}^{2}\) is proportional to \(\mu _{12}^{2}\) itself and it is logarithmically sensitive to the cut off scale as it is the only source of lepton number violation [41]. This results in the stabilization of vev hierarchy \(v\ll v^{'}\) against radiative corrections. The physical Higgs boson after SSB is given by:

$$\begin{aligned} H^{+}= & {} \phi ^{+}cos\beta - \Phi ^{+}sin\beta ,\nonumber \\ A= & {} \phi ^{0,i}cos\beta - \Phi ^{0,i}sin\beta \end{aligned}$$
(2.8)
$$\begin{aligned} H^0= & {} \phi ^{0,r}cos\alpha - \Phi ^{0,r}sin\alpha ,\nonumber \\ h= & {} \Phi ^{0,r}cos\alpha + \Phi ^{0,r}sin\alpha , \end{aligned}$$
(2.9)

where the mixing angle \(\alpha \) is given as follows:

$$\begin{aligned} tan 2\alpha \simeq 2\frac{v}{v^{'}}\frac{-\mu _{12}^{2}+(\lambda _{3}+\lambda _{4}+\lambda _{5})v^{'}v}{-\mu _{12}^{2}+\lambda _{1}v^{'}v}. \end{aligned}$$
(2.10)

Further after neglecting \({\mathcal {O}}(v^{2})\) and \({\mathcal {O}}(\mu _{12}^{2})\), the masses of the physical Higgs bosons are as follows:

$$\begin{aligned} m_{H^{+}}^{2}\simeq & {} m_{\phi }^{2}+1/2(\lambda _{3}v^{'2}), m_{A}^{2}\simeq m_{H}^{2}\nonumber \\\simeq & {} m_{H^{+}}^{2}+1/2(\lambda _{4}+\lambda _{5})v^{'2}, m_{h}\simeq \lambda _{1}v^{'2} \end{aligned}$$
(2.11)

The effective Lagrangian for the extended \(\nu \)2HDM can be expressed as:

$$\begin{aligned} {\mathcal {L}} \subset Y \bar{l_{L}} {\tilde{\phi }}N + \lambda {\bar{\xi }}\eta N + \frac{1}{2} \bar{N^{c}}m_{N} N+ m_{\xi } \xi {\bar{\xi }}+h.c . \end{aligned}$$
(2.12)

From Eq. (2.12), the light neutrino mass can derived as [13]:

$$\begin{aligned} m_{\nu }=-\frac{v^{2}}{2} Y M^{-1}_{N} Y^{T}= U_{PMNS} m^{d}_{\nu } U^{T}_{PMNS} \end{aligned}$$
(2.13)

where, \(m^{d}_{\nu }=diag(m_{1},m_{2},m_{3})\) is the diagonal neutrino mass matrix and \(U_{PMNS}\) is the PMNS (Pontecorvo–Maki–Nakagawa–Sakata) matrix.

Fig. 1
figure 1

\(Br(\mu \rightarrow e\gamma )\) as a function of \(m_{H^{+}}\) for NH and IH. The dashed horizontal lines are the recent upper bounds

Full size image

We also realize the extension of the \(\nu \)2HDM with the help of discrete flavor symmetry group \(A_{4}\otimes Z_{4}\). To maintain the stability of the dark sector, a discrete \(Z_{2}\) symmetry is also introduced under which the extended particles are charged. Particle content of the model with its charge assignment are given in the Table 1. \(\eta \) and \(\xi \) fields are charged under \(Z_{2}\) symmetry, considering them to be the dark sector which donot acquire any vev and thus remain invisible. We introduce five flavon fields \(\rho \), \(\rho ^{'}\), \(\rho ^{''}\), \(\zeta \), \(\zeta ^{'}\) in order to break the flavor symmetry so as to generate the required mass structures.

Now, the Lagrangian for the charged lepton sector can be written as:

$$\begin{aligned} {\mathcal {L}}= & {} \frac{y_{e}}{\Lambda } (\bar{l_{L}}\Phi \rho )l_{R_{1}}+\frac{y_{\mu }}{\Lambda } (\bar{l_{L}}\Phi \rho )l_{R_{2}}\nonumber \\&+\frac{y_{\tau }}{\Lambda } (\bar{l_{L}}\Phi \rho )l_{R_{3}}+h.c \end{aligned}$$
(2.14)

where \(\Lambda \) denotes the cut off scale. The term \(\bar{l_L}H\rho \) in the Eq. (2.14) transforms as 1, \(1^{'}\) and \(1^{''}\) respectively under \(A_{4}\) for initial three terms in order to obtain a diagonal charged lepton mass matrix [42].

The effective Lagrangain for Dirac mass is given by:

$$\begin{aligned} {\mathcal {L}}= & {} \frac{y_{1}}{\Lambda } (\bar{l_{L}}{\tilde{\phi }}\rho ) N_{1}+\frac{y_{2}}{\Lambda } (\bar{l_{L}}{\tilde{\phi }}\rho ^{'}) N_{2}\nonumber \\&+\frac{y_{3}}{\Lambda } (\bar{l_{L}}{\tilde{\phi }}\rho ^{''}) N_{3} \end{aligned}$$
(2.15)

and the right handed fermions are represented by the Lagrangian:

$$\begin{aligned} \frac{1}{2}\beta _{1}\zeta \bar{N_{1}^{c}}N_{1}+\frac{1}{2}\beta _{2}\zeta ^{'} \bar{N_{2}^{c}}N_{2}+\frac{1}{2}\beta _{3}\zeta \bar{N_{3}^{c}}N_{3}+h.c \end{aligned}$$
(2.16)

The flavon alignments considered in the model are given below:

$$\begin{aligned} \langle \rho \rangle&=(\omega ,\omega ,\omega ), \langle \rho ^{'}\rangle =(0,-\omega ,\omega ),\nonumber \\ \langle \rho ^{''}\rangle&=(\omega ,\omega ,\omega ), \langle \zeta \rangle =\langle \zeta ^{'}\rangle =\omega \end{aligned}$$
(2.17)

Using this alignment we can have diagonal charge lepton matrix given by:

$$\begin{aligned} m_{l}=\frac{\langle \Phi \rangle \omega }{\Lambda }\left[ \begin{array}{ccc} y_{e}&{}\quad 0&{}\quad 0\\ 0&{}\quad y_{\mu }&{}\quad 0\\ 0&{}\quad 0&{}\quad y_{\tau } \end{array}\right] , \end{aligned}$$
(2.18)

and the Dirac mass matrix will take the form represented as:

$$\begin{aligned} m_{D}=\frac{\langle \phi \rangle \omega }{\Lambda }\left[ \begin{array}{ccc} y_{1}&{}\quad y_{2}&{}\quad y_{3}\\ y_{1}&{}\quad 0&{}\quad y_{3}\\ y_{1}&{}\quad -y_{2}&{}\quad y_{3} \end{array}\right] , \end{aligned}$$
(2.19)

here \(y_{1}\),\(y_{2}\) and \(y_{3}\) stands for the Yukawa couplings. We further express the Dirc mass matrix in terms of the model parameters as:

$$\begin{aligned} m_{D}=\left[ \begin{array}{ccc} a&{}\quad b&{}\quad c\\ a&{}\quad 0&{}\quad c\\ a&{}\quad -b&{}\quad c \end{array}\right] , \end{aligned}$$
(2.20)

where, \(a=\frac{\langle \phi \rangle \omega }{\Lambda }y_{1}\), \(b=\frac{\langle \phi \rangle \omega }{\Lambda }y_{2}\) and \(c=\frac{\langle \phi \rangle \omega }{\Lambda }y_{3}\) are the model parameters which actually represents the Dirac masses. Because of the additional \(Z_{4}\) symmetry the right handed neutrino mass matrix will be a diagonal one given as:

$$\begin{aligned} M_{N}=\left[ \begin{array}{ccc} \beta _{1}\omega &{}\quad 0&{}\quad 0\\ 0&{}\quad \beta _{2}\omega &{}\quad 0\\ 0&{}\quad 0&{}\quad \beta _{3}\omega \end{array}\right] , \end{aligned}$$
(2.21)

where \(\beta _{1}\),\(\beta _{2}\) and \(\beta _{3}\) are the Majorana couplings. Incorporating the mass matrices and Yukawa coupling matrix obtained from the model in Eq. (2.13), we can finally calculate the active neutrino mass.

Bounds from Lepton flavor violating process

It is well known that lepton flavor violating processes put significant bound on the model parameter space. The size of the LFV is controlled by the lepton number violating Yukawa couplings \(Y_{ij}\). The MEG collaboration has been able to set the impressive bound on muon decay Br\((\mu \rightarrow e\gamma )<4.2 \times 10^{-13}\) [43]. The branching ratio of \(l_{\alpha }\rightarrow l_{\beta }\gamma \) is given by [44]-

$$\begin{aligned} \text {Br}(\mu \rightarrow e\gamma )=\frac{3\alpha }{64\pi G^{2}}\bigg |\sum _{i} \frac{Y_{\mu i}Y_{ei}^{*}}{m^{2}_{H^{+}}} F(\Delta ^{N_{i}}_{H^{+}})\bigg | \end{aligned}$$
(2.22)

where G is the Fermi constant and \(\Delta ^{N_{i}}_{H^{+}}= (\frac{M_{N_{i}}}{m_{H^{+}}})^{2}\) and the loop function is expressed as :

$$\begin{aligned} F(x)= \frac{1}{6(1-x)^{4}}(1-6x^{2}+3x^{2}+2x^{3}-6x^{2}lnx) \end{aligned}$$
(2.23)

The Yukawa couplings obtained from the model, i.e. found to be in the range \(10^{-1}-10^{-6}\) are consistent with the bounds of lepton flavor violating process. From Fig. 1, we see that the branching ratio value for the process \(\mu \rightarrow e\gamma \) is below the experimental bound given by MEG collaboration for both NH and IH, thereby, making the Yukawa couplings viable in explaining leptogenesis and dark matter.

3 Leptogenesis

We study leptogenesis in this work, which is a consequence of the out-of-equilibrium decay of \( N_{1}\rightarrow l_{L}\phi ^{*},\bar{ l_{L}}\phi \) [4]. The lepton asymmetry produced only by the decay of \(N_{1}\) is converted into the baryon asymmetry of the Universe (BAU) by the electro-weak sphaleron phase transitions [45]. In case of vanilla leptogenesis, there exists an absolute lower bound on the mass of the lightest RHN to be \(M_{N_{1}} \simeq 10^{9}\) GeV [46, 47]. However, the limit on the lightest RHN mass scale can be lowered to 10 TeV [5, 24] in certain scenarios. With the consideration of a hierarchical mass spectrum \( m_{\phi }\ll M_{N_{1}}\ll M_{N_{2,3}}\) and the incorporation of the Yukawa couplings from the model, the CP-asymmetry term is given by [24]:

$$\begin{aligned} \epsilon _{1}= & {} \frac{1}{8\pi ( Y^{\dagger } Y)_{11}}\sum _{j\ne 1} Im[( Y^{\dagger } Y)^{2}]_{1j}\nonumber \\&\times \left[ f( r_{j1},\eta _{1})- \frac{\sqrt{ r_{j1}}}{ r_{j1}-1}(1-\eta _{1})^{2}\right] , \end{aligned}$$
(3.1)

where, Y represents the Yukawa coupling matrix and the term \(f( r_{j1},\eta _{1})\) is expressed as:

$$\begin{aligned} f( r_{j1},\eta _{1})= & {} \sqrt{ r_{j1}}\left[ 1+ \frac{(1-2\eta _{1}+ r_{j1})}{(1-\eta _{1})^{2}} ln\right. \nonumber \\&\times \left. (\frac{ r_{j1}-\eta _{1}^{2}}{1-2\eta _{1}+ r_{j1}})\right] , \end{aligned}$$
(3.2)

with \( r_{j1}= \left( \frac{ M_{N_{j}}}{ M_{N_{1}}}\right) ^{2}\) and \(\eta _{1}\equiv \left( \frac{ m_{H^{0}}}{ M_{N_{1}}}\right) ^{2}\).

The decay rate equation for \( N_{1}\) which is given by,

$$\begin{aligned} \Gamma _{1}= & {} \frac{ M_{N_{1}}}{8\pi }( Y^{\dagger } Y)_{11}\left[ 1- \left( \frac{ m_{H^{0}}}{ M_{N_{1}}}\right) ^{2}\right] ^{2}\nonumber \\= & {} \frac{ M_{N_{1}}}{8\pi }( Y^{\dagger } Y)_{11}(1-\eta _{1})^{2} \end{aligned}$$
(3.3)

Meanwhile, the washout effect is checked by the decay parameter

$$\begin{aligned} K_{N_{1}}= \frac{\Gamma _{1}}{ H(z=1)}, \end{aligned}$$
(3.4)

where, \(\Gamma _{1}\) is the decay width of \( N_{1}\), H is the Hubble parameter and \(z=M_{N_{1}}/T\) with T being the temperature of the thermal bath. We can express H in terms of T and the corresponding equation is given by:

$$\begin{aligned} H = \sqrt{\frac{8\pi ^{3}g_{*}}{90}}\dfrac{T^{2}}{M_{Pl}}. \end{aligned}$$
(3.5)

In Eq. (3.5), \(g_{*}\) stands for the effective number of relativistic degrees of freedom and \(M_{Pl}\simeq 1.22\times 10^{19}\) GeV is the Planck mass.

Using the parametrization for Yukawa coupling, i.e. [48]

$$\begin{aligned} Y= \frac{\sqrt{2}}{v}U_{PMNS}(m^{diag}_{\nu })^{1/2}R(M^{diag}_{N})^{1/2}, \end{aligned}$$
(3.6)

we can verify the following relation:

$$\begin{aligned} Y^{\dagger }Y= \frac{2}{v^{2}}(M^{diag}_{N})^{1/2}R^{\dagger }(m^{diag}_{\nu })R(M^{diag}_{N})^{1/2}. \end{aligned}$$
(3.7)

Here, R is an orthogonal matrix satisfying the condition \(R^{T}R\)=1 which is parametrized as:

$$\begin{aligned} R= & {} \begin{pmatrix} cos\omega _{12} &{}\quad -sin\omega _{12} &{}\quad 0 \\ sin\omega _{12} &{}\quad cos\omega _{12} &{}\quad 0\\ 0 &{}\quad 0 &{}\quad 1\\ \end{pmatrix} \begin{pmatrix} cos\omega _{13} &{}\quad 0 &{}\quad -sin\omega _{13} \\ 0 &{}\quad 1 &{}\quad 0\\ sin\omega _{13} &{} \quad 0 &{}\quad cos\omega _{13}\\ \end{pmatrix}\nonumber \\&\times \begin{pmatrix} 1 &{} \quad 0 &{}\quad 0 \\ 0 &{}\quad cos\omega _{23} &{}\quad -sin\omega _{23}\\ 0 &{}\quad sin\omega _{23} &{}\quad cos\omega _{23}\\ \end{pmatrix} \end{aligned}$$
(3.8)

where, \(\omega _{12,13,23}\) represents the arbitrary complex angles. Thus, we can now express the decay parameter \( K_{N_{1}}\) interms of the arbitrary angles by the following equation:

$$\begin{aligned} K_{N_{1}}\simeq 897 \left( tan\beta \right) ^{2}\frac{|((m_{\nu }^{diag})^{R})_{11}|}{eV}, \end{aligned}$$
(3.9)

which can also be verified from Eq. (3.7). In the above equation, \((m_{\nu }^{diag})^{R}= R^{\dagger }m_{\nu }^{diag}R\), which further gives:

$$\begin{aligned} (m_{\nu }^{diag})^{R})_{11}= & {} m_{1}cos\omega _{12}^{2}cos\omega _{13}^{2}\nonumber \\&+ m_{2}sin\omega _{12}^{2}cos\omega _{13}^{2}+m_{3}sin\omega _{13}^{2} \end{aligned}$$
(3.10)

From Eqs. (3.9) and (3.10), we can see that \(K_{N_{1}}\) has dependency on the arbitrary complex angles \(\omega _{12,13}\) but is independent of \(\omega _{23}\). This relationship implies that by proper variation of these arbitrary angles, we can have a comparative analysis of the different phenomenologies associated with it. In our work, we have chosen some benchmark values for \(\omega _{13,23}\) for which we have varied \(\omega _{12}\). We showcase the variations in BAU as well as DM phenomenology for two different values of \(\omega _{12}\), however, keeping \(\omega _{13,23}\) fixed. Also, the vev of the scalar doublet \(\phi \) plays a vital role in the determination of \(K_{N_{1}}\), which further has significant impact on baryogenesis. As mentioned in various literatures [49, 50], the \(\Delta L\)= 0 washout processes are crucial in context of small values u. Thereby, the condition \(\Gamma _{1}/M_{N_{1}} \ll 1\) for low scale seesaw is satisfied. Thus, the Boltzmann equations for the evolution of the abundance of \(N_{1}\) and \(N_{B-L}\) are given by [46],

$$\begin{aligned} \frac{dn_{N_{1}}}{dz}= & {} -D_{1}(n_{N_{1}} - n_{N_{1}}^{eq}), \end{aligned}$$
(3.11)
$$\begin{aligned} \frac{dn_{B-L}}{dz}= & {} -\epsilon _{1}D_{1}(n_{N_{1}} - n_{N_{1}}^{eq})- W_{1}n_{B-L}, \end{aligned}$$
(3.12)

respectively. \(n_{N_{1}}^{eq}= \frac{z^{2}}{2}K_{1}(z)\) is the equilibrium number density of \(N_{1}\), where \(K_{i}(z)\) is the modified Bessel function of \(i^{th}\) type and

$$\begin{aligned} D_{1}\equiv \frac{\Gamma _{1}}{Hz} = K_{N_{1}}z\frac{K_{1}(z)}{K_{2}(z)} \end{aligned}$$
(3.13)

gives the measure of the total decay rate with respect to the Hubble rate, and \( W_{1}= \frac{\Gamma _{W}}{Hz}\) is the total washout rate. Again, \(W_{1}= W_{1D}+W_{\Delta L=2}\) [5], i.e the total washout term is the sum of the washout due to inverse decays \(l\phi ,{\bar{l}}\phi ^{*}\rightarrow N_{1}\) (\(W_{1D}= \frac{1}{4}K_{N_{1}}z^{3}K_{1}(z)\)) and the washout due to the \(\Delta L= 2\) scatterings \(l\phi \leftrightarrow {\bar{l}}\phi ^{*},ll \leftrightarrow \phi ^{*}\phi ^{*}\) which is given by [24],

$$\begin{aligned} W_{\Delta L=2} \simeq \dfrac{18\sqrt{10}M_{Pl}}{\pi ^{4}g_{l}\sqrt{g_{*}}z^{2}v^{4}}\left( \frac{2\pi ^{2}}{\lambda _{5}}\right) ^{2}M_{N_{1}}{\bar{m}}^{2}. \end{aligned}$$
(3.14)

Here, \(g_{l}\) is the internal degrees of freedom for the SM leptons, and \({\bar{m}}\) is the absolute neutrino mass scale, defined by:

$$\begin{aligned} {\bar{m}}^{2} = m_{1}^{2}+m_{2}^{2}+m_{3}^{2} \end{aligned}$$
(3.15)

The final B-L asymmetry \( n_{B-L}^{f}\) is evaluated by numerically calculating Eqs. (3.11) and (3.12) before the sphaleron freeze-out. This is converted into the baryon-to-photon ratio given by [24]:

$$\begin{aligned} n_{B}= \frac{3}{4}\frac{g_{*}^{0}}{g_{*}}a_{sph}n_{B-L}^{f}\simeq 9.2\times 10^{-3}n_{B-L}^{f}, \end{aligned}$$
(3.16)

In Eq. (3.16), \(g_{*}= 110.75\) is the effective relativistic degrees of freedom at the time when final lepton asymmetry was produced, \(g_{*}^{0}= \frac{43}{11}\) is the effective degrees of freedom at the recombination epoch and \( a_{sph}=\frac{8}{23}\) is the sphaleron conversion factor taking two Higgs doublet into consideration. The Planck limit 2018 gives a bound on the observed BAU (\( n_{B}\)) to be \((6.04\pm 0.08)\times 10^{-10}\) [51]. Therefore, in our work, we have chosen the free parameters such that we can generate the observed BAU for NH/IH. As mentioned earlier, we have fixed the arbitrary angles \(\omega _{13,23}\) bearing values \(10^{-3}+10^{-3}i\) and \(10^{-1}+10^{-1}i\) respectively. Whereas, we have chosen two benchmark values for \(\omega _{12}\), i.e \(10^{-2}+10^{-2}i\) and \(10^{-12}+10^{-12}i\) which shows quite a significant change in the cosmological phenomenologies carried out in this work. In addition to this, we have taken a certain parameter space for \(v= 0.1{-}30\) GeV to show its dependence on BAU for both NH/IH.

4 Dark matter

In this work, we consider a FIMP type dark matter with the real Yukawa coupling, \(\lambda \ll 1\) [31] between the right handed neutrino N with the dark sector, i.e. singlet fermion \(\xi \) and singlet scalar \(\eta \). As of the DM type taken into account, it is known that the coupling is so weak that it never reaches thermalization even within the dark sector. We consider the singlet fermion (\(\xi \)) to be a probable dark matter candidate. Due to the feeble interaction of the DM candidate, its relic abundance is generated via the freeze-in mechanism [52]. We can obtain the expression for relic abundance by solving the Boltzmann equation:

$$\begin{aligned} \frac{dY_{\xi }}{dz}= DY_{N_{1}}\mathrm{BR}_{\xi } \end{aligned}$$
(4.1)

where, \(Y_{\xi }\) and \(Y_{N_{1}}\) are the abundance of DM candidate \(\xi \) and right handed neutrino \(N_{1}\) respectively, and \(\mathrm{BR}_{\xi }\) is the branching ratio of \(N_{1}\longrightarrow \xi \eta \). The inverse decay process, i.e. \(\xi \eta \longrightarrow N_{1}\) is neglected and a hierarchical criteria of \(\mathrm{BR}_{\xi }\equiv \mathrm{BR} (N_{1}\longrightarrow \xi \eta )\ll \mathrm{BR}_{SM}\equiv \mathrm{BR} (N_{1}\longrightarrow SM)\simeq 1\) is considered due to the FIMP nature of \(\xi \). Alongside, for the decay \(N_{1}\longrightarrow \xi \eta \) to obey out of equilibrium condition, it is crucial that \(\mathrm{BR}_{\xi }< 10^{-2}\). In our work, as we have solved the model parameters so as to find the value of \( K_{N_{1}}\), we see that it is consistent in satisfying the bounds for relic abundance and streaming length. From Eq. (4.1), an asymptotic abundance of the FIMP DM \(\xi \) can be approximated by the relation [31]:

$$\begin{aligned} Y_{\xi }(\infty )\simeq Y_{N_{1}}(0)\mathrm{BR}_{\xi }\left( 1+\frac{15\pi \zeta (5)}{16\zeta (3)}K_{N_{1}}\right) \end{aligned}$$
(4.2)

Therefore, the expression for relic abundance in agreement with the asymptotic abundance is given by:

$$\begin{aligned} \Omega _{\xi }^{ FIMP}h^2= & {} \frac{m_\xi s_{0}Y_{\xi }(\infty )}{\rho _{c}}h^2 \simeq 0.12\nonumber \\&\times \left( \frac{m_{\xi }}{keV}\right) (\mathrm{BR}_{\xi } \times 10^{3})\left( 0.009+\frac{K_{N_{1}}}{44}\right) \nonumber \\ \end{aligned}$$
(4.3)

where, \(\rho _{c}= 1.05371 \times 10^{-5}h^{2}\,\mathrm{GeV}\,\mathrm{cm}^{-3}\) is the critical density of the Universe and \(s_{0}= 2891.2\,\mathrm{cm}^{-3}\) [53] is the current entropy density and \(h=0.72\) is the Hubble parameter. One of the frequently and significantly considered cosmological constraints that comes into play while considering light DM candidate is the free streaming limit. It provides stringent bounds on the FIMP DM mass. Due to non-trivial velocity dispersion, the free streaming of dark matter particles strike outs the matter density perturbations and consequently the structure formation on scales \(\lambda < \lambda _{FS} \). Subsequently, small structure formation gives the most robust constraint on the free streaming length, viz. \(\lambda _{FS}< {\mathcal {O}}(0.1)\) Mpc [54]. The free streaming length is defined as the average distance covered by a particle without confronting collision [31];

$$\begin{aligned} \lambda _{FS}= & {} \int _{a_{rh}}^{a_{eq}} \frac{\langle v_{\xi }\rangle }{a^{2}H}da \nonumber \\\simeq & {} \frac{a_{NR}}{H_{0}\sqrt{\Omega _{R}}}\left( 0.62+ln\left( \frac{a_{eq}}{a_{NR}}\right) \right) \end{aligned}$$
(4.4)

where, \(\langle v_{\xi }\rangle \) is the average velocity at given time of the FIMP DM \(\xi \), \(a_{rh}\) and \(a_{eq}\) are the scale factors at reheating and equilibrium respectively. The values of the cosmological parameters used are \(H_{0}= 67.3\,\mathrm{km}\, { \mathrm s}^{-1}\,{ \mathrm Mpc}^{-1}\), \(\Omega _{R}= 9.3\times 10^{-5}\) and \(a_{eq}= 2.9\times 10^{-4}\) [55]. Again, the non-relativistic scale factor for FIMP DM is expressed as [40]:

$$\begin{aligned} a_{NR}= \frac{T_{0}}{2m_{\xi }}\left( \frac{g_{*,0}}{g_{*,rh}}\right) K_{N_{1}}^{-1/2} \end{aligned}$$
(4.5)

with \(g_{*,0}= 3.91\), \(g_{*,rh}= 106.75\) and \(T_{0}= 2.35\times 10^{-4}\,\mathrm{eV}\). There are different decoupling and production mechanism of DM and thereby, the free streaming length is different for each of the relativistic decoupling scenarios such as hot, FIMP DM. As the free streaming length is highly dependent on the production mechanism, thus, the specific free streaming length for FIMP DM can be given by [40]:

$$\begin{aligned} \lambda _{FS}\simeq & {} 2.8\times 10^{-4}\left( \frac{keV}{m_{\xi }}\right) \left( \frac{50}{K_{N_{1}}}\right) ^{1/2}\nonumber \\&\times \left( 1+0.09 ln\left[ \left( \frac{m_{\xi }}{keV}\right) \left( \frac{K_{N_{1}}}{50}\right) ^{1/2}\right] \right) \mathrm{Mpc} \end{aligned}$$
(4.6)

From the above equation, we can clearly say that the free streaming length is relied upon the decay parameter and the DM mass. Hence, we do a detailed analysis on these factors in our work for both NH/IH.

Fig. 2
figure 2

Baryon asymmetry of the Universe as a function of lightest RHN (\(N_{1}\)) is shown in the first row. The second and third row depicts BAU as a function of \( K_{N_{1}}\) and tan\(\beta \) respectively. We take two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for NH/IH

Full size image
Fig. 3
figure 3

Yukawa coupling term \(YY^{\dagger }_{11}\) as a function of BAU for two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for NH/IH

Full size image
Fig. 4
figure 4

Baryon asymmetry of the Universe as a function of relic abundance of the dark matter candidate \(\xi \) for NH/IH with the consideration of two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\)

Full size image
Fig. 5
figure 5

Decay parameter \( K_{N_{1}}\) as a function of lightest neutrino mass for two different values of arbitrary angle \(\omega _{12}\) with benchmark values \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for NH/IH

Full size image

5 Analysis and results

In the flavor symmetric realisation of the extended \(\nu \)2HDM [40], we have explicitly studied the implication of the Yukawa coupling matrix and the rotational matrix used for parametrization. On numerically solving the Dirac masses given by a,b and c which constitute the Yukawa coupling matrix, we eventually find the range of the coupling matrix Y. Incorporating the parameter space of the Yukawa coupling matrix in Eq. (3.6), we can deduce the active neutrino mass matrix. It has been found from the model we have worked upon that the summation of the active neutrino is consistent with the Planck limit, i.e. \(\sum _{i=1,2,3}m_{i} < 0.11 \) eV [6]. Consecutively, we have also analysed the phenomenology of \(0\nu \beta \beta \) [56,57,58] in our work. It is indeed a well known and significant experimental technique of detecting neutrino mass, thereby, making it relatable with observations in the on-going experiments. The various experiments in this regard are KamLAND-Zen [59, 60], KATRIN [61, 62], GERDA [63, 64], etc. The phenomenon that is measured in these experiments is the effective mass of the active neutrinos which is given by the expression:

$$\begin{aligned} |m_{\beta \beta }|= |c_{12}^{2}c_{13}^{2}m_{1} + s_{12}^{2}c_{13}^{2}m_{2}e^{2i\alpha } + s_{13}^{2}m_{3}e^{2i\beta }| \end{aligned}$$
(5.1)

where, \(c_{ij}\)= \(\cos \theta _{ij}\) and \(s_{ij}\)= \(\sin \theta _{ij}\) are the elements of the UPMNS matrix. It is crucial to obey the effective mass constraint so as to make the model sensitive to the on-going as well as future collider signatures. Throughout the work, we have considered a specific range of the right handed neutrinos, viz. \(M_{N_{1}}= 10^4\) GeV–\(10^{6}\) GeV, \(M_{N_{2}}= 10^7\) GeV–\(5\times 10^{7}\) GeV and \(M_{N_{3}}= 10^8\) GeV–\(5\times 10^{8}\) GeV. Also, the lightest active neutrino mass is taken in the span of \(m_{l}= 10^{-11}\)–\(10^{-13}\) eV in case of both NH/IH which is a crucial parameter required to achieve baryogenesis in TeV scale [5]. In our study, we have shown the variations that may occur due to the choice of arbitrary angles of the rotational matrix given in Eq. (3.10). Since, the dependency of BAU and dark matter phenomenology is influenced by the decay parameter \( K_{N_{1}}\), thus we can generate desired values of \( K_{N_{1}}\) by fine tuning the arbitrary angles \(\omega _{12}\) and \(\omega _{13}\) [40]. In our case, we have kept \(\omega _{13,23}\) fixed at \(10^{-3}+10^{-3}i\) and \(10^{-2}+10^{-2}i\) respectively and simultaneously chosen two different values of \(\omega _{12}\) as \(10^{-1}+10^{-1}i\) and \(10^{-12}+10^{-12}i\). In Fig. 2, we have shown co-relation plots between BAU and parameters such as \(M_{N_{1}}\), \( K_{N_{1}}\) and tan\(\beta \) respectively for NH and IH. From the first row of Fig. 2, we see that the observed baryogenesis is generated for \(M_{N_{1}}= 10^{4}\)–\(10^{6}\) GeV for NH, however, the choice of the arbitrary angle \(\omega _{12}= 10^{-1}+10^{-1}i\) gives large number of points satisfying the Planck limit compared to that of \(\omega _{12}= 10^{-12}+10^{-12}i\). For \(\omega _{12}= 10^{-12}+10^{-12}i\), the region \(M_{N_{1}}= 5\times 10^{4}- 10^{6}\)GeV has prominent points satisfying the desired BAU in case of NH. For IH, we donot observe much change in the variation of BAU w.r.t \(M_{N_{1}}\) obeying the Planck bound. Also the variation due to the arbitrary angles are almost similar for both the hierarchies. In the second row of Fig. 2, an interesting result is seen which constraints the parameter space of \( K_{N_{1}}\) satisfying the Planck limit depending upon the choice of arbitrary angle \(\omega _{12}\). The points satisfying BAU for \(\omega _{12}= 10^{-12}+10^{-12}i\) are very scanty, whereas for \(\omega _{12}= 10^{-1}+10^{-1}i\) it is seen that we have abundant points for the decay parameter space \(K_{N_{1}}= 10{-}100\), thereby producing thermal leptogenesis and also serving the DM candidate to be a FIMP type for both NH/IH [31, 40]. Again from third row of Fig. 2, we have obtained a clear distinction between the range of tan\(\beta \) satisfying the Planck limit for BAU w.r.t the choice of \(\omega _{12}\). We see that for \(\omega _{12}= 10^{-1}+10^{-1}i\), the range of tan\(\beta \) from 10–100 is seen to produce the observed BAU, whereas for \(\omega _{12}= 10^{-12}+10^{-12}i\) the region of tan\(\beta \) satisfying the BAU limit is constrained from 10–50 with very less points as compared to that for \(\omega _{12}= 10^{-1}+10^{-1}i\) in case of NH. The results obtained for IH is almost similar to that for NH. We have obtained a crucial result from Fig. 3 which remains almost the same for both the mass heirarchies. Here, the parameter space of the Yukawa coupling element \((Y^{\dagger }Y)_{11}\) which has a significant contribution in the CP asymmetry is constrained for \(\omega _{12}= 10^{-12}+10^{-12}i\) in a small region obeying the BAU limit. Also we have obtained very few points in the parameter space satisfying the Planck bound incase of lower value of \(\omega _{12}\). However, we have quite a wide range of \((Y^{\dagger }Y)_{11}\) corresponding to \(\omega _{12}= 10^{-1}+10^{-1}i\) satisfying the Planck bound for BAU, i.e. \((Y^{\dagger }Y)_{11}= 10^{-8}-10^{-5}\). Figure 4 is a co-relation plot between the relic abundance and the BAU. Here, we see that there are sufficient common points for the choice of \(\omega _{12}=10^{-1}+10^{-1}i\) satisfying both the bounds simultaneously, though we have almost negligible number of points obeying both the bounds incase of \(\omega _{12}=10^{-12}+10^{-12}i\). The results are similar for both the heirarchies.

From Fig. 5, we see a divergence in the plot of decay parameter w.r.t the lightest active neutrino mass. For \(\omega _{12}= 10^{-1}+10^{-1}i\), the decay parameter \( K_{N_{1}}\) in the allowed region, i.e. 10–100 from the thermal leptogenesis point of view is obtained for light neutrino mass in the space \(10^{-19}-10^{-21}\). This range of \(K_{N_{1}}\) corresponds to warm DM samples. Whereas, for \(\omega _{12}= 10^{-12}+10^{-12}i\), there is a shift in the mass range of lightest active neutrino. However, there is hardly any change in the effective mass curve depicted in Fig. 6 irrespective of the two different choice of \(\omega _{12}\) for NH as well as IH. Typically for a warm DM, the branching ratio \(\mathrm{Br}_{\xi }> 10^{-4}\), whereas for a cold DM it is \(\mathrm{Br}_{\xi }\lesssim 10^{-3}\).

Fig. 6
figure 6

Effective mass of the neutrinos w.r.t the lightest active neutrino is being showed for NH/IH with two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\). The horizontal black line signify the KamLAND-Zen upper limit on the effective mass of neutrinos

Full size image
Fig. 7
figure 7

The plot shows variation of relic abundance w.r.t the dark matter mass for NH/IH. Two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for NH/IH is chosen

Full size image
Fig. 8
figure 8

The plot shows variation of relic abundance w.r.t the decay parameter \( K_{N_{1}}\) for NH/IH for two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\)

Full size image
Fig. 9
figure 9

The plot shows variation of relic abundance w.r.t the branching ratio of the decay \(N_{1}\longrightarrow \xi \chi \) for two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for NH/IH

Full size image
Fig. 10
figure 10

The plot shows variation of free streaming length w.r.t the decay parameter \( K_{N_{1}}\) for NH/IH. Here, the variation is shown for two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\)

Full size image
Fig. 11
figure 11

The plot shows variation of free streaming length w.r.t the relic abundance of \(\xi \) for two different values of arbitrary angle \(\omega _{12}\) and benchmark values of \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for NH/IH

Full size image
Fig. 12
figure 12

Contour plot relating the model parameters a and b w.r.t the relic abundance (\(\Omega _{DM}h^{2}\)) of the DM candidate (\(\xi \)) (left panel), right panel shows variation of b and c with \(\Omega _{DM}h^{2}\) and in the middle we show a similar plot for a and c. This result corresponds to benchmark points \(\omega _{12}= 10^{-1}+10^{-1}i\), \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for NH

Full size image

For further deduction of the relic abundance numerically, we choose the branching ratio in the range \(\mathrm{Br}_{\xi }= 10^{-2}\)–\(10^{-7}\). From Fig. 7, we see that for \(\omega _{12}= 10^{-1}+10^{-1}i\), we get observed relic abundance for DM mass in the range 5–100 GeV which corresponds to warm DM, whereas for \(\omega _{12}= 10^{-12}+10^{-12}i\), smaller mass of \(\xi \) signifying hot DM also satisfies the relic abundance limit. In a plot between relic abundance and decay parameter (\( K_{N_{1}}\)) as shown in Fig. 8, we observe that the parameter space for \( K_{N_{1}}=10{-}100\) satisfies the Planck limit for relic abundance for \(\omega _{12}= 10^{-1}+10^{-1}i\) whereas for \(\omega _{12}= 10^{-1}+10^{-1}i\), the same region of \( K_{N_{1}}\) hardly satisfies relic abundance constraint. However, for satisfying the small structure formation constraint, \( K_{N_{1}}\) must not fall in the very weak washout region, therefore, \(\omega _{12}= 10^{-12}+10^{-12}i\) is less favorable for both NH/IH. Also as mentioned earlier the preferable branching ratio range for warm DM satisfying the relic abundance bound is obeyed for \(\omega _{12}= 10^{-1}+10^{-1}i\) as depicted in Fig. 9. We know that the most stringent bound on \(r_{FS}\) comes from the small structure formation, \(r_{FS}< 0.1\). A relationship between \(r_{FS}\) and \( K_{N_{1}}\) is shown in Fig. 10, where we have all the points for both the cases of \(\omega _{12}\) in the allowed region. As we know that \( K_{N_{1}}\) is inversely proportional to \(r_{FS}\), thus, very small values of \( K_{N_{1}}\) donot satisfy the small structure formation bound. An analysis on the points satisfying both relic abundance and small structure formation is been shown in Fig. 11, wherein we see that almost all the points corresponding \(r_{FS}= 0.01{-}0.1\) satisfy the relic abundance bound. Also from Fig. 7, we have obtained the allowed range of warm DM mass satisying the relic abundance to be 5–100 KeV, thus we can summarise that this range of DM mass also abides the streaming length constraint. We have further shown some contour plots between the Dirac masses a, b and c w.r.t the allowed range for relic abundance of the dark matter, i.e. \(\Omega _{DM}h^{2}= 0.1186 \pm 0.0020\) given by Planck data. Thus, in Figs. 12, 13 we have shown the parameter space of a, b and c which satisfies the relic abundance limit corresponding to \(\omega _{12}=10^{-1}+10^{-1}i\) for both NH and IH respectively. Again similar plots are depicted for \(\omega _{12}=10^{-12}+10^{-12}i\) in Figs. 14, 15. Summarising the allowed parameter space of the Dirac masses from the above mentioned figures, we finally represent it in tabular form in Tables 2 and 3.

Fig. 13
figure 13

Contour plot relating the model parameters a and b w.r.t the relic abundance (\(\Omega _{DM}h^{2}\)) of the DM candidate (\(\xi \)) (left panel), right panel shows variation of b and c with \(\Omega _{DM}h^{2}\) and in the middle we show a similar plot for a and c. This result corresponds to benchmark points \(\omega _{12}= 10^{-1}+10^{-1}i\), \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for IH

Full size image
Fig. 14
figure 14

Contour plot relating the Dirac masses a and b w.r.t the relic abundance (\(\Omega _{DM}h^{2}\)) of the DM candidate (\(\xi \)) (left panel), right panel shows variation of b and c with \(\Omega _{DM}h^{2}\) and in the middle we show a similar plot for a and c. This result corresponds to benchmark points \(\omega _{12}= 10^{-12}+10^{-12}i\), \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for NH

Full size image
Fig. 15
figure 15

Contour plot relating the Dirac masses a and b w.r.t the relic abundance (\(\Omega _{DM}h^{2}\)) of the DM candidate (\(\xi \)) (left panel), right panel shows variation of b and c with \(\Omega _{DM}h^{2}\) and in the middle we show a similar plot for a and c. This result corresponds to benchmark points \(\omega _{12}= 10^{-12}+10^{-12}i\), \(\omega _{13}= 10^{-3}+10^{-3}i\) and \(\omega _{23}= 10^{-2}+10^{-2}i\) for IH

Full size image
Table 2 Dirac masses of the model and their respective parameter space satisfying the relic abundance of the DM candidate \(\xi \) for \(\omega _{12}=10^{-1}+10^{-1}i\)
Full size table
Table 3 Dirac masses of the model and their respective parameter space satisfying the relic abundance of the DM candidate \(\xi \) for \(\omega _{12}=10^{-12}+10^{-12}i\)
Full size table

6 Conclusion

Our work basically showcases a comparative analysis of different phenomenological consequences corresponding to variation in the arbitrary angles of the rotational matrix. As mentioned in the earlier sections, we have realised the extended \(\nu \)2HDM with the help of flavor symmetries \(A_{4}\otimes Z_{4}\). It is an interesting model as it can accomodate both neutrino as well as DM phenomenologies. Also, we can generate the light neutrino mass at the tree level by consideration of an extra scalar doublet which is assigned L=-1 [21]. From the model, we numerically calculate the Yukawa coupling matrix, which plays the source for the various phenomenologies that we have studied. Also, we have chosen the rotational matrix R and the values of the arbitrary angles such that we can draw crucial conclusions of its impact on the cosmological phenomena we have discussed. Following various literatures [5, 55], we have chosen the values of the RHN masses, lightest active neutrino mass in the range \(10^{-13}-10^{-11}\)eV, vev of the scalar doublet (\(\phi \)) as already mentioned in Sect. 5 so as to achieve TeV scale leptogenesis. We have also co-related the BAU with the values of tan\(\beta \) and analyzed its variation for different \(\omega _{12}\) values. From Figs. 2, 3 and 4 we can conclude that the choice \(\omega _{12}= 10^{-1}+10^{-1}i\) is more preferable in satisfying the BAU limit given by Planck data. Also, we have obtained a constraint range of the free parameters as well as the Yukawa couplings corresponding to the two choices of the arbitrary angles. In plot (4) we obtain points which abide by the constraint from both BAU and relic abundance simultaneously which is a vital part of the analysis. We have also generated some important results in context with the decay parameter. Depending on \(\omega _{12}\) values, we can generalise the transition in the relation between \( K_{N_{1}}\) and \(m_{l}\) for both NH and IH. Considering the DM scenario, it is viable to have a warm DM candidate as it obeys the constraint coming from small structure formation and relic abundance w.r.t to the allowed parameter space of decay parameter in comparison to hot DM. Simultaneously, we can get a warm DM source when the decay parameter donot fall in the very weak washout region. Therefore, it is seen from Fig. 5 that for \(\omega _{12}=10^{-1}+10^{-1}i\) we obtain the preferable range of \( K_{N_{1}}\) which further explains the warm DM. For warm DM, \(m_{\xi }\sim 10\) KeV gives significant observed relic abundance, however, small DM mass correspond to hot DM which again is constrained by small structure formation. Again, a difference in the branching ratio range is obtained depending on the two different values of \(\omega _{12}\). So, we can say that \(\mathrm{Br}_{\xi }\simeq 10^{-5}-10^{-3}\) is the prefered range for \(\omega _{12}= 10^{-1}+10^{-1}i\) obeying the Planck limit for relic abundance, whereas for \(\omega _{12}= 10^{-12}+10^{-12}i\), \(\mathrm{Br}_{\xi }\simeq 10^{-6}-10^{-3}\) produces the observed relic abundance. Summarizing the above results, it can be said that for the choice of \(\omega _{12}= 10^{-1}+10^{-1}i\), the value of \( K_{N_{1}}\) falls in the weak washout region which further is successful in generating the desired BAU, relic abundance and also the small structure formation for both NH and IH. Also, the Yukawa couplings obtained from the model are successful in producing the branching ratio Br \((\mu \rightarrow e\gamma )<4.2 \times 10^{-13}\) as shown in Fig. 1. Thus, the model can explain the neutrino mass, leptogenesis and dark matter for the choice of the free parameters considered and the values of Dirac masses obtained.