Type-I thermal leptogenesis in $Z_3$-symmetric three Higgs doublet model

Abstract

Our present work explores the possibility of neutrino mass generation through Type-I see-saw mechanism and provides an explanation of the baryon asymmetry of the Universe via thermal leptogenesis in the framework of a $Z_3$-symmetric three Higgs doublet model (3HDM) augmented with three right-handed neutrinos. Here the thermal leptogenesis is initiated by the out-of-equilibrium decay of the lightest heavy neutrino $N_1$. The constraints arising out of the scalar sector put strong bound on the model parameter $\tan \beta$, which in turn takes part in the computation of the lepton asymmetry $\epsilon$. Lepton asymmetry being converted partially into the baryon asymmetry by electroweak sphelaron processes, will account for the required baryon asymmetry satisfying the current data. We therefore analyse the parameter space consistent with the constraints arising from neutrino oscillation, lepton asymmetry and baryon asymmetry together, last one turning out to be the most stringent one.

1 Introduction

While the Standard Model (SM) particle spectrum is complete after Higgs discovery [1, 2], there are several reasons to believe that this is not the ultimate story but only an effective theory valid up to some high energy scale, above which some other theory takes over. Like other shortcomings of SM, explanation of neutrino mass generation necessitates a beyond-the-SM (BSM) scenario incorporating different types of see-saw mechanisms [3,4,5,6,7,8,9]. Addition of right-handed (RH) neutrinos, which are singlets under SM gauge group, results in the creation of neutrino mass via Type-I see-saw mechanism [3,4,5,6,7]. Besides, the observed imbalance between the number of baryons and antibaryons is yet another issue that remains unaddressed within the ambit of SM [10,11,12]. The dynamic generation of baryon asymmetry needs to comply with three Sakharov conditions [13, 14], which require : (a) baryon number violation, (b) C or CP-violation, (c) out-of-equilibrium condition. The current data reads [15]:

$$\begin{aligned} \eta _B \equiv \frac{n_B - n_{\overline{B}}}{n_\gamma } = (6.12 \pm 0.04) \times 10^{-10}. \end{aligned}$$

(1)

$n_B, ~ n_{\overline{B}}, ~ n_\gamma$ being number densities of baryons, anti-baryons and photons respectively. Thus new physics (NP) needs to be introduced to compensate the due amount of baryon asymmetry within SM.

The out-of-equilibrium decay of the RH heavy neutrinos in Type-I see-saw mechanism, induces leptogenesis [16, 17], that can make up for the aforementioned baryon imbalance. Complex Yukawa couplings give rise to CP-violation, thereby fulfilling the required criteria for generating baryon asymmetry. At the epoch of generation of asymmetry, the decay rate being slower than the expansion rate of the Universe, out-of-equilibrium condition is automatically fulfilled. Finally a partial conversion of lepton asymmetry (created during the out-of-equilibrium decay of heavy neutrinos) to baryon asymmetry, occurs through $(B+L)$ violating electroweak (EW) sphelaron processes [18].

As stated earlier, the inability of SM, to address the issues of neutrino mass generation and baryon asymmetry calls for a BSM scenario. A particular way in this direction is to extend the SM by spin-0 degrees of freedom only. Moreover, extending the SM scalar sector by $SU(2)_L$ doublets only, is an attractive choice since the tree level electroweak $\rho$-parameter is kept intact. The most minimal multi-doublet extension comprises two Higgs doublets leading up to what is known as two Higgs doublet models (2HDM) [19, 20]. However, as there is no fundamental principle to pinpoint the exact number of doublets present, more non-minimal extensions are also possible. In fact, three Higgs doublet models (3HDMs) [21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45] have been attracting attention for quite some time now. The main motivation of 3HDM lies in the fact that, the masses and mixings of three fermionic generations can be properly reproduced, when these three doublets are connected to the three fermionic generations via appropriate symmetries. Examples of such discrete symmetries include $A_4, S_4, S_3, Z_3$ etc. A $Z_3$-symmetric 3HDM [21, 35, 45] resembles the democratic 3HDM (where three doublets individually couple to up-type quarks, down-type quarks and leptons) [46,47,48] via proper $Z_3$-charge assignment to quarks and leptons as will be discussed later. Another important aspect of this $Z_3$-symmetric 3HDM is to promote the natural flavour conservation (NFC) by prohibiting the tree level flavour changing neutral currents (FCNCs).

In this paper, we uphold the $Z_3$-symmetric 3HDM augmented with three heavy RH neutrinos as a possible framework to address the two aforementioned shortcomings of SM. In particular, here we shall focus on thermal leptogenesis [17, 49,50,51,52,53,54,55], which allows hierarchical heavy neutrino masses, mass of one of them being much smaller than others. Besides, only thermal generation and out-of-equilibrium decay of lightest heavy neutrino will play the crucial role in generating lepton asymmetry. In presence of three RH neutrinos, mass generation of light neutrinos will be possible via Type-I see-saw mechanism. As can be seen later, the entire parameter space will be constrained by the restrictions coming from the scalar sector, as well as the more stringent constraints arising from neutrino oscillation data, lepton asymmetry and baryon asymmetry respectively. Thermal type-I leptogenesis in a minimal scenario containing RH neutrinos along with SM Higgs doublet has been analysed in [56,57,58]. Natural and thermal leptogenesis has been studied in the framework of 2HDM extended by RH neutrinos in [59, 60]. Analysis of the scalar sector of the $Z_3$-symmetric 3HDM along with three heavy RH neutrinos has not been performed earlier in light of the theoretical, experimental constraints. In addition, any study of type-I thermal leptogenesis has not been done within this particular model. Thus there is a huge impulse for analysing this model in light of type-I thermal leptogenesis. There can be another variant of leptogenesis, in which the CP-asymmetry is enhanced by considering the mass-splitting between any two of the heavy neutrinos to be comparable with their decay width. This type of leptogenesis is termed as Resonant leptogenesis [61, 62]. Since the lower bound on the heavy neutrino mass is relaxed in this case, collider searches involving these neutrinos are feasible in the future colliders. To understand the importance of flavor effects on leptogenesis, we refer the readers to go through the papers [63,64,65,66,67,68]. However we shall restrict ourselves in studying thermal leptogenesis in this paper and shall not consider the other variants.

This paper is structured as follows. Section 2 contains the information regarding the particle content of the model considered for analysis. Section 3 comprises of detailed discussion of several constraints imposed on the parameter space. In Sect. 4, we elaborate the fitting of neutrino oscillation data using Casas Ibarra parametrization. Section 5 deals with thermal leptogenesis, i.e. solutions of Boltzman equations. In Sect. 6, we present analysis and results. Finally we summarize and conclude in Sect. 7.

2 Model

In this analysis, we consider $Z_3$-symmetric 3HDM comprising of three $SU(2)_L$ doublets $\phi _1, \phi _2$ and $\phi _3$ each with hyper-charge $Y= +1$,¹ augmented with three heavy RH neutrinos $N_{1R}$, $N_{2R}$, $N_{3R}$. For simplicity, we shall denote these three heavy neutrinos as $N_1, N_2, N_3$ throughout the analysis. The complete description of different sets of quantum numbers assigned to all the particles can be found in Table 1.

Table 1 Different quantum numbers assigned to the particles in the model. Here $\omega = e^{\frac{2 i \pi }{3}}$

2.1 $Z_3$-symmetric Scalar Lagrangian

Following the quantum numbers assigned to the doublets, as mentioned in Table 1, the $Z_3$-symmetric scalar potential involving $\phi _1, \phi _2$ and $\phi _3$ can be written as [45],

$$\begin{aligned} V(\phi _1, \phi _2, \phi _3)&= m_{11}^2 (\phi _1^\dag \phi _1) + m_{22}^2 (\phi _2^\dag \phi _2) + m_{33}^2 (\phi _3^\dag \phi _3) \nonumber \\&\quad + \frac{\lambda _1}{2} (\phi _1^\dag \phi _1)^2 + \frac{\lambda _2}{2} (\phi _2^\dag \phi _2)^2 + \frac{\lambda _3}{2} (\phi _3^\dag \phi _3)^2 \nonumber \\&\quad + \lambda _4 (\phi _1^\dag \phi _1)(\phi _2^\dag \phi _2) + \lambda _5 (\phi _1^\dag \phi _1)(\phi _3^\dag \phi _3)\nonumber \\&\quad + \lambda _6 (\phi _2^\dag \phi _2)(\phi _3^\dag \phi _3) + \lambda _7 (\phi _1^\dag \phi _2)(\phi _2^\dag \phi _1) \nonumber \\&\quad + \lambda _8 (\phi _1^\dag \phi _3)(\phi _3^\dag \phi _1) + \lambda _9 (\phi _2^\dag \phi _3)(\phi _3^\dag \phi _2) \nonumber \\&\quad + \left[ \lambda _{10} (\phi _1^\dag \phi _2)(\phi _1^\dag \phi _3) + \lambda _{11} (\phi _1^\dag \phi _2)(\phi _3^\dag \phi _2) \right. \nonumber \\&\quad \left. + \lambda _{12} (\phi _1^\dag \phi _3)(\phi _2^\dag \phi _3)+ \mathrm{h.c.}\right] \end{aligned}$$

(2)

After symmetry breaking, $\phi _i$ can be expressed as,

$$\begin{aligned} \phi _i = \begin{pmatrix} h_i^+ \\ \frac{1}{\sqrt{2}} (v_i + h_i + i \rho _i) \end{pmatrix}, i=1,2,3 \end{aligned}$$

(3)

$v_i$ being the vacuum expectation value (VEV) of $\phi _i$. Two important parameters of the model $\tan \beta$ and $\tan \gamma$ can be expressed as the ratios of VEVs of doublets : $\tan \beta = \frac{\sqrt{v_1^2 + v_2^2}}{v_3}, \tan \gamma = \frac{v_2}{v_1}$. Therefore $v_1, v_2$ and $v_3$ can be written in terms of the mixing angles $\beta$ and $\gamma$ as :

$$\begin{aligned} v_1= & {} v \sin \beta ~\cos \gamma , \nonumber \\ v_2= & {} v \sin \beta ~ \sin \gamma , \nonumber \\ v_3= & {} v \cos \beta \,, ~\mathrm{with}~ v = \sqrt{v_1^2 + v_2^2 + v_3^2 } = 246~ \mathrm{GeV} \end{aligned}$$

(4)

The quartic couplings $[\lambda _1, \lambda _2,... \lambda _{12}]$ and the doublet VEVs $[v_1, v_2, v_3]$ are taken to be real to avoid any kind of CP-violation in the scalar potential. The particle spectrum of the model comprises of seven physical scalars, namely $h, H_1, H_2, A_1, A_2, H_1^{\pm }, H_2^{\pm }$. Twelve quartic couplings $[\lambda _1, \lambda _2,... \lambda _{12}]$ can be rewritten in terms of the aforementioned seven physical masses ($M_h, M_{H_1}, M_{H_2}, M_{A_1}, M_{A_2}, M_{H_1^{\pm }}, M_{H_2^{\pm }}$) and five mixing angles, i.e. three in the CP-even sector ($\alpha _1, \alpha _2, \alpha _3$), one in CP-odd sector ($\gamma _1$) and one in charged scalar sector ($\gamma _2$) [45]. The lightest neutral physical state h resembles the SM Higgs boson with mass 125 GeV at the Alignment limit defined as : $\alpha _1 = \gamma , \alpha _2 + \beta = \frac{\pi }{2}$ [45].

The details of the scalar sector of $Z_3$-symmetric 3HDM including the basis transformations from flavor basis to mass basis etc. can be found in [45]. To avoid repetition, we shall not provide the same details here.

2.2 Yukawa Lagrangian

Due to the particular assignment of $Z_3$-charges to the fields (shown in Table 1), the flavor changing neutral currents (FCNCs) are forbidden in this model. Up-type, down-type quarks and leptons will acquire masses through the couplings with $\phi _1, \phi _2$ and $\phi _3$ respectively. Due to the presence of three heavy RH neutrinos, SM light neutrinos can also acquire masses via Type-I see-saw mechanism, only $\phi _3$ being responsible for the mass generation of neutrinos.

Thus we can write down the $Z_3$-symmetric Yukawa Lagrangian along with the Majorana mass terms for the heavy neutrinos as:

$$\begin{aligned} -\mathcal {L}_Y&= y_1 \overline{L}_1 \tilde{\phi _3} N_1 + y_2 \overline{L}_1 \tilde{\phi _3} N_2 + y_3 \overline{L}_1 \tilde{\phi _3} N_3 \nonumber \\&\quad + y_4 \overline{L}_2 \tilde{\phi _3} N_1 + y_5 \overline{L}_2 \tilde{\phi _3} N_2 + y_6 \overline{L}_2 \tilde{\phi _3} N_3 \nonumber \\&\quad + y_7 \overline{L}_3 \tilde{\phi _3} N_1 + y_8 \overline{L}_3 \tilde{\phi _3} N_2 + y_9 \overline{L}_3 \tilde{\phi _3} N_3 \nonumber \\&\quad + \frac{1}{2} M_1 \overline{{N}_{1}^{c}} N_1 + \frac{1}{2} M_2 \overline{{N}_{2}^{c}} N_2 + \frac{1}{2} M_3 \overline{{N}_{3}^{c}} N_3+ \mathrm{h.c.}\nonumber \\ \end{aligned}$$

(5)

Here $L_i$ are left-handed (LH) lepton doublets and $\tilde{\phi _i} = i \sigma _2 \phi _i^{*}$. As mentioned earlier, only $\phi _3$ will be responsible for generating SM light neutrino masses. Yukawa couplings $y_j$ are taken to be complex for generating CP-asymmetry in leptogenesis. The real and imaginary parts of the Yukawa couplings $y_j$ are constrained by recent neutrino oscillation data [69], as will be discussed elaborately in Sect. 4.

3 Constraints to be considered

For the analysis, we shall consider a multi-dimensional parameter space, spanned by the following independent parameters : $\tan \beta , \gamma , \gamma _1 , \gamma _2, M_h, M_{H_1}, M_{H_2}, M_{A_1}, M_{A_2}, M_{H_1^{\pm }}, M_{H_2^{\pm }}, \alpha _3$ ($\alpha _1, \alpha _2$ are connected to $\gamma _1$ and $\gamma _2$ respectively through the alignment conditions : $\alpha _1 = \gamma , ~ \alpha _2 + \beta = \frac{\pi }{2}$, thus are not independent). Since Alignment limit will be imposed strictly, the lightest Higgs h being SM-like, $M_h$ is taken to be 125 GeV. We have checked that the variation of $\alpha _3$ hardly induces any change in the parameter space. The effect of scanning over the other variables like physical masses and mixing angles surpasses the mild effect of changing $\alpha _3$. To illustrate this, we refer to the plot of $M_{H_1}$ vs. $M_{H_1^+}$ plane for $\tan \beta = 3$ and three different values of $\alpha _3 = \frac{\pi }{6}, \frac{\pi }{4}, \frac{\pi }{2}$ in Fig. 1. It shows that for three different values of $\alpha _3$, the parameter space in the mass plane changes only mildly. The same conclusion can be drawn for the other masses and $\tan \beta$ as well. Therefore, to simplify the numerical scans, we fix $\alpha _3 = \frac{\pi }{4}$ throughout the rest of the analysis.

Fig. 1

Plot of the parameter space in $M_{H_1}$ vs. $M_{H_1^+}$ plane for $\alpha _3 = \frac{\pi }{6}, \frac{\pi }{4}, \frac{\pi }{2}$ and $\tan \beta = 3$

In addition to these, the constraints to be imposed on the parameter space are discussed below.

3.1 Theoretical constraints

• The real quartic couplings $[\lambda _1, \lambda _2,... \lambda _{12}]$ are taken to be pertubative , i.e. $|\lambda _i| \le 4\pi$.

• Yukawa couplings $y_j$ ($j = 1,2,...9$) are constrained from the neutrino oscillation data and constraints arising from leptogenesis, as will be discussed later. There is also an upper bound of $|y_j| \le \sqrt{4\pi }$ arising from perturbativity.

• Boundedness of the scalar potential (Eq. (2)) can be ensured by satisfying following stability conditions involving the quartic couplings:

• 1. $\lambda _1, \lambda _2, \lambda _3 \ge 0$

• 2. $\lambda _4 \ge - \sqrt{ \lambda _1 \lambda _2}$

• 3. $\lambda _5 \ge - \sqrt{ \lambda _1 \lambda _3}$

• 4. $\lambda _6 \ge - \sqrt{ \lambda _2 \lambda _3}$

• 5. $\lambda _5 + \lambda _8 \ge - \sqrt{\lambda _1 \lambda _3}$

• 6. $\lambda _6 + \lambda _9 \ge - \sqrt{\lambda _2 \lambda _3}$

• 7. $\lambda _4 + \lambda _7 \ge - \sqrt{\lambda _1 \lambda _2}$

• 8. $|\lambda _{10}|, |\lambda _{11}|, |\lambda _{12}| < |\lambda _i|,~ i=1,2,...,9$.

Here first seven conditions come from the phase-invariant part of the scalar potential which includes all the terms in Eq. (2) except last three terms. The last stability condition arises only from the $Z_3$-symmetry ensuring terms (last three terms in Eq. (2)).

3.2 Constraints from oblique parameters

In presence of additional scalars in the model, the oblique parameters S, T, U will be modified accordingly. The present limits on their deviation from SM values are [70]:

$$\begin{aligned} \Delta S= & {} 0.02 \pm 0.01, \nonumber \\ \Delta T= & {} 0.07 \pm 0.12, \nonumber \\ \Delta U= & {} 0.00 \pm 0.09. \end{aligned}$$

(6)

Specially the $Z_3$-symmetric 3HDM parameter space is sensitive to the deviation of T-parameter from SM value, because this deviation controls the mass-splitting between the charged and the neutral scalars. We have ensured the compatibility with T-parameter constraint by keeping the mass-splitting between the charged and the neutral scalars $\sim$ 50 GeV.

3.3 Constraints on Higgs signal-strengths from LHC data

To make the parameter space compatible with the current LHC data, one has to compute Higgs signal strengths in different Higgs decay channels. For the decay channel $h \rightarrow X Y$, the signal strength $\mu _{X Y}$ can be computed as the ratio of cross section of Higgs production via $p-p$ collision times the branching ratio of Higgs decay into the channel $h \rightarrow X Y$ in 3HDM and the same quantity measured in the SM:

$$\begin{aligned} \mu _{X Y} = \frac{\sigma ^\mathrm{{3HDM}}(pp \rightarrow h)~ \mathrm{BR^{3HDM}}(h \rightarrow X Y)}{\sigma ^\mathrm{{SM}}(pp \rightarrow h)~ \mathrm{BR^{SM}}(h \rightarrow X Y)}. \end{aligned}$$

(7)

Among all Higgs production process, the dominant contribution at the LHC comes from the gluon–gluon fusion process mediated by heavy quarks in triangular loops. The parton-level cross section can be written as [71]

$$\begin{aligned} \sigma (gg \rightarrow h) = \frac{\pi ^2}{8 M_h} \Gamma (h \rightarrow gg)~ \delta (\hat{s} - M_h^2), \end{aligned}$$

(8)

$\hat{s}$ being gluon-gluon invariant energy squared.

Using Eqs. (7) and (8), one can rewrite the signal strength $\mu _{X Y}$ as:

$$\begin{aligned} \mu _{X Y}= & {} \frac{\sigma ^\mathrm{{3HDM}}(gg \rightarrow h)}{\sigma ^\mathrm{{SM}}(gg \rightarrow h)} ~\frac{\Gamma _{X Y}^\mathrm{{3HDM}}(h \rightarrow X Y )}{\Gamma _\mathrm{{tot}}^\mathrm{{3HDM}}}\nonumber \\&~\frac{\Gamma _\mathrm{{tot}}^\mathrm{{SM}}}{\Gamma _{X Y}^\mathrm{{SM}}(h \rightarrow X Y)}, \nonumber \\= & {} \frac{\Gamma ^\mathrm{{3HDM}}(h \rightarrow gg)}{\Gamma ^\mathrm{{SM}}(h \rightarrow gg)} ~\frac{\Gamma _{X Y}^\mathrm{{3HDM}}(h \rightarrow X Y)}{\Gamma _\mathrm{{tot}}^\mathrm{{3HDM}}}\nonumber \\&~\frac{\Gamma _\mathrm{{tot}}^\mathrm{{SM}}}{\Gamma _{X Y}^\mathrm{{SM}}(h \rightarrow X Y)}. \end{aligned}$$

(9)

where $\Gamma _\mathrm{{tot}}$ stands for the total decay width.

Since the Alignment limit is being invoked strictly, the lightest Higgs h being SM like, the Higgs signal strengths in the $WW, ZZ, b \overline{b}, \tau ^+ \tau ^-$ mode are satisfied automatically. $\Gamma (h \rightarrow \gamma \gamma )$ receives an extra contribution coming from the charged Higgs mediated loop and are modified. At the exact Alignment limit, the total Higgs decay width coincides with that of the SM Higgs h. Thus the signal strength $\mu _{h \rightarrow \gamma \gamma }$ can be approximated to $\frac{\Gamma ^\mathrm{{3HDM}}(h \rightarrow \gamma \gamma )}{\Gamma ^\mathrm{{SM}}(h \rightarrow \gamma \gamma )}$. Expressions for the decay width $\Gamma (h \rightarrow \gamma \gamma )$ can be found in appendix A. We have used $2\sigma$-deviation from the allowed values of signal strength to scan the parameter space [72].

4 Fitting of neutrino-data

As mentioned earlier, the Yukawa couplings need to be complex in order to generate lepton asymmetry required for leptogenesis. Following the Yukawa Lagrangian in Eq. (5), after symmetry breaking, the Dirac mass matrix $M_D$ can be computed as:

$$\begin{aligned} M_D= & {} \frac{v_3}{\sqrt{2}} ~ Y_{ij} = \frac{v_3}{\sqrt{2}} \begin{pmatrix} y_1 &{} y_2 &{} y_3 \\ y_4 &{} y_5 &{} y_6 \\ y_7 &{} y_8 &{} y_9 \end{pmatrix}\nonumber \\= & {} \frac{v \cos \beta }{\sqrt{2}} \begin{pmatrix} (y_{1R} + i y_{1I}) &{} (y_{2R} + i y_{2I}) &{} (y_{3R} + i y_{3I}) \\ (y_{4R} + i y_{4I}) &{} (y_{5R} + i y_{5I}) &{} (y_{6R} + i y_{6I}) \\ (y_{7R} + i y_{7I}) &{} (y_{8R} + i y_{8I}) &{} (y_{9R} + i y_{9I}) \end{pmatrix}.\nonumber \\ \end{aligned}$$

(10)

Here complex Yukawa couplings $y_j$s are decomposed into real and imaginary parts as : $y_{jR}$ and $y_{jI}$ respectively. Majorana mass matrix $M_R$ is assumed to be diagonal for simplicity :

$$\begin{aligned} M_R = \begin{pmatrix} M_1 &{} 0 &{} 0 \\ 0 &{} M_2 &{} 0 \\ 0 &{} 0 &{} M_3 \end{pmatrix}. \end{aligned}$$

(11)

For the mass generation of light neutrinos through Type-I see-saw mechanism, the neutrino mass matrix can be expressed in terms of Dirac mass matrix $M_D$ and Majorana mass matrix $M_R$ as:

$$\begin{aligned} M_\nu = - M_D M_R^{-1} M_D^T. \end{aligned}$$

(12)

$M_\nu$ can be diagonalised to get the light neutrino masses by the transformation:

$$\begin{aligned} U_\mathrm{PMNS}^T~ M_\nu ~U_\mathrm{PMNS} = \mathrm{diag}(m_1,m_2,m_3) = \widehat{M_\nu }. \end{aligned}$$

(13)

where $m_1, m_2, m_3$ are three light neutrino masses, $U_\mathrm{PMNS}$ is the Pontecorvo–Maki–Nakagawa–Sakata matrix (PMNS) matrix and can be written as:

$$\begin{aligned} U_\mathrm{PMNS} = \begin{pmatrix} c_{13}c_{12} &{} c_{13} s_{12} &{} s_{13}e^{-i \delta _\mathrm{CP}} \\ -s_{12}c_{23}-c_{12}s_{23}s_{13}e^{i \delta _{CP}} &{} c_{12} c_{23}-s_{12} s_{23} s_{13} e^{i \delta _\mathrm{CP}}&{} s_{23} c_{13} \\ s_{12} s_{23} - c_{12} c_{23} s_{13}e^{i \delta _\mathrm{CP}} &{} -c_{12} s_{23} - s_{12} c_{23} s_{13} e^{i \delta _\mathrm{CP}} &{} c_{23} c_{13} \end{pmatrix}, \end{aligned}$$

(14)

where $c_{ij} \equiv \cos \theta _{ij} , s_{ij} \equiv \sin \theta _{ij}$ and $\delta _{CP}$ is the CP-violating phase. To parametrize the elements of $M_D$, one can use the parametrization proposed by Casas and Ibarra (CI) [73], as will be mentioned in detail in the next subsection.

4.1 Casas Ibarra parametrization

According to the CI parametrization² [73], $M_D$ can be rewritten as:

$$\begin{aligned} M_D = \frac{v_3}{\sqrt{2}} Y_{ij} = i~U_\mathrm{PMNS} \sqrt{\widehat{M_\nu }} ~\mathcal {O} ~\sqrt{M_R} . \end{aligned}$$

(15)

$\mathcal {O}$ being a general complex orthogonal matrix, with complex angles $\theta , \chi , \psi$, can be expressed as [17],

$$\begin{aligned} \mathcal {O} = \begin{pmatrix} c_{\chi } c_{\psi } &{} c_{\chi }s_{\psi } &{} s_{\chi } \\ -c_{\theta } s_{\psi } - s_{\theta } s_{\chi } c_{\psi } &{} c_{\theta } c_{\psi } - s_{\theta } s_{\chi } s_{\psi } &{} s_{\theta } c_{\chi } \\ s_{\theta } s_{\psi } - c_{\theta } s_{\chi } c_{\psi } &{} -s_{\theta } c_{\psi } - c_{\theta } s_{\chi } s_{\psi } &{} c_{\theta } c_{\chi } \end{pmatrix}, \end{aligned}$$

(16)

where $c_\alpha , s_\alpha$ are the shorthand notations for $\cos \alpha$ and $\sin \alpha$ respectively. Here $\widehat{M_\nu }$ is the diagonal light neutrino mass matrix.

The angles $\theta , \chi , \psi$ in Eq. (16), can be complex in general, but for our analysis, we have chosen the phase associated with the angles to be zero, i.e. the angles are chosen to be real for simplicity. One of the three light neutrinos is taken to be massless, i.e. $m_1 = 0$. We have considered the Normal hierarchy (NH) among $m_1, m_2, m_3$. From Eq. (15), it is evident that one can evaluate the elements of matrix $M_D$, i.e. the real and imaginary parts of Yukawa couplings (18 real parameters), in terms of the elements of $U_\mathrm{PMNS}$ matrix (which is known from neutrino oscillation data), angles $\theta , \chi , \psi$ in orthogonal matrix $\mathcal {O}$ and $M_1, M_2, M_3$ in $M_R$. Complex $U_\mathrm{PMNS}$ matrix at the right hand side of Eq. (15) in turn necessitates complex Yukawa couplings $y_j$s in $M_D$ at the left hand side of the same equation. Here we have solved the real and imaginary parts of the Yukawa couplings using Casas Ibarra parametrization, in terms of the matrix elements of $U_\mathrm{PMNS}$, $M_\nu , \mathcal {O}, M_R$ and $v_3$ following Eqs. (10) and (15), to make them consistent with the neutrino oscillation data. Thus the real and imaginary parts of the Yukawa couplings pick up a $\beta$- dependence. Discussions regarding this will be elaborated in Sect. 5.

5 Leptogenesis

During this analysis, we aim to explore that portion of the parameter space, where the model parameters satisfy the constraints coming from neutrino oscillation data, as well as the current bound on baryon asymmetry. The main mechanism of generating baryon asymmetry here is leptogenesis, through which the lepton asymmetry is produced. In this scenario, the lepton asymmetry is originated by the CP-violating, out-of-equilibrium decay of the lightest heavy RH Majorana neutrino $N_1$. In the limit of hierarchical neutrino masses, i.e. $M_1<< M_2, M_3$, the dominant contribution for generating the lepton asymmetry stems from the decay of $N_1$ only, since the processes mediated by $N_1$ before its out-of-equilibrium decay, abolish the lepton asymmetry created by the decay of $N_2, N_3$ at $T \sim M_1$. Therefore we have to solve two coupled Boltzmann equations involving $Y_{N_1}$ and $Y_{B-L}$. The simultaneous solution of the first and second Boltzmann equations yield comoving density $Y_{N_{1}}$ of the lightest heavy RH-neutrino $N_{1}$ and comoving density $Y_{B-L}$ of $B-L$ asymmetry respectively. Here $Y_{N_{1}}$ ($Y_{B-L}$) is defined as actual number density of $N_{1}$ (${B-L}$ asymmetry) divided by the entropy density $\overline{s}$ of the universe. Entropy density $\overline{s}$ can be written as:

$$\begin{aligned} \overline{s} = \frac{2 \pi ^2}{45} g_{eff} T^3. \end{aligned}$$

(17)

Here T is the temperature³ and $g_{eff}$ is the total effective degrees of freedom which includes all the physical particles of the model. Detailed calculation of $g_{eff}$ is given in appendix D.

In general, the Boltzmann equations for $N_{1}$ and the $(B-L)$ asymmetry can be written as [74],

$$\begin{aligned} \frac{\text {d} Y_{N_{1}}}{\text {d} z}= & {} - \frac{z}{s H(M_{1})} \Big [ \Big (\frac{Y_{N_{1}}}{Y^{eq}_{N_{1}}} - 1\Big )[\gamma _{D_{1}} + 2 \gamma ^{1}_{\phi ,s} + 4 \gamma ^{1}_{\phi ,t}]\Big ], \nonumber \\ \end{aligned}$$

(18)

$$\begin{aligned} \frac{\text {d} Y_{B-L}}{\text {d} z}= & {} - \frac{z}{s H(M_{1})} \Big [\left\{ \frac{1}{2} \frac{Y_{B-L}}{Y^{eq}_{l}} + \epsilon ~\Big (\frac{Y_{N_{1}}}{Y^{eq}_{N_{1}}} - 1\Big ) \right\} \gamma _{D_{1}} \nonumber \\&+ \frac{Y_{B-L}}{Y^{eq}_{l}}\left\{ 2 \gamma _{N,s} + 2 \gamma _{N,t} + 2 \gamma ^{1}_{\phi ,t} + \frac{Y_{N_{1}}}{Y^{eq}_{N_{1}}} \gamma ^{1}_{\phi ,s} \right\} \Big ] \,,\nonumber \\ \end{aligned}$$

(19)

where $z = \frac{M_1}{T}$ and $H(M_1)$ is the Hubble parameter at $T = M_1$: $H(T=M_{1}) = 1.66 ~g_{eff}^{1/2} \frac{T^{2}}{M_\mathrm{{Pl}}}|_{T=M_{1}}$, $M_\mathrm{{Pl}} = 10^{19}$ GeV being Planck scale. $Y_{N_1}^{eq}, Y_l^{eq}$ are the comoving densities at equilibrium. We solve these two equations with initial conditions:

$$\begin{aligned} Y_{N_1}(0) = Y_{N_1}^{eq} , ~\mathrm{and}~ Y_{B-L}(0) = 0. \end{aligned}$$

(20)

at $T>> M_1$.

Different $\gamma$s in Eqs. (18) and (19) are space-time densities of the scattering processes at equilibrium depicted in Fig. 2. In the first Boltzmann equation (Eq. (18)), $\gamma _{D_{1}}$ denotes the contribution from the decay of $N_{1}$. $\gamma ^{1}_{\phi ,s}$ and $\gamma ^{1}_{\phi ,t}$ originate from the lepton number-violating ($\Delta L = 1$) s-channel and t-channel washout processes via Higgs-mediation. The factor of $^{\prime }{^{\prime }}2^{\prime }{^{\prime }}$ in front of $\gamma ^{1}_{\phi ,s}$ comes due to the Majorana nature of $N_{1}$. The factor $^{\prime }{^{\prime }}4^{\prime }{^{\prime }}$ in front of $\gamma ^{1}_{\phi ,t}$ accounts for the Majorana nature of $N_1$ as well as two t-channel washout scattering processes mediated by $N_{1}$ ($N_{1} t \rightarrow l q$ and $N_{1} \overline{q} \rightarrow l \overline{t}$) [58]. $\gamma ^{1}_{\phi ,s}$ and $\gamma ^{1}_{\phi ,t}$ also contribute in the second Boltzmann equation. $\gamma _{N,s}$ and $\gamma _{N,t}$ in Eq. (19) come from $\Delta L = 2$ lepton number-violating s-channel and t-channel scattering processes via $N_{1}$. The expressions of $\gamma _{D_{1}}$, $\gamma ^{1}_{\phi ,s}$, $\gamma ^{1}_{\phi ,t}$, $\gamma _{N,s}$ and $\gamma _{N,t}$ can be found in appendix B. However, in our model, since $\phi _3$ does not couple with the quarks from the requirement of zero FCNC (see the quantum number assignments in Table 1), no contributions will be drawn from $\gamma ^{1}_{\phi ,s}$ and $\gamma ^{1}_{\phi ,t}$ (Fig. 1c, d, e). Only surviving processes contributing to the washout will be s-channel and t-channel processes mediated by $N_1$ (Fig. 1b, f). In our model, due to the quantum number assignment, $\phi \equiv \phi _3$ in Fig. 2.

The number densities of particles with mass M and temperature T can be written as:

$$\begin{aligned} N_{eq} = \frac{g M^2 T}{2 \pi ^2} K_2(\frac{M_1}{T}). \end{aligned}$$

(21)

g being the number of degrees of freedom of corresponding particles, $K_2$ being second modified Bessel function of second kind.

Fig. 2

a Decay of lightest heavy RH-neutrino $N_{1}$ (contributes to $\gamma _{D_{1}}$), b $\Delta L = 2$, s-channel scattering via $N_{1}$ (contributes to $\gamma _{N,s}$), c $\Delta L = 1$, s-channel scattering via Higgs (contributes to $\gamma ^{1}_{\phi ,s}$), d, e $\Delta L = 1$, t-channel scattering via Higgs (contributes to $\gamma ^{1}_{\phi ,t}$), f $\Delta L = 2$, t-channel scattering via $N_{1}$ (contributes to $\gamma _{N,t}$)

Fig. 3

Diagrams contributing to the CP-asymmetry $\epsilon$: a tree-level decay of $N_1$, b vertex correction, c self-energy diagram

$\epsilon _{\alpha \alpha }$ being the CP-asymmetry generated through the out-of-equilibrium decay of $N_{1}$ (decaying to $\phi _3$ and $l_\alpha$, $\alpha$ being the flavor of the lepton)) [17], the total lepton asymmetry can be computed by summing over the flavor indices, i.e. $\epsilon = \sum _\alpha \epsilon _{\alpha \alpha }$. Here the CP-asymmetry for a single flavor $\alpha$ can be calculated as [17]:

$$\begin{aligned} \epsilon _{\alpha \alpha } := \frac{\Gamma (N_{1} \rightarrow \phi + l_{\alpha }) - \Gamma (N_{1} \rightarrow \overline{\phi } + \overline{l}_{\alpha })}{\Gamma (N_{1} \rightarrow \phi + l) + \Gamma (N_{1} \rightarrow \overline{\phi } + \overline{l})} \end{aligned}$$

(22)

Considering the interference between the amplitudes of tree-level decay of $N_1$, (Fig. 2a), one-loop vertex-correction (Fig. 2b) and self-energy diagram (Fig. 2c), $\epsilon _{\alpha \alpha }$ can be calculated as [17]:

$$\begin{aligned} \epsilon _{\alpha \alpha }= & {} \frac{1}{8 \pi }~ \frac{1}{(Y^\dag Y)_{11}} \sum _j \mathrm{Im} \{Y^*_{\alpha 1} ~(Y^\dag Y)_{1j} ~Y_{\alpha j}\}~ g(x_j) \nonumber \\&+ \frac{1}{8 \pi }~ \frac{1}{(Y^\dag Y)_{11}} \sum _j \mathrm{Im} \{Y^*_{\alpha 1} ~(Y^\dag Y)_{j1} ~Y_{\alpha j}\}~ \frac{1}{1-x_j} \,. \nonumber \\ \end{aligned}$$

(23)

Here $\epsilon _{\alpha \alpha }$ in Eq. (23) can be written in this form assuming not too degenerate heavy neutrino spectrum, i.e. $M_i - M_j>> \Gamma _{N_1}$, $\Gamma _{N_1}$ being total tree-level decay width of $N_1$.⁴ The first term in Eq. (23) comes from the interference between the diagrams in Fig. 3, which violate both lepton flavor and lepton number. The second term in Eq. (23) violates lepton flavor, but conserves lepton number and hence does not contribute to the total lepton asymmetry $\epsilon$.

Thus total lepton asymmetry is computed by summing over the flavor indices [17]:

$$\begin{aligned} \epsilon = \sum _\alpha \epsilon _{\alpha \alpha } = \frac{1}{8 \pi (Y^\dag Y)_{11}} \sum _j \mathrm{Im} \left\{ \left[ (Y^\dag Y)_{1j}\right] ^2\right\} g(x_j) \,.\nonumber \\ \end{aligned}$$

(24)

where $x_j = \frac{M_j^2}{M_1^2}$.

After taking the approximation $x>> 1$,

$$\begin{aligned} g(x) = \sqrt{x} \left[ \frac{1}{1-x} + 1 - (1+x)~ \mathrm{In} \left( \frac{1+x}{x}\right) \right] \rightarrow - \frac{3}{2 \sqrt{x}} \,. \nonumber \\ \end{aligned}$$

(25)

With $x_j = \frac{M_j^2}{M_1^2}$, $\epsilon$ becomes,

$$\begin{aligned} \epsilon = - \frac{3}{16 \pi (Y^\dag Y)_{11}} M_1 \left[ \frac{\mathrm{Im} \{(Y^\dag Y)_{12}^2\}}{M_2} + \frac{\mathrm{Im} \{(Y^\dag Y)_{13}^2\}}{M_3} \right] . \end{aligned}$$

(26)

Expressions for $(Y^\dag Y)_{11},~ \mathrm{Im} \{(Y^\dag Y)_{12}^2\},~ \mathrm{Im} \{(Y^\dag Y)_{13}^2\}$ are relegated to appendix C.

Fig. 4

This figure depicts parameter space in $M_{H_1^+}~ \mathrm{vs.} ~M_{H_1}$ plane (upper panel left), $M_{H_1^+} ~\mathrm{vs.}~M_{A_1}$ plane (upper panel right), $M_{H_2^+}~ \mathrm{vs.}~ M_{H_2}$ plane (lower panel left), $M_{H_2^+}~ \mathrm{vs.}~ M_{A_2}$ plane (lower panel right). The cyan, green and red regions represent the parameter space allowed by all constraints in the scalar sector (mentioned in Sect. 3) for $\tan \beta = 3,~4,~5$ respectively

The lepton asymmetry generated in the out-of-equilibrium decay of $N_1$, is converted into baryon asymmetry through $(B+L)$ violating sphelaron transitions [18, 75]. The conversion of lepton asymmetry to baryon asymmetry being terminated at the freeze-out temperature of the sphelaron process, $T_\mathrm{sph} \sim 150$ GeV [76], the resultant baryon number is computed at $T_\mathrm{sph}$ as [77]:

$$\begin{aligned} Y_{B} = \bigg ( \frac{8 N_{f} + 4 N_{H}}{22 N_{f} + 13 N_{H} } \bigg ) Y_{B-L} (z_\mathrm{sph}). \end{aligned}$$

(27)

where $N_{f}$ is the number of generations of the fermion families and $N_{H}$ is the number of the Higgs doublets and $Y_{B-L} (z_\mathrm{sph})$ is the solution of Boltzman equations at $z = z_\mathrm{sph} = \frac{M_1}{T_\mathrm{sph}}$. For our model $N_f = 3 , N_H = 3$.

6 Analysis and results

To analyse the multi-dimensional parameter space compatible with the aforementioned theoretical and experimental constraints, we have considered the model parameters $\tan \beta , \gamma , \gamma _1 , \gamma _2, \alpha _3, M_h, M_{H_1}, M_{H_2}, M_{A_1}, M_{A_2}, M_{H_1^{\pm }}, M_{H_2^{\pm }}$ as independent, and have varied them (except $M_h$ and $\alpha _3$) within the following window:

$$\begin{aligned}&2.5< \tan \beta< 10.0 , ~ - \pi< \gamma , \gamma _1 , \gamma _2< \pi , \nonumber \\&300~ \mathrm{GeV}< M_{H_1}, M_{H_2}, M_{A_1}, M_{A_2}, M_{H_1^{\pm }}, M_{H_2^{\pm }} < 500~ \mathrm{GeV} \,.\nonumber \\ \end{aligned}$$

(28)

The dependent parameters $\alpha _1, \alpha _2$ can be expressed in terms of the independent ones. Mixing angles in the CP-even sector $\alpha _1, \alpha _2$ are fixed by the relations $\alpha _1 = \gamma , \alpha _2 + \beta = \frac{\pi }{2}$ at the Alignment limit and we have fixed $M_h = 125$ GeV and $\alpha _3 = \frac{\pi }{4}$ throughout the analysis.

After doing an extensive scan over the parameter space, subject to all the aforementioned constraints listed in Sect. 3, larger values of $\tan \beta$, i.e. $\tan \beta > 5.5$, are ruled out particularly from the perturbativity constraints ($|\lambda _i| \le 4 \pi$) on the quartic couplings. Therefore we have presented the results for three discrete values of $\tan \beta$, i.e. $\tan \beta = 3,~4,~5$. Since doublet $\phi _3$ is responsible for generating masses of the light neutrinos due to $Z_3$-symmetry (Eq. (10)), $\tan \beta$ plays a crucial role and subsequently enters into the calculation of lepton asymmetry (Eq. (26)). Throughout the analysis, we have used those values of $\tan \beta$, which are filtered out by the constraints in the scalar sector mentioned in Sect. 3. At exact Alignment limit, h being the SM-like Higgs boson, masses of other non-standard heavier scalars range from 200-450 GeV depending on $\tan \beta$. Figure 4 depicts the parameter space spanned by one charged and one neutral scalar(s). Lower the value of $\tan \beta$, parameter space with higher masses of non-standard scalars becomes accessible. The masses of non-standard scalars are functions of the quartic couplings $\lambda _i$s ($i=1,2,...12$) and the mixing angles $\beta , \gamma , \gamma _1, \gamma _2, \alpha _1, \alpha _2, \alpha _3$ etc. Other mixing angles other than “$\beta$” also play a role in constructing the physical masses. Thus a simple dependence of the physical masses on $\tan \beta$ alone cannot be derived by neglecting the effect of varying other parameters. Here one can at most comment that the parameter space with lower values of $\tan \beta$ and higher values of physical masses comply with all the constraints mentioned in section III. It is clearly evident that with rise of $\tan \beta$, the available parameter space consistent with all the constraints, shrinks from the cyan colored region with $\tan \beta = 3$ to red colored region with $\tan \beta = 5$ in Fig. 4. Constraints coming from T-parameter restrict the mass-splittings between the heavy neutral and charged scalars within 50 GeV. These mass splittings result in the sharp edges in the plots of Fig. 4.

After putting an upper bound on $\tan \beta$ from the constraints in the scalar sector, let us now move on to explore the status of the parameter space in the neutrino sector. As discussed earlier, using CI parametrization the real and imaginary parts of Yukawa couplings are solved in terms of $v_3$, the elements in the PMNS matrix, complex angles $\theta , \phi , \psi$ and $M_1, M_2, M_3$. The elements in the $U_\mathrm{PMNS}$ matrix, constrained by neutrino oscillation data, are fixed at their central values [69],

$$\begin{aligned}&\sin ^2 \theta _{12} = 0.31 , ~ \sin ^2 \theta _{23} = 0.58 , ~ \sin ^2 \theta _{13} = 0.02241, \nonumber \\&\Delta m_{21}^2 = 7.39 \times 10^{-5} \mathrm{eV}^2 , ~ \Delta m_{31}^2 = 2.525 \times 10^{-3} \mathrm{eV}^2 , \nonumber \\&\delta _\mathrm{CP} = 215^\circ \,. \end{aligned}$$

(29)

Assuming the phases associated with the three complex angles $\theta , \phi , \psi$ in the matrix $\mathcal {O}$ (Eq. (16)) to be zero, they are varied within the region : $- \pi< \phi< \pi , -\pi< \psi < \pi$ and we fix $\theta$ at $\theta = \frac{\pi }{4}$ for simplicity. The variation of these angles will in turn incorporate variations in the real ($y_{jR}$) and imaginary parts ($y_{jI}$) of Yukawa couplings in $M_D$, which are absolutely compatible with the neutrino oscillation data [69].

Fig. 5

Parameter space in $Y_B-|\epsilon |$ plane with varying $M_1, M_2, M_3$. Blue, green and red points correspond to the regions with $\tan \beta = 3,~4,~5$. The Orange band signifies $2 \sigma$-deviation from the central value of $Y_B$ mentioned in introduction

Fig. 6

Region allowed by neutrino oscillation data in $M_1~ \mathrm{vs.} ~ |\epsilon |$ plane. Green region corresponds to the points with no constraint on $|\epsilon |$, whereas red points satisfy : $1 \times 10^{-6}< |\epsilon | < 1.5 \times 10^{-6}$. Figures are drawn for $\tan \beta = 3$ (upper panel left), $\tan \beta = 4$ (upper panel right), $\tan \beta = 5$ (lower panel)

Fig. 7

Parameter space in real vs. imaginary part of Yukawa coupling $y_1$. Plots are given for $\tan \beta = 3$ (upper panel left), $\tan \beta = 4$ (lower panel left), $\tan \beta = 5$ (lower panel right). Light green, deep green and red region correspond to the parameter space allowed by only neutrino oscillation data, neutrino oscillation data + lepton asymmetry, neutrino oscillation data + lepton asymmetry + baryon asymmetry respectively. The right plot in the upper panel is the zoomed version of the figure at the left for $\tan \beta = 3$

Fig. 8

Parameter space in real vs. imaginary part of Yukawa coupling $y_4$. Plots are given for $\tan \beta = 3$ (upper panel left), $\tan \beta = 4$ (upper panel right), $\tan \beta = 5$ (lower panel). Light green, deep green and red region correspond to the parameter space allowed by only neutrino oscillation data, neutrino oscillation data + lepton asymmetry, neutrino oscillation data + lepton asymmetry + baryon asymmetry respectively

Fig. 9

Parameter space in real vs. imaginary part of Yukawa coupling $y_5$. Plots are given for $\tan \beta = 3$ (upper panel left), $\tan \beta = 4$ (upper panel right), $\tan \beta = 5$ (lower panel). Light green, deep green and red region correspond to the parameter space allowed by only neutrino oscillation data, neutrino oscillation data + lepton asymmetry, neutrino oscillation data + lepton asymmetry + baryon asymmetry respectively

Fig. 10

Parameter space in real vs. imaginary part of Yukawa coupling $y_8$. Plots are given for $\tan \beta = 3$ (upper panel left), $\tan \beta = 4$ (upper panel right), $\tan \beta = 5$ (lower panel). Light green, deep green and red region correspond to the parameter space allowed by only neutrino oscillation data, neutrino oscillation data + lepton asymmetry, neutrino oscillation data + lepton asymmetry + baryon asymmetry respectively

In thermal leptogenesis there exists a lower bound of $10^9$ GeV on $M_1$, i.e. $M_1 \ge 10^9$ GeV, which is known as Davidson-Ibarra bound [17, 78]. In addition, we consider the heavy neutrino masses to be hierarchical. In Fig. 5, the red, green and blue points in “$Y_B-|\epsilon |$” plane represent the points filtered out by neutrino oscillation data for $\tan \beta = 5, ~4, ~3$ respectively (with $10^9 ~\mathrm{GeV}< M_1< 10^{11} ~\mathrm{GeV}, 10^{13} ~\mathrm{GeV}< M_2, M_3 < 10^{16}~ \mathrm{GeV}$). The narrow orange band represents the region of the parameter space consistent with the current baryon asymmetry data. The red, green and blue points lying within the orange band thus comply with the neutrino oscillation data and observed baryon asymmetry of the Universe. Therefore the red, blue and green points above the orange band are ruled out by the current baryon asymmetry data. Whereas for points lying below the orange band, leptogenesis fails to produce adequate matter anti-matter asymmetry. Due to the interplay of the model parameters like real and imaginary parts of Yukawa couplings, $\tan \beta , ~ M_1, ~ M_2, ~ M_3$, the allowed parameter space gets larger for higher values of $\tan \beta$. Figure 5 shows that with increase of $\tan \beta$, larger values of $|\epsilon |$ are attainable. The common parameter space in $Y_B$ vs. $|\epsilon |$ plane in Fig. 5 (crowded by the blue, green and red points lying within the orange band), indicates that the production of sufficient baryon asymmetry requires $|\epsilon | \sim 10^{-6}$ for $\tan \beta = 3,4,5$.⁵ While its order of magnitude remains the same for all three values of tan$\beta$ (= 3, 4, 5) taken, the most restrictive bound $1 \times 10^{-6}< |\epsilon | < 1.5 \times 10^{-6}$ is obtained for tan$\beta$ = 3 (blue points in Fig. 5). In other words, the points lying inside the orange band and obeying $1 \times 10^{-6}< |\epsilon | < 1.5 \times 10^{-6}$ lead to the requisite $Y_B$ irrespective of tan$\beta$.

In Fig. 6, the variation of $M_1$ with respect to $|\epsilon |$ have been shown for three different values of $\tan \beta$. For the green points, only neutrino oscillation data has been satisfied. The red band on top of the green region signifies the reduced parameter space in $M_1~ \mathrm{vs. } ~ |\epsilon |$ plane after applying the aforementioned bound on $|\epsilon |$. Here it is needed to be clarified that all the red points do not lead to 100 $\%$ of the observed baryon asymmetry. Some points in the red region, depending on the other parameters in the Boltzman equation, indeed lead to 100$\%$ of the baryon asymmetry. The green region at the left of the red band corresponds to the under production of baryon asymmetry, since from Fig. 5, for $|\epsilon | < 1 \times 10^{-6}$, there is hardly any point leading to exact (within the orange band) or excess (points above the orange band) baryon asymmetry. The green region at the right of the red band in Fig. 6, partially corresponds to the under production of the same following Fig. 5. Apart from this, another major fraction of the green region at the right of the red band refers to the overproduction of baryon asymmetry. A very small fraction of the green points at the right of the red band, corresponds to 100$\%$ baryon asymmetry. For higher $\tan \beta$, most of the green region at the right contains points with overproduced baryon asymmetry (as can be seen from Fig. 5 also). Increase in $\tan \beta$ makes lower values of $M_1$ allowed, which are compatible with both neutrino oscillation data and the bound from lepton asymmetry. It can be inferred that this constraint on $|\epsilon |$ disfavors the portion of the parameter space with $M_1 < 4 \times 10^{10}$ GeV, $2.5 \times 10^{10}$ GeV, $1.5 \times 10^{10}$ GeV for $\tan \beta = 3,~4,~5$ respectively in our model. Thus the lower bound of $10^9$ GeV on $M_1$ for thermal leptogenesis is uplifted after being filtered out by all constraints.

From Eq. (26) it can be seen that for fixed values of Yukawa couplings and $M_1$, $|\epsilon |$ decreases with increasing $M_2$ and $M_3$. From Fig. 5 it can be concluded that the parameter space with $|\epsilon |$ smaller than $\sim 10^{-6}$ (and hence too large $M_2, M_3$ for fixed values of Yukawa couplings and $M_1$) cannot produce sufficient baryon asymmetry. Thus we choose to explore the parameter region of Yukawa couplings corresponding to $M_1 = 10^{11}$ GeV, and varied $M_2, M_3$ from $10^{13}$ GeV to $10^{16}$ GeV for all values of $\tan \beta$, so that adequate baryon asymmetry can be produced. Real and imaginary parts of the Yukawa couplings $y_j$, already being compatible with neutrino oscillation data, get additional constraints coming from the lepton asymmetry $|\epsilon |$ (Eq. (26)). All points satisfying the neutrino oscillation data and the constraints coming from lepton asymmetry, are further validated by the baryon asymmetry constraint (Eq. (1)). At this point, a clear distinction between the two constraints coming from the lepton asymmetry and observed baryon asymmetry is required. The parameter space compatible with the observed baryon asymmetry (orange band) in Fig. 4 is a subset of the parameter space satisfied by the constraint on lepton asymmetry (includes points residing within, above and below the orange band). After satisfying the aforementioned bound on $|\epsilon |$, the baryon asymmetry is under produced and over produced for the green, blue and red points lying below and above the orange band respectively. Thus all points satisfying the lepton asymmetry bound may not comply with the observed baryon asymmetry data. This is consistent with Eq. (19), since the solution of the second Boltzman equation depends not only on $\epsilon$, but also on some other parameters like $\gamma _{N,s}, \gamma _{N,t}$ etc.

From Figs. 7, 8, 9, 10, it can be inferred that the available parameter spaces spanned by the real and imaginary parts of the Yukawa couplings ($y_{j}$), at $\tan \beta =3,~4,~5$, shrink gradually after applying the following constraints sequentially : (i) neutrino oscillation data (light green region), (ii) lepton asymmetry (deep green region), (iii) baryon asymmetry (red region). Thus the bound on the Yukawa couplings coming from baryon asymmetry comes out to be the most stringent among all. From Eqs. (26) and (C1) it can be computed that for a given set of positive Yukawa couplings, $\epsilon$ is the same as computed with the similar set of Yukawa couplings which are equal in magnitude but opposite in sign. Therefore in Figs. 7, 8, 9, 10, both the dark green and red regions are separated and show a symmetric pattern with respect to the origin (0, 0). Here we want to clarify that from Figs. 7, 8, 9, 10, it seems that the (0,0) points (where the real and imaginary parts of the Yukawa couplings are zero) are allowed by the constraints, which is misleading. For clarification, we would refer to Fig. 7b, which shows the zoomed version of Fig. 7a, where it is clearly seen that the (0,0) point is disallowed by the constraints. This conclusion is true for the other Yukawa couplings too.

In Figs. 7, 8, 9, 10, the range of the real and imaginary part of Yukawa couplings filtered out by the neutrino oscillation data (represented by light green region) and constraint on lepton asymmetry (represented by dark green region), go on increasing with an increase in $\tan \beta$. Thus one can observe that after surviving through first two constraints, larger values of couplings become accessible for higher values of $\tan \beta$. For example, in the light green region in Fig. 7, $y_{1R}$ ranges from $\sim -0.0035$ to $\sim 0.0035$ for $\tan \beta = 3$. Whereas, for $\tan \beta = 4$ and 5, the corresponding range extends to: $-0.0045< y_{1R} < 0.0045$ and $-0.0058< y_{1R} < 0.0058$ respectively. Likewise the imaginary part of $y_1$ behaves in a similar manner with increasing $\tan \beta$. The red regions do not follow the same pattern, i.e. the red regions cover smaller Yukawa couplings as $\tan \beta$ grows in Figs. 7, 8, 9, 10. This observation holds for real and imaginary parts of the other couplings $y_{4}, y_{5}, y_{8}$ too (Figs. 8, 9, 10). The shape of the distribution in Figs. 7, 8 (elliptical) are different from that in Figs. 9, 10 (quadrilateral). Sharp upper and lower edges in the $y_{5I} ~\mathrm{vs. }~ y_{5R}$ and $y_{8I} ~\mathrm{vs. }~ y_{8R}$ plane (Figs. 9, 10), parallel to $y_{5R}$ and $y_{8R}$ axes respectively, correspond to the perturbativity limits imposed on the Yukawa couplings, i.e. $|y_j| \le \sqrt{4 \pi }$. Due to the perturbative requirement, upper and lower portions of the plots for $y_5$ and $y_8$ have been chopped off. Since the real and imaginary parts of $y_1$ and $y_4$ lie well within the perturbative limit already, the shape of the parameter space has no sharp edge in Figs. 7, 8. Here we have presented the plots for the Yukawa couplings $y_1, y_4, y_5, y_8$ only, because the shape of the plots for the rest of the Yukawa couplings $y_2, y_3, y_6, y_7, y_9$ resembles with that of the presented ones.

7 Summary and conclusion

In this analysis, we have explored the possibility of neutrino mass generation via Type-I see-saw mechanism and baryogenesis via thermal leptogenesis in the context of $Z_3$-symmetric 3HDM accompanied by three RH singlet neutrinos. According to the criteria of thermal leptogenesis, we consider hierarchical masses between three heavy neutrinos: $M_1<< M_2,~M_3$; $M_1$ having the lower limit of $10^9$ GeV. The thermal production and out-of-equilibrium decay of the lightest heavy neutrino $N_1$ gives rise to lepton asymmetry, which in turn is partially converted to baryon asymmetry via EW sphelaron processes.

An important model parameter $\tan \beta$, relevant for lepton asymmetry calculation, has been filtered out by different constraints in the scalar sector. Among all of the constraints, the requirement of perturbativity of all quartic couplings, rules out the region of the parameter space with $\tan \beta > 5.5$. Thus we proceed with three discrete values of $\tan \beta$, i.e. $\tan \beta = 3, ~4, ~5$, to make a comparative study of the parameter space in the neutrino sector.

Among the three doublets, $\phi _3$ only being responsible for the neutrino mass generation due to $Z_3$-quantum number assignment, the Yukawa Lagrangian contains nine complex Yukawa couplings, i.e. 18 free parameters (real and imaginary parts of nine complex Yukawa couplings) to fit neutrino oscillation data, which is further simplified by CI parametrization. Three RH singlet neutrinos couple to the SM neutrinos via doublet $\phi _3$ only, and generate mass of light neutrinos via Type-I see-saw mechanism. From the decay of $N_1$, both lepton asymmetry $\epsilon$ and baryon asymmetry $Y_B$ are calculated at the points satisfying neutrino oscillation data with varying $M_1, M_2, M_3$. It is found that the available parameter space in $Y_B -\epsilon$ plane shrinks with decreasing $\tan \beta$. To be consistent with the current bound on baryon asymmetry, one has to take $\epsilon \sim 10^{-6}$. Thus for the rest of the study, we have imposed a conservative limit of $1 \times 10^{-6}< |\epsilon | < 1.5 \times 10^{-6}$ on $\epsilon$ for all $\tan \beta$. This constraint immediately uplifts the lower bound on $M_1$ from $10^9$ GeV to $\sim 4 \times 10^{10}$ GeV, $\sim 2.5 \times 10^{10}$ GeV, $\sim 1.5 \times 10^{10}$ GeV for $\tan \beta = 3, ~4,~ 5$ respectively in our model. The available parameter space for a fixed $\tan \beta$ in the real vs. imaginary part of complex Yukawa coupling plane is diminished after applying three constraints sequentially: neutrino oscillation data, lepton asymmetry, baryon asymmetry. The last constraint turns out to be the most stringent among all.

Appendices

Appendix A: Decay width of $h \rightarrow \gamma \gamma$ in 3HDM

Amplitude and decay width of the process $h \rightarrow \gamma \gamma$ can be written as [79]:

$$\begin{aligned} \mathcal {M}^{\text {3HDM}}_{h \rightarrow \gamma \gamma }= & {} \sum _f N_f Q_f^2 f_{hff} A_{1/2}\Big (\frac{M^2_h}{4 M^2_f}\Big ) + f_{h VV} A_1\Big (\frac{M^2_h}{4 M^2_W}\Big )\nonumber \\&+ \sum _{i=1}^2 \frac{\lambda _{h H_i^+ H_i^-} v}{2 M^2_{H_i^+}} A_0\Big (\frac{M^2_h}{4 M^2_{H_i^+}}\Big ) \end{aligned}$$

(A1a)

$$\begin{aligned} \Gamma ^{\text {3HDM}}_{h \rightarrow \gamma \gamma }= & {} \frac{G_F \alpha ^2 M_h^3}{128 \sqrt{2} \pi ^3} |\mathcal {M}^{\text {3HDM}}_{h \rightarrow \gamma \gamma }|^2, \end{aligned}$$

(A1b)

where $N_f, Q_f, G_F$ and $\alpha$ denote respectively color factor, charge of fermion, the Fermi constant and the QED fine-structure constant. For quarks $N_f = 3$. $\lambda _{h H_i^+ H_i^-}$ is $h H_i^+ H_i^-$ ($i = 1,2$) coupling. $f_{hff}, f_{hVV}$ are scale factors of hff, hVV couplings with respect to SM. When the alignment limit is strictly enforced,

$$\begin{aligned} f_{hff} = f_{hVV} = 1 \end{aligned}$$

(A2)

The loop functions are listed below.

$$\begin{aligned} A_{1/2}(x)= & {} \frac{2}{x^2}\big ((x + (x -1)f(x)\big ), \end{aligned}$$

(A3a)

$$\begin{aligned} A_1(x)= & {} -\frac{1}{x^2}\big ((2 x^2 + 3 x + 3(2 x -1)f(x)\big ), \end{aligned}$$

(A3b)

$$\begin{aligned} A_0(x)= & {} -\frac{1}{x^2}\big (x - f(x)\big ), \end{aligned}$$

(A3c)

$$\begin{aligned} \text {with} ~~f(x)= & {} \text {arcsin}^2(\sqrt{x}); ~~~x \le 1 \nonumber \\&= -\frac{1}{4}\Bigg [\text {log}\frac{1+\sqrt{1 - x^{-1}}}{1-\sqrt{1 - x^{-1}}} -i\pi \Bigg ]^2; ~~~x > 1.\nonumber \\ \end{aligned}$$

(A3d)

where $A_{1/2}(x), A_1(x)$ and $A_0(x)$ are the respective amplitudes for the spin-$\frac{1}{2}$, spin-1 and spin-0 particles in the loop.

$\lambda _{h H_1^+ H_1^-}, ~\lambda _{h H_2^+ H_2^-}$ can be expressed in terms of quartic couplings and mixing angles as,

$$\begin{aligned}&\lambda _{h H_1^+ H_1^-} = -\cos \beta _1 (\cos \alpha _3 (v \sin \beta _1 \cos \beta _2 \sin \gamma _2\nonumber \\&\quad \cos \gamma _2 (\lambda _9 \sin \beta _2+\lambda _{11}) \nonumber \\&\quad +\lambda _3 v \sin ^2 \beta _2 \sin ^2 \gamma _2 \nonumber \\&\quad + \lambda _6 v \sin ^2 \beta _2 \cos ^2 \gamma _2) \nonumber \\&\quad +\cos \beta _2 (-\lambda _{12} v \sin \gamma _2 \cos \gamma _2 \nonumber \\&\quad (\sin \alpha _3 \sin \beta _1 + \sin \beta _2) \nonumber \\&\quad +\lambda _2 v \sin \alpha _3 \sin \beta _1 \sin \beta _2 \cos ^2 \gamma _2 \nonumber \\&\quad + \lambda _6 v \sin \alpha _3 \sin \beta _1 \sin \beta _2 \sin ^2 \gamma _2) \nonumber \\&\quad + \lambda _9 v \sin \alpha _3 \sin ^2 \beta _2 \sin \gamma _2 \cos \gamma _2\nonumber \\&\quad - \lambda _{11} v \sin \beta _1 \cos ^2 \beta _2 \sin \gamma _2 \cos \gamma _2) \nonumber \\&\quad +\cos ^2 \beta _1 (v \cos ^2 \beta _2 (\lambda _4 \cos ^2 \gamma _2 \nonumber \\&\quad + \lambda _5 ~\sin ^2 \gamma _2) \nonumber \\&\quad -\cos \beta _2 \sin \gamma _2 \cos \gamma _2 \nonumber \\&\quad (\lambda _{11} v \sin \alpha _3 \sin \beta _2+\lambda _{12} v \cos \alpha _3 \sin \beta _2)) \nonumber \\&\quad +\sin \beta _1 (\sin \alpha _3 (\lambda _9 v \sin \beta _1 \cos \nonumber \\&\quad \beta _2 \sin \gamma _2 \cos \gamma _2 +\lambda _3 v \sin \beta _2 \sin ^2 \gamma _2 \nonumber \\&\quad + \lambda _6 v \sin \beta _2 \cos ^2 \gamma _2)\nonumber \\&\quad -\cos \alpha _3 (\cos \beta _2 (\lambda _2 v \sin \beta _1 \cos ^2 \gamma _2 \nonumber \\&\quad + \lambda _6~ v \sin \beta _1 \sin ^2 \gamma _2)\nonumber \\&\quad +\lambda _9 v \sin \beta _2 \sin \gamma _2 \cos \gamma _2)) \end{aligned}$$

(A4)

$$\begin{aligned}&\lambda _{h H_2^+ H_2^-} = -\cos \beta _1 (\sin \gamma _2 \cos \gamma _2\nonumber \\&\quad (\cos \beta _2 (\lambda _{12} v \sin \alpha _3 \sin \beta _1+\lambda _{12} v \sin \beta _2) \nonumber \\&\quad -\lambda _9 v \sin \alpha _3 \sin ^2\beta _2\nonumber \\&\quad +\lambda _{11} v \sin \beta _1 \cos ^2\beta _2) \nonumber \\&+\cos \alpha _3 (-\cos \beta _2 \sin \gamma _2 \cos \gamma _2\nonumber \\&\quad (\lambda _9 v \sin \beta _1 \sin \beta _2 \nonumber \\&\quad +\lambda _{11} v \sin \beta _1)+\lambda _3 v \sin ^2\beta _2 \cos ^2\gamma _2\nonumber \\&\quad +\lambda _6 v \sin ^2\beta _2 \sin ^2\gamma _2) \nonumber \\&\quad +\lambda _2 v \sin \alpha _3 \sin \beta _1 \sin \beta _2 \cos \beta _2 \sin ^2\gamma _2 \nonumber \\&\quad +\lambda _6 v \sin \alpha _3 \sin \beta _1 \nonumber \\&\quad \sin \beta _2 \cos \beta _2 \cos ^2 \gamma _2)\nonumber \\&\quad +\frac{1}{4} \cos ^2\beta _1 (2 \cos \beta _2 \sin 2\gamma _2 \nonumber \\&\quad (\lambda _{11} v \sin \alpha _3 \sin \beta _2 \nonumber \\&+\lambda _{12} v \cos \alpha _3 \sin \beta _2)+2 v \cos ^2\beta _2 \nonumber \\&\quad (\cos 2\gamma _2 (\lambda _5-\lambda _4)+\lambda _4+\lambda _5))\nonumber \\&+\sin \beta _1 (\sin ^2\gamma _2 (\lambda _6 v \sin \alpha _3 \sin \beta _2\nonumber \\&\quad -\lambda _2 v \cos \alpha _3 \sin \beta _1 \cos \beta _2) \nonumber \\&\quad +\cos ^2\gamma _2 (\lambda _3 v \sin \alpha _3 \sin \beta _2\nonumber \\&\quad -\lambda _6 v \cos \alpha _3 \sin \beta _1 \cos \beta _2) \nonumber \\&\quad +\sin \gamma _2 \cos \gamma _2 (\lambda _9 v \cos \alpha _3 \sin \beta _2\nonumber \\&\quad -\lambda _9 v \sin \alpha _3 \sin \beta _1 \cos \beta _2)) \end{aligned}$$

(A5)

Appendix B: Formulas for reduced cross sections

Expression for $\gamma _{D_1}$ can be written as [74]:

$$\begin{aligned} \gamma _{D_1} = \gamma _{eq} = N_{N_1}^{eq}~ \frac{K_{1}(z)}{K_{2}(z)}~ \Gamma _{N_1} \,, \mathrm{with}~ z = \frac{M_1}{T} \end{aligned}$$

$N_{N_1}^{eq}$ being the equilibrium number density of the lightest RH neutrino $N_1$. Here $K_1$ and $K_2$ are the first and second modified Bessel functions of second kind respectively and $\Gamma _{N_1}$ is the total decay width of $N_1$.

For decay of $N_1$, $\gamma _{eq}$ can be written as [74],

$$\begin{aligned} \gamma _{eq} = \frac{T}{64 \pi ^{4}} \int _{M^{2}_{1}}^{\infty } ~~ ds \hat{\sigma }(s) ~~ \sqrt{s} ~~ K_{1}(\frac{\sqrt{s}}{T}) \,. \end{aligned}$$

(B1)

where s⁶ is the square of center of mass energy and $\hat{\sigma }(s)$ is reduced cross section, which can be expressed in terms of actual cross section for two body scattering $a + b \rightarrow i + j + ...$ as [74] :

$$\begin{aligned} \hat{\sigma }(s) = \frac{8}{s}\left[ (p_a.p_b)^2 - M_a^2 M_b^2\right] \sigma (s) \,, \end{aligned}$$

(B2)

with $p_k$ and $M_k$ being three momentum and mass of particle k.

Decay width of $N_1$ at tree level,

$$\begin{aligned} \Gamma _{N_1}:= & {} \Gamma (N_1 \rightarrow \phi _3^{\dagger } + l) + \Gamma (N_1 \rightarrow \phi _3 + \overline{l}) \nonumber \\&= \frac{\alpha }{\text {sin}^{2}\theta _W} \frac{M_{1}}{4} \frac{(M^{\dagger }_{D} M_{D})_{11}}{M^{2}_{W}} \end{aligned}$$

(B3)

with $\alpha , \theta _W$ being the Fine structure constant and the Weinberg angle.

The reduced cross-section of $N_{1}$ decay is given by [74],

$$\begin{aligned} \hat{\sigma }_{N_{1}}(s)= & {} \frac{\alpha ^{2}}{\text {sin}^{4}\theta _W} \frac{2 \pi }{M^{4}_{W}}\frac{1}{x} a_{1}(M^{\dagger }_{D} M_{D})_{11}^2 \Big [\frac{x}{a_{1}} + \frac{2 x}{D_{1}(x)} + \frac{x^{2}}{2 D^{2}_{1}(x)} \nonumber \\&- \Big (1 + 2\frac{x + a_{1}}{D_{1}(x)}\Big ) ~\text {ln}\Big (\frac{x + a_{1}}{a_{1}}\Big ) \Big ], \end{aligned}$$

(B4)

where, $x = \frac{s}{M_1^2}, ~ a_1 =1, ~ \frac{1}{D_{1}(x)} := \frac{x - a_{1}}{(x-a_{1})^{2} + a_{1} c_{1}}$, with $c_{1} := \Big (\frac{\Gamma _{N_{1}}}{M_{1}}\Big )^{2}$

The reduced cross-section for L-violating t-channel process (via $N_{1}$) is [74],

$$\begin{aligned}&\hat{\sigma }_{N_{1},t} (s) = \frac{2 \pi \alpha ^2 a_1}{M^{4}_{W} \text {sin}^{4}\theta } (M^{\dagger }_{D} M_{D})^{2}_{11} \nonumber \\&\quad \times \Big [\frac{1}{2 a_1}~\frac{x}{x+a_{1}} + \frac{1}{x + 2 a_{1}} \text {ln}\Big (\frac{x + a_{1}}{a_{1}}\Big )\Big ]. \end{aligned}$$

(B5)

The reduced cross-section for s-channel process $N_1 + l \rightarrow \overline{t} + q$ (mediated by $\phi _3$) is⁷ [74],

$$\begin{aligned} \hat{\sigma }^{1}_{\phi ,s}(s) = \frac{3 \pi \alpha ^{2} M^{2}_{t}}{M^{4}_{W} \text {sin}^{4}\theta _W} (M^{\dagger }_{D} M_{D})_{11} \Big (\frac{x - a_{1}}{x}\Big )^{2}. \end{aligned}$$

(B6)

The reduced cross-section for t-channel process $N_1 + t \rightarrow \overline{l} + q$ (mediated by $\phi _3$) is [74],

$$\begin{aligned}&\hat{\sigma }^{1}_{\phi ,t}(s) = \frac{3 \pi \alpha ^{2} M^{2}_{t}}{M^{4}_{W} \text {sin}^{4}\theta _W} (M^{\dagger }_{D} M_{D})_{11}\nonumber \\&\quad \times \Big [\frac{x - a_{1}}{x} + \frac{a_{1}}{x}~ \text {ln} \Big (\frac{x - a_{1} + y~'}{y~'}\Big )\Big ], \end{aligned}$$

(B7)

where $y~' = \frac{M^{2}_h}{M^{2}_{1}}$.

Appendix C: Expressions of $(Y^\dag Y)_{11},~ \mathrm{Im} \{(Y^\dag Y)_{12}^2\}, \mathrm{Im} \{(Y^\dag Y)_{13}^2\}$

Expressions for $(Y^\dag Y)_{11},~ \mathrm{Im} \{(Y^\dag Y)_{12}^2\},~ \mathrm{Im} \{(Y^\dag Y)_{13}^2\}$ in Eq. (26) can be written in terms of real and imaginary parts of Yukawa couplings as :

$$\begin{aligned}&(Y^\dag Y)_{11} = y_{1R}^2 + y_{1I}^2 + y_{4R}^2 + y_{4I}^2 + y_{7R}^2 + y_{7I}^2 \,. \nonumber \\&\mathrm{Im} \{(Y^\dag Y)_{12}^2\} = y_{1R}^2 y_{2R} y_{2I}-2 y_{1R} y_{1I} y_{2R}^2 \nonumber \\&\quad +2 y_{1R} y_{1I} y_{2I}^2+2 y_{1R} y_{2R} y_{4R} y_{5I}\nonumber \\&\quad -2 y_{1R} y_{2R} y_{4I} y_{5R} \nonumber \\&\quad +2 y_{1R} y_{2R} y_{7R} y_{8I}-2 y_{1R} y_{2R} y_{7I} y_{8R}\nonumber \\&\quad +2 y_{1R} y_{2I} y_{4R} y_{5R}+2 y_{1R} y_{2I} y_{4I} y_{5I} \nonumber \\&\quad +2 y_{1R} y_{2I} y_{7R} y_{8R}+2 y_{1R} y_{2I} y_{7I} y_{8I}\nonumber \\&\quad -2 y_{1I}^2 y_{2R} y_{2I}-2 y_{1I} y_{2R} y_{4R} y_{5R} \nonumber \\&\quad -2 y_{1I} y_{2R} y_{4I} y_{5I}-2 y_{1I} y_{2R} y_{7R} y_{8R}\nonumber \\&\quad -2 y_{1I} y_{2R} y_{7I} y_{8I} +2 y_{1I} y_{2I} y_{4R} y_{5I} \nonumber \\&\quad -2 y_{1I} y_{2I} y_{4I} y_{5R}+2 y_{1I} y_{2I} y_{7R} y_{8I}\nonumber \\&\quad -2 y_{1I} y_{2I} y_{7I} y_{8R}+2 y_{4R}^2 y_{5R} y_{5I} \nonumber \\&\quad -2 y_{4R} y_{4I} y_{5R}^2+2 y_{4R} y_{4I} y_{5I}^2\nonumber \\&\quad +2 y_{4R} y_{5R} y_{7R} y_{8I} -2 y_{4R} y_{5R} y_{7I} y_{8R}+2 y_{4R} y_{5I} y_{7R} y_{8R} \nonumber \\&\quad +2 y_{4R} y_{5I} y_{7I} y_{8I}-2 y_{4I}^2 y_{5R} y_{5I}-2 y_{4I} y_{5R} y_{7R} y_{8R}\nonumber \\&\quad -2 y_{4I} y_{5R} y_{7I} y_{8I}+2 y_{4I} y_{5I} y_{7R} y_{8I} \nonumber \\&\quad -2 y_{4I} y_{5I} y_{7I} y_{8R}+2 y_{7R}^2 y_{8R} y_{8I}-2 y_{7R} y_{7I} y_{8R}^2\nonumber \\&\quad +2 y_{7R} y_{7I} y_{8I}^2-2 y_{7I}^2 y_{8R} y_{8I} \,. \nonumber \\&\mathrm{Im} \{(Y^\dag Y)_{13}^2\} = 2 y_{1R}^2 y_{3R} y_{3I}-2 y_{1R} y_{1I} y_{3R}^2\nonumber \\&\quad +2 y_{1R} y_{1I} y_{3I}^2+2 y_{1R} y_{3R} y_{4R} y_{6I} -2 y_{1R} y_{3R} y_{4I} y_{6R} \nonumber \\&\quad +2 y_{1R} y_{3R} y_{7R} y_{9I}-2 y_{1R} y_{3R} y_{7I} y_{9R}\nonumber \\&\quad +2 y_{1R} y_{3I} y_{4R} y_{6R}+2 y_{1R} y_{3I} y_{4I} y_{6I} \nonumber \\&+2 y_{1R} y_{3I} y_{7R} y_{9R}+2 y_{1R} y_{3I} y_{7I} y_{9I}\nonumber \\&\quad -2 y_{1I}^2 y_{3R} y_{3I}-2 y_{1I} y_{3R} y_{4R} y_{6R} \nonumber \\&\quad -2 y_{1I} y_{3R} y_{4I} y_{6I}-2 y_{1I} y_{3R} y_{7R} y_{9R}-2 y_{1I} y_{3R} y_{7I} y_{9I}\nonumber \\&\quad +2 y_{1I} y_{3I} y_{4R} y_{6I} \nonumber \\&\quad -2 y_{1I} y_{3I} y_{4I} y_{6R}+2 y_{1I} y_{3I} y_{7R} y_{9I} -2 y_{1I} y_{3I} y_{7I} y_{9R}\nonumber \\&\quad +2 y_{4R}^2 y_{6R} y_{6I}-2 y_{4R} y_{4I} y_{6R}^2 \nonumber \\&\quad +2 y_{4R} y_{4I} y_{6I}^2+2 y_{4R} y_{6R} y_{7R} y_{9I}\nonumber \\&\quad -2 y_{4R} y_{6R} y_{7I} y_{9R} +2 y_{4R} y_{6I} y_{7R} y_{9R} \nonumber \\&\quad +2 y_{4R} y_{6I} y_{7I} y_{9I}-2 y_{4I}^2 y_{6R} y_{6I}\nonumber \\&\quad -2 y_{4I} y_{6R} y_{7R} y_{9R} -2 y_{4I} y_{6R} y_{7I} y_{9I} +2 y_{4I} y_{6I} y_{7R} y_{9I} \nonumber \\&\quad -2 y_{4I} y_{6I} y_{7I} y_{9R}+2 y_{7R}^2 y_{9R} y_{9I}\nonumber \\&\quad -2 y_{7R} y_{7I} y_{9R}^2+2 y_{7R} y_{7I} y_{9I}^2-2 y_{7I}^2 y_{9R} y_{9I}. \end{aligned}$$

(C1)

Appendix D: Computation of bosonic and fermionic degrees of freedom (D.O.F)

For fermionic sector D.O.F can be computed as [80],

$$\begin{aligned} g_{\text {fermion}}= & {} g_{\text {quark}} + g_{\text {lepton}} + g_{\text {neutrino}} + g_{\text {RH-neutrino}} \nonumber \\&= (6 \times 3\times 2 \times 2) \nonumber \\&+ (3 \times 2 \times 2) + (3 \times 2 + 3 \times 2) = 96. \end{aligned}$$

(D1)

Similarly for the bosonic counter part, D.O.F can be calculated as [80],

$$\begin{aligned} g_{\text {boson}}= & {} g_{\text {gluon}} + g_{\text {weak}} + g_{\text {photon}} + g_{\text {Higgs}} + g_{\text {BSM-Higgs}} \nonumber \\&= (8 \times 2) + (3 \times 3) + 2 + 1 + 8 = 36. \end{aligned}$$

(D2)

For 3HDM the total effective D.O.F can be obtained by summing up the D.O.Fs in fermion and bosonic sectors,

$$\begin{aligned} g_{eff} (T > 174~ \mathrm{GeV}) = 36 + \frac{7}{8} \times 96 = 120. \end{aligned}$$

(D3)