$Q_{6}$ as the flavor symmetry in a non-minimal SUSY $SU(5)$ model

Abstract

We present a non-minimal renormalizable SUSY $SU(5)$ model, with extended Higgs sector and right-handed neutrinos, where the flavor sector exhibits a $Q_{6}$ flavor symmetry. We analyzed the simplest version of this model, in which R-parity is conserved and the right-handed neutrino masses in the flavor doublet are considered with and without degeneracy. We find the generic form of the mass matrices both in the quark and lepton sectors. We reproduce, according to current data, the mixing in the CKM matrix. In the leptonic sector, in the general case where the right-handed neutrino masses are not degenerate, we find that the values for the solar, atmospheric, and reactor mixing angles are in very good agreement with the experimental data, both for a normal and an inverted hierarchy. In the particular case where the right-handed neutrinos masses are degenerate, the model predicts a strong inverted hierarchy spectrum and a sum rule among the neutrino masses. In this case the atmospheric and solar angles are in very good agreement with experimental data, and the reactor one is different from zero, albeit too small ($\theta ^{\ell ^{\mathrm{th}}}_{13} \sim 3.38^{\circ }$). This value constitutes a lower bound for $\theta _{13}$ in the general case. We also find the range of the values for the neutrino masses in each case.

1 Introduction

Understanding the flavor sector of the Standard Model (SM) has been a puzzle for a long time, due to the large differences in the Yukawa couplings of the different fermions. The hierarchy between the fundamental particles, the amount of CP violation, and the structure of the CKM matrix remain as open problems [1]. In spite of these subtle facts, the success of the SM is remarkable.

One of the strategies to deal with the flavor problem in the quark and lepton sectors has been to study it in the framework of textures (zeros) in the mass matrices. The textures have been explored for a long time, as an attempt to eliminate the irrelevant free parameters in the Yukawa sector. As is well known, the Fritzsch [2, 3] textures can accommodate the quarks’ mixing angles, in terms of the quarks’ masses. This approach seems to work out correctly because the Cabbibo angle is obtained with great accuracy [2]. However, this framework presents some problems with the top mass and the $V_{\mathrm{cb}}$ element of the CKM matrix [4, 5], as can be seen in [6, 7]. Recently, deviations to the Fritzsch textures have appeared in order to overcome these problems. Moreover, the charged lepton and neutrino sector have been included in this kind of ansatz and consistent results on the PMNS matrix [8, 9] have been observed in this generic approximation [7]. As alternative textures to the Fritzsch ones, the Nearest Neighbor Interaction (NNI) textures can also reproduce very well the flavor mixing in the quark and lepton sectors [10, 11]; it is well known that the Fritzsch textures can be obtained from the NNI ones as a limiting case [10].

Non-Abelian flavor symmetries have played an important role in model building; these symmetries are considered as an elegant way to obtain the NNI textures in the fermion mass matrices [10, 11]. In particular, the flavor symmetry group $Q_{6}$ [12] has been proposed as responsible of these kind of textures in the quarks as well as in the leptonic sector [13–17]. The rich phenomenology that $Q_{6}$ provides in the SUSY scenario is remarkable; one of these features is to prohibit the dangerous terms that mediate fast proton decay, rather than invoking the $R$-parity symmetry [14, 15]. The immediate question that arises is, how does the $Q_{6}$ symmetry work within a GUT framework? In particular, the main question that we will address here is if the SUSY $SU(5)$ models are compatible with the $Q_{6}$ group in order to accommodate masses and mixings for fermions.

From a theoretical point of view, grand unification ideas are well motivated and there are fundamental reasons for studying them [18–21]. In particular, the $SU(5)$ model [21] is considered to be one of the best scenarios to unify the electroweak and strong interactions. However, the model itself faces serious phenomenological and theoretical problems, one of them being that active neutrinos are massless [22, 23]; another one is that the unification of gauge couplings is not quite good with the current precision data. The simplicity of the $SU(5)$ can be retained even if it is promoted to a supersymmetric model. The SUSY $SU(5)$ [24–26] version, with $R$-parity conserved, gives a much better unification of the gauge couplings, but new couplings (five-dimensional operators [27, 28]) can yield the proton decay [29–31], and therefore, they can exclude the SUSY version [32–34]. On the other hand, generic studies on the minimal SUSY $SU(5)$ model have made it clear that it may not be ruled out by the experimental data on the proton decay rates [35–37]. Taking into account the neutrino mass problem, the SUSY $SU(5)$ version (SUSY models in general) provides elegant mechanisms to generate massive neutrinos via $R$-parity violation [38, 39]. But still, the simplest way to give mass to the left-handed neutrinos in the minimal $SU(5)$ or SUSY scenario is to consider three right-handed neutrinos (RHNs), which are singlets under the gauge group and invoke the type I see-saw mechanism [40–44].

There are interesting SUSY $SU(5)$ models [45–47], which can very well reproduce the CKM and PMNS matrices in agreement with the experimental results, but where the simplicity in the matter content has been left aside. Most of these models include a large number of flavons, which are required to accommodate correctly the mixings. A different approach consists of extending the Higgs sector of the models. By itself, the SUSY $SU(5)$ matter content provides the tools to accommodate masses and mixings via the extension of the Higgs scalar sector, although gauge coupling unification may be compromised. This philosophy has been worked out with success in non-supersymmetric [48–50] and supersymmetric [13–17] scenarios, where the concept of flavor has been extended to the Higgs sector.

Therefore, we propose here a renormalizable SUSY $SU(5)$ model where the $Q_{6}$ group plays an important role in the flavor sector. The need to extend the scalar sector is evident in order to accommodate masses and mixings for quarks and leptons without breaking explicitly the flavor symmetry, so that three families of $H^\mathrm{u}$ and $H^\mathrm{d}$ $5$-plets are introduced. At the same time, three RHNs are included by hand, and the type I see-saw mechanism is invoked to get small masses for the active neutrinos. At low energies when the first and second RHN masses are degenerate, a lower bound for the reactor angle, $\theta _{13}^{\ell }$, is obtained. Namely, consistent results are obtained for the CKM matrix in the quark sector, while in the leptonic sector, the values obtained for the atmospheric and solar mixing angles are: $\theta ^{\ell ^{\mathrm{th}}}_{23} = ( 46.18^{+0.66}_{-0.65} )^{\circ }$ and $\theta ^{\ell ^{\mathrm{th}}}_{12} = ( 36.62 \pm 4.06 )^{\circ }$, which are consistent with experimental data. However, the reactor mixing angle value $\theta ^{\ell ^{\mathrm{th}}}_{13} = (3.38^{+0.03}_{-0.02} )^{\circ }$ is not in good agreement with the central values of the global fits but it is still within the error bar of the different experiments, and fairly large in comparison to the tribimaximal scenario. This value corresponds to a lower bound for the reactor mixing angle of the more general case where the RHNs are not mass degenerate, similar to the case of $S_3$ non-supersymmetric models, where relaxing the degeneracy condition in the RHN masses gives the right value for $\theta _{13}^{\ell }$ [50]. So, in order to enhance the theoretical value of the reactor mixing angle, which must be in accordance with the current experimental data, we assume that the two first RHNs, $N_{1}$ and $N_{2}$, have different masses in the Majorana mass term. This assumption allows one to get the following values for the normal (inverted) hierarchy: $\theta _{12}^{ \ell ^{\mathrm{th}} } = ( 34.71_{-0.98}^{+0.91} )^{\circ }$ $[ ( 34.73_{-1.11}^{+0.89} )^{\circ } ]$, $\theta _{23}^{ \ell ^{\mathrm{th}} } = ( 45.83_{-3.98}^{+4.49} )^{\circ }$ $[ ( 48.57_{-2.76}^{+2.07} )^{\circ } ]$, and $\theta _{13}^{ \ell ^\mathrm{th} } = ( 8.77_{-0.32}^{+0.40} )^{\circ }$ $[ ( 8.93_{-0.39}^{+0.33} )^{\circ } ]$.

The paper is organized as follows: In Sect. 2, we build the extended SUSY $SU(5)$ model which obeys the $Q_{6}$ flavor symmetry, in addition, we explain the required matter content to get a unified scenario and we do stress the strong assumptions that we will make in the model; at the same time, the ansatz to increase the value of reactor mixing angle is discussed. The textures in the quark mass matrices, and therefore their consequences in the lepton sector, are analyzed in Sect. 3. Also, we describe how to diagonalize the mass matrices for each sector, with particular attention to the neutrino sector. In Sect. 4, we present and discuss the results about the mixing angles for each sector. Finally, we give conclusions on this preliminary analysis of the model.

2 SUSY $SU(5)\otimes Q_{6}$ model

We will consider the SUSY $SU(5)$ model where three right-handed neutrinos have been included in the matter content. In addition, the scalar sector has to be necessarily extended as we will show. Thus, the matter content that will be used in the present model is displayed on Table 1, where the assignments under $Q_{6}$ flavor symmetry are shown.

Table 1 Matter content in the SUSY $SU(5) \otimes Q_{6}$ model

Let us comment on our notation and the matter content: $\Phi ^{a}_{b}$ stands for the $\mathbf{24 }$ adjoint scalar representation which breaks ($\langle \Phi \rangle = \sigma \text {diag} ( 1, 1, 1, -3/2, -3/2 )$) the SUSY $SU(5)$ gauge group to the MSSM; there are three families of Higgs type $H^\mathrm{u}_{i}$ and $H^\mathrm{d}_{j}$. Here, we ought to stress a point. Because we have extended the Higgs sector, gauge coupling unification is not guaranteed, since the Renormalization Group Equations (RGEs) depend strongly on the number of Higgs families [51, 52]. This is a drawback in the present model, and it is the price we have to pay for going beyond the minimal SUSY $SU(5)$ model. A proper analysis of the RGE evolution, including also the right-handed neutrinos will be left for a future work. On the other hand, we do need to include the $H_{45}$ and ${H}_{\bar{45}}$ scalar representations so as to fix the incorrect relation $\mathbf{M}_\mathrm{d} = \mathbf{M}^\mathrm{T}_\mathrm{e}$, although there are other ways to achieve it; see [53, 54]. In addition, there is flavon, $Y_\mathrm{B}$, which has already been used in supersymmetric models [13–17] in order to have a flavor invariant Majorana mass matrix. Regarding the fermion sector, $N_{i}$ denotes the RHN, which is a singlet under the $SU(5)$ gauge group; $F_{i}$ and $T_{j}$ stand for the $\mathbf{5}$-plets and the antisymmetric $\mathbf{10}$-plet, respectively. Here, $a, b,c$ are $SU(5)$ indices, and $i,j$ are family indices. More explicitly,

$$\begin{aligned} \begin{aligned} F_{i a}&= (d^\mathrm{c}, L); \quad T^{ab}_{j} = \dfrac{1}{\sqrt{2}} ( u^\mathrm{c}, q, \ell ^\mathrm{c} ); \\&\qquad L = \begin{pmatrix} \nu _{\ell } \\ \ell \end{pmatrix}; \quad q = \begin{pmatrix} u \\ d \end{pmatrix}, \\ H^{\mathrm{u} a}&= \begin{pmatrix} \mathtt H ^\mathrm{u} \\ \mathbf H ^\mathrm{u} \end{pmatrix}; \quad H^\mathrm{d}_{b}= \begin{pmatrix} \mathtt H ^\mathrm{d} \\ \mathbf H ^\mathrm{d} \end{pmatrix};\\&\qquad \mathbf H ^\mathrm{u}= \begin{pmatrix} h^{+ \mathrm{u}} \\ h^{0 \mathrm{u}} \end{pmatrix}; \quad \mathbf H ^\mathrm{d}= \begin{pmatrix} h^{0 \mathrm{d}} \\ h^{- \mathrm{d}} \end{pmatrix}. \end{aligned} \end{aligned}$$

(1)

Here, $\mathtt H ^\mathrm{u}$ and $\mathtt H ^\mathrm{d}$ are the colored triplet scalars that mediate proton decay. For the time being, it will be assumed that these are heavy enough to keep the proton lifetime bounded and under control. In addition, we have to point out that this subtle issue will be left aside, since we are only interested in studying the masses and mixings when implementing $Q_{6}$ as a flavor symmetry in this model. On the other hand, $\mathbf H ^\mathrm{d}$ and $\mathbf H ^\mathrm{u}$ are the weak doublets of the MSSM group. In consequence, we employ the following vacuum expectation values (VEVs) to get the fermion mass matrices:

$$\begin{aligned} \begin{aligned} \langle \mathbf H ^{\mathrm{u} }\rangle&= \begin{pmatrix} 0 \\ h^{0\mathrm{u}} \end{pmatrix}, \qquad \langle \mathbf H ^\mathrm{d} \rangle = \begin{pmatrix} h^{0 \mathrm{d}} \\ 0 \end{pmatrix},\\&\qquad \langle H_{45} \rangle ^{\alpha 5}_{\alpha } = v_{45},\qquad \langle H_{45} \rangle ^{4 5}_{4} = -3 v_{45} \\ \langle H_{\bar{45}} \rangle ^{\alpha }_{\alpha 5}&= v_{\bar{45}},\qquad \langle H_{\bar{45}} \rangle ^{4}_{4 5} = -3 v_{\bar{45}},\\&\qquad \langle Y_\mathrm{B} \rangle =v_\mathrm{B} \qquad \alpha ,\beta =1,2,3. \end{aligned} \end{aligned}$$

(2)

Having presented the assigned matter fields under the $Q_{6}$ flavor symmetry, we will now introduce the superpotential, gauge invariant under the $Q_{6}$ discrete group. All necessary details of the multiplication rules of $Q_{6}$ are given in the appendix, where a brief review of this dihedral group is offered. The trilinear terms in the superpotential are given by

$$\begin{aligned} W&=\sqrt{2} y^\mathrm{d}_{1} ( F_{1} T_{2} - F_{2} T_{1} ) H^\mathrm{d}_{3} + \sqrt{2} y^\mathrm{d}_{2} ( F_{1} T_{3} H^\mathrm{d}_{2} - F_{2} T_{3} H^\mathrm{d}_{1}) \nonumber \\&\quad + \sqrt{2} y^\mathrm{d}_{3} F_{3} ( T_{1} H^\mathrm{d}_{2} - T_{2} H^\mathrm{d}_{1} ) + \sqrt{2} y^\mathrm{d}_{4} F_{3} T_{3} H^\mathrm{d}_{3}\nonumber \\&\quad + \dfrac{ y^\mathrm{u}_{1} }{4} ( T_{1} T_{2} - T_{2} T_{1} ) H^\mathrm{u}_{3} + \dfrac{ y^\mathrm{u}_{2} }{4} ( T_{1} T_{3} H^\mathrm{u}_{2} - T_{2} T_{3} H^\mathrm{u}_{1} ) \nonumber \\&\quad + \dfrac{ y^\mathrm{u}_{3} }{4} T_{3} ( T_{1} H^\mathrm{u}_{2} -T_{2} H^\mathrm{u}_{1} ) + \dfrac{ y^\mathrm{u}_{4} }{4} T_{3} T_{3} H^\mathrm{u}_{3}\nonumber \\&\quad + \sqrt{2} Y_{1} ( F_{1} T_{2} - F_{2} T_{1} ) H_{\bar{45}} + \sqrt{2} Y_{2} F_{3} T_{3} H_{\bar{45}} \nonumber \\&\quad + \dfrac{ \tilde{Y}_{1}}{4} ( T_{1} T_{2} - T_{2} T_{1} ) H_{45} + \dfrac{ \tilde{Y}_{2}}{4} T_{3} T_{3} H_{45}\nonumber \\&\quad + y^{n}_{1} ( N^\mathrm{c}_{1} F_{2} - N^\mathrm{c}_{2} F_{1} ) H^\mathrm{u}_{3} + y^{n}_{2} ( N^\mathrm{c}_{1} F_{3} H^\mathrm{u}_{2} - N^\mathrm{c}_{2} F_{3} H^\mathrm{u}_{1} ) \nonumber \\&\quad + y^{n}_{3} N^\mathrm{c}_{3} ( F_{1} H^\mathrm{u}_{2} + F_{2} H^\mathrm{u}_{1} ) + M_{R_{1}} ( N^\mathrm{c}_{1} N^\mathrm{c}_{1} + N^\mathrm{c}_{2} N^\mathrm{c}_{2} ) \nonumber \\&\quad + y^{m}_{2} N^\mathrm{c}_{3} Y_\mathrm{B} N^\mathrm{c}_{3}. \end{aligned}$$

(3)

It is important to remember that SUSY must be broken via soft breaking terms, so these soft breaking terms should be included in a complete study of the full scalar potential, but we do not include them in this preliminary analysis. The scalar superpotential is

$$\begin{aligned} W_{\mathrm{s}}&= \mu _{\Phi } \mathrm{Tr} ( \Phi ^{2} ) + \lambda _{\Phi } \mathrm{Tr} ( \Phi ^{3} ) + m_{45} H_{\bar{45}} H_{45} \nonumber \\&\quad + \mu _{1} ( H^\mathrm{u}_{1} H^\mathrm{d}_{2} - H^\mathrm{u}_{2} H^\mathrm{d}_{1} ) + \mu _{2} H^\mathrm{u}_{3} H^\mathrm{d}_{3}\nonumber \\&\quad + \lambda _{1} ( H^\mathrm{u}_{1} \Phi H^\mathrm{d}_{2} - H^\mathrm{u}_{2} \Phi H^\mathrm{d}_{1} ) + \lambda _{2} H^\mathrm{u}_{3} \Phi H^\mathrm{d}_{3} \nonumber \\&\quad + a_{1} H_{\bar{45}} H_{45} \Phi + a_{2} H_{\bar{45}} \Phi H^\mathrm{u}_{3} + a_{3} H^\mathrm{d}_{3} \Phi H_{45}. \end{aligned}$$

(4)

From the superpotential given in Eq. (3), one finds the MSSM effective superpotential that contains the Yukawa mass term after spontaneous symmetry breaking via the VEVs of the $\mathbf H ^\mathrm{u}$ and $\mathbf H ^\mathrm{d}$ weak doublets scalar superfields. We will work in the following basis:

$$\begin{aligned} \mathcal {L} = \mathcal {L}^{\mathrm{q}} + \mathcal {L}^{\mathrm{l}} \end{aligned}$$

(5)

with

$$\begin{aligned} \mathcal {L}^{\mathrm{q}}= & {} - \bar{d}_{i \mathrm{R}} ( \mathbf{M}_{\mathrm{d}} )_{ij} d_{j \mathrm{L}} - \bar{u}_{i \mathrm{R}} ( \mathbf{M}_{\mathrm{u}} )_{ij} u_{j \mathrm{L}} + \mathrm{h.c.}, \nonumber \\ \mathcal {L}^{\mathrm{l}}= & {} \bar{\ell }_{i \mathrm{R}} ( \mathbf{M}_{\ell } )_{ij} \ell _{j \mathrm{L}} - \bar{N}_{i \mathrm{R}} ( \mathbf{M}_{\mathrm{D}} )_{ij} \nu _{j \mathrm{L}} \\&- \frac{1}{2} \bar{N}_{i \mathrm{R}} ( \mathbf{M}_{\mathrm{R}} )_{ij} N^\mathrm{c}_{j \mathrm{R}} + \mathrm{h.c.}. \nonumber \end{aligned}$$

(6)

In general, the up, down, and charged lepton mass matrices are given by (see [55])

$$\begin{aligned} \mathbf{M}_\mathrm{u}= & {} \dfrac{ ( \mathbf{Y}^\mathrm{u} + \mathbf{Y}^{\mathrm{u T}} ) }{2} \langle \mathbf H ^\mathrm{u} \rangle - ( \tilde{\mathbf{Y}}^{\mathrm{T}} - \tilde{\mathbf{Y}} ) v_{45}, \quad \mathbf{M}_{\mathrm{d}} = \mathbf{Y}^\mathrm{d} \langle \mathbf H ^\mathrm{d} \rangle + 2 \mathbf{Y} v_{\bar{45}}, \nonumber \\ \mathbf{M}_{\ell }= & {} \mathbf{Y}^{\mathrm{d T}} \langle \mathbf H ^\mathrm{d} \rangle -6 \mathbf{Y}^{\mathrm{T}} v_{\bar{45}}. \end{aligned}$$

(7)

In this particular model, from Eqs. (3) and (7) we see that the up, down, and charged lepton mass matrices have, respectively, the following structures:

$$\begin{aligned} \mathbf{M}_{\mathrm{u}}&= \begin{pmatrix} 0 &{} \quad -2\tilde{Y}_{1} v_{45} &{} \quad \bar{y}^\mathrm{u} h^{0 \mathrm{u}}_{2} \\ 2 \tilde{Y}_{1} v_{45} &{}\quad 0 &{}\quad - \bar{y}^\mathrm{u} h^{0 \mathrm{u}}_{1} \\ \bar{y}^\mathrm{u} h^{0 \mathrm{u}}_{2} &{}\quad - \bar{y}^\mathrm{u}h^{0 \mathrm{u}}_{1} &{}\quad y^\mathrm{u}_{4} h^{0 \mathrm{u}}_{3} \end{pmatrix},\nonumber \\ \mathbf{M}_{\mathrm{d}}&= \begin{pmatrix} 0 &{}\quad y^\mathrm{d}_{1} h^{0 \mathrm{d}}_{3} + 2 Y_{1} v_{\bar{45}} &{} \quad y^\mathrm{d}_{2} h^{0 \mathrm{d}}_{2} \\ - y^\mathrm{d}_{1} h^{0 \mathrm{d}}_{3} - 2 Y_{1} v_{\bar{45}} &{}\quad 0 &{}\quad - y^\mathrm{d}_{2} h^{0 \mathrm{d}}_{1} \\ y^\mathrm{d}_{3} h^{0 \mathrm{d}}_{2} &{}\quad - y^\mathrm{d}_{3} h^{0 \mathrm{d}}_{1} &{}\quad y^\mathrm{d}_{4} h^{0 \mathrm{d}}_{3} + 2Y_{2} v_{\bar{45}} \end{pmatrix}; \nonumber \\ \mathbf{M}_{\ell }&= \begin{pmatrix} 0 &{}\quad - \left( y^\mathrm{d}_{1} h^{0 \mathrm{d}}_{3} - 6 Y_{1} v_{\bar{45}} \right) &{}\quad y^\mathrm{d}_{3} h^{0 \mathrm{d}}_{2} \\ y^\mathrm{d}_{1} h^{0 \mathrm{d}}_{3} - 6 Y_{1} v_{\bar{45}} &{}\quad 0 &{}\quad - y^\mathrm{d}_{3} h^{0 \mathrm{d}}_{1} \\ y^\mathrm{d}_{2} h^{0 \mathrm{d}}_{2} &{}\quad - y^\mathrm{d}_{2} h^{0 \mathrm{d}}_{1} &{}\quad y^\mathrm{d}_{4} h^{0 \mathrm{d}}_{3} - 6 Y_{2} v_{\bar{45}} \end{pmatrix}, \end{aligned}$$

(8)

where $\bar{y}^\mathrm{u} \equiv ( y^\mathrm{u}_{2} + y^\mathrm{u}_{3} )/2$. As can be seen, the $\mathbf{M}_{\mathrm{u}}$ mass matrix turns out almost symmetric due to the flavor structure. At the same time, we were able to correct the wrong relationship between the down quarks and the charged leptons; this was achieved including the $H_{\bar{45}}$ scalar representation. From Eq. (3), we obtain the Dirac (see Eq. 9) and Majorana mass matrices; the latter are strictly given by $\mathbf{m}_{\mathrm{R}} = \text {diag} ( M_{\mathrm{R}_{1}}, M_{\mathrm{R}_{1}}, M_{\mathrm{R}_{3}} = y^{m}_{2}v_\mathrm{B})$. We would like to mention at this point the ansatz to enhance the $\theta _{13}^{l}$ value, and its subtle ingredient, which consists in assuming that two RHNs, $N_{1}$ and $N_{2}$, are not degenerate. Thus, the Majorana mass matrix is given in Eq. (9), and two cases will be studied later: (a) degenerate RHNs, (b) non-degenerate RHNs. We have

$$\begin{aligned} \mathbf{M}_{\mathrm{D}} =&\begin{pmatrix} 0 &{}\quad y^{n}_{1} h^{0 \mathrm{u}}_{3} &{}\quad y^{n}_{2} h^{0 \mathrm{u}}_{2} \\ -y^{n}_{1} h^{0 \mathrm{u}}_{3}&{} 0 &{}-y^{n}_{2} h^{0 \mathrm{u}}_{1} \\ y^{n}_{3} h^{0 \mathrm{u}}_{2} &{}\quad y^{n}_{3} h^{0 \mathrm{u}}_{1} &{}\quad 0 \end{pmatrix} \quad \text {and} \nonumber \\ \mathbf{M}_{\mathrm{R}}&= \begin{pmatrix} M_{\mathrm{R}_{1}}&{}\quad 0 &{}\quad 0 \\ 0 &{}\quad M_{\mathrm{R}_{2}} &{}\quad 0 \\ 0 &{} &{}\quad M_{\mathrm{R}_{3}} \end{pmatrix}. \end{aligned}$$

(9)

Therefore, corresponding to the type I see-saw mechanism, the neutrino mass term is given by

$$\begin{aligned} \mathcal {L}_{\nu }= & {} -\bar{N}_{i \mathrm{R}} ( \mathbf{M}_{\mathrm{D}} )_{ij} \nu _{j \mathrm{L}} - \frac{1}{2} \bar{N}_{i \mathrm{R}} ( \mathbf{M}_{\mathrm{R}} )_{ij} N^\mathrm{c}_{j \mathrm{R}} \nonumber \\&+\,\mathrm{h.c.} = - \frac{1}{2} \bar{\nu }^\mathrm{C}_{\mathrm{L}} \mathbf{M}_{\nu } \nu _{\mathrm{L}} - \frac{1}{2} \bar{N}_{i\mathrm{R}} ( \mathbf{M}_{\mathrm{R}} )_{ij} N^\mathrm{c}_{j \mathrm{R}}, \end{aligned}$$

(10)

where the $\mathbf{M}_{\nu } = \mathbf{M}^{\mathrm{T}}_{\mathrm{D}} \mathbf{M}_{\mathrm{R}}^{-1} \mathbf{M}_{\mathrm{D}}$ effective neutrino mass matrix has the following structure:

$$\begin{aligned} \mathbf{M}_{\nu } = \left( \begin{array}{lll} \frac{ \left( y^{n}_{1} h^{0 \mathrm{u}}_{3} \right) ^{2}}{ M_{\mathrm{R}_{2}} } + \frac{ \left( y^{n}_{3} h^{0 \mathrm{u}}_{2} \right) ^{2}}{ M_{\mathrm{R}_{3}} } &{} \frac{ \left( y^{n}_{3} \right) ^{2} h^{0 \mathrm{u}}_{2} h^{0 \mathrm{u}}_{1} }{ M_{\mathrm{R}_{3}} } &{} \frac{ y^{n}_{1} h^{0 \mathrm{u}}_{3} y^{n}_{2} h^{0 \mathrm{u}}_{1} }{ M_{\mathrm{R}_{2}} }\\ \frac{ \left( y^{n}_{3} \right) ^{2} h^{0 \mathrm{u}}_{2} h^{0 \mathrm{u}}_{1} }{ M_{\mathrm{R}_{3}} } &{} \frac{ \left( y^{n}_{1} h^{0 \mathrm{u}}_{3} \right) ^{2}}{ M_{\mathrm{R}_{1}} } + \frac{ \left( y^{n}_{3} h^{0 \mathrm{u}}_{1} \right) ^{2}}{ M_{\mathrm{R}_{3}} } &{} \frac{ y^{n}_{1} h^{0 \mathrm{u}}_{3} y^{n}_{2} h^{0 \mathrm{u}}_{2} }{ M_{\mathrm{R}_{1}} }\\ \frac{ y^{n}_{1} h^{0 \mathrm{u}}_{3} y^{n}_{2} h^{0 \mathrm{u}}_{1} }{ M_{R_{2}} } &{} \frac{ y^{n}_{1} h^{0 \mathrm{u}}_{3} y^{n}_{2} h^{0 \mathrm{u}}_{2} }{ M_{\mathrm{R}_{1}} } &{} \frac{ \left( y^{n}_{2} h^{0 \mathrm{u}}_{2} \right) ^{2}}{ M_{\mathrm{R}_{1}} } + \frac{ \left( y^{n}_{2} h^{0 \mathrm{u}}_{1} \right) ^{2}}{ M_{\mathrm{R}_{2}} } \end{array} \right) . \end{aligned}$$

(11)

At first sight the mass matrix $M_{\nu }$ may seem very complicated to be diagonalized analytically. Also, it has 12 real free parameters, since it is a complex symmetric matrix. Thus, the number of parameters in the matrix $M_{\nu }$ is very large. However, it is well known that an effective way to reduce the number of parameters in a mass matrix is to perform a similarity transformation, through which it is possible to go to a basis where the matrix will have some texture zeros [56]. Then, in complete analogy with the work done on the $S_{3}$-flavor symmetry [50, 57–59] and due to the form of $\mathbf{M}_{\nu }$, we can rotate the left-handed neutrino field as follows: $\nu _{\mathrm{L}} = \mathbf{U}_{\nu } \tilde{\nu }_{\mathrm{L}}$, where $\mathbf{U}_{\nu } = \mathbf{\textit{u} }_{\theta } \mathbf{u}_{\nu }$ so that for a normal (inverted) hierarchy in the neutrino masses one gets:¹

$$\begin{aligned} {\tilde{\mathbf{M}} }_{\nu } = \text {diag} ( m_{\nu _{1[3]}}, m_{\nu _{2[1]}}, m_{\nu _{3[2]}} ) = \mathbf{u}^{\mathrm{T}}_{\nu }\mathbf{m}_{\nu } \mathbf{u}_{\nu }, \end{aligned}$$

(12)

where

$$\begin{aligned} \mathbf{m}_{\nu }= & {} \mathbf{\textit{u} }_{\theta }^{\mathrm{T}} \mathbf{M}_{\nu } \mathbf{\textit{u} }_{\theta } = \left( \begin{array}{c@{\quad }c@{\quad }c} b_{\nu } &{} a_{\nu } &{} c_{\nu } \\ a_{\nu } &{} \mu _{0} &{} 0 \\ c_{\nu } &{} 0 &{} d_{\nu } \end{array} \right) \nonumber \\&\quad \text {and} \quad \mathbf{\textit{u} }_{\theta } = \left( \begin{array}{c@{\quad }c@{\quad }c} \cos \theta &{} 0 &{} -\sin \theta \\ \sin \theta &{} 0 &{} \cos \theta \\ 0 &{} 1 &{} 0 \end{array} \right) , \end{aligned}$$

(13)

with

$$\begin{aligned} \tan \theta&= \frac{ M_{\mathrm{R}_{2}} }{ M_{\mathrm{R}_{1}} } \frac{ h^{0 \mathrm{u}}_{2} }{ h^{0 \mathrm{u}}_{1} }, \quad \mu _{0} = \frac{ ( y_{2}^{n} )^{2} }{ M_{\mathrm{R}_{1}} M_{\mathrm{R}_{2}} } [ ( h^{0 \mathrm{u}}_{2} )^{2} M_{R_{2}} \nonumber \\&\qquad + ( h^{0 \mathrm{u}}_{1} )^{2} M_{\mathrm{R}_{1}}], \nonumber \\ a_{\nu } =&\frac{ \cos \theta y_{1}^{n} h^{0 \mathrm{u}}_{3} y_{2}^{n} }{ M_{\mathrm{R}_{2} } M_{ \mathrm{R}_{1} }^{2} h^{0 \mathrm{u}}_{1} } \left( ( h^{0 \mathrm{u}}_{1} )^{2} M_{ \mathrm{R}_{1} }^{2} + ( h^{0\mathrm{u}}_{2} )^{2} M_{\mathrm{R}_{2}}^{2} \right) , \nonumber \\ b_{\nu }&= \frac{ \cos ^{2} \theta }{ M_{\mathrm{R}_{1}}^{2} } \bigg [ \frac{ ( y_{1}^{n} h^{0\mathrm{u}}_{3} )^{2} }{ M_{ \mathrm{R}_{1} } M_{ \mathrm{R}_{2} } ( h^{0 \mathrm{u}}_{1} )^{2} } ( M_{\mathrm{R}_{1}}^{3} ( h^{0 \mathrm{u}}_{1} )^{2} + M_{\mathrm{R}_{2}}^{3} ( h^{0 \mathrm{u}}_{2} )^{2} ) \nonumber \\&\qquad + \frac{ ( y_{3}^{n} h^{0 \mathrm{u}}_{2})^{2} }{ M_{\mathrm{R}_{3}} } ( M_{\mathrm{R}_{1}} + M_{\mathrm{R}_{2}} )^{2} \bigg ], \nonumber \\ c_{\nu } =&\frac{ h^{0 \mathrm{u}}_{2} \cos ^{2} \theta }{ h^{0 \mathrm{u}}_{1} M_{\mathrm{R}_{1}}^{2} } \bigg [ ( y_{1}^{n} h^{0 \mathrm{u}}_{3} )^{2} ( M_{\mathrm{R}_{2}} - M_{\mathrm{R}_{1}} )+ \frac{ ( y_{3}^{n} )^{2} }{ M_{\mathrm{R}_{3}} } \nonumber \\&\quad \times ( M_{\mathrm{R}_{1}} + M_{\mathrm{R}_{2}} ) (( h^{0\mathrm{u}}_{1} )^{2} M_{\mathrm{R}_{1}} - ( h^{0 \mathrm{u}}_{2} )^{2} M_{\mathrm{R}_{2}} ) \bigg ], \; \text {and} \; \nonumber \\ d_{\nu }&= \frac{ \cos ^{2} \theta }{ ( M_{\mathrm{R}_{1}} h^{0 \mathrm{u}}_{1} )^{2} } \bigg [ ( y_{1}^{n} h^{0 \mathrm{u}}_{3} )^{2} ( M_{\mathrm{R}_{2}} ( h^{0 \mathrm{u}}_{2} )^{2} \nonumber \\&\quad + M_{\mathrm{R}_{1}} ( h^{0 \mathrm{u}}_{1} )^{2} ) + \frac{ ( y_{3}^{n} )^{2} }{ M_{\mathrm{R}_{3}} } ( M_{\mathrm{R}_{1}} ( h^{0 \mathrm{u}}_{1} )^{2} - M_{\mathrm{R}_{2}} ( h^{0 \mathrm{u}}_{2} )^{2} )^{2} \bigg ].\quad \end{aligned}$$

(14)

The mass matrix $\mathbf{m}_{\nu }$, Eq. (13), has one texture zero, so it has ten real effective free parameters. But, as we will show later on, when we relate $\mathbf{m}_{\nu }$ with a matrix with two texture zeros of class I [50, 57–60], the number of real effective free parameters is reduced to only four.

3 Masses and mixings in the NNI scenario

There are two ways to obtain the Fritzsch and NNI textures in the up-, down-quark, and charged lepton sector, respectively. The first scenario consists in taking the condition $h^{0 \mathrm{u}}_{2}=0=h^{0 \mathrm{d}}_{2}$ on the VEVs, and the second one consists in assuming $h^{0 \mathrm{u}}_{2}=h^{0\mathrm{u}}_{1}\equiv h^{0\mathrm{u}}$ and $h^{0 \mathrm{d}}_{2}=h^{0 \mathrm{d}}_{1}\equiv h^{0 \mathrm{d}}$. Without loss of generality, in the next subsection we will describe the second way.

3.1 Quark and lepton masses

First, if we assume that $h^{0 \mathrm{u}}_{2}=h^{0\mathrm{u}}_{1}\equiv h^{0\mathrm{u}}$ and $h^{0 \mathrm{d}}_{2}=h^{0\mathrm{d}}_{1}\equiv h^{0\mathrm{d}}$, we obtain the following mass matrices:

$$\begin{aligned}&\mathbf{M}_{j} = \begin{pmatrix} 0 &{}\quad \pm A_{j} &{}\quad B_{j} \\ \mp A_{j} &{}\quad 0 &{}\quad -B_{j} \\ C_{j} &{}\quad -C_{j} &{}\quad D_{j} \end{pmatrix}, \quad \text {and}\nonumber \\&\mathbf{M}_{\mathrm{u}} = \begin{pmatrix} 0 &{} \quad -A_{\mathrm{u}} &{}\quad B_{\mathrm{u}} \\ A_{\mathrm{u}} &{}\quad 0 &{}\quad -B_{\mathrm{u}} \\ B_{\mathrm{u}} &{}\quad -B_{\mathrm{u}} &{}\quad D_{\mathrm{u}} \end{pmatrix}. \end{aligned}$$

(15)

Here, the subindex $j$ denotes the charged lepton and quark-down mass matrices, namely $j= \ell , d$, while the upper (lower) sign corresponds to the $\mathbf{M}_{\mathrm{d}}$ ($\mathbf{M}_{\ell }$) mass matrix. In addition, the explicit form of the matrix elements is given in Eq. (8). The $\mathbf{M}_{j}$ and $\mathbf{M}_{\mathrm{u}}$ mass matrices contain implicitly the $NNI$ and $Fritzsch$ textures, respectively, which appear explicitly as follows: the above mass matrices are diagonalized by unitary matrices, $\mathbf{U}_{k (\mathrm{R,L})}$ with the subindex $k=d, \ell , u$. According to Eq. (6) one obtains $\tilde{\mathbf{M}}_{ k } = \mathbf{U}^{\dagger }_{k \mathrm{R}} \hat{ \mathbf{M} }_{k} \mathbf{U}_{k \mathrm{L}}$, in general. Here, $\tilde{\mathbf{M}}_{k} = \text {diag} ( \tilde{m}_{k_{1}}, \tilde{m}_{k_{2}}, 1 )$. For simplicity, we have normalized the above expressions so that $\tilde{m}_{k_{1}} = m_{k_{1}}/m_{k_{3}}$, $\tilde{m}_{k_{2}} = m_{k_{2}}/m_{k_{3}}$, and $\hat{\mathbf{M}}_{k} = \mathbf{M}_{k}/m_{k_{3}}$ are dimensionless parameters. Then, taking the unitary matrices as $\mathbf{U}_{k (\mathrm{R, L})} = \mathbf{U}_{\pi /4} \mathbf{u}_{k (\mathrm{R, L})}$, one easily gets $\tilde{\mathbf{M}}_{k}=\mathbf{u}^{\dagger }_{k \mathrm{R}} \mathbf{m}_{k} \mathbf{u}_{k \mathrm{L}}$ where

$$\begin{aligned}&\mathbf{m}_{k} = \mathbf{U}^{\mathrm{T}}_{\pi /4} \hat{\mathbf{M}}_{k} \mathbf{U}_{\pi /4} = \begin{pmatrix} 0 &{}\quad \pm \tilde{A}_{k} &{}\quad 0 \\ \mp \tilde{A}_{k} &{}\quad 0 &{}\quad -\sqrt{2}\tilde{B}_{k} \\ 0 &{} \quad -\sqrt{2}\tilde{C}_{k} &{}\quad \tilde{D}_{k} \end{pmatrix}, \; \text {and}\; \nonumber \\&\qquad \mathbf{U}_{\pi /4} = \begin{pmatrix} \frac{1}{ \sqrt{2} } &{} \quad - \frac{1}{ \sqrt{2} } &{}\quad 0 \\ \frac{1}{ \sqrt{2} } &{}\quad \frac{1}{ \sqrt{2} } &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad 1 \end{pmatrix}. \end{aligned}$$

(16)

We should point out that the $\sqrt{2}$ factor will be absorbed in the $\tilde{B}$ and $\tilde{C}$ dimensionless free parameters for the quark and lepton sectors, respectively. In addition, notice that for the $\mathbf{m}_{\mathrm{u}}$ mass matrix given in Eq. (16), $\tilde{C}_{\mathrm{u}}=\tilde{B}_{\mathrm{u}}$, according to Eq. (15). In the neutrino sector, the degeneracy, $h^{0 \mathrm{u}}_{2}=h^{0 \mathrm{u}}_{1}\equiv h^{0 \mathrm{u}}$, in the effective mass matrix given in Eq. (13) substantially reduces the free parameters when two RHNs are degenerate; this is not true for the non-degenerate case where this assumption does not modify the functional structure of the effective neutrino mass matrix given in Eq. (13), because this degeneracy only simplifies a little the form of the entries of the matrix $\mathbf{m}_{\nu }$; for more details see Appendix B.

Let us add a comment on the degeneracy on the vacuum expectation values. As we have remarked, the conditions $h^{0 \mathrm{u}}_{2}=h^{0 \mathrm{u}}_{1}\equiv h^{0 \mathrm{u}}$ and $h^{0 \mathrm{d}}_{2}=h^{0 \mathrm{d}}_{1}\equiv h^{0 \mathrm{d}}$ have been assumed so far, it is not clear yet that these relations will arise in a natural way upon minimization of the scalar potential. We expect that it will be the case and the study of the full scalar potential may be done along the lines given in [13, 16].

3.2 Quark and lepton mixings

We will describe briefly how to diagonalize the mass matrices, $\mathbf{m}_{j}$ and $\mathbf{m}_{\mathrm{u}}$ $(j=d,\ell )$, respectively. Let us first start with the down quark and charged lepton mass matrices which have the NNI textures; we will not enter in great detail since these kind of matrices have been well studied in [10, 11]. The above mentioned description is applied to the $\mathbf{m}_{\mathrm{u}}$ mass matrix where the Fritzsch texture [2, 3, 7] is present. For a pedagogical method to diagonalize these mass matrices see [48].

Going back to the expression $\tilde{\mathbf{M}}_{k}=\mathbf{u}^{\dagger }_{k \mathrm{R}} \mathbf{m}_{k} \mathbf{u}_{k \mathrm{L}}$, we are interested in obtaining the $\mathbf{u}_{k \mathrm{L}}$ left-handed matrices that appear in the $CKM$ matrix, and for this we must build the bilinear form ${\tilde{\mathbf{M}}}^{\dagger }_{k} {\tilde{\mathbf{M}}}_{k} = \mathbf{u}^{\dagger }_{k \mathrm{L}} \mathbf{m}^{\dagger }_{k} \mathbf{m }_{k} \mathbf{u}_{k \mathrm{L}}$. From this relation, we can factorize the CP phases that come from $\mathbf{m }^{\dagger }_{k} \mathbf{m }_{k} = \mathbf{Q}_{k} ( \mathbf{m}^{\dagger }_{k} \mathbf{m}_{k} ) \mathbf{Q}^{\dagger }_{k}$, see [61], such that

$$\begin{aligned} \mathbf{Q}_{k} =&\, \text {diag} ( 1, \mathrm{e}^{-i \eta _{k_{2}}}, \mathrm{e}^{-i \eta _{k_{3}}} ) \quad \text { and} \nonumber \\&( \mathbf{m}^{\dagger }_{k} \mathbf{m}_{k} ) = \begin{pmatrix} \vert \tilde{A}_{k} \vert ^2 &{}\quad 0 &{}\quad \vert \tilde{A}_{k} \vert \vert \tilde{B}_{k} \vert \\ 0 &{}\quad \vert \tilde{A}_{k} \vert ^2 + \vert \tilde{C}_{k} \vert ^2 &{}\quad \vert \tilde{C}_{k} \vert \vert \tilde{D}_{k} \vert \\ \vert \tilde{A}_{k} \vert \vert \tilde{B}_{k} \vert &{}\quad \vert \tilde{C}_{k} \vert \vert \tilde{D}_{k} \vert &{}\quad \vert \tilde{B}_{k} \vert ^2 +\vert \tilde{D}_{k} \vert ^2 \end{pmatrix}; \end{aligned}$$

(17)

we should keep in mind that the $\vert \tilde{A}_{k} \vert$, the $\vert \tilde{B}_{k} \vert$, $\vert \tilde{C}_{k} \vert$, and the $\vert \tilde{D}_{k} \vert$ free parameters are real and dimensionless, and that for the up-quark mass matrix we see that $\vert \tilde{B}_{\mathrm{u}} \vert = \vert \tilde{C}_{\mathrm{u}} \vert$.

Having factorized out the phases associated with CP violation in the bilinear form, we choose $\mathbf{u}_{k \mathrm{L}} = \mathbf{Q}_{k} \mathbf{O}_{k \mathrm{L}}$, where $\mathbf{O}_{k \mathrm{L}}$ is the orthogonal matrix that diagonalizes to the $( \mathbf{m}^{\dagger }_{k} \mathbf{m }_{k} )$ matrix. The matrix $\mathbf{O}_{k \mathrm{L}}$ is given by

$$\begin{aligned} \mathbf{O}_{k \mathrm{L}}= ( \mid f_{1} \rangle , \mid f_{2} \rangle , \mid f_{3} \rangle ), \end{aligned}$$

(18)

where the three eigenvectors have the following form:

$$\begin{aligned}&\mid f_{i} \rangle = N_{f_{i}} \begin{pmatrix} ( \tilde{m}^{2}_{ k_{i} }- \vert \tilde{A}_{k} \vert ^2 - \vert \tilde{C}_{k} \vert ^2 ) \vert \tilde{A}_{k} \vert \vert \tilde{B}_{k} \vert \\ ( \tilde{m}^{2}_{k_{i}} - \vert \tilde{A}_{k} \vert ^2 ) \vert \tilde{C}_{k} \vert \vert \tilde{D}_{k} \vert \\ ( \tilde{m}^{2}_{k_{i}} - \vert \tilde{A}_{k} \vert ^2 ) ( \tilde{m}^{2}_{k_{i}} - \vert \tilde{A}_{k} \vert ^2 - \vert \tilde{C}_{k} \vert ^2 ) \end{pmatrix},\nonumber \\&\mid f_{3} \rangle = N_{f_{3}} \begin{pmatrix} ( 1-\vert \tilde{A}_{k} \vert ^2-\vert \tilde{C}_{k} \vert ^2 ) \vert \tilde{A}_{k} \vert \vert \tilde{B}_{k} \vert \\ ( 1 -\vert \tilde{A}_{k} \vert ^2 ) \vert \tilde{C}_{k} \vert \vert \tilde{D}_{k} \vert \\ ( 1 - \vert \tilde{A}_{k} \vert ^2 ) ( 1 - \vert \tilde{A}_{k} \vert ^2 - \vert \tilde{C}_{k} \vert ^2 ) \end{pmatrix}. \end{aligned}$$

(19)

Here, $N_{f_{i}}$ ($i=1,2$) and $N_{f_{3}}$ stand for the normalization factors whose definition must be read directly from the above expression.

On the other hand, for the down quarks and charged leptons, three free parameters can be fixed in terms of the physical masses and $\vert \tilde{D}_{j} \vert \equiv y_{j}$, with $j = d, \ell$ [10, 11]. Explicitly, these are given by

$$\begin{aligned}&\vert \tilde{A}_{j} \vert = \frac{ q_{j} }{ y_{j} }, \, \vert \tilde{B}_{j} \vert = \sqrt{ \frac{ 1 + P_{j} - y^{4}_{j} - R_{j} }{2} - \left( \frac{ q_{j} }{ y_{j} } \right) ^{2} }, \nonumber \\&\qquad \vert \tilde{C}_{j} \vert = \sqrt{ \frac{ 1 + P_{j} - y^{4}_{j} + R_{j} }{2} - \left( \frac{ q_{j} }{ y_{j} } \right) ^{2}}, \end{aligned}$$

(20)

where

$$\begin{aligned}&P_{j} = \tilde{m}_{ j_{1} }^{2} + \tilde{m}_{ j_{2} }^{2}, \, q_{j} = \root 4 \of { \tilde{m}_{ j_{1} }^{2} \tilde{m}_{ j_{2} }^{2} },\nonumber \\&\qquad R_{j} = \sqrt{( 1 + P_{j} - y^{4}_{j} )^{2} - 4 ( P_{j} + q^{4}_{j} ) + 8 q^{2}_{j} y^{2}_{j} }. \end{aligned}$$

(21)

In the above expressions there is only one free parameter which is $y_{j}$. This parameter should be tuned in order to get reliable mixing matrices as we will see later. So far, we have found the $\mathbf{U}_{j,\mathrm{L}}$ left-handed matrices that diagonalize the $\mathbf{M}_{j}$ mass matrices which have the NNI textures. Let us now focus on $\mathbf{m}_{\mathrm{u}}$. Going back to Eq. (17), we must remember that $\vert \tilde{B}_{\mathrm{u}} \vert = \vert \tilde{C}_{\mathrm{u}} \vert$; then one can determine the three free parameters in terms of the physical masses. Explicitly, we obtain

$$\begin{aligned}&\vert \tilde{A}_{\mathrm{u}} \vert = \sqrt{ \dfrac{\tilde{m}_{\mathrm{u}} \tilde{m}_{\mathrm{c}} }{ 1 - \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}} } },\nonumber \\&\vert \tilde{B}_{\mathrm{u}} \vert = \sqrt{ \dfrac{ ( 1 - \tilde{m}_{\mathrm{c}} ) ( 1 + \tilde{m}_{\mathrm{u}} )( \tilde{m}_{\mathrm{c}} - \tilde{m}_{\mathrm{u}} ) }{ 1 - \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}} } } ,\nonumber \\&\vert \tilde{D}_{\mathrm{u}} \vert = 1 - \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}}. \end{aligned}$$

(22)

Following the same procedure, the $\mathbf{O}_{\mathrm{u} \mathrm{L}}$ orthogonal matrix that diagonalizes $(\mathbf{m}^{\dagger }_{\mathrm{u}} \mathbf{m}_{\mathrm{u}})$ is fixed in terms of above parameters. Thus, using the expression given in Eq. (19), we get

$$\begin{aligned} \mathbf{O}_{\mathrm{u L}}= \begin{pmatrix} - \sqrt{ \frac{ \tilde{m}_{\mathrm{c}} ( 1 - \tilde{m}_{\mathrm{c}} ) }{ ( 1 - \tilde{m}_{\mathrm{u}} ) ( \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}} ) \mathcal{G}_{\mathrm{u}} } } &{} -\sqrt{ \frac{ \tilde{m}_{\mathrm{u}} ( 1 + \tilde{m}_{\mathrm{u}} ) }{ ( 1 + \tilde{m}_{\mathrm{c}} ) ( \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}} ) \mathcal{G}_{\mathrm{u}} } } &{} \sqrt{ \frac{ \tilde{m}_{\mathrm{u}} \tilde{m}_{\mathrm{c}} ( \tilde{m}_{\mathrm{c}} - \tilde{m}_{\mathrm{u}}) }{( 1 - \tilde{m}_{\mathrm{u}} )( 1 + \tilde{m}_{\mathrm{c}} ) \mathcal{G}_{\mathrm{u}} } }\\ -\sqrt{ \frac{ \tilde{m}_{\mathrm{u}} ( 1 - \tilde{m}_{\mathrm{c}} ) }{ ( 1 - \tilde{m}_{\mathrm{u}} ) ( \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}} ) } } &{} \sqrt{ \frac{ \tilde{m}_{\mathrm{c}} ( 1 + \tilde{m}_{\mathrm{u}} ) }{ ( 1 + \tilde{m}_{\mathrm{c}} ) ( \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}} ) } } &{} \sqrt{ \frac{ ( \tilde{m}_{\mathrm{c}} - \tilde{m}_{\mathrm{u}} ) }{ ( 1 - \tilde{m}_{\mathrm{u}} ) ( 1 + \tilde{m}_{\mathrm{c}} ) } } \\ \sqrt{ \frac{ \tilde{m}_{\mathrm{u}} ( 1 + \tilde{m}_{\mathrm{u}} )( \tilde{m}_{\mathrm{c}} - \tilde{m}_{\mathrm{u}} ) }{ ( 1 - \tilde{m}_{\mathrm{u}} )( \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}} ) \mathcal{G}_{\mathrm{u}} } } &{} - \sqrt{ \frac{ \tilde{m}_{\mathrm{c}} ( 1 - \tilde{m}_{\mathrm{c}} ) ( \tilde{m}_{\mathrm{c}} - \tilde{m}_{\mathrm{u}} ) }{ ( 1 + \tilde{m}_{\mathrm{c}} ) ( \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}} ) \mathcal{G}_{\mathrm{u}} } } &{} \sqrt{ \frac{ ( 1 + \tilde{m}_{\mathrm{u}} )( 1 - \tilde{m}_{\mathrm{c}} ) }{ ( 1 - \tilde{m}_{\mathrm{u}} )( 1 + \tilde{m}_{\mathrm{c}} ) \mathcal{G}_{\mathrm{u}} } } \end{pmatrix}, \end{aligned}$$

(23)

where $\mathcal{G}_{\mathrm{u}} \equiv ( 1 - \tilde{m}_{\mathrm{c}} + \tilde{m}_{\mathrm{u}})$. From the expressions in Eq. (22) we see that the real orthogonal matrix $\mathbf{O}_{\mathrm{u L}}$ does not have free parameters, since this only depends on the up-quark mass ratios.

Therefore, the full left-handed unitary matrices that diagonalize the charged lepton, down-, and up-quark mass matrices are given by

$$\begin{aligned} \mathbf{U}_{k \mathrm{L}} = \mathbf{U}_{\pi /4} \mathbf{u}_{k \mathrm{L}} = \mathbf{U}_{\pi /4} \mathbf{Q}_{k} \mathbf{O}_{k \mathrm{L}}, \quad k = u, d, \ell . \end{aligned}$$

(24)

Thus, the CKM mixing matrix may be completely determined and given by

$$\begin{aligned} \mathbf{V}_{\mathrm{CKM}}= \mathbf{U}^{\dagger }_{\mathrm{u L}} \mathbf{U}_{\mathrm{d L}}= & {} ( \mathbf{U}_{\pi /4} \mathbf{Q}_{\mathrm{u}} \mathbf{O}_{\mathrm{u L}})^{\dagger } \mathbf{U}_{\pi /4} \mathbf{Q}_{\mathrm{d}} \mathbf{O}_{\mathrm{d L}} \nonumber \\= & {} \mathbf{O}^{\mathrm{T}}_{\mathrm{u L}} \mathbf{Q}_{\mathrm{q}} \mathbf{O}_{\mathrm{d L}}, \end{aligned}$$

(25)

where we have defined $\mathbf{Q}_{\mathrm{q}} \equiv \mathbf{Q}^{\dagger }_{\mathrm{u}} \mathbf{Q}_{\mathrm{d}} = \text {diag} ( 1, \, \mathrm{e}^{ i\alpha }, \, \mathrm{e}^{i \beta } )$ with the phase factors $\alpha = \eta _{\mathrm{u}_{2}} - \eta _{\mathrm{d}_{2}}$ and $\beta = \eta _{\mathrm{u}_{3}} - \eta _{\mathrm{d}_{3}}$, which come from the quark mass matrices. These two phases can be related with the unique phase $\delta _{\mathrm{KM}}$ of the angle–phase parametrization used in the PDG [62] by means of the expression $\sin \delta _{\mathrm{KM}} = \mathcal{J}_{\mathrm{q}} ( 1 -| V_{\mathrm{ub}} |^{2})/ | V_{\mathrm{ud}} | | V_{\mathrm{tb}} | | V_{\mathrm{us}} | | V_{\mathrm{cb}} | | V_{\mathrm{ub}} |$ where $\mathcal{J}_{\mathrm{q}}$ is the Jarlskog invariant. Also, the rotation matrix $\mathbf{U}_{\pi /4}$ is unobservable in the quark flavor mixings. In this way, the quark mixing matrix $V_{\mathrm{CKM}}$ has only three free parameters, which are $y_{\mathrm{d}}$, $\alpha$, and $\beta$, since the quark mass ratios are not treated as free parameters because we allow their values to vary within the experimental measurements region reported by PDG [62]. In addition the CKM matrix can be obtained analytically or numerically, however, we are now just interested in getting a numerical expression for it which will be done in the next section.

Now, in the leptonic sector the flavor mixing matrix PMNS is defined as [56]

$$\begin{aligned} \mathbf{V}_{\mathrm{PMNS}} = \mathbf{U}^{\dagger }_{\ell \mathrm{L}} \mathbf{U}_{\nu } \mathbf{K}. \end{aligned}$$

(26)

Here, the unitary matrix of charge leptons $\mathbf{U}_{\ell \mathrm{L}}$ is written as $\mathbf{U}_{\ell \mathrm{L}} = \mathbf{U}_{\pi /4} \mathbf{Q}_{\ell } \mathbf{O}_{\ell \mathrm{L}}$. The explicit form of unitary matrix of neutrinos, $\mathbf{U}_{\nu }$, will be obtained in the next sections. We should point out that we will neglect the $\mathbf{K}$ Majorana CP phases, which are unobservable in the magnitudes of entries of the leptonic mixing matrix.

Before diagonalizing the effective neutrino mass matrix, let us show the set of neutrino observables which is considered along the analytic and numerical analysis [63, 64]. This is given below

$$\begin{aligned}&\Delta m^{2}_{\odot } ( 10^{-5}\, \text {eV}^{2} ) = m^{2}_{ \nu _{2} } - m^{2}_{ \nu _{1} } = 7.59^{+0.20}_{-0.18} \nonumber \\&\Delta m^{2}_{\text {ATM}} ( 10^{-3}\, \text {eV}^{2} ) = | m^{2}_{ \nu _{3} } - m^{2}_{ \nu _{1} } | \nonumber \\&\quad = 2.50^{+0.09}_{-0.16} [ -2.40^{+0.08}_{-0.09} ] \nonumber \\&\sin ^{2}\theta _{12}^{\ell ^{\mathrm{ex}}} = 0.312^{+0.017}_{-0.015} \nonumber \\&\sin ^{2}\theta _{23}^{\ell ^{ex}} = 0.52^{+0.06}_{-0.07} [ 0.52 \pm 0.06 ] \nonumber \\&\sin ^{2}\theta _{13}^{\ell ^{\mathrm{ex}}} = 0.013^{+0.007}_{-0.005} [ 0.016^{+0.008}_{-0.006} ] \nonumber \\&\sin ^{2}2\theta _{13}^{\ell ^{\mathrm{ex}}} = 0.076 \pm 0.068 \quad ( \text {MINOS} ). \end{aligned}$$

(27)

Here, the data appearing in square parentheses stand for the inverted case. Leaving aside the experimental results, we focus on diagonalize the neutrino mass matrix $\mathbf{m}_{\nu }$. For this purpose in this paper we will consider two cases: (a) the first two masses of the RHNs are degenerate. (b) the RHNs masses are not degenerate.

In the first case we can only obtain a lower bound for the value of the reactor angle [50]. But it is important to consider the case when $M_{\mathrm{R}_{1}} = M_{\mathrm{R}_{2}}$, because it gives us an idea of which are the allowed values for the free parameters present in the leptonic mixing matrix.

3.2.1 The masses of the right-handed neutrinos with degeneration

From the expressions in Eqs. (66) and (67), given in Appendix B, it is very easy to see that when the masses of the first two RHNs are degenerate, $M_{\mathrm{R}_{1}} = M_{\mathrm{R}_{2}}$, the effective neutrino mass matrix $\mathbf{m}_{\nu }$ is reduced to a block matrix, as shown below:

$$\begin{aligned}&\mathbf{m}_{\nu } = \mathbf{\textit{u} }^{\mathrm{T}}_{\theta =\pi /4} \mathbf{M}_{\nu } \mathbf{\textit{u}}_{\theta =\pi /4} = \begin{pmatrix} A^{2}_{\nu } + 2 B^{2}_{\nu } &{}\quad \sqrt{2}A_{\nu } C_{\nu } &{}\quad 0 \\ \sqrt{2} A_{\nu } C_{\nu } &{}\quad 2 C^{2}_{\nu } &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad A^{2}_{\nu } \end{pmatrix}\nonumber \\&\qquad \text {and} \quad \mathbf{\textit{u} }_{\theta =\pi /4} = \begin{pmatrix} \frac{1}{\sqrt{2}} &{}\quad 0 &{} \quad -\frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} &{}\quad 0 &{}\quad \frac{1}{\sqrt{2}} \\ 0 &{}\quad 1 &{}\quad 0 \end{pmatrix}. \end{aligned}$$

(28)

The $\mathbf{m}_{\nu }$ block matrix can easily be diagonalized. First, let us factorize the CP phases of $\mathbf{m}_{\nu }$ [61]. Therefore, $\mathbf{m}_{\nu } = \mathbf{P}_{\nu } {\hat{\mathbf{m}} }_{\nu } \mathbf{P}_{\nu }$, where $\mathbf{P}_{\nu }$ and ${\hat{\mathbf{m}}}_{\nu }$ are given in Eq. (29). We can associate immediately $\vert A_{\nu } \vert ^2 =m_{\nu _{3}}$. Thus, we just have to diagonalize the left upper block of $\mathbf{m}_{\nu }$

$$\begin{aligned}&\mathbf{P}_{\nu } = \begin{pmatrix} \mathrm{e}^{i\eta _{\nu _{1}}} &{}\quad 0 &{}\quad 0 \\ 0 &{}\quad \mathrm{e}^{i\eta _{\nu _{2}}} &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad \mathrm{e}^{i\eta _{\nu _{1}}} \end{pmatrix} \, \text {and} \nonumber \\&\qquad {\hat{\mathbf{m}} }_{\nu }= \begin{pmatrix} \vert A_{\nu } \vert ^2 + 2 \vert B_{\nu } \vert ^2 &{}\quad \sqrt{2} \vert A_{\nu } \vert \vert C_{\nu } \vert &{}\quad 0 \\ \sqrt{2} \vert A_{\nu } \vert \vert C_{\nu } \vert &{}\quad 2 \vert C_{\nu } \vert ^2 &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad \vert A_{\nu }\vert ^2 \end{pmatrix}. \end{aligned}$$

(29)

A necessary condition for factorizing the phases in the above way is that the $A^{2}_{\nu }$ and $B^{2}_{\nu }$ phases be aligned, although they may be different in magnitude. On the other hand, we appropriately choose $\mathbf{u}_{\nu } = \mathbf{P}^{\dagger }_{\nu } \mathbf{O}_{\nu }$. Here, $\mathbf{O}_{\nu }$ is a real orthogonal matrix that diagonalizes the ${\hat{\mathbf{m}}}_{\nu }$ matrix. Using the left upper block of $\mathbf{m}_{\nu }$, we fix the $\vert B_{\nu } \vert ^2$ and $\vert C_{\nu }\vert ^2$ free parameters through the following equations:

$$\begin{aligned}&m_{\nu _{2}} + m_{\nu _{1}} = \vert A_{\nu } \vert ^2+ 2 \vert B_{\nu } \vert ^2 + 2 \vert C_{\nu } \vert ^2\,\, \text {and} \nonumber \\&\qquad \qquad m_{\nu _{2}} m_{\nu _{1}} = 4 \vert B_{\nu } \vert ^2 \vert C_{\nu } \vert ^2. \end{aligned}$$

(30)

Solving for the rest of the free parameters we find that

$$\begin{aligned}&\vert B_{\nu } \vert ^2_{\mp } = \frac{1}{4} ( m_{\nu _{2}} + m_{\nu _{1}} - m_{\nu _{3}} \mp R_{\nu } )\,\, \text {and} \nonumber \\&\qquad \qquad \vert C_{\nu } \vert ^2_{\pm } = \frac{1}{4} ( m_{\nu _{2}} + m_{\nu _{1}} - m_{\nu _{3}} \pm R_{\nu } ), \end{aligned}$$

(31)

where

$$\begin{aligned} R_{\nu } \equiv \sqrt{ ( m_{\nu _{2}} + m_{\nu _{1}} - m_{\nu _{3}} )^2 - 4 m_{\nu _{2}} m_{\nu _{1}}}. \end{aligned}$$

(32)

As we can observe, there are two solutions for $\vert B_{\nu } \vert ^2$ and $\vert C_{\nu } \vert ^2$, respectively. However, following a straightforward analysis, it is clear that one solution is discarded by demanding that two free parameters (for the normal and for the inverted hierarchy) should be real and positive definite since they come from a real symmetric matrix. As a result of this, we realize that the normal spectrum is ruled out. For the inverted case ($m_{\nu _{2}}>m_{\nu _{1}}>m_{\nu _{3}}$), $\vert B_{\nu } \vert ^2_{-}$, and $\vert C_{\nu } \vert ^2_{+}$ turn out to be real and positive, if and only if the $m_{\nu _{3}}$ lightest neutrino mass is very small. Actually, from the definition of $R_{\nu }$ we obtain the following sum rule:

$$\begin{aligned} m_{\nu _{3}} \le \big ( \sqrt{m_{\nu _{2}}} - \sqrt{m_{\nu _{1}}}\big )^2, \end{aligned}$$

(33)

where the equality in the above expression means an upper bound for the $m_{\nu _{3}}$ lightest mass. Having fixed $\vert B_{\nu } \vert ^2_{-}$ and $\vert C_{\nu } \vert ^2_{+}$ in terms of the physical neutrino masses, the $\mathbf{O}_{\nu }$ matrix is well determined by them. Explicitly, $\mathbf{O}_{\nu }$ is given by

$$\begin{aligned} \mathbf{O}_{\nu } = \begin{pmatrix} \sqrt{\dfrac{m_{\nu _{3}}(m_{\nu _{2}}+ m_{\nu _{1}}-m_{\nu _{3}} + R_{\nu } ) }{ ( m_{\nu _{2}} - m_{\nu _{1}} ) ( m_{\nu _{2}} - m_{\nu _{1}} + m_{\nu _{3}} - R_{\nu } ) }} &{}\quad \sqrt{ \dfrac{ m_{\nu _{3} } ( m_{\nu _{2}} + m_{\nu _{1}} - m_{\nu _{3}} + R_{\nu } ) }{ ( m_{\nu _{2}} - m_{\nu _{1}} ) ( m_{\nu _{2}} - m_{\nu _{1}} - m_{\nu _{3}} + R_{\nu })}} &{} 0 \\ - \sqrt{ \dfrac{ m_{\nu _{2} } - m_{\nu _{1}} + m_{\nu _{3}} - R_{\nu } }{ 2( m_{\nu _{2}} - m_{\nu _{1}} ) } } &{} \sqrt{ \dfrac{ m_{\nu _{2}} - m_{\nu _{1}} - m_{3\nu } + R_{\nu } }{2( m_{\nu _{2}} - m_{\nu _{1}} ) } } &{} 0 \\ 0 &{} 0 &{} 1 \end{pmatrix}. \end{aligned}$$

(34)

Therefore, the $\mathbf{M}_{\nu }$ neutrino mass matrix is diagonalized by the following unitary matrix:

$$\begin{aligned} \mathbf{U}_{\nu } = \mathbf{\textit{u} }_{\pi /4} \mathbf{P}^{\dagger }_{\nu } \mathbf{O}_{\nu }. \end{aligned}$$

(35)

Then one sees that the leptonic mixing matrix takes the form

$$\begin{aligned} \mathbf{V}_{\mathrm{PMNS}} = \mathbf{U}^{\dagger }_{\ell } \mathbf{U}_{\nu } = \mathbf{O}^{\mathrm{T}}_{\ell \mathrm{L}} \mathbf{Q}^{\dagger }_{\ell } \mathbf{S}_{23} \mathbf{P}^{\dagger }_{\nu } \mathbf{O}_{\nu }. \end{aligned}$$

(36)

Here, $\mathbf{S}_{23} = \mathbf{U}^{\mathrm{T}}_{ \pi /4} \mathbf{\textit{u} }_{\theta = \pi /4}$ is the permutation matrix which is an element of the $S_{3}$ family group; at the same time, it is an element of the $Q_{6}$ family since $S_{3}$ is a subgroup of it. Therefore, the PMNS mixing matrix has the following form:

$$\begin{aligned} \mathbf{V}_{\mathrm{PMNS}}= \begin{pmatrix} O_{11 \ell } O_{11 \nu } + O_{31\ell } O_{21\nu } \mathrm{e}^{ i \bar{\eta }_{3 { e}} } &{}\quad O_{11 \ell } O_{12 \nu } + O_{31\ell } O_{22\nu } \mathrm{e}^{ i \bar{\eta }_{3 { e}} } &{}\quad O_{21 \ell } \mathrm{e}^{ i\eta _{2 { e}} } \\ O_{12 \ell } O_{11 \nu } + O_{32 \ell } O_{21 \nu } \mathrm{e}^{ i \bar{\eta }_{3 { e}} } &{}\quad O_{12 \ell } O_{12 \nu } + O_{32\ell } O_{22\nu } \mathrm{e}^{ i \bar{\eta }_{3 { e}} } &{}\quad O_{22 \ell } \mathrm{e}^{i \eta _{2 { e}} } \\ O_{13 \ell } O_{11 \nu } + O_{33 \ell } O_{21 \nu } \mathrm{e}^{ i \bar{\eta }_{3 { e}} } &{}\quad O_{13 \ell } O_{12 \nu } + O_{33\ell } O_{22\nu } \mathrm{e}^{ i \bar{\eta }_{3 { e}} } &{}\quad O_{23\ell } \mathrm{e}^{ i \eta _{2 { e}} } , \end{pmatrix} \end{aligned}$$

(37)

where $\eta _{2 { e}}$ and $\bar{\eta }_{3 { e}}$ are phases that coming from lepton mass matrices. In this case the $V_{\mathrm{PMNS}}$ mixing matrix has three free parameters which are $\eta _{2 { e}}$, $\bar{\eta }_{3 { e}}$, $y_{l}$. The charged lepton mass ratios are given by experimental data [62], while the neutrino masses are determined by means of the sum rule, Eq. (33), and the neutrino mass squared splittings, Eq. (27). Comparing this matrix with the standard parametrization given in [65], we find that the reactor, the atmospheric, and the solar mixing angles are well determined as follows:

$$\begin{aligned}&\vert \sin \theta _{13}^{\ell ^{\mathrm{th}}} \vert = \vert O_{21\ell } \vert , \quad \vert \sin \theta _{23}^{\ell ^{\mathrm{th}}} \vert = \dfrac{ \vert O_{22\ell } \vert }{ \sqrt{ 1 - \vert O_{21\ell } \vert ^2} },\nonumber \\&\qquad \vert \tan \theta _{12}^{\ell ^{\mathrm{th}}} \vert ^2 = \dfrac{ \vert O_{11 \ell } O_{12 \nu } + O_{31 \ell } O_{22\nu } \mathrm{e}^{ i \bar{\eta }_{3 { e}} } \vert ^2 }{ \vert O_{11 \ell } O_{11 \nu } + O_{31 \ell } O_{21 \nu } \mathrm{e}^{ i \bar{\eta }_{3 { e}} } \vert ^2 }. \end{aligned}$$

(38)

Let us point out a remarkable coincidence between the above formulas and those showed in [48, 49], their functional behavior seems to be the same, at least. As we already commented briefly, it is not a surprise since the $Q_{6}$ family group is the double covering of the $S_{3}$ one, so that in this particular model the $\mathbf{S}_{23}$ presence in the leptonic sector is not simply a coincidence. Of course, we expect that our results turn out being different from the $S_3$ case, since the charged lepton and neutrino contributions are different in both models, as we will see next.

3.2.2 The masses of the right-handed neutrinos without degeneration

Now, we consider the case where the RHN masses are not degenerate. In this case we can have both hierarchies of the neutrino masses, as we will. The effective neutrino mass matrix $\mathbf{m}_{\nu }$ can be written in polar form as $P_{\nu } \bar{\mathbf{m}}_{\nu } P_{\nu }$, where $\bar{\mathbf{m}}_{\nu }$ is a symmetric real matrix and $P_{\nu } = \text {diag}( 1, \mathrm{e}^{i \alpha _{1} }, \mathrm{e}^{i \alpha _{2} } )$ is a diagonal matrix of phases with $2\alpha _{1} = \arg \{\mu _{0}\} - \arg \{b_{\nu }\}$, $2\alpha _{2} = \arg \{ d_{\nu } \} - \arg \{b_{\nu }\}$, $2\arg \{c_{\nu }\} = \arg \{b_{\nu }\} + \arg \{d_{\nu }\}$, and $2\arg \{a_{\nu }\} = \arg \{b_{\nu }\} + \arg \{\mu _{0}\}$ [50, 61]. The symmetric real matrix with one texture zero, $\bar{\mathbf{m}}_{\nu }$, can be expressed in terms of a matrix with two texture zeros class I as [50]

$$\begin{aligned} \bar{\mathbf{m}}_{\nu } = \mu _{0} \mathbb {I}_{3 \times 3} + \mathbf{M}'_{\nu } \end{aligned}$$

(39)

where the matrix $\mathbf{M}'_{\nu }$ written in terms of its eigenvalues, for a normal (inverted) hierarchy, is [60]

$$\begin{aligned} \widetilde{\mathbf{M}}'_{\nu } = \frac{ \mathbf{M}'_{\nu } }{ \sigma _{3[2]} } \left( \begin{array}{ccc} \widetilde{\sigma }_{1[3]} - \widetilde{\sigma }_{2[1]} + \delta _{\nu } &{} \sqrt{ \frac{ \widetilde{\sigma }_{1[3]} \widetilde{\sigma }_{2[1]} }{ 1 - \delta _{\nu } } } &{} \sqrt{ \frac{ \delta _{\nu } }{ 1 - \delta _{\nu } } f_{\nu 1[3] } f_{\nu 2[1] } } \\ \sqrt{ \frac{ \widetilde{\sigma }_{1[3]} \widetilde{\sigma }_{2[1]} }{ 1 - \delta _{\nu } } } &{} 0 &{} 0 \\ \sqrt{ \frac{ \delta _{\nu } }{ 1 - \delta _{\nu } } f_{\nu 1[3] } f_{\nu 2[1]} } &{} 0 &{} 1 - \delta _{\nu } \end{array} \right) , \end{aligned}$$

(40)

where

$$\begin{aligned}&f_{\nu 1[3]} = ( 1 - \widetilde{\sigma }_{1[3]} - \delta _{\nu } ), \quad f_{\nu 2[1]} = ( 1 + \widetilde{\sigma }_{2[1]} - \delta _{\nu } ),\nonumber \\&\widetilde{\sigma }_{1[3]} = \frac{ \widetilde{m}_{\nu _{1[3]}} - \widetilde{\mu } }{ 1 - \widetilde{\mu }}, \quad \widetilde{\sigma }_{2[1]} = \frac{ | \widetilde{m}_{\nu _{2[1]}} - \widetilde{\mu } | }{ 1 - \widetilde{\mu } } , \nonumber \\&\qquad \widetilde{m}_{\nu _{1[3]}} = \frac{ m_{\nu _{1[3]}} }{ m_{\nu _{3[2]}} }, \quad \widetilde{m}_{\nu _{2[1]}} = \frac{ m_{\nu _{2[1]}} }{ m_{\nu _{3[2]}} }, \quad \widetilde{\mu } = \frac{ |\mu _{0}| }{ m_{\nu _{3[2]}} }. \end{aligned}$$

(41)

From the expressions for the mass parameters, Eq. (41), we obtain the following constraint: $\widetilde{m}_{\nu _{1[3]}} > \widetilde{\mu }$. The parameter $\delta _{\nu }$ is defined as $\delta _{\nu } = 1 - \widetilde{d}_{\nu } + \widetilde{\mu }$ with $\widetilde{d}_{\nu } = | d_{\nu } |/\sigma _{3[2]}$, and has a range of values $1 - \widetilde{\sigma }_{\nu _{1[3]}} > \delta _{\nu } > 0$, which is equivalent to $m_{\nu _{3[2]}} > |d_{\nu }| > m_{\nu _{1[3]}}$. From the above it can be seen that strictly speaking $|d_{\nu }|$ is not a free parameter of the mass matrix $\mathbf{M}_{\nu }$ and, consequently, of $\mathbf{m}_{\nu }$, because it must meet the above condition. The numerical values consistent with this condition are determined from the experimental data on neutrino oscillations. Now, reparameterized in terms of its eigenvalues, the orthogonal real matrix that diagonalizes the mass matrix $\widetilde{\mathbf{M}}'_{\nu }$ (and thus $\bar{\mathbf{m}}_{\nu }$) is

$$\begin{aligned} \mathbf{O}_{\nu } = \left( \begin{array}{ccc} \sqrt{ \frac{ \widetilde{\sigma }_{1[3]} ( 1 - \delta _{\nu } ) f_{\nu 1[3]} }{ D_{\nu 1[3]} } } &{} \sqrt{ \frac{ \widetilde{\sigma }_{2[1]} ( 1 - \delta _{\nu } ) f_{\nu 2[1]} }{ D_{\nu 2[1]} } } &{} \sqrt{ \frac{ \delta _{\nu } ( 1 - \delta _{\nu } ) }{ D_{ \nu 3[2]} } } \\ \sqrt{ \frac{ \widetilde{\sigma }_{2[1]} f_{\nu 1[3]} }{ D_{\nu 1[3] } } } &{} -\sqrt{ \frac{ \widetilde{\sigma }_{1[3] } f_{\nu 2[1] } }{ D_{\nu 2[1] }} } &{} \sqrt{ \frac{ \widetilde{\sigma }_{1[3] } \widetilde{\sigma }_{2[1] } \delta _{\nu } }{ D_{\nu 3[2] } } } \\ -\sqrt{ \frac{ \widetilde{\sigma }_{1[3]} \delta _{\nu } f_{\nu 2[1] } }{ D_{\nu 1[3] } } } &{} -\sqrt{ \frac{ \widetilde{\sigma }_{2[1]} \delta _{\nu } f_{\nu 1[3] } }{ D_{\nu 2[1] } } } &{} \sqrt{ \frac{ f_{\nu 1[3] } f_{\nu 2[1] } }{ D_{\nu 3[2] } } } \end{array} \right) , \end{aligned}$$

(42)

where

$$\begin{aligned}&D_{\nu 1[3]} = ( 1 - \delta _{\nu } ) ( \widetilde{\sigma }_{1[3]} + \widetilde{\sigma }_{2[1]} ) ( 1 - \widetilde{\sigma }_{1[3]} ), \nonumber \\&D_{\nu 2[1]} = ( 1 - \delta _{\nu } ) ( \widetilde{\sigma }_{1[3]} + \widetilde{\sigma }_{2[1]} ) ( 1 + \widetilde{\sigma }_{2[1]} ), \nonumber \\&D_{\nu 3[2]} = ( 1 - \delta _{\nu } ) ( 1 - \widetilde{\sigma }_{1[3]} ) ( 1 + \widetilde{\sigma }_{2[1]} ). \end{aligned}$$

(43)

Then one sees that the leptonic mixing matrix takes the form

$$\begin{aligned} \mathbf{V}_{\mathrm{PMNS}} = \mathbf{O}^{\mathrm{T}}_{\ell \mathrm{L}} \mathbf{Q}^{\dagger }_{\ell } \mathbf{S}_{23} \mathbf{P}^{\dagger }_{\nu } \mathbf{O}_{\nu }. \end{aligned}$$

(44)

Here $\mathbf{S}_{23} = \mathbf{U}^{\mathrm{T}}_{ \pi /4} \mathbf{\textit{u} }_{\theta }$ is not the permutation matrix show in the previous section; its explicit form is

$$\begin{aligned} \mathbf{S}_{23} = \left( \begin{array}{c@{\quad }c@{\quad }c} S_{\theta + \frac{\pi }{4} } &{} 0 &{} -C_{\theta + \frac{\pi }{4} } \\ C_{\theta + \frac{\pi }{4} } &{} 0 &{} S_{\theta + \frac{\pi }{4} } \\ 0 &{} 1 &{} 0 \end{array} \right) , \end{aligned}$$

(45)

where $S_{\theta + \frac{\pi }{4}} = \sin ( \theta + \frac{\pi }{4} )$ and $C_{\theta + \frac{\pi }{4}} = -\cos ( \theta + \frac{\pi }{4} )$, with $\theta = \arctan \{ \frac{ M_{\mathrm{R}_{2}} }{ M_{\mathrm{R}_{1}} } \}$. Thus, the parameter $\theta$ measures the degeneracy between the first two right-handed neutrino masses, and may give us a hint about the hierarchy that obeys the mass spectrum of right-handed neutrinos. Therefore, the PMNS mixing matrix has the following form:

$$\begin{aligned} \mathbf{V}_{\mathrm{PMNS}} = \begin{pmatrix} V_{e1} &{}\quad V_{e2} &{}\quad V_{e3} \\ V_{\mu 1} &{}\quad V_{\mu 2} &{}\quad V_{\mu 3} \\ V_{\tau 1} &{}\quad V_{\tau 2} &{}\quad V_{\tau 3} \end{pmatrix}, \end{aligned}$$

(46)

where

$$\begin{aligned} V_{e1}= & {} ( O_{11\ell } S_{\frac{\pi }{4} +\theta } + O_{21\ell } C_{\frac{\pi }{4} + \theta } \mathrm{e}^{i\eta _{1}} ) O_{11\nu } + O_{31 \ell } O_{21 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{21\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{11\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{31\nu } \nonumber \\ V_{e2}= & {} ( O_{11\ell } S_{\frac{\pi }{4} +\theta } + O_{21\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{1}} ) O_{12\nu } + O_{31 \ell } O_{22 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{21\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{11\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{32\nu } \nonumber \\ V_{e3}= & {} ( O_{11\ell } S_{\frac{\pi }{4} +\theta } + O_{21\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{1}} ) O_{13\nu } + O_{31 \ell } O_{23 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{21\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{11\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{33\nu } \nonumber \\ V_{\mu 1}= & {} ( O_{12\ell } S_{\frac{\pi }{4} +\theta } + O_{22\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{1}} ) O_{11\nu } + O_{32 \ell } O_{21 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{22\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{12\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{31\nu } \nonumber \\ V_{\mu 2}= & {} ( O_{12\ell } S_{\frac{\pi }{4} +\theta } + O_{22\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{1}} ) O_{12\nu } + O_{32 \ell } O_{22 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{22\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{12\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{32\nu } \nonumber \\ V_{\mu 3}= & {} ( O_{12\ell } S_{\frac{\pi }{4} +\theta } + O_{22\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{1}} ) O_{13\nu } + O_{32 \ell } O_{23 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{22\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{12\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{33\nu } \nonumber \\ V_{\tau 1}= & {} ( O_{13\ell } S_{\frac{\pi }{4} +\theta } + O_{23\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{1}} ) O_{11\nu } + O_{33 \ell } O_{21 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{23\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{13\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{31\nu } \nonumber \\ V_{\tau 2}= & {} ( O_{13\ell } S_{\frac{\pi }{4} +\theta } + O_{23\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{1}} ) O_{12\nu } + O_{33 \ell } O_{22 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{23\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{13\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{32\nu } \nonumber \\ V_{\tau 3}= & {} ( O_{13\ell } S_{\frac{\pi }{4} +\theta } + O_{23\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{1}} ) O_{13\nu } + O_{33 \ell } O_{23 \nu } \mathrm{e}^{i\bar{ \eta }_{3e} } \nonumber \\&+\, ( O_{23\ell } S_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\eta _{e2}} - O_{13\ell } C_{\frac{\pi }{4} +\theta } \mathrm{e}^{i\alpha _{2}} ) O_{33\nu } \end{aligned}$$

(47)

with $\eta _{2e}= \eta _{1} + \alpha _{2}$ and $\bar{\eta }_{3e} = \eta _{2} + \alpha _{1}$. Now, for this case the $V_{\mathrm{PMNS}}$ mixing matrix has nine free parameters which are $y_{\ell }$, $\theta$, $\eta _{1}$, $\alpha _{2}$, $\eta _{2e}$, $\bar{\eta }_{3e}$, $m_{\nu _{3[2]}}$, $\widetilde{\mu }_{0}$, and $\delta _{\nu }$. From the above expressions it is easy to see that the case analyzed in the previous section is a particular case of the one discussed in this section. In other words, if $\theta = \pi /4$ we have $\tan \theta = 1$, which implies $M_{R_{1}} = M_{R_{2}}$.

4 Numerical analysis for the mixing matrices

4.1 CKM mixing matrix

The CKM matrix is defined as $\mathbf{V}_{\mathrm{CKM}}= \mathbf{U}^{\dagger }_{\mathrm{u L}} \mathbf{U}_{\mathrm{d L}} = \mathbf{O}^{T}_{\mathrm{u L}} \mathbf{Q}_{\mathrm{q}} \mathbf{O}_{\mathrm{d L}}$, with the form of the mass matrices found in the previous sections. So far, there are three free parameters ($y_{\mathrm{d}}$ and two CP-violating phases in $\mathbf{Q}_{\mathrm{q}}$) if the quark mass ratios are taken as inputs. As is well known, the physical masses depend on the scale at which they are measured, in this model the CKM matrix may be obtained numerically with masses at the GUT scale. However, the mass ratios do not change drastically at different energy scales as one can verify directly from [66]. Therefore, we will assume that the form of the mass matrices will remain the same from the GUT scale to the electroweak scale. Of course, in the more detailed analysis that is currently in progress, the effects of the extra Higgs fields in the model and the running of the renormalization group will be taken into account. Thus, for the rest of the analysis we will assume we are already at the electroweak scale. At low energies, we have used the following values for the quark mass ratios given in [60, 67]:

$$\begin{aligned}&m_\mathrm{u}/m_{\mathrm{t}} = (1.73 \pm 0.75) \times 10^{-5}, \nonumber \\&\qquad m_\mathrm{c}/m_{\mathrm{t}} = (3.46 \pm 0.43) \times 10^{-3}, \nonumber \\&m_\mathrm{d}/m_{\mathrm{b}} = (1.12 \pm 0.007) \times 10^{-3}, \nonumber \\&\qquad m_{\mathrm{s}}/m_{\mathrm{b}}= (2.32 \pm 0.84) \times 10^{-2}. \end{aligned}$$

(48)

In order to obtain the numerical values for the three free parameters, we perform a $\chi ^{2}$ analysis on the parameter space, to find their best fit points. It is built as in [60, 67]

$$\begin{aligned} \chi ^2= & {} \dfrac{ ( |V^{\mathrm{th}}_{\mathrm{ud}}| - |V^{\mathrm{ex}}_{\mathrm{ud}}|)^2 }{ \sigma ^{2}_{V_{\mathrm{ud}}} } + \dfrac{ ( |V^{\mathrm{th}}_{\mathrm{us}}| - |V^{\mathrm{ex}}_{\mathrm{us}}| )^2 }{ \sigma ^{2}_{V_{\mathrm{us}} } } \nonumber \\&+\, \dfrac{ ( |V^{\mathrm{th}}_{\mathrm{ub}}| - |V^{\mathrm{ex}}_{\mathrm{ub}}| )^2 }{ \sigma ^{2}_{V_{\mathrm{ub}}} } + \dfrac{ ( \mathcal {J}^{\mathrm{th}}_{\mathrm{q}} - \mathcal {J}^{\mathrm{ex}}_{\mathrm{q}})^2 }{ \sigma ^{2}_{\mathcal {J}_{\mathrm{q}}}} \end{aligned}$$

(49)

where we have taken the following experimental values for the $V_{\mathrm{CKM}}$ elements we used to construct the $\chi ^2$ function [65]:

$$\begin{aligned}&| V^{\mathrm{ex}}_{\mathrm{ud}} | = 0.97427 \pm 0.00015, \nonumber \\&\quad | V^{\mathrm{ex}}_{\mathrm{us}} | = 0.2253 \pm 0.007, \nonumber \\&| V^{\mathrm{ex}}_{\mathrm{ub}} | = 0.00351 \pm 0.00015, \nonumber \\&\quad \mathcal {J}^{\mathrm{ex}}_{\mathrm{q}} = (2.96 \pm 0.18) \times 10^{-5}. \end{aligned}$$

(50)

Notice that using the Jarlskog invariant in the $\chi ^2$ function implies unitarity as a constraint. The best values for the free parameters are thus found to be

$$\begin{aligned}&y_{\mathrm{d}} = 0.981977^{+0.002843 }_{-0.002117}, \quad \alpha = ( 168.78^{+2.10}_{-1.11} )^{\circ }, \nonumber \\&\qquad \beta = ( 132^{+105}_{-25} )^{\circ }, \end{aligned}$$

(51)

at 70 % C. L with $\chi ^{2} = 0.0515$ as the minimal value, see Fig. 1. These correspond to the following values for the $V_{\mathrm{CKM}}$ elements:

$$\begin{aligned} |V^{\mathrm{th}}_{\mathrm{ud}}|= & {} 0.97428^{+0.00016}_{-0.00014},\quad |V^{\mathrm{th}}_{\mathrm{us}}| = 0.2252^{+0.00062}_{-0.00067} \nonumber \\ |V^{\mathrm{th}}_{\mathrm{ub}}|= & {} 0.00351^{+0.00017}_{-0.00016}, \quad \mathcal {J}_{\mathrm{q}} = 2.95^{+0.20}_{-0.19} \times 10^{-5}. \end{aligned}$$

(52)

Fig. 1

Allowed region for the three free parameters in the quark sector at $70~\%$ (green) and $90~\%$ (orange) confidence level

4.2 PMNS mixing matrix

4.2.1 The masses of right-handed neutrinos with degeneration

As we observe from Eq. (38), the reactor and atmospheric angles turn out to be independent of the neutrino masses. These observables only depend explicitly on the $y_{\ell }$ free parameter, the charged lepton masses and the $\eta _{2 e}$ Dirac phase; the latter may be ignored since we are only interested in the absolute values of the two mixing angles. On the other hand, the solar mixing angle depends on the $y_{\ell }$ free parameter, the $\bar{\eta }_{3e}$ phase, as well as the charged lepton and neutrino masses.

In order to show that in this model we can describe the lepton masses and mixing we will also make a $\chi ^{2}$ fit using the theoretical expressions for the atmospheric and reactor angles given in Eq. (38) and compare them with the current experimental data for these mixing angles. Thus, for the atmospheric mixing angle we consider the following experimental value: $\sin ^2\theta ^{\ell ^{\mathrm{ex}}}_{23}= 0.52 \pm 0.06$. Within this theoretical framework we will consider first the particular case when the first and second right-handed neutrinos are mass degenerate. In this case we can only determine a lower bound for the value of the reactor angle [50]. To perform the $\chi ^{2}$ fit we considered the values for the reactor angle reported by the MINOS experiment: $\sin ^2 2\theta ^{\ell ^{\mathrm{ex}}}_{13}=0.076\pm 0.068$. We considered also the following charged lepton masses values: $m_{\mathrm{e}}=0.51099$ MeV, $m_{\mu }= 105.6583$ MeV and $m_{\tau }=1776.82$ MeV [62]. As a result of this $\chi ^{2}$ analysis, we found that for the best fit with $\chi ^2=0.85$ as the minimum value, the free parameter $y_{\ell }$ at $1\sigma$ has the following range of values: $y_{\ell }= 0.8478^{+0,0045}_{-0.0046}$. Moreover, the best theoretical values for the atmospheric and reactor angles that come from this analysis, at $1\sigma$, are:

$$\begin{aligned}&\sin ^{2} \theta ^{\ell ^{\mathrm{th}}}_{23} = 0.5206_{-0.0113}^{+0.0115} \rightarrow \theta ^{\ell ^{\mathrm{th}}}_{23} = ( 46.18^{+0.66}_{-0.65} )^{\circ }, \nonumber \\&\sin ^{2} 2 \theta ^{\ell ^{\mathrm{th}}}_{13} = 0.01386_{-0.00025}^{+0.00016} \rightarrow \theta ^{\ell ^{\mathrm{th}}}_{13} = ( 3.38^{+0.03}_{-0.02} )^{\circ }. \end{aligned}$$

(53)

As can be observed, the atmospheric angle value is in good agreement with the experimental values, however, the obtained reactor mixing value is smaller than the central value obtained in the global fits. In fact, this theoretical value of the reactor angle is more than $3 \sigma$ away from the values reported in Refs. [68, 69] (Fig. 2). But it is worth noting that our model predicts that the reactor angle must be different from zero and that is large in comparison to the one obtained from the tribimaximal mixing matrix. Moreover, as already stated above, it is a lower bound for the value obtained in a more general model where the right-handed neutrino masses are not degenerate. In Fig. 2, we show the dependence of the atmospheric and reactor angles on the parameter $y_{\ell }$.

Fig. 2

Atmospheric and reactor mixing angle values at $65~\%$ (orange) and $95~\%$ (yellow) of C.L.

Having determined the allowed values for the $y_{\ell }$ free parameter, the solar mixing angle just depends on the neutrino masses and one Dirac phase. We can reduce further the free parameters, noticing that the neutrino masses can be determined using the sum rule given in Eq. (33). This is written in terms of the observables $\Delta m^{2}_{\odot }$ and $\Delta m^{2}_{\mathrm{ATM}}$ as

$$\begin{aligned} m_{\nu _{3}} \le \left( \root 4 \of { m^{2}_{\nu _{3}} + \Delta m^{2}_{\odot } + \Delta m^{2}_{ \text {ATM} } } - \root 4 \of { m^{2}_{\nu _{3}} + \Delta m^{2}_{ \text {ATM} }} \right) ^2.\nonumber \\ \end{aligned}$$

(54)

From the above expression and using the experimental results of $\Delta m^{2}_{\odot }$ and $\Delta m^{2}_{\mathrm{ATM} }$, we can get an upper bound for the $m_{\nu _{3}}$ lightest neutrino mass. Therefore, the allowed values for the lightest neutrino mass are: $0\le m_{\nu _{3}}\le 4\times 10^{-6}$ eV. As a result, the $m_{\nu _{2}}$ and $m_{\nu _{1}}$ neutrino masses are easily calculated in the following way:

$$\begin{aligned} m_{\nu _{2}}&= \sqrt{ \Delta m^{2}_{\odot } + \Delta m^{2}_{ \text {ATM} } } \sqrt{ 1 + \dfrac{m^{2}_{\nu _{3}} }{ \Delta m^{2}_{\odot } + \Delta m^{2}_{ \text {ATM} } } }\nonumber \\&\approx \! 5.0 \pm 0.087 \times 10^{-2}~\mathrm{eV},\nonumber \\ m_{\nu _{1}}&= \sqrt{ \Delta m^{2}_{ \text {ATM} } } \sqrt{ 1 + \dfrac{m^{2}_{\nu _{3}} }{ \Delta m^{2}_{ \text {ATM} } } } \nonumber \\&\approx \, 4.90 \pm 0.089 \times 10^{-2}~\mathrm{eV}. \end{aligned}$$

(55)

As we already commented, this version of our model, where the first two RHN masses are degenerate, predicts an inverted ordering among the neutrino masses, the above values clarify explicitly our statement. Since we have calculated the neutrino masses, these cease to be considered as free parameters in the solar mixing angle expression given in Eq. (38). Furthermore, we easily obtain the solar mixing value that is allowed by the fixed free parameters, which are $y_{\ell }$ and the neutrino masses. Actually, with the particular value for the $\bar{\eta }_{3 e}=\pi$ Dirac phase, and using the $y_{\ell }$ value at $90~\%$ at C.L., we predict the following value for the solar mixing angle:

$$\begin{aligned} \tan ^{2} \theta ^{\ell ^{\mathrm{th}}}_{12} = 0.552 \pm 0.078 \rightarrow \theta ^{\ell ^{\mathrm{th}}}_{12} = ( 36.62 \pm 4.06 )^{\circ } \end{aligned}$$

(56)

where we have used $m_{\nu _{2}}= 0.05080$ eV, $m_{\nu _{1}}=0.04987$ eV and $m_{\nu _{3}}=3.9\times 10^{-6}$ eV. As we observe the solar mixing angle value in Eq. (56) is in good agreement with the experimental data. In Fig. 3 we show explicitly the dependence of the solar angle on the $y_{\ell }$ parameter and the $m_{\nu _{3}}$ lightest neutrino mass, respectively.

Fig. 3

Solar mixing angle values as a function of the $y_{\ell }$ parameter and the $m_{\nu _{3}}$ neutrino mass considering $m_{\nu _{2}}= 0.05080$ eV, $m_{\nu _{1}}=0.04987$ eV, and $\bar{\eta }_{3 e}=\pi$

It is clear from these results that the solar angle has a strong dependence on the $m_{\nu _{3}}$ mass, and that the sum rule for the neutrino masses does play an important role to determine the above mixing angle in good agreement with the experimental data.

4.2.2 The masses of right-handed neutrinos without degeneration

In the previous section we obtained that the reactor mixing angle, $\theta ^{\ell }_{13}$, has a nonzero value, whereby the shape of lepton mixing matrix PMNS is not consistent with tribimaximal scenario. However, this value of the reactor mixing angle is small compared to the central value from the global fits [70], and it would correspond to a lower bound.

Now, in order to reproduce the current numerical values of the reactor angle reported by global fits, we break the degeneracy of the first two right-handed neutrinos, namely $M_{\mathrm{R}_{1}} \ne M_{\mathrm{R}_{2}}$. Also, we defined the $\chi ^{2}$ function by

$$\begin{aligned} \chi ^{2} = \frac{ ( S_{12}^{2,\mathrm{ex}} - S_{12}^{2,\mathrm{th}} )^{2} }{\sigma _{ S_{12}^{2,\mathrm{ex}} }^{2} } +\frac{ ( S_{23}^{2,\mathrm{ex}} - S_{23}^{2,\mathrm{th}} )^{2} }{\sigma _{ S_{23}^{2,\mathrm{ex}} }^{2} } +\frac{ ( S_{13}^{2,\mathrm{ex}} - S_{13}^{2,\mathrm{th}} )^{2} }{\sigma _{ S_{13}^{2,\mathrm{ex}} }^{2} } \end{aligned}$$

(57)

where $S_{ij}^{2,\mathrm{ex(th)}} \equiv \sin ^{2} \theta _{ij}^{\ell }$ with $i,j=1,2,3$. The terms with superscript th are the theoretical expressions obtained from Eq. (46), while the terms with superscript ex are the experimental data with uncertainty $\sigma _{ S_{13} }$ given in Eq. (27). A consequence of considering the most general case with non-degenerate right- handed neutrino masses is an increasing of the number of free parameters in the PMNS matrix. In a first scan of the parameter space, we find that the $\eta _{1}$, $\alpha _{2}$, $\eta _{2e}$, and $\bar{\eta }_{3e}$ phases are not correlated among each other, nor with the parameters $m_{\nu _{3[2]}}$, $\widetilde{\mu }_{0}$, and $\delta _{\nu }$. Thus we consider that the $\eta _{1}$, $\alpha _{2}$, $\eta _{2e}$, and $\bar{\eta }_{3e}$ phases vary in the range of 0 to $2\pi$. From the analysis done in the previous section we consider $y_{\ell } \in [ 0.75, 0.96 ]$. Then, to perform the $\chi ^{2}$ fit, the neutrino masses are written as

$$\begin{aligned}&\widetilde{m}_{\nu _{1}} = \dfrac{ m_{\nu _{1}} }{ m_{\nu _{3}} } = \sqrt{ 1 - \frac{ \Delta m^{2}_{ \text {ATM} } }{ m_{\nu _{3}}^{2}} }, \nonumber \\&\widetilde{m}_{\nu _{2}} = \dfrac{ m_{\nu _{2}} }{ m_{\nu _{3}} } = \sqrt{ 1 - \frac{ \Delta m^{2}_{ 32 } }{ m_{\nu _{3}}^{2} } } \text {(NH),} \nonumber \\&\widetilde{m}_{\nu _{3}} = \dfrac{ m_{\nu _{3}} }{ m_{\nu _{2}} } = \sqrt{ 1 - \frac{ \Delta m^{2}_{23} }{ m_{\nu _{2}} } },\nonumber \\&\widetilde{m}_{\nu _{1}} = \dfrac{ m_{\nu _{1}} }{ m_{\nu _{2}} } = \sqrt{ 1 - \frac{ \Delta m^{2}_{ \odot } }{ m_{\nu _{2}}^{2} } } \text {(IH),} \end{aligned}$$

(58)

where $\Delta m^{2}_{32} = \Delta m^{2}_{ \text {ATM} } - \Delta m^{2}_{ \odot }$ and $\Delta m^{2}_{23} = \Delta m^{2}_{ \text {ATM} } + \Delta m^{2}_{ \odot }$. The upper (lower) row corresponds to a normal (inverted) hierarchy. The above expressions only have one free parameter, which is $m_{\nu _{3[2]}}$. This parameter must satisfy the condition $m_{\nu _{3[2]}} < \Delta m^{2}_{ 32[\odot ] }$ and $\sum _{i} m_{\nu _{i}} < 0.23$ eV. The last one was reported by the Planck collaboration [71]. Thus in the $\chi ^{2}$ fit we assume that $m_{\nu _{3[2]}}$ is not a free parameter, since it is constrained by experimental data. Thus, with $\theta = 283^{\circ }[121^{\circ }]$ for a normal (inverted) hierarchy, we see that $\widetilde{\mu }_{0}$ and $\delta _{\nu }$ are the only free parameters in the fit.

Finally, as result of the $\chi ^{2}$ analysis we found that for the best fit point $\chi ^{2}=0.020 [0.014]$ for a normal (inverted) hierarchy. Also, the neutrino masses at $1~\sigma$ are:

$$\begin{aligned}&m_{\nu _{3}} = \left\{ \begin{array}{l} ( 5.35_{-1.73}^{+4.32} ) \times 10^{-2}\,\text {eV}\\ ( 4.44_{-3.87}^{+4.21} ) \times 10^{-2}\,\text {eV} \end{array} \right. , \, m_{\nu _{2}} = \left\{ \begin{array}{l} ( 2.01_{-0.98}^{+6.42} ) \times 10^{-2}\,\text {eV}\\ ( 6.71_{-1.81}^{+3.25} ) \times 10^{-2}\,\text {eV} \end{array} \right. , \nonumber \\&\qquad \qquad m_{\nu _{1}} = \left\{ \begin{array}{l} ( 1.08_{-1.30}^{+6.59} ) \times 10^{-2}\,\text {eV}\\ ( 6.65_{-1.83}^{+3.27} ) \times 10^{-2}\,\text {eV} \end{array} \right. . \end{aligned}$$

(59)

The free parameters $\widetilde{\mu }_{0}$ and $\delta _{\nu }$ at $1~\sigma$ are:

$$\begin{aligned} \begin{array}{l} \widetilde{\mu }_{0} = \left\{ \begin{array}{l} 0.22_{-0.20}^{+0.63} \\ 0.56_{-0.56}^{+0.29} \end{array} \right. \quad \text {and} \quad \delta _{\nu } = \left\{ \begin{array}{l} 0.75_{-0.15}^{+0.24} \\ 0.73_{-0.09}^{+0.25} \end{array} \right. . \end{array} \end{aligned}$$

(60)

We obtain the following numerical values for the leptonic mixing angles, at $1~\sigma$:

$$\begin{aligned}&\sin ^{2} \theta _{12}^{\ell ^{\mathrm{th}}} = \left\{ \begin{array}{l} 0.324_{-0.016}^{+0.015} \\ 0.325_{-0.018}^{+0.014} \end{array} \right. ,\,\, \sin ^{2} \theta _{23}^{\ell ^{\mathrm{th}}} = \left\{ \begin{array}{l} 0.515_{-0.069}^{+0.078} \\ 0.562_{-0.048}^{+0.036} \end{array} \right. ,\nonumber \\&\qquad \qquad \sin ^{2} \theta _{13}^{\ell ^{\mathrm{th}}} = \left\{ \begin{array}{l} 0.0232_{-0.0016}^{+0.0021} \\ 0.0241_{-0.0021}^{+0.0018} \end{array} \right. ,\end{aligned}$$

(61)

$$\begin{aligned}&\theta _{12}^{\ell ^{\mathrm{th}}} = \left\{ \begin{array}{l} ( 34.71_{-0.98}^{+0.91} )^{\circ } \\ ( 34.73_{-1.11}^{+0.89} )^{\circ } \end{array} \right. ,\quad \theta _{23}^{\ell ^{\mathrm{th}}} = \left\{ \begin{array}{l} ( 45.83_{-3.98}^{+4.49} )^{\circ } \\ ( 48.57_{-2.76}^{+2.07} )^{\circ } \end{array} \right. ,\nonumber \\&\qquad \qquad \theta _{13}^{\ell ^{\mathrm{th}}} = \left\{ \begin{array}{l} ( 8.77_{-0.32}^{+0.40} )^{\circ } \\ ( 8.93_{-0.39}^{+0.33} )^{\circ } \end{array} \right. , \end{aligned}$$

(62)

which are in very good agreement with the last global fit reported in Ref. [70]. The upper (lower) row corresponds to a normal (inverted) hierarchy. In Fig. 4 we show the allowed regions, at $69$ and $95\,\%$ C.L., for the sine-squared of the lepton mixing angles considering a normal and inverted hierarchy.

Fig. 4

Allowed regions for the lepton mixing angles considering a normal and inverted hierarchy (NH and IH) in the mass spectrum of neutrinos. The turquoise region is at $90\,\%$, the red region is at $69~\%$. The blue solid lines delimit the experimental data of $\sin ^{2} \theta _{12(23)}^{\ell }$ at $1\sigma$. The blue dashed lines delimit the experimental data of $\sin ^{2} \theta _{13}^{\ell }$ at $1\sigma$ [70]

Before closing this section, a comment is in order. As is well known, the effective neutrino mass matrix, $\mathbf {M}_{\nu }$, is sensitive to the effect of the running of the mass matrix parameters from high to low energy [72, 73]. Actually, small mixing angles could get a notable enhancement, as remarked in [72]. Thus, in the scenario where the $\theta _{13}$ is tiny, it would be interesting to make a running of the mixing parameters to know how much the reactor angle changes, but we will leave this for a future work.

5 Outlook and remarks

We have studied a non-minimal supersymmetric $SU(5)$ model where the $Q_{6}$ flavor symmetry plays an important role in accommodating the masses and mixings for quarks and leptons. For the former sector, the CKM mixing matrix has been obtained with great accuracy and it is consistent with the experimental results. For leptons we considered two cases: (a) the first two masses of RHNs are degenerate, (b) the RHNs masses are not degenerate.

On the one hand, for the case (a) the flavor symmetry implies an appealing sum rule for the neutrino masses, which leads to an inverted hierarchy and is crucial to determine the solar mixing angle. As a main result in this case, we see that the atmospheric $\theta ^{\ell ^{\mathrm{th}}}_{23} = ( 46.18^{+0.66}_{-0.65} )^{\circ }$ and solar angle $\theta ^{\ell ^{\mathrm{th}}}_{12} = ( 36.62\,\pm \,4.06 )^{\circ }$ are in good agreement with the experimental data, however, the reactor angle value, $\theta ^{\ell ^{\mathrm{th}}}_{13} = ( 3.38^{+0.03}_{-0.02} )^{\circ }$, is more than $3 \sigma$ away from the central value from the global fits. It is worth pointing out that the model predicts a nonzero value for this angle, unlike the tribimaximal case, and that this value constitutes the lower bound for $\theta _{13}$ in the more general model, where the right-handed neutrinos in the $Q_6$ doublet are not mass degenerate.

On the other hand, for the more general case (b), where the RHNs masses are not degenerate, we obtained a value for the reactor mixing angle in very good agreement with the last experimental data or global fits, as is the case in the non-supersymmetric $S_3$ models [50]. Namely, in this case it is possible that the neutrino mass spectrum obeys a normal or an inverted hierarchy. Thus, we find that the leptonic mixing angles have the following theoretical values for a normal (inverted) hierarchy: $\theta _{12}^{ \ell ^{\mathrm{th}} } = ( 34.71_{-0.98}^{+0.91} )^{\circ }$ $[ ( 34.73_{-1.11}^{+0.89} )^{\circ } ]$, $\theta _{23}^{ \ell ^{\mathrm{th}} } = ( 45.83_{-3.98}^{+4.49} )^{\circ }$ $[ ( 48.57_{-2.76}^{+2.07} )^{\circ } ]$, and $\theta _{13}^{ \ell ^{\mathrm{th}} } = ( 8.77_{-0.32}^{+0.40} )^{\circ }$ $[ ( 8.93_{-0.39}^{+0.33} )^{\circ } ]$.

Although in this preliminary analysis we have found the form of the mass and mixing matrices for quarks and leptons and we have shown that they lead to realistic values, we have left aside subtle issues as the full analysis of the scalar superpotential, the details of the proton decay, the running of the mass parameters from high to low scale energy, and all the phenomenology that the model provides by itself. These topics will be taken into account in a complete study of the model, as we already commented. In general, the model seems to work out very well and it may be considered as a realistic one.

Appendices

Appendix A: $Q_{6}$ flavor symmetry

The $Q_{6}$ group has 12 elements, which are contained in six conjugacy classes; therefore, it contains six irreducible representations. We will use the notation given in [15], there are various notations and extensive studies of this group; see for example [12, 74]. The $Q_{6}$ family symmetry has two two-dimensional irreducible representations, denoted by $\mathbf{2}_{1}$, and $\mathbf{2}_{2}$, and four one-dimensional ones, which are denoted by $\mathbf{1}_{+,0}$, $\mathbf{1}_{+,2}$, $\mathbf{1}_{-,1}$, and $\mathbf{1}_{-,3}$. As is well known, $\mathbf{2}_{1}$ is a pseudo real and $\mathbf{2}_{2}$ is a real representation. In addition, for $\mathbf{1}_{\pm ,n}$ we see that $n=0,1,2,3$ is the factor $\exp (in\pi /2)$, which appears in the matrix given by $\mathbf{B}$. The $\pm$ stands for the change of sign under the transformation given by the $\mathbf{A}$ matrix. Therefore, the first two one-dimensional representations are real and the two last ones are complex conjugate to each other.

$$\begin{aligned} Q_{6} = \lbrace 1, \mathbf{A}, \mathbf{A}^{2}, \mathbf{A}^{3}, \mathbf{A}^{4}, \mathbf{A}^{5}, \mathbf{B}, \mathbf{A}\mathbf{B}, \mathbf{A}^{2}\mathbf{B}, \mathbf{A}^{3} \mathbf{B}, \mathbf{A}^{4} \mathbf{B}, \mathbf{A}^{5} \mathbf{B} \rbrace , \end{aligned}$$

(63)

where the $\mathbf{A}$ and $\mathbf{B}$ are two-dimensional matrices whose explicit forms are given by

$$\begin{aligned} \mathbf{A} = \begin{pmatrix} \cos (\pi /3 ) &{}\quad \sin (\pi /3 ) \\ -\sin (\pi /3 ) &{}\quad \cos (\pi /3 ) \end{pmatrix} \quad \text {and} \quad \mathbf{B} = \begin{pmatrix} i &{}\quad 0 \\ 0 &{}\quad -i \end{pmatrix}. \end{aligned}$$

(64)

Let us write the multiplication rules among the six irreducible representations, which will be useful to build a phenomenological model:

$$\begin{aligned} \mathbf{1}_{+,2}\otimes \mathbf{1}_{+,2}&= \mathbf{1}_{+,0},\quad \mathbf{1}_{-,3} \otimes \mathbf{1}_{-,3} = \mathbf{1}_{+,2},\quad \mathbf{1}_{-,1} \otimes \mathbf{1}_{-,1} = \mathbf{1}_{+,2},\quad \mathbf{1}_{-,1} \otimes \mathbf{1}_{-,3} = \mathbf{1}_{+,0},\nonumber \\ \mathbf{1}_{+,2} \otimes \mathbf{1}_{-,1}&= \mathbf{1}_{-,3},\quad \mathbf{1}_{+,2} \otimes \mathbf{1}_{-,3} = \mathbf{1}_{-,1},\quad \mathbf{2}_{1} \otimes \mathbf{1}_{+,2} = \mathbf{2}_{1},\quad \mathbf{2}_{1} \otimes \mathbf{1}_{-,3} = \mathbf{2}_{2},\nonumber \\ \mathbf{2}_{1} \otimes \mathbf{1}_{-,1}&= \mathbf{2}_{2},\quad \mathbf{2}_{2} \otimes \mathbf{1}_{+,2} = \mathbf{2}_{2},\quad \mathbf{2}_{2} \otimes \mathbf{1}_{-,3} = \mathbf{2}_{1},\quad \mathbf{2}_{2} \otimes \mathbf{1}_{-,1} = \mathbf{2}_{1}; \nonumber \\ \overbrace{\begin{pmatrix} x_{1} \\ x_{2} \end{pmatrix}}^{\mathbf{2}_{1}} \, \otimes \, \overbrace{\begin{pmatrix} y_{1} \\ y_{2} \end{pmatrix}}^{\mathbf{2}_{1}}&= \overbrace{(x_{1}y_{2}-x_{2}y_{1})}^{\mathbf{1}_{+,0}} \quad +\quad \overbrace{(x_{1}y_{1}+x_{2}y_{2})}^{\mathbf{1}_{+,2}} \quad + \quad \overbrace{\begin{pmatrix} -x_{1}y_{2}-x_{2}y_{1} \\ x_{1}y_{1}-x_{2}y_{2} \end{pmatrix}}^{\mathbf{2}_{2}} \nonumber \\ \overbrace{\begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix}}^{\mathbf{2}_{2}} \otimes \overbrace{\begin{pmatrix} b_{1} \\ b_{2} \end{pmatrix}}^{\mathbf{2}_{2}}&=\overbrace{(a_{1}b_{1}+a_{2}b_{2})}^{\mathbf{1}_{+,0}}\quad + \quad \overbrace{(a_{1}b_{2}- a_{2}b_{1})}^{\mathbf{1}_{+,2}} \quad + \quad \overbrace{\begin{pmatrix} -a_{1}b_{1}+a_{2}b_{2} \\ a_{1}b_{2}+a_{2}b_{1} \end{pmatrix}}^{\mathbf{2}_{2}} \nonumber \\ \overbrace{\begin{pmatrix} x_{1} \\ x_{2} \end{pmatrix}}^{\mathbf{2}_{1}} \otimes \overbrace{\begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix}}^{\mathbf{2}_{2}}&=\overbrace{(x_{1}a_{2}+x_{2}a_{1})}^{\mathbf{1}_{-,3}}\quad + \quad \overbrace{(x_{1}a_{1}-x_{2}a_{2})}^{\mathbf{1}_{-,1}} \quad + \quad \overbrace{\begin{pmatrix} x_{1}a_{1}+x_{2}a_{2} \\ x_{1}a_{2}-x_{2}a_{1} \end{pmatrix}}^{\mathbf{2}_{1}}, \end{aligned}$$

(65)

Appendix B: Neutrino mass matrix

The degeneracy in the vacuum expectation values for two scalar fields, does not modify the functional structure of the effective neutrino mass matrix $\mathbf{m}_{\nu }$, Eq. (13), because it still is a matrix with one texture zero. But the parameters $a_{\nu }$, $b_{\nu }$, $c_{\nu }$, $d_{\nu }$, and $\mu _{0}$ given in Eq. (14) are simplified a little. Therefore, the expressions in Eqs. (13) and (14) take the following form:

$$\begin{aligned}&\mathbf{m}_{\nu } = \mathbf{\textit{u} }_{\theta }^{\mathrm{T}} \mathbf{M}_{\nu } \mathbf{\textit{u} }_{\theta } = \left( \begin{array}{l@{\quad }l@{\quad }l} b_{\nu } &{} a_{\nu } &{} c_{\nu } \\ a_{\nu } &{} \mu _{0} &{} 0 \\ c_{\nu } &{} 0 &{} d_{\nu } \end{array} \right) \qquad \text {and} \nonumber \\&\qquad \quad \mathbf{\textit{u} }_{\theta } = \left( \begin{array}{c@{\quad }c@{\quad }c} \cos \theta &{} 0 &{} -\sin \theta \\ \sin \theta &{} 0 &{} \cos \theta \\ 0 &{} 1 &{} 0 \end{array} \right) , \end{aligned}$$

(66)

with

$$\begin{aligned} \tan \theta= & {} \frac{ M_{\mathrm{R}_{2}} }{ M_{\mathrm{R}_{1}} }, \quad \mu _{0} = \frac{ ( y_{2}^{n} h^{0 \mathrm{u}} )^{2} }{ M_{\mathrm{R}_{1}} M_{\mathrm{R}_{2}} } [ M_{\mathrm{R}_{2}} + M_{\mathrm{R}_{1}} ], \nonumber \\&\quad a_{\nu } = \frac{ \cos \theta y_{1}^{n} h^{0 \mathrm{u}}_{3} y_{2}^{n} h^{0 \mathrm{u}} }{ M_{\mathrm{R}_{2}} M_{\mathrm{R}_{1}}^{2} } ( M_{\mathrm{R}_{1}}^{2} + M_{\mathrm{R}_{2}}^{2} ), \nonumber \\ b_{\nu }= & {} \frac{\cos ^{2} \theta }{ M_{\mathrm{R}_{1}}^{2} } \bigg [ \frac{ ( y_{1}^{n} h^{0 \mathrm{u}}_{3} )^{2} }{ M_{\mathrm{R}_{1}} M_{\mathrm{R}_{2} } } ( M_{\mathrm{R}_{1}}^{3} + M_{\mathrm{R}_{2}}^{3} ) \nonumber \\&+ \frac{ ( y_{3}^{n} h^{0 \mathrm{u}} )^{2} }{ M_{\mathrm{R}_{3}} } ( M_{\mathrm{R}_{1}} + M_{\mathrm{R}_{2}} )^{2} \bigg ], \nonumber \\ c_{\nu }= & {} \frac{ \cos ^{2} \theta }{ M_{\mathrm{R}_{1}}^{2} } \bigg [ ( y_{1}^{n} h^{0 \mathrm{u}}_{3} )^{2} ( M_{\mathrm{R}_{2}} - M_{\mathrm{R}_{1}} ) \nonumber \\&+ \frac{ ( y_{3}^{n} h^{0 \mathrm{u}} )^{2} }{ M_{\mathrm{R}_{3}} } ( M_{\mathrm{R}_{1}}^{2} - M_{\mathrm{R}_{2}}^{2} ) \bigg ], \text {and} \;\nonumber \\ d_{\nu }= & {} \frac{ \cos ^{2} \theta }{ ( M_{\mathrm{R}_{1}} )^{2} } \left[ ( y_{1}^{n} h^{0 \mathrm{u}}_{3} )^{2} ( M_{\mathrm{R}_{2}} + M_{\mathrm{R}_{1}} )\right. \nonumber \\&\left. + \frac{ ( y_{3}^{n} h^{0 \mathrm{u}} )^{2} }{ M_{\mathrm{R}_{3}} } ( M_{\mathrm{R}_{1}} - M_{\mathrm{R}_{2}} )^{2} \right] . \end{aligned}$$

(67)

From the previous expressions is easy to notice that when the masses of the first two right-handed neutrinos are degenerate, $M_{\mathrm{R}_{1}} = M_{\mathrm{R}_{2}}$, the parameter $c_{\nu }$ becomes zero and the rotation angle takes the value $\theta = \pi /4$. Hence, the effective neutrino mass matrix $\mathbf{m}_{\nu }$ is reduced to a block matrix.