1 Introduction

How to explain and understand tiny neutrino masses and the fermion mixings, respectively, in and beyond the Standard Model (SM) is still an open question. Up to now, it is not clear if there is an organizing principle in the Yukawa sector that explains the almost diagonal CKM mixing matrix and its PMNS counterpart, which has large mixing values.

The pronounced hierarchy among the quark masses, \(m_{t}\gg m_{c}\gg m_{u}\) and \(m_{b}\gg m_{s}\gg m_{d}\), could be behind the small mixing angles that parametrize the CKM, which depend strongly on the mass ratios [1,2,3]. From a phenomenological point of view, the hierarchy among the fermion masses may be understood by means of textures (zeros) in the fermion mass matrices [1,2,3,4]. On the theoretical side, mass textures can be generated dynamically by non-abelian discrete symmetries [5,6,7,8,9]. The Fritzsch [10,11,12] and the NNI [13,14,15,16] textures are hierarchical, however, only the latter can accommodate with good accuracy the CKM matrix.

In the lepton sector, the hierarchy seems to work differently in the mixings since the charged lepton masses are hierarchical, \(m_{\tau }\gg m_{\mu }\gg m_{e}\), but the active neutrino masses exhibit a weak hierarchy [17, 18] that may be responsible for the large mixing values. If the neutrinos obey a normal mass ordering, large mixings can also be obtained by the Fritzsch and NNI textures [17,18,19]. It is worth mentioning that non-hierarchical fermion mass matrices could also accommodate the lepton mixing angles [20, 21]. Nevertheless, the hierarchy might have nothing to do with the mixing [22,23,24,25], since large mixings might be explained by discrete symmetries which were motivated mainly by the experimental values, \(\theta _{23}\approx 45^{\circ }\) and \(\theta _{13}\approx 0^{\circ }\). The \(\mu \leftrightarrow \tau \) symmetry [26,27,28,29,30,31] was proposed to be behind the atmospheric and reactor mixing angle values. This symmetry predicts exactly that \(\theta _{23}=45^{\circ }\) and \(\theta _{13}=0^{\circ }\), which were consistent with the experimental data many years ago. The tri-bimaximal (TB) mixing pattern [8, 32, 33] was suggested for obtaining the above angles plus \(\sin {\theta _{12}}={1/\sqrt{3}}\), for the solar angle which coincides approximately with the experimental value. An intriguing fact is that the above mixing pattern does not depend on the lepton masses up to corrections to the lower order in the mixing matrices.

To face the neutrino masses and mixings problems, one has to go beyond the SM, and it has to be extended or replaced by a new framework where ideally both issues can be explained. Along this line of thought, one of the best motivated candidates to replace the SM is the baryon number minus lepton number (B–L) gauge model, \(\text {SM}\otimes U(1)_{B-L}\), which may come from the Grand Unified Theory (GUT) \(\text {SO}(10)\) [34, 35] or from the unified model \(S(3)_{C}\otimes \text {SU}(3)_{L}\otimes U(1)_{X}\otimes U(1)_{N}\) [36, 37]. The breaking mass scale of the B–L model to the SM is related with the mass of the three right-handed neutrinos (RHNs) that are included to cancel anomalies, explaining, at tree level, the tiny neutrino masses by means of the type I see-saw mechanism [38,39,40,41,42,43,44] (for another mechanism in B–L see [45]). Apart from neutrino masses and mixings, leptogenesis, dark matter and inflation also have found a realization in the B–L model [46,47,48,49,50,51,52]. Due to all those features, in our point of view, the renormalizable B–L model has the main ingredients to address the problem of the quark and lepton masses and their contrasting mixing matrices.

Moving on to the mixing, the \(\mathbf{S}_{3}\) non-abelian group has been proposed as the underlying flavor symmetry in different frameworks [53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82]. One motivation to use this discrete symmetry in the lepton sector is to generate the \(\mu \leftrightarrow \tau \) symmetry [26,27,28,29,30,31] or the TB mixing matrix [83, 84]. In the quark sector, the \(\mathbf{S}_{3}\) symmetry can give rise to the Fritzsch and generalized Fritzsch mass textures [61, 85]. More recently, it was shown that the NNI mass textures are hidden in the \(\mathbf{S}_{3}\) flavor symmetry [68], this last novel fact will be highlighted as part of our motivation in the present work.

Therefore, we make a scalar extension of the B–L gauge model where the \(\mathbf{S}_{3}\) non-abelian discrete group drives mainly the Yukawa sector. Motived by the large and small hierarchies among the quark and active neutrino masses, respectively, the quark and lepton families are not treated on the same footing under the assignment of the discrete group. As a consequence, NNI textures appear in the quark sector, leading to the CKM mixing matrix, whereas in the lepton sector, a soft breaking of the \(\mu \leftrightarrow \tau \) symmetry in the effective neutrino mass that comes from type I see-saw mechanism provides a non-maximal atmospheric angle and a non-zero reactor angle.

The plan of this paper is as follows: the B–L gauge model and the \(\mathbf{S}_{3}\) flavor symmetry are described briefly in Sect. 2, and the fermion masses and mixings will be discussed in Sect. 3. In Sect. 4 some conclusions are drawn.

2 Flavored B–L Model

The B–L gauge model is based on the \(\text {SU}(3)_{c}\otimes \text {SU}(2)_{L}\otimes U(1)_{Y}\otimes U(1)_{B-L}\) gauge group where, apart from the SM fields, three \(N_{i}\) RHNs and a \(\phi \) singlet scalar field are added to the matter content. Under B–L, the quantum numbers for quarks, leptons and Higgs (\(\phi \)) are 1 / 3, \(-1\) and 0 (\(-2\)), respectively. The allowed Lagrangian is

$$\begin{aligned} {\mathcal {L}}_{B-L}={\mathcal {L}}_{\text {SM}}-y^{D}{\bar{L}}{\tilde{H}}N-\frac{1}{2}y^{N}{\bar{N}}^{c}\phi N-V\left( H,\phi \right) \end{aligned}$$
(1)

with

$$\begin{aligned} V\left( H,\phi \right) =\mu ^{2}_{BL}\phi ^{\dagger } \phi +\frac{\lambda _{BL}}{2}\left( \phi ^{\dagger }\phi \right) ^{2}-\lambda ^{H \phi }\left( H^{\dagger } H \right) \left( \phi ^{\dagger } \phi \right) , \end{aligned}$$
(2)

where \({\tilde{H}}_{i}=i\sigma _{2}H^{*}_{i}\). Spontaneous symmetry breaking of \(U(1)_{B-L}\) happens usually at high energies, so the breaking scale is larger than the electroweak scale, \(\phi _{0}\gg v\). In this first stage, the RHNs become massive particles; along with this, an extra gauge boson, \(Z_{B-L}\), appears as a result of breaking the gauge group. The rest of the particles turn out to be massive when the Higgs scalars acquire their vacuum expectation value (vevs) and tiny active neutrino masses are explained by the type I see-saw mechanism. We have

$$\begin{aligned} \langle H\rangle = \frac{1}{\sqrt{2}}\left( \begin{array}{c} 0 \\ v \\ \end{array} \right) , \quad \langle \phi \rangle =\frac{\phi _{0}}{\sqrt{2}}. \end{aligned}$$
(3)

On the other hand, let us describe briefly the non-Abelian group \(\mathbf{S}_{3}\), which is the permutation group of three objects; it has three irreducible representations: two 1-dimensional, \(\mathbf{1}_{S}\) and \(\mathbf{1}_{A}\), and one 2-dimensional representation, \(\mathbf{2}\) (for a detailed study see [5]). Thus, the three dimensional real representation can be decomposed as \(\mathbf{3}_{S}=\mathbf{2}\oplus \mathbf{1}_{S}\) or \(\mathbf{3}_{A}=\mathbf{2}\oplus \mathbf{1}_{A}\). The multiplication rules among the irreducible representations are

$$\begin{aligned}&\mathbf{1}_{S}\otimes \mathbf{1}_{S}=\mathbf{1}_{S},\quad \mathbf{1}_{S}\otimes \mathbf{1}_{A}=\mathbf{1}_{A},\nonumber \\&\mathbf{1}_{A}\otimes \mathbf{1}_{S}=\mathbf{1}_{A},\quad \mathbf{1}_{A}\otimes \mathbf{1}_{A}=\mathbf{1}_{S},\nonumber \\&\mathbf{1}_{S}\otimes \mathbf{2}=\mathbf{2},\quad \mathbf{1}_{A}\otimes \mathbf{2}=\mathbf{2},\quad \mathbf{2}\otimes \mathbf{1}_{S}=\mathbf{2},\quad \mathbf{2}\otimes \mathbf{1}_{A}=\mathbf{2};\nonumber \\&\begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix}_{\mathbf{2}} \otimes \begin{pmatrix} b_{1} \\ b_{2} \end{pmatrix}_{\mathbf{2}} \nonumber \\&\quad = \left( a_{1}b_{1}{+}a_{2}b_{2}\right) _{\mathbf{1}_{S}} \oplus \left( a_{1}b_{2}{-}a_{2}b_{1}\right) _{\mathbf{1}_{A}} \oplus \begin{pmatrix} a_{1}b_{2}+a_{2}b_{1} \\ a_{1}b_{1}-a_{2}b_{2} \end{pmatrix}_{\mathbf{2}}. \end{aligned}$$
(4)
Table 1 Flavored \(B-L\) model. Here, \(I=1,2\) and \(J=2,3\)
Full size table

Having introduced the theoretical framework and the flavor symmetry that will play an important role in the present work, it is worthwhile to point out that the present idea has been developed in the framework of left–right symmetric model (LRSM) [86]. However, there are substantial differences between the LRSM and B–L models in their minimal versions: (a) due to the gauge symmetry the LRSM contains more scalars fields in comparison to the B–L model; (b) as a consequence the latter one has a simpler Yukawa mass term than the LRSM, which allows us to work with fewer couplings in the fermionic mass matrices; (c) the effective neutrino mass matrix has two contributions due to the type I and II see-saw mechanism in the LRSM (the latter one usually is neglected by hand), whereas the type I see-saw mechanism works in the B–L model. The above statements stand for some advantages to study fermion masses and mixings in the B–L model; moreover, the LRSM appears to be complicated if the scalar sector is augmented. From these comments, one may conclude that the current work is a simple comparison with the LRSM, however, we emphasize that the quark sector makes the difference between the present work and that developed in [86]. As we will see later, in the current study the CKM matrix is understood by hierarchical mass matrices, which is not the case in [86]; this last statement can be verified in [87] which is an extended version of the previous work [86].

Now, let us remark important points about the scalar sector and the family assignment under the flavor symmetry in our model. Due to the flavor symmetry, three Higgs doublets have been added in this model to obtain the CKM mixing matrix. At the same time, as we will see, the charged leptons and Dirac neutrinos mass matrices are built so as to be diagonal, so the mixing will come from the RHN mass matrix. Then three singlets scalars fields, \(\phi _{i}\), are needed to accomplish this. Along with this, in order to try to explain naively the contrasting values between the CKM and PMNS mixing matrices, let us point out a crucial difference in the way the quark and lepton families have been assigned under the irreducible representations of \(\mathbf{S}_{3}\). Hierarchy among the fermion masses suggests that both in the quark and Higgs sector, the first and second family are put together in a flavor doublet \(\mathbf{2}\) and the third family in a singlet \(\mathbf{1}_{S}\). On the contrary, for the leptons, the first family has been assigned to a singlet \(\mathbf{1}_{S}\) and the second and third families to a doublet \(\mathbf{2}\). As a consequence of this assignment, the hierarchical NNI textures are hidden in the quark mass matrices. In the lepton sector, on the other hand, the lepton mixings can be understood from an approximated \(\mu \leftrightarrow \tau \) symmetry in the effective neutrino mass matrix [86].

We ought to comment that the above flavor symmetry assignment may be incompatible with \(\text {SO}(10)\) multiplets, however, this assignment could be realized in the \(S(3)_{C}\otimes \text {SU}(3)_{L}\otimes U(1)_{X}\otimes U(1)_{N}\) model (see for instance [36, 37]).

In Table 1, the full assignment for the matter content is shown. The \({\mathbf{Z}}_{\mathbf{2}}\) symmetry has been added in order to prohibit some Yukawa couplings in the lepton sector, but this is not enough to obtain diagonal mass matrices. Thus, an extra symmetry will be imposed below.

Thus, the most general form for the Yukawa interaction Lagrangian that respects the \(\mathbf{S}_{3}\otimes \mathbf{Z}_{2}\) flavor symmetry and the gauge group is given as

$$\begin{aligned} -{\mathcal {L}}_{Y}&=y^{d}_{1}\left[ {\bar{Q}}_{1 L}\left( H_{1} d_{2 R}+H_{2} d_{1 R}\right) \right. \nonumber \\&\quad \left. +{\bar{Q}}_{2 L}\left( H_{1} d_{1 R}-H_{2}d_{2 R}\right) \right] \nonumber \\&\quad +y^{d}_{2}\left[ {\bar{Q}}_{1 L}H_{3}d_{1 R}+{\bar{Q}}_{2 L}H_{3} d_{2 R}\right] \nonumber \\&\quad +y^{d}_{3}\left[ {\bar{Q}}_{1 L}H_{1}+{\bar{Q}}_{2 L}H_{2}\right] d_{3 R}\nonumber \\&\quad +y^{d}_{4}{\bar{Q}}_{3 L}\left[ H_{1}d_{1 R}+H_{2}d_{2 R}\right] +y^{d}_{5}{\bar{Q}}_{3 L}H_{3}d_{3 R}\nonumber \\&\quad +y^{u}_{i} \left( H\rightarrow {\tilde{H}},d_{R}\rightarrow u_{R}\right) \nonumber \\&\quad +y^{e}_{1}{\bar{L}}_{1}H_{3}e_{1 R}+y^{e}_{2}\left[ ({\bar{L}}_{2}H_{2}\right. \nonumber \\&\quad \left. +{\bar{L}}_{3}H_{1})e_{2 R}+({\bar{L}}_{2}H_{1}-{\bar{L}}_{3}H_{2})e_{3 R} \right] \nonumber \\&\quad +y^{e}_{3}\left[ {\bar{L}}_{2}H_{3}e_{2 R}+{\bar{L}}_{3}H_{3}e_{3 R}\right] \nonumber \\&\quad + y^{D}_{i}\left( H\rightarrow {\tilde{H}},e_{R}\rightarrow N\right) +y^{N}_{1}{\bar{N}}^{c}_{1}\phi _{3}N_{1}\nonumber \\&\quad +y^{N}_{2}\left[ {\bar{N}}^{c}_{1}\left( \phi _{1 }N_{2}+\phi _{2}N_{3}\right) \right. \nonumber \\&\quad \left. +\left( {\bar{N}}^{c}_{2}\phi _{1}+{\bar{N}}^{c}_{3}\phi _{2}\right) N_{1}\right] \nonumber \\&\quad +y^{N}_{3}\left[ {\bar{N}}^{c}_{2}\phi _{3 }N_{2}+{\bar{N}}^{c}_{3}\phi _{3}N_{3}\right] +h.c. \end{aligned}$$
(5)

At this stage, an extra symmetry \(\mathbf{Z}^{e}_{2}\) is used to obtain diagonal charged and neutrinos Dirac mass matrices. This symmetry does not modify the Majorana mass matrix form. Explicitly, in the above Lagrangian, we require that

$$\begin{aligned} L_{3}\leftrightarrow -L_{3},\quad e_{3 R}\leftrightarrow -e_{3 R},\quad N_{3 }\leftrightarrow -N_{3},\quad \phi _{2 }\leftrightarrow -\phi _{2}, \end{aligned}$$
(6)

so the off-diagonal entries 23 and 32 in the lepton sector are absent. Then this allows one to identify properly the charged lepton masses; at the same time, we can speak strictly about the \(\mu \leftrightarrow \tau \) symmetry in the effective neutrino mass.

On the other hand, it is convenient to point out that the scalar potential of the SM with three families of Higgs, \(V(H_{i})\), and the representation \(\mathbf{3}_{S}=\mathbf{2}\oplus \mathbf{1}_{S}\) has been studied in [53, 88,89,90,91,92,93,94]. So, in the \(B-L\) model, the flavored gauge scalar potential (together with Eq. (6)) is given by \(V(H_{i},\phi _{i})=V(H_{i})+V(\phi _{i})+V(H_{i},\phi _{i})\) where the first term has already been analyzed in the mentioned works; the second and third terms are given as

$$\begin{aligned}&V(\phi _{i}) +V(H_{i},\phi _{i})=\mu ^{2}_{1BL}\left( \phi ^{\dagger }_{1}\phi _{1}{+}\phi ^{\dagger }_{2}\phi _{2}\right) {+}\mu ^{2}_{2BL}\left( \phi ^{\dagger }_{3}\phi _{3}\right) \nonumber \\&\quad +\lambda ^{\phi }_{1}\left( \phi ^{\dagger }_{1}\phi _{1}+\phi ^{\dagger }_{2}\phi _{2}\right) ^{2}+\lambda ^{\phi }_{2}\left( \phi ^{\dagger }_{1}\phi _{2}-\phi ^{\dagger }_{2}\phi _{1}\right) ^{2}\nonumber \\&\quad +\lambda ^{\phi }_{5}\left( \phi ^{\dagger }_{3}\phi _{3}\right) \left( \phi ^{\dagger }_{1}\phi _{1}+\phi ^{\dagger }_{2}\phi _{2}\right) \nonumber \\&\quad +\lambda ^{\phi }_{3}\left[ \left( \phi ^{\dagger }_{1}\phi _{2}+\phi ^{\dagger }_{2}\phi _{1}\right) ^{2}+\left( \phi ^{\dagger }_{1}\phi _{1}-\phi ^{\dagger }_{2}\phi _{2}\right) ^{2}\right] \nonumber \\&\quad +\lambda ^{\phi }_{6}\left[ \left( \phi ^{\dagger }_{3}\phi _{1}\right) \left( \phi ^{\dagger }_{1}\phi _{3}\right) +\left( \phi ^{\dagger }_{3}\phi _{2}\right) \left( \phi ^{\dagger }_{2}\phi _{3}\right) \right] \nonumber \\&\quad +\lambda ^{\phi }_{7}\left[ \left( \phi ^{\dagger }_{3}\phi _{1}\right) ^{2}\!+\! \left( \phi ^{\dagger }_{3}\phi _{2}\right) ^{2}\!+\!\text {h.c.}\right] \!+\! \lambda ^{\phi }_{8}\left( \phi ^{\dagger }_{3}\phi _{3}\right) ^{2}\nonumber \\&\quad +\lambda ^{H\phi }_{1}\left( H^{\dagger }_{1}H_{1}+H^{\dagger }_{2}H_{2}\right) \left( \phi ^{\dagger }_{1}\phi _{1}+\phi ^{\dagger }_{2}\phi _{2}\right) \nonumber \\&\quad +\lambda ^{H\phi }_{4}\left( H^{\dagger }_{3}H_{2}\right) \left( \phi ^{\dagger }_{1}\phi _{1}\!-\!\phi ^{\dagger }_{2}\phi _{2}\!\right) \nonumber \\&\quad +\lambda ^{H \phi }_{5}\left( H^{\dagger }_{1}H_{2}+H^{\dagger }_{2}H_{1}\right) \left( \phi ^{\dagger }_{3}\phi _{1}\right) \nonumber \\&\quad +\lambda ^{H \phi }_{6}\left( H^{\dagger }_{2}H_{3}\right) \left( \phi ^{\dagger }_{1}\phi _{1}-\phi ^{\dagger }_{2}\phi _{2}\right) \nonumber \\&\quad +\lambda ^{H \phi }_{7}\left( H^{\dagger }_{1}H_{2}+H^{\dagger }_{2}H_{1}\right) \left( \phi ^{\dagger }_{1}\phi _{3}\right) \nonumber \\&\quad +\lambda ^{H \phi }_{8}\left( H^{\dagger }_{3}H_{3}\right) \left( \phi ^{\dagger }_{1}\phi _{1}+\phi ^{\dagger }_{2}\phi _{2}\right) \nonumber \\&\quad +\lambda ^{H \phi }_{9}\left( H^{\dagger }_{1}H_{1}+H^{\dagger }_{2}H_{2}\right) \left( \phi ^{\dagger }_{3}\phi _{3}\right) \nonumber \\&\quad +\lambda ^{H \phi }_{10}\left( H^{\dagger }_{1}H_{3}\right) \left( \phi ^{\dagger }_{3}\phi _{1}\right) \nonumber \\&\quad + \lambda ^{H \phi }_{11}\left( H^{\dagger }_{3}H_{1}\right) \left( \phi ^{\dagger }_{1}\phi _{3}\right) +\lambda ^{H \phi }_{12}\left( H^{\dagger }_{3}H_{1}\right) \left( \phi ^{\dagger }_{3}\phi _{1}\right) \nonumber \\&\quad +\lambda ^{H \phi }_{13}\left( H^{\dagger }_{1}H_{3}\right) \left( \phi ^{\dagger }_{1}\phi _{3}\right) +\lambda ^{H \phi }_{14}\left( H^{\dagger }_{3}H_{3}\right) \left( \phi ^{\dagger }_{3}\phi _{3}\right) , \end{aligned}$$
(7)

where the factor of 1 / 2, in the second term of Eq. (2), has been absorbed in the \(\lambda ^{\phi }_{i}\equiv \lambda _{BL}\) parameter. Then, assuming that all parameters in the scalar potential are real, the minimization condition for the complete scalar potential are given by

$$\begin{aligned} 2\mu ^{2}_{1}&=-2\gamma \left( v^{2}_{1}+v^{2}_{2}\right) -6\lambda _{4}v_{2}v_{3}-\lambda v^{2}_{3}\nonumber \\&\quad +\lambda ^{H\phi }_{1}\left( \phi ^{2}_{01}+\phi ^{2}_{02}\right) \nonumber \\&\quad +\left[ \lambda ^{H\phi }_{II}\frac{v_{2}}{v_{1}}+\lambda ^{H\phi }_{III}\frac{v_{3}}{2v_{1}} \right] \phi _{01}\phi _{03}+\lambda ^{H\phi }_{9}\phi ^{2}_{03}. \end{aligned}$$
(8)
$$\begin{aligned} 2\mu ^{2}_{1}&=-2\gamma \left( v^{2}_{1}+v^{2}_{2}\right) -3\lambda _{4}\frac{v_{3}}{v_{1}}\left( v^{2}_{1}-v^{2}_{2}\right) -\lambda v^{2}_{3}\nonumber \\&\quad +\lambda ^{H\phi }_{1}\left( \phi ^{2}_{01}+\phi ^{2}_{02}\right) +\lambda ^{H\phi }_{I}\frac{v_{3}}{2v_{2}}\left( \phi ^{2}_{01}-\phi ^{2 }_{02}\right) \nonumber \\&\quad +\lambda ^{H\phi }_{II}\frac{v_{1}}{v_{2}} \phi _{01}\phi _{03}+\lambda ^{H\phi }_{9}\phi ^{2}_{03}. \end{aligned}$$
(9)
$$\begin{aligned} 2\mu ^{2}_{2}&=-\lambda _{4}\frac{v_{2}}{v_{3}}\left( 3v^{2}_{1}-v^{2}_{2}\right) -\lambda \left( v^{2}_{1}+v^{2}_{2}\right) -2\lambda _{8}v^{2}_{3}\nonumber \\&\quad +\lambda ^{H\phi }_{I}\frac{v_{2}}{2v_{3}}\left( \phi ^{2}_{01}\!-\!\phi ^{2 }_{02}\right) \!+\!\lambda ^{H\phi }_{8}\left( \phi ^{2}_{01}\!+\!\phi ^{2 }_{02}\right) \nonumber \\&\quad +\lambda ^{H\phi }_{III}\frac{v_{1}}{2v_{3}}\phi _{01}\phi _{03}+\lambda ^{H\phi }_{14}\phi ^{2}_{03}. \end{aligned}$$
(10)
$$\begin{aligned} 2\mu ^{2}_{1BL}&=-2\gamma _{BL}\left( \phi ^{2}_{01}+\phi ^{2}_{02}\right) - \lambda _{BL}\phi ^{2}_{03}\nonumber \\&\quad +\lambda ^{H \phi }_{1}\left( v^{2}_{1}+v^{2}_{2}\right) +\lambda ^{H \phi }_{I} v_{2}v_{3}+\lambda ^{H \phi }_{II}\frac{\phi _{03}}{\phi _{01}}v_{1}v_{2}\nonumber \\&\quad +\lambda ^{H \phi }_{8}v^{2}_{3}+\lambda ^{H \phi }_{III}\frac{\phi _{03}}{\phi _{01}}v_{1}v_{3}.\end{aligned}$$
(11)
$$\begin{aligned} 2\mu ^{2}_{1BL}&=-2\gamma _{BL}\left( \phi ^{2}_{01}+\phi ^{2}_{02}\right) - \lambda _{BL}\phi ^{2}_{03}+\lambda ^{H \phi }_{1}\left( v^{2}_{1}+v^{2}_{2}\right) \nonumber \\&\quad -\lambda ^{H \phi }_{I} v_{2}v_{3}+\lambda ^{H \phi }_{8}v^{2}_{3}. \end{aligned}$$
(12)
$$\begin{aligned} 2\mu ^{2}_{2BL}&=-\lambda _{BL}\left( \phi ^{2}_{01}+\phi ^{2}_{02}\right) -2\lambda ^{\phi }_{8}\phi ^{2}_{03}+\lambda ^{H \phi }_{II}\frac{\phi _{01}}{\phi _{03}}v_{1}v_{2}\nonumber \\&\quad +\lambda ^{H \phi }_{9}\left( v^{2}_{1}+v^{2}_{2}\right) +\lambda ^{H \phi }_{III}\frac{\phi _{01}}{2\phi _{03}}v_{1}v_{3}+\lambda ^{H\phi }_{14}v^{2}_{3}, \end{aligned}$$
(13)

where

$$\begin{aligned} \lambda&= \lambda _{5}+\lambda _{6}+2\lambda _{7},\qquad \gamma =\lambda _{1}+\lambda _{3}; \end{aligned}$$
(14)
$$\begin{aligned} \lambda _{BL}&= \lambda ^{\phi }_{5}+\lambda ^{\phi }_{6}+2\lambda ^{\phi }_{7},\qquad \gamma _{BL}=\lambda ^{\phi }_{1}+\lambda ^{\phi }_{3}; \end{aligned}$$
(15)
$$\begin{aligned} \lambda ^{H\phi }_{I}&=\lambda ^{H\phi }_{4}+\lambda ^{H\phi }_{6},\qquad \lambda ^{H\phi }_{II}=\lambda ^{H\phi }_{5}+\lambda ^{H\phi }_{7},\nonumber \\ \lambda ^{H\phi }_{III}&=\lambda ^{H\phi }_{10}+ \lambda ^{H\phi }_{11}+\lambda ^{H\phi }_{12}+ \lambda ^{H\phi }_{14}. \end{aligned}$$
(16)

It is not the purpose of this paper to analyze the scalar potential in detail, but some things can be noted. The potential of the three Higgs S3 model (S3–3H) has been analyzed in some detail in Refs. [93, 94]. In our case, the breaking of the \(U(1)_{B-L}\) symmetry at a scale larger than the electroweak scale \(\phi _{0i}>> v_i\) will give rise to a massive \(Z_{B-L}\) gauge boson. After electroweak symmetry breaking the remaining degrees of freedom from the \(U(1)_{B-L}\) part will mix with the ones coming from the electroweak doublets \(H_i\), which transform under \(\mathbf {S}_\mathbf{3}\), and which give rise to a number of neutral, charged and pseudoscalar Higgs bosons, one of which will correspond to the SM one. This will provide two scales in the model, and some of the scalars will be naturally heavier than the others, but it is clear from the potential that there will also be mixing among them. This rich scalar structure will give rise to FCNCs, a detailed analysis of which, together with the experimental Higgs bounds, will place constraints on the available parameter space of the model. In the limit where the couplings of the \(B-L\) part go to zero the S3–3H model will be recovered. In general, the phenomenology of the models will be different, not only because of the extra heavy scalar sector and a \(Z_{B-L}\) boson, but also because there may also be mixing of the B–L sector and the S3–3H one.

To get an idea what possible scenarios could be for the scalar particles in our model let us consider the limit that there is no mixing between the B–L part and the S3–3H one. This situation will correspond to two separate sectors at very different scales \(\phi _{0i}>>v_{i}\). As already mentioned, in the limit where the couplings of the B–L part go to zero the S3–3H model will be recovered. In the S3–3H model, after electroweak symmetry breaking, the scalar sector consists of three neutral scalars, \(h_0\), \(H_{1,2}\) one of which is identified with the Higgs boson of the SM (say \(H_2\)), four charged scalars, \(H_{1,2}^{\pm }\), and two pseudoscalars, \(A_{1,2}\). There are two scenarios possible. In one case the \(\lambda _4\) coupling is absent, which implies a continuous symmetry of the potential \(\text {SO}(2)\). Upon breaking of the electroweak symmetry this gives rise to a massless Goldstone boson \(h_0\) [90, 93], i.e. one of the three neutral scalars remains massless. The other two scalars \(H_{1,2}\) can be parameterized in a similar way to the two Higgs doublet model (2HDM) and a decoupling or alignment limit defined. This decoupling limit refers to the fact that only one of the two scalars will be coupled to the gauge bosons, and it is identified with the SM one; the other scalar is orthogonal to it and has no couplings with the gauge bosons. But, in contrast to the 2HDM, this decoupling limit does not imply that the decoupled scalar is necessarily heavier than the other one. On the other hand, if the \(\lambda _4\) term is present, the continuous \(\text {SO}(2)\) symmetry is not there; instead upon electroweak symmetry breaking there is a residual \(Z_2\) symmetry left from the breaking of \(S_3\). Now the three neutral scalars acquire mass, but one of them is not coupled to the gauge bosons, due to the \(Z_2\) symmetry, and in the other two the decoupling limit described above applies [93]. Although the three neutral scalars can have masses in the same energy range, a study from a model with \(\mathbf {S}_\mathbf{3}\) symmetry and four Higgs doublets, where the fourth one is inert and the couplings of the other three are like in S3–3H, shows that upon certain considerations it is possible to satisfy the Higgs bounds and have regions in parameter space that are compatible with the latest experimental results [95].

Examination of the B–L part with no mixing terms shows that it resembles the situation of the S3–3H model with the \(\lambda _4\) term set to zero, that is, there exists an \(\text {SO}(2)\) symmetry in this sector too. In this case, after the \(\phi 's\) acquire vevs, besides the massive gauge boson \(Z_{B-L}\), there will be three neutral scalars, one of them massless, and two pseudoscalars. The massive states will be heavier than in the S3–3H part, since we have assumed \(\phi _i>>v_{i}\), giving two disconnected scalar sectors and one candidate to the SM Higgs boson in the decoupling limit described above, plus the massless scalar. In this case, since the \(\lambda _4^{\phi }\) coupling is forbidden by the \(Z_2^e\) symmetry, the only way to avoid the Goldstone boson is to break this symmetry softly.

Upon considering the mixing of the B–L part and the S3–3H one, the \(\text {SO}(2)\) or \(Z_2\) symmetries of the potential will not be present, they will be broken by the mixing terms. In general, all the scalars will acquire masses. Since the mixing terms have to be very small to comply with the experimental bounds, it will still be possible to define a decoupling limit in the sense described above, where one of the neutral scalars of the S3–3H part can be identified with the SM one, although the expressions will be more complicated due to the mixing terms. The viability of this decoupling limit will impose constraints on the possible values of these mixed couplings.

Moving to the fermionic sector, the Yukawa Lagrangian in the standard basis is

$$\begin{aligned} -{\mathcal {L}}_{Y}&={\bar{q}}_{i L} \left( \mathbf{M}_{q} \right) _{ij}q_{j R}+{\bar{\ell }}_{i L} \left( \mathbf{M}_{\ell }\right) _{ij}\ell _{j R} \nonumber \\&\quad +\dfrac{1}{2}{\bar{\nu }}_{i L}\left( \mathbf{M}_{\nu }\right) _{ij}\nu ^{c}_{j L }+\dfrac{1}{2}{\bar{N}}^{c}_{i}\left( \mathbf{M}_{R}\right) _{ij}N_{j }+h.c. \end{aligned}$$
(17)

where the type I see-saw mechanism has been realized, \(\mathbf{M}_{\nu }=-\mathbf{M}_{D} \mathbf{M}^{-1}_{R} \mathbf{M}^{T}_{D}\). From Eq. (5), the mass matrices have the following form:

$$\begin{aligned}&\mathbf{M}_{q}=\begin{pmatrix} a_{q}+b^{\prime }_{q} &{} b_{q} &{} c_{q} \\ b_{q} &{} a_{q}-b^{\prime }_{q} &{} c^{\prime }_{q} \\ f_{q} &{} f^{\prime }_{q} &{} g_{q} \end{pmatrix},\nonumber \\&\mathbf{M}_{\ell }=\begin{pmatrix} a_{\ell } &{} 0 &{} 0 \\ 0 &{} b_{\ell }+c_{\ell } &{} 0 \\ 0 &{} 0 &{} b_{\ell }-c_{\ell } \end{pmatrix}, \nonumber \\&\mathbf{M}_{R}=\begin{pmatrix} a_{R} &{} b_{R} &{} b^{\prime }_{R} \\ b_{R} &{} c_{R} &{} 0 \\ b^{\prime }_{R} &{} 0 &{} c_{R} \end{pmatrix}, \end{aligned}$$
(18)

where the \(q= u, d\) and \(\ell =e, D\). Explicitly, the matrix elements for the quarks and lepton sectors are given as

$$\begin{aligned}&a_{q}=y^{q}_{2}\langle H_{3}\rangle ,\quad b^{\prime }_{q}=y^{q}_{1} \langle H_{2}\rangle , \quad b_{q}=y^{q}_{1} \langle H_{1}\rangle ,\nonumber \\&c_{q}=y^{q}_{3} \langle H_{1}\rangle ,\quad c^{\prime }_{q}=y^{q}_{3} \langle H_{2}\rangle ,\quad f_{q}=y^{q}_{4} \langle H_{1}\rangle ;\nonumber \\&f^{\prime }_{q}= y^{q}_{4} \langle H_{2}\rangle ,\quad g_{q}=y^{q}_{5} \langle H_{3}\rangle ,\quad a_{\ell }=y^{\ell }_{1}\langle H_{3}\rangle ,\nonumber \\&b_{\ell }=y^{\ell }_{3}\langle H_{3}\rangle ,\quad c_{\ell }=y^{\ell }_{2}\langle H_{2}\rangle ,\quad a_{R}=y^{N}_{1} \langle \phi _{3}\rangle ;\nonumber \\&b_{R}=y^{N}_{2} \langle \phi _{1}\rangle ,\quad b^{\prime }_{R}=y^{N}_{2} \langle \phi _{2}\rangle ,\quad c_{R}=y^{N}_{3}\langle \phi _{3}\rangle . \end{aligned}$$
(19)

Here, it is convenient to remark that the number of Yukawa couplings that appear in the flavored B–L model is reduced to half in comparison with the flavored LRSM scenario [86].

3 Masses and mixings

3.1 Quark sector: NNI textures

The quark mass matrix, \(\mathbf{M}_{q}\), has already been obtained by means of the \(\mathbf{S}_{3}\) flavor symmetry [54,55,56,57,58,59,60, 65,66,67,68]. However, it is important to point out, as shown in [68], that this mass matrix possesses implicitly a kind of NNI textures,Footnote 1 but with one more free parameter than the canonical NNI ones [13,14,15,16], which only contain four. This is relevant, since it shows that NNI textures are hidden in the \(\mathbf{S}_{3}\) flavor symmetry [68], so it may not be necessary to use larger discrete groups, for example the \(\mathbf{Q}_{6}\) symmetry [96,97,98,99,100,101,102], to understand the mixing by hierarchical mass matrices, although extending the symmetry group may be necessary in other contexts.

Having emphasized the above fact, we obtain simultaneously the NNI textures and the broken \(\mu \leftrightarrow \tau \) symmetry, in the quark and lepton sector, respectively, within an \(\mathbf{S}_{3}\) flavored B–L gauge model. Although the NNI textures and the \(\mu \leftrightarrow \tau \) symmetry have been studied quite widely in the literature, neither has been explored in the present theoretical framework.

Let us comment on how to get the NNI textures (for more details on other textures see cite [68]). Take the quark mass matrix, \(\mathbf{M}_{q}\), that is diagonalized by the unitary matrices \(\mathbf{U}_{q(R, L)}\) such that \(\hat{\mathbf{M}}_{q}=\text {diag.}\left( m_{q_{1}}, m_{q_{2}}, m_{q_{3}}\right) =\mathbf{U}^{\dagger }_{q L}{} \mathbf{M}_{q}{} \mathbf{U}_{q R}\). Now, we apply the rotation \(\mathbf{U}_{\theta }\) (\(\mathbf{U}_{q(R, L)}=\mathbf{U}_{\theta } \mathbf{u}_{q (R, L)}\)) to \(\mathbf{M}_{q}\) to obtain

$$\begin{aligned}&\mathbf{m}_{q}=\mathbf{U}^{T}_{\theta }{} \mathbf{M}_{q} \mathbf{U}_{\theta }= \begin{pmatrix} a_{q} &{} \frac{2}{\sqrt{3}}b^{\prime }_{q} &{} 0 \\ \frac{2}{\sqrt{3}}b^{\prime }_{q} &{} a_{q} &{} \frac{2}{\sqrt{3}}c^{\prime }_{q} \\ 0 &{} \frac{2}{\sqrt{3}}f^{\prime }_{q} &{} g_{q} \end{pmatrix},\nonumber \\&\mathbf{U}_{\theta }=\begin{pmatrix} \cos {\theta } &{} \sin {\theta } &{} 0 \\ -\sin {\theta } &{} \cos {\theta } &{} 0 \\ 0 &{} 0 &{} 1 \end{pmatrix}, \end{aligned}$$
(20)

with the following conditions:

$$\begin{aligned}&\tan {\theta }=\frac{c_{q}}{c^{\prime }_{q}}=\frac{f_{q}}{f^{\prime }_{q}}=\frac{\langle H_{1}\rangle }{\langle H_{2}\rangle }\qquad \text {and}\nonumber \\&\tan {2\theta }=\frac{b^{\prime }_{q}}{b_{q}}=\frac{\langle H_{2}\rangle }{\langle H_{1}\rangle }~, \end{aligned}$$
(21)

which give us the relation \(\langle H_{2}\rangle =\pm \sqrt{3} \langle H_{1}\rangle \),Footnote 2 then \(\theta =\pi /6\) as was shown in [68]. Notice that \(\mathbf{m}_{q}\) can be written as

$$\begin{aligned} \mathbf{m}_{q}=a_{q}{} \mathbf{1}+\overbrace{\begin{pmatrix} 0 &{} \frac{2}{\sqrt{3}}b^{\prime }_{q} &{} 0 \\ \frac{2}{\sqrt{3}}b^{\prime }_{q} &{} 0 &{} \frac{2}{\sqrt{3}}c^{\prime }_{q} \\ 0 &{} \frac{2}{\sqrt{3}}f^{\prime }_{q} &{} g_{q}-a_{q} \end{pmatrix}}^{\mathbf{m^{\prime }_{q}}}. \end{aligned}$$
(22)

If \(\mathbf{m}_{q}\) was a hermitian matrix (\(f^{\prime }_{q}=c^{\prime *}_{q}\)), this would imply that \(\mathbf{u}^{\dagger }_{q L}\mathbf{u}_{q R}=\mathbf{1}\) and \({\mathbf{m}}^{\prime }_{\mathbf{q}}\) would be like the Fritzsch textures, so that to diagonalize \(\mathbf{m}_{q}\) is equivalent to do so in \(\mathbf{m}^{\prime }_{q}\); this means \(\hat{\mathbf{M}}^{\prime }_{q}=\text {diag.}\left( m_{q_{1}}-a_{q}, m_{q_{2}}-a_{q}, m_{q_{3}}-a_{q}\right) =\mathbf{u}^{\dagger }_{q L}\mathbf{m}^{\prime }_{q}{} \mathbf{u}_{q R}\). However, in the present framework, \(\mathbf{m}_{q}\) is not hermitian and \(a_{q}\ne 0\), in general, so an exact diagonalization of \(\mathbf{m}_{q}\) might produce a different result from the one expected if \(a_{q}=0\) (in this benchmark the NNI textures appear). Along with this, if \(a_{q}\) was considered as a perturbation to \(\mathbf{m}^{\prime }_{q}\), one would expect a modified NNI texture. Here, for simplicity, in order to not include extra discrete symmetries to prohibit the second term in the Yukawa mass term (see Eq. (5)), which gives rise to \(a_{q}\), let us adopt the benchmark where \(a_{q}=0\), which means that \(y^{q}_{2}=0\). In this way, the NNI textures appear in the quark mass matrix so these hierarchical matrices fit the CKM matrix very well.

In this framework, we find the \(\mathbf{u}_{f R}\) and \(\mathbf{u}_{f L}\) unitary matrices that diagonalize \(\mathbf{m}_{q}\). Then we must build the bilineal forms: \({\hat{\mathbf{M}}}_{q} {\hat{\mathbf{M}}}^{\dagger }_{q}=\mathbf{u}^{\dagger }_{q L} \mathbf{m}_{q} \mathbf{m }^{\dagger }_{q} \mathbf{u}_{q L}\) and \({\hat{\mathbf{M}}}^{\dagger }_{q} {\hat{\mathbf{M}}}_{q}=\mathbf{u}^{\dagger }_{q R} \mathbf{m}^{\dagger }_{q} \mathbf{m }_{q} \mathbf{u}_{q R}\); however, in this work we will only need to obtain the \(\mathbf{u}_{q L}\) left-handed matrix which occurs in the CKM matrix. This is given by \(\mathbf{u}_{q L}=\mathbf{Q}_{q L}\mathbf{O}_{q L}\) where the former matrix contains the CP-violating phases, \( \mathbf{Q}_{q} = \text {diag} \left( 1, \exp i\eta _{q_{2}}, \exp i\eta _{q_{3}} \right) \), that comes from \(\mathbf{m}_{q} \mathbf{m }^{\dagger }_{q}\). \(\mathbf{O}_{q L}\) is a real orthogonal matrix and it is parametrized as

$$\begin{aligned} \mathbf{O}_{q L}= \begin{pmatrix} -\sqrt{\dfrac{{\tilde{m}}_{q_{2}} (\rho ^{q}_{-}-R^{q}) K^{q}_{+}}{4 y_{q} \delta ^{q}_{1} \kappa ^{q}_{1} }} &{} -\sqrt{\dfrac{{\tilde{m}}_{q_{1}} (\sigma ^{q}_{+}-R^{q}) K^{f}_{+}}{4 y_{q} \delta ^{q}_{2} \kappa ^{q}_{2} }} &{} \sqrt{\dfrac{{\tilde{m}}_{q_{1}} {\tilde{m}}_{q_{2}} (\sigma ^{q}_{-}+R^{q}) K^{q}_{+}}{4 y_{q} \delta ^{q}_{3} \kappa ^{q}_{3} }} \\ -\sqrt{\dfrac{{\tilde{m}}_{q_{1}} \kappa ^{q}_{1} K^{q}_{-}}{\delta ^{q}_{1}(\rho ^{q}_{-}-R^{q}) }} &{} \sqrt{\dfrac{{\tilde{m}}_{q_{2}} \kappa ^{q}_{2} K^{q}_{-}}{\delta ^{q}_{2}(\sigma ^{q}_{+}-R^{q}) }} &{} \sqrt{\dfrac{\kappa ^{q}_{3} K^{q}_{-}}{\delta ^{q}_{3}(\sigma ^{q}_{-}+R^{q}) }} \\ \sqrt{\dfrac{{\tilde{m}}_{q_{1}} \kappa ^{q}_{1}(\rho ^{q}_{-}-R^{q})}{2 y_{q}\delta ^{q}_{1}}} &{} -\sqrt{\dfrac{{\tilde{m}}_{q_{2}} \kappa ^{q}_{2}(\sigma ^{q}_{+}-R^{q})}{2 y_{q}\delta ^{q}_{2}}} &{} \sqrt{\dfrac{\kappa ^{q}_{3}(\sigma ^{q}_{-}+R^{q})}{2 y_{q}\delta ^{q}_{3}}} \end{pmatrix} \end{aligned}$$
(23)

with

$$\begin{aligned}&\rho ^{q}_{\pm }\equiv 1+{\tilde{m}}^{2}_{q_{2}}\pm {\tilde{m}}^{2}_{q_{1}}-y^{2}_{q},\quad \sigma ^{q}_{\pm }\equiv 1-{\tilde{m}}^{2}_{q_{2}}\pm ({\tilde{m}}^{2}_{q_{1}}-y^{2}_{q}),\nonumber \\&\delta ^{q}_{(1, 2)}\equiv (1-{\tilde{m}}^{2}_{q_{(1, 2)}})({\tilde{m}}^{2}_{q_{2}}-{\tilde{m}}^{2}_{q_{1}});\nonumber \\&\delta ^{q}_{3}\equiv (1-{\tilde{m}}^{2}_{q_{1}})(1-{\tilde{m}}^{2}_{q_{2}}),\quad \kappa ^{q}_{1} \equiv {\tilde{m}}_{q_{2}}-{\tilde{m}}_{q_{1}}y_{q},\nonumber \\&\kappa ^{q}_{2}\equiv {\tilde{m}}_{q_{2}}y_{q}-{\tilde{m}}_{q_{1}},\quad \kappa ^{q}_{3}\equiv y_{q}-{\tilde{m}}_{q_{1}}{\tilde{m}}_{q_{2}};\nonumber \\&R^{q}\equiv \sqrt{\rho ^{q 2}_{+}-4({\tilde{m}}^{2}_{q_{2}}+{\tilde{m}}^{2}_{q_{1}}+{\tilde{m}}^{2}_{q_{2}}{\tilde{m}}^{2}_{q_{1}}-2{\tilde{m}}_{q_{1}}{\tilde{m}}_{q_{2}}y_{q})},\nonumber \\&K^{q}_{\pm } \equiv y_{q}(\rho ^{q}_{+}\pm R^{q})-2{\tilde{m}}_{q_{1}}{\tilde{m}}_{q_{2}}. \end{aligned}$$
(24)

In the above expressions, all the parameters have been normalized by the heaviest physical quark mass, \(m_{q_{3}}\). Along with this, from the above parametrization, \(y_{q}\equiv \vert g_{q}\vert /m_{q_{3}}\) is the only dimensionless free parameter that cannot be fixed in terms of the physical masses; but it is constrained by \(1>y_{q}>{\tilde{m}}_{q_{2}}>{\tilde{m}}_{q_{1}}\). Therefore, the left-handed mixing matrix that occurs in the CKM matrix is given by \(\mathbf{U}_{q L}= \mathbf{U}_{\theta }{} \mathbf{Q}_{q}{} \mathbf{O}_{q L}\) where \(q=u, d\). Finally, the CKM mixing matrix is written as

$$\begin{aligned} \mathbf{V}_\text {PMNS}= & {} \mathbf{O}^{T}_{u L}{} \mathbf{P}_{q} \mathbf{O}_{d L}, \quad \mathbf{P}_{q}=\mathbf{Q}^{\dagger }_{u}{} \mathbf{Q}_{d}\nonumber \\= & {} \text {diag.}\left( 1, e^{i\eta _{q_{1}}}, e^{i\eta _{q_{2}}} \right) . \end{aligned}$$
(25)

This CKM mixing matrix has four free parameters, namely \(y_{u}\), \(y_{d}\), and two phases \(\eta _{q_{1}}\) and \(\eta _{q_{2}}\), which could be obtained numerically; in this work, the physical quark masses (at \(m_{Z}\) scale) will be taken (just central values) as inputs: \(m_{u}=1.45~\text {MeV}\), \(m_{c}=635~\text {MeV}\), \(m_{t}=172.1~\text {GeV}\) and \(m_{d}=2.9~\text {MeV}\), \(m_{s}=57.7~\text {MeV}\), \(m_{b}=2.82~\text {GeV}\) [103]. In the following, a naive \(\chi ^{2}\) analysis will be performed to tune the free parameters. Then we define

$$\begin{aligned}&\chi ^{2}\left( y_{u},y_{d}, \eta _{q_{1}}, \eta _{q_{2}}\right) \nonumber \\&\quad =\frac{\left( \left| V\text {th}_\text {ud}\right| -V^\text {ex}_\text {ud}\right) ^{2}}{\sigma ^{2}_\text {ud}}+\frac{\left( \left| V\text {th}_\text {us}\right| -V^\text {ex}_\text {us}\right) ^{2}}{\sigma ^{2}_\text {us}}\nonumber \\&\qquad +\frac{\left( \left| V\text {th}_\text {ub}\right| -V^\text {ex}_\text {ub}\right) ^{2}}{\sigma ^{2}_\text {ub}}+\frac{\left( \left| J\text {th}\right| -J^\text {ex}\right) ^{2}}{\sigma ^{2}_{J}}, \end{aligned}$$
(26)

where the experimental values are given as [104]

$$\begin{aligned}&V^\text {ex}_\text {ud}=0.97434^{+0.00011}_{-0.00012},\\&V^\text {ex}_\text {us}=0.22506\pm 0.00050,\quad V^\text {ex}_\text {ub}=0.00357\pm 0.00015, \end{aligned}$$

and

$$\begin{aligned}&J\text {th}=Im\left[ V\text {th}_\text {us}~V\text {th}_\text {cb}~ V^{*\text {th}}_\text {cs}~ V^{*\text {th}}_\text {ub} \right] ,\\&J^\text {ex}=3.04^{+0.21}_{-0.20}\times 10^{-5}. \end{aligned}$$

Then we obtain the following values for the free parameters that fit the mixing values up to \(2\sigma \):

$$\begin{aligned}&y_{u}=0.996068,\quad y_{d}=0.922299,\nonumber \\&\eta _{q_{1}}=4.48161,\quad \eta _{q_{2}}=3.64887; \end{aligned}$$
(27)

with these values one obtains

$$\begin{aligned}&\left| V\text {th}_{CKM}\right| =\begin{pmatrix} 0.97433 &{} 0.22505 &{} 0.00356 \\ 0.22490 &{} 0.97359 &{} 0.03926 \\ 0.00901 &{} 0.03831 &{} 0.99922 \end{pmatrix},\nonumber \\&J\text {th}=3.04008\times 10^{-5}. \end{aligned}$$
(28)

As can be seen, these values are in good agreement with the experimental data, this is not a surprise since the NNI textures work quite well in the quark sector.

3.2 Lepton sector: broken \(\mu \leftrightarrow \tau \) symmetry

As we already mentioned, the lepton mass matrices have already been diagonalized in the framework of the LRSM [86], where a systematic study was realized on the mixing angles. Therefore, we will just mention the relevant points and comment on the results.

The \(\mathbf{M}_{e}\) mass matrix is complex and diagonal, then one can identify straightforwardly the physical masses; since the \(\mathbf{M}_{e}\) mass matrix is diagonalized by \(\mathbf{U}_{e L}=\mathbf{S}_{23}{} \mathbf{P}_{e}\) and \(\mathbf{U}_{e R}=\mathbf{S}_{23}\mathbf{P}^{\dagger }_{e}\), this is, \({\hat{\mathbf{M}}_{e}}=\text {Diag.}(\vert m_{e}\vert , \vert m_{\mu }\vert ,\vert m_{\tau }\vert )=\mathbf{U}^{\dagger }_{e L}\mathbf{M}_{e}{} \mathbf{U}_{e R} =\mathbf{P}^{\dagger }{} \mathbf{m}_{e}{} \mathbf{P}^{\dagger }_{e}\) with \(\mathbf{m}_{e}=\mathbf{S}^{T}_{23}{} \mathbf{M}_{e}{} \mathbf{S}_{23}\). After factorizing the phases, we have \(\mathbf{m}_{e}=\mathbf{P}_{e}{\bar{\mathbf{m}}}_{\mathbf{e}}{} \mathbf{P}_{e}\)  where

$$\begin{aligned}&\mathbf{m}_{e}=\text {Diag.}(m_{e}, m_{\mu }, m_{\tau }),\quad \mathbf{S}_{23}=\begin{pmatrix} 1 &{} 0 &{} 0 \\ 0 &{} 0 &{} 1 \\ 0 &{} 1 &{} 0 \end{pmatrix},\nonumber \\&\mathbf{P}_{e}=\text {diag.}(e^{i\eta _{e}/2}, e^{i\eta _{\mu }/2}, e^{i\eta _{\tau }/2}) . \end{aligned}$$
(29)

As a result, one obtains \(\vert m_{e}\vert =\vert a_{e}\vert \), \(\vert m_{\mu }\vert =\vert b_{e}-c_{e}\vert \) and \(\vert m_{\tau }\vert =\vert b_{e}+c_{e}\vert \).

On the other hand, the effective neutrino mass matrix \(\mathbf{M}_{\nu }=\mathbf{M}_{D}\mathbf{M}^{-1}_{R}{} \mathbf{M}^{T}_{D}\) is given by

$$\begin{aligned}&\mathbf{M}_{\nu }=\begin{pmatrix} \mathcal {X}a^{2}_{D}&{} -a_{D}\mathcal {Y}(b_{D}+c_{D}) &{} -a_{D}\mathcal {Y}(b_{D}-c_{D}) \\ -a_{D}\mathcal {Y}(b_{D}+c_{D})&{} \mathcal {W}(b_{D}+c_{D})^{2} &{} \mathcal {Z}(b^{2}_{D}-c^{2}_{D}) \\ -a_{D}\mathcal {Y}(b_{D}-c_{D})&{} \mathcal {Z}(b^{2}_{D}-c^{2}_{D}) &{} \mathcal {W}(b_{D}-c_{D})^{2} \end{pmatrix},\nonumber \\&\mathbf{M}^{-1}_{R}\equiv \begin{pmatrix} \mathcal {X}&{} -\mathcal {Y} &{} -\mathcal {Y} \\ -\mathcal {Y} &{} \mathcal {W} &{} \mathcal {Z} \\ -\mathcal {Y} &{} \mathcal {Z} &{} \mathcal {W} \end{pmatrix} , \end{aligned}$$
(30)

where the Dirac (\(\ell =D\)) and right-handed neutrino mass matrices are given in Eq. (18). In the latter mass matrix, we have assumed the vacuum alignment \(\langle \phi _{1}\rangle =\langle \phi _{2}\rangle \). Now, as a hypothesis, we will assume that \(b_{D}\) is larger than \(c_{D}\); in this way the effective mass matrix can be written as

$$\begin{aligned} \mathbf{M}_{\nu }\equiv \begin{pmatrix} m^{0}_{ee}&{} -m^{0}_{e\mu }(1+\epsilon ) &{} -m^{0}_{e\mu }(1-\epsilon ) \\ -m^{0}_{e\mu }(1+\epsilon )&{} m^{0}_{\mu \mu }(1+\epsilon )^{2} &{} m^{0}_{\mu \tau }(1-\epsilon ^{2}) \\ -m^{0}_{e\mu }(1-\epsilon )&{} m^{0}_{\mu \tau }(1-\epsilon ^{2}) &{} m^{0}_{\mu \mu }(1-\epsilon )^{2} \end{pmatrix} , \end{aligned}$$
(31)

where \(m^{0}_{ee}\equiv \mathcal {X}a^{2}_{D}\), \(m^{0}_{e\mu } \equiv \mathcal {Y}a_{D}b_{D}\), \(m^{0}_{\mu \mu }\equiv \mathcal {W}b^{2}_{D}\) and \(m^{0}_{\mu \tau }\equiv \mathcal {Z}b^{2}_{D}\) are complex. Here, \(\epsilon \equiv c_{D}/b_{D}\) is a complex parameter which will be considered as a perturbation to the effective mass matrix such that \(\vert \epsilon \vert \lll 1\). In order to softly break the \(\mu \leftrightarrow \tau \) symmetry, we require that \(\vert \epsilon \vert \le 0.3\), so we will neglect the \(\epsilon ^{2}\) quadratic terms in the above matrix hereafter and a perturbative diagonalization will be carried out.

In order to cancel the \(\mathbf{S}_{23}\) contribution, which comes from the charged lepton sector, we proceed as follows. We know that \(\hat{\mathbf{M}}_{\nu }=\text {diag.}(m_{\nu _{1}}, m_{\nu _{2}}, m_{\nu _{3}})=\mathbf{U}^{\dagger }_{\nu }{} \mathbf{M}_{\nu }{} \mathbf{U}^{*}_{\nu }\), then \(\mathbf{U}_{\nu }=\mathbf{S}_{23}{} \mathbf{\mathcal {U}_{\nu }}\) where the latter mixing matrix will be obtained below. Then \(\hat{\mathbf{M}}_{\nu }=\mathbf{\mathcal {U}^{\dagger }_{\nu }}{} \mathbf{\mathcal {M}_{\nu }}{\mathcal {U}^{*}_{\nu }}\) with

$$\begin{aligned} {\mathcal {M}_{\nu }}&\approx \begin{pmatrix} m^{0}_{e e} &{} -m^{0}_{e\mu } &{} -m^{0}_{e\mu } \\ -m^{0}_{e\mu } &{} m^{0}_{\mu \mu } &{} m^{0}_{\mu \tau } \\ -m^{0}_{e\mu } &{} m^{0}_{\mu \tau } &{} m^{0}_{\mu \mu } \end{pmatrix}\nonumber \\&\quad +\begin{pmatrix} 0 &{} m^{0}_{e \mu }~ \epsilon &{} -m^{0}_{e\mu } ~\epsilon \\ -m^{0}_{e\mu }~ \epsilon &{} -2 m^{0}_{\mu \mu }~ \epsilon &{} 0 \\ -m^{0}_{e\mu }~ \epsilon &{} 0 &{} 2 m^{0}_{\mu \mu }~ \epsilon \end{pmatrix}\nonumber \\&={\mathcal {M}^{0}_{\nu }}+{\mathcal {M}^{\epsilon }_{\nu }}. \end{aligned}$$
(32)

Notice that \({\mathcal {M}^{0}_{\nu }}\) possesses the \(\mu \)–\(\tau \) symmetry and this is diagonalized by

$$\begin{aligned} {\mathcal {U}}^{0}_{\nu }=\begin{pmatrix} \cos {\theta }_{\nu }~e^{i\eta _{\nu }} &{} \sin {\theta }_{\nu }~e^{i\eta _{\nu }} &{} 0 \\ -\frac{\sin {\theta }_{\nu }}{\sqrt{2}}&{} \frac{\cos {\theta }_{\nu }}{\sqrt{2}} &{} -\frac{1}{\sqrt{2}} \\ -\frac{\sin {\theta }_{\nu }}{\sqrt{2}}&{} \frac{\cos {\theta }_{\nu }}{\sqrt{2}} &{} \frac{1}{\sqrt{2}} \end{pmatrix} , \end{aligned}$$
(33)

where the matrix elements \({\mathcal {M}^{0}_{\nu }}= {\mathcal {U}}^{0}_{\nu } \hat{\mathbf{M}}^{0}_{\nu } {\mathcal {U}}^{0 T}_{\nu } \) are written as

$$\begin{aligned} m^{0}_{e e}&=(m^{0}_{\nu _{1}}\cos ^{2}{\theta }_{\nu }+m^{0}_{\nu _{2}}\sin ^{2}{\theta }_{\nu })e^{2i\eta _{\nu }},\nonumber \\ -m^{0}_{e\mu }&=\frac{1}{\sqrt{2}}\cos {\theta }_{\nu }\sin {\theta }_{\nu }(m^{0}_{\nu _{2}}-m^{0}_{\nu _{1}})e^{i\eta _{\nu }};\nonumber \\ m^{0}_{\mu \mu }&=\frac{1}{2}(m^{0}_{\nu _{1}}\sin ^{2}{\theta }_{\nu }+m^{0}_{\nu _{2}}\cos ^{2}{\theta }_{\nu }+m^{0}_{\nu _{3}}),\nonumber \\ m^{0}_{\mu \tau }&=\frac{1}{2}(m^{0}_{\nu _{1}}\sin ^{2}{\theta }_{\nu }+m^{0}_{\nu _{2}}\cos ^{2}{\theta }_{\nu }-m^{0}_{\nu _{3}}). \end{aligned}$$
(34)

Including the perturbation, \({\mathcal {M}^{\epsilon }_{\nu }}\), applying \({\mathcal {U}^{0}_{\nu }}\) one gets \({\mathcal {M}}_{\nu }={ \mathcal {U}^{0\dagger }_{\nu }}({\mathcal {M}^{0}_{\nu }}+{ \mathcal {M}^{\epsilon }_{\nu }}){\mathcal {U}^{0 *}_{\nu }}\). Explicitly

$$\begin{aligned}&{\mathcal {M}}_{\nu }=\text {Diag.}(m^{0}_{\nu _{1}}, m^{0}_{\nu _{2}}, m^{0}_{\nu _{3}})\nonumber \\&\quad +\begin{pmatrix} 0 &{} 0 &{}-\sin {\theta _{\nu }}(m^{0}_{\nu _{3}}+m^{0}_{\nu _{1}})~\epsilon \\ 0 &{} 0 &{} \cos {\theta _{\nu }}(m^{0}_{\nu _{3}}+m^{0}_{\nu _{2}})~\epsilon \\ -\sin {\theta _{\nu }}(m^{0}_{\nu _{3}}+m^{0}_{\nu _{1}})~\epsilon &{} \cos {\theta _{\nu }}(m^{0}_{\nu _{3}}+m^{0}_{\nu _{2}})~\epsilon &{} 0 \end{pmatrix} . \end{aligned}$$
(35)

The contribution of the second matrix to the mixing one is given by

$$\begin{aligned} {\mathcal {U}}^{\epsilon }_{\nu }\approx \begin{pmatrix} N_{1}&{} 0 &{} -N_{3}\sin {\theta }~r_{1}~\epsilon \\ 0 &{} N_{2} &{} N_{3}\cos {\theta _{\nu }}~r_{2}~\epsilon \\ N_{1}\sin {\theta _{\nu }}~r_{1}~\epsilon &{} -N_{2}\cos {\theta _{\nu }}~r_{2}~\epsilon &{} N_{3} \end{pmatrix}, \end{aligned}$$
(36)

where \(N_{1}\), \(N_{2}\) and \(N_{3}\) are the normalization factors, which are given as

$$\begin{aligned}&N_{1}=\frac{1}{\sqrt{1+\sin ^{2}{\theta _{\nu }}\vert r_{1}\epsilon \vert ^{2}}} ,\nonumber \\&N_{2}=\frac{1}{\sqrt{1+\cos ^{2}{\theta _{\nu }}\vert r_{2}\epsilon \vert ^{2}}},\nonumber \\&N_{3}=\frac{1}{\sqrt{1+\sin ^{2}{\theta _{\nu }}\vert r_{1}\epsilon \vert ^{2}+\cos ^{2}{\theta _{\nu }}\vert r_{2}\epsilon \vert ^{2}}}, \end{aligned}$$
(37)

with \(r_{(1, 2)}\equiv (m^{0}_{\nu _{3}}+m^{0}_{\nu _{(1, 2)}})/(m^{0}_{\nu _{3}}-m^{0}_{\nu _{(1, 2)}})\). Finally, the effective mass matrix given in Eq. (31) is diagonalized approximately by \(\mathbf{U}_{\nu }\approx \mathbf{S}_{23}{ \mathcal {U}^{0}_{\nu }{\mathcal {U}^{\epsilon }_{\nu }}}\). Therefore, the theoretical PMNS mixing matrix is written as \(V_{PMNS}=\mathbf{U}^{\dagger }_{e L}{} \mathbf{U}_{\nu }=\mathbf{P}^{\dagger }_{e}\mathbf{\mathcal {U}^{0}_{\nu }{\mathcal {U}^{\epsilon }_{\nu }}}\). Explicitly,

$$\begin{aligned}&\mathbf{V}_{PMNS}\nonumber \\&\quad =\mathbf{P}^{\prime \dagger }_{e}\begin{pmatrix} \cos {\theta _{\nu }}N_{1} &{} \sin {\theta _{\nu }}N_{2} &{} \sin {2\theta _{\nu }}\frac{N_{3}}{2}(r_{2}-r_{1})~\epsilon \\ -\frac{\sin {\theta _{\nu }}}{\sqrt{2}}N_{1}(1+r_{1}~\epsilon )&{} \frac{\cos {\theta _{\nu }}}{\sqrt{2}}N_{2}(1+r_{2}~\epsilon ) &{} -\frac{N_{3}}{\sqrt{2}}\left[ 1-\epsilon ~r_{3}\right] \\ -\frac{\sin {\theta _{\nu }}}{\sqrt{2}}N_{1}(1-r_{1}~\epsilon )&{} \frac{\cos {\theta _{\nu }}}{\sqrt{2}}N_{2}(1-r_{2}~\epsilon ) &{} \frac{N_{3}}{\sqrt{2}}\left[ 1+\epsilon ~ r_{3}\right] \end{pmatrix} , \end{aligned}$$
(38)

where the Dirac phase, \(\eta _{\nu }\), has been factorized in the first entry of \(\mathbf{P}^{\prime \dagger }_{e}\) and \(r_{3}\equiv r_{2}\cos ^{2}{\theta _{\nu }}+r_{1}\sin ^{2}{\theta _{\nu }}\). On the other hand, comparing the magnitude of entries in the \(\mathbf{V}_{PMNS}\) with the mixing matrix in the standard parametrization of the PMNS, we obtain the following expressions for the lepton mixing angles:

$$\begin{aligned} \sin ^{2}{\theta }_{13}&=\vert \mathbf{V}_{13}\vert ^{2} =\frac{\sin ^{2}{2\theta _{\nu }}}{4}N^{2}_{3}\vert \epsilon \vert ^{2}~\vert r_{2}-r_{1} \vert ^{2};\nonumber \\ \sin ^{2}{\theta }_{23}&=\dfrac{\vert \mathbf{V}_{23}\vert ^{2}}{1-\vert \mathbf{V}_{13}\vert ^{2}}=\dfrac{N^{2}_{3}}{2}\frac{\vert 1-\epsilon ~r_{3} \vert ^{2}}{1- \sin ^{2}{\theta _{13}}},\nonumber \\ \sin ^{2}{\theta _{12}}&=\dfrac{\vert \mathbf{V}_{12}\vert ^{2}}{1-\vert \mathbf{V}_{13}\vert ^{2}}= \dfrac{N^{2}_{2}\sin ^{2}{\theta _{\nu }}}{1-\sin ^{2}{\theta }_{13}}. \end{aligned}$$
(39)

Notice that, in general, the reactor and atmospheric angles depend strongly on the active neutrino masses and therefore on the Majorana phases; also the reactor angle depends on the magnitude of the parameter \(\epsilon \) but the atmospheric one has a clear dependency on the \(\epsilon \) phase, which turns out to be relevant for reaching the allowed value.

Fig. 1
figure 1

From left to right: the atmospheric angle versus a the reactor angle, b \(\vert m^0_{\nu _{3}}\vert \) and \(\left| \epsilon \right| \), and c \(m_{ee}\) versus \(\vert m^0_{\nu _{3}}\vert \) and \(\left| \epsilon \right| \). Inverted hierarchy: blue and green points stand for \(\left| m_{\nu _{3}}\right| \) and \(\left| \epsilon \right| \), respectively. The dot-dashed, dashed and thick lines stand for \(1~\sigma \), \(2~\sigma \) and \(3~\sigma \) of C.L.

Full size image
Fig. 2
figure 2

From left to right: the atmospheric angle versus a the reactor angle, b \(m_0\) and \(\left| \epsilon \right| \), and c \(m_{ee}\) versus \(m_0\) and \(\left| \epsilon \right| \). Degenerate hierarchy: blue and green points stand for \(m_{0}\) and \(\left| \epsilon \right| \), respectively. The dot-dashed, dashed and thick lines stand for \(1~\sigma \), \(2~\sigma \) and \(3~\sigma \) of C.L.

Full size image

In particular, as shown in [86], in the regime of a soft breaking of the \(\mu \leftrightarrow \tau \) symmetry, as one can see from Eq. (39) that \(\theta _{12}\approx \theta _{\nu }\); then this parameter was considered as an input to determine the reactor and atmospheric angles. In these circumstances, the normal hierarchy was ruled out by experimental data. Along with this, the most viable cases for inverted and degenerate hierarchy were those where the CP parities in the neutrino masses are \({ \mathcal {M}^{0}_{\nu }}=\text {diag.}\left( m^{0}_{\nu _{1}}, m^{0}_{\nu _{2}}, m^{0}_{\nu _{3}} \right) =\text {diag.}\left( - \left| m^{0}_{\nu _{1}}\right| , \left| m^{0}_{\nu _{2}}\right| ,-\left| m^{0}_{\nu _{3}}\right| \right) \) where

$$\begin{aligned}&\vert m^{0}_{\nu _{2}}\vert =\sqrt{\Delta m^{2}_{13}+\Delta m^{2}_{21}+\vert m^{0 }_{\nu _{3}}\vert ^{2}},\nonumber \\&\vert m^{0}_{\nu _{1}}\vert =\sqrt{\Delta m^{2}_{13}+\vert m^{0}_{\nu _{3}}\vert ^{2}},\qquad \text {Inverted Hierarchy}\nonumber \\&\vert m^{0}_{\nu _{3}} \vert = \sqrt{\Delta m^{2}_{31}+m^{2}_{0}},\nonumber \\&\vert m^{0}_{\nu _{2}} \vert =\sqrt{\Delta m^{2}_{21}+m^{2}_{0}}, \qquad \text {Degenerate Hierarchy}, \end{aligned}$$
(40)

with \(m_{0}\gtrsim 0.1~\text {eV}\) as the common mass. At the same time, for the inverted (degenerate) hierarchy the associated phase of \(\epsilon =\left| \epsilon \right| e^{\alpha _{\epsilon }}\) has to be 0 (\(\pi \)) to reach the allowed values for the reactor and atmospheric angles.

In order to show that there is a parameter space for the \(\epsilon \), \(\left| m^{0}_{\nu _{3}} \right| \) and \(m_{0}\), we have made scattered plots where we require that the reactor and atmospheric angles lie within \(3\sigma \) of their experimental values, whereas the squared mass scales lie within \(2\sigma \) [105]. We allow \(\left| \epsilon \right| \) and \(\left| m^{0}_{\nu _{3}}\right| \) (\(m_{0}\)) to vary from \(0-0.3\) and \(0-0.1~\text {eV}\) (\(0.06-0.2~\text {eV}\)), respectively. Figures 1 and 2 show the atmospheric angle versus the reactor angle in panel (a), and versus \(\vert m^{0}_{\nu _{3}}\vert \) (\(m_{0}\)) (in green) and \(\left| \epsilon \right| \) (in blue) in panel (b). At the same time, as a model prediction the effective neutrino mass rate for neutrinoless double beta decay [106,107,108,109] is displayed for inverted and degenerate ordering in panel (c).

4 Conclusions

An economical scalar extension of the B–L gauge model has been built for fermion masses and mixings. We have stressed that the very pronounced and the smaller hierarchy among the quark and active neutrino masses, respectively, are the main motivations to make an unusual assignment for the fermion families under the \(\mathbf{S}_{3}\) discrete symmetry, which becomes fundamental to understanding the contrasting values between the CKM and PMNS mixing matrices. The large hierarchy in the quark masses is reflected in the hierarchical NNI textures that hijack the quark mass matrices, and therefore, the CKM mixing. On the other hand, the lepton mixing might be explained by a soft breaking of the \(\mu \leftrightarrow \tau \) symmetry, where a set of values for the relevant free parameters was found to be consistent with the last experimental data on lepton observables. The model also has a rich scalar sector, providing opportunities for its experimental testing.

Last but not least, this naive work remarks that the non-abelian group, \(\mathbf{S}_{3}\), together with two \(\mathbf{Z}_{2}\) parities, may be considered as the underlying flavor symmetry at low energies that allows us to understand the fermion masses and mixings, even though the lepton sector is limited in the sense that the Dirac CP-violating and Majorana phases are not predicted in the model.