The anomalous $U(1)_{\mathrm{anom}}$  symmetry and flavors from an SU(5) $\times$ SU(5)$'$ GUT in $\mathbf{Z}_{12-I}$ orbifold compactification

Abstract

In string compactifications, frequently the anomalous U(1) gauge symmetry appears which belongs to $E_8 \times E_8'$ of the heterotic string. This anomalous U(1) gauge boson obtains mass at the compactification scale ($\approx 10^{18\,}\,\text {GeV}$) by absorbing one pseudoscalar (corresponding to the model-independent axion) from the second rank antisymmetric tensor field $B_{MN}$. Below the compactification scale a global symmetry $U(1)_{\mathrm{anom}}$ results whose charge $Q_\mathrm{anom}$ is the original gauge U(1) charge. This is the most natural global symmetry, realizing the “invisible” axion. This global symmetry $U(1)_{\mathrm{anom}}$ is suitable for a flavor symmetry. In the simplest compactification model with the flipped SU(5) grand unification, all the low energy parameters are calculated in terms of the vacuum expectation values of the standard model singlets.

1 Introduction

In the effective theory low energy global symmetries are of fundamental importance for the strong CP solutions [1] and cosmology [2,3,4]. Using the bottom-up approach, the Kim–Shifman–Vainstein–Zhakarov (KSVZ) axion model [5, 6] and the Dine–Fischer–Srednicki–Zhitnitsky (DFSZ) axion model [7, 8] are of practical interest.¹ However, these global symmetries might be badly broken by gravitational effects [11,12,13].

On the other hand, a consistent top-down approach, a so-called string model, does not allow any global symmetry. In the compactifications of the heterotic string [14] there always exists the pseudoscalar from the second rank antisymmetric tensor field $B_{\mu \nu }\,(\mu ,\nu =1,2,3,4)$ [15], which is the so-called “model-independent axion (MI-axion)” [16]. If the MI axion is physical, its decay constant is of order $10^{15\,}\,\text {GeV}$ [17], which is shown in Fig. 1 of Ref. [18]. When this MI-axion degree is removed at the compactification scale a global U(1) symmetry can survive in breaking down to realize the “invisible” axions at the intermediate scale $M_\mathrm{int}$   [5,6,7,8]. This happens in the compactifications with an anomalous U(1) gauge symmetry [19, 20]. The anomalous U(1) gauge symmetry is a U(1) subgroup of the ${E_8\times E_8'}$ gauge group and the corresponding gauge boson obtains mass at the compactification scale ($\approx 10^{18\,}\,\text {GeV}$) by absorbing the MI-axion degree. In this case a global symmetry called $U(1)_{\mathrm{anom}}$ is surviving down to a lower energy scale. Note that the so-called “model-dependent axions (MD-axions)” from $B_{MN} \,(M,N=5,\ldots ,10)$ [21] do not match to any U(1) subgroup of ${E_8\times E_8'}$ because the heterotic string has only one anomalous U(1) gauge symmetry. Because there is no global symmetry except the $U(1)_{\mathrm{anom}}$ global symmetry, the MD-axions must be removed at the compactification scale unless they become accidentally light [22]. Thus, those used in Refs. [23, 24] must be accidentally realized. In the string compactification accidental symmetries are pointed out to be related to axions [25, 26] and R symmetry [27]. Here we identify $U(1)_{\mathrm{anom}}$ as the needed Peccei–Quinn (PQ) symmetry [1] for the “invisible” axion [5,6,7,8]. Now a reasonable compactification model with an $U(1)_{\mathrm{anom}}$ symmetry can be considered in full detail. In this paper the model presented in Ref. [28] is chosen, based on $\mathbf{Z}_{12-I}$ orbifold compactification.² The model presented in Ref. [30] could also have been chosen. This model, however, contains many more singlets and hence is more complicated to be completely presented here. Even though the analyses are presented in the specific model, the current method can be applied to any model in order to obtain complete knowledge on the “invisible” axion and flavor parameters.

The low energy gauge group obtained in [28] is $SU(5) \times U(1)_X$ $\times U(1)^6 \times SU(5)' \times SU(2)'$ where the primed non-Abelian groups are from the hidden sector $E_8'$. The first two factors $SU(5) \times U(1)_X$ are the so-called rank-5 flipped-SU(5) [31,32,33]. Being a GUT, the flipped-SU(5) must resolve the doublet–triplet splitting problem in the Higgs quintets $\mathbf{5}$ and $\overline{\mathbf{5}}$: “Why are the color triplets superheavy while Higgs doublets remain light?” In this paper it is shown how the splitting is realized in terms of the complete spectrum in the model. It is in fact done beyond the dimensional analyses.

The global symmetry $U(1)_{\mathrm{anom}}$ is beyond the flipped-SU(5). Hence it can be used as a family symmetry. Since all the quantum numbers $Q_\mathrm{anom}$ of the global symmetry $U(1)_{\mathrm{anom}}$are known, the order of magnitude of all the Yukawa couplings can be obtained, resolving the family parameters. That is, the mass matrices of the SM fermions can be obtained. Basically, it turns out that the flavor matrices are given by the multiples of the Yukawa coupling constants [27] instead of by the mass power suppressions via the Froggatt–Nielsen powers [34].

In Sect. 2 the definition of the quantum numbers and expression for $Q_\mathrm{anom}$ are presented in terms of six U(1) gauge charges of ${E_8\times E_8'}$. It is derived which pair of $\mathbf{5}$ and $\overline{\mathbf{5}}$ is remaining toward the needed pair in the SUSY SM. Here we also discuss the ’t Hooft mechanism which is working for the transfer of the global symmetry down to the axion window. In Sect. 3, the mass scales in the model where $U(1)_{\mathrm{anom}}$ is surviving as a PQ symmetry down to an intermediate scale, are discussed. In Sect. 4, the Yukawa mass matrices of $Q_\mathrm{em}=+\frac{2}{3}$ and $-\frac{1}{3}$ quarks, $Q_\mathrm{em}=-1$ charged leptons, and light SM neutrinos are presented. Section 5 is our conclusion. In the appendix the ’t Hooft mechanism in the compactification process is discussed. For this occasion the correct entries for the previous tables of Refs. [35, 36], taking into account its erratum, are presented.

2 Global charges and one pair of Higgs doublets in SUSY

In the open string theory with n Chan–Paton factors string amplitudes are U(n) invariant. This U(n), constructed with n(fundamental)–$\bar{n}$(anti-fundamental), is the world sheet global symmetry; viz. p. 374 of Ref. [37]. In the target space this is coordinate ($x_M$) dependent and hence U(n) is promoted to a gauge symmetry, which is the reason that the string theory does not allow any global symmetry. The basic reason might be the string movement in the world sheet. However, if the location of the string is fixed at a fixed point, variation of the string in the world sheet is not allowed. Hence in the target space global symmetry may not be promoted to a gauge symmetry. Fixed points are present in the symmetric orbifold compactifications and the existence of a global symmetry is not ruled out. However, in the smooth compactifications there will be no global symmetry. Thus an anomalous U(1) may not arise in smooth compactifications. To obtain a global symmetry producing an anomalous U(1) and hence the “invisible” axion, the orbifold compactification is being considered.

An ${E_8\times E_8'}$ heterotic string model compactified on $\mathbf{Z}_{12-I}$ orbifold gives SU(5)$\times$SU(5)$'\times$SU(2)$'$ with seven U(1)s [28]. This GUT model has been studied for various aspects in Refs. [35, 36, 38, 39]. Extra U(1)s are just a problem in orbifold compactification. In the Calabi–Yau compactifications for example, the rank due to extra U(1)s is easily reduced. But there is one important U(1) factor which is a part of the flipped SU(5) GUT [31]. Because of the difficulty of obtaining an adjoint representation for a Higgs multiplet for breaking SU(5), the flipped SU(5) is probably the most favorable GUT in the orbifold compactification [37].³ First we pay attention to the factor SU(5)$_\mathrm{flip}$ where our definition of SU(5)$_\mathrm{flip}$ is containing a gauge group U(1): $SU(5) \times U(1)_X$. The second factor is the anomalous U(1). Except these two U(1) factors, U(1)$_X$ and $U(1)_{\mathrm{anom}}$, the non-Abelian gauge group is SU(5)$\times$SU(5)$'\times$SU(2)$'$ and the rest of the anomaly free factors are $\tilde{\mathrm{U}}(1)^5$. Since the rank of the original gauge group ${E_8\times E_8'}$ is 16, the total number of U(1) factors is 7. Their charges are named X and $Q_i\,(i=1,\ldots ,6)$. $Q_\mathrm{anom}$ is a linear combination of $Q_i\,(i=1,\ldots ,6)$. In Refs. [35, 36] these charges are defined on the lattice as⁴

$$\begin{aligned}&X=(-\,2,-\,2,-\,2,-\,2,-\,2\,;\,0^3)(\,0^8)', \end{aligned}$$

(1)

$$\begin{aligned}&Q_\mathrm{anom}= 84Q_1+147Q_2 -42Q_3-63Q_5- 9Q_6, \end{aligned}$$

(2)

where

$$\begin{aligned} Q_1= & {} (0^5;12,0,0)(0^8)',\\ Q_2= & {} (0^5;0,12,0)(0^8)',\\ Q_3= & {} (0^5;0,0,12)(0^8)',\\ Q_4= & {} (0^8)(0^4,0;12,-\,12,0)',\\ Q_5= & {} (0^8)(0^4,0;-\,6,-\,6,12)',\\ Q_6= & {} (0^8)(-\,6,-\,6,-\,6,-\,6,18;0,0,6)'. \end{aligned}$$

In the orbifold compactification, there frequently appears an anomalous U(1)$_A$ gauge field ${A}_\mu$ from a subgroup of ${E_8\times E_8'}$ [19, 20]. The charge of this anomalous U(1)$_A$ is given in Eq. (2). In addition, the anomaly cancellation in 10 dimensions (10D) requires the so-called Green–Schwarz (GS) term in terms of the second rank antisymmetric tensor field $B_{MN}\,(M,N=1,2,\dots ,10)$ [15]. This GS term always introduces the MI axion $a_\mathrm{MI}$, $\partial _\mu a_\mathrm{MI}\propto \epsilon _{\mu \nu \rho \sigma } H^{\nu \rho \sigma }\,(\mu ,\mathrm etc.=1,2,3,4)$ where $H^{\nu \rho \sigma }$ is the field strength of $B^{\rho \sigma }$ [16]. In this compactification with U(1)$_A$, ${A}_\mu$ absorbs $a_\mathrm{MI}$ to become massive at the compactification scale $m_A\approx 10^{18\,}\,\text {GeV}$.⁵ Below the scale $m_A$ there remains a global symmetry which is called $U(1)_{\mathrm{anom}}$. Its charge is given by $Q_\mathrm{anom}$ presented in Eq. (2). In detail it works as follows. Suppose that five $\tilde{\mathrm{U}}(1)$ charges out of $Q_{1,\dots ,6}$ are broken and there is only one gauge symmetry remaining which we identify as $U(1)_{\mathrm{anom}}$. Then two continuous parameters can be considered, the MI-axion direction and the phase of the $U(1)_{\mathrm{anom}}$ transformation. Out of two continuous directions, only one phase or pseudoscalar is absorbed by the U(1)$_\mathrm{anom}$ gauge boson and one continuous direction survives. The remaining continuous degree corresponds to a global symmetry which is called the ’t Hooft mechanism [43, 44]: “If both a gauge symmetry and a global symmetry are broken by one scalar vacuum expectation value (VEV), the gauge symmetry is broken and a global symmetry is surviving”. The resulting global charge is a linear combination of the original gauge and global charges. This will briefly be reviewed in the appendix. This counting of pseudoscalar degrees is not affected by changing the scales of the VEVs. Thus, when the anomalous U(1) is arising at the compactification scale, the gauge symmetry $U(1)_{\mathrm{anom}}$ is broken and the MI-axion degree is removed and in addition the global $U(1)_{\mathrm{anom}}$ symmetry below the compactification scale results. The dilaton partner of the MI axion must remain heavy as in the usual Higgs mechanism since it does not find its partner below the compactification scale.

2.1 No gauged anomalous U(1) below the compactification scale

There have been many discussions on the Fayet–Iliopoulos (FI) D-term for $U(1)_{\mathrm{anom}}$ gauge symmetry at the GUT scale, e.g. in Ref. [45]. However, there is no gauged U(1) corresponding to the anomalous symmetry below the compactification scale. In models with a hierarchy between the compactification scale and the GUT scale there is no need to consider the $U(1)_{\mathrm{anom}}$  D-term, $\int d^4\theta \,\xi D$, below the compactification scale. The $\xi$ term is in the D-term potential, $\frac{1}{2} D^2$, with $D=-\xi -e\phi ^*Q_a\phi$ where e is the $U(1)_{\mathrm{anom}}$  gauge coupling. Our $U(1)_{\mathrm{anom}}$ is derived from the orbifold compactification of the ${E_8\times E_8'}$ heterotic string. After compactification of the six internal space $M_4\times K$ can be considered where $M_4$ is the Minkowski space and K is the internal space. In Fig. 1 some relevant fields living in K are shown. The effective symmetry group in the $M_4$ Lagrangian is gauge symmetries times some discrete groups without any global symmetry except that corresponding to $B_{\mu \nu }$. The 4D scalar $B_{ij}$ are called the MD-axions, $a_n\ (n\ge 1)$, determined by the topology of the internal space. Reference [45] shows that the classical symmetries corresponding to $a_n\rightarrow a_n+(\mathrm{constant})$ are broken by the world-sheet instanton effects. Figure 1 shows these fields living in K with some U(1) gauge fields. In the compactification of ${E_8\times E_8'}$ heterotic string there appears only one anomalous U(1) if any of such terms are present. If so, the corresponding gauge boson obtains mass by absorbing $a_\mathrm{MI}$ as shown in the appendix and Fig. 1b. All the other U(1)s are anomaly free. $a_n$ are not absorbed to gauge bosons at this stage. Now the massless states $a_n$ and non-anomalous six U(1)s below the compactification scale can be considered as well as their K$\ddot{\mathrm{a}}$hler potentials and FI D terms. However, for $U(1)_{\mathrm{anom}}$  we do not need to consider the corresponding D term.

Fig. 1

Compactification, leading to $M_4\times K$. The parallelogram depicts $M_4$, $\ell$ is a compactification size of K and L in one direction is shown pictorially as a neck

For a consistency check, consider Fig. 1a again. If the throat is not cut, the space is still 10D. In this 10D the $U(1)_{\mathrm{anom}}$ subgroup of ${E_8\times E_8'}$ can be considered. Let us consider the subgroup SU(3) $_C\times$ SU(2)$_L\times$ U(1)$_Y\times$ U(1)$_\mathrm{anom}$ $\times \prod _{i=1}^5\tilde{U}(1)_i$ of ${E_8\times E_8'}$ where $\tilde{U}(1)_i$ are anomaly free. Before cutting the throat ${E_8\times E_8'}$ is broken in order to separate $U(1)_{\mathrm{anom}}$ by the VEV of an appropriate adjoint representation (in 4D language) in the bulk. Of course, this adjoint representation is not present in our massless spectrum but is good enough to see the resulting effective low energy theory. If $U(1)_{\mathrm{anom}}$ is separated in this way it obtains mass by absorbing the MI axion by the VEVs of $F_{ij}$ as shown in the appendix. Then superheavy masses are assigned to the adjoint representation introduced. So far, nothing has been introduced violating the effective symmetry SU(3) $_C\times$ SU(2)$_L\times$ U(1)$_Y\times$ U(1)$_\mathrm{anom}$ $\times \prod _{i=1}^5\tilde{U}(1)_i$. Next, the throat is cut to obtain the effective 4D theory which is SU(3)$_C\times$SU(2)$_L\times$U(1)$_Y$ $\times \prod _{i=1}^5\tilde{U}(1)_i$. There cannot be a FI D term for $U(1)_{\mathrm{anom}}$ in this interpretation.

Even if we consider the FI term with a nonvanishing $\xi$, in case there is no hierarchy between the compactification scale and the GUT scale [46], it can be shown that a global symmetry can be derived below the scale of the anomalous gauge boson mass. Therefore, the following equation can be considered, including the D term with a nonvanishing $\xi$,

$$\begin{aligned}&\frac{1}{2} \partial ^\mu a_\mathrm{MI}\partial _\mu a_\mathrm{MI} + M_\mathrm{MI}A_\mu \partial ^\mu a_\mathrm{MI} +\left| - \xi +e\sum _{a}\phi _a^*Q_a \phi _a\right| ^2\nonumber \\&\qquad +\Big [|(\partial _\mu -ie A_\mu )\phi _1|^2+ \cdots \Big ]\nonumber \\&\quad = (M_\mathrm{MI} \partial ^\mu a_\mathrm{MI}-eV_1\partial ^\mu a_1)A_\mu +\cdots , \end{aligned}$$

(3)

where $\phi _a$ are assumed to carry only the anomalous charge for a moment, not carrying any non-anomalous charge Y and $\tilde{Q}_i\,(i=1,\dots ,5)$. Let one $\phi _a$, say $\phi _1$ develops a VEV, ${V_1}$, by minimizing the FI term. Here, two phase fields, $a_\mathrm{MI}$ and $a_1$ [= the phase of $\phi _1\,(=(V_1+\rho _1)e^{ia_1/V_1})/\sqrt{2}$] are considered and only one Goldstone boson is absorbed to $A_\mu$,

$$\begin{aligned} \sqrt{M_\mathrm{MI}^2+e^2V_1^2}\, (\cos \theta _G\, a_\mathrm{MI}-\sin \theta _G\, a_1) \end{aligned}$$

(4)

where $\tan \theta _G=eV_1/M_\mathrm{MI}$. The orthogonal direction

$$\begin{aligned} \theta '=\cos \theta _G\, \frac{a_\mathrm{MI}}{|M_\mathrm{MI}|}+\sin \theta _G\,\frac{a_1}{|eV_1|} \end{aligned}$$

(5)

is surviving as a global direction below the scale $\sqrt{M_\mathrm{MI}^2+e^2V_1^2}$. With this global symmetry, breaking the five non-anomalous $\tilde{\mathrm{U}}(1)$’s around the GUT scale can be considered, leaving only one global symmetry to the axion window. If one gauge symmetry $U(1)_{\mathrm{anom}}$ were the whole story for the global symmetry, the next lower scale VEV of a scalar carrying nonzero VEV (probably at a GUT scale) will break the global $U(1)_{\mathrm{anom}}$. In the orbifold compactification however, there appear many gauge U(1)s (six in our example) which are anomaly-free except the above $U(1)_{\mathrm{anom}}$, say U(1)$_4$ in our example. The D term of U(1)$_4$ is $|\phi _2^*Q_4\phi _2|^2$. Generally, $\phi _2$ carries the $U(1)_{\mathrm{anom}}$ charge, for example in Table 2 any $\phi _2$ carrying nonzero $Q_4$ also carries $Q_\mathrm{anom}$. So, the VEV of $\phi _2$ will break both U(1)$_4$ and the global $U(1)_{\mathrm{anom}}$ obtained above. Below $\langle \phi _2\rangle$ appears the global symmetry $U(1)_{\mathrm{anom}}$. Applying the ’t Hooft mechanism repeatedly until all anomaly-free gauge U(1)s are broken except U(1)$_Y$, the global $U(1)_{\mathrm{anom}}$ interpretable as $U(1)_\mathrm{PQ}$ from string compactification is obtained.⁶ Then, at the intermediate scale $10^{9-11}$ GeV, one $U(1)_{\mathrm{anom}}$  breaking VEV $f_a$ of a SM singlet scalar $\Phi$ breaks the global symmetry $U(1)_{\mathrm{anom}}$ spontaneously and the needed “invisible” axion at the intermediate scale results. The global symmetry whose shift angle $\theta '$ is broken at an intermediate scale to create the “invisible” axion by some scalar field carrying the anomalous charge. To observe this only one scalar field $\phi _1$ develops a VEV $V_1$ as before. Now a global charge surviving below the scale $\sqrt{M_\mathrm{MI}^2+e^2V_1^2}$ is defined as⁷

$$\begin{aligned} \hat{Q}'={Q}_\mathrm{anom}+x\,{Q}_a \end{aligned}$$

(6)

where ${Q}_a$ is an anomaly-free gauge U(1) charge. For the condition $\hat{Q}'|\phi _1\rangle =0$ so that $\hat{Q}'$ is a good generator of the global symmetry, $x=-{Q}_\mathrm{anom}(\phi _1)/Q_a(\phi _1)$ is fixed. Since $Q_a$ is an anomaly-free generator, a constant multiple of $Q_a$ in Eq. (6) can be added to give the same anomaly coefficient for U(1)$'$–SU(3)$_c$–SU(3)$_c$. A new global charge can be taken

$$\begin{aligned} \hat{Q}'={Q}_\mathrm{anom}. \end{aligned}$$

(7)

Actually, a more general proof was given in Ref. [43]. Consider $\hat{Q}'={Q}_\mathrm{anom}+x_1 Q_a+x_2Q_b$ where $Q_a$ and $Q_b$ are anomaly-free gauge charges. Then Tr $\hat{Q}'=Q_\mathrm{anom}$ and Tr $\hat{Q}'Q_pQ_q=Q_\mathrm{anom}Q_pQ_q$ where $\{p,q\}=\{a,b\}$. In fact, in Ref. [43] instead of $Q_\mathrm{anom}$ of Eq. (2), it was shown to have exactly the same traces with

$$\begin{aligned} \hat{Q}'=63(0^5;16,28,-\,8)(0^5;6,6,-\,12)'. \end{aligned}$$

(8)

In the tables of the present paper $Q_\mathrm{anom}$ is used, and in the Tables of [43] $\hat{Q}'$ is used but the anomaly-related quantities such as Tr $\hat{Q}'Q_\mathrm{color}^\alpha Q^\beta _\mathrm{color}$ and Tr $\hat{Q}'Q_\mathrm{em}Q_\mathrm{em}$ are exactly the same in both cases.

Table 1 The $SU(5) \times U(1)_X$  states. Here, + represents helicity $+\frac{1}{2}$ and – represents helicity $-\frac{1}{2}$. Sum of $Q_\mathrm{anom}$ is multiplied by the index of the fundamental representation of SU(3)$_c$, $\frac{1}{2}$. The PQ symmetry, being chiral, counts quark and antiquark in the same way. The right-handed states in $T_3$ and $T_5$ are converted to the left-handed ones of $T_9$ and $T_7$, respectively. The bold entries in the column $Q_\mathrm{anom}$ are $Q_\mathrm{anom}/126$

2.2 “Invisible” axion in the axion window

For simplicity’s sake, a hierarchy is assumed between the compactification and the GUT scales. $\phi$ is chosen, not carrying any gauge charge. The VEV of $\phi$ is assumed to be at the axion window and breaks the $U(1)_{\mathrm{anom}}$ global symmetry. In this case, the actual global symmetry breaking scale is a mixture of two effects: the MI-axion direction and the hypothetical intermediate scale axion direction (the phase of $\phi$). When the original anomalous gauge charge of $\phi$ is $Q_a$, $\phi \,(\sim ve^{ia_\phi /v})$, the QCD axion a created at the intermediate scale (determined by the VEV of $\phi$) is a combination of $a_\phi$ and $a_\mathrm{MI}$,

$$\begin{aligned} a= & {} \cos \theta \,a_\phi +\sin \theta \,a_\mathrm{MI},~~\mathrm{with}~\sin \theta \nonumber \\= & {} \frac{gQ_av}{\sqrt{M^2_\mathrm{MI}+g^2Q_a^2v^2}} \end{aligned}$$

(9)

where the antisymmetric tensor field strength is the MI-axion, $H_{\mu \nu \rho }=M_{MI}\epsilon _{\mu \nu \rho \sigma } \partial ^\sigma a_{MI}$ [16]. Thus, for $v\ll M_{MI}$ the desired “invisible” axion at the intermediate scale is obtained.

Here we stress again that the exact global symmetries from the string compactification require anomalous gauge symmetries after compactification. From the ${E_8\times E_8'}$ heterotic string there is only one such anomalous gauge symmetry as discussed above.⁸ Any other global symmetries must be accidental as discussed for QCD axions in [25, 26] and for axion-like particles in [48].

2.3 Three families in the flipped SU(5)

The three families of the minimal supersymmetric standard model (MSSM) and one pair of Higgs doublets are required as a result. To have the “invisible” axion, it is further required that $U(1)_{\mathrm{anom}}$ is broken at the intermediate scale, $M_\mathrm{int}\approx 10^{11\,}\,\text {GeV}$. At the SU(5)$_\mathrm{flip}$ GUT level, three copies of $\overline{\mathbf{10}}_{-1}\oplus {\mathbf{5}}_{+3}\oplus {\mathbf{1}}_{-5}$ are necessary. In Table 1 SU(5) non-singlet fields with the global quantum numbers where the axion–photon–photon couplings are presented in the last column are presented. One family appears in the untwisted sector U and two families appear in the twisted sector $T_4^0$.

In the SU(5)$_\mathrm{flip}$ GUT, a pair of $\overline{\mathbf{10}}_{-1}\oplus {\mathbf{10}}_{+1}$ is needed to break the rank 5 group SU(5)$_\mathrm{flip}$ down to the rank 4 group SU(3)$_C\times$SU(2)$_L\times$U(1)$_Y$. They appear in $T_3^0$ and $T_9^0$ in Table 1. The vacuum expectation values (VEVs) of these pairs achieve the doublet–triplet splitting discussed in Sect. 4.1.

Table 2 Left-handed $SU(5) \times U(1)_X$ $\times$SU(5)$'\times$ SU(2)$'$ singlet states. $(N^L)_{j}$ is the notation for the oscillator mode of the oscillating string with j denoting the coordinate in the internal space. $\mathcal{P}(f_{0,+,-})$ is the multiplicity of the corresponding spectrum in the twisted sector $T^{0,+,-}$. In this table there is only $\mathcal{P}(f_0)$. The right-handed states in $T_3$ and $T_5$ are converted to the left-handed ones of $T_9$ and $T_7$, respectively

For the Higgs quintets $T_6$ has pairs with multiplicity 3. More importantly, two pairs appear in $T_4^0$ and one pair appears in $T_7^0$. The two pairs appearing in $T_4^0$ are not distinguished and the Higgsino mass matrix elements are democratic. Their Yukawa couplings take the form of $C_5C_{6}\sigma _{1}$ that conserves the $U(1)_{\mathrm{anom}}$ symmetry,

$$\begin{aligned} M_\mathrm{demo}^{2\times 2}= \begin{pmatrix} M&{}\quad M \\ M&{}\quad M \end{pmatrix} \end{aligned}$$

(10)

where $M\sim \langle \sigma _{1}\rangle$. The $\sigma _1$ multiplicity is 3, as shown in Table 2. The twisted sector $T_4^0$ satisfies the $\mathbf{Z}_3$ orbifold selection rules and the multiplicity 3 belonging to the permutation symmetry $S_3$ splits into $S_3$ representations $\mathbf{2}\oplus \mathbf{1}$. Three $\sigma _1$’s under $S_3$ in the $\mathbf{Z}_3$ compactification can be combined to [49, 50]

$$\begin{aligned} \Phi _0= & {} \frac{1}{\sqrt{3}} \left( \sigma _1^{a}+ \sigma _1^{b}+\sigma _1^{c} \right) ,\nonumber \\ \Phi _+= & {} \frac{1}{\sqrt{3}}\left( \sigma _1^{a}+ \omega \,\sigma _1^{b}+ \bar{\omega }\,\sigma _1^{c} \right) ,\\ \Phi _-= & {} \frac{1}{\sqrt{3}}\left( \sigma _1^{a}+{\bar{\omega }}\,\sigma _1^{b} +\omega \,\sigma _1^{c} \right) ,\nonumber \end{aligned}$$

(11)

where $\omega =e^{2\pi i/3}$ and ${\bar{\omega }}=e^{4\pi i/3}$ are the cube roots of unity. $\Phi _0$ is a singlet 1 and $\Phi _+$ and $\Phi _-$ form a doublet 2. Suppose that $\langle \Phi _0\rangle \ne 0$ and $\langle \Phi _+\rangle =\langle \Phi _-\rangle = 0$. $C_5$ and $C_6$ belong to 2 of $S_3$, and their multiplication gives ${\mathbf{2}}\times {\mathbf{2}}={\mathbf{1}}\oplus {\mathbf{1}}' \oplus {\mathbf{2}}$,

$$\begin{aligned} \Psi _0= & {} \frac{1}{\sqrt{3}} \left( C_5^{(1)}C_6^{(1)} +C_5^{(2)}C_6^{(2)}\right) ,\nonumber \\ \Psi _0'= & {} \frac{1}{\sqrt{3}} \left( C_5^{(1)}C_6^{(1)} -C_5^{(2)}C_6^{(2)}\right) ,\\ \Psi _+= & {} \frac{1}{\sqrt{3}} \left( C_5^{(1)}C_6^{(2)} +\omega \,C_5^{(2)}C_6^{(1)}\right) ,\nonumber \\ \Psi _-= & {} \frac{1}{\sqrt{3}} \left( C_5^{(1)}C_6^{(2)} +\bar{\omega }\,C_5^{(2)}C_6^{(1)}\right) ,\nonumber \end{aligned}$$

(12)

where $\Psi _+$ and $\Psi _-$ form a doublet under interchange $(1)\leftrightarrow (2)$. Thus, the singlet VEV $\langle \Phi _0\rangle$ can couple with either $\Psi _0$ or $\Psi _0'$. In this way one pair becomes superheavy. This result is equivalent to the democratic mass matrix (10). Namely, the determinant of $M_\mathrm{demo}^{2\times 2}$ is 0 and only one pair obtains mass 2M. The remaining pair is massless at this stage. For the three pairs in $T_6$ the Higgsino mass matrix can be studied in the same way. Since it belongs to $T_6$ we consider the $\mathbf{Z}_2$ and $S_2$ permutation which allows only the following singlet combinations:

$$\begin{aligned} \Psi ^{(a)}= & {} \frac{1}{\sqrt{3}} \left( C_7^{(1)}C_8^{(1)} +C_7^{(2)}C_8^{(2)}+C_7^{(3)}C_8^{(3)}\right) ,\nonumber \\ \Psi ^{(b)}= & {} \frac{1}{\sqrt{3}} \left( C_7^{(1)}C_8^{(1)} -C_7^{(2)}C_8^{(2)}+C_7^{(3)}C_8^{(3)}\right) ,\\ \Psi ^{(c)}= & {} \frac{1}{\sqrt{3}} \left( C_7^{(1)}C_8^{(1)} +C_7^{(2)}C_8^{(2)}-C_7^{(3)}C_8^{(3)}\right) ,\nonumber \\ \Psi ^{(d)}= & {} \frac{1}{\sqrt{3}} \left( C_7^{(1)}C_8^{(1)} -C_7^{(2)}C_8^{(2)}-C_7^{(3)}C_8^{(3)}\right) .\nonumber \end{aligned}$$

(13)

If we require the invariance of the mass matrix under $S_2$, i.e. under the interchange of any two pairs out of (1), (2) and (3), only the term $\Psi ^{(a)}$ is allowed. Then only one pair obtains a superheavy mass and two pairs remain light. Again it is like taking a democratic mass matrix. Even if the two pairs from $T_6$ remain light, their contribution to the unification point of couplings is null because they are the SU(5)$_\mathrm{flip}$ complete multiplets. However, the absolute magnitude of the gauge coupling constant at the unification point is affected. Nevertheless we will not discuss these complete multiplets anymore in this paper since the massless pairs do not affect our discussion on the flavor problems.

There are the Yukawa couplings $C_5C_{11}\sigma _{21}$ and $C_6C_{12}\sigma _{21}$ which conserve the $U(1)_{\mathrm{anom}}$ symmetry. Among the remaining two light pairs (one from $T_4^0$ and the other from $T_7^0$) one obtains mass and finally there will be only one light pair left. We have the $3\times 3$ Higgsino mass matrix⁹

$$\begin{aligned}&~~H_u^{(T4)_1}~~\, H_u^{(T4)_2}~~ H_u^{(T7)}\nonumber \\ M_\mathrm{Higgsino}^{3\times 3}= & {} \begin{pmatrix} ~~M~&{}~~\quad M\quad &{}\quad m~~\\ ~~M~&{}~~\quad M\quad &{}\quad m~~\\ ~~m~&{}~~\quad m\quad &{}\quad 0~~ \end{pmatrix} \begin{array}{c}H_d^{(T4)_1}\\ H_d^{(T4)_2}\\ H_d^{(T7)} \end{array} \end{aligned}$$

(14)

where $m\sim \langle \sigma _{21}\rangle$. As expected, the determinant of $M_\mathrm{Higgsino}^{3\times 3}$ is 0, and there remain two light pairs as far as the (33) element is 0. The heaviest eigenstate of (14) is

$$\begin{aligned} \Psi ^{M_c}=\frac{1}{\sqrt{2}}\left( \Psi ^{T_4^0}_1+\Psi ^{T_4^0}_2\right) , ~\mathrm{mass}=2M, \end{aligned}$$

(15)

where $\Psi$ is $H_{u,d}$. The Higgsino pair of the MSSM contains

$$\begin{aligned} \Psi ^{0}=\frac{1}{\sqrt{2}}\left( \Psi ^{T_4^0}_1-\Psi ^{T_4^0}_2\right) , ~\mathrm{mass}=0. \end{aligned}$$

(16)

The other state with a nonzero (33) element will be presented later.

Table 3 Electromagnetically charged singlets

3 Mass scales

Below the Planck scale $M_\mathrm{P}$ four scales are being considered: the compactification scale allowing large masses ${M_\mathrm{vec}}$ to vector-like pairs, the GUT scale $M_\mathrm{GUT}$, the intermediate $M_\mathrm{int}$ and the electroweak scale $v_\mathrm{ew}$. The principle of removing vector-like pairs is just the gauge principle as emphasized in [43]. If extra symmetries are introduced one must include another assumption(s) how those extra symmetries are broken. The hierarchy of scales that we consider is

$$\begin{aligned}&\mathrm{E}_8\times \mathrm{E}_8' \Big |_{M_\mathrm{vec}} {\quad \longrightarrow \quad } \mathrm{GUT}{\quad \longrightarrow \quad } \Big |_{M_\mathrm{GUT}}\\&\quad \text { SM~ and ~``invisible''~axion} \Big |_{M_\mathrm{int}} {\longrightarrow \quad } \mathrm{SU(3)_c}\times \mathrm{U(1)_{em}} \Big |_{v_\mathrm{ew}}\\ \end{aligned}$$

where ${M_\mathrm{vec}}^2$ is the order of the string tension, $\alpha ^{\prime \,-1}$. The particles removed at the compactification scale are the vector-like sets. In Table 3 charged singlets are listed. The vector-like sets, including the charges $Q_\mathrm{anom}$ and $\mathbf{Z}_{12}$ orbifolds, must be removed at $M_\mathrm{vec}$ . The U(1)s which can be broken at the GUT scales are five anomaly free $\tilde{\mathrm{U}}(1)$’s. $U(1)_{\mathrm{anom}}$ global symmetry is required to be broken at the axion window and works as a global symmetry at the GUT scale. Any singlet of Table 3 can have a GUT scale VEV, leaving a global symmetry below ${M_\mathrm{GUT}}$ via the ’t Hooft mechanism [44]. This process can be repeated in order to break all five anomaly free $\tilde{\mathrm{U}}(1)$’s, leaving only $U(1)_{\mathrm{anom}}$ global symmetry below ${M_\mathrm{GUT}}$. The doublet–triplet splitting can be of the form “colored particles=GUT scale and $H_{u,d}$=light”. Even if the masses of colored particles are a bit smaller than $M_{GUT}$, proton stability can be achieved. Proton decay by dimension 6 operators of quarks and leptons (by the exchange of colored scalars) is helped by the Yukawa couplings for the first family members by order $10^{-5.5}$. Then it is not problematic. With the SUSY assumption dimension 5 operators of quark and lepton superfields, $W_4\propto \overline{\mathbf{10}}_{-1}\cdot \overline{\mathbf{10}}_{-1}\cdot \overline{\mathbf{10}}_{-1} \cdot \mathbf{5}_{+3}$ are the leading contribution. This is disastrously dangerous if the colored scalar masses are somewhat smaller than ${M_\mathrm{GUT}}$. There can arise a $\mathbf{Z}_4$ from a subgroup of an anomaly free $\tilde{\mathrm{U}}(1)$ gauge group, eliminating $W_4$ as shown in [38]. There is no fast proton decay problem [51].

There are two GUT scale sets,

$$\begin{aligned} C_7^{T_6}+C_8^{T_6},~C_{11}^{T_3}+C_{12}^{T_9}. \end{aligned}$$

(17)

These are $SU(5) \times U(1)_X$ non-singlets and these vector-like sets cannot be external light fields. In addition the superpotential terms cannot be generated through the intermediate states of these vector-like fields. Diagrams with these internal $M_\mathrm{GUT}$ mass states must contain loops which cannot generate superpotential terms because of the non-renomalization theorem. Thus high dimension superpotential terms composed of light fields cannot be generated with a suppression factor ${M_\mathrm{GUT}}$ or $M_\mathrm{vec}$   in our model.

If a high dimension superpotential term of light fields is generated, the relevant mass suppression factor must be the intermediate scale $M_\mathrm{int}$ . This conclusion, depending on our detailed model, is very different from the general strategy in the MSSM where the $\mu$ term for example has the suppression factor ${M_\mathrm{GUT}}$ or $M_\mathrm{P}$ [10].

4 Yukawa couplings

Now, we search for a possibility for a nonzero (33) element of $M_\mathrm{Higgsino}^{3\times 3}$ in Eq. (14) which respects the $U(1)_{\mathrm{anom}}$ symmetry and the $\mathbf{Z}_{12-I}$ selection rules. At the level of dimension 10 there appears one

$$\begin{aligned} W\propto & {} \frac{1}{M_\mathrm{int}^7} H_u^{(T_7^0)}H_d^{(T_7^0)}\,\sigma _{12}^{(T3)} \sigma _{23}^{(T_7^0)}\,\sigma _{21}^{(T_7^0)}\sigma _{21}^{(T_7^0)} \,\sigma _{8}^{(T6)}\nonumber \\&\times \,\sigma _{8}^{(T6)} \sigma _{8}^{(T6)}\sigma _{1}^{(T_4^0)} \end{aligned}$$

(18)

where we used the $M_\mathrm{int}$  as the suppression factor. This term introduces a nonzero entry $\varepsilon$ in the (33) element which cannot be very small because the VEVs of the singlets are also at the scale $M_\mathrm{int}$ . In this case, the remaining two eigenstates of (14) are

$$\begin{aligned}&\frac{1}{N}\begin{pmatrix} 1\\ 1\\ -\frac{M}{m}+\frac{\varepsilon }{2m}-\sqrt{2+\left( \frac{M}{m}-\frac{\varepsilon }{2m}\right) ^2} \end{pmatrix} ,\nonumber \\&\quad \mathrm{mass}=M+\frac{\varepsilon }{2}-\sqrt{2m^2+\left( M-\frac{\varepsilon }{2}\right) ^2} \end{aligned}$$

(19)

and the massless one is still Eq. (16),

$$\begin{aligned} \mathrm{\mathbf{5}} ^{T_4^0},\mathrm{\overline{\mathbf{5}}} ^{T_4^0}=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\ -1\\ 0\end{pmatrix}, ~\mathrm{mass}=0. \end{aligned}$$

(20)

At this level, the Higgs doublets are those appearing in $T_4^0$. However, the doublets from $T_4^0$ obtain mass at the electroweak scale when soft masses of order $m_{3/2}^2$ are introduced, which is a well-known fact in the supergravity phenomenology. This will not be discussed here.

Table 4 Three left-handed states belonging to the vector-like spectrum appearing in Fig. 2

Fig. 2

A high dimensional term. The fractional numbers in the brackets are $Q_\mathrm{anom}/126$

4.1 Doublet–triplet splitting

For the successful MSSM, the multiplet (20) must split into heavy colored ones and light Higgs doublets. This doublet–triplet splitting problem is achieved by the VEVs of $\overline{\mathbf{10}}_{-1}$ and $\mathbf{10}_{+1}$ appearing in $T_3$ and $T_9$. Here it is explicitly shown from the detailed string compactification model discussed above. It is shown that the suppression factor is the mass of the vector-like pair ${M_\mathrm{vec}}$. The VEVs of ${\overline{\mathbf{10}}}_{-1}$ and ${\mathbf{10}}_{+1}$ give mass to the colored triplets of the Higgs quintets, $\mathrm{\mathbf{5}}_{-2} ^{T_4^0}$ and $\mathrm{\overline{\mathbf{5}}}^{T_4^0}_{+2}$. But the problem is the scale for the effective operator. We find the following operators:

$$\begin{aligned}&\mathrm{VEV~of~\overline{\mathbf{10}}}{:}~\frac{1}{{M_\mathrm{vec}}M_\mathrm{int}} \, \langle C_{11}^{T3}\rangle \langle \sigma _{21}^{T_1^0} \rangle \langle \sigma _{21}^{T_1^0} \rangle \, C_{11}^{T3}\,C_6^{T_4^0}\nonumber \\&\rightarrow d_\mathrm{from\, {\overline{\mathbf{10}}}_{-1}}\,d^c_\mathrm{from\, {\overline{\mathbf{5}}}_{+2}} , \end{aligned}$$

(21)

$$\begin{aligned}&\mathrm{VEV~of~\mathbf{10}}{:}~\frac{1}{{M_\mathrm{vec}}M_\mathrm{int}} \, \langle C_{12}^{T9}\rangle \langle \sigma _{19}^{T_1^0} \rangle \langle \sigma _{20}^{T_1^0} \rangle \, C_{12}^{T9}\,C_5^{T_4^0}\nonumber \\&\rightarrow d^c_\mathrm{from\, {\mathbf{10}}_{+1}}\, d_\mathrm{from\, {\mathbf{5}}_{-2}}, \end{aligned}$$

(22)

where the $1/{M_\mathrm{vec}}$ suppression exists because there appear heavy vector-like states in the tree diagram.¹⁰ Two thick lines of Fig. 2 are vector-like states of Ref. [28] because there is no massless (left-handed) states in $T_8^0$ and $T_5^0$. The cross in the RHS is of the order $M_\mathrm{int}$ because ${\mathbf{1}}_0^{T_7^0}\varvec{(\frac{-54}{7})}$ is present in Table 2. On the other hand the cross in the LHS is expected to be of the order ${M_\mathrm{vec}}$ because ${\overline{\mathbf{10}}}^{T_4^0}\varvec{(\frac{-21}{7})}$ does not appear in Table 2. Even though the left-handed states of Table 2 do not contain a state with $Q_\mathrm{anom}=\frac{-21}{7}$, the vector-like pairs ($\mathbf{10}$ and $\overline{\mathbf{10}}$ shown as two thick arrows) can fulfill the quantum numbers because we will not require the masslessness conditions in the orbifold selection rules for the vector-like states. Also, the thick arrow line for $\overline{\mathbf{5}}_{+2}$ does not appear in Table 2. These three states are denoted by the ‘vector-like states’ which are shown in Table 4. The left-mover and right-mover masses in the heterotic string with $\mathbf{Z}_{12-I}$ compactification in the $k^\mathrm{th}$ twisted sector are given

$$\begin{aligned}&M_L^2= \frac{(P+kV)^2}{2}-\frac{2\tilde{c}_k}{2}+\frac{2\tilde{N}_L}{2}, \end{aligned}$$

(23)

$$\begin{aligned}&M_R^2= \frac{(\tilde{s}+ k\phi )^2}{2} -\frac{2c_k }{2}+\frac{2\tilde{N}_R}{2}, \end{aligned}$$

(24)

where $\tilde{N}_{L,R}$ are the oscillator numbers and $2\tilde{c}_{4,8}=\frac{3}{2}$ and $2{c}_{4,8}=\frac{1}{2}$ are given in Ref. [41]. In the model of [41], the L and R vectors are

$$\begin{aligned}&V=\left( 0^5;\frac{-1}{6},\frac{-1}{6},\frac{-1}{6}\right) \left( 0^5;\frac{+\,1}{4},\frac{+\,1}{4},\frac{-2}{4}\right) ', \end{aligned}$$

(25)

$$\begin{aligned}&\phi =\left( \frac{5}{12},\frac{4}{12},\frac{1}{12}\right) . \end{aligned}$$

(26)

The QCD-color field in ${\overline{\mathbf{5}}}_{+2}$ finds the partner in ${\overline{\mathbf{10}}}_{-1}$, both of which are shown in red in Fig. 2. Similarly, ${\mathbf{5}}_{-2}$ finds the partner in ${\mathbf{10}}_{+1}$. Thus there remain only massless Higgs doublets from Eq. (20), $H_u$ and $H_d$. The doublet–triplet splitting is realized. One pair of ${\mathbf{5}}_{-2}$ and ${\overline{\mathbf{5}}}_{+2}$ needs one pair of ${\mathbf{10}}_{+1}$ and ${\overline{\mathbf{10}}}_{-1}$ for the splitting.

Table 5 The SU(5)$'$ representations. Notations are the same as in Table 1

In view of the longevity of proton, note that the dangerous $\overline{\mathbf{10}}_H \cdot \overline{\mathbf{10}}\cdot \overline{\mathbf{10}}\cdot \mathbf{5}$ and $\overline{\mathbf{10}}_H\cdot \mathbf{5}\cdot \mathbf{5}\cdot \mathbf{1}$ couplings are allowed in $SU(5) \times U(1)_X$, judging from the gauge quantum numbers alone. In the ordinary SO(10), which is the covering group of $SU(5) \times U(1)_X$, these terms are forbidden by imposing the R-parity quantum numbers $-\,1$ and $+\,1$, respectively, for the matter and Higgs fields. In our case the GUT scale Higgs boson $\overline{\mathbf{10}}_H$ and $\mathbf{10}_H$ are $C_{11}$ and $C_{12}$ of Table 1. These are from $T_3^0$ and $T_9^0$. The orbifold selection rules allow for the superpotential term $C_{11}C_{12}$ but do not allow for $C_{11} C_4C_4 C_3$ and $C_{11} C_3C_3\cdot \mathbf{1}$ from the fields in Table 1. In the latter operator containing the L-violating $LLE^c$, $\mathbf{1}$ may be chosen from $T_1^0$ such as $\sigma _{19,20,21}$ of Table 2. However, $LLE^c$ alone does not trigger proton decay. Basically, the R-parity interpretation for the proton longevity in SO(10) GUT, up to dimension 5 operators, is automatic from our orbifold selection rules [37] in case of the $SU(5) \times U(1)_X$. However, a complete study of the proton longevity is outside the scope of this paper.

4.2 The CKM and PMNS matrices

Earlier an attempt was made to obtain a CKM matrix from standard-like models implied by a $\mathbf{Z}_2\times \mathbf{Z}_2$ fermionic construction [42]. Obtaining CKM and PMNS matrices based on the model of Ref. [28] was attempted. As commented on before, if a high dimension superpotential term of light fields is generated, the relevant mass suppression factor must be the intermediate scale $M_\mathrm{int}$ . Suppose we have an effective operator for the $Q_\mathrm{em}=\frac{2}{3}$ quarks,

$$\begin{aligned} \frac{1}{M^n} \mathrm{(SM~singlets~of~ Table~II )}\cdot \overline{\mathbf{10}}_{-1}\mathbf{5}_3\mathbf{5}_{-2}. \end{aligned}$$

(27)

In Tables 5 and 6, all the particles that transform non-trivially under SU(5)$'\times$SU(2)$'$ are listed. There is no vector-like pair including $Q_\mathrm{anom}$ charges. Thus there is no tree diagram of an intermediate state with mass $M_\mathrm{vec}$ and any operator with sub-GUT scale fields must have the mass suppression parameter $M_\mathrm{int}$ . Thus the suppression mass in Eq. (27) must be $M_\mathrm{int}$ .

Table 6 The SU(2)$'$ representations. Notations are the same as in Table 1. We listed only the upper component of SU(2)$'$ from which the lower component can be obtained by applying $T^-$ of SU(2)$'$

With this in mind the Yukawa matrices and the fermion masses are discussed. The $Q_\mathrm{em}=+\frac{2}{3}$ quark mass matrix, consistent with the $U(1)_{\mathrm{anom}}$ symmetry, the orbifold selection rules and the multiplicity 2 conditions of the Higgs doublets and matter fermions in $T_4^0$, is

$$\begin{aligned}&{\mathbf{5}}_3^U\quad {\mathbf{5}}_3^{T_4^A} ~~{\mathbf{5}}_3^{T_4^S}&\nonumber \\ M_u\propto \begin{array}{c} {\overline{\mathbf{10}}}_{-1}^U\\ {\overline{\mathbf{10}}}_{-1}^{T_4^A} \\ {\overline{\mathbf{10}}}_{-1}^{T_4^S}\end{array}&\begin{pmatrix}\frac{\sigma _2\sigma _4}{M_\mathrm{int}^2},&{}0,&{} \frac{\alpha _1\sigma _4}{M_\mathrm{int}}\\ 0,&{}0,&{}1\\ \frac{\alpha _2\sigma _2}{M_\mathrm{int}},&{} 1,&{}0 \end{pmatrix} v_u&,~~\mathrm{with}~\langle H_u\rangle =\frac{v_u}{\sqrt{2}},\nonumber \\ \end{aligned}$$

(28)

where superscripts S and A in $T_4^0$ denote the symmetric and antisymmetric combinations of the multiplicity 2 fields and $\alpha _1$ and $\alpha _2$ are coupling parameters. Similarly, the $Q_\mathrm{em}=-\frac{1}{3}$ quark mass matrix is given by

$$\begin{aligned}&{\overline{\mathbf{10}}}_{-1}^U~~{\overline{\mathbf{10}}}_{-1}^{T_4^A} ~~{\overline{\mathbf{10}}}_{-1}^{T_4^S}&\nonumber \\ M_d\propto \begin{array}{c} {\overline{\mathbf{10}}}_{-1}^U\\ {\overline{\mathbf{10}}}_{-1}^{T_4^A} \\ {\overline{\mathbf{10}}}_{-1}^{T_4^S}\end{array}&\begin{pmatrix}\frac{\sigma _4^2}{M_\mathrm{int}^2},&{}\quad 0~,&{} \quad \frac{\beta _1\sigma _4}{M_\mathrm{int}}\\ ~ 0~~,&{}\quad 0~,&{}1\\ \frac{\beta _1\sigma _4}{M_\mathrm{int}},&{}\quad 1~,&{}0 \end{pmatrix} v_d&,~~\mathrm{with}~\langle H_d\rangle =\frac{v_d}{\sqrt{2}},\nonumber \\ \end{aligned}$$

(29)

where $\beta _1$ is a coupling parameter.

In Table 3 charged singlets are listed that will be needed for the charged lepton Yukawa couplings. Here ${\mathbf{1}}_{-5}$ in U and $T_4^0$ are $Q_\mathrm{em}=+1$ charged leptons. Thus the $Q_\mathrm{em}=-1$ charged lepton mass matrix is

$$\begin{aligned}&{\mathbf{5}}_3^U\quad {\mathbf{5}}_3^{T_4^A} ~~{\mathbf{5}}_3^{T_4^S}&\nonumber \\ M_e\propto \begin{array}{c} {\mathbf{1}}_{-5}^U\\ {\mathbf{1}}_{-5}^{T_4^A} \\ {\mathbf{1}}_{-5}^{T_4^S}\end{array}&\begin{pmatrix}\frac{\sigma _2\sigma _3}{M_\mathrm{int}^2},&{}0,&{} \frac{\gamma _1\sigma _3}{M_\mathrm{int}}\\ 0,&{}0,&{}1\\ \frac{\gamma _2\sigma _2}{M_\mathrm{int}},&{} 1,&{}0 \end{pmatrix}v_d&. \end{aligned}$$

(30)

Similarly, the neutrino mass matrix can be written

$$\begin{aligned}&{\mathbf{5}}_{+3}^{U}\qquad {\mathbf{5}}_{+3}^{T_4^A} \qquad {\mathbf{5}}_{+3}^{T_4^S}&\nonumber \\ M_\nu \propto \begin{array}{c} {\mathbf{5}}_{+3}^{U}\\ {\mathbf{5}}_{+3}^{T_4^A}\\ {\mathbf{5}}_{+3}^{T_4^S}\end{array}&\begin{pmatrix}\frac{\sigma _4\sigma _6^2\sigma _{13}\sigma _{16}}{M_\mathrm{int}^5},&{}0,&{} \frac{\{\sigma _2^2\sigma _4,\, \sigma _2\sigma _5\sigma _9\}}{M_\mathrm{int}^3}\\ 0,&{}0,&{}\frac{\sigma _1^3\sigma _4^2}{M_\mathrm{int}^5}\\ \frac{\{\sigma _2^2\sigma _4,\, \sigma _2\sigma _5\sigma _9\}}{M_\mathrm{int}^3},&{} \frac{\sigma _1^3\sigma _4^2}{M_\mathrm{int}^5},&{}0 \end{pmatrix}\frac{ v_u^2}{M_\mathrm{int}}.\nonumber \\ \end{aligned}$$

(31)

The effective interaction for $M_\nu$ based on SU(3) $_C\times$ SU(2) $_L\times$ U(1)$_Y\times$ U(1)$_\mathrm{anom}$ introduces two $H_u$ insertions. Note that u-type quarks obtain mass from the coupling $\overline{\mathbf{10}}_{-1}\cdot \mathbf{5}_{+3}\cdot \mathbf{5}_{H_u}$ where $\mathbf{5}_{H_u}$ is $\mathbf{5}_{-2}$ in the flipped SU(5). Hence the neutrino mass matrix comes from the square of this which introduces $v_u^2$. The coefficient of ${\mathbf{5}}_{+3}{\mathbf{5}}_{+3} v_u^2$ can be $\langle \overline{\mathbf{10}}_{-1}\rangle ^2/(\mathrm suppression mass)^3={M_\mathrm{GUT}}^2/(\mathrm suppression~ mass)^3$ which we take as $1/M_\mathrm{int}$. In addition there are nonzero factors of $\sigma ^3/M_\mathrm{int}^3$ or $\sigma ^5/M_\mathrm{int}^5$. As illustration, $\sigma /M_\mathrm{int}=0.1-0.5$ and the largest neutrino mass $m_{\nu _\mathrm{max}}=0.5\,\,\mathrm {eV}$ are taken. Then $M_\mathrm{int}\approx 10^{8\,}-10^{11\,}\,\text {GeV}$. Obtaining $M_\mathrm{int}$ from first principles is beyond the scope of this paper.

Inspecting the mass matrices it is concluded that a physical phase in $\sigma _4$ leads to the CKM and PMNS phases.¹¹ For example, there can be a phase generated by the following superpotential:

$$\begin{aligned} W=m\sigma _6\sigma _8+\frac{1}{M^2}\sigma _6\sigma _7 \sigma _2\sigma _4^2 \end{aligned}$$

(32)

where m and M are real parameters, and all fields develop nonvanishing VEVs. Then

$$\begin{aligned} \sigma _4=\pm i\,\left| m M^2\,\frac{\sigma _8}{\sigma _2\sigma _7}\right| ^{1/2}e^{i(\delta _8-\delta _2-\delta _7)/2} \end{aligned}$$

(33)

where $\delta _i$ are the phases of $\sigma _i$. If $\delta _8=\delta _2=\delta _7=0$, the CKM and PMNS phases are determined as $\pm \frac{\pi }{2}$.

The form of mass matrices in Eqs. (28), (29), (30), and (31) can describe the quark and lepton masses successfully by various ratios of the singlet VEVs. Here, however, we will not try to find the relevant ratios.

4.3 The axion–photon–photon coupling

Because all the $Q_\mathrm{anom}$ charges of the electromagnetically charged fermions are known, the axion–photon–photon coupling $c_{a\gamma \gamma }$ can be calculated. In Tables 5 and 6 the typos in the previous tables [35, 36] are corrected. Summing the $Q_a^{\gamma \gamma }$ columns of Tables 1, 3, 5, and 6, we obtain

$$\begin{aligned} c_{a\gamma \gamma }\simeq \frac{-9312}{-3492}-2=\frac{2}{3}, \end{aligned}$$

(34)

which must be the case if $U(1)_{\mathrm{anom}}$ is the PQ symmetry [39]. The subtraction of $\approx 2$ for $m_u/m_d\simeq 0.5$ is due to the contribution from the condensation of light quarks.

5 Conclusion

The only allowed global symmetry $U(1)_{\mathrm{anom}}$ from the heterotic string is used to find the flavor structure of the SM. This global symmetry is the most natural choice for the “invisible” axion from string theory. In addition, $U(1)_{\mathrm{anom}}$ is describing a flavor symmetry. In the flipped SU(5) grand unification of Ref. [28] the mass matrices of quarks and leptons are calculated. It turns out that the fermion mass hierarchy in the SM results from the number of powers of Yukawa couplings, which is a common case in string compactification. Also it is shown how the doublet–triplet splitting in the flipped SU(5) GUT is realized in the model.

Appendix: On the ’t Hooft mechanism

Superstring axions arising from the mother gauge symmetry ${E_8\times E_8'}$ have been considered very early [52]. However, the case has been realized in [53,54,55] with the anomalous U(1) [19, 20] after the orbifold compactification is known which is possible even from the mother gauge symmetry SO(32) because the anomalous U(1) may arise from the spectrum in the twisted sectors. Basically, Barr excluded SO(32) from the untwisted sector spectrum [52]. In all these cases the working principle is the ’t Hooft mechanism with the anomalous gauge U(1) from compactification where the U(1) is a subgroup of ${E_8\times E_8'}$ or SO(32). In this string compactification the Green–Schwarz (GS) counter term [15] is essential.

The conventional ’t Hooft mechanism in gauge theories [44] is the following. Out of two continuous transformation parameters $\alpha (x)$ and $\beta$, acting on the field $\phi$,

$$\begin{aligned} \phi \rightarrow e^{i\alpha (x) Q_\mathrm{gauge}} e^{i\beta Q_\mathrm{global}}\phi , \end{aligned}$$

(35)

the $\alpha$ direction becomes the longitudinal mode of heavy gauge boson. The above transformation can be rewritten

$$\begin{aligned} \phi \rightarrow e^{i(\alpha (x)+\beta ) Q_\mathrm{gauge}} e^{i\beta (Q_\mathrm{global}-Q_\mathrm{gauge} )}\phi . \end{aligned}$$

(36)

Redefining the local direction as $\alpha '(x)=\alpha (x)+\beta$ the transformation is obtained

$$\begin{aligned} \phi \rightarrow e^{i \alpha '(x) Q_\mathrm{gauge}} e^{i\beta (Q_\mathrm{global}-Q_\mathrm{gauge} )}\phi . \end{aligned}$$

(37)

Thus the charge $Q_\mathrm{global}-Q_\mathrm{gauge}$ is reinterpreted as the new global charge and is not broken by the VEV, $\langle \phi \rangle$. Basically, the direction $\beta$ remains as the unbroken continuous direction.

When considering the MI-axion kinetic energy term in 10D [15],

$$\begin{aligned} -\frac{3\kappa ^2}{2 g^4\,\varphi ^2}H_{MNP}H^{MNP}, \quad \mathrm{with}\quad {M,N,P}=\{1,2,\dots ,10\},\nonumber \\ \end{aligned}$$

(38)

which is parametrized after compactification to 4D

$$\begin{aligned} \frac{1}{2\cdot 3! M_{MI}^2}H_{\mu \nu \rho }H^{\mu \nu \rho },\quad \mathrm{with}\quad {\mu ,\nu ,\rho }=\{1,2,3,4\}. \end{aligned}$$

(39)

The GS action in the differential form is [15]

$$\begin{aligned} S_1'= & {} \frac{c}{108\,000}\int \left\{ 30B\left[ (\mathrm{tr}_1 F^2)^2+( \mathrm{tr}_2 F^2)^2- \mathrm{tr}_1 F^2\,\mathrm{tr}_2F^2 \right] \right. \nonumber \\&\left. +\cdots \right\} \end{aligned}$$

(40)

where $\mathrm{tr}_1$ and $\mathrm{tr}_2$ are for the E$_8$ and E$_8'$ representations, which is relevant for our model discussed in the paper. In 4D, it leads to an interaction

$$\begin{aligned} S_1'\propto & {} -\frac{c}{10800}\left\{ H_{\mu \nu \rho }A_\sigma \,\epsilon ^{\mu \nu \rho \sigma }\epsilon ^{ijklmn}\langle F_{ij}\rangle \langle F_{kl}\rangle \langle F_{mn}\rangle \right. \nonumber \\&\left. +\cdots \right\} \rightarrow \frac{1}{3!}\epsilon _{\mu \nu \rho \sigma } H^{\mu \nu \rho } A^{\sigma } \end{aligned}$$

(41)

where we have taken VEVs of internal gauge field strengths, $F_{ij}$ etc. The superstring carries the electric field $A_\mu$, whose group space value is simply denoted A, a kind of matrix.

Fig. 3

The flux $\langle F_{ij}\rangle$ at a fixed point

Fig. 4

The GS terms

First, it is noted how the VEVs are assigned to the field strengths in Eq. (41). Then ’t Hooft mechanism is discussed. The line integral around a fixed point gives an integer. However, in the orbifold compactification we need to line-integrate only a part of $2\pi$. In Fig. 3 the (limegreen) fundamental domain of $Z_3$ is shown. The line integral is for the matrix value gauge field $A_\mu$ which in our case is denoted by the shift vector and Wilson lines, $V,V\pm a_i$ in the (ij) internal plane. By Stokes’ theorem, it is $\frac{1}{3}$ of the surface integral of $F_{ij}$ which is assumed to be located at the fixed point $F_{ij}\delta ^2(\sigma -\sigma _0)$ where $\sigma _0$ is the location of the fixed point in the two-torus. This delta function limit of a zero-length string must be neglected for the string tension contribution. In this way, VEVs are assigned to field strengths in Eq. (41).

It is noted that one scalar degree $H_{\mu \nu \rho }$ is the MI axion [16]

$$\begin{aligned} H_{\mu \nu \rho }=M_{MI}\epsilon _{\mu \nu \rho \sigma }\,\partial ^\sigma a_{MI}. \end{aligned}$$

(42)

In Fig. 4a, the first term of Eq. (41) is shown, transferring one derivative of $F_{\mu \nu }$ to $B_{\rho \sigma }$. Equations (39) and (42) result in the following:

$$\begin{aligned} \frac{1}{2} \partial ^\mu a_{MI}\partial _\mu a_{MI}+ M_{MI}A_\mu \partial ^\mu a_{MI}. \end{aligned}$$

(43)

The global direction with respect to the ’t Hooft mechanism is the $a_{MI}$ direction. Equation (43) can be expressed by adding the contribution of Fig. 4b [which is obtained from Fig. 4a],

$$\begin{aligned}&\frac{1}{2} (\partial _\mu a_{MI})^2+ M_{MI}A_\mu \partial ^\mu a_{MI}\nonumber \\&\quad +\,\frac{1}{2}M_{MI}^2A_\mu A^\mu \rightarrow \frac{1}{2} M_{MI}^2\left( A_\mu +\frac{1}{M_{MI}}\partial _\mu a_{MI}\right) ^2. \end{aligned}$$

(44)

Thus the MI-axion degree is completely removed below the gauge boson mass scale where the heavy anomalous gauge boson including the longitudinal degree is defined to be $\tilde{A}_\mu =A_\mu +(1/M_{MI})\partial _\mu a_{MI}$. The MI axion is dynamicaly removed from the theory except that it couples to the anomaly [56]. This coupling is defining a $\theta$ parameter at low energy depending on some cosmological determination of $\langle a_\mathrm{MI}\rangle$. The resulting global symmetry is broken by the usual Higgs mechanism and it is expected that it is achieved at the intermediate scale.

Now, the charge of the U(1)$_a$ subgroup of ${E_8\times E_8'}$ is denoted $Q_a$ that was anomalous in the beginning. Supposing that $\phi$ carries nonzero gauge charge $Q_a$. All terms involving $\phi$ should respect the original U(1)$_a$ gauge symmetry. Thus the potential V is a function of $\phi ^*\phi$ only.

Below $M_{MI}$ we do not consider the FI term because there is no U(1)$_a$ gauge symmetry. However, to discuss the scales of the U(1)$_\mathrm{anom}$ breaking, we consider the $\phi$ couplings together with the $a_{MI}$ degree. The $\phi$ coupling to $A_\mu$ is given for $\phi =(\frac{v+\rho }{\sqrt{2}})e^{ia_{\phi }/v}$,

$$\begin{aligned} |D_\mu \phi |^2= & {} |(\partial _\mu -ig Q_{a}A_\mu )\phi |^2_{\rho =0}\nonumber \\= & {} \frac{1}{2}(\partial _\mu a_{\phi })^2-gQ_a A_\mu \partial ^\mu a_{\phi }+\frac{g^2}{2}Q_a^2v^2 A_\mu ^2 \nonumber \\= & {} \frac{g^2}{2}Q_a^2v^2\left( A_\mu -\frac{1}{gQ_a v}\partial ^\mu a_{\phi }\right) ^2. \end{aligned}$$

(45)

The gauge boson $A_\mu$ obtains its mass from two contributions, Eqs. (44) and (45); this can be written as

$$\begin{aligned}&\frac{1}{2}\left( M_{MI}^2+g^2Q_a^2v^2 \right) (A_\mu )^2\nonumber \\&\quad +\,A_\mu (M_{MI}\partial ^\mu a_{MI} -g Q_a v \partial ^\mu a_{\phi })\nonumber \\&\quad +\,\frac{1}{2}\left[ (\partial _\mu a_{MI})^2+(\partial ^\mu a_{\phi })^2 \right] . \qquad \qquad (9) \end{aligned}$$

(46)

A new global degree is defined, a, interpretable as the “invisible” axion

$$\begin{aligned} a=\cos \theta \,a_{\phi } +\sin \theta \,a_{MI} \end{aligned}$$

(47)

where

$$\begin{aligned} \sin \theta =\frac{g Q_a v}{\sqrt{M_{MI}^2+g^2Q_a^2v^2}}. \end{aligned}$$

(48)

Thus, if $v\ll M_{MI}$, the global symmetry breaking scale can be at the intermediate scale. The axion has the dominant component from the phase of $\phi$. The determination of v in the effective field theory framework is such that the coefficient of $\phi ^*\phi$ in the potential is given by $-(\mathrm intermediate scale)^2$.