Texture zeros of quark mass matrices at ﬁxed point τ = ω in modular ﬂavor symmetry

We study systematically derivation of the speciﬁc texture zeros, that is the nearest neighbor interaction (NNI) form of the quark mass matrices at the ﬁxed point τ = ω in modular ﬂavor symmetric models. We present models that the NNI forms of the quark mass matrices are simply realized at the ﬁxed point τ = ω in the A 4 modular ﬂavor symmetry by taking account multi-Higgs ﬁelds. Such texture zero structure originates from the ST charge of the residual symmetry Z 3 of SL (2 , Z ). The NNI form can be realized at the ﬁxed point τ = ω in A 4 and S 4 modular ﬂavor models with two pairs of Higgs ﬁelds when we assign properly modular weights to Yukawa couplings and A 4 and S 4 representations to three generations of quarks. We need four pairs of Higgs ﬁelds to realize the NNI form in A 5 modular ﬂavor models.


Introduction
In order to understand the flavor mixing and the CP violation of the quark and lepton sectors, many works were made to find Ansatz for fermion mass matrices and discussed its predictions.The Fritzsch Ansatz [1,2] was a typical example.This approach leads to the texture zero analysis where some elements of mass matrices are required to be zero to reduce the degrees of freedom in mass matrices.Some famous works have been made in the texture zeros [3][4][5].
Along with those works, the nearest neighbor interaction (NNI) form1 is considered as a "general" form of both up-and down-types quark mass matrices because this form is achieved by the transformation that leaves the left-handed gauge interaction invariant [6].Based on the NNI form, some works appeared to explain the flavor mixing of quarks and leptons [7][8][9][10][11].The NNI form is a desirable base to derive the Fritzsch-type quark mass matrix while the NNI form is a general form of quark mass matrices.Therefore, it is important to study quark models to realize the NNI form explicitly.
Phenomenological studies of the lepton flavors have been done based on A 4 [30][31][32], S 4 [33][34][35] and A 5 [16,17].Furthermore, phenomenological studies have been developed in many works.Among them, the texture zeros of quarks and leptons have been discussed in the context of the assignment of the weight for the chiral superfields [36,37].
However, the realization of texture zeros is not necessary to adjust the weight of the chiral superfields.For example, the fermion mass matrix has the texture zero structure at τ = ω = e 2πi/3 due to the Z 3 symmetry independent of the weights [38,39] although the flavor mixing are not reproduced.That is, the theory becomes special due to residual symmetries at the fixed points of SL(2, Z) [40,41].The fixed point τ = ω is also favored from the moduli stabilization.This fixed point has the highest probability in the moduli stabilization due to three-form background [42].Other moduli stabilization mechanisms were studied in modular flavor models [44][45][46][47].The fixed points are also useful to stabilize dark matter candidates [48].
Recently, the CP violation at τ = ω has been discussed in magnetized orbifold models with multi-Higgs modes [49].Magnetized orbifold models are interesting compactification from higher dimensional theory such as superstring theory.They lead to a four-dimensional chiral theory, where the generation number is determined by the size of magnetic flux in the compact space [50,51].The four-dimensional low-energy effective field theory has the modular symmetry [18,[52][53][54][55][56][57].Realization of quark and lepton masses and their mixing angles was studied [58][59][60].Their texture structures were also studied [61].These magnetized orbifold models lead to multi-Higgs modes, while generic string compactification also leads more than one candidates for Higgs fields.
In this work, we present models that the NNI forms of the quark mass matrices are simply realized at the fixed point τ = ω of the modular symmetry by taking account multi-Higgs fields.We use the A 4 modular symmetry as well as S 4 and A 5 .The quark mass matrices with texture zeros, which are consistent with observed CKM matrix elements, are also derived.These models are also simple examples that the CP is violated even at τ = ω.
The paper is organized as follows.In section 2, we present a simple example of the NNI form at τ = ω in A 4 modular symmetry.In section 3, we study a generic model systematically.Section 4 is our conclusion.We summarize group theoretical aspects of A 4 , S 4 , and A 5 in Appendix A and the modular forms of level N = 3 in Appendix B.
2 NNI form of quark mass matrices at τ = ω

Quark mass matrices with mluti-Higgs
In this section, we present a simple model of quark mass matrices in the level N = 3 modular symmetry (A 4 modular flavor symmetry) with the multi-Higgs at τ = ω, which is referred to as Model 1.We assign the A 4 representation and the weights for the relevant chiral superfields as • up and down sector Higgs fields which are summarized in Table 1.Then, the superpotential terms of the up-type quark masses and down-type quark masses are written by where the decompositions of the tensor products are The superpotential terms are rewritten as: Finally, the quark mass matrices are given as: where the chiralities of the mass matrix, L and R are defined as [M u(d) ] LR .

ST -eigenstate base at τ = ω
Let us discuss the mass matrices at τ = ω in the ST -eigenstates.The ST -transformation of the A 4 triplet of the left-handed quarks Q is where representations of S and T are given explicitly for the triplet in Appendix A. The STeigenstate Q ′ is obtained by using the unitary matrix U L as follows: The ST -eigenstates are Q ′ ≡ U † L Q. On the other hand, right-handed quarks, which are singlets (1, 1 ′ , 1 ′′ ), are the eigenstates of ST ; that is, the ST -transformation is Higgs fields are also the ST -eigenstates since they are singlets (1, 1 ′′ ).Therefore, STtransformation of them is In the ST -eigenstates, the quark mass matrices are given as: where Their ratios at τ = ω are obtained as Now, imposing α 1 u,d = 0, we obtain the NNI forms for both the up-type and the down-type quark mass matrices.Therefore, the quark masses and the CKM matrix are reproduced taking relevant values of parameters.It is noticed that the flavor mixing is not realized in the case of one Higgs doublets for up-and down-type quark sectors.Thus, the NNI forms at τ = ω are simply obtained unless the vacuum expectation values (VEVs) of two-Higgs vanish.The general discussion is presented in section 3.
The CP symmetry is not violated at τ = ω in modular flavor symmetric models with a pair of Higgs fields because of the T symmetry [45].However, the models with multi-Higgs fields can break the CP symmetry at the fixed point τ = ω even if all of the Higgs VEVs are real [49].Thus, the CP phase appears in our models, in general.Our models are interesting from the viewpoint of the CP violation, too.
The non-vanishing VEVs of both Higgs fields H 1 u,d and H 2 u,d are important to realize the NNI forms.We expect the scenario that these Higgs fields have a µ-matrix to mix them, Then, a light linear combination develops its VEV, which includes H 1 u,d and H 2 u,d .However, the above assignment of modular weights for the Higgs fields allows the µ-term of only µ 11 , and the others vanish.That is, the mixing does not occur.When we assume the singlet S with the A 4 1 ′ representation develops its VEV, the (1, 2) and (2, 1) elements appear as µ 12 = µ 21 = λ S like the next-to-minimal supersymmetric standard model.
Alternatively, we can assign the modular weights to fields as shown in Table 2.We refer to this model as Model 2. By this assignment, we can obtain the same quark mass matrices as one in Eqs.(15) and (16).The superpotential terms for Higgs µ-terms are given in terms of the weight 4 modular forms as: where 3 ) are given in Appendix B.
The superpotential W H is explicitly given as: Then, mass eigenvalues are written by and the corresponding Higgs directions are written by up to normalization factors.For example, when µ 11 ≫ µ 12 , we obtain Then, the light mode corresponds to the following direction: up to a normalization factor.Heavier Higgs modes would contribute to flavor changing processes.They depend on the mass of the heavier modes, which are free parameters in the above model.Those flavor changing processes are suppressed when heavier modes are heavy enough.Studies on flavor changing processes would be important if we have a scenario to predict the mass scale of heavier modes.That is beyond our scope.

Three pairs of Higgs fields
Similarly, we can study three pairs of Higgs fields with the A 4 (1, 1 ′′ , 1 ′ ) representations.We add another pair of Higgs fields H 3 u,d with the A 4 1 ′ representation of the modular weights, 0 and −2 in models 1 and 2, respectively.Then, the mass matrices are modified as follows: in both models 1 and 2. Thus, this model can lead to a quite generic mass matrix.For example, by setting some of α i u,d , β i u,d , γ i u,d to be zero, we can drive some of texture zero structures including the NNI form.In addition, we can assume to reduce the number of free parameters and realize a certain form of mass matrices.Thus, the different assignment of the A 4 singlets (1, 1 ′′ , 1 ′ ) for Higgs leads to different texture zeros.

Generic models
In the previous section, the quark mass matrices are discussed in the specific modular symmetry of N = 3 in order to show the derivation of NNI forms clearly.Similarly, we can study a generic mode leading to the NNI forms including S 4 and A 5 modular flavor symmetries.

Residual Z 3 symmetry
Because of (ST ) 3 = 1, each field under the ST basis has the Z (ST ) 3 charge.Here, let us discuss the ST charge assignment without specifying the finite modular groups.At first, consider the case of the single Higgs field of up-type quark sector for simplicity as seen in Table 3. Quarks may belong to a multiplet such as a triplet, but we study just the ST charge.
The modular forms transform as: where the Z (ST ) 3 charge q i is defined as including the automorphic factor.Then, the superpotential for up-type quark mass matrix is given as where coefficients of the singlet components are written as: Under the ST transformation of where ω 2k Y ρ jk (ST ) corresponds to the ST charge.Therefore, we obtain non-vanishing components of the mass matrix from the assignment of ST charges for fields in Table 3: Then, the mass matrix is given as: If we take ST charge for the Higgs field is 1 as in Table 4, the mass matrix is given as: Table 4: ST charges of fields of up-type quark sector Finally, taking ST -charge for the Higgs field to be 2 as in Table 5, the mass matrix is: Table 5: ST charges of fields of up-type quark sector By combining these matrices, we can obtain the NNI form as well as other textures.Thus, the Z charge.Hence, we can not realize the above form, which can be derived by the Z (ST ) 3 symmetry.On the other hand, at the limit τ → i∞, the Z (T ) N symmetry remains for Γ N , i.e.T N = 1.Such a residual symmetry may be useful to construct the NNI form.However, the limit τ → i∞ corresponds to decompactification in extra dimensional theory such as superstring theory.
The above discussion is generic without specifying representations.When we specify the representations, the mass matrices are constrained more.Table 6 shows irreducible representations of A 4 , S 4 , and A 5 .In addition, Tables 7, 8, 9 show the relations between representations and weighs of modular forms of A 4 , S 4 , A 5 , respectively.For example, when left-handed and right-handed quarks are assigned to the same triplet, the mass matrix must be symmetric.In addition, the coefficients α i u,d , β i u,d , γ i u.d are not independent parameters, but must be related.We do not have a sufficient number of free parameters to realize realistic masses and mixing angles.Thus, it is important how to assign three generations to irreducible representations.We study this point.
We classify the structures of the superpotential without specifying the finite modular groups.We consider models with two pairs of Higgs fields.They can correspond to either two singlets or a doublet.Three generations of quarks are constructed by combining singlets, doublets and triplets of any finite modular groups.Table 10 shows all possible representation combinations for up-type quarks and two pairs of up-sector Higgs fields.In order to realize NNI forms, Q and q must be decomposed into 1 ⊕ 1 ω ⊕ 1 ω 2 , and H u must be decomposed into 1 ⊕ 1 ω , 1 ⊕ 1 ω 2 or 1 ω ⊕ 1 ω 2 at τ = ω, where 1 ω k denotes the singlet with the Z (ST ) 3 charge k.Note that we need three or more independent parameters for each Higgs fields to realize the NNI form.We show the structures of the superpotential in each case below.
For H u = singlet ⊕ singlet, the superpotential relevant to up-type quark mass is given by For H u = doublet, it is given by Both Yukawa matrices have a sufficient number of free parameters to realized the NNI form.We show them by using A 4 models in the following subsection.
For H u = singlet ⊕ singlet, the superpotential relevant to up-type quark mass is given by For H u = doublet, it is given by Both Yukawa matrices have a sufficient number of free parameters to realized the NNI form.In the case of II', the superpotential is given by exchanging Q and q in the above.However, these assignments can not be realized by A 4 , S 4 , or A 5 symmetry.
III. Q = singlet ⊕ singlet ⊕ singlet, q = triplet: For H u = singlet ⊕ singlet, the superpotential relevant to up-type quark mass is given by For H u = doublet, it is given by Both Yukawa matrices have a sufficient number of free parameters to realized the NNI form.We show them by using A 4 models in the following subsection.In the case of III', the superpotential is given by exchanging Q and q in the above.Models 1 and 2 correspond to this case.
IV. Q = singlet ⊕ doublet, q = singlet ⊕ doublet: For H u = singlet ⊕ singlet, the superpotential relevant to up-type quark mass is given by For H u = doublet, it is given by Both Yukawa matrices have a sufficient number of free parameters to realized the NNI form.We show them by using S 4 models in the following subsection.
V. Q = singlet ⊕ doublet, q = triplet: For H u = singlet ⊕ singlet, the superpotential relevant to up-type quark mass is given by For H u = doublet, it is given by In the case of V', the superpotential is given by exchanging Q and q in the above.The number of free parameters in all of these cases is insufficient to lead to the NNI form.
VI. Q = triplet, q = triplet: For H u = singlet ⊕ singlet, the superpotential relevant to up-type quark mass is given by For H u = doublet, it is given by The number of free parameters in all of these cases is insufficient to lead to the NNI form.

A 4 models
The A 4 group has three singlets 1, 1 ′′ , 1 ′ , and a triplet 3 as irreducible representations.Thus, the cases I, III, III', and VI are possible.The Yukawa couplings, which are written by modular forms, also have irreducible representations.Table 7 shows which representations appear in modular forms for fixed weights.In general, different modular forms with the same representation appears for fixed weights, while modular forms with some representations do not appear.We denote Yukawa coupling of weight k Y by where we put indexes such as 1 j and 3 m , because different modular forms with the same representation appear for a fixed weight.At τ = ω, on ST -eigenbasis, they are transformed as under ST -transformation.Note that τ = ω is a fixed point of ST -transformation as ST ω = ω. with although we can discuss other choices similarly.
Since the ST -invariances restrict Yukawa couplings, the structures of the mass matrix as well as the superpotential are also constrained.For example, let us study the superpotential and the mass matrix in the case I.In this case, quark doublet and right-handed up-type quark singlets are assigned into (1, 1 ′′ , 1 ′ ); up-sector Higgs fields are assigned into either of (1, 1 ′′ ), (1, 1 ′ ) or (1 ′′ , 1 ′ ).We focus on up-sector Higgs fields with (1, 1 ′′ ).Then the superpotential at τ = ω is given by By use of the multiplication rule in Appendix A.1, it follows that The up-type quark mass matrix is obtained as Note that all nonzero elements in the mass matrix are written by independent parameters.Therefore we can realize NNI forms in the case I.
Next, let us study the mass matrix and the superpotential in the case III.In this case, quark doublet is assigned into (1, 1 ′′ , 1); right-handed up-type quark singlets are assigned into the triplet 3; up-sector Higgs fields are assigned into either of (1, 1 ′′ ), (1, 1 ′ ) or (1 ′′ , 1 ′ ).We focus on up-sector Higgs fields with (1, 1 ′′ ).Then the superpotential at τ = ω is given by By use of the multiplication rule in Appendix A.1, it follows that The up-type quark mass matrix is obtained as All nonzero elements in this mass matrix are obtained by independent parameters.Therefore we can realize NNI forms in the case III.Also, it can be realized in the case III'.Indeed, models 1 and 2 correspond to this case.As a result, NNI forms can be found in only the cases I, III and III'.However, the number of free parameters in the case VI is not large enough to realize the NNI form.

S 4 models
The S 4 group has two singlets, 1, 1 ′ , a doublet 2, and two triplets 3 and 3 ′ .Thus, cases IV, V, V', and VI are possible.In S 4 model, Yukawa couplings of weight k Y are denoted by At τ = ω, on ST -eigenbasis, they are transformed as under ST -transformation.As in A 4 models, ST -invariances of Yukawa couplings at τ = ω mean that for k Y = 6n, the following Yukawa couplings remain non-vanishing at τ = ω, while the others vanish.For k Y = 6n + 4, only the following Yukawa couplings remain non-vanishing, while for k Y = 6n + 2, only the following Yukawa couplings remain non-vanishing.In what follows, we focus on the case only (Y with Note that in this choice we have four kinds of nonzero Yukawa couplings, that is, four independent parameters corresponding to irreducible representations of S 4 , 1, 1 ′ , 3 and 3 ′ .In the other choices, we never realize NNI forms in S 4 models because the number of free parameters is not sufficiently large. To find NNI forms, we consider the superpotential in the case IV.In this case, quark doublet and right-handed up-type quark singlets are assigned into (1,2) or (1 ′ , 2); up-sector Higgs fields are assigned into the doublet 2. We focus on quark doublet with (1, 2) and right-handed up-type quark singlets with (1,2).Then the superpotential at τ = ω is given by By use of the multiplication rule in Appendix A.2, it follows that the up-type quark mass matrix is obtained as All nonzero elements in this mass matrix can be written by the parameters α 1,2 , β 1,2 and γ 1,2 , which are independent.Therefore we can realize NNI forms in the case IV.
In a similar way, we can find the structures of the mass matrices in all cases.The number of free parameters is not sufficient in cases V, V', and VI in order to lead to the NNI form.As a result, NNI forms can be found in only the case IV.

A 5 models
In A 5 group, there is only one singlet 1 and no doublets.Thus two pairs of Higgs fields cannot have different ST charges and we never find NNI forms.Table 11 shows our results in models with two pairs of Higgs fields.On the other hand, we can realize the NNI form in A 5 modular symmetric models with four pairs of Higgs fields 2 .The A 5 group has one singlet 1, two triplets 3 and 3 ′ , one four-dimensional representation 4 and one five-dimensional representation 5.In A 5 model, Yukawa couplings of weight k Y are denoted by At τ = ω, on ST -eigenbasis, they are transformed as under ST -transformation.As in A 4 and S 4 models, ST -invariances of Yukawa couplings at τ = ω mean that for k Y = 6n, the following Yukawa couplings remain non-vanishing at τ = ω, while the other vanish.For k Y = 6n + 4, only the following Yukawa couplings remain non-vanishing at τ = ω, while for k Y = 6n + 2, only the following Yukawa couplings remain non-vanishing.In what follows, we focus on the case only (Y with although we can discuss other choices similarly.Note that in this choice we have four kinds of nonzero Yukawa couplings, that is, four independent parameters corresponding to irreducible representations of A 5 , 3, 3 ′ , 4 and 5. Next we consider the superpotential.Since Higgs fields must contain two different STcharge modes to realize the NNI forms, four pairs of Higgs fields should be assigned into one singlet and one triplet, (1,3) or (1, 3 ′ ).Similarly, quark doublet and right-handed up-type quark singlets must contain three different ST -charge modes and they should be assigned into triplets, 3 or 3 ′ .We focus on quark doublet with 3, right-handed up-type quark singlets with 3 and up-sector Higgs fields with (1,3).Then the superpotential at τ = ω is given by where By use of the multiplication rule in Appendix A.3, it follows that To find NNI forms, here we assume H 3 u = 0. Then the up-type quark mass matrix is obtained as This mass matrix has six kinds of independent parameters, α 3 k u1 , α 3 k u2 , α , α 4m u2 , α 5n u1 and α 5n u2 .By taking appropriate liner combinations of them, it is possible to realize the following form matrix, where α 1,2 , β 1,2 and γ 1,2 are liner functions of the above six parameters.Therefore we can realize NNI forms in this A 5 model with four pairs of Higgs fields.

Summary
The NNI form is a desirable base to derive the Fritzsch-type quark mass matrix with specific texture zeros although the NNI form is a general form of quark mass matrices.We have studied the flavor models of quarks systematically to realize the NNI form of mass matrices explicitly.
The NNI form of quark mass matrices are derived in modular flavor symmetric models at the fixed point τ = ω.We have presented models that the NNI forms of the quark mass matrices are simply realized at the fixed point τ = ω in the A 4 modular flavor symmetry by taking account multi-Higgs fields.Those are also simple examples that the CP is violated even at τ = ω in the case of the finite modular symmetry with multi-Higgs fields.We show that such texture zero structure originates from the ST charge of the residual symmetry Z 3 of SL(2, Z).
The NNI form can be realized at the fixed point τ = ω in A 4 and S 4 modular flavor models with two pairs of Higgs fields, when we assign properly modular weights to Yukawa couplings and A 4 and S 4 representations to three generations of quarks.We need four pairs of Higgs fields to realize the NNI form in A 5 modular flavor models.Thus, the modular flavor models with multi-Higgs fields at the fixed point τ = ω leads to successful quark mass matrices.We can extend our analysis to the lepton sector.Extension to the charged lepton mass matrix and Dirac neutrino mass matrix is straightforward, and we obtain the same results.Similarly, we can study the right-handed neutrino mass matrices.For the right-handed neutrino sector, the symmetric assignments are possible in Table 10, i.e. cases I, IV, and VI, and parameters in the mass matrix should be symmetric.Since the Higgs VEVs do not appear in the right-handed neutrino mass matrix, we replace the Higgs VEV of the trivial singlet by a constant with setting the Higgs VEVs with other representation to be zero.Case VI has a limited form of the mass matrix with one free parameter, while case I has a generic (3 × 3) symmetric mass matrix.It is interesting to study more phenomenological aspects on the flavor physics based on our systematical analysis at the fixed point τ = ω in the near future.
In A 4 group, there are four irreducible representations, three singlet 1, 1 ′′ and 1 ′ and one triplet 3.Each irreducible representation is given by in the T -diagonal basis.In the ST -diagonal basis, they have the following ST -eigenvalues.
Their multiplication rules are shown in Table 12.

A.2 S 4
The generators S and T of S 4 group satisfy the following algebraic relations: In S 4 group, there are five irreducible representations, two singlet 1 and 1 ′ , one doublet 2 and two triplets 3 and 3 ′ .Each irreducible representation is given by in the T -diagonal basis.In the ST -diagonal basis, they have the following ST -eigenvalues.
Tensor product Table 13: Multiplication rule in irreducible representations of S 4 .The second column shows decompositions in the T -diagonal basis and the third column shows ones in the ST -diagonal basis.

5
NNI forms I, III, III' IV NoneTable 11: NNI forms in A 4 , S 4 and A 5 models with two pairs of Higgs fields.

Table 14 : 1 b 1 +a 2 b 3 +a 3 b 2 ⊕ a 2 b 3 −a 3 b 2 a 1 b 2 −a 2 b 1 a 3 b 1 −a 1 b 3
Multiplication rule in irreducible representations of A 5 .The first column shows tensor product decompositions in the T -diagonal basis and the second column shows ones in the ST -diagonal basis.3⊗ 3 = 1 ⊕ 3 ⊕ 5 (a i b j ) T -diagonal basisST -diagonal basis a

Table 2 :
Assignments of Model 2

Table 10 :
All possible representation combinations for up-type quarks and two pairs of up-sector Higgs fields.
and it vanishes unless k Y = 6n.The other Yukawa couplings have similar behaviors.Then, it is found that when k Y = 6n, Y 1 j (ω) and Y 1 3m (ω) are non-vanishing, while the other Yukawa couplings vanish.Similarly, only Y 1 ′′

Table 12 :
Multiplication rule in irreducible representations of A 4 .The second column shows decompositions in the T -diagonal basis and the third column shows ones in the ST -basis.