Modular Origin of Mass Hierarchy: Froggatt-Nielsen like Mechanism

We study Froggatt-Nielsen (FN) like flavor models with modular symmetry. The FN mechanism is a convincing solution to the flavor puzzle in quark sector. The FN mechanism requires an extra $U(1)$ gauge symmetry which is broken at high energy. Alternatively, in the framework of modular symmetry the modular weights can play the role of the FN charges of the extra $U(1)$ symmetry. Based on the FN-like mechanism with modular symmetry we present new flavor models for quark sector. Assuming that the three generations have a common representation under modular symmetry, our models simply reproduce the FN-like Yukawa matrices. We also show that the realistic mass hierarchy and mixing angles, which are related each other through the modular parameters and a scalar vev, can be realized in models with several finite modular groups (and their double covering groups) without unnatural hierarchical parameters.


Introduction
The origin of the flavor structure of the Standard Model (SM) is one of the most challenging problems in the particle physics. The observed mass eigenvalues of the matter fields have a large hierarchy, which is more than 10 6 among them. This hierarchy can not be explained in the framework of the SM, since the Yukawa couplings are free parameters in the SM. Thus we require a beyond the SM mechanism naturally reproducing it.
The modular symmetry is a recently proposed solution for the flavor puzzle [1,2]. In this model, the action is assumed to be invariant under the (inhomogeneous) modular group Γ P SL(2, Z), which is the quotient group of SL(2, Z) divided by its center {I, −I}. In this model, coupling constants are no longer free parameters, but modular forms. The modular forms are specific holomorphic functions of the complex parameter known as the modulus. They form unitary representations of the quotient group of the modular group: Γ N = Γ/Γ(N ). Γ(N ) is known as a principle congruence subgroup of Γ. Γ N is isomorphic to a non-Abelian discrete group. In particular, it is isomorphic to a finite group when the level N is lower than 6 [3]. Therefore coupling constants, as well as the dynamical fields, transform as unitary representations of the non-Abelian finite group in this model. This is attractive for the particle phenomenologist since discrete symmetries are well known candidate solutions for the flavor puzzle, especially for the lepton flavor structure, [4,5,6]. 1 Indeed, various modular invariant models which successfully reproduce the SM have been constructed in these years, e.g., for Γ 2 [10,11,12,13], Γ 3 [1,2,10,13,14,15,16,17,18,19,20], Γ 4 [13,16,21,22,23,24], Γ 5 [13,24,25], Γ 7 [26], and the double covering groups of the modular groups [13,27,28,29,30,31,32,33]. A combined symmetry of the modular symmetry with the conventional flavor symmetries or CP-symmetry is also considered in [34,35,36,37,38,39,40,41].
The Froggatt-Nielsen (FN) mechanism is another well-known possible solution for the flavor puzzle [42]. In the original FN mechanism, the matter fields such as the left-handed quarks Q i and the right-handed quarks u c i and d c i are assumed to be charged under an extra gauge group denoted by U (1) F N , which prohibits the tree level Yukawa coupling except for the top quark. An extra scalar field φ, which is a trivial singlet under the SM gauge group, is introduced to spontaneously break U (1) F N . Since the effective Yukawa couplings are given by higher order couplings suppressed by its vacuum expectation value ( φ /Λ) n , where n is a difference of U (1) F N charges between generations, the mass hierarchy is controlled by the U (1) F N charges of the quark fields. It is interesting that in the FN model the mass ratios are also related to the mixing angles, so that it naturally explains a realistic mass hierarchy and the mixing angles of the quark sector simultaneously if φ /Λ is chosen to the Cabibbo angle [9].
We consider FN-like mechanism in the framework of modular symmetry. In analogous to the FN mechanism, the modular weights of the fermion fields play the role of U (1) F N charges. An extra SM singlet φ with a negative modular weight is introduced, which compensates the modular weights of the fermion fields. The effective couplings are given by higher dimensional operators suppressed by powers of (φ/Λ) n , where n is a difference of modular weights between generations. The Yukawa coupling itself is also controlled by the modular weights of the fermion fields, since the modular forms are classified by modular weights. This FN-like mechanism based on modular symmetry has been recently considered to explain the large mass hierarchy in [24,43,44,45]. 2 We note that while in the previous model the mass hierarchy originates from powers of the weighton vev, the resulting quark mass matrices do not fully simulate the FN-like structure, so that the relationship between two origins of mass hierarchy and the small mixing angles becomes more subtle than that of the FN model.
In this paper, we present new flavor models for quark sector based on the FN-like mechanism with modular symmetry. In our models, it is assumed that the three generations of singlet quarks have a common representation under the modular symmetry, so that we see the same order of suppression factors appearing in each column or each row of the Yukawa matrices, where the two different suppression factors (the exponentially suppressed factor in the modular forms and a power suppression by φ/Λ) are incorporated, respectively. Thus our models can simply reproduce the characteristic structure of the FN-like mass matrices. We illustrate this mechanism in models with different finite modular groups of Γ 3 , Γ 4 and Γ 5 in detail. Following Ref. [43], we analyze an approximate expression for the mass ratios and the mixing angles, where a hierarchical mass structure which relates to the mixing angles is easily obtained with a suitable choice of the modular parameters and the singlet vev. We then numerically confirm this mechanism by a fit analysis, where we find parameter sets for the realistic mass hierarchy and mixing angles with O(1) coefficients. The validity of the approximate estimation and the stability against changes of free parameters are also numerically investigated through the parameter dependence of the results. This paper is organized as follows. In section 2, we briefly review the modular symmetry. We also explain possible two origins of the hierarchy for the modular symmetry; one is the hierarchy among the modular forms, and the other is FN mechanism. In section 3, we consider Froggatt-Nielsen like superpotential with the modular group of level 3. In this model, hierarchical mass matrix is realized by cooperation between the above two possible origins. In section 4, we generalize the previous model to the modular group of higher levels. In section 5, we investigate our models statistically. Section 6 is devoted to the conclusion. We also review the modular forms of level 3, 4, and 5 in Appendix.

Modular symmetry and the Froggatt-Nielsen mechanism
In this section, we briefly review the modular symmetry and the Froggatt-Nielsen mechanism. We also introduce our notations mostly based on [1]. The modular symmetry is a recently proposed model building framework. In this model, the action is assumed to include a complex parameter known as the complex structure modulus τ ∈ H. The complex structure parameterizes the geometry of torus. Torus is invariant under the linear fractional transformation, where γ = a b c d ∈ SL(2, Z) since τ and γτ generate the same lattice. It is obvious that both γ and −γ equivalently act on τ . Thus the torus is invariant under P SL(2, Z) ≡ SL(2, Z)/Z 2 . This is the (inhomogeneous) modular group Γ. The modular group is generated by two generators, and these generators satisfying the following relations where I is the identity. In this paper, we abuse elements of SL(2, Z) to denote the corresponding elements of the modular group SL(2, Z)/Z 2 . For instance, S is the element of SL(2, Z) originally, but the same symbol also denotes the equivalence class in the modular group. The modular group acts on the effective action. For instance, coupling constants such as Yukawa couplings depend on the moduli τ . They transform under the modular group through the modular transformation of τ . To construct a modular invariant action, coupling constants should form representations of the modular group, and such functions are known as the modular forms. Before considering modular forms, we introduce the principal congruence subgroup of level N in Γ, which is usually represented by Γ(N ). Γ(N ) is given by The modular forms of level N and weight k are holomorphic functions satisfying the following transformation for any γ in Γ(N ). The prefactor (cτ + d) k is so-called automorphy factor. Since linear combinations of the modular forms of level N and weight k are also modular forms of level N and weight k, they form a linear space, which is denoted by M k (Γ(N )). M k (Γ(N )) is finite dimensional. The modular forms form unitary representations of the quotient group Γ N = Γ/Γ(N ) up to the automorphy factor, where {f i } is a basis of M k (Γ N ), and ρ is a unitary representation of Γ N . The generators of Γ N are satisfying the following relations, 3 Γ N is isomorphic to the non-Abelian finite group when N is smaller than 6: [3]. Note that the definition of the automorphy factor has ambiguity since the modular group is divided by its center. Hence the modular forms are well-defined only if the modular weight is even. If we consider the double covering group of the modular group instead of the usual modular group, modular forms of odd modular weights can be defined as well. The double covering group of the modular group is known as homogeneous modular group Γ . To distinguish Γ from Γ , Γ is called inhomogeneous modular group. Γ is nothing but SL(2, Z) itself, and there are no ambiguities of sign of c and d. The principal congruence subgroup of level N in the homogeneous modular group, and its quotient group Γ N is similarly obtained as where Γ (N ) is a subgroup of Γ whose element is equivalent to I mod N . The generator of Γ N are satisfying the similar relations: Γ N is isomorphic to non-Abelian finite group when N < 6, too: Γ 2 = Γ 2 S 3 , Γ 3 T , Γ 4 S 4 and Γ 5 A 5 [27]. The modular symmetry is originally inspired by string compactifications [49,50,51,52,53,54]. From the stringy perspective, the coupling constants and dynamical fields transform under the homogeneous modular group rather than the inhomogeneous modular group [36,35,55,56,57,58,59] and metaplectic group [60]. Hence we consider modular symmetric model based on Γ in this paper.
Throughout this paper, we assume global supersymmetry. The matter fields such as quarks are denoted by chiral superfields. We assume that they transform as the modular forms of level N and weight k, where Φ i is a matter field and ρ Φ (γ) is a unitary matrix. The action of chiral superfields Φ i is given by two functions, the Kähler potential K and the superpotential W , . 3 We abuse elements of SL(2, Z) 2 to denote elements of Γ N too.
The typical modular invariant action is given as 4 where f i 1 i 2 ...in (τ ) is a modular form of weight k satisfying so that the modular weight of the superpotential is zero. 5 The modular weight plays a similar role of the charge of U (1) gauge symmetry.
..Φ in ) 1 denotes the trivial singlet component of the tensor product of the chiral superfields and the modular form. We consider canonically normalized couplings rather than the holomorphic couplings. The canonically normalized fieldΦ i is given bỹ and the canonically normalized n-point couplingf i 1 i 2 ...in is given bỹ where k is the modular weight of f i 1 i 2 ...in itself. In order to be invariant under modular symmetry, the allowed Yukawa couplings are also given in term of the modular forms, which are classified by modular weights. In the followings, we discuss the properties of the modular forms.

Hierarchy in the modular forms
The modular forms naturally have a hierarchy. To illustrate this point clearly, we consider the modular forms of level 3 as a concrete example. The modular forms of level 3 and weight k form a k + 1 dimensional linear space M k (Γ(3)) [27,28]. The modular forms of weight 1 are given byê This term implies that the modular weight of the superpotential is h rather than zero, and the modular invariant condition (13) is changed to k + k i1 + ... + k in = h. However it is always possible to cancel it by shifting the modular weight of the chiral superfields. Therefore we assume (13) throughout this paper.
where η(τ ) is the Dedekind eta function, Note that since we consider the homogeneous modular group, odd weight modular forms can be defined. The modular transformations ofê 1 andê 2 are given by and we can check that the action of the modular group is closed in this space. Since the modular forms form representations of Γ 3 , they can be decomposed to the irreducible representations of Γ 3 T . The irreducible representations of T are as follows, 6 1, 1 , 1 , 2, 2 , 2 , 3.
The modular forms of higher weights are constructed by the tensor products of the modular forms of weight 1, e.g., the modular forms of weight 2 are given by and they form 3 of T . We can obtain the higher weight modular forms in a similar way. The modular forms have q-expansion. Y Thus Y 1 Y 2 for large Im τ . This is a general feature for the modular forms which transform as 2 of T . The matrix representation of ρ 2 (T ) are given by It is clear that q 1/M transforms under T as Thus the modular forms which transform as 2 of T must have the q-expansion of the following form, where C (k) n and D (k) n are coefficients independent of τ . Thus we can generally approximate the modular forms of 2 as for large Im τ . More precisely, the leading terms of the q-expansions are not uniquely determined by the matrix representations of T . They always have ambiguity of integer powers of q. Thus the modular forms are approximated as where M 1 and M 2 are appropriate integer numbers. To determine the correct hierarchy, we must calculate the explicit forms of the modular forms. Nevertheless we obtain the hierarchical values in either case since the powers of q of the leading terms can not be the same. The matrix representation of T for other representations are given in Appendix A.
The modular forms of the other representations have q-expansion of (27) and they are approximated by Therefore the modular forms naturally have large hierarchy for large Im τ . This feature is general for the modular group of other levels. We summarize the result of Γ 4 in Appendix B, and that of Γ 5 in Appendix C.

Froggatt-Nielsen mechanism
The Froggatt-Nielsen (FN) mechanism is a well-known candidate solution for the flavor puzzle. In FN mechanism, one introduces an extra Abelian gauge group U (1) F N , and assumes that the matter fields such as the left-handed quark field Q i and the right-handed quark fields u c i , d c i are charged under U (1) F N . We assume that Higgs fields H u and H d are neutral under U (1) F N . The Yukawa couplings are prohibited by U (1) F N unless the sum of U (1) F N charges of the corresponding fields are canceled. We introduce an extra scalar field φ which has the U (1) F N charge of −1. We also assume that φ is the trivial singlet under the SM gauge group. After the U (1) F N is spontaneously broken down at high energy scale Λ, φ obtains vacuum expectation value φ and the effective superpotential for the quarks would be given by ij are free parameters supposed to be order 1. Ifφ is sufficiently small (and the U (1) F N charges of the quarks are large enough), effective Yukawa couplings much less than 1 are obtained.
A typical example for the quark charges reproducing the observed mass eigenvalues and the mixing angles is given by In this case, Yukawa couplings of order 1 are prohibited except for the top quark, and the light quark mass terms are given by nonrenormalizable higher order terms. The quark mass matrix is given by [47] The left-handed quarks Q i are on the left side of the mass matrix, and the right-handed quarks u c i , d c i are on the right side in our notation [48]. The mass matrix is diagonalized by , and the Cabbibo-Kobayashi-Maskawa (CKM) matrix is given by The mass eigenvalues of (29) are approximated by m u ∼φ 6 v u , m c ∼φ 4 v u and m t ∼ v u for the up sector, and they are given as m d ∼φ 5 v d , m s ∼φ 3 v d and m b ∼φv u for the down sector.
In terms of the mixing angles, we consider two-flavor model at first for simplicity. Suppose that the mass matrix of the up and down quarks are given by where i , δ j are small parameters. The eigenvalues of M u are obtained by the following eigenvalue equations, We obtain The diagonalizing matrix is given by The calculation for the down sector is completely parallel, and we obtain the CKM matrix . . . required, but a similar role is played by the modular group. In this section, we explain this idea. We also explain a difference between our model and the previous ones. We consider a modular symmetric models with Γ 3 = T at first. It is straightforward to generalize the idea to the modular group of another level. Suppose that the quark fields and their representations under the finite modular group are denoted by ρ Q , ρ u , ρ d , respectively. We can set the modular weights of the Higgs fields to zero without loss of generality since they can be absorbed by shifting the modular weight of the other fields. We also assume that the Higgs fields are the trivial singlet of the modular group. The tree-level superpotential is given by where the Yukawa couplings Y are the modular forms of weight −k Q i − k u j and −k Q i − k d j . Suppose ρ Q = 3 and ρ u = ρ d = 1 of T , and their modular weights are given by where odd modular weights to the quark fields are allowed if we consider the double covering group of the modular group. In this case, the Yukawa couplings must be 3. We show the modular forms of level 3 and weight k < 7 in Table 1. The tree level couplings are prohibited except for u c 3 because of the absence of the triplet modular forms for the odd weights. Hence the modular invariant superpotential is obtained as where Y 3,I and Y 3,II denote the two modular forms of weight 6 which transform as the triplet under T . γ u and r u are arbitrary coefficients supposed to be order 1. The explicit form of Y (6) 3,I and Y (6) 3,II are given by (99). γ u is real and r u is complex since the phase of γ u can be absorbed by redefinition of u c 3 . This superpotential would corresponds to the Yukawa couplings of the top quark. The other couplings are given as non-renormalizable higher order couplings. We introduce a new chiral superfield φ whose modular weight is −1. This φ is called weighton since it carries the unit of modular weight [43]. We assume that φ is the trivial singlet under both T and the SM gauge group. After breaking the modular symmetry, φ develops its vev, and the effective superpotential should be given by whereφ denotes φ /Λ and Λ is the cutoff scale. α f , β f , γ f and r f , where f denotes u or d, are order 1 parameters. The superscript tri explicitly indicates that the left-handed quarks form the triplet. In our basis, the irreducible decomposition of 3 ⊗ 3 is given by If Im τ is large, the modular forms Y (k) 1,2,3 are approximated by q 0,1/3,2/3 respectively (see (27)). Then the superpotential is approximated for large Im τ as and the mass matrix is approximated by We can always exchange the indices of the quark fields freely. We redefine the left-handed quark fields Q i as 7 The Clebsch-Gordon (CG) coefficients of T in our notation are summarized in Appendix A. and the mass matrix is rewritten as The determinants of these two matrices are proportional toφ 4 q, and the largest eigenvalue is of order ofφ 0 q 0 . Thus we have a natural hierarchy. It is the same as the FN mechanism (29). If q 1/3 ∼φ 3/2 , we obtain the FN-like mass matrix. Thus we expect that these mass matrices reproduce the mass hierarchies and the mixing angles of the quarks. Note that the right-handed quarks u c i (and d c i ) must be the same representation to realize FN-like mass matrix. For example, suppose u c 2 is assigned to 1 , and u c 3 to 1 , and the other right-handed quarks are the trivial singlet 1. In this case, the effective superpotential of the up sector is changed to The CG coefficients in terms of (3 × 3) 1 , and (3 × 3) 1 are summarized in (96). The mass matrix is approximated by The eigenvalues of (43) are approximated byφ 3 v u ,φ 1 v u , andφ 0 v u in the limit of Im τ → +∞. It may reproduce the small values of the mixing angles since it is close to diagonal matrix. Indeed, in the previous modular symmetric models, the mass matrices are the same form as (43). In this case, however, the source of the mass hierarchy and that of the Cabbibo angles are independent each other [43]. On the other hand in the case of the "FNlike" mass matrices in (41) they are related each other through the moduli parameters and the singlet vev. As it will be discussed later, the empirical relations between mass ratios and the mixing angles in (31) can also be realized with O(1) coefficients. For our purpose it is important to assume that the right-handed quarks are the same representation under modular symmetry. To construct a mass matrix similar to (41), we have a restriction on the modular weights too. Since the components of the modular forms are aligned in the same order at each row in our model, the weight of the Yukawa couplings of each generation must be different to realize the full rank matrix. The modular forms of weight higher than 5 are required at least. The modular forms of modular weight higher than 5 is also required for the CP-phase. The complex phases of the modular forms do not affect the CP-phase for large Im τ , since the mass matrix is approximated by (41), and the phase of q, i.e., Re τ , can be absorbed by field redefinition of Q i . The phase of coefficients α f , β f and γ f , as well as that of φ are also absorbed by u i and d j . Hence r u and r d in (36) are the only source of CP-violation in our model. 8 Such a CP-phase appears if M k (Γ(3)) has multiple triplets. It is satisfied when the modular weight is higher than 5.
We also comment on the sub-leading terms. Suppose that the leading Yukawa term is given by then we have sub-leading couplings, The sub-leading term is suppressed byφ 2 compared to the leading term because the weight of the modular forms which is 3 of T must be even and positive. As shown in the following analysis,φ is of order 10 −2 for the realistic models, and we can omit the subleading terms. This also implies that the smallest value of k (the power ofφ in the leading Yukawa term) should be lower than 2 in general. The only exception is the Yukawa term in which the modular weight of the Yukawa coupling is the lowest, that is, = 2 in the case of Y ( ) 3 . In this case it is possible to assign an arbitrary positive power ofφ at the leading order, since there is no modular form with lower modular weight < 2 for Y ( ) 3 (See Table 1).
Finally, we obtain the general superpotential of our FN-like model with Γ 3 : where I, J, K, L, M, N and k i , i are integer numbers satisfying the following conditions: As mentioned above, if k i or i is equal to 2, the corresponding capital index can be arbitrary integer number, otherwise the capital indices must be 0 or 1.
is the smallest weight full rank model, which has the smallest number of free parameters because the dimension of M k (Γ(3)) monotonically increases as the weight k increases.

Models with the singlet left-handed quarks
We construct a similar model by exchanging the representations of Q i and (u c i , d c i ). Suppose that Q i are the trivial singlet of T , and u c i and d c i form the triplet of T , then we obtain the following superpotential: where In this case, the powers ofφ and the modular weights of the fields satisfy the following conditions, The mass matrices are approximated as We also obtain the hierarchical mass matrix. In fact, this mass matrix is the transposed matrix of the previous one in (41).
is the model with the lowest weight modular forms. In this case, the constraints (49) implies

Numerical analysis of mass ratios and the mixing angles of Γ 3 models
The origin of the modular symmetry is the geometrical symmetry of the extra dimensions. Hence we should evaluate the Yukawa couplings at the compactification scale. We assume that the compactification scale is the GUT scale (2 × 10 16 GeV). The Yukawa couplings at high energy scale receives quantum corrections, and they are given by solving the renormalization group equation. They depend on the physics beyond the standard model. In this paper, we assume a minimal SUSY breaking scenario with tan β = 5 [64,65]. At the GUT scale, the Yukawa couplings are calculated as We explicitly show 1σ interval for every observable. In the following analysis, we concentrate on the ratios of the Yukawa couplings rather than the Yukawa couplings themselves, since the overall factor is irrelevant to our study. The ratios of the Yukawa couplings are calculated as Similarly, the mixing angles and CP -phase consistent with the experimental results at the GUT scale are given by θ obs 12 = 0.22736 ± 0.00142, θ obs 23 = 0.03585 ± 0.00670, θ obs 13 = 0.003145 ± 0.000490, δ obs CP = 1.206 ± 0.108. Our notation of the mixing angles and the CP-phase is based on the PDG [48]. The quark sector has 9 observables to fit.
In this section, we analyze the mass hierarchy and the mixing angles of our FN-like model. The superpotential with Γ 3 are summarized in (46) and (48). We consider the superpotential with the lowest weight modular forms.

Model with triplet left-handed quarks
First we consider the FN-like model based on the superpotential of (46). Before investigating the numerical analysis, we should study the structure of the mass matrix analytically. The physical mass matrix is given in terms of the canonically normalized Yukawa couplings in (15) as where we note that in order to clearly see a FN-like hierarchy we redefine the indices of the quark fields as where we implicitly assume the following conditions 9 Thus the largest Yukawa coupling for the down sector would be γ dφ N Y 1 , while that for the up sector γ uφ K (Y 1,I + r u Y 1,II ). Using explicit q-expansions in Appendix A we obtain an approximate estimation of the mixing angles for large Im τ as We see that the approximate mixing angles do not depend onφ, α u,d , β u,d , and γ u,d . The first relation implies 15|q 1/3 | ∼ θ C ∼ 10 −1 , which is satisfied when Im τ ∼ 2.4. The other two conditions are rewritten as Then we can realize the observed values for r u = O(1). We also obtain a natural hierarchical structure for the mass ratios which are suppressed by powers of θ C andφ as for the up sector, and for the down sector, where we use Im τ ∼ 2.4. We solve these equations under the conditions (55). For simplicity we assumeφ = 10 −2 for the later estimation. We then find a solution of I = 2, J = 1, K = 0, M = 1, N = 1, and L = 1 or 2, which can reproduce the observed mass ratios in (52) with O(1) coefficients of and r u,d . To confirm our analysis we construct an explicit model which satisfies the above conditions. The representations and modular weights of the quark fields are summarized in Table 2. The mass matrix is given by (53) with I = 2, J = 1, K = 0, and L = 2, M = N = 1. We set Re τ = 0 since the complex phase factor in the modular forms is negligibly small for large Im τ . We also assume absolute value of r f is 1 at first. Thus we have 8 free parameters: α u /γ u , β u /γ u , r u , α d /γ d , β d /γ d , r d , Im τ and γ d /γ u .φ is not counted as a d.o.f since it is absorbed by the coefficients. The best fit parameters in our search are given by The most hierarchical parameter is β u /α u = 12.7 in this parameter set. The FN-like mass matrices are successfully obtained as, As the result, we obtain the following mass eigenvalues and mixing angles, In this case, though we have only 8 free parameters, all mixing angles and mass eigenvalues except for the mixing angles θ 13 are reproduced within 2σ range of the observed values. In this case, χ 2 , which is given by where Observables = {y u /y t , y c /y t , y d /y b , y s /y b , y b /y t , σ 12 , σ 23 , σ 13 , δ CP }, is estimated as If we relax the restriction on |r|, we can realize the observed values more precisely, while the number of free parameters is more than observed values. A benchmark value is obtained as The most hierarchical parameter is β u /γ d = 8.61 in this parameter set, and all the coefficients can be the same order. The mass matrix is given by We obtain the following mass eigenvalues and mixing angles In this case, all the observables are within 0.4σ range, and χ 2 ∼ 0.1. Hence we can realize the realistic values without hierarchical parameters.

Model with singlet left-handed quarks
Here we consider a model with singlet left-handed quarks based on the superpotential of (48). Changing the flavor indices of the right-handed quarks for both up and down sectors as 3,I + r u Y 1,I + r u Y 3,I + r d Y 2,I + r d Y 1,I + r d Y which is the transposed matrix of (53). In the case of the lowest weight modular forms, we have additional conditions of (51), which implies I − J = L − M and J − K = M − N . Thus the same powers ofφ arise in the mass ratios for both the up and down sectors. In fact, an approximate expression of the mass ratios are given by Therefore unnatural hierarchical coefficients are inevitable to obtain the realistic mass hierarchy.
We show an explicit model. The modular weights and representations for the best fit model are summarized in Table 3. The mass matrices are given by (64) with I = L = 2, J = M = 1, K = N = 0. The best fit values are given by where hierarchical coefficients are required. We obtain the mass matrix, where χ 2 ∼ 20. Even though we relax the constraints on |r|, we can not realize the observed values in this model. As shown above, in the case of the lowest modular weight forms, hierarchical parameters should be required even if we consider other modular groups of higher levels. Thus singlet left-handed quark model is not suitable for our purpose. Hereafter we only consider the models where the left-handed quarks form a triplet of the modular group. 4 Froggatt-Nielsen like mechanism with the modular groups of higher levels.
It is straightforward to generalize this mechanism to the modular group of the other levels.
The only requirement is the existence of triplet modular forms which have hierarchical components. This condition is satisfied for the modular group of level N ≥ 3.

FN-like mechanism with the modular group of level 4
The algebra of Γ 4 is summarized in Appendix B. Γ 4 is isomorphic to S 4 SL(2, Z 4 ). It has four triplet representations, 3,3, 3 and3 . The matrix representations of T in this algebra are summarized in Table 8 in Appendix B. The triplet modular forms are approximated as for large Im τ . Thus the FN-like hierarchical mass matrix can be realized in the similar way. We consider four classes of the modular invariant superpotentials: f denotes the flavor u or d, and φ is the trivial singlet carrying modular weight −1, i.e., weighton. We can assign f c i to the trivial singlet 3 of Γ 4 without loss of generality. Q c i is assigned to 3 for W i ,3 for W 2 , 3 for W 3 , and3 for W 4 . The CG coefficients of the tensor products of the triplets are given by (101). We obtain the mass matrix, is the i-th component of the corresponding triplet modular form of weight k j . The mass matrix is approximated as for large Im τ . We obtain the FN-like mass matrix and hierarchical mass eigenvalues. We can choose the superpotential of the up sector and that of the down sector from W 1 , W 2 , W 3 and W 4 individually. Thus we have 16 classes of FN-like models with Γ 4 in principal. However, we find that it is difficult to obtain the observed mixing angles if we use a different type of the superpotential for each sector. As shown in (67) the position of the largest component is different for each representation. If we use a different representation for each sector, we can not obtain FN-like Yukawa matrices for both sectors simultaneously, where the order of the contribution to a mixing angle from each sector could be different, and one of the mixing angles may become large. 10 We have the same constraints on the power ofφ as those in Γ 3 . Namely, the powers ofφ are 0 or 1 in general, but they can be arbitrary positive integer if the corresponding Yukawa coupling is the lowest weight modular form. The lowest weight is 1 for3, 2 for 3 , 3 for3 and 4 for 3.

Lowest weight models
The superpotential W i has several free parameters. They are proportional to the number of the triplet modular forms in the superpotential. In order to minimize the number of the free parameters we consider the model with the lowest weight modular forms. They are given by 3 3 3 3 3 The total number of the free parameters are the same as that of the superpotential of Γ 3 . α f , β f , γ f are real numbers, and r f is a complex number. Thus we have five real parameters for each sector. I can be an arbitrary positive integer number, but J and K are restricted to 0 or 1.

FN-like mechanism with the modular group of level 5
The algebra of Γ 5 is summarized in Appendix C. Γ 5 is isomorphic to A 5 SL(2, Z 5 ) having two triplets 3 and 3 . Their matrix representations are shown in Table 10. The triplet modular forms are approximated as for large Im τ . We can construct the FN-like model in the same way. We have two classes of the FN-like superpotential, with the triplet left-handed quarks. We assume that f c i is the trivial singlet, and Q i form a triplet of Γ 5 . The CG coefficients are summarized in (103). The mass matrices are approximated as for large Im τ . Thus we obtain FN-like hierarchical eigenvalues for both cases. We have 2 possibilities of the representations of the Yukawa couplings both for the up and down sectors. We note, however, that since the tensor product of 3 and 3 has no singlet, the Yukawa couplings and Q should be the same representation. Therefore we have 2 possibilities of either 3 or 3 for both up and down sectors. We have the same constraints on the power ofφ as in the previous models. In this case, the lowest weight is 2 both for 3 and 3 .

Lowest weight models
The superpotential including the lowest weight modular forms are given by 3 3 3 3 The number of the free parameters are the same as that of the superpotential of Γ 3 and Γ 4 . α f , β f , γ f are real, and r f is a complex number. We have five real parameters for each sector.

Realistic models without hierarchical parameters with Γ 4 and Γ 5
In this subsection, we analyze the model with the modular group of level 4 and 5. We show some typical models for illustration purpose.

Yukawa couplings of 3 representation in Γ 4
We consider the FN-like mechanism based on the superpotential W 1 in (71), i.e., the Yukawa couplings are 3 in Γ 4 . We obtain the mass matrix We assume I ≥ J ≥ K and L ≥ M ≥ N to obtain a FN-like matrix. We also study other possibilities, but the above mass matrix is the best one. The mixing angles are approximated by  The first condition implies 9|q 1/4 | ∼ θ C ∼ 10 −1 , which implies Im τ ∼ 2.8. The remaining conditions are rewritten as and the realistic mixing angles are realized naturally with |r u − 0.6| ∼ 10 −1 . Then the mass ratios are approximated as To confirm the above analysis, we construct an explicit example. The representations and the modular weights of the quark fields are summarized in Table 4. The mass matrix is given by (75) with I = 2, J = 1, K = 0, and L = 2, M = N = 1. The best fit parameter in our analysis is given by The largest hierarchy comes from β u /β d = 12.9. We set Re τ = 0 and |r f | = 1 again. We obtain the following hierarchical mass matrices, The mass eigenvalues and mixing angles are given as The parameter which is the most apart from the observed value is θ 13 , and (θ 13 − θ obs 13 )/σ 13 ∼ 2.3σ. We obtain χ 2 ∼ 12, and almost all the parameters are within 2σ range.
Thus all parameters are included within 0.1σ interval, and χ 2 < 0.01.

Yukawa couplings of3 in Γ 4
The q-expansions of the modular forms of3 in Γ 4 are different from the modular forms of 3. If the superpotential is given by W 4 in (71), the approximated mass matrix is given by M 4 in (70). This matrix is quite interesting because the mixing angles are approximated by These relations are nothing but approximate mixing angles predicted in the FN mechanism. Thus this model seems to be the most promising candidate. We consider the model with the following mass matrices, The precise q-expansion of the modular forms are summarized in Appendix B. The mixing angles are estimated as 1,I + r d Y 3,I + r u Y 1,I + r u Y The first relation implies 10|q 1/4 | ∼ θ C ∼ 10 −1 , which implies Im τ ∼ 2.9. Substituting this relation, we obtain Thus we obtain approximate relation of the mixing angles. The mass ratios are approxi- mated as  Table 5. This model generates the mass matrix with I = 2, J = 1, K = 0, L = 2, M = N = 1. In this model, we can not find a parameter set whose χ 2 < 50 with |r f | = 1. We relax this constraint. The best fit parameters in our analysis are given by α u /γ u =1.516, β u /γ u = 2.908, r u = 0.4535e 2.051i , The largest hierarchy is β u /β d = 8.6 among them. The mass matrix is given by The mixing angles and the mass ratios are calculated as y u /y t = 5.70 × 10 −6 , y c /y t = 2.68 × 10 −3 , y d /y b = 6.92. × 10 −4 , y s /y b = 1.37 × 10 −2 , y b /y t = 1.30 × 10 −2 , θ 12 = 0.227, θ 23 = 0.359, θ 13 = 0.00314, δ CP = 1.21, and we can realize all the parameters within 0.1σ range. χ 2 < 0.01.

Yukawa couplings of 3 in Γ 5
We investigate the model based on W 2 in (74). The explicit form of the q-expansion of the modular forms are summarized in Appendix C. We also assume the lightest up-type quark (up-quark) corresponds to the modular form of weight 4, and the second lightest up-type quark (charm quark) corresponds to the modular form of weight 2. Thus the mass matrix is given by The approximate mixing angles are given by Thus 2|q 1/5 | ∼ θ C ∼ 10 −1 , which implies Im τ ∼ 2.4. We obtain In this case we can realize the observed values if | 1 ru − 1 r d | ∼ 10 −1 . On the other hand, conditions for the mass eigenvalues are given by Table 6: The representations and the modular weights of the quarks. This is an explicit model for superpotential with Yukawa coupling of 3 in A 5 .
In this case χ 2 ∼ 0.3 and realistic observed values are reproduced.

Stability of parameters
In the previous section we find realistic models which reproduce the observed quark mass ratios and mixing angles with O(1) parameters. Our approximate estimations show that the mixing angles depend only on Im τ and r u,d . We expect that they are rather stable prediction in our model, if the overall coefficients are O(1). We study the validity of   the approximations used in our models and the stability of the results against changes of the free parameters. For these purposes, we investigate the coefficient dependence of the results. In the followings we use the previous model of Table 6 in Γ 5 as an example.
To study the stability, we see the distributions of the results, where we randomly generate four free parameters of (α u /γ u , β u /γ u , α d /γ d , β d /γ d ). Each parameter is generated as where p follows the uniform distribution with minimum −1 and maximum 1. x best is the best fit value of the coefficients (91). Therefore each of free coefficients fluctuates within the range of x best √ 10 < x < √ 10x best to keep the same order of magnitude. Distributions of the mixing angles and the CP-phase are shown in Figure 1. From the figure we find that the realistic values of the mixing angles are realized without fine-tuning of the free coefficients. Especially θ 12 as well as θ 23 are localized around the observed values, where approximately half of the configurations reproduces the observed value of θ 12 within 2σ range, and all the configurations reproduce the observed value of θ 23 . While some configurations are out of 2σ range for θ 13 , its order is realized without any fine-tuning as we expected. δ CP is also naturally realized, and more than half of the configurations can reproduce the observed value.   Figure 2 shows the distributions of the mass ratios. Figures 2a and 2b show that y u /y t and y c /y t are uniformly distributed as we expected. On the other hand, the mass ratios of the down sectors in Figures 2c and 2d do not follow the uniform distribution. It is consistent with the fact that M d is less hierarchical than M u and the largest components of the mass matrix is exchangeable if the coefficients are significantly apart from the best fit point (See also Figure 3a for singular behavior). However, the mass ratios are localized around the observed values even for M d . Thus the physical parameters are stable under perturbations of the coefficients.
The parameter dependence of the mixing angles and the mass eigenvalues are shown in Figure 3. The mass ratios in up sector and down sector depend on |r u | and |r d |, respectively. We also find that θ 12 drastically changes, although it is independent of |r u,d | in our approximation (89). This is because large |r d | can exchange the largest component of M d as well and our approximation is no longer valid in such regions. In fact, Figure  3b shows that θ 12 have a peak around |r d | ∼ 3, and Figure 3a shows that y s and y d get closer at the same point, where our approximation becomes invalid and the mixing angles are unstable. Apart from such region, the order of all the mixing angles are correctly reproduced. We also show the α u,d dependence in comparison. In contrast to r u,d , the mixing angles are almost independent of α u and α d , which is consistent with the approximation in (89).
Similar results can be obtained in the models with Γ 3 and Γ 4 . Therefore, in any cases of our FN-like models the approximate estimation is valid and useful to analyze the relation between the mass ratios and the mixing angles, especially for the up-type mass matrix due to its large hierarchy. On the other hand, the down-type mass matrix is less hierarchical and the mixing angles may fluctuate depending on the coefficients in the down sectors. Nevertheless all of our models can reproduce the correct orders of the mixing angles and the mass ratios under perturbations of the O(1) coefficients.      Mixing angles and CP-phase 12 23 13 CP (f ) α d dependence of mixing angles.  Mixing angles and CP-phase 12 23 13 CP (h) α u dependence of mixing angles. Figure 3: Parameter dependence of the mass ratios, mixing angles and CP-phase. We show |r u,d | dependence since the mixing angles are explicitly depend on them at the leading order. We also show α d and α u dependence of the parameters for comparison purpose. The observables (except for the corresponding mass eigenvalues) are almost independent of α u and α d .

Conclusion
We have studied the mass hierarchy and the mixing angles in quark sector based on the FN-like mechanism in the framework of the modular symmetry. We have assumed that each of singlet quarks have a common representation under the modular symmetry but with a different modular weight, which is important to obtain the CP-phase as well as a full rank mass matrix. We have introduced a scalar φ with a negative modular weight. The allowed Yukawa couplings are then suppressed by powers of the scalar vev ( φ /Λ) due to the FN-like mechanism, and the mass hierarchy is originated from powers of φ /Λ and the hierarchical modular forms. Using q-expansions of the modular forms we have analyzed an approximate expression of the Yukawa matrices, where the same order of suppression factors appears in each column or each row of the Yukawa matrices, so that our models can simply realize the FN-like matrices for the up sector and down sector simultaneously. We have illustrated this mechanism in models with different finite modular groups of Γ 3 , Γ 4 and Γ 5 in detail. As the result, all of our models can reproduce the correct orders of observed mass ratios and mixing angles by choosing the modular parameters and the singlet vev. The best model is constructed in the model with Γ 4 , where the Yukawa couplings and the left-handed quarks Q have to be the same representation to obtain small mixing angles. In this model, we have approximately reproduced all the 9 observables by tuning 8 parameters, for which we do not require any unnatural hierarchical coefficients. A statistical investigation has also been carried out to study the stability of our results. We have shown that the approximate estimation is valid and our results are stable against perturbations of the O(1) coefficients, especially for the up sector due to its large hierarchical structure. Throughout this paper, we have not studied the lepton sector, while it is interesting to see if the lepton mass spectra and the neutrino mixing angles can be realized based on this models. We assume that φ is stabilized and is safely decoupled. To stabilize the weighton, we require superpotetial in terms of φ. The superpotential of φ is modular invariant as well. Thus it is also restricted by the weight and representations. We should investigate its vacuum structure to see if φ can develop our desired value of vev. The potential of φ is also interesting from the phenomenological point of view. φ may be related to beyond the SM physics, such as SUSY breaking, soft-terms, and µ-terms. The deviation between Majonara mass scale and GUT scale may be originated from the VEV of φ, too. We assume φ is the trivial singlet of the finite modular group. φ can be non-trivial singlet such as1 in S 4 , and such φ possibly change the phenomenology. In this case φ 4 is the trivial singlet, and it looks like a flavon of the usual Z 4 flavor symmetric models. Such a flavon may be important for physics around the standard model [66]. The stringy origin of our model is also unclear and it should be investigated. However, these topics are beyond our scope in this paper, and we will study them elsewhere. In this appendix, we briefly review the modular forms of level 3 and develop our notation. Complete explanation is not our purpose. We would provide a minimal toolkit necessary for our analysis. Γ 3 is isomorphic to T . T has seven irreducible representations, T is generated by S, T and R. Thus it is sufficient to study the matrix representations of these two elements. The matrix representations of T have ambiguities. In this paper, we follow the notation of [27]. The matrix representations are summarized in Table 7. Table 7: The matrix representations of S and T . ω = e 2πi/3 . Our notation is base on [27].
The irreducible decomposition of the tensor product of the singlets are trivial: We also study the irreducible decomposition of the tensor product including 3, i.e., Clebsh-Gordon coefficients. It is given by (96)

Modular forms
The modular forms of level 3 and weight k are given by tensor products of the modular forms of level 3 and weight 1 (19). In this appendix, we consider modular forms which are 3 of T . The complete set of the modular forms whose weights are lower than 6 can be found in [27]. The modular forms of weight 2 are given by the tensor product, We can obtain the modular forms of higher weights as These expansions are consistent with (27). Table 8: The matrix representations of the generators of Γ 4 . Our notation is base on [29].

B The modular forms of level 4
Γ 4 is isomorphic to S 4 SL(2, Z 4 ), which is a double covering group of S 4 . S 4 has the following irreducible representations, 1,1, 1 ,1 , 2,2, 3,3, 3 ,3 . (100) Our notation follows [29]. The matrix representations are summarized in Table 8. The complete table of the irreducible decomposition of the tensor products, and their CG coefficients also be found in [29]. We summarize necessary part here.1 corresponds to e iπ/2 .1 2 = 1 , and (1 ) 2 = 1. Thus the irreducible decomposition of the tensor product of the singlets are trivial: We also require the irreducible decomposition of the tensor products including the triplets, and their Clebsh-Gordon coefficients. They are classified to two cases. For the first case, the irreducible decomposition are given by and the CG-coefficients for these five cases are given as follows For the second case, the irreducible decomposition of the tensor products are given by and the CG-coefficients are summarized by
our Yukawa couplings are triplets. 11 The modular forms of weight 2 are given by Y The complete set of the modular forms whose weights are lower than 8 can be found in [29].
The modular forms of weight 3 are given by The modular forms of weight 4 are given by Y (4) The modular forms of weight 5 are given by

Modular forms
The modular forms of level 5 and weight k are given by tensor products of the modular forms of level 5 and weight 1. The modular forms of weight 1 form6 of A 5 , and they are given by where k r 1 ,r 2 (τ ) is the Klein form defined by k r 1 ,r 2 (τ ) = q (r 1 −1)/2 with (r 1 , r 2 ) being a pair of rational number. z ≡ τ r 1 + r 2 and q z ≡ e 2πiz . The modular forms of higher weights are constructed by its products.
triplet modular forms always be positive and even for Γ 5 . The complete set of the modular forms whose weights are lower than 6 can be found in [31]. The triplet modular forms of