Modular symmetry at level 6 and a new route towards finite modular groups

We propose to construct the finite modular groups from the quotient of two principal congruence subgroups as $\Gamma(N')/\Gamma(N")$, and the modular group $SL(2,\mathbb{Z})$ is extended to a principal congruence subgroup $\Gamma(N')$. The original modular invariant theory is reproduced when $N'=1$. We perform a comprehensive study of $\Gamma'_6$ modular symmetry corresponding to $N'=1$ and $N"=6$, five types of models for lepton masses and mixing with $\Gamma'_6$ modular symmetry are discussed and some example models are studied numerically. The case of $N'=2$ and $N"=6$ is considered, the finite modular group is $\Gamma(2)/\Gamma(6)\cong T'$, and a benchmark model is constructed.


Introduction
The neutrino masses and lepton mixing parameters have been precisely measured by a large number of neutrino oscillation experiments. If neutrinos are Majorana particles, the lepton mixing are described by three mixing angles, one Dirac CP violation phase δ CP and two Majorana CP violation phases α 21 and α 31 . The 3σ ranges of lepton mixing angles and neutrino masses for normal ordering (NO) spectrum are determined to be [1] 0.02032 ≤ sin 2 θ 13 ≤ 0.02410, 0.269 ≤ sin 2 θ 12 ≤ 0.343, 0.415 ≤ sin 2 θ 23 ≤ 0.616, 6.82 eV 2 ≤ ∆m 2 21 × 10 5 ≤ 8.04 eV 2 , 2.435 eV 2 ≤ ∆m 2 31 × 10 3 ≤ 2.598 eV 2 , (1.1) where θ 12 , θ 13 and θ 23 are three lepton mixing angles in the standard parametrization [2], and ∆m 2 21 ≡ m 2 2 − m 2 1 and ∆m 2 31 ≡ m 2 3 − m 2 1 are two mass squared differences. Similar results for inverted ordering (IO) spectrum have been obtained [1]. The Dirac phase δ CP is less constrained by the current data, and the forthcoming long-baseline experiments are expected to make precise measurement of δ CP . The Majorana phases of the neutrino mixing matrix can not have any effect in neutrino oscillations, and almost nothing is known about their possible values at present.
Understanding the origin of fermion masses and mixing patterns is a greatest challenge in particle physics. It was found that the non-abelian finite discrete flavor symmetry is particularly suitable to explain the large lepton mixing angles [3][4][5][6][7][8][9]. Certain mixing patterns can be obtained if the discrete flavor group is broken down to different remnant subgroups in the charged lepton and neutrino sectors. In the paradigm of traditional discrete flavor symmetry, the flavor group is spontaneously broken by the vacuum expectation values of a set of flavons. In this approach, additional dynamics of vacuum alignment, field content and auxiliary symmetry are necessary to achieve the desired non-trivial vacuum configuration so that the resulting models are rather complicated and the scalar potential has to be cleverly designed.
However, the origin of finite flavor symmetry is still unknown. Recently modular symmetry which naturally appears in string theory has been suggested as the origin of such a finite flavor symmetry [10]. The homogeneous (inhomogeneous) finite modular groups Γ N (Γ N ) appearing as the quotient group of the two dimensional (projective) special linear group over the principal congruence subgroups is considered in the framework of the modular invariance approach. In this modular approach, the flavon fields other than the modulus τ are unnecessary and the Yukawa couplings are modular forms which are holomorphic functions of the complex modulus τ . The flavor symmetry will be broken when the modulus τ gets vacuum expectation value (VEV). Moreover, the modular invariance and supersymmetry completely determine all higher dimensional operators in the superpotential. As the lepton mixing parameters and masses are only fixed by a small number of free parameters in modular invariant lepton models, the resulting models have strong predictive power.
Recently the modular weight k has been extended to a rational number, and the corresponding finite modular group will be extended to its metaplectic covers [63,64]. It is remarkable that the modular symmetry also has the merit of conventional abelian flavor symmetry group, the structure of the modular form can produce texture zeros of fermion mass matrices exactly [30,59], the modular weights can play the role of Froggatt-Nielsen charges [34,49], and the hierarchical charged lepton masses can arise solely due to the proximity of the modulus τ to a residual symmetry conserved point [15,41,65]. Furthermore, the modular invariance approach has been extended to the quark sector to explain the quark mixing and quark masses. Then a unified description of leptons and quarks can be achieved from a common finite discrete modular group [22,36,40,59,61,63]. In order to further improve the predictive power of the modular invariance approach, the setup of combining modular symmetry with the generalized CP (gCP) symmetry has been discussed [66,67]. The consistency of modular and gCP symmetries requires that the complex modulus τ transforms as τ CP −→ −τ * up to modular transformations [66][67][68][69][70][71]. In this context, the VEV of τ is the unique source of both modular and gCP symmetries. In the basis where the representation matrices of both modular generators S and T are symmetric, the gCP transformation would reduce to canonical CP and all the coupling constants would be real. Moreover, the possibility of coexistence of multiple moduli has been investigated [46,72].
In the present work, we generalized the modular invariance approach [10] by replacing the modular group SL(2, Z) with the principal congruence subgroup Γ(N ), the original modular invariant theory is the special case of N = 1. The finite modular group is the quotient of two principal congruence subgroups Γ(N )/Γ(N ), and the known homogeneous finite modular groups can be reproduced. As examples, we study two cases of Γ(N ) = Γ(1) and Γ(N ) = Γ(2) with Γ(N ) = Γ (6) to construct models of lepton masses and mixing. The modular symmetry is extended to combine with the gCP symmetry, and invariance under gCP requires all coupling constants to be real in our working basis. We find that the modular forms of weight k = 1 and level N = 6 can be obtained from the modular forms of Γ(3) with weight 1 and they can be expressed in terms of the products of Dedekind eta functions. A systematical analysis of the modular invariant lepton models with Γ 6 modular symmetry is performed. The finite modular group Γ 6 has abundant two-dimensional representations which provides new model building possibilities. The left-handed lepton fields, the righthanded charged lepton and the right-handed neutrinos can be assigned to a triplet, doublet plus singlet or three singlets of Γ 6 . We are concerned with the phenomenologically viable models with small number of free parameters. We have discussed five types of lepton models classified according to the representation assignments of the lepton fields under Γ 6 , and some examples models are presented. The neutrino masses are generated by the effective Weinberg operator in type IV model and by the seesaw mechanism in other types of models. Three right-handed neutrinos are introduced in type I, II, III models while there are two righthanded neutrinos in type V models so that the lightest neutrino is massless. Furthermore, we consider the case of Γ(N ) = Γ(2) and Γ(N ) = Γ(6), the finite modular group is Γ(2)/Γ(6) = Γ 3 ∼ = T . The modular forms of level 6 can be arranged into T multiplets, and a benchmark model is constructed.
The outline of this paper is as follows. The concept of modular symmetry is recapitulated and a new route towards finite modular groups is given in section 2. In section 3, we give the formalism of the generalized modular invariant supersymmetry theory and modular forms of level 6 are presented. In section 4, lepton models with Γ 6 modular symmetry are studied, and we have discussed five types of models differing in the representation assignments of the lepton fields under Γ 6 . As an example, a modular invariant model based on Γ(2) modular symmetry with finite modular group T is given in section 5. We conclude the paper in section 6. The group theory of the Γ 6 modular group is presented and the Clebsch-Gordan (CG) coefficients in our working basis are reported in appendix A. The linearly independent higher integral weight modular multiplets of level 6 with finite modular group Γ 6 are listed in appendix B. We present the decomposition of level 6 modular forms with respect to the T modular symmetry in appendix C,.

A new route towards finite modular groups
The finite modular groups and their cover groups have been widely studied as the flavor symmetry groups. They usually arise from the quotient of the group SL(2, Z) or P SL(2, Z) modulo its normal subgroups Γ(N ), where SL(2, Z) ≡ Γ called the full modular group is defined as The group SL(2, Z) has infinite elements and it can be generated by two generators S and T with which fulfill the relations The infinite normal subgroups Γ(N ) which are called principal congruence subgroup of level N are defined as Note that Γ(1) = SL(2, Z) and T N ∈ Γ(N ). The homogeneous finite modular group Γ N is the quotient group Γ/Γ(N ) 1 : Note that additional relations are necessary to render the group Γ N finite for N ≥ 6 [73]. In the present work, we are concerned with the modular group Γ 6 which satisfies the defining relations The above construction of finite modular group is not unique. The quotient of two congruence subgroups of SL(2, Z) can also give finite groups. Let N = p n 1 1 . . . p n k k be the prime factorization of the positive integer N , where n 1 , . . . , n k ∈ N and p 1 , . . . , p k are distinct primes. The prime factorization of all positive integers less than 21 is listed in table 1. It is well known that Chinese remainder theorem gives the isomorphism of rings [74]: Then this gives isomorphism [75]: SL(2, Z/N Z) ∼ = SL(2, Z/p n 1 1 Z) × · · · × SL(2, Z/p n k k Z) . (2.8) Using isomorphism SL(2, Z/N Z) ∼ = Γ(1)/Γ(N ), we have N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1 2 3 2 2 5 2 · 3 7 2 3 3 2 2 · 5 11 2 2 · 3 13 2 · 7 3 · 5 2 4 17 2 · 3 2 19 2 2 · 5 (2.10) As concrete examples, we have 2 This implies that quotient of the principal congruence subgroups can also give rise to the homogeneous finite modular groups Γ N . If one considers the more general congruent subgroups of SL(2, Z), other finite modular groups can be constructed by the quotient procedure. Besides the principal congruence subgroups, there are two another important congruence subgroups [76] where * means any positive integer small than N . Obviously we have relation Γ(N ) ⊂ Γ 1 (N ) ⊂ Γ 0 (N ) ⊂ Γ. In fact, it is known that Γ(N ) and Γ 1 (N ) are the normal sungroups of Γ 1 (N ) and Γ 0 (N ) respectively, and [76] 14) where Z N are cyclic groups of order N , also known as additive group of integers modulo N . And Z * N is known as the multiplicative group of integers modulo N , and its order is equal to φ(N ) ≡ N p|N 1 − 1 p which is Euler's totient function. Z * N is a finite abelian group and it reduces to a cyclic group for prime N . Since both Z N and Z * N are abelian groups and they only have one-dimensional irreducible representations, we don't consider the cases of Z N and Z * N as flavor symmetry in the following. We denote the two principal congruence subgroups as Γ(N ) and Γ(N ) in the following, N is divisible by N so that Γ(N ) is a normal subgroup of Γ(N ). Γ(N ) acts on the upper half plane H = {τ ∈ C| τ > 0} by the linear fractional transformation As a consequence, the fundamental domain becomes D = H/Γ(N ) in which no two points are related by the modular transformations of Γ(N ). The modular forms of the modular subgroup Γ(N ) with weight k are holomorphic functions f (τ ) satisfying the property The modular forms f i (τ ) span a finite dimensional linear space, and they are usual modular forms of level N . It is possible to choose a basis such that the transformations of modular forms f i (τ ) under Γ(N ) can be described by [10,58] where ρ(γ) is the irreducible representation of quotient group Γ(N )/Γ(N ).
3 Modular symmetry and modular forms of level 6 We generalize the modular invariant supersymmetry theory [10] by replacing Γ ∼ = SL(2, Z) with the principal congruence subgroup Γ(N ), the original modular invariant theory is the special case of N = 1. Notice that this generalization is already contained in the more general automorphic supersymmetric framework [72]. The field content consists of a set of chiral matter superfields Φ I and a modulus superfield τ , their modular transformation under Γ(N ) are given by: where k I is called the modular weight of matter field Φ I , and ρ I (γ) is the unitary representation of quotient group Γ(N )/Γ(N ). The Kähler potential is still taken to be the minimal [10] K(Φ I ,Φ I ; τ,τ ) = −hΛ 2 log(−iτ + iτ ) + which leads to the kinetic terms for matter fields and the modulus field τ after the modular symmetry breaking induced by non-vanishing VEV of τ . The superpotential W(Φ I , τ ) can be expanded in power series of the involved supermultiplets Φ I , where Y I 1 ...In (τ ) is a modular multiplet of weight k Y and level N as introduced in the previous section, and it fulfills Modular invariance entails that each term of the W(Φ I , τ ) should satisfy the following conditions: In this work, we will study the normal subgroup Γ(N ) = Γ(6) as an example, and consider two cases of Γ(N ) = Γ(1) and Γ(N ) = Γ(2) as the full modular symmetries to construct the corresponding lepton models. Accordingly the finite modular groups are Γ(1)/Γ(6) = Γ 6 and Γ(2)/Γ(6) = Γ 3 ∼ = T respectively.

Modular forms of level N = 6
The modular forms of weight k and level N form a linear space M k (Γ(N )), and the general dimension formula of M k (Γ(N )) is [76,77] dimM k (Γ(N )) = (k − 1)N + 6 24 N 2 for 12 > N > 2, k ≥ 1 or N > 2, k ≥ 2, where the product is over the prime divisors p of N . For the concerned level N = 6, the above dimensional formula is Therefore there are six linear independent modular forms of weight k = 1 at level 6.

Generalized CP consistent with Γ 6 modular symmetry
In order to make modular invariant models more predictive, we shall assume the theory is invariant under both modular symmetry and gCP symmetry. It is well known that the modular symmetry SL(2, Z) can consistently combine the gCP symmetry [66][67][68][69][70][71], and the modulus τ transforms under CP as τ If the gCP symmetry is imposed, the modular symmetry The action of Γ * on the complex modulus τ is given by Under the action of gCP transformation, the matter fields and the modular form multiplets transform as where X r is the gCP transformation matrix in the irreducible representation r. The consistency between the modular symmetry and gCP symmetry requires the following condition has to be fulfilled [66,67] where χ(γ) called character is a homomorphism of SL(2, Z) to {+1, −1} with χ(S) = χ(T ) = 1 or χ(S) = χ(T ) = −1, and u(γ) is an outer automorphism of the modular group, The gCP transformation with χ(S) = χ(T ) = −1 will leads to vanishing mixing angles [54,60], consequently we consider the case with trivial character χ(S) = χ(T ) = 1. Then the consistency condition in Eq. (3.24) for the generators γ = S, T takes the following form, The representation matrices of generators S and T are unitary and symmetric in our working basis, as shown in table 5. Therefore the gCP transformation matrix X r is fixed to be unity matrix up to an overall phase, i.e.
Moreover, the CG coefficients given in appendix A are real, hence gCP invariance requires all coupling constants to be real in the following models.

Decomposing modular forms of level
Here T is the double covering of the tetrahedron group A 4 . The representation matrices ofã andb in different irreducible representations of T are given in table 6. From the general properties of modular forms [58], we know that the modular forms of Γ(6) can be decomposed into the irreducible multiplets of the finite group Γ 1 ii denote the two weight 1 modular forms transforming in the represen- 2 (τ ) arise from the different representation bases used in the present work and [58]. As shown in section 3.2, all couplings would be real if we impose gCP as a symmetry on the theory.

Decomposing modular forms of level
Similar to previous case, Γ(6) is a normal subgroup of Γ(3), and we have wherec = S 3 T 3 S,d = T 3 are the generators of Γ(3). We summarize the irreducible representation matrices ofc andd in table 6. Analogously the six weight 1 modular forms Y i (τ ) can be arranged into two singlets and two doublets of S 3 as follows  Similarly the decomposition of higher weight modular forms under S 3 can also be performed, we will not enumerate them here. It is interesting to note that the modular weight can be generic nonnegative integer while only even weight modular forms are allowed in the original modular invariant theory with Γ 2 ∼ = S 3 modular symmetry [10,58] 4 Model building based on Γ 6 modular symmetry In this section we shall construct some typical lepton models based on the Γ 6 modular symmetry, and no flavon other than complex modulus τ will be introduced. The modular symmetry is broken by the VEV of τ . If gCP symmetry is imposed on the theory, the VEV τ also results in CP violation. Without loss of generality, we can limit the value of τ in the fundamental domain D Γ of SL(2, Z): Other values of τ are related by modular symmetry to those in D Γ . In the present work, the construction of modular models is guided by the principle of minimality and simplicity. The neutrinos are assumed to be Majorana particles in these models, and the two Higgs doublets fields H u,d are assumed to be trivial singlet 1 0 0 under Γ 6 with vanishing modular weights. The modular group Γ 6 is isomorphic to S 3 ×T , the direct products of the irreducible representations of S 3 and T give all irreducible representations of Γ 6 . Therefore all models based on S 3 and T modular symmetry can be reproduced from Γ 6 . Besides the singlet, doublet and triplet representations, Γ 6 has the four-dimensional representations 4 0 , 4 1 , 4 2 and six-dimensional representations 6 which don't appear in both S 3 and T modular symmetries, and there are level 6 modular forms in the above high dimensional representations. Hence the modular group Γ 6 opens up new model building possibilities not available in these two finite modular groups. The lepton fields are assigned to be three singlets, singlet plus doublet or a triplet of Γ 6 multiplets. We consider five possible representation assignments in the present work, as summarized in table 2. More free parameters are needed to explain the experimental data for other assignments, consequently we don't show them in this work. In the following, we present the five types of models with Γ 6 modular symmetry together with come typical examples.

Type I models
In the first type of models, the three generations of left-handed (LH) lepton doublets L are assigned to a Γ 6 triplet 3 r with the modular weight k L , and the three right-handed (RH) charged leptons E c 1,2,3 are Γ 6 singlets transforming as 1 s k with weights k E c 1,2,3 . In these modes, the neutrino masses are described by the type I seesaw mechanism with 3RHN.

Charged lepton sector
In this kind of models, the modular form which couples to E c i and L should be in the representation 3 t of Γ 6 with t ≡ −(r + s) (mod 2). Thus the most general superpotential for the charged lepton masses can be written as : where f i (Y ) (i = 1, 2, 3) stand for the level 6 modular forms which transform as 3 t given in table 7. The three terms in Eq. (4.2) have similar form, and they contribute to three rows of the charged lepton mass matrix respectively. Guided by the principle of minimality and simplicity, we shall consider the lower weight modular forms in the following. As an example, let us discuss all possible forms of the first row of the charged lepton mass matrix. Without loss of generality, we assume L ∼ 3 0 and E c Then we can read out the first row of the charged lepton mass matrix as follows where w = 2 for k L + k E c 1 = 2 and w = 4 for k L + k E c 1 = 4, and the permutation matrix 3 0 , and this convention will be used throughout the paper. For the assignment E c 1 ∼ 1 1 i , the possible values of the modular weights k L + k E c 1 are 4 and 6, and accordingly the modular function 3 1 , respectively. For these two cases, the first row of the charged lepton mass matrix reads as Therefore one can easily check the following identity is satisfied The factor Y can be absorbed into the coupling α, thus R w,i and R w,i give the same set of possible forms of the first row of the charged lepton mass matrix. Hence it is sufficient to consider the assignments of L ∼ 3 0 and E c 1 ∼ 1 0 i with k L + k E c 1 = 2 and k L + k E c 1 = 4, and the first row of the charged lepton mass matrix can take the 6 forms shown in Eq. (4.3). Analogously the other two rows of the charged lepton mass matrix can also take these six forms. Notice that any two rows of a charged lepton mass matrix should not be proportional to each other, otherwise at least one charged lepton would be massless. In the end, we find there are total 20 possible charged lepton mass matrices (up to row permutations). They can be written as Table 3: The 20 possible charged lepton mass matrices and the corresponding representations and modular weights of charged leptons for the type I models, where k ci ≡ k L + k E c i denote the sum of the modular weight of the charged lepton fields.

Neutrino sector
In this kind of models, the neutrino masses are generated through the type I seesaw mechanism. The three right-handed neutrinos are embedded into a triplet 3 r of Γ 6 . Hence the most general superpotential for the neutrino masses can be generally written as where Y D and Y N denote the modular form multiplets of Γ 6 . In the case of k N c = 0 and k N c = 1, the corresponding modular multiplets Y N are constants and Y We proceed to discuss the possible form of the Dirac neutrino mass matrix. If the left-handed lepton L and right-handed neutrinos N c transform as L ∼ 3 0 , N c ∼ 3 0 or L ∼ 3 1 , N c ∼ 3 1 , modular invariance requires Y D should transform as 1 0 0 , 1 0 1 , 1 0 2 or 3 0 under Γ 6 . The lowest non-vanishing weight is k L + k N c = 2, and the Yukawa coupling is given by Then the Dirac neutrino mass matrix takes the following form where the phase of h 1 can be absorbed by field redefinition, while the relative phase of h 1 and h 2 can not be eliminated. For the second possible assignment L ∼ 3 0 , N c ∼ 3 1 or L ∼ 3 1 , N c ∼ 3 0 , the resulting Dirac neutrino mass matrix can be obtained from Eq. (4.12) by permutation of columns. After taking into account the charged lepton sector, no new textures of lepton mass matrices are obtained. Notice that the group Γ 6 is isomorphic to S 3 × T , where T is the double cover of A 4 , and the representations 1 0 k (k = 0, 1, 2) and 3 0 are the direct products of the A 4 representations with the trivial unit representation of S 3 . As a result, the Γ 6 modular symmetry can not be distinguished from A 4 in the representations 1 0 k and 3 0 . As we have shown, in this type of models it is sufficient to consider the assignments of L ∼ 3 0 , N c ∼ 3 0 and E c 1,2,3 ∼ 1 0 k . Hence the type I models coincide with the A 4 modular models which have been systematically analyzed in [40]. We have checked that the numerical results for mixing parameters are the same as those of [40].

Type II models
In this type of models, the charged lepton sector is the same as that of type I models discussed in section 4.1.1. The light neutrino masses arise from the type I seesaw mechanism with 3RHN, we assume that one right-handed neutrino N c 1 transforms as singlet of Γ 6 and the other two N c d = (N c 2 , N c 3 ) form a doublet of Γ 6 . As an example, we consider the transformation assignment N c 1 ∼ 1 0 0 , N c d ∼ 2 0 with the modular weights k N c 1 = k N c d = 0, the modular weight of the left-handed lepton doublets L is taken to be k L = 2. Then the most general superpotential for neutrino masses is Then one can read out the Dirac and Majorana mass matrices as follows (4.14) Then the light neutrino mass matrix m ν only depend on the combinations h 2 1 /g 1 and h 2 2 /g 2 besides the complex modulus τ . The phase of h 2 1 /g 1 is unphysical and consequently h 2 1 /g 1 can be taken real, while the phase of h 2 2 /g 2 can not be removed by a field redefinition. If gCP invariance is imposed, both h 2 1 /g 1 and h 2 2 /g 2 are required to be real, i.e. their phases are 0 or π.

Numerical results
The charged lepton mass matrices can take the twenty forms C i (i = 1, · · · , 20) given in table 3, and they all have three real coupling constants α, β and γ. Thus all the six lepton masses, three lepton mixing angles and three CP violation phases depend on six dimensionless parameters τ, arg (g 1 h 2 2 /(g 2 h 2 1 )) , β/α, γ/α, and two overall scales αv d and h 2 1 v 2 u /(g 1 Λ). If gCP symmetry is considered, the parameter arg (g 1 h 2 2 /(g 2 h 2 1 )) can only take two values 0 and π. We shall perform a numerical analysis for each possible model without gCP and with gCP. In order to quantitatively assess whether a model can accommodate the experimental data on lepton masses and mixing parameters, we perform a conventional χ 2 analysis, and the χ 2 function is defined as charged lepton sector without gCP charged lepton sector with gCP where σ i , µ i and P i refer to the 1σ deviations, the global best fit values and the theoretical predictions for the seven dimensionless observable quantities as functions of free parameters: m e /m µ , m µ /m τ , sin 2 θ 12 , sin 2 θ 13 , sin 2 θ 23 , δ CP , ∆m 2 21 /∆m 2 31 .  We find that five of the twenty models are phenomenologically viable for both models without gCP and with gCP. The predictions for lepton mixing parameters, neutrino masses Figure 1: The experimentally allowed values of the complex modulus τ and the correlations between the neutrino mixing parameters in the example model of type II. The green and red regions denote points compatible with experimental data for models without and with gCP respectively. The light blue bounds represent the 1σ and 3σ ranges respectively [1]. The orange (cyan) dashed lines in the last panel represent the most general allowed regions for NO (IO) neutrino masses respectively, where the neutrino oscillation parameter are varied within their 3σ ranges. The present upper limit m ββ < (61−165) meV from KamLAND-Zen [79] is shown by horizontal grey band. The vertical grey exclusion band denotes the current bound from the cosmological data of i m i < 0.120eV at 95% confidence level given by the Planck collaboration [81]. and the best fit values of the input parameters for the five phenomenologically viable models are summarized in table 4. Here the parameter m ββ refers to the effective Majorana mass in neutrinoless double beta decay and m i is the sum of the light neutrino masses. At present, the most stringent bounds are m ββ < (61 − 165) meV from the KamLAND-Zen experiment [79] and m i < 0.12 eV from the Planck Collaboration [80]. We see that the minimal values of the χ 2 are quite small, the experimentally measured lepton masses and mixing angles can be accommodated very well, and the values of m ββ and i m i are much below the present experimental bounds [79,80].
In order to show the viability and predictions of this type of models, we shall give detailed numerical results of the model in which the charged lepton sector is C 6 . We comprehensively scan the parameter space and require all the observables lie in their experimentally preferred 3σ regions, some interesting correlations among the input parameters and observables are obtained, as shown in figure 1. Here the results without gCP and with gCP are displayed in green and red respectively. From figure 1, we find that the atmospheric mixing θ 23 and Dirac CP phase δ CP can take all possible values in their 3σ regions if gCP is not considered. However, all couplings are real and τ is the unique source of CP violation if gCP invariance is required in the model. As a consequence, the three CP violation phases are constrained to lie in narrow ranges in figure 1.

Type III models
In this type of models, the left-handed lepton fields L are assigned to a triplet of Γ 6 , and the right-handed charged leptons are the direct sum of singlet and doublet. Without loss of generality, we assume that E c 1 with modular weight k E c 1 transforms as singlet 1 r k under Γ 6 , and the right-handed charged leptons E c d = (E c 2 , E c 3 ) with modular weight k E c d transform as a doublet of Γ 6 . The most general superpotential for the charged lepton masses is of the following form: where f 1 (Y ) and f d (Y ) are functions of modular forms of level 6 given in appendix B. The charged lepton mass matrix is denoted as where v i (i = 1, 2, 3) are three rows of charged lepton mass matrix m e . From the analysis in section 4.1.1, we find that the first row v 1 can take the six possible forms in Eq. (4.3). Now let us consider the possible forms of the second row v 2 and the third row v 3 . For the representation assignment L ∼ 3 r and E c d ∼ 2 i , modular invariance requires the presence of modular form in the representation 6. From table 7, we know that the lowest weight modular multiplet transforming as 6 under Γ 6 is of weight 2, and accordingly the charged lepton mass term is which leads to the second and third rows of the charged lepton mass matrix to be where matrices P 2 and P 3 are shown in Eq. (A.7). As the general form of the three rows of the charged lepton mass matrix have been obtained, then one can obtain the charged lepton mass matrix m e for all possible models. As the hermitian combination m † e m e will take the same form for r = 0 and r = 1. Then the effect of P r 2 in above equation can be absorbed by right-handed charged leptons.
If the lepton fields L and E c d transform as L ∼ 3 s and E c d ∼ 2 r i respectively, from the multiplication rules 2 r i ⊗ 3 s = 2 t 0 ⊕ 2 t 1 ⊕ 2 t 2 and 2 r i ⊗ 2 s j = 1 t m ⊕ 3 t , we find that the doublet modular forms Y . Without loss of generality, we take E c d ∼ 2 0 i and L ∼ 3 0 with k L + k E c d = 1. The second term in Eq. (4.19) can be written as Then one can straightforwardly obtain Guided by the principle of minimality and simplicity, the modular form Y can also take to be Y Then we can take E c d ∼ 2 1 i and L ∼ 3 0 with k L + k E c d = 3, and the charged lepton Yukawa coupling reads as We give an example model in the following. Both the left-handed lepton L and the righthanded neutrinos N c transform as 3 0 under Γ 6 , and the right-handed charged leptons E c 1 and E c d transform as 1 0 0 and 2 2 , respectively. The modular weights take to be Although the predictions for neutrino mixing angels are compatible with experimental data, the light neutrino masses are quasi-degenerate and the effective Majorana mass m ββ is larger than the latest upper bound. We have considered other type II models, and we find the neutrino mass spectrum tends to be degenerate and they are usually disfavored by the data of neutrinoless double beta decay.

Type IV models
In this type of models, both left-handed leptons L and right-handed charged leptons E c are assumed to transform as singlet plus doublet under Γ 6 . Since the Γ 6 modular group has six one-dimensional representations 1 r k and nine two-dimensional representation 2 k , 2 r k with r = 0, 1 and k = 0, 1, 2, there are a lot of possible representation assignment for the lepton fields, and we find many of them can explain the experimental data. We will not shall all these models one by one. In order to see how well this type of models agree with the experimental data on mixing parameters and lepton masses, we will present an example model. We assume that neutrinos masses are described by the Weinberg operator, the transformation properties and modular weights of the lepton fields are given by Then the superpotential for the lepton mass is determined to be Note that the term proportional to g 3 gives null contributions because the contraction 2 1 1 ⊗ 2 1 1 → 1 0 2 is antisymmetric. In comparison with previous types of models, the modular form Y (5) 4 0 in the four-dimensional representation of Γ 6 is involved in this model. We can straightforwardly read off the lepton mass matrices as follow, (4.32) The phases of the couplings α, β, γ and g 1 can be absorbed by lepton fields while g 2 is complex without gCP. This model can agree with the experimental data very well for certain values of the input parameters. We find χ 2 min = 2.871 and the best fit values of the free parameters are determined to be τ = −0.329, τ = 1.080, β/α = 105.467, γ/α = 12.954, |g 2 /g 1 | = 0.816, arg(g 2 /g 1 ) = 0.958π, If gCP is imposed on the model, the coupling g 2 would be real and it can be either positive or negative. We find that the minimum value of χ 2 function is χ 2 min = 2.914 when the input parameters take the following values τ = −0.334, τ = 1.092, β/α = 105.717, γ/α = 12.696, g 2 /g 1 = −0.810, We numerically scan over the parameter space of the model, the lepton masses and mixing parameters are required to lie in the experimentally preferred 3σ regions. The allowed region of the modulus τ and the correlations among different observables are shown in figure 2, where the red and green regions are the results with gCP and without gCP, respectively. We see that the model is very predictive and the neutrino mixing parameters are determined to vary in small regions. After including the gCP symmetry, the predictive power is improved further and the allowed regions shrink drastically.

Type V models
As the finite modular group Γ 6 has nine two-dimensional irreducible representations 2 k and 2 r k with r = 0, 1, k = 0, 1, 2, it is convenient to construct minimal seesaw models involving two right-handed neutrinos in Γ 6 modular symmetry. The two right-handed neutrinos N c = (N c 1 , N c 2 ) T are embedded into a doublet of Γ 6 and its modular weight is k N c . In order to take full advantage of the abundant two-dimensional representations of Γ 6 , the three generations of left-handed lepton doublets are arranged into one singlet and one doublet of Γ 6 . The three right-handed charged leptons are assumed to transform as a triplet under Γ 6 otherwise more modular invariant terms as well as more free parameters would be needed. There are many different possible representation assignments for the matter fields, and it is too lengthy to list them. For illustration, we give one example, the transformation properties and modular weights of the lepton fields are given by which completely fixes the superpotential of the lepton masses as follows, We impose gCP symmetry such that all coupling constants are real. Notice that the neutrino Yukawa coupling between L 1 and N c is forbidden by modular invariance. The neutrino Dirac mass matrix and Majorana mass matrix are given by (4.39) The predictions for the lepton mixing parameters and lepton masses agree rather well with their measured values for certain values of the input parameters. The best fit point is found to be given by Similar to previous cases, a numerical analysis is performed, and the results are plotted in figure 3. It is worth noting that the Dirac CP phase δ CP is limited to a very narrow range. 5 Model building based on Γ(2) modular symmetry with finite modular group T Applying the general formalism outlined in section 3, we consider construction of modular models for N = 2 and N = 6 in this section, the full flavor symmetry is the Γ(2) modular group. It is worth mentioning that the Γ(2) modular symmetry was used to understand the quantum hall effect [82]. In this context, the inequivalent vacuum of the modulus τ is given by the fundamental domain of Γ(2): which is displayed in figure 4. The modular forms of level 6 can be arranged into irreducible multiplets of the finite modular group Γ(2)/Γ(6) ∼ = T , as is summarized in table 8. It can be seen that at a given weight, there are several modular multiplets in the same representation of T . Comparing with the modular forms of level 3 and their decomposition under Γ 3 ∼ = T [58], we see there are more modular multiplets at a given weight. The T group has three one-dimensional representations 1 0 k with k = 0, 1, 2, three two-dimensional representations 2 0 k and a three-dimensional representation 3 0 . The Kronecker products between different representations of T can be straightforwardly obtained from Eq. (A.6) as follows, Moreover, one can easily obtain the CG coefficients of T group from those of Γ 6 in appendix A.
Re(τ ) Im(τ ) In the same fashion as section 4, one can also systematically analyze the possible modular models based on Γ(2) modular symmetry with finite modular group T . It is too lengthy to present all possibilities here, for illustration we shall give an example model in the following. The left-handed lepton doublets L are assigned to a triplet 3 0 of T with the modular weight k L = 1. The three right-handed charged leptons transform as E c 1 under T , and their modular weights are k E c 1 = 1 and k E c d = 0, respectively. We assume the neutrino masses are described by the effective Weinberg operator. From the Kronecker we know that all the six weight 2 modular multiplets Y 3 0 ii and Y (2) 3 0 iii can couple with the operator LLH u H u to fulfill modular invariance. Thus the superpotential for neutrino masses is given by From the CG coefficients given in appendix A, we can easily read off the light neutrino mass matrix. Furthermore, the Yukawa interactions for the charged leptons are of the following form which leads to the charged lepton mass matrix (5.5) All the coupling constants in Eqs. (5.3,5.4) are real due to gCP invariance. This model contains enough free parameters to fit the experimental data, and we find the best fit values of all observables can be reproduced for the following values of the input parameters: τ = −0.0263, τ = 1.789, α 2 /α 1 = 285.994, α 3 /α 1 = 199.982, β 1 /α 1 = 623.202, β 2 /α 1 = 886.017, β 3 /α 1 = 3.243, g 2 /g 1 = 972.843 , g 3 /g 1 = 469.822, g 4 /g 1 = 455.144, g 5 /g 1 = 789.301, g 6 /g 1 = 172.997 . (5.6) The resulting models usually contain a large number of free parameters if the modular flavor symmetry is some principal congruence subgroup Γ(N ) rather than the modular group SL(2, Z) ≡ Γ(1), consequently the predictive power is reduced to certain extent. The topdown construction in string theory generally lead to both modular symmetry and traditional flavor symmetry, the full flavor symmetry named as eclectic flavor group is a nontrivial product of traditional favor group and finite modular group [83][84][85][86][87]. The couplings in modular invariant models are subject to strong constraints from traditional flavor symmetry, and some terms allowed by the modular group could be forbidden by the traditional flavor group. Thus we expect that the modular invariant models based on Γ(N ) and finite modular group can become much more predictive by combining with certain traditional flavor group in the paradigm of eclectic flavor group.

Conclusion
Modular symmetry is a promising framework to understand the hierarchical masses and flavor mixing patterns of quarks and leptons [10]. In the present work, we generalize the modular invariance approach [10] and a new route towards finite modular groups is proposed. The theory is assumed to be invariant under the principal congruence subgroup Γ(N ), the finite modular group is the quotient of two principal congruence subgroups Γ(N )/Γ(N ). The known homogeneous finite modular groups can be reproduced, as shown in Eq. (2.11), and the original modular invariant theory is the special case of N = 1.
For N = 1 and N = 6, the finite modular group is the homogeneous finite modular group Γ(1)/Γ(6) = Γ 6 which is isomorphic to S 3 ×T . The gCP symmetry consistent with the modular symmetry is considered. The representation matrices of both modular generators S and T are symmetric and unitary in all irreducible representations of Γ 6 , consequently the gCP transformation is a unit matrix up to modular transformations and all coupling constants are real if gCP invariance is imposed. We have constructed the weight 1 modular forms of level 6 in terms of the Dedekind eta function, and they can be arranged into a doublet and a quartet of Γ 6 . We also give the expressions of the linearly independent higher weight modular forms which are the tensor products of weight 1 modular forms.
We have performed a comprehensive analysis of models for lepton masses and mixing with Γ 6 modular symmetry, and five types of models summarized in table 2 have been studied. In the type I models, both left-handed leptons L and right-handed neutrinos N c are assumed to be triplets of Γ 6 , the right-handed charged fermions are singlets of Γ 6 . It turns out that the type I models with small number of free parameters coincide with the A 4 modular models discussed in [40]. Different from the type I models, the three right-handed neutrinos are assigned to the direct product of a singlet and a doublet of Γ 6 in the type II models. We find that five of the twenty modular models can accommodate the experimental data no matter whether gCP symmetry is included or not, as shown in table 4. The type III models differ from the type I models in the assignment of right-handed charged leptons which are assumed to be singlet plus doublet of Γ 6 . The light neutrino masses arise from the effective Weinberg operator and the minimal seesaw with two right-handed neutrinos in the type IV and type V models respectively. Moreover, we numerically scan over the parameter space of some example models, and the correlations among the mixing parameters are plotted.
Furthermore, we consider the modular invariant models for N = 2 and N = 6. Then the full modular flavor symmetry is Γ(2) and the finite modular group is Γ(2)/Γ(6) ∼ = T .
The modular forms of level 6 can be decomposed into multiplets of T . From table 8 we see that there are a number of modular forms in the same representation of T at a given weight. As a consequence, the resulting model has a plenty of free parameters. We expect that the independent free parameters could be reduced considerably if the modular symmetry is combined with the traditional flavor symmetry for instance in the context of eclectic flavor group [83][84][85][86][87].
It is well-known that top quark is much heavier than other quark masses. Therefore the first and the second generations of quark fields are usually assigned to a doublet of certain flavor symmetry group while the third generation is assigned to a singlet. The modular group Γ 6 has abundant one-dimensional and two-dimensional irreducible representations. It is interesting to apply the Γ 6 modular symmetry to understand the hierarchical quark masses and mixing angles and construct a quark-lepton unification model which can describe the experimentally measured values of both leptons and quarks for a common modulus τ . This is beyond the scope of present work, and the results of this work pave the way for such future studies.
A Group Theory of Γ 6 The homogeneous finite modular group Γ 6 can be generated by the modular generators S and T obeying the multiplication rules 4 : From Eq. (2.8), we know that Γ 6 is isomorphic to Γ 2 × Γ 3 = S 3 × T where Γ 3 ∼ = T is the double covering of the tetrahedron group A 4 and Γ 2 ∼ = S 3 is the permutation group of order 3.
The group Γ 6 ∼ = S 3 × T can also be generated by four generatorsã,b,c andd fulfilling the following relationsã whereã andb are the generators of T , andc andd are the generators of S 3 . It is easy to check that the multiplication rules of Γ 6 in above two bases are fulfilled bỹ In the same fashion, the following relations can also be obtained The 144 elements of the modular group Γ 6 can be divided into 21 conjugacy classes and it has 104 non-trivial abelian subgroups which include seven Z 2 subgroups, thirteen Z 3 subgroups, twelve Z 4 subgroups, three K 4 subgroups, thirty-seven Z 6 subgroups, nine Z 2 × Z 4 , four Z 3 ×Z 3 subgroups, three Z 12 subgroups, twelve Z 2 ×Z 6 subgroups and four Z 3 ×Z 6 subgroups. The Γ 6 group has twenty-one irreducible representations: six singlet representations which are labeled as 1 r k , nine two-dimensional representations defined as 2 k and 2 r k , two threedimensional representations 3 r , three four-dimensional representations labeled as 4 k and one six-dimensional representation 6, where the superscript r and the subscript k can take the values r = 0, 1 and k = 0, 1, 2, respectively. In the present work, the explicit forms of the generators S and T (ã,b,c andd) in the twenty-one irreducible representations are summarized in table 5 (table 6), where the matrices a 2 , b 2 , a 3 and b 3 in the two tables are taken to be From the representation matrices of generators in table 5 or table 6, one can easy to obtain the character table of Γ 6 . Then the Kronecker products between different irreducible representations read as Table 5: The representation matrices of the generators S and T for the twenty-one irreducible representations of Γ 6 in the chosen basis, where ω = e 2πi/3 , r = 0, 1, k = 0, 1, 2 and matrices a 2 , b 2 , a 3 and b 3 are shown in Eq. (A.5). Table 6: The representation matrices of the generatorsã,b,c andd for the irreducible representations of groups T , S 3 and Γ 6 in the chosen basis, where ω = e 2πi/3 , r = 0, 1 and k = 0, 1, 2. In the present work, the representation matrices of T are related to those of [88] by the similarity transformation u 2 = diag(1, e 5πi 12 )P 2 for two-dimensional representations 2 0 k and u 3 = diag(1, ω 2 , ω) for three-dimensional representation 3 0 . The representation matrices of generators of S 3 can be found in [4]. The irreducible representations of Γ 6 = S 3 ×T can be obtained from the direct products of those of S 3 and T [89]. For example, the two-dimensional representation 2 k of Γ 6 is exactly the direct product 2 0 × 1 0 k .