Modular $S_4$ and $A_4$ Symmetries and Their Fixed Points: New Predictive Examples of Lepton Mixing

In the modular symmetry approach to neutrino models, the flavour symmetry emerges as a finite subgroup $\Gamma_N$ of the modular symmetry, broken by the vacuum expectation value (VEV) of a modulus field $\tau$. If the VEV of the modulus $\tau$ takes some special value, a residual subgroup of $\Gamma_N$ would be preserved. We derive the fixed points $\tau_S=i$, $\tau_{ST}=(-1+i\sqrt{3})/2$, $\tau_{TS}=(1+i\sqrt{3})/2$, $\tau_T=i\infty$ in the fundamental domain which are invariant under the modular transformations indicated. We then generalise these fixed points to $\tau_f=\gamma\tau_S$, $\gamma\tau_{ST}$, $\gamma\tau_{TS}$ and $\gamma\tau_{T}$ in the upper half complex plane, and show that it is sufficient to consider $\gamma\in\Gamma_{N}$. Focussing on level $N=4$, corresponding to the flavour group $S_4$, we consider all the resulting triplet modular forms at these fixed points up to weight 6. We then apply the results to lepton mixing, with different residual subgroups in the charged lepton sector and each of the right-handed neutrinos sectors. In the minimal case of two right-handed neutrinos, we find three phenomenologically viable cases in which the light neutrino mass matrix only depends on three free parameters, and the lepton mixing takes the trimaximal TM1 pattern for two examples. One of these cases corresponds to a new Littlest Modular Seesaw based on CSD$(n)$ with $n=1+\sqrt{6}\approx 3.45$, intermediate between CSD$(3)$ and CSD$(4)$. Finally, we generalize the results to examples with three right-handed neutrinos, also considering the level $N=3$ case, corresponding to $A_4$ flavour symmetry.


Abstract
In the modular symmetry approach to neutrino models, the flavour symmetry emerges as a finite subgroup Γ N of the modular symmetry, broken by the vacuum expectation value (VEV) of a modulus field τ . If the VEV of the modulus τ takes some special value, a residual subgroup of Γ N would be preserved. We derive the fixed points τ S = i, τ ST = (−1 + i √ 3)/2, τ T S = (1 + i √ 3)/2, τ T = i∞ in the fundamental domain which are invariant under the modular transformations indicated. We then generalise these fixed points to τ f = γτ S , γτ ST , γτ T S and γτ T in the upper half complex plane, and show that it is sufficient to consider γ ∈ Γ N . Focussing on level N = 4, corresponding to the flavour group S 4 , we consider all the resulting triplet modular forms at these fixed points up to weight 6. We then apply the results to lepton mixing, with different residual subgroups in the charged lepton sector and each of the right-handed neutrinos sectors. In the minimal case of two right-handed neutrinos, we find three phenomenologically viable cases in which the light neutrino mass matrix only depends on three free parameters, and the lepton mixing takes the trimaximal TM1 pattern for two examples. One of these cases corresponds to a new Littlest Modular Seesaw based on CSD(n) with n = 1 + √ 6 ≈ 3.45, intermediate between CSD (3) and CSD (4). Finally, we generalize the results to examples with three right-handed neutrinos, also considering the level N = 3 case, corresponding to A 4 flavour symmetry.

Introduction
It is well-known that there are huge mass hierarchies among the quarks and leptons, and the quark mixing angles are small while the lepton sector has two large mixing angles θ 12 , θ 23 and one small mixing angles θ 13 which is of the same order of magnitude as the quark Cabibbo mixing angle [1]. The origin of the flavour structure of the quarks and leptons such as the mass hierarchies, mixing angles and CP violation phases is a big mystery of particle physics. Considerable effort has been devoted to understanding fermion masses and flavour mixing from flavour symmetry for decades. In particular, it is found that the observed lepton mixing angles can be naturally explained by the non-abelian discrete flavour symmetry, see Refs. [2][3][4][5][6][7] for review. Moreover, the leptonic CP violation phases can be predicted and the precisely measured quark CKM mixing matrix can be accommodated if the discrete flavour symmetry is combined with generalized CP symmetry [8][9][10][11].
In the usual paradigm of discrete flavour symmetry, the standard model gauge symmetry is extended by certain finite flavour symmetry at high energy scale which is subsequently broken down to different subgroups in the neutrino and charged lepton sectors at low energy. In general some flavon fields which are standard model singlets are required to realize the flavour symmetry breaking. The flavon fields usually obtain vacuum expectation values (VEVs) along specific directions in order to reproduce phenomenologically viable lepton mixing angles. As a consequence, the scalar potential of discrete flavour symmetry models is rather elaborate, and certain auxiliary abelian symmetries are usually needed to forbid dangerous operators.
Recently, modular symmetry has been suggested as the origin of flavour symmetry [12]. In this new framework, flavon fields might not be needed and the flavour symmetry can be uniquely broken by the VEV of the modulus τ . Moreover, all higher-dimensional operators in the superpotential are completely determined by modular invariance if supersymmetry is exact. In modular invariant models, the Yukawa couplings transform nontrivially under the modular symmetry and they are modular forms which are holomorphic functions of τ [12]. Models with modular flavour symmetry can be highly predictive, the neutrino masses and mixing parameters can be predicted in terms of few input parameters, although the predictive power of this framework may be reduced by the Kähler potential which is less constrained by modular symmetry [13].
It has been realised that, if the VEV of the modulus τ takes some special value, a residual subgroup of the finite modular symmetry group Γ N would be preserved. The phenomenological implications of the residual modular symmetry have been discussed in the context of modular A 4 [21], S 4 [26] and A 5 [30] symmetries. If the modular symmetry is broken down to a residual Z 3 (or Z 5 ) subgroup in charged lepton sector and to a Z 2 subgroup in the neutrino sector, the trimaximal TM1 and TM2 mixing patterns can be obtained [21,26].
In this paper, we derive the fixed points τ S = i, τ ST = (−1 + i √ 3)/2, τ T S = (1 + i √ 3)/2, τ T = i∞ in the fundamental domain which are invariant under the modular transformations indicated. We then generalise these fixed points to τ f = γτ S , γτ ST , γτ T S and γτ T in the upper half complex plane, and show that it is sufficient to consider γ ∈ Γ N . Focussing on level N = 4, corresponding to the flavour group S 4 , we consider all the resulting triplet modular forms at these fixed points up to weight 6. We then apply the results to obtain new predictive examples of lepton mixing, with different residual subgroups in the charged lepton sector and each of the right-handed neutrinos sectors. In the minimal case of two right-handed neutrinos, we find three phenomenologically viable cases in which the light neutrino mass matrix only depends on three free parameters, and the lepton mixing takes the trimaximal TM 1 pattern for two examples. Finally, we generalize the results to examples with three right-handed neutrinos, also considering the level N = 3 case, corresponding to A 4 flavour symmetry, listing the values of modular forms at the fixed points in this case also.
It is interesting to compare the modular symmetry approach here to the tri-direct CP approach in usual discrete flavour symmetry based on the residual symmetry [43,44], which can also give rise to very predictive models such as the Littlest seesaw model based on CSD(n) with n = 3, 4 or a variant [45][46][47][48]. In the present work, we shall also follow the tri-direct approach but in the context of modular invariant models and without CP. In the minimal scenario, with two right-handed neutrinos, the two columns of the Dirac neutrino mass matrix are modular forms which are aligned along certain directions at some special values of τ . One of the minimal S 4 cases corresponds to a new Littlest Modular Seesaw based on CSD(n) with n = 1 + √ 6 ≈ 3.45, intermediate between CSD (3) and CSD(4). The paper is organized as follows. In section 2, we give a brief review on the modular symmetry and the modular forms of level 4 are constructed from the products of the Dedekind η function. In section 3, we analyze the nontrivial fixed points τ f of the complex modulus which preserves a residual modular subgroup. We find there are infinity fixed modulus with the form τ f = γτ S , γτ ST , γτ T S and γτ T , where γ is an arbitrary modular transformation and However, the independent alignments of modular forms at the fixed points are finite. In section 4, we present the framework of tri-direct modular model, the minimal scenario is based on the two right-handed neutrinos model, and we generalize this approach to the three right-handed neutrinos case. In order to show concrete examples, we analyze the tri-direct modular models for the S 4 group. Furthermore, in section 5 we present the cases which can also be obtained, if the modular symmetry is A 4 instead of S 4 . We conclude in section 6. Finally the group theory of S 4 as well as the Clebsch-Gordan coefficients in our basis are collected in Appendix A.
2 Modular symmetry and modular form multiplets of level N = 4 The modular group Γ is the group of linear fraction transformations which acts on the complex modulus τ in the upper half complex plane as follow, We note that the map is an isomorphism from the modular group to the projective matrix group P SL(2, Z) ∼ = SL(2, Z)/{±I}, where SL(2, Z) is the group of two-by-two matrices with integer entries and determinant equal to one. It is obvious that therefore we identify a b c d is the same as −a −b −c −d (4) in matrix notation. The modular group Γ can be generated by two generators S and T which are represented by the following two matrices of P SL(2, Z), We can check that the generators S and T obey the relations, The principal congruence subgroup of level N is the subgroup which is an infinite normal subgroup of SL(2, Z). It is easy to see that T N is an element of Γ(N ). The projective principal congruence subgroup is defined as Γ(N ) = Γ(N )/{±I} for N = 1, 2. For the values of N ≥ 3, we have Γ(N ) = Γ(N ) because Γ(N ) doesn't contain the element −I. The quotient group Γ N ≡ Γ/Γ(N ) is the finite modular group, and it can be obtained by further imposing the condition T N = 1 besides those in Eq. (7). A crucial element of the modular invariance approach is the modular form f (τ ) of weight k and level N . The modular form f (τ ) is a holomorphic function of the complex modulus τ and it is required to transform under the action of Γ(N ) as follows, The modular forms of weight k and level N span a linear space of finite dimension. It is always possible to choose a basis in this linear space such that the modular forms can be arranged into some modular multiplets f r ≡ (f 1 (τ ), f 2 (τ ), ...) T which transform as irreducible representation r of the finite modular group Γ N for even k [12,32], i.e.
where γ is the representative element of the coset γΓ(N ) in Γ N , and ρ r (γ) is the representation matrix of the element γ in the irreducible representation r.
In this case, we have a 0 = −d 0 . The fixed modulus τ 0 in Eq. (30) is of the following form, Since τ 0 is in fundamental region with | τ 0 | ≤ 1 2 and |τ 0 | ≥ 1, thus we have the following constraint, which implies Hence the modular transformation γ 0 takes the form Notice that S and −S lead to the same linear fractional transformation, as shown in Eq. (4). Accordingly the fixed point is The parameter d 0 can be expressed in terms of a 0 as The solution for τ 0 in Eq. (30) is simplified into The requirement of τ 0 ∈ F entails which leads to As a consequence, γ 0 and τ 0 are determined to be or In this case, the parameter d 0 = −1 − a 0 and the solution of τ 0 is The condition τ 0 ∈ F imposes the following constraints, Hence the parameters a 0 , b 0 , c 0 and d 0 are determined to be The modular transformation γ 0 and the corresponding fixed point τ 0 are found to be given by or We see that are the fixed points of the modular transformation ±ST and ±T S respectively. Moreover they are related by modular transformation T , Furthermore, there is a fourth fixed point τ 0 = i∞. It is easy to check that i∞ is invariant under the action of T as T (i∞) = i∞ + 1 = i∞. We shall denote τ T ≡ i∞ in the following. In short, we have only the following four nontrivial fixed points in the fundamental domain, In general, a modular multiplet Y (k) r (τ ) of weight k in the irreducible representation r of the finite modular group Γ N satisfies the following property, where J k (γ, τ ) is the so-called automorphy factor [50], At the fixed point τ 0 , Eq. (51) gives us, Hence the modular multiplets Y r (τ 0 ) at the fixed point τ 0 is actually the eigenvector of the representation matrix ρ r (γ 0 ) with eigenvalue J −1 k (γ 0 , τ 0 ). It is straightforward to obtain Therefore the automorphy factor J k (γ 0 , τ 0 ) at the fixed points τ S , τ ST , τ T S and τ T must be a phase with unit absolute value. We find that the weight two modular forms at these fixed points take the following values, 3 (τ S ) is an eigenvector of the representation matrix ρ 3 (S) with eigenvalue −1. However ρ 3 (S) has two degenerate eigenvalues −1, consequently the alignment of Y 3 (τ S ) can not be uniquely fixed by Eq. (54). It is remarkable that modular symmetry helps to break the degeneracy in some sense and fix Y

Fixed points in upper half complex plane
Let us consider the general fixed points τ f of the modular transformation γ f in the upper half complex plane. By definition, τ f must be related to some modulus τ ∈ F by certain modular transformation γ , i.e., τ f = γ τ . Then the fixed point condition γ f τ f = τ f becomes Multiplying γ −1 from left on both sides of the above equation, we can obtain which is exactly the fixed point condition Eq. (26) inside the fundamental domain. Hence we have Hence the fixed point τ f of the modular group and the corresponding stabilizer γ f are given by where γ is an arbitrary modular symmetry element. It is straightforward to check γ f τ f = τ f which means that τ f is really the fixed point of γ f . Hence all fixed points are related to τ S , τ ST , τ T S , and τ T by modular transformations. For illustration, we display part of the fixed points of the modular group in figure 1. Since there are infinity fixed points, it is impossible to show all the fixed points. From Eq. (60) we see that only the modular symmetry transformation conjugate to γ 0 can have fixed point. In other words, γ f and γ 0 must belong to the same conjugacy class. The value of the modular form at the fixed point which implies that the direction of Y r (τ f ) is proportional to ρ r (γ )Y r (τ 0 ). The modular group Γ can be decomposed into disjoint union of right cosets of the principal congruence subgroup Γ(N ) in Γ, where we taken into account the fact Γ N = Γ/Γ(N ) such that the finite modular group Γ N can be regarded as the system of right coset representatives of Γ(N ) in Γ. As a consequence, any element γ ∈ Γ can always be written as . The nonequivalent directions of the modular forms Y r (τ f ) at fixed points are actually generated by the finite modular group Γ N . Although there are infinite numbers of fixed points τ f , it sufficient to only consider these fixed points τ f which belong to the orbits Γ N τ S , Γ N τ ST , Γ N τ T S and Γ N τ T . Moreover, for any fixed point τ 0 = τ S , τ ST , τ T S or τ T in the fundamental domain, the corresponding stabilizer subgroup is an abelian subgroup generated by γ 0 and it is denoted as Stab Γ N (τ 0 ) ≡ γ 0 . We can decompose the modular group Γ N into the right coset of the stabilizer subgroup where A i is the right coset representative element. Thus the orbit Γ N τ 0 can be generally simplified into Hence the number of the independent modulus in the orbit Γ N τ 0 is equal to |Γ N |/|Stab Γ (τ 0 )| which is the number of distinct right cosets of Stab Γ N (τ 0 ). Here the notation |G| denotes the order of a group G. Furthermore, after some algebra, we can show that the modular multiplet Y r (τ f ) has the following property which gives rise to This means that the modular multiplet Y r (τ f ) at the fixed point τ f is an eigenvector of the representation matrix ρ r (γ f ) with the eigenvalue J −1 k (γ 0 , τ 0 ) given in Eq. (55). We give the nontrivial and nonequivalent fixed points and the corresponding alignments of the triplet modular forms of level 4 in table 2. As we shall show in the present work, these alignments at the fixed points could give a rich phenomenology of neutrino mixing in the framework of tri-direct modular approach.
The alignments of triplet modular forms Y 3,3 (γτ S ) of level 4 up to weight 6 γ γτ S Y 2 ) The alignments of triplet modular forms Y 3,3 (γτ ST ) of level 4 up to weight 6 γ γτ ST Y 3,II (γτ T S ), Y The alignments of triplet modular forms Y 3,3 (γτ T ) of level 4 up to weight 6 γ γτ T Y 3,I (γτ T ), Y 3,II (γτ T ) Y 3,3 (τ f ) and Y (6) 3,3 (τ f ) of level 4 at the fixed point τ f . As shown in section 3, it is sufficient to only consider the fixed points τ f = γτ S , γτ ST , γτ T S , γτ T with γ ∈ S 4 . In the second column of the table, we have identified the modulus parameter τ with T 4 τ = τ + 4, the reason is T 4 = 1 in the S 4 group and Y r (τ ) = Y r (τ + 4) for any modular multiplet Y r .

Residual modular symmetry and its implication
We assume that neutrinos are Majorana particles in the following. The charged lepton and neutrino mass terms in modular invariant approach can be generally written as where the two-component notation of the fermion fields is used. The fields L i and E c i = e c , µ c , τ c stand for the left-handed lepton doublets and the right-handed charged leptons respectively, H u and H d are Higgs doublets, and Y e (τ ) and Y ν (τ ) are modular forms. The neutral components of the Higgs fields acquire vacuum expectation values H 0 u,d = v u,d after the electroweak symmetry breaking, then the charged lepton and neutrino mass matrices can be read off as Under a generic modular transformation γ ∈ Γ, the lepton fields L = (L 1 , L 2 , L 3 ) and E c = (e c , µ c , τ c ) T transform as where −k L and −k E are the modular weights of L and E c respectively. Modular invariance requires that the summation of the modular weights of each term in Eq. (66) should be vanishing and the mass matrices m e (τ ) and m ν (τ ) should transform under a modular transformation as In modular invariant models, the vacuum expectation value of the complex modulus τ is the unique source of flavour symmetry breaking. If τ is at some fixed point τ = τ f , a residual subgroup γ n f : n ∈ Z generated by the stabilizer γ f would be preserved.
where J k L +k E (γ f , τ f ) = J k L +k E (γ 0 , τ 0 ) and J 2k L (γ f , τ f ) = J 2k L (γ 0 , τ 0 ) are certain phase factors 3 , as shown in Eq. (55). From Eq. (70), we know that the hermitian combination of the charged lepton mass matrix m † e (τ f )m e (τ f ) should be subject to the following constraint, which implies ρ L (γ f ) and m † e (τ f )m e (τ f ) are commutable. The representation matrix ρ L (γ f ) can be diagonalized by a unitary matrix U e , where ρ L (γ f ) is a diagonal unitary matrix and the non-vanishing entries are eigenvalues of ρ L (γ f ). Then we can see If the diagonal entries of ρ L (γ f ) are non-degenerate, then U † e m † e (τ f )m e (τ f )U e is also diagonal since only a diagonal matrix is invariant when conjugated by a non-degenerate phase matrix. If ρ L (γ f ) has two degenerate diagonal entries, only one column of U e can be determined at the fixed point τ f . In the same fashion, we can show that the neutrino mass matrix m ν (τ f ) is also diagonalized by the unitary transformation U e in Eq. (72). As a result, the lepton mixing matrix would have six zeros for non-degenerate ρ L (γ f ) or four zeros for partially degenerate ρ L (γ f ) [21,26], and this is not consistent with the experimental data. Therefore there are no phenomenologically viable models with one common fixed point τ f in both neutrino and charged lepton sectors.
In order to accommodate the observed lepton mixing angles, it is assumed the complex modulus τ in the charged lepton and neutrino mass matrices takes two different values τ f,e and τ f,ν which break the modular symmetry into the residual subgroups generated by γ f,e and γ f,ν respectively in the charged lepton and neutrino sectors. In this approach, the trimaximal TM2 lepton mixing pattern can be obtained from the A 4 modular group [21] and the models giving TM1 mixing are constructed with multiple modular S 4 groups [38,39]. Inspired by the tri-direct CP approach [43,44,48], in the following we shall present the tri-direct modular model which has three fixed moduli.

New Predictive Examples of Lepton Mixing
The tri-direct approach is based on the minimal seesaw model with two right-handed neutrinos which are denoted as N c atm and N c sol [43,44,48]. In this section we shall generalize the tri-direct approach to the modular invairance models, and the flavon fields would be replaced by modular forms. The superpotential for the charged lepton and neutrino masses in the tri-direct modular model is of the following form where we don't specify whether the above operators arise from only one independent or several independent modular invariant contractions. We assign the lepton doublets L to triplet of the finite modular group Γ N , while both right-handed neutrinos N c atm and N c sol transform as singlets of Γ N . Then modular invariance requires Y atm (τ ) and Y sol (τ ) are triplet modular forms. For instance, in the case of N = 4, we can assume L ∼ 3, N c atm ∼ 1, The cross term N c atm N c sol could be forbidden by proper weight assignment of N c atm and N c sol . Motivated by the idea of tri-direct CP approach [43,44,48], we shall consider the scenario of three moduli in the theory: τ e and its vacuum expectation value (VEV) at certain fixed point τ f,e break the modular symmetry in the charged lepton sector, τ atm and τ sol are responsible for the breaking of modular symmetry in the atmospheric and solar neutrino sectors respectively. Similar to section 3.3, the charged lepton mass matrix is determined to be m e (τ e ) = y e Y e (τ e )v d and it is diagonalized by the unitary matrix U e fulfilling U † e ρ L (γ f,e )U e = ρ L (γ f,e ) at the fixed point τ e = τ f,e . Notice that U e is determined up to permutations and phases of its column vectors since the order of the charged lepton masses is undefined in our framework. Subsequently we can read out the neutrino Dirac mass matrix and the Majorana mass matrix of right-handed neutrinos, where the Clebsch-Gordan coefficients in both contractions y atm LY atm N c atm H u and y sol LY sol N c sol H u are omitted for notation simplicity. The effective light neutrino mass matrix is given by the seesaw formula m ν = −m D m N m T D which implies Table 3: The assignments for the modular forms Y atm and Y sol and the VEVs of τ atm and τ sol for the three phenomenologically viable cases of tri-direct modular approach with two right-handed neutrinos.
We shall assume that the vacuum expectation values of τ atm and τ sol are certain fixed points τ atm = τ f,atm and τ sol = τ f,sol so that some residual modular groups generated by γ f,atm and γ f,sol are preserved. In order to show concrete examples and find new interesting models, we shall perform a comprehensive study for the modular group S 4 . Note that we leave the construction of tri-direct modular model in future work.

Examples of N = 4
We have considered all possible fixed points of the three moduli τ e , τ atm , τ sol and the corresponding alignments of triplets modular forms shown in table 2. We find three possible cases which can accommodate the experimental data on neutrino masses and mixing angles. The assignments for the modular forms Y atm and Y sol and the vacuum expectation values of the moduli τ atm and τ sol are summarized in table 3.
If the VEV of the modulus τ e is τ e = τ ST , the modular symmetry would be broken down to the residual subgroup G e = Z ST 3 in the charged lepton sector. Since the representation matrix of the element ST is diagonal, the charged lepton diagonalization matrix would be a unit matrix U e = 1 3×3 up to column permutations, and the lepton flavour mixing completely originates from the neutrino sector. From table 2 we can read out the alignments of the triplet atmospheric and solar modular forms at the fixed points, Notice that the case B corresponds to the CSD(n) model with n = 1+ √ 6, the two columns of the Dirac mass matrix are proportional to (0, 1, −1) and (1, n, 2 − n) respectively in CSD(n) model [45][46][47][48]. The predictive Littlest seesaw model and its variant are the cases of n = 3 [45,47,[53][54][55], n = 4 [46,[56][57][58] and n = −1/2 [48] respectively. It has been shown that the CSD(n) model can be reproduced from the S 4 flavour symmetry in the tri-direct CP approach [43,44], and the parameter n is constrained to be a generic real parameter by the S 4 flavour symmetry and CP symmetry [43,44]. Here the modular symmetry can fix the alignment parameter n to be 1 + √ 6. This is a remarkable advantage of modular symmetry with respect to discrete flavour symmetry. From Eq. (76), we find the neutrino mass matrix is predicted to be Case η/π m a (meV) m s /m a χ 2 min sin 2 θ 13 sin 2 θ 12 sin 2 θ 23 δ CP /π β/π m 2 (meV) m 3 (meV) m ee (meV) It is notable that only three free parameters m a , m s and η are involved in the neutrino mass matrix. It is straightforward to check that the column vector (2, −1, −1) T is an eigenvector of m ν with vanishing eigenvalue. Therefore the neutrino mass spectrum is normal ordering, and lightest neutrino is massless m 1 = 0, and the lepton mixing matrix is determined to be the TM1 pattern, Using the general formulae for lepton mixing angles, CP violating phases and neutrino masses in the tri-direct model given in [43,44], we find the experimental data can be accommodated very well for certain values of m a , m s and η. We present the results of χ 2 analysis for the different cases in table 4. The contour plots of sin 2 θ 13 , sin 2 θ 12 , sin 2 θ 23 and m 2 2 /m 2 3 in the plane r versus η are shown in figure 2 where r = m s /m a .

G
The residual symmetry of the charged lepton sector would be G e = Z ST 2 4 for the fixed point τ f,e = −1. In the case, the unitary transformation U e is of the following form, There are only one independent case which can be compatible with experimental data. The triplet modular forms Y atm and Y sol are aligned along the directions The most general form of the neutrino mass matrix reads as The lepton mixing matrix turns out to be of the form  Figure 2: The contour plots of sin 2 θ 12 , sin 2 θ 13 , sin 2 θ 23 and m 2 2 /m 2 3 in the η/π − r plane for cases A, B and C. The cyan, red, green and blue areas denote the 3σ regions of sin 2 θ 23 , sin 2 θ 13 and m 2 2 /m 2 3 respectively. The solid lines denote the 3 sigma upper bounds, the thin lines denote the 3 sigma lower bounds and the dashed lines refer to their best fit values, as adopted from NuFIT 4.1 [59].
Notice that the first column of the mixing matrix looks much more complex although the experimental data can be accommodated. The predictions for lepton mixing parameters and neutrino masses are listed in table 4.
For the cases A, B and C, both Y atm and Y sol are triplet modular forms with weights 2 or 4, and their values at fixed points are uniquely fixed, as shown in table 2. However, at weight 6 there are two linearly independent modular forms Y 3,I and Y (6) 3,II transforming as 3 under S 4 . As a consequence, if either Y atm or Y sol is a weight 6 modular form in the representation 3, it alignment would not be fixed uniquely by the residual modular symmetry. For instance, we assign G e = Z ST which implies The solar modular form Y sol is generally a linear combination of Y 3,I and Y 3,II , and it can be written as where r 1,2 and η 1,2 are real parameters. Then the light neutrino mass matrix can be parameterized as In this scenario, the neutrino mass matrix depends on five free parameters m a , r 1 , r 2 , η 1 and η 2 . The experimental data can be reproduced very well, e.g. m a = 20.362 meV, r 1 = 0.826, η 1 = −0.588π, r 2 = 0.427, η 2 = 0.062π , Choosing other possible values of the fixed points τ f,e , τ f,atm and τ f,sol , we can find similar models compatible with data. We shall not list all the possibilities since there are many viable cases.

Tri-direct modular model with three right-handed neutrinos
The above tri-direct modular approach can be straightforwardly extended to the conventional seesaw model with three right-handed neutrinos denote as N c atm , N c sol and N c dec . Then the superpotential for the charged lepton and neutrino masses is In this scenario, the theory has four moduli: τ e in the charged lepton sector, τ atm , τ sol and τ dec associated with the atmospheric, solar and "decoupled" right-handed neutrinos sectors respectively. Analogous to the two right-handed neutrinos case discussed above, we assume the modular symmetry is broken to different residual subgroups through the VEVs of τ e , τ atm , τ sol and τ dec . Many viable model can be found in this scenario. For illustration, we shall present one example in the following, and we assume the residual modular symmetry in the charged lepton sector and From table 2 we can read off the alignments of the modular forms as follow Consequently the neutrino mass matrix is given by where r 1,2 and η 1,2 are real free parameters. Excellent agreement with experimental data can be achieved in this case, and a numerical benchmark is The modular group Γ(3) has been extensively studied in the literature [12,14,15,[18][19][20][21][22]. The finite modular group Γ 3 is isomorphic to A 4 . In the present work we shall adopt the same convention as [12,23]. In the triplet representation 3, the A 4 generators S and T are represented by The weight 2 modular forms Y transform as a triplet of A 4 and it can be constructed in terms of η(τ ) as follow [12], There are five weight 4 modular forms given by The weight 6 modular forms can be decomposed as 1 ⊕ 3 ⊕ 3 under A 4 [12], The alignments of triplet modular forms Y 3,3 (γτ S ) of level 3 up to weight 6 γ γτ S Y (0, 0, 1) (0, 1, 0) (1, 0, 0) The alignments of triplet modular forms Y 3,3 (γτ T S ) of level 3 up to weight 6 γ γτ T S Y 3,I (γτ T ), Y 3,3 (τ f ) of level 3 at the fixed point τ f = γτ S , γτ ST , γτ T S , γτ T with γ ∈ A 4 . We have identified the modulus parameter τ with T 3 τ = τ + 3 in the second column because Y r (τ ) = Y r (τ + 3) for any modular multiplet Y r of level 3.
As shown in section 3, the modular group has infinite nontrivial fixed points while the nonequivalent alignments of the modular forms at fixed points are finite. It is sufficient to only consider the fixed points Γ N τ S , Γ N τ ST , Γ N τ T S and Γ N τ T . We report the nonequivalent fixed points and the alignments of the triplet modular forms for N = 3 in table 5.
In the framework of tri-direct modular model with two-right handed neutrinos, we have considered all possible residual symmetries in different sectors, yet no viable models can be found if the modular weights of Y atm and Y sol are equal to 2 or 4. Some models compatible with experimental data can be obtained if either Y atm or Y sol is a weight 6 modular form transforming as 3 under A 4 such that its alignment is not fixed uniquely. We give one example in the following, the residual symmetry of the charged lepton sector is The neutrino mass matrix is of the form of Eq. (88), and the measured values of the mixing parameters and neutrino masses can be accommodated very well, e.g.
In the tri-direct modular model with three right-handed neutrinos, we can also find many phenomenologically viable models from A 4 modular symmetry. We shall not list all the possibilities but give an example, we take G e = Z ST 3 and the atmospheric, solar and "decoupled" modular forms are assigned to transform as 3 , τ f,atm = τ S = i, τ f,sol = τ S = i, τ f,dec = T Sτ ST = 2 + ω . (102) The neutrino mass matrix in this case is of the following form The agreement with experiment data is optimised for m a = 48.353 meV, r 1 = 0.705, η 1 = 0.066π, r 2 = 0.859, η 2 = 1.037π , and accordingly the neutrino masses and mixing parameters are determined to be

Conclusion
Models based on modular flavour symmetry are highly predictive. In the most economical version of modular symmetry models, the VEV of the modulus τ is the unique source of the flavour symmetry breaking, and the flavon fields are not absolutely necessary. If the VEV of τ takes some special values, certain residual subgroup of the modular symmetry would be preserved. Inside the fundamental domain, we have derived the four nontrivial fixed points and Z T N respectively with N being the level of modular group. Other fixed points of modulus are related to τ S , τ ST , τ T S and τ T = i∞ through modular transformation. The most general form of the fixed modulus is given by τ f = γτ S , γτ ST , γτ T S , γτ T with γ ∈ Γ. Although the number of fixed point τ f is infinity, the inequivalent directions of the modular forms Y r (τ f ) at fixed points are finite. We have shown that it is sufficient to only consider the values of Y r (τ f ) with τ f given by Γ N τ S , Γ N τ ST , Γ N τ T S and Γ N τ T .
Before applying these results to lepton mixing, we first proved that it is necessary for the neutrino and charged lepton sectors to have different residual symmetry, since if they shared a common symmetry value of τ f , the lepton mixing would contain four or six zeros which is excluded by present data. The approach to lepton mixing followed here is motivated by the tri-direct CP approach in usual discrete flavour symmetry, however here we find that it is unnecessary to CP, as we discuss below. For example, in the minimal scenario with two righthanded neutrinos, we assumed that the modular symmetry is broken to different residual subgroups in the charged lepton sector, atmospheric neutrino sector and solar neutrino sector. In this case, the two columns of the Dirac neutrino mass matrix are triplet modular forms whose alignments are determined by residual symmetry at fixed points τ f . Focussing on level N = 4, corresponding to the flavour group S 4 , we considered all the resulting triplet modular forms at the above fixed points up to weight 6. We then applied the results to lepton mixing, with different residual subgroups in the charged lepton sector and each of the right-handed neutrinos sectors. In the minimal case of two righthanded neutrinos, we found three phenomenologically viable cases, denoted A, B and C. We showed that if only weight 2 and weight 4 modular forms are involved in the neutrino Yukawa coupling, the light neutrino mass matrix involves three real parameters and three independent cases can be compatible with the experimental data, as summarized in table 3. The cases A and B lead to TM1 mixing pattern, and the first column of the lepton mixing matrix is approximately (0.800, 0.421, 0.428) T for the case C. The case B corresponds to the CSD(n) model with n = 1 + √ 6 ≈ 3.45, intermediate between CSD (3) and CSD(4). We find it remarkable that, even without CP, the modular symmetry uniquely fixes the value of n to be a unique real number (to be compared to the usual tri-direct approach, where n takes an arbitrary real value, assuming a CP symmetry). If a higher weight (e.g., weight 6) modular form enters in the neutrino Yukawa coupling, its alignment is not constant vector any more at fixed point τ f and more free parameters are involved in the light neutrino mass matrix. Hence we find many cases can described the measured values of lepton masses and mixing in this scenario, and we give one example for illustration. Furthermore, we generalized the tri-direct modular approach to the models with three right-handed neutrinos, and many cases in agreement with experimental data can also be found.
In the same fashion, we also studied the possible tri-direct modular models for the level N = 3 case, corresponding to A 4 flavour symmetry. In the minimal model with two righthanded neutrinos, no models can be compatible with data if the modular weights of Y atm and Y sol are equal to 2 or 4. However, phenomenologically viable models can be found if the modular weights of either Y atm or Y sol are greater than 4 or three right-handed neutrinos are involved.
In the tri-direct modular approach here, the physical results only depend on the structure of modular symmetry group and the assumed residual symmetries. The details of the mechanism realising the desired residual symmetries are irrelevant. It seems that at least three complex moduli τ e , τ atm and τ sol responsible for the breaking of modular symmetry in the charged lepton sector, atmospheric neutrino sector and solar neutrino sector may be needed. It would be interesting to implement one of the successful cases A, B, C in a concrete model based on multiple modular symmetries [38,39] or some other mechanism which can lead to different values of τ in the charged lepton, atmospheric neutrino and solar neutrino sectors, and this is left for future work. where ω = e 2πi/3 = −1/2 + i √ 3/2 is a cubic root of unity.

Appendix
A Group theory of Γ 4 ∼ = S 4 The finite modular group Γ 4 has two generators S and T which fulfill the following rations The finite modular group Γ 4 is isomorphic to the permutation group S 4 of four objects. In order to see the correlation between S 4 and tri-bimaximal mixing and the connection to S 3 , A 4 groups more easily, it is convenient to generate the S 4 group in terms of three generatorŝ S,T andÛ with the multiplication rules [60,61], where nC k denotes a conjugacy class with n elements and the subscript k is the order of the elements. The group S 4 has five irreducible representations: two singlets 1 and 1 , one doublet 2 and two triplets 3 and 3 . In the present work, we choose the same basis as that of [61]. The explicit forms of S and T in each of the irreducible representations are summarized in table 6. Notice that the triplet representations 3 and 3 correspond to 3 and 3 of Refs. [25,26] respectively. The Clebsch-Gordan coefficients in our working basis can be found in [61], for completeness we present them in the following. We shall use α i to denote the elements of first representation and β i stands for the elements of the second representation of the tensor product.