1 Introduction

Thanks to various experiments involving neutrino oscillations [1,2,3,4,5], the mixing matrix in the leptonic sector is known to a good accuracy [6, 7]. The question then arises as to whether one can understand the mixing matrix from some theoretical considerations [8,9,10]. The considerations would definitely involve some symmetry. One can look at the experimentally allowed values for the elements of the leptonic mixing matrix, also called the PMNS matrix, and try to guess a symmetry that would fit the values. This approach has resulted, based on experimental data available at different points of time, in various schemes of neutrino mixing such as the bimaximal (BM) [11, 12] or the tribimaximal (TBM) [13, 14]. To explain these patterns, a different approach was considered, in which one writes down the Lagrangian at some high energy scale like the grand unified scale, and then predicts the patterns by considering the running of various parameters down to low scales. Literature of this approach is vast; see Refs. [8,9,10] and references therein. There is yet another approach [15,16,17,18,19] in which one starts with identifying symmetries of the mass terms in the physical basis, i.e., the basis in which the mass matrices are diagonal. One assumes that these symmetries are remnants of a bigger symmetry group that is presumably valid at some large energy scale. One tries to identify this bigger group by starting with the remnant symmetries as generators and building the group by imposing suitable conditions on the generators. The bigger symmetry then dictates the structure of the PMNS matrix: if not the whole matrix, at least some elements of it. Luckily in this approach, one is not required to write down any Lagrangian for high energy scale. Simple group-theoretical considerations give us the PMNS matrix.

We assume that the neutrinos are Majorana particles, in which case the symmetries associated with the neutrino field redefinitions must be involutions, i.e., the square of the symmetry transformations must yield the identity transformation. In the charged lepton sector, the redefinitions can be different. For the sake of simplicity, one assumes a discrete symmetry in this sector as well [15]. At first, people tried to obtain the TBM matrix with this assumption, because the TBM was consistent with experimental data at that time. When the experimental data confirming a non-zero value of \(\theta _{13}\) poured in [2, 3], the work continued with the same philosophy, but with the changed data. In such attempts [18, 19], it was first assumed that the eigenvalues of the symmetry generators of the charged leptons are non-degenerate. However, degeneracy cannot be ruled out from any physical consideration, and a large number of authors have explored various symmetries, with or without degenerate eigenvalues [20,21,22,23,24,25,26,27,28]. They have identified several groups that can predict at least some of the elements of the PMNS matrix.

In this work, we perform a systematic analysis of the possibility that all symmetries in the leptonic sector of the Lagrangian are generated by involutions, i.e., elements which are of order 2. A class of these groups are called Coxeter groups, a term that will be defined in Sect. 6, where such groups will be discussed in detail. Some groups of this kind were arrived at by Lam [29] while searching for “built-in” symmetries in SO(10) grand unified models. In addition, many of the groups that we encounter in our search have been discussed sporadically by other authors (see Table II of Ref. [10] and the references therein), and therefore many of the results that we obtain are not new. The novelty of our approach is the use of the theory of Coxeter groups. We use the theory to identify all Coxeter groups with four or fewer generators, and discuss their relevance in this context. Further, it also helps us identifying other involution groups, because they must be subgroups of infinite Coxeter groups.

When one deals with groups generated by involutions only, degeneracy in the eigenvalues cannot be avoided since there exist only two distinct eigenvalues for each generator. Therefore, after a general analysis of the symmetries of the mass terms and currents and the presentation of some earlier results in Sect. 2, we discuss how to tackle degenerate phases in Sect. 3. We summarize the experimental data in Sect. 4 and set up the strategy for the search of groups that might be consistent with the data. In Sect. 6 we introduce Coxeter groups and perform the analysis with finite Coxeter groups with four or less generators. In Sect. 7 we consider other finite groups generated by involutions only, by treating them as subgroups of infinite Coxeter groups, as argued before. We summarize our findings and our outlook in Sect. 8.

2 Consideration of symmetries

We start with a brief review of the method [18, 19]. If the Lagrangian of the leptonic sector is written in terms of the mass eigenstates, it contains the following terms:

$$\begin{aligned} \mathscr {L}= & {} \sum _{\ell ,\alpha } \left[ \frac{g}{\surd 2} \left( \bar{\ell }U_{\ell \alpha } \gamma ^\mu L \nu _\alpha W_\mu ^+ + \text{ h.c. } \right) \right. \nonumber \\&\left. - M_\ell \bar{\ell }\ell - \frac{1}{2} m_\alpha \nu _\alpha ^\top C \nu _\alpha \right] , \end{aligned}$$
(2.1)

where \(\ell =e,\mu ,\tau \), whereas the neutrino eigenstates are indexed by \(\alpha =1,2,3\). Note that here and anywhere else, we use the Einstein summation convention for Lorentz indices, but not for indices used for differentiating different particles. The neutrinos have been assumed to be Majorana particles. There are of course other terms in the Lagrangian that involves the leptons, but they are not crucial for the argument that follows.

The mass terms in Eq. (2.1) admit the following symmetries:

$$\begin{aligned} \nu _\alpha\longrightarrow & {} \eta _\alpha \nu _\alpha , \end{aligned}$$
(2.2a)
$$\begin{aligned} \ell\longrightarrow & {} e^{i\phi _\ell } \ell . \end{aligned}$$
(2.2b)

Because each neutrino eigenstate has to satisfy the Majorana condition,

$$\begin{aligned} \bar{\nu }_\alpha = \nu ^\top C \end{aligned}$$
(2.3)

for some unitary matrix C, each \(\eta _\alpha \) must be real, i.e.,

$$\begin{aligned} \eta _\alpha = \pm 1 \qquad \forall \alpha . \end{aligned}$$
(2.4)

The maximum possible symmetry in Eq. (2.2) is therefore \([Z_2]^3\) from the neutrino sector and \(\mathrm [U(1)]^3\) from the charged leptons. The interaction of leptons with the Z-boson and the Higgs boson also obey this symmetry, which is why we have not written them down in Eq. (2.1). On the other hand, the charged current interaction term of Eq. (2.1) remains invariant if we augment the transformations of Eq. (2.2) by the rule

$$\begin{aligned} U_{\ell \alpha } \longrightarrow U_{\ell \alpha } e^{-i\phi _\ell } \eta _\alpha . \end{aligned}$$
(2.5)

This is then the arbitrariness in the definition of the PMNS matrix U.

Because of this arbitrariness, it would be pointless to try to predict the PMNS matrix U. However, one can try whether one can somehow use the symmetry of Eq. (2.5) to get some information as regards the absolute values of the matrix elements of U. To this end, we rewrite Eq. (2.2) in the form

$$\begin{aligned} \varvec{\nu }\longrightarrow & {} S \varvec{\nu }, \end{aligned}$$
(2.6a)
$$\begin{aligned} \varvec{\ell }\longrightarrow & {} T \varvec{\ell }, \end{aligned}$$
(2.6b)

where the bold letters indicate column matrices in flavor space, whereas S and T are diagonal matrices. Now, in order to make the problem more tractable, one assumes that

$$\begin{aligned} \det S = \det T = 1, \end{aligned}$$
(2.7)

so that

$$\begin{aligned} \eta _1 \eta _2 \eta _3= & {} 1, \end{aligned}$$
(2.8a)
$$\begin{aligned} \phi _e + \phi _\mu + \phi _\tau= & {} 0 \,\mathrm{mod}\,2\pi . \end{aligned}$$
(2.8b)

Then the symmetry for the neutrino fields can be generated by the matrices

$$\begin{aligned} S_1= & {} \mathrm{diag}(1,-1,-1), \qquad S_2 = \mathrm{diag}(-1,1,-1), \nonumber \\ S_3= & {} \mathrm{diag}(-1,-1,1). \end{aligned}$$
(2.9)

Since

$$\begin{aligned} S_1 S_2 S_3 = 1, \end{aligned}$$
(2.10)

there are only two independent operations, and therefore the symmetry is at most \(Z_2 \times Z_2\). On the other hand, the symmetry in the charged lepton sector, already reduced to \(\mathrm [U(1)]^2\) because of Eq. (2.8b), is further reduced by assuming that T is the representation of a discrete group. The discrete group must then be of the form \(Z_n \times Z_{n'}\).

Let us now discuss the whole thing from the flavor basis of neutrinos. The neutrino fields in this basis are given by

$$\begin{aligned} \tilde{\varvec{\nu }} = U \varvec{\nu }, \end{aligned}$$
(2.11)

with the tilde indicating the basis in which the charged lepton mass matrix is diagonal. The neutrino mass matrix in this basis takes the form

$$\begin{aligned} \tilde{\varvec{m}} = U^* \varvec{m} U^\dagger . \end{aligned}$$
(2.12)

The symmetry of Eq. (2.6) now becomes

$$\begin{aligned} \tilde{\varvec{\nu }} \longrightarrow S' \tilde{\varvec{\nu }}, \qquad \tilde{\varvec{m}} \longrightarrow S'^\top \tilde{\varvec{m}} S' \end{aligned}$$
(2.13)

where

$$\begin{aligned} S' = U S U^\dagger , \end{aligned}$$
(2.14)

S being any of the matrices of Eq. (2.9). There is no reason to assume that \(S'\) and T commute. Depending on their structures, we can deduce the moduluses of different elements of the PMNS matrix U.

As the simplest example, we can consider that the flavor groups in neutrino and charged lepton sectors each has one generator only. Since Eq. (2.14) implies that \(S'^2=1\), it means that the symmetry group will be \(Z_2\) for the neutrino sector and \(Z_n\), for some n, in the charged lepton sector. The \(Z_2\) group of the neutrino sector must be generated by one of the matrices \(S'_\alpha \), defined through Eq. (2.14). One can then introduce the group element

$$\begin{aligned} W_\alpha = S'_\alpha T, \end{aligned}$$
(2.15)

and assume that this element also has a finite order. The group now will be defined by the relations

$$\begin{aligned} S_\alpha '^2 = T^n = W_\alpha ^p = 1, \end{aligned}$$
(2.16)

which defines the von Dyck group D(2, n, p).

In order to obtain the elements of the PMNS matrix, one evaluates

$$\begin{aligned} a_\alpha \equiv \mathrm{Tr}(W_\alpha ) = \sum _\ell (2|U_{\ell \alpha }|^2 -1)e^{i \phi _\ell }. \end{aligned}$$
(2.17)

Using real and imaginary parts of the above equation with unitary condition for U, one can exactly solve the absolute values of one column of the PMNS matrix. The solution is given by [18, 19]

$$\begin{aligned} |U_{e\alpha }|^2= & {} \frac{ \,\mathrm{Re}(a_\alpha ) \cos \frac{\phi _e}{2} + \cos \frac{3\phi _e}{2} - \,\mathrm{Im}(a_\alpha ) \sin \frac{\phi _e}{2} }{4\sin {\frac{\phi _e-\phi _\mu }{2}} \sin {\frac{\phi _\tau -\phi _e}{2}}}, \nonumber \\ \end{aligned}$$
(2.18)

with \(|U_{\mu \alpha }|^2\) and \(|U_{\tau \alpha }|^2\) obtained by making cyclic permutation of the indices e, \(\mu \), \(\tau \). Similarly, if we have two \(Z_2\) symmetries in the neutrino sector, we can obtain expressions for another column. Hence, by using unitarity conditions, the remaining column can also be determined, which means that we would know the absolute values of all entries of the PMNS matrix.

3 Degenerate phases

An expressions like that in Eq. (2.18) cannot be the most general formula for determination of the PMNS matrix elements [23]. One can easily see that they break down when any two of the phases \(\phi _e, \phi _\mu \) and \(\phi _\tau \) are equal. There can be three possibilities with a two-fold degeneracy:

$$\begin{aligned} T_e= & {} \mathrm{diag}\Big (e^{-2 i \phi _e},e^{i \phi _e},e^{ i \phi _e} \Big ),\nonumber \\ T_\mu= & {} \mathrm{diag}\Big ( e^{i \phi _\mu },e^{-2 i \phi _\mu },e^{ i \phi _\mu } \Big ), \\ T_\tau= & {} \mathrm{diag}\Big (e^{i \phi _\tau },e^{i \phi _\tau },e^{- 2 i \phi _\tau } \Big ).\nonumber \end{aligned}$$
(3.1)

In analogy with Eqs. (2.15) and (2.17), we now define the following quantities:

$$\begin{aligned} W_{\ell \alpha }= & {} T_\ell S'_\alpha , \end{aligned}$$
(3.2a)
$$\begin{aligned} a_{\ell \alpha }= & {} \mathrm{Tr}(W_{\ell \alpha }) = \mathrm{Tr}(T_\ell U S_\alpha U^\dagger ). \end{aligned}$$
(3.2b)

The traces are easily determined and one obtains

$$\begin{aligned} a_{\ell \alpha } = - e^{-2i \phi _\ell } + 2 |U_{\ell \alpha }|^2 (e^{-2i \phi _\ell } - e^{i\phi _\ell }). \end{aligned}$$
(3.3)

For generators of order 2, \(\phi _\ell =\pi \). This fact has several important consequences. First, we see from Eq. (3.1) that now the generators \(T_\ell \) are of the form

$$\begin{aligned} T_e= & {} \mathrm{diag}(1,-1,-1), \qquad T_\mu = \mathrm{diag}(-1,1,-1), \nonumber \\ T_\tau= & {} \mathrm{diag}(-1,-1,1), \end{aligned}$$
(3.4)

so that

$$\begin{aligned} T_e T_\mu T_\tau = 1, \end{aligned}$$
(3.5)

implying that only two of these generators are independent. Second, we can now use Eq. (3.3) to write

$$\begin{aligned} |U_{\ell \alpha }|^2 = \frac{1}{4} \Big (1 + a_{\ell \alpha }\Big ), \end{aligned}$$
(3.6)

which shows that \(a_{\ell \alpha }\) will have to be real.

Note that \(W_{\ell \alpha }\), defined in Eq. (3.2a), must be an element of the symmetry group. Suppose that the order of this element is \(p_{\ell \alpha }\). The quantity \(a_{\ell \alpha }\), being the trace of \(W_{\ell \alpha }\), is therefore nothing but the sum of three eigenvalues of \(W_{\ell \alpha }\), i.e., of three \(p_{\ell \alpha }^\mathrm{th}\) roots of unity. Let us denote these three roots by \(e^{i\theta _1}\), \(e^{i\theta _2}\) and \(e^{i\theta _3}\), where each of these \(\theta \)’s is of the form \(2\pi k/p_{\ell \alpha }\), with possibly different integral values of k but the same value of \(p_{\ell \alpha }\), that is, the order of the element \(W_{\ell \alpha }\). Since by Eq. (2.7) the determinant is unity, we obtain

$$\begin{aligned} \theta _1 + \theta _2 + \theta _3 = 0 \,\mathrm{mod}\,2\pi . \end{aligned}$$
(3.7)

Further, since the sum of the roots is real, we have

$$\begin{aligned} \sin \theta _1 + \sin \theta _2 + \sin \theta _3 = 0. \end{aligned}$$
(3.8)

Using Eq. (3.7) to eliminate \(\theta _3\), we can write Eq. (3.8) as

$$\begin{aligned} \sin \theta _1 + \sin \theta _2 = \sin (\theta _1+\theta _2), \end{aligned}$$
(3.9)

which can be rewritten in the form

$$\begin{aligned} \tan {\theta _1 \over 2} = - \tan {\theta _2 \over 2}. \end{aligned}$$
(3.10)

The most general solution of this equation is

$$\begin{aligned} \theta _1 = 2\pi m - \theta _2 \end{aligned}$$
(3.11)

for some integer m. Using this relation along with Eq. (3.7), we can determine all three eigenvalues of \(W_{\ell \alpha }\) in terms of one parameter, and write

$$\begin{aligned} a_{\ell \alpha }= & {} \exp \left( 2\pi i {k_{\ell \alpha } \over p_{\ell \alpha }} \right) + \exp \left( - 2\pi i {k_{\ell \alpha } \over p_{\ell \alpha }} \right) + 1 \nonumber \\= & {} 1 + 2 \cos \left( 2\pi {k_{\ell \alpha } \over p_{\ell \alpha }} \right) , \end{aligned}$$
(3.12)

restoring the definition of the \(\theta \)’s. We can then use Eqs. (3.6) and (3.12) to obtain

$$\begin{aligned} |U_{\ell \alpha }|^2 = \frac{1}{2} \left[ 1 + \cos \Big ({2\pi \frac{k_{\ell \alpha }}{p_{\ell \alpha }}} \Big ) \right] = \cos ^2 \Big ({\pi \frac{k_{\ell \alpha }}{p_{\ell \alpha }}} \Big ). \end{aligned}$$
(3.13)

The task is now to find different combinations of \(p_{\ell \alpha }\) and \(k_{\ell \alpha }\) that will produce values of \(|U_{\ell \alpha }|^2\) that fall within the experimentally allowed ranges. Before embarking on this journey, we summarize the experimental results that we are trying to fit.

4 Confronting experimental results

The PMNS matrix is written in terms of three angles and three CP-violating phases [30]:

$$\begin{aligned} U= & {} \left( \begin{array}{ccc} c_{12} c_{13} &{}\quad s_{12} c_{13} &{}\quad s_{13} e^{-i\delta } \\ -s_{12} c_{23} - c_{12} s_{23} s_{13} e^{i\delta } &{}\quad c_{12} c_{23} - s_{12} s_{23} s_{13} e^{i\delta } &{}\quad s_{23} c_{13} \\ s_{12} s_{23} - c_{12} c_{23} s_{13} e^{i\delta } &{}\quad - c_{12} s_{23} - s_{12} c_{23} s_{13} e^{i\delta } &{}\quad c_{23} c_{13} \\ \end{array}\right) \nonumber \\&\times \mathrm{diag}(1, e^{i \beta _2}, e^{i\beta _3}), \end{aligned}$$
(4.1)

where, for example, \(c_{12} = \cos \theta _{12}\) and \(s_{12} = \sin \theta _{12}\). There is no limit on the phases \(\beta _2\), \(\beta _3\) and \(\delta \) at the \(3\sigma \) level. The \(3\sigma \) limits on the other parameters are as follows [6, 7]:

(4.2)

Using these limits, we find \(3\sigma \) limits on the absolute values of the elements of the PMNS matrix. For normal hierarchy (NH), the limits are as follows:

$$\begin{aligned} |U^2| = \left( \begin{array}{ccc} 0.630\,\, \hbox {to}\,\, 0.736 &{}\quad 0.244\,\, \hbox {to}\,\, 0.347 &{}\quad 0.0185\,\, \hbox {to}\,\, 0.0246 \\ 0.0432\,\, \hbox {to}\,\, 0.299 &{}\quad 0.180\,\, \hbox {to}\,\, 0.532 &{}\quad 0.370\,\, \hbox {to}\,\, 0.604 \\ 0.0403\,\, \hbox {to}\,\, 0.295 &{}\quad 0.180\,\, \hbox {to}\,\, 0.530 &{}\quad 0.375\,\, \hbox {to}\,\, 0.609 \end{array}\right) , \nonumber \\ \end{aligned}$$
(4.3a)

whereas, for the inverted hierarchy (IH), the limits are slightly different:

$$\begin{aligned} |U^2| = \left( \begin{array}{ccc} 0.630 \text{ to } 0.736 &{}\quad 0.244 \text{ to } 0.347 &{}\quad 0.0186 \text{ to } 0.0248 \\ 0.0389 \text{ to } 0.298 &{}\quad 0.168 \text{ to } 0.529 &{}\quad 0.374 \text{ to } 0.625 \\ 0.0409 \text{ to } 0.302 &{}\quad 0.182 \text{ to } 0.546 &{}\quad 0.354 \text{ to } 0.605 \end{array}\right) . \nonumber \\ \end{aligned}$$
(4.3b)

It should be noted that the ranges indicated here do not pertain to values of the matrix elements of \(U^2\). Rather, each entry denotes the range of the range of modulus squared of an element of the PMNS matrix U. These are the ranges that we will use for checking the feasibility of getting a particular Coxeter group.

In the approach that we are going to take, all elements of the PMNS matrix cannot be found in general. We will discuss, depending on a particular choice of the group, how many elements of the PMNS matrix can be predicted, and will check how they fare in the light of experimental data.

In this pursuit, we will use Eq. (3.13). In order to avoid double counting and unnecessary work, it is useful to keep the following guidelines in mind.

  1. 1.

    The solution \(k_{\ell \alpha }=0\) is not allowed for any \(p_{\ell \alpha }\), because it gives \(|U_{\ell \alpha }|^2=1\), which is not allowed for any element of the PMNS matrix.

  2. 2.

    If \(p_{\ell \alpha }\) is even, the value \(k_{\ell \alpha }=\frac{1}{2} p_{\ell \alpha }\) is not allowed as well, because it gives \(U_{\ell \alpha }=0\), which is unacceptable for any element of the PMNS matrix.

  3. 3.

    Values of \(k_{\ell \alpha }\) with

    $$\begin{aligned} k_{\ell \alpha } > \frac{1}{2} p_{\ell \alpha } \end{aligned}$$
    (4.4)

    do not produce any new value of \(|U_{\ell \alpha }|^2\) that is not already encountered with smaller values of \(k_{\ell \alpha }\). Hence, these are irrelevant for our search.

  4. 4.

    Since only the ratio of \(k_{\ell \alpha }\) and \(p_{\ell \alpha }\) appears in Eq. (3.13), any common factor in the two numbers is irrelevant.

Combining these guidelines, we see that we only need to check for the values

$$\begin{aligned} 0< k_{\ell \alpha } < \frac{1}{2} p_{\ell \alpha }, \end{aligned}$$
(4.5)

with

$$\begin{aligned} \gcd (k_{\ell \alpha }, p_{\ell \alpha }) = 1. \end{aligned}$$
(4.6)

For \(p_{\ell \alpha } \le 5\), we present the result of these checks. We have

(4.7)

Fortunately, the availability of the solutions is not sensitive to the small differences of the allowed values that appear in Eq. (4.3a) and in Eq. (4.3b), so our subsequent analysis apply equally well for both hierarchies.

If the number of generators of the group is more than two, there is a different kind of relation that we will need to satisfy. To understand the point, let us assume that we have a group with two \(S'\)-type and one T-type involution generators. There will be two different combinations \(W_{\ell \alpha }\), and therefore two elements of the same column will be determined through Eq. (3.13). However, it is important to notice that, using Eqs. (2.10) and (2.14), and the unitarity of the PMNS matrix U, we can write

$$\begin{aligned} S'_1 S'_2 S'_3 = 1, \end{aligned}$$
(4.8)

which means that even the third \(S'\)-type matrix is also an element of the group. Instead of taking \(S'_1\) and \(S'_2\), say, as the generators, we could have also chosen \(S'_1\) and \(S'_3\), along with the T-type generator. If we had done that, the modulus of the third element of the column of the PMNS matrix would also have been determined by a relation of the form given in Eq. (3.13). The unitarity condition on the three elements of the same column would have then ensured that

$$\begin{aligned} \sum _{\alpha =1,2,3} \cos ^2 \Big ( \pi {k_{\ell \alpha } \over p_{\ell \alpha }} \Big ) = 1. \end{aligned}$$
(4.9)

Similarly, if we had considered a group with two T-type and one \(S'\)-type involution generators, we would have obtained

$$\begin{aligned} \sum _{\ell =e,\mu ,\tau } \cos ^2 \Big ( \pi {k_{\ell \alpha } \over p_{\ell \alpha }} \Big ) = 1. \end{aligned}$$
(4.10)

Both kinds of equations are of the same form,

$$\begin{aligned} \cos ^2 \Big ( \frac{\pi n_1}{N} \Big ) + \cos ^2 \Big ( \frac{\pi n_2}{N} \Big ) + \cos ^2 \Big (\frac{\pi n_3}{N} \Big ) = 1, \end{aligned}$$
(4.11)

with a suitably defined N, which can be the LCM of the numbers \(p_{\ell \alpha }\). If we consider a group with four generators, both Eqs. (4.9) and (4.10) will apply, and therefore there will be six equations of the form given in Eq. (4.11) that need to be satisfied, one for each row and for each column.

There are trivial solutions to Eq. (4.11) in which at least one of the cosines is zero. For example, if N is even, we have solutions in which one of the \(n_i\)’s is equal to zero and the other two equal to N / 2, i.e., one of the cosine-squared values is equal to 1 and the other zero. Or we can have solutions like

$$\begin{aligned} n_1 + n_2 = \frac{1}{2} N, \qquad n_3 = \frac{1}{2} N, \end{aligned}$$
(4.12)

along with permutations of the set of numbers \(n_1\), \(n_2\) and \(n_3\). Such solutions will not be useful for us, because they would imply zeros as elements of the PMNS matrix, which are untenable by experimental results. We need to find solutions of Eq. (4.11) where each of the cosines is non-zero. It can be analytically shown that [24, 25], subject to the guidelines summarized in Eqs. (4.5) and (4.6), the only solutions of Eq. (4.11) are these:

$$\begin{aligned} \begin{array}{cll} \hline N &{} \{n_1, n_2, n_3\} &{} \hbox {Values of }|U_{\ell \alpha }|^2 \\ \hline 12 &{} \{3, 4, 4 \} &{} \left\{ \frac{1}{2}, \frac{1}{4}, \frac{1}{4} \right\} \\ 15 &{} \{3, 5, 6 \} &{} \left\{ \frac{1}{8}(3+\surd 5), \frac{1}{4}, \frac{1}{8}(3-\surd 5) \right\} \\ \hline \end{array} \end{aligned}$$
(4.13)

For each choice of N, we have also given, in the last column, the mod-squared values of the entries of the PMNS matrix in the row or column for which that value of N applies.

One interesting point to note is that one cannot obtain the TBM form if the generators of the flavor group are involutions. The reason is that Eq. (3.13) shows that the absolute-squared values of each element must be of the form of the cosine squared of an angle which is a rational multiple of \(\pi \), and the TBM form contains absolute-squared values equal to \(\frac{1}{3}\), \(\frac{1}{6}\) etc. which are not. Of course, one can obtain TBM form when not all T-type and S-type generators are taken as involutions [10, 31,32,33,34,35,36,37]. This issue of arbitrariness in the choice of generators will be elaborated in Sect. 6.1. The BM matrix can be obtained with involution generators, but we will not consider it further since it contains a zero element, just as the TBM does.

5 Requirement of irreducible representations

Experimental data show that none of the elements of U is zero. This fact has an important implication on the T and \(S'\) generators, as we show now.

In the basis in which the T generators are diagonal, the relation between the \(S'\) generators and the PMNS matrix can be read from Eq. (2.14):

$$\begin{aligned} \Big ( S'_\alpha \Big )_{\ell \ell '} = {\left\{ \begin{array}{ll} 2 U_{\ell \alpha } U^*_{\ell '\alpha } - 1 &{}\quad \hbox {if }\ell =\ell ', \\ 2 U_{\ell \alpha } U^*_{\ell '\alpha } &{}\quad \hbox {otherwise}, \end{array}\right. } \end{aligned}$$
(5.1)

using the S generators given in Eq. (2.9). This shows that the off-diagonal elements of \(S'_\alpha \) cannot be zero, and therefore \(S'_\alpha \) cannot be block diagonal. Conversely, if \(S'_\alpha \) has to be block diagonal, some of its off-diagonal elements must vanish, requiring some elements of U to vanish. Thus, the statement that all elements of U are non-zero is equivalent to the statement that the \(S'\) generators are not block diagonal in the representation in which the T generators are. Therefore, the representation comprising the T and \(S'\) generators has to be an irreducible representation (irrep) in order that all elements of the PMNS matrix are non-zero. Since we are dealing with three generations of fermions, it means that the flavor group must have 3-dimensional irreps.

The point can be made in another way [38]. Suppose we take a set of matrices \(M_I\) and try to find a matrix M that commutes with each matrix in the set. Obviously, the unit matrix, or any multiple of it, will be solutions to the problem. If there is no other solution, then by Schur’s theorem the matrices are irreducible. If we only have two different T’s of Eq. (3.4) in the set \(M_I\), then it is straightforward to show that the general solution for M is a diagonal matrix. If now we also put one of the \(S'\) matrices in the collection \(M_I\), then M can only be a multiple of the identity matrix provided \(S'\) has no zero element. Hence, with two T-type and one \(S'\)-type generators, we need 3-dimensional irreps. The same is true if we have to generators of the \(S'\)-type and one of the T-type, and the proof is the same if we change over to the basis in which the \(S'\) generators are diagonal.

If, however, we have only one T-type and one \(S'\)-type generators, that is not the case. Without loss of generality, let us say that the set \(M_I\) contains the T-type generator \(T_e\). The most general M that commutes with it has the form

$$\begin{aligned} M = \left( \begin{array}{ccc} a &{}\quad 0 &{}\quad 0 \\ 0 &{}\quad m_{22} &{}\quad m_{23} \\ 0 &{}\quad m_{32} &{}\quad m_{33} \end{array} \right) . \end{aligned}$$
(5.2)

Suppose now we put the requirement that this M should commute with \(S'_1\). That would require

$$\begin{aligned} m'_{12} = m'_{13} = m'_{21} = m'_{31} = 0, \end{aligned}$$
(5.3)

where these are elements of the matrix \(M' = U^\dagger M U\). The four zeros will give four homogeneous equations for the quantities \(m_{22}-a\), \(m_{23}\), \(m_{32}\) and \(m_{33}-a\). Using the unitarity of the PMNS matrix U, one can easily show that the determinant of the co-efficient matrix vanishes, implying that non-zero solutions are possible. This means that the 3-dimensional matrices are reducible, implying that a block-diagonal solution for \(S'\) can be obtained. That would give zero elements in U, as expected from Eq. (5.1). In fact, later when we discuss groups with two generators, we show explicitly that those are groups which do not have any 3-dimensional irreducible representation.

Thus, if we encounter a group that has only 1- and 2-dimensional irreps, it is useless for us. A group that admits a 3-dimensional irrep is certainly fine. If a group does not have 3-dimensional irrep but has higher dimensional irreps, it also cannot be ruled out. The reason is the following. We are going to follow the Coxeter diagrams, which imply the number of generators and some relations between them, as exemplified in Eq. (6.4). Thus, the groups that we will find will be the ones which are consistent with some conditions on the generators. Suppose a given set of conditions on a fixed number of generators specifies a group G if we assume that there are no other condition connecting the generators. If we impose an extra condition, we will still obtain a group H which will be a subgroup of G. Even if G does not have a 3-dimensional irrep, it is possible that H does, and maybe this subgroup is responsible for the structure of the PMNS matrix. Thus, a flavor group can be acceptable provided it has irreps of dimension 3 or more.

It is to be understood that this constraint has nothing to do with the specific choice of the flavor group. Eq. (5.1) holds irrespective of the underlying group structure: it just says that, in a basis in which the T-generators are diagonal, the matrix U diagonalizes the matrices \(S'_\alpha \). Thus, this constraint has to be obeyed in any model of this form, irrespective of the T- and \(S'\)-type generators. The condition that is crucial for this conclusion is the absence of zeros in the PMNS matrix, as we have mentioned at the beginning of this section.

We want to emphasize that our discussion pertains to the case where the flavor symmetries are exact. If the symmetry is broken, explicitly or spontaneously, then symmetry breaking terms can contribute to the PMNS matrix as well and give a form that is not block diagonal. There are discussions of such scenarios in the literature [39]. We do not discuss this possibility.

6 Finite Coxeter groups

6.1 Coxeter groups and Coxeter diagrams

In Sect. 3, we initiated the discussion of groups generated by involutions. One subclass of such groups are called Coxeter groups, whose definition includes one more condition: the entire group can be specified once one knows the order of binary products of the generators. The generic presentation of Coxeter groups is therefore of the form

$$\begin{aligned} \left\langle r_i | (r_ir_j)^{q_{ij}} \right\rangle , \end{aligned}$$
(6.1)

where i and j run from 1 up to the number of generators. Because the order of each generator is 2, we have the further constraint that

$$\begin{aligned} q_{ii} = 1 \qquad \forall i. \end{aligned}$$
(6.2)

(Once again recall that we are nowhere using implied summation on repeated indices.) It can also be proved easily that

$$\begin{aligned} q_{ij} = q_{ji}. \end{aligned}$$
(6.3)

In order to specify a particular Coxeter group, one therefore needs the following pieces of information:

  1. 1.

    The number of generators.

  2. 2.

    The orders of binary products of the form \(r_ir_j\) with \(i<j\).

If no power of the product of a particular pair of generators is equal to the identity element, the corresponding \(q_{ij}\) is taken to be infinity.

Coxeter diagrams constitute a pictorial way for depicting Coxeter groups. In a Coxeter diagram, each generator is depicted by a blob. If \(q_{ij}=3\) for some particular value of i and j, then the \(i \mathrm{th}\) and \(j \mathrm{th}\) blobs are joined by a line. If \(q_{ij}>3\), there is still a line, but the value of \(q_{ij}\) is written above or below the line. If \(q_{ij}=2\), there is no line joining the dots. For example, the diagram would imply that the presentation of the Coxeter group is

$$\begin{aligned} \left\langle r_1,r_2,r_3 \Big | r_1^2, r_2^2, r_3^2, (r_1r_2)^3, (r_2r_3)^3, (r_1r_3)^2 \right\rangle , \end{aligned}$$
(6.4)

where on the left we give a list of the generators and on the right we give the combinations which are equal to the identity element.

It has to be commented that the presentation of a group is not unique. For example, one can start with the elements \(r_1\), \(r_2\) and \(r_3'=r_2r_3\) to generate the whole group given in Eq. (6.4). In this case, the generator \(r_3'\) would be an element of order 3, and the presentation of the group will be

$$\begin{aligned} \left\langle r_1,r_2,r_3' \Big | r_1^2, r_2^2, (r_2r_3')^2, (r_1r_2)^3, r_3'^3, (r_1r_2r_3')^2 \right\rangle . \end{aligned}$$
(6.5)

It will be hard to guess from this presentation that it is a Coxeter group. However, it must be, since the group is the same as that in Eq. (6.4). Even the number of generators for the same group might be different in two different presentations. In our discussion, in order to avoid such confusions, we will always talk about Coxeter groups with the involution generators, and use the minimum number of such generators necessary to write the presentation. Thus, when we talk about a group with two generators, we mean a group with two involution generators unless something to the contrary is explicitly stated.

Fig. 1
figure 1

Classification of irreducible finite Coxeter groups through Coxeter diagrams. More Coxeter groups can be obtained by taking products of the groups given here. We have not included the diagrams of \(E_6\), \(E_7\) and \(E_8\), all of which have more than four generators and are therefore irrelevant for us. Our nomenclature of the groups follows Ref. [40]. In each case, the subscript denotes the number of generators in the presentation of the group. Products of the groups shown here qualify as Coxeter groups as well

Full size image

In the present context, we will consider only Coxeter groups with two or three or four generators because, according to the restrictions put forth in Eq. (2.8), we can have at most two generators of the S type and two of the T type. Apart from a few exceptional groups, all finite Coxeter groups with connected Coxeter diagrams fall into one or another of four infinite series, called \(A_N\), \(B_N\), \(D_N\) and \(I_2(p)\) where the subscripts denote the number of generators. In Fig. 1, we have shown the Coxeter diagrams for all four series, and all exceptional groups whose number of generators is less than or equal to four. Although the notations for the diagrams is more or less universal, the names of the groups are not. There are several conventions, and we have followed the one of Ref. [40].

As a passing comment, note that the Coxeter groups \(A_N\) are really permutation groups. Any permutation involving n objects can be generated by combining permutations which are transpositions of two adjacent elements. Thus, the group \(S_n\) can be generated by transpositions like \(\tau _{1,2}\), \(\tau _{2,3}\) and so on, up to \(\tau _{n-1,n}\). It is also easy to see that products of any two transposition will have order 2 if there are no common objects, and order 3 if there is one common object. Comparing with the diagram of the \(A_N\) groups, we therefore see that

$$\begin{aligned} S_n = A_{n-1}. \end{aligned}$$
(6.6)

From Fig. 1, we see that the only irreducible finite Coxeter groups whose number of generators is between two and four are the following:

$$\begin{aligned}&A_2, A_3, A_4; \end{aligned}$$
(6.7a)
$$\begin{aligned}&B_2, B_3, B_4; \end{aligned}$$
(6.7b)
$$\begin{aligned}&D_2, D_3, D_4;\end{aligned}$$
(6.7c)
$$\begin{aligned}&I_2(p) \qquad \hbox {with }p=3,4,\ldots ;\end{aligned}$$
(6.7d)
$$\begin{aligned}&F_4, H_3, H_4. \end{aligned}$$
(6.7e)

Some groups appear more than once in this list. For example, a look at Fig. 1 suggests the following equivalences:

$$\begin{aligned} I_2(3)= & {} D_2 = A_2, \end{aligned}$$
(6.8a)
$$\begin{aligned} D_3= & {} A_3, \end{aligned}$$
(6.8b)
$$\begin{aligned} I_2(4)= & {} B_2. \end{aligned}$$
(6.8c)

We will keep these coincidences in mind while identifying distinct groups and discuss whether they give experimentally acceptable result for the elements of the PMNS matrix. In doing so, a few general points should be kept in mind, which we summarize here.

Given any Coxeter diagram, we have to discuss the possibility of assigning each of its dots to a generator of the type \(S'_\alpha \) or of the form \(T_\ell \). For this purpose, we need to remember an important result that can be proved easily: if \(q_{ij}=2\) in Eq. (6.1) for some specific values of i and j, then \(r_i\) and \(r_j\) commute. Now, note that the \(S'_\alpha \)’s commute with one another. Thus, if we assign the generator \(S'_1\) to one of the dots of a Coxeter diagram, any other dot connected to it cannot be assigned to another \(S'\) generator: it will have to be a T generator. For the same reason, two connected dots cannot be assigned to two T generators.

6.2 Groups with two generators

From the discussion at the end of Sect. 6.1, it is clear that if we look for a group with two generators, one of these generators must be \(S'_\alpha \) for some value of \(\alpha \), and the other would be \(T_\ell \) for some \(\ell \). Therefore, we will be able to determine only one element of the PMNS matrix, viz., the element \(U_{\ell \alpha }\).

The number of elements predicted might be disappointingly low, but there is no reason to be unhappy about it, because we now argue that none of these groups is viable so long as we use only the involution generators. Looking at Eq. (6.8), we see that we need to consider only \(I_2(p)\) groups, because all other groups with two generators is isomorphic to either \(I_2(3)\) or \(I_2(4)\). However, the \(I_2(p)\) groups are dihedral groups, i.e., symmetry groups of regular polygons, as is obvious from the presentation implied by the Coxeter diagrams: \(\left\langle a,b | a^2, b^2, (ab)^p \right\rangle \). It is known that the dihedral groups have only 1= or 2-dimensional irreducible representations. Thus, they are unacceptable for us.

6.3 Groups with three generators

With three generators, Fig. 1 shows us that we have the groups \(A_3\), \(B_3\) and \(H_3\). The diagrams of these three groups look the same: three dots joined by two links. They differ only in the order of one link. Recalling that we cannot assign two \(S'\) generators or two T generators to the ends of any link, we must alternate the two types of generators. Thus, we will either get the generators to be of \(S'TS'\) type, with whatever indices, or of the type \(TS'T\). In the first case, we will be able to determine the absolute values of two elements belonging to the same row of the PMNS matrix, whereas in the second case we will be able to do the same for two elements belonging to the same column. In either case, we will be able to tell the remaining element of the same column or row, as the case may be, by using a suitable unitarity relation. According to the discussion of Sect. 4, this extra value thus obtained must also be of the form given in Eq. (3.13), i.e., should be a solution of an equation of the form Eq. (4.9) or Eq. (4.10).

Because we obtain a full row or a full column, it is not necessary to check whether the group has any 3-dimensional irrep. If the only irreps are 1 and 2-dimensional, each row and each column must have at least one element equal to zero. Thus, if we just determine one row or one column and do not find a zero in it, it guarantees that the representation is not reducible. On the other hand, if we obtain at least one zero, it does not guarantee that the representation is reducible, but it does say that the group should be discarded because none of the entries in Eq. (4.3) is consistent with zero.

Table 1 Allowed generators for the group \(A_3\) (Note: This is the Coxeter nomenclature. This group is more commonly known as \(S_4\), and the GAP identifier of this group is [24,12])
Full size table

The group \(A_3\): In this case, both links have order 3. Looking at Eq. (4.7), we see that the choice of k is unique. For each possible choice of the generators, we give the list of matrix element whose absolute values are determined through Eq. (3.6), and both these values should be \(\frac{1}{4}\) as shown in Eq. (4.7). Unitarity of the mixing matrix would dictate that the absolute square of the remaining element of the same row or same column should be equal to \(\frac{1}{2}\), as discussed above. We look for all combinations of generators and list the acceptable ones in Table 1.

This group is isomorphic to the permutation group \(S_4\), as pointed out in Eq. (6.6), and has been encountered by various authors earlier. However, this does not necessarily mean that their analysis is same as that of ours, or that they obtain the same values of the PMNS elements as we do. As we commented before, the presentation of any group is not unique. Some authors [31, 41, 42], while considering the group \(S_4\) in models inspired by grand unified theories, did not take all the generators of \(S_4\) to be involutions and obtained different results. In contrast, we consider only involution generators. There are also computer searches [23] of finite groups with three generators. We will discuss their results shortly.

Table 2 Allowed generators for the group \(B_3\) (Note: This is the Coxeter nomenclature. The group has GAP identifier \([{ 48,48}]\) and is isomorphic to \(A_3 \times Z_2\) or \(S_4 \times Z_2\) [43].)
Full size table
Table 3 Allowed generators for the group \(H_3\) (Note: This group has GAP identifier [120,35])
Full size table

The group \(B_3\): Here, one link has order 4 and one has order 3. As in Fig. 1, we take the left link to have order 4. Because of this link of order 4, this time we will have to distinguish which generator corresponds to the right dot and which one to the left, something that was not important for the previous case. We list all allowed possibilities in Table 2. This group appeared in the discussion of Ref. [44], but only as a factor in the semi-direct product \((Z_2 \times Z_2 \times Z_2) \rtimes B_3\).

The group \(H_3\): As in Fig. 1, we take the left link to have order 5. The right link then has order 3. The modulus-squared value of the matrix element corresponding to the right link is \(\frac{1}{4}\), according to Eq. (4.7). The value coming from the left link can be either \(\frac{1}{8}(3+\surd 5)=0.6545\) or \(\frac{1}{8}(3-\surd 5)=0.0955\), depending on the value of \(k_{\ell \alpha }\). In either case, the modulus-squared values of the three elements in one row or one column should be \(\frac{1}{8}(3+\surd 5)\), \(\frac{1}{8}(3-\surd 5)\) and \(\frac{1}{4}\), in whatever order. There is only one element in Eq. (4.3) that can be as big as 0.6545, and that is \(|U_{e1}|^2\). In the first row, there is no choice of generators which can give rise to those values in Table 3. The first column, however, is acceptable, and we show the choice of generators which can give rise to those values. This group is isomorphic to direct product of alternating group \(\mathcal{A}_5\) (not to be confused with the Coxeter group \(A_5\)) and \(Z_2\).

Discussion of previous work: Fixing a column or a row of the PMNS matrix by considering three involution generators was considered by Lavoura and Ludl [23]. They made computer searches in the GAP [45] database. This database lists finite groups, including their properties and representations. Each group is denoted by two numbers in the form [a, b]. The first number is the cardinality of the group, and the second one is an arbitrary serial number assigned to groups of equal cardinality. These GAP identifiers have been mentioned in the captions of the tables for the groups discussed above.

For \(N=12\) solution of Eq. (4.13), the smallest group found by Lavoura and Ludl is \( [2,4 12]\) which is nothing but the \(A_3\) group. We also got the same solution for the \(B_3\) group whose cardinality is 48. This is a bigger group containing the \(A_3\) group. By the definition of the group \(B_3\),

$$\begin{aligned} (T_\mu S'_2)^4 = (T_\tau S'_2)^3 =1, \end{aligned}$$
(6.9)

which helps us determine two elements of the second column of the PMNS matrix. For the remaining element of the same column, we can deduce

$$\begin{aligned} (T_e S'_2)^3= & {} (S'_2 T_e)^3, \end{aligned}$$
(6.10)

by using relations like those in Eq. (3.5) and

$$\begin{aligned} T_\tau S'_2 T_\tau= & {} S'_2 T_\tau S'_2, \nonumber \\ T_\mu S'_2 T_\mu= & {} S'_2 T_\mu S'_2 T_\mu S'_2, \end{aligned}$$
(6.11)

which follow from Eq. (6.9). From this, it is trivial to show that the sixth power of \(T_e S'_2\) or of \(S'_2 T_e\) is equal to the identity element. Thus, when one chooses the lowest value, i.e., 3, for \(p_{e2}\), the order of \(T_eS'_2\), then because of an extra relation, instead of the \(B_3\) group a subgroup of that group is really being considered. And that subgroup is \(A_3\). Thus \(B_3\) has twice as many elements as \(A_3\). In the GAP [45] database, this group is called [48, 48].

For the \(N=15\) case, the smallest group they found has SmallGroup Id as [60, 5] whereas we found \(H_3\), whose cardinality is 120 and GAP [45] database Id [120, 35], as the smallest group with the \(N=15\) solution. This is because we consider only Coxeter groups, and [60, 5], a subgroup of \(H_3\), is not a Coxeter group. Lavoura and Ludl [23] found some bigger groups as well, but we suspect that those solutions involve non-involution generators.

6.4 Groups with four generators

From Fig. 1, we see that the finite groups with four generators are \(A_4\), \(B_4\), \(D_4\), \(F_4\) and \(H_4\). Let us give the final result first: none of these groups is allowed. We explain the reasons in what follows.

Let us discuss \(D_4\) first. Its Coxeter diagram has one nodal blob, from which three links come out to meet the three other blobs. If we assign an \(S'\) generator to the nodal blob, we must assign T-type generators to all other blobs. However, there are only two T-type independent generators, so this group is untenable. Even if we venture to put three different T-type generators on the three blobs, unitarity condition is not fulfilled since we get \(|U_{\ell \alpha }|^2 = \frac{1}{4}\) for all elements in a column. The argument with a T-type generator as the middle blob is the same, and need not be repeated.

For all other groups the list, the Coxeter diagrams are linear. Because the \(S'\)-type and T-type generators alternate on a line, one of the two extreme blobs must correspond to an \(S'\)-type generator and the other one to a T-type generator. There is no line joining these two blobs, which means that there is a \(W_{\ell \alpha }\) which is of order 2. But \(p_{\ell \alpha }=2\) does not give any acceptable solution, as noted in Eq. (3.13).

Let us now ignore the constraint that all the calculated PMNS elements must comply with the experimentally observed values within a \(3\sigma \) limit and examine whether any of these groups can even satisfy the unitarity conditions. There is at least one pair of T and \(S'\) generators which are not connected by a link, i.e., the corresponding \(W_{\ell \alpha }\) has \(p_{\ell \alpha }=2\). Therefore, one entry of the calculated PMNS matrix must be zero or unity, depending on the value of \(k_{\ell \alpha }\). If \(k_{\ell \alpha }=0\) and therefore \(|U_{\ell \alpha }|^2=1\) for one element, the other two in the same row and in the same column must be zero, and so there will be one eigenstate that will not mix with the other two. On the contrary, if \(k_{\ell \alpha }=1\), that particular element vanishes, and so the sum of absolute-squared values of two other elements of the PMNS matrix in the same row must be unity. This is true also for the column containing this zero entry.

For the \(A_4\) group, \(p_{\ell \alpha }=3\) for all the links and \(|U_{\ell \alpha }|^2=\frac{1}{4}\). Thus two such entries cannot add up to 1, and one obtains a contradiction since the matrix should be unitary. The group \(H_4\) has \(p_{\ell \alpha }=3,5\). There are no two allowed values of \(|U_{\ell \alpha }|^2\), corresponding to these values of \(p_{\ell \alpha }\), which add up to 1, and hence the fate of this group is the same. For the groups \(B_4\) and \(F_4\), one link has values of \(p_{\ell \alpha }=4\), which can give \(|U_{\ell \alpha }|^2=\frac{1}{2}\), but then the other two have \(p_{\ell \alpha }=3\), which can produce a maximum of \(\frac{1}{4}\). Thus we shall get at most \(\frac{3}{4}\) as the required sum. So we see that none of the 4-generator groups can satisfy the constraints of unitarity: we meet a contradiction.

6.5 Direct product groups

So far, the groups we have discussed may be called irreducible Coxeter groups. We can also consider reducible ones, i.e., groups which are direct products of more than one Coxeter groups. If first we look at groups with four generators, we have the following options:

$$\begin{aligned} \begin{array}{ll} \text{(a) } &{} A_1 \times X_3, \\ \text{(b) } &{} \Big ( A_1 \Big )^2 \times I_2(p), \\ \text{(c) } &{} \Big ( A_1 \Big )^4, \\ \text{(d) } &{} I_2(p)\times I_2(p'), \end{array} \end{aligned}$$
(6.12)

where \(X_3\) can be any Coxeter group with three connected generators. For any of the choices of \(X_3\), the Coxeter diagram will consist of one isolated blob, and the other three arranged in a manner shown in Fig. 1. Suppose the isolated one represents a T-type generator. Then it is not connected with any of the \(S'\) generators, implying that two elements in a row are zero. If the isolated one is an \(S'\) generator, then two elements in a column are zero. None of these options is acceptable. The next two options will also have at least one generator which is not connected to any other generator, and therefore gives zeros in the PMNS matrix.

If we take the last option of Eq. (6.12), then the Coxeter diagram will have two disconnected parts, each with two blobs. Each disconnected part will have to contain one T-type and one \(S'\)-type generator. Here again, each T blob will not be linked with one \(S'\) blob that is in the other connected part, and therefore there should be two zeros in the PMNS matrix.

Groups with two or three generators will have the same problem with disconnected parts in the Coxeter diagram, and are untenable.

7 More involution groups

We see that, among the finite Coxeter groups, the only acceptable solutions have three generators, which give only one column or one row of the PMNS matrix. Since we could not find any group with four generators, we cannot determine the entire PMNS matrix. Because of this, we now look for other options. We recall that our analysis is based on the relations derived in Sects. 3 and 4, especially on the formula for PMNS matrix elements in Eq. (3.13) and its correspondence with experimental data in Eq. (4.7). It should be noticed that none of these key formulas depend on the Coxeter nature of the group. In fact, the definition of Coxeter groups was not even given until Sect. 6. Rather, these formulas depend on the fact that all generators of the group are involutions. Therefore, we take this opportunity to explore other groups which are generated by involutions, but whose presentation is not of the form given in Eq. (6.1), and which therefore do not qualify as Coxeter groups.

To this end, we employ the following strategy. We start with Eq. (4.11) which also depends only on the fact that the group is generated by involutions. Irrespective of the experimental data, we first analyze how many solutions to this equation can be obtained, and we outline some criteria for selecting a subset of them. Then we make computer searches for these solutions, using four involution generators.

First, let us look at the solutions of Eq. (4.11). There can be various solutions which involve vanishing elements of the PMNS matrix. We disregard them and enumerate only the solutions where none of the PMNS elements is zero.

  1. 1.

    The solution with \(N=12\) presented in Eq. (4.13) can be used for each row and each column, producing the mod-squared values of the elements of a unitary matrix:

    $$\begin{aligned} |U^2|_{12} = \left( \begin{array}{ccc} \frac{1}{2} &{}\quad \frac{1}{4} &{}\quad \frac{1}{4}\\ \frac{1}{4} &{}\quad \frac{1}{2} &{}\quad \frac{1}{4} \\ \frac{1}{4} &{}\quad \frac{1}{4} &{}\quad \frac{1}{2} \end{array} \right) . \end{aligned}$$
    (7.1a)

    Of course it will have to be understood that variations can be obtained by reshuffling rows or columns.

  2. 2.

    One can also use the solution with \(N=15\) from Eq. (4.13) to construct the following values of the mod-squared values of the elements:

    $$\begin{aligned} |U^2|_{15} = \left( \begin{array}{ccc} \frac{1}{8}(3+\surd 5) &{}\quad \frac{1}{4} &{}\quad \frac{1}{8}(3-\surd 5)\\ \frac{1}{8}(3-\surd 5) &{}\quad \frac{1}{8}(3+\surd 5) &{}\quad \frac{1}{4} \\ \frac{1}{4} &{}\quad \frac{1}{8}(3-\surd 5) &{}\quad \frac{1}{8}(3+\surd 5) \end{array} \right) .\nonumber \\ \end{aligned}$$
    (7.1b)

    As before, reshuffling of rows or columns is allowed.

  3. 3.

    There can also be solutions where some rows will come from the \(N=12\) solution and some from the \(N=15\) solution. Of course, the latter kind must come in pairs because the occurrence of the irrational value of either \(\cos ^2(\pi /5)\) or \(\cos ^2(2\pi /5)\) cannot be consistent with the unitarity condition without the occurrence of the other. Thus, the solution is

    $$ \begin{aligned} |U^2|_{12 \& 15} = \left( \begin{array}{ccc} \frac{1}{8}(3+\surd 5) &{}\quad \frac{1}{4} &{}\quad \frac{1}{8}(3-\surd 5)\\ \frac{1}{8}(3-\surd 5) &{}\quad \frac{1}{4} &{}\quad \frac{1}{8}(3+\surd 5)\\ \frac{1}{4} &{}\quad \frac{1}{2} &{}\quad \frac{1}{4} \end{array} \right) . \end{aligned}$$
    (7.1c)

    Here also, reshuffling of rows or columns produces acceptable alternatives.

Note that none of the matrices shown in Eq. (7.1) is consistent with the experimental data for all elements of the PMNS matrix. The agreement is very bad for the form shown in Eq. (7.1b), where no row or no column is fully consistent with the data. The form of Eq. (7.1a) can be consistent with the data for the lower two rows of the PMNS matrix. In the top row, it misses one element by a small amount, and does not agree for the other two elements within the \(3\sigma \) limits. The solution of Eq. (7.1c) is definitely consistent with the values of five elements, and among the remaining ones, it misses one very narrowly. With little to choose from between these two forms, we perform the search for all three forms, showing that even these forms are very difficult to obtain.

The search is made by using GAP [45]. We take four involution generators, since that will determine the absolute values of all elements of the PMNS matrix. For the sake of definiteness, let us call these four generators \(T_e\), \(T_\mu \), \(S'_1\), \(S'_2\). We feed a presentation involving these generators of the following form and obtain the group:

$$\begin{aligned} \left\langle T_e, T_\mu , S'_1, S'_2 \right.&\Big | T_e^2, T_\mu ^2, S_1'^2, S_2'^2, (T_e T_\mu )^2, (S_1' S_2')^2, \nonumber \\&(T_e S_1')^{P_{e1}}, (T_e S_2')^{P_{e2}}, (T_e S_3')^{P_{e3}}, \nonumber \\&(T_\mu S_1')^{P_{\mu 1}}, (T_\mu S_2')^{P_{\mu 2}}, (T_\mu S_3')^{P_{\mu 3}}, \nonumber \\&\left. (T_\tau S_1')^{P_{\tau 1}}, (T_\tau S_2')^{P_{\tau 2}}, (T_\tau S_3')^{P_{\tau 3}} \right\rangle , \end{aligned}$$
(7.2)

where \(T_\tau = T_e T_\mu \) and \(S_3' = S_1' S_2'\). It should be noted that \(T_\tau \) and \(S'_3\) are not generators of the group: these notations have been introduced only for the sake of brevity.

The first four conditions of Eq. (7.2) indicate that all generators are involutions. The next two conditions state that \(T_e\) commutes with \(T_\mu \) whereas \(S_1'\) commutes with \(S_2'\). These six conditions of the presentation should be the same for all searches.

Each of the other conditions announces the order of one group element. If we had only the orders of the elements \(T_e S_1'\), \(T_e S_2'\), \(T_\mu S_1'\) and \(T_\mu S_2'\), and if these orders had corresponded to the orders obtainable from the Coxeter diagrams of Fig. 1 with four generators, the resulting group would have been a finite Coxeter group. However, we exclude that possibility by making some exceptions, as we explain now.

Table 4 Result of the search described in the text. For each kind of PMNS matrix elements, we give the possible finite subgroup of a Coxeter group. The numbers given under the columns marked “cardinality” and “serial Id”, taken together, form what has been called the SmallGroup Id in the GAP internet archives [45]
Full size table

Suppose, for the sake of definiteness, that we are trying to see how one might obtain the PMNS matrix of the form shown in Eq. (7.1a). In each row, one of the elements is equal to \(\frac{1}{2}\), which must come from \(k/p=\frac{1}{4}\). Previously, we used Eq. (4.6) to conclude that therefore these elements should have order \(p=4\). But now we allow for multiples of 4. As long as we keep the same value of k / p, we can obtain the same value of \(|U_{\ell \alpha }|^2\). For this reason, we write the order of the element with a uppercase P in Eq. (7.2), indicating that k and p need not be relatively prime. In our search, we allow for a GCD up to 3 for each pair of k and p.

Even after making that adjustment, if we restrict the presentation table of Eq. (7.2) only to the first six declarations and the orders of \(T_e S_1'\), \(T_e S_2'\), \(T_\mu S_1'\) and \(T_\mu S_2'\) only, we would obtain a Coxeter group because of the definition given in Eq. (6.1). The group would be infinite, because it would not correspond to any of the diagrams of Fig. 1. But we have more conditions. We take these other conditions in such a way that they provide us with an acceptable value of \(|U_{\ell \alpha }|^2\) that can satisfy Eq. (4.11).

Let us explain the strategy in some more detail. Suppose we want to obtain the PMNS matrix of the form given in Eq. (7.1a), with the top left element equal to \(\frac{1}{2}\). As said before, it needs \(k/p=\frac{1}{4}\). Since k is an integer, it means that p must be a multiple of 4, so we denote it by \(4a_1\), where \(a_1\) is an integer. Similarly, since the other elements of this row should come from \(k/p=\frac{1}{3}\), we can denote them by \(3a_2\) and \(3a_3\), where \(a_2\) and \(a_3\) are integers. Proceeding in this fashion, we obtain the following forms for the orders \(P_{\ell \alpha }\) of the group elements responsible for different elements of the PMNS matrix:

$$\begin{aligned} \left[ \begin{array}{ccc} P_{e1} &{}\quad P_{e2} &{}\quad P_{e3} \\ P_{\mu 1} &{}\quad P_{\mu 2} &{}\quad P_{\mu 3} \\ P_{\tau 1} &{}\quad P_{\tau 2} &{}\quad P_{\tau 3} \\ \end{array} \right] _{\mathrm{Eq.}\, (7.1\mathrm{a})} = \left[ \begin{array}{ccc} 4a_1 &{}\quad 3a_2 &{}\quad 3a_3 \\ 3b_1 &{}\quad 4b_2 &{}\quad 3b_3 \\ 3c_1 &{}\quad 3c_2 &{}\quad 4c_3 \\ \end{array} \right] , \nonumber \\ \end{aligned}$$
(7.3a)
$$\begin{aligned} \left[ \begin{array}{ccc} P_{e1} &{}\quad P_{e2} &{}\quad P_{e3} \\ P_{\mu 1} &{}\quad P_{\mu 2} &{}\quad P_{\mu 3} \\ P_{\tau 1} &{}\quad P_{\tau 2} &{}\quad P_{\tau 3} \\ \end{array} \right] _{\mathrm{Eq.}\, (7.1\mathrm{b})} = \left[ \begin{array}{ccc} 5a_1 &{}\quad 3a_2 &{}\quad 5a_3 \\ 5b_1 &{}\quad 5b_2 &{}\quad 3b_3 \\ 3c_1 &{}\quad 5c_2 &{}\quad 5c_3 \\ \end{array} \right] ,\nonumber \\ \end{aligned}$$
(7.3b)
$$\begin{aligned} \left[ \begin{array}{ccc} P_{e1} &{}\quad P_{e2} &{}\quad P_{e3} \\ P_{\mu 1} &{}\quad P_{\mu 2} &{}\quad P_{\mu 3} \\ P_{\tau 1} &{}\quad P_{\tau 2} &{}\quad P_{\tau 3} \\ \end{array} \right] _{\mathrm{Eq.} \,(7.1\mathrm{c})} = \left[ \begin{array}{ccc} 5a_1 &{}\quad 3a_2 &{}\quad 5a_3 \\ 5b_1 &{}\quad 3b_2 &{}\quad 5b_3 \\ 3c_1 &{}\quad 4c_2 &{}\quad 3c_3 \\ \end{array} \right] .\nonumber \\ \end{aligned}$$
(7.3c)

For each case, it has to be remembered that we allow for reshuffling of rows and columns. For a search in each category, we take the value of each of the integers to run from 1 to 3.

Because there are extra conditions now, we do not obtain a Coxeter group. Rather, we obtain a subgroup of a Coxeter group that will be obtained by deleting all conditions involving \(T_\tau =T_e T_\mu \) and \(S'_3=S'_1 S'_2\) from Eq. (7.2), which will be an infinite group because it does not appear in Fig. 1. Using GAP [45], we search whether the resulting group with any given choice of the integers \(a_1\) to \(c_3\) is finite. We limit our search to groups with number of elements between 9 and 1200. Smaller groups are not considered because they do not have any 3-dimensional irrep, and larger groups are probably less interesting because they are not very economical. Whenever we find a finite group with cardinality in this range, we announce it in Table 4. And finally, also from GAP, we find whether the group obtained has any 3-dimensional irreducible representation. The result is also shown in Table 4.

We see that, for groups with four involution generators and with cardinality up to 1200, we obtain no solution of the form advocated in Eq. (7.1b). For the form given in Eq. (7.1c), we find one solution in the same range. This group has the GAP Id [1080, 260], and is the \(\Sigma (360\times 3)\) group [22] as shown in Table 4. For the form given in Eq. (7.1a), we find some groups, but none of them has any 3-dimensional irrep. In fact, in desperation we continued the search somewhat further for this case and found that the next available finite group has 1728 elements, but this one also does not have any 3-dimensional irrep. We did not continue further.

8 Summary

We have considered all finite Coxeter groups whose number of generators range from two to four which appear in Fig. 1, as well as their direct products. We have identified the involution generators with residual symmetry generators in the low energy leptonic sector, and tried to see if they give an acceptable PMNS matrix. It would have been great if we could find a group with four generators to satisfy all known experimental constraints, because then we could have predicted the magnitudes of all elements of the PMNS matrix. Unfortunately, our analysis shows that such groups are not compatible with experimental results, in concurrence with results obtained by earlier authors who have considered many of the groups discussed in the present work. There are acceptable solutions with Coxeter groups with three generators, although these are only partial solutions. With a three generator group, we can calculate the magnitudes of elements in one row or in one column of the PMNS matrix. Such solutions have been shown in Sect. 6.3. For a group with two generators, only one element can be calculated. However, no such group is acceptable, for reasons described in Sect. 6.2.

We then searched for other groups that can be generated by four involution generators. We identify the very few types of PMNS matrices that can be generated from any involutionary group, and tried to see whether any involutionary group with four generators can provide that pattern. Within the limits in which we have performed the search, we have obtained a few such subgroups, which we have listed in Table 4. Some of these groups do not have any 3-dimensional irreducible representation and are therefore unacceptable. There is only one solution that is consistent with non-zero entries for each element, and has a 3-dimensional irrep as well.

One of our working assumptions was that the determinant of each generator is equal to 1, as announced in Eq. (2.7). For \(Z_2\) generators, the only other possibility is to have determinant equal to \(-1\). If we allow this possibility as well, the argument leading to Eq. (4.11) will be modified. For products of the form \(T_\ell S'_\alpha \) which will have a negative determinant, the cosines appearing in Eq. (4.11) will be replaced by sines. Using \(\sin x = \cos (\frac{1}{2}\pi -x)\), we can turn these equations into equations involving cosines again, just like Eq. (4.11). Thus we will not get any new solutions.