The growth of abelian sections

Abstract

Given an abstract group G, we study the function $ab_n(G) := \sup _{|G:H| \le n} |H/[H,H]|$. If G has no abelian composition factors, then $ab_n(G)$ is bounded by a polynomial: as a consequence, we find a sharp upper bound for the representation growth of these groups.

1 Introduction

Let G be an abstract group. There are many functions in group theory that describe G starting “from the top”, the most famous probably being the subgroup growth [14], which counts the number of subgroups of any given index. The main goal of this article is to study the abelian quotients of these subgroups, i.e., the function

$$\begin{aligned} ab_n(G) \> := \> \sup _{|G:H| \le n} |H/H'| \> , \end{aligned}$$

where $H':=[H,H]$ is the commutator subgroup of H. We call it the abelianization growth of G. As we will see, this is really a matter of finite groups, and it is quite related to the representation theory of G. Let us define $Rep_n(G)$ as the number of (pairwise inequivalent) irreducible complex G-representations of degree at most n and finite image. We will refer to this function as the representation growth of G. We remark that the usual definition of $Rep_n(G)$ involves representations with infinite image as well: for technical reasons, we will consider only finite images. Representation growth was introduced in the early 2000s, and was especially studied for arithmetic groups (we refer the reader to the survey [9]). On the other hand, the quantitative study of abelian sections has emerged recently. Perhaps the first question one can ask about $ab_n(G)$ is whether it really grows. With some effort made in Section 2, results from [16, 17] give the following tight bound (all the logarithms in this paper are to base 2):

Theorem 1

If G is any residually finite infinite group, then

$$\begin{aligned} ab_{n}(G) \> \ge \> n^{\> 1/32 \log \log n} \end{aligned}$$

for infinitely many n.

On the other hand, there exists a residually finite infinite group G such that

$$\begin{aligned} ab_{n}(G) \> \le \> n^{\> \beta /\log \log n} \end{aligned}$$

for some fixed $\beta >0$ and every $n \ge 3$.

Sections 3 and 4 concern both $ab_n(G)$ and $Rep_n(G)$. As it is a common observation that $ab_1(G) = Rep_1(G)$ when G is finite, it happens that $ab_n(G)$ and $Rep_n(G)$ are still related at larger scales. Indeed, [13, Lemma 2.6 (a)] (where $ab_n(G)$ appears in passing) provides

$$\begin{aligned} ab_n(G) \> \le \> n \cdot Rep_n(G) \end{aligned}$$

(1)

for every finite group G and $n \ge 1$. On the other hand, we show

Proposition 2

There exists an absolute constant $C>0$ such that if G is d-generated for some $d \ge 1$, then

$$\begin{aligned} Rep_n(G) \> \le \> (n! \cdot ab_{(n+1)!}(G))^{ \> Cd \> \cdot \> (\log ab_{(n+1)!}(G) \> + \> n \log n)} \end{aligned}$$

for every $n \ge 1$.

Moreover, we construct residually finite infinite groups with arbitrarily fast abelianization and representation growths (Proposition 23). We move to study more in detail the groups without abelian composition factors.

Theorem 3

There exists an absolute constant $\alpha >0$ such that the following holds. Let $C \ge 1$, $D \ge 0$. Let G be a group with D abelian composition factors, and suppose that all of them have order at most C. Then

$$\begin{aligned} ab_n(G) \> \le \> C^D \cdot n^\alpha \end{aligned}$$

for every $n \ge 1$.

This result depends on the classification of finite simple groups. One construction of Kassabov and Nikolov [8] provides a finitely generated residually finite group without abelian composition factors and representation growth of factorial type (see Example 18). Thus, $ab_n(G)$ and $Rep_n(G)$ can be quite different in general. In the range of d-generated cartesian products without abelian composition factors, they define admissible [8, Theorem 1.8] a representation growth which is bounded below a factorial. We prove that the same bound holds for all groups without abelian composition factors.

Theorem 4

There exists an absolute constant $\alpha '>0$ such that, if G is a d-generated group without abelian composition factors, then

$$\begin{aligned} Rep_n(G) \> \le \> (n!)^{\alpha ' d} \end{aligned}$$

for every $n \ge 1$.

The proof uses $ab_n(G) \le n^\alpha$, the Jordan theorem on finite subgroups of $GL_n(\mathbb {C})$, and that the number of boundedly generated groups of order n without abelian composition factors is polynomially bounded [6]. In Section 5, we generalize $ab_n(G)$ and $Rep_n(G)$ to transitive group actions, in place of the regular action of G on itself: while (1) resists in the generalized setting, the same is not true for any inequality in the direction of Proposition 2.

2 Basic properties of $ab_n(G)$

By definition, we have $ab_1(G)=|G/G'|$. We accept infinite values for $ab_n(G)$, and if G is finite, then $ab_{|G|}(G)$ is the size of the largest abelian section of G. One might wonder whether “$\sup$” is often unnecessary in the definition of $ab_n(G)$, and “$\max$” is acceptable. A satisfactory answer comes from the starting point of the theory of subgroup growth, i.e., the fact that a finitely generated group has finitely many subgroups of index n, for every fixed $n \ge 1$. More precisely, the following result, where $Sub_n(G)$ denotes the number of subgroups of index at most n in G, can be deduced from [14, Corollary 3.4].

Lemma 5

Let G be a d-generated group, for some $d \ge 1$. Then $Sub_n(G)$ is finite for every $n \ge 1$, and

$$\begin{aligned} Sub_n(G) \> \le \> (n!)^{d} \end{aligned}$$

for all $n \ge 4$. Moreover, if G is solvable, then

$$\begin{aligned} Sub_n(G) \> \le \> 4^{d n} \end{aligned}$$

is true for all sufficiently large n.

We develop the theory of abelianization growth in a general context and use the hypothesis of finite generation only when really needed.

Remark 6

If Z(G) denotes the center of G, then $(HZ(G))'=H'$ for every $H \leqslant G$. Thus, $ab_n(G)$ is determined by some subgroup which contains the center, for every $n \ge 1$.

The following are easily checked (slightly more general results will be proved in Sect. 5).

Lemma 7

(Hereditary properties) Let $H \leqslant G$ and $n \ge 1$. We have

• (i) $ab_n(H) \le ab_{|G:H|n}(G)$;

• (ii) $ab_n(G) \le |G:H| \cdot ab_n(H)$.

Moreover, if $N \lhd G$, then $ab_n(G/N) \le ab_n(G)$ for all $n \ge 1$.

2.1 Residually finite and profinite groups

Let $\mathcal {G}$ be a family of finite groups. We define

$$\begin{aligned} ab_n(\mathcal {G}) \> := \> \sup _{G \in \mathcal {G}} ab_n(G) \> . \end{aligned}$$

It is obvious that $ab_n(\mathcal {G}) \le ab_n(\mathcal {F})$ for all $n \ge 1$ if $\mathcal {G}$ and $\mathcal {F}$ are two families of finite groups and $\mathcal {G} \subseteq \mathcal {F}$. We recall a definition and introduce its finitary version.

Definition

A (typically infinite) group G has FAb (Finite Abelianizations) if $|H/H'|$ is finite for every subgroup $H \leqslant G$ of finite index.

A family of finite groups $\mathcal {G}$ has BAb (Bounded Abelianizations) if $ab_n(\mathcal {G})$ is finite for all $n \ge 1$.

We stress that the definition of FAb is slightly weaker than the property of having $ab_n(G)<\infty$ for every $n \ge 1$. Indeed, the two are equivalent when G is finitely generated. The following is useful to construct groups with FAb.

Lemma 8

If $(G_k)_{k \ge 1}$ is a sequence of finite perfect groups, then the Cartesian product $G := \prod _{k=1}^{\infty } G_k$ has FAb.

Proof

Let $H \leqslant G$ be a subgroup of finite index. Let $I_H$ be the set of indices k such that H intersects the direct factor $G_k$ properly. Of course, H is contained in $\prod _{k \notin I_H} G_k \times \prod _{k \in I_H} M_k$, where $M_k<G_k$ are maximal subgroups. Since H has finite index, $I_H$ is a finite set. Then, we use the fact that the $G_k$’s are perfect to write $|H/H'| \le \prod _{k \in I_H} |M_k| < \infty$. $\square$

The following group has FAb, but $ab_{60}(G)=\infty$.

Example 9

Let $p \ge 2$ be a prime, let $\mathbb {F}_p$ be the field with p elements, and consider the permutational wreath product $(\mathbb {F}_p)^5 \rtimes \mathop {\mathrm {Alt}}(5)$. The abelianization of this group has size p, and it is unbounded. We overcome this problem by considering the subspace of $(\mathbb {F}_p)^5$ which is constituted by the vectors whose coordinates sum to zero. This is a four-dimensional space, the so-called deleted permutation module, and let us denote it by $V_p$. We restrict the semidirect product to this subspace and define

$$\begin{aligned} G_p \> := \> V_p \rtimes \mathop {\mathrm {Alt}}(5) \>, \end{aligned}$$

where the action is the one described above. Let $G:=\prod _{p {\hbox {prime}}} G_p$ be the cartesian product of the $G_p$’s. Since $G_p$ is perfect for every p, G has FAb from Lemma 8. On the other hand, for every p, it is easy to find a subgroup of G of index $|\mathop {\mathrm {Alt}}(5)|=60$, whose abelianization is precisely $V_p$.

A stronger condition assures that $ab_n(G)$ is finite for every $n \ge 1$.

Lemma 10

Let $(G_k)_{k \ge 1}$ be a sequence of finite perfect groups with BAb. If $G:=\prod _{k=1}^{\infty } G_k$, then $ab_n(G)$ is finite for every $n \ge 1$.

Proof

Let $\mathcal {G}:=(G_k)_{k \ge 1}$, and let $H \leqslant G$ be a subgroup of finite index. Arguing as in the proof of Lemma 8, we see that $I_H$ is a finite set. Moreover, $2^{|I_H|} \le \prod _{k \in I_H} |G_k:M_k| \le |G:H|$, and so $|I_H| \le \log |G:H|$. Since the $G_k$’s are perfect, for every $n \ge 1$ we have

$$\begin{aligned} ab_n(G)&\> = \> \sup _{|G:H| \le n} |H/H'| \\&\> \le \> ( \sup _{G_k \in \mathcal {G}} ab_n(G_k) )^{|I_H|} \\&\> \le \> ( ab_n(\mathcal {G}) )^{\log n} \> , \end{aligned}$$

and the conclusion follows because $\mathcal {G}$ has BAb. $\square$

Now we see how $ab_n(G)$ is related to the finite quotients of G.

Proposition 11

(Reduction to finite groups) Let G be an arbitrary group, and let $\mathcal {G}$ be the family of its finite quotients. Then

$$\begin{aligned} ab_n(\mathcal {G}) \> \le \> ab_n(G) \end{aligned}$$

for all $n \ge 1$. Moreover, equality holds in the following cases:

• (i) G has FAb;

• (ii) G is finitely generated and residually finite (so, in this case $\mathcal {G}$ has BAb if and only if G has FAb).

Proof

From the last part of Lemma 7, for every $n \ge 1$,

$$\begin{aligned} ab_n(\mathcal {G}) = \sup _{G/N \in \mathcal {G}} ab_n(G/N) \le \sup _{G/N \in \mathcal {G}} ab_n(G) = ab_n(G) . \end{aligned}$$

Now let G have FAb, and let $H \leqslant G$ be a subgroup of finite index. It follows that $|G:H'|=|G:H||H:H'|$ is finite. If $N := \cap _{x \in G} (H')^x \leqslant H'$ is the normal core of $H'$ in G, we have $|G:N|<\infty$ and

$$\begin{aligned} |H/H'| = |(H/N):(H/N)'| , \quad \hbox {and} \quad |G:H| = |(G/N):(H/N)| . \end{aligned}$$

Since $H/N \leqslant G/N \in \mathcal {G}$, we obtain $ab_n(G) \le ab_n(\mathcal {G})$.

Finally, let G be finitely generated and residually finite. If G has FAb, then the conclusion follows from (i), so suppose that G does not have FAb. Since G is finitely generated, there are only finitely many subgroups for any given index. Then there exists a minimal integer, say m, such that G has a subgroup of index m with infinite abelianization. Hence, $ab_{m-1}(G) <\infty$, while $ab_m(G)=\infty$. For all $n<m$, we have $ab_n(\mathcal {G}) = ab_n(G)$ by the same argument we used above. It remains to prove that $ab_m(\mathcal {G})=\infty$. Let $H \leqslant G$ be a subgroup of index m and infinite abelianization. Since $H/H'$ is infinite and residually finite, there exists a subgroup $J/H' \leqslant H/H'$ of finite but arbitrarily large index |H : J|. Passing to the normal core $N:=\cap _{x \in G} J^x$, there exists a normal subgroup $N \lhd G$ inside J.

Consider $G/N \in \mathcal {G}$, and $H/N \leqslant G/N$ as a subgroup of index m. We have

$$\begin{aligned} |(H/N):(H/N)'| = |H:NH'| \ge |H:J| . \end{aligned}$$

Since |H : J| is arbitrarily large, the result follows. $\square$

Three remarks about Proposition 11:

• The hypotheses in (i) and (ii) are necessary. If $G \cong \mathbb {Q}$, the additive group of the rationals, then $\mathcal {G}=\{ 1 \}$, while $ab_n(G)=\infty$ for all $n \ge 1$;

• Essentially the same proof works if G is the inverse limit of the inverse system $\mathcal {G}$;

• If G satisfies one among (i)–(ii), then one can take the profinite completion ${\widetilde{G}}$ and conclude that $ab_n(G)=ab_n({\widetilde{G}})$ for all $n \ge 1$. However, we stress that the approach with families is more general, because not all the families of finite groups arise as finite quotients of an infinite mother group. As we will see with Proposition 24, not all the techniques of profinite groups work for families.

For the proof of Theorem 1, we will use a number of known results. We start with the following two.

Theorem 12

(Theorem 1 in [16]) Every finite group G of size at least 3 contains an abelian section of size at least $|G|^{1/32 \log \log |G|}$.

Theorem 13

(Theorem 1 in [17]) Let $\mathcal {G}:=(\mathop {\mathrm {Sym}}(k))_{k \ge 5}$. Then there exists an absolute constant $\alpha >0$ such that

$$\begin{aligned} ab_n(\mathcal {G}) \> \le \> n^{\> \alpha /\log \log n} \end{aligned}$$

for every $n \ge 3$.

From Lemma 7 (i), it is easy to see that Theorem 13 remains true with $\mathop {\mathrm {Alt}}(k)$ in place of $\mathop {\mathrm {Sym}}(k)$. The next result is very useful, especially when combined with Proposition 11.

Theorem 14

(Theorem 1.2 in [8]) If a cartesian product of alternating groups $\prod _{k=5}^{\infty } \mathop {\mathrm {Alt}}(k)^{f(k)}$ is topologically finitely generated, then it is the profinite completion of a finitely generated residually finite group.

We also need a numerical claim.

Lemma 15

If x and y are real numbers and $8 \le x \le y$, then

$$\begin{aligned} \frac{x}{\log x} + \frac{ y}{2 \log y} \> \le \> \frac{x+y}{\log (x+y)} \> . \end{aligned}$$

Proof

We have equality when $x=y=8$, and it is easy to check the case $x=y \ge 8$. Moreover, for every fixed x, the function $f(y) := \frac{x+y}{\log (x+y)} - \frac{ y}{2 \log y}$ is not decreasing. $\square$

Proof of Theorem 1

The lower bound for $ab_n(G)$ follows from Lemma 7 and Theorem 12. In fact, for every $N \lhd G$ of finite index one has

$$\begin{aligned} ab_{|G/N|}(G) \>\ge \> ab_{|G/N|}(G/N) \> \ge \> |G/N|^{1/32\log \log |G/N|} \> . \end{aligned}$$

If G is residually finite, then there exist normal subgroups of finite but arbitrarily large index.

To prove the second part, we construct a residually finite infinite group which attains the bound of the symmetric groups (this is not obvious, because there is no infinite group whose finite quotients are precisely the symmetric groups). Let $(a_j)_{j \ge 1}$ be a sequence of positive integers to be fixed later, and let $G := \prod _{j=1}^{\infty } \mathop {\mathrm {Alt}}(a_j)$. We remark immediately that G has FAb from Lemma 8. We will show that $ab_n(G) \le n^{\beta /\log \log n}$ for all $n \ge 3$. From Theorem 14, G is the profinite completion of a finitely generated residually finite group, and so, the proof would follow from Proposition 11. To estimate $ab_n(G)$, we work with its finite quotients. Indeed G is the inverse limit of $(G_k)_{k \ge 1}$, where $G_k := \prod _{j=1}^k \mathop {\mathrm {Alt}}(a_j)$ for all $k \ge 1$. From Proposition 11 (i), it is enough to prove that for every $k \ge 0$ and $H \leqslant G_{k+1}$ one has

$$\begin{aligned} \log |H/H'| \> \le \> \beta \cdot \frac{\log |G_{k+1}:H|}{\log \log |G_{k+1}:H|} \> , \end{aligned}$$

(2)

where $\beta :=2 \alpha$, and $\alpha >0$ is the constant in Theorem 13. We accomplish this task by induction on k. Arguing as in [17, Sect. 4], we can suppose $H=A \times B$ for some $A \leqslant G_k$ and $B \leqslant \mathop {\mathrm {Alt}}(a_{k+1})$. By the inductive hypothesis, we have

$$\begin{aligned} \log |H/H'|&\> = \> \log |A/A'| + \log |B/B'| \\&\> \le \> \beta \frac{\log |G_k:A|}{\log \log |G_k:A|} + \alpha \frac{\log (|\mathop {\mathrm {Alt}}(a_{k+1})|/|B|)}{\log \log (|\mathop {\mathrm {Alt}}(a_{k+1})|/|B|)} \> . \end{aligned}$$

Since $(\mathop {\mathrm {Alt}}(a_j))_{j \ge 1}$ has BAb, $(G_k)_{k \ge 1}$ has BAb from Lemma 10 and Proposition 11. Therefore, we can assume that $|G_k:A|$ is at least $2^8$ (otherwise, the first term is bounded by an absolute constant). Moreover, since $\mathop {\mathrm {Alt}}(a_{k+1})$ is perfect, we can assume $|\mathop {\mathrm {Alt}}(a_{k+1})|/|B| \ge a_{k+1}$. Let $x:=\log |G_k:A|$ and $y:=\log (|\mathop {\mathrm {Alt}}(a_{k+1})|/|B|)$. Now, if $(a_j)_{j \ge 1}$ is sufficiently fast, then the condition $8 \le x \le y$ is satisfied, and so, (2) follows from Lemma 15. $\square$

Here is a characterization of BAb. Although we will not use it, it can be interesting in itself.

Lemma 16

Let $\mathcal {G}$ be a family of finite groups. The following are equivalent:

• (i) For every normal subgroup $N \lhd G \in \mathcal {G}$, $|N/N'|$ can be bounded just from |G : N|;

• (ii) $\mathcal {G}$ has BAb;

• (iii) Every solvable section H/N of every $G \in \mathcal {G}$ has size bounded just from |G : H| and its derived length.

Proof

“(iii) $\Rightarrow$ (i)” is obvious.

“(i) $\Rightarrow$ (ii)” Let $H \leqslant G \in \mathcal {G}$, and let $N := \cap _{x \in G} H^x$ be its normal core. We have

$$\begin{aligned} |H/H'|&\> = \> |H:NH'| |NH':H'| \\&\> \le \> |G:N| |N:N \cap H'| \\&\> \le \> |G:N| |N:N'| \> , \end{aligned}$$

and the last term can be bounded from |G : N| by hypothesis. Since G/N is a permutation group over |G : H| elements, we have $|G/N| \le |G:H|!$.

“(ii) $\Rightarrow$ (iii)” We work by induction on the derived length $\ell$. The case $\ell =1$ is the definition of BAb, so suppose that H/N is a solvable section of $G \in \mathcal {G}$, and let $\ell +1$ be its derived length. We have $|H/N|=|H:NH'||NH':N|$, and $|H:NH'|$ can be bounded from |G : H|, because $|H:NH'| \le |H:H'|$. Moreover, $NH'/N \cong (H/N)'$ is a solvable section of derived length $\ell$, and so, by induction $|NH':N|$ can be bounded from $|G:NH'|$, and

$$\begin{aligned} |G:NH'| = |G:H||H:NH'| \le |G:H||H:H'| . \end{aligned}$$

Since this can be bounded from |G : H| by hypothesis, the proof follows. $\square$

The previous proof shows that the abelianization growth can be (crudely) estimated by a check on abelian quotients of normal subgroups. In particular, we remark that $|G/\cap _{x \in G} H^x| \le 3^{|G:H|}$ if $G/\cap _{x \in G} H^x$ is solvable [4].

3 Abelianization and representation growths

The counting of representation growth usually concerns a fixed infinite group G. The corresponding notion for a family of finite groups $\mathcal {G}$ is

$$\begin{aligned} Rep_n(\mathcal {G}) \> := \> \sup _{G \in \mathcal {G}} Rep_n(G) \> . \end{aligned}$$

If representations with infinite image are considered, the matter of representation growth cannot be reduced to the matter of finite groups, not even for a finitely generated residually finite group. With our restricted definition, we have

Lemma 17

Let G be an arbitrary group, and let $\mathcal {G}$ be the family of its finite quotients. Then

$$\begin{aligned} Rep_n(G) \> = \> Rep_n(\mathcal {G}) \end{aligned}$$

for every $n \ge 1$.

Proof

For every $N \lhd G$ of finite index, a representation of G/N can be seen as a representation of G with finite image. It follows that

$$\begin{aligned} Rep_n(\mathcal {G}) = \sup _{G/N \in \mathcal {G}} Rep_n(G/N) \le Rep_n(G) . \end{aligned}$$

To prove equality, we have to show that two different G-representations with finite image are already different in some finite quotient of G. Let $\pi , \rho \in Irr_n(G)$ for some $n \ge 1$, and let $\pi \ne \rho$. Since both $Ker(\pi )$ and $Ker(\rho )$ have finite index in G, then $N:=Ker(\pi ) \cap Ker(\rho )$ has finite index, and $\pi$ and $\rho$ are distinct representations of G/N. $\square$

Essentially, the next is taken from [8].

Example 18

For every $d \ge 3$ and $k \ge 5$, the direct product $(\mathop {\mathrm {Alt}}(k))^{(k!/2)^{d-2}}$ is d-generated [18, Lemma 2]. Thus, from Theorem 14 and the discussion below [8, Theorem 1.4] in that paper, there exists a $d'$-generated residually finite group G with profinite completion $\prod _{k=5}^{\infty } (\mathop {\mathrm {Alt}}(k))^{(k!/2)^{d-2}}$, where $d':=22(d+1)$. For every $k \ge 5$, G has at least $(k!/2)^{d-2}$ different representations of degree $k-1$, one for each factor in the corresponding direct product. It follows that

$$\begin{aligned} Rep_n(G) \> \ge \> ((n+1)!/2)^{d-2} \> \ge \> (n!)^{d-2} \ge (n!)^{d'/50} \end{aligned}$$

for every $n \ge 4$.

Now we recall two results in representation theory: the first is the famous Jordan theorem about the finite subgroups of $GL_n(\mathbb {C})$.

Theorem 19

(Jordan function) There exists a function $j: \mathbb {N}_+ \rightarrow \mathbb {N}_+$ such that the following hold:

• (i) if G is a finite subgroup of $GL_n(\mathbb {C})$, then G has an abelian normal subgroup of index at most j(n) [7];

• (ii) $j(n) \le (n+1)!$ for every $n \ge 71$ [2, Theorem A].

The second result is the representation growth version of Lemma 7, and it is a clever application of the Frobenius reciprocity law. We give our proof, because it will be useful in Sect. 5.

Lemma 20

(Lemma 2.2 in [13]) Let G be a finite group, $H \leqslant G$ and $n \ge 1$. We have

• (i) $Rep_n(H) \le |G:H| \cdot Rep_{n|G:H|}(G)$;

• (ii) $Rep_n(G) \le |G:H| \cdot Rep_n(H)$.

Proof

For every $n \ge 1$, let us denote by $Irr_n(G)$ the set of the irreducible G-representations (up to isomorphism) of degree at most n. We define a map $\phi : Irr_n(H) \rightarrow Irr_{n|G:H|}(G)$ in the following way: for every $\rho \in Irr_n(H)$, choose $\phi (\rho )$ as an irreducible constituent of the induced representation $\rho ^{\uparrow G}$. Whenever $\phi (\rho )$ is a constituent of $\pi ^{\uparrow G}$ for some $\pi \in Irr_n(H)$, by Frobenius reciprocity $\pi$ is a constituent of the restriction $\phi (\rho )_{\downarrow H}$. Fix $\rho \in Irr_n(H)$, and let $\pi$ be of minimal degree among the irreducible representations in $\phi (\rho )_{\downarrow H}$. We have

$$\begin{aligned} |\phi ^{-1}\phi (\rho )| \dim (\pi )&\> \le \> \dim (\phi (\rho )_{\downarrow H}) \\&\> = \> \dim (\phi (\rho )) \\&\> \le \> \dim (\pi ^{\uparrow G}) \\&\> = \> |G:H| \dim (\pi ) \> . \end{aligned}$$

Thus, the fibers of $\phi$ have size at most |G : H|, and (i) follows.

For (ii), similarly define a map $\varphi : Irr_n(G) \rightarrow Irr_n(H)$ in the following way: for every $\rho \in Irr_n(G)$, choose $\varphi (\rho )$ as an irreducible constituent of $\rho _{\downarrow H}$. Whenever $\varphi (\rho )$ is a constituent of $\pi _{\downarrow H}$ for some $\pi \in Irr_n(G)$, by Frobenius reciprocity $\pi$ is a constituent of $\varphi (\rho )^{\uparrow G}$. Fix $\rho \in Irr_n(G)$, and let $\pi$ be of minimal degree among the irreducible representations in $\varphi (\rho )^{\uparrow G}$ We have

$$\begin{aligned} |\varphi ^{-1}\varphi (\rho )| \dim (\pi )&\> \le \> \dim (\varphi (\rho )^{\uparrow G}) \\&\> = \> |G:H| \dim (\varphi (\rho )) \\&\> \le \> |G:H| \dim (\pi _{\downarrow H})\\&\> = \> |G:H| \dim (\pi ) \> , \end{aligned}$$

and the proof follows as before. We remark that (1) is obtained by setting $n=1$ in (i). $\square$

Proof of Proposition 2

Fix $n \ge 1$. From Theorem 19 (i), for every representation $\rho : G \rightarrow GL_n(\mathbb {C})$, the finite image $\rho (G)$ has an abelian subgroup of index at most j(n). Let $A_\rho \leqslant G$ be the pre-image of such an abelian subgroup. By the correspondence theorem we have $|G:A_\rho | = |\rho (G):\rho (A_\rho )| \le j(n)$. Moreover, by the abelianity of $\rho (A_\rho )$,

$$\begin{aligned} |\rho (A_\rho )| \le |A_\rho /(A_\rho )'| \le ab_{|G:A_\rho |}(G) \le ab_{j(n)}(G) , \end{aligned}$$

and then,

$$\begin{aligned} |\rho (G)| = |\rho (G):\rho (A_\rho )| \cdot |\rho (A_\rho )| \le j(n) \cdot ab_{j(n)}(G) . \end{aligned}$$

From the main result of Lubotzky [12], and Remark 1 in that paper, there exist at most $|\rho (G)|^{3d \cdot \log |\rho (G)|}$ abstract groups of order at least $|\rho (G)|$. For each of these abstract groups, there are at most $|\rho (G)|$ irreducible representations up to equivalence, the exact number being the number of conjugacy classes. Then, there are at most $|\rho (G)|$ possible embeddings into $GL_n(\mathbb {C})$ up to conjugation. The number of homomorphisms from G to one fixed embedding is at most $|\rho (G)|^d$. To sum up

$$\begin{aligned} Rep_n(G) \le |\rho (G)|^{3d \log |\rho (G)|+1+d} \le (j(n) \cdot ab_{j(n)}(G))^{ 4d \log ( j(n) \cdot ab_{j(n)}(G))} . \end{aligned}$$

When $n \le 70$, the proof follows by choosing the constant C in the statement sufficiently large. For $n \ge 71$, we use Theorem 19 (ii) and the trivial inequality $(n+1)! \le (n!)^2$. $\square$

We can improve Proposition 2 in some cases. A finite group is called monomial if each of its irreducible representations is induced from some one-dimensional representation of some subgroup. It is known that the class of monomial groups lies between those of nilpotent and solvable groups [5, Corollary 5.13] (originally attributed to Taketa).

Lemma 21

If G is a d-generated monomial group, then

$$\begin{aligned} Rep_n(G) \> \le \> 4^{d n} \cdot ab_n(G) \end{aligned}$$

for all sufficiently large n.

Proof

For all $n \ge 1$, every representation of degree at most n is induced by a one-dimensional representation of some subgroup $H \leqslant G$ of index at most n. This implies that

$$\begin{aligned} Rep_n(G) \le Sub_n(G) \cdot Rep_{_1}(H) \le Sub_n(G) \cdot ab_n(G) . \end{aligned}$$

As we said above, monomial groups are solvable, and so, the proof follows from Lemma 5. $\square$

Combining (1) and Lemma 21, we obtain

Corollary 22

Let $\mathcal {G}$ be a family of d-generated monomial groups, for some fixed $d \ge 1$. Then $ab_n(\mathcal {G})$ is at most exponential if and only if $Rep_n(\mathcal {G})$ is at most exponential.

3.1 Upper and lower bounds

In general, there is no upper bound for $ab_n(G)$ and $Rep_n(G)$. To see this, we modify the construction of Example 9.

Proposition 23

For every function $f: \mathbb {N}_+ \rightarrow \mathbb {N}_+$, there exists a finitely generated FAb residually finite group G such that $ab_n(G) \ge f(n)$ for infinitely many n. The same is true for $Rep_n(G)$.

Proof

The statement for $Rep_n(G)$ follows from that for $ab_n(G)$ and (1). We divide the proof for $ab_n(G)$ in three steps:

Step 1. Let $(p_k)_{k \ge 5}$ be a sequence of primes to be fixed later, and consider the permutational wreath product $(\mathbb {F}_{p_k})^k \rtimes \mathop {\mathrm {Alt}}(k)$. Let $V_k$ be the $(k-1)$-dimensional subspace of $(\mathbb {F}_{p_k})^k$ which is constituted by the vectors whose coordinates sum to zero. We restrict the semidirect product to this subspace and define

$$\begin{aligned} G_k \> := \> V_k \rtimes \mathop {\mathrm {Alt}}(k) \end{aligned}$$

for all $k \ge 5$. Let $G:=\prod _{k \ge 5} G_k$ be the cartesian product of the $G_k$’s.

Step 2. Since $G_k$ is perfect for every $k \ge 5$, G has FAb from Lemma 8 (actually, we could use Lemma 10, but this is not required). On the other hand, let $H := G_5 \times ... \times G_{k-1} \times V_k \leqslant G_k$. We have $|G:H|=k!/2$, and $|H/H'| = (p_k)^{k-1}$. Hence, for a sufficiently fast sequence of primes $(p_k)_{k \ge 5}$, we obtain $ab_{k!/2}(G) \ge f(k!/2)$ for every $k \ge 5$.

Step 3. By Theorem 14, $\prod _{k \ge 5} \mathop {\mathrm {Alt}}(k)$ is the profinite completion of a finitely generated residually finite group. If $v \in V_k$ is a non-trivial element, then $G_k$ is generated by $\mathop {\mathrm {Alt}}(k)$ and $[v,\mathop {\mathrm {Alt}}(k)]$, and so, [8, Lemma 2.4] gives that G itself is the profinite completion of a finitely generated residually finite group. The proof is completed by Proposition 11. $\square$

We conclude this section with some remarks about lower bounds. It follows from the definitions of $ab_n(\mathcal {G})$ and $Rep_n(\mathcal {G})$ that

$$\begin{aligned} ab_{|G|}(\mathcal {G}) \> \ge \> ab_{|G|}(G) \hbox { and } Rep_{|G|}(\mathcal {G}) \> \ge \> Rep_{|G|}(G) \end{aligned}$$

(3)

for every family of finite groups $\mathcal {G}$ and every $G \in \mathcal {G}$. Of course, $ab_{|G|}(G)$ is the size of the largest abelian section of G. From a standard result in representation theory, $Rep_{|G|}(G)$ equals the number of conjugacy classes of G, which we denote by k(G). The inequalities in (3) are not effective if $ab_n(\mathcal {G})$ or $Rep_n(\mathcal {G})$ are fast (in the extreme case, if $\mathcal {G}$ does not have BAb). However, we saw in the proof of Theorem 1 that the first inequality is surprisingly precise in giving a general lower bound (by “general” here we mean that it is true for every $\mathcal {G}$). Can one do the same for $Rep_n(\mathcal {G})$? The best known lower bound for k(G) is due to the companion Baumeister–Maroti–Tong-Viet [1], and it is the following: for every $\varepsilon >0$, there exists $C_\varepsilon >0$ such that

$$\begin{aligned} k(G) \> \ge \> C_\varepsilon \cdot \frac{\log |G|}{(\log \log |G|)^{3+\varepsilon }} \end{aligned}$$

(4)

for every finite group G of size at least 3. On the other side, there are arbitrarily large groups satisfying $k(G) \le C \cdot (\log |G| / \log \log |G|)^2$ (see [15, pag. 375]). When combined to (3), (4) yields the best lower bound, we can provide at the moment, for the representation growth of every family of arbitrarily large finite groups. It is worth noticing that Craven [3] used a different idea to obtain a better bound, which holds only for a fixed infinite profinite group G, but not for an arbitrary family of finite groups.

Proposition 24

(Proposition 2.2 in [3]) For every $\varepsilon >0$, there exists $C_\varepsilon >0$ such that the following holds. If G is an infinite, finitely generated profinite group with FAb, then

$$\begin{aligned} Rep_n(G) \> \ge \> C_\varepsilon \cdot (\log n)(\log \log n)^{1-\varepsilon } \end{aligned}$$

for infinitely many n.

In short, Craven’s proof distinguishes if G has finitely many or infinitely many maximal subgroups. In the first case, G is virtually pro-nilpotent, while in the second case it maps onto arbitrarily large finite groups with trivial Frattini subgroup. In both cases, the representation growth is as fast as claimed. As the second part of Theorem 1 points out, the same approach fails for $ab_n(G)$, essentially because of the relatively slow abelianization growth of alternating groups.

4 Polynomial abelianization growth

Definition

Let G be a group, and let $\mathcal {G}$ be a family of finite groups. We say that G ($\mathcal {G}$, respectively) has polynomial abelianization growth if there exist absolute constants $C,\alpha >0$ such that

$$\begin{aligned} ab_n(G) \> \le \> C n^\alpha \end{aligned}$$

for every $n \ge 1$ ($ab_n(\mathcal {G}) \le C n^\alpha$, respectively). The notion of polynomial representation growth is defined similarly.

We remark that (1) says that PRG implies PAG in every residually finite group. Here are some cases where PAG is realized.

Lemma 25

Let G be a finite group, and $\varepsilon >0$. The following hold:

• (i) if $|G/G'| \le \varepsilon ^{-1}$, and $|G:H| \ge |G|^\varepsilon$ for every proper subgroup $H < G$, then

  $$\begin{aligned} ab_n(G) \> \le \> \varepsilon ^{-1} \cdot n^{\varepsilon ^{-1} -1} \end{aligned}$$

  for every $n \ge 1$;

• (ii) if $\dim (\pi ) \ge |G|^\varepsilon$ for every non-trivial irreducible representation $\pi$ of G, then

  $$\begin{aligned} Rep_n(G) \> \le \> n^{\varepsilon ^{-1} -1} \end{aligned}$$

  for every $n \ge 1$.

Proof

(i) For every proper subgroup $H<G$, we have

$$\begin{aligned} |H| \le |G|^{1-\varepsilon } = |G:H|^{1-\varepsilon }|H|^{1-\varepsilon } , \end{aligned}$$

and arranging the terms we obtain $|H|^\varepsilon \le |G:H|^{1-\varepsilon }$, so that

$$\begin{aligned} |H/H'| \le |H| \le |G:H|^{\varepsilon ^{-1}-1} . \end{aligned}$$

Combined with $|G/G'| \le \varepsilon ^{-1}$, this gives the desired inequality.

(ii) By hypothesis, we have $Rep_n(G) = 1$ for every $n < |G|^\varepsilon$. Let $n \ge |G|^\varepsilon$, and let $r_n(G)$ be the number of (inequivalent) irreducible representations of degree exactly n. Since $|G| = \sum _{\pi \in Irr(G)} (\dim \pi )^2$, we have

$$\begin{aligned} r_n(G) \le |G|/n^2 \le n^{\varepsilon ^{-1} -2} . \end{aligned}$$

Finally

$$\begin{aligned} Rep_n(G)&\> = \> 1 + \sum _{j=\lceil |G|^\varepsilon \rceil }^n r_j(G) \\&\> \le \> 1 + (n-\lceil |G|^\varepsilon \rceil +1) n^{\varepsilon ^{-1} -2} \\&\> \le \> n^{\varepsilon ^{-1} -1} \> . \end{aligned}$$

$\square$

Sometimes the condition $\dim (\pi ) \ge |G|^\varepsilon$ for every non-trivial $\pi$ and fixed $\varepsilon >0$ is called $\varepsilon$- quasirandomness. It is a property shared by quasisimple groups of Lie type of bounded Lie rank [10], and it implies what is asked in Lemma 25 (i). On the other hand, Theorem 3 and Example 18 show that PAG does not imply PRG in general, not even for finitely generated groups. We are going to prove Theorem 3. When H is an arbitrary group and $N \lhd H$, one has

$$\begin{aligned} |H/H'|&\> = \> |H:NH'| \cdot |NH': H'| \\&\> = \> |(H/N):(H/N)'| \cdot |N : N \cap H'| \> , \end{aligned}$$

and so, $|H/H'| \le |(H/N):(H/N)'||N:N'|$, because $N' \leqslant N \cap H'$.

Lemma 26

(Product formula) Let $N \lhd G$. Then, for every $n \ge 1$,

$$\begin{aligned} ab_n (G) \> \le \> \max _{j \> | \> n} \> ( ab_{n/j} (G/N) \cdot ab_j (N) ) \> . \end{aligned}$$

In particular,

$$\begin{aligned} ab_n (G/N) \> \le \> ab_n (G) \> \le \> ab_n (G/N) \cdot ab_n (N) \end{aligned}$$

for all $n \ge 1$.

Proof

Let $n \ge 1$ and $H \leqslant G$ with $|G:H| \le n$. From the previous inequality, we have

$$\begin{aligned} |H/H'|&\> \le \> |(H/N \cap H):(H/N \cap H)'| \cdot |N \cap H : (N \cap H)'| \\&\> = \> |(NH/N):(NH/N)'| \cdot |N \cap H : (N \cap H)'| \> . \end{aligned}$$

Moreover,

$$\begin{aligned} |(G/N):(NH/N)| = |G:NH|, \hbox {and} |N: N \cap H| = |NH:H| . \end{aligned}$$

Let $j:=|NH:H|$. The three computations above combined give

$$\begin{aligned} |H/H'| \le ab_{|G:H|/j}(G/N) \cdot ab_{j}(N) . \end{aligned}$$

The proof follows. $\square$

We divide the proof of Theorem 3 in three steps:

• 1. Reduction to G finite. This is required to use the classification of non-abelian finite simple groups. Let G be a group with D abelian composition factors, which size is at most C. Let $H \leqslant G$ be a subgroup of finite index, and let $N:=\cap _{x \in G} H^x$ be its normal core. Arguing as in the proof of Lemma 16, we have

  $$\begin{aligned} |H/H'|&\> = \> |H:NH'| |NH':H'| \\&\> \le \> |G:N| |N:N \cap H'| \\&\> \le \> |G:H|! \cdot |N:N'| \> . \end{aligned}$$

  By the Jordan–Hölder theorem, we have $|N/N'| \le C^D$. Thus, G has FAb, and the reduction follows from Proposition 11 (i).

• 2. One simple group. Let S be a non-abelian finite simple group. This step follows from the work of Liebeck and Shalev [11] on character degrees of finite simple groups: they proved the following theorem about the zeta function

  $$\begin{aligned} \zeta _S(t) \> := \> \sum _{\pi \in Irr(S)} (\dim \pi )^{-t} \> , \end{aligned}$$

  where $\pi$ ranges among the (inequivalent) irreducible representations of S.

  Theorem 27

  (Theorem 1.1 in [11]) Let S be a non-abelian finite simple group. If $t>1$, then

  $$\begin{aligned} \zeta _S(t) \rightarrow 1 \hbox { as } |S| \rightarrow +\infty . \end{aligned}$$

  We remark that a weaker fact is needed to prove Theorem 3, namely that there exists $t_0$ such that $\zeta _S(t_0)$ is bounded when $|S| \rightarrow +\infty$. Now, it is easy to see that the zeta function is a sum involving representation growth. Let $r_n(S)$ be the number of (inequivalent) irreducible representations of degree exactly n. Arranging the terms, we can write

  $$\begin{aligned} \zeta _S(2) \> = \> \sum _{\pi \in Irr(S)} (\dim \pi )^{-2} \> = \> \sum _{n \ge 1} r_n(S) n^{-2} \> . \end{aligned}$$

  Since the series converges when |S| grows, there exists an absolute constant $c>0$ such that $r_n(S) \le c n^2$ for all n and S. Hence,

  $$\begin{aligned} Rep_n(S) \> = \> \sum _{j =1}^n r_j(S) \> \le \> c n^3 \> , \end{aligned}$$

  and combining this inequality with (1), we obtain

  $$\begin{aligned} ab_n (S) \> \le \> n \cdot Rep_n(S) \> \le \> cn^4 \> . \end{aligned}$$

  We point out that it would be very interesting to have a CFSG-free proof of the inequality $|H/H'| \le |G:H|^\alpha$ in non-abelian finite simple groups.

• 3. Mixing simple groups. Since S is perfect, we can stretch $cn^4$ to a monic polynomial $n^\alpha$. In fact, for $n = 2$ we have

  $$\begin{aligned} cn^4 = n^{4 + \log (c)} , \end{aligned}$$

  while “<” holds for all larger n. Thus, we can write $ab_n(S) \le n^\alpha$ for some fixed $\alpha >0$ and every $n \ge 1$. Now let G be a finite group with D abelian composition factors of size at most C. We work by induction on the composition length, so let $N \lhd G$ be a maximal normal subgroup. If the simple quotient G/N is non-abelian, then $ab_n(G/N) \le n^\alpha$ by the previous step, and $ab_n(N) \le C^D n^{\alpha }$ by induction. Using Lemma 26, we obtain

  $$\begin{aligned} ab_n(G) \le \max _{j \> | \> n} \> ((n/j)^\alpha C^D (j)^\alpha ) = C^D n^\alpha \end{aligned}$$

  for all $n \ge 1$. When some abelian quotient G/N appears, then by induction $ab_n(N) \le C^{D-1} n^{\alpha }$. Let $H \leqslant G$. Since $|NH/N| \le |G/N| \le C$, we can write

  $$\begin{aligned} |H/H'|&\> \le \> |(H/N \cap H):(H/N \cap H)'| \cdot |N \cap H : (N \cap H)'| \\&\> \le \> |NH/N| \cdot ab_{_{|N:N \cap H|}}(N) \\&\> \le \> C \cdot C^{D-1} |N:N \cap H|^\alpha \\&\> \le \> C^D |G:H|^\alpha . \end{aligned}$$

  This concludes the proof of Theorem 3.

Remark 28

It is not true that mixing (via group extensions) infinitely many groups with PAG, we always do obtain a group with PAG. Actually, we do if only finitely many of the starting groups are not perfect. Moreover, (1) and Proposition 2 imply that, for a family $\mathcal {G}$ of finite groups which are all d-generated for a fixed d, $ab_n(\mathcal {G})$ is finite for every $n \ge 1$ if and only if the same is true for $Rep_n(\mathcal {G})$. Let $G^k := G \times ... \times G$ be the direct product of k copies of a perfect group G, and let $\mathcal {G} := (G^k)_{k \ge 1}$. Then $\mathcal {G}$ has BAb, but certainly $Rep_{|G|}(\mathcal {G})$ is not finite. This shows that the dependence on d in Proposition 2 is necessary.

Characterizing groups with PAG (or even PRG) seems to be a serious task:

Example 29

(p-groups with PRG) Highlighting a nice connection, Lubotzky and Martin [13] proved that an arithmetic group has polynomial representation growth if and only if it has the so-called congruence subgroup property (informally, this means that all the finite index subgroups arise from the arithmetic structure of the group). This property is own, for example, by $SL_r(\mathbb {Z})$ when $r \ge 3$ (notice that $SL_2(\mathbb {Z})$ does not even have FAb, because of its free subgroups of finite index). Let $G:=SL_r(\mathbb {Z})$, and for fixed $p \ge 2$ and every $k \ge 1$ define

$$\begin{aligned} G_k \> := \> Ker \{ \> SL_r(\mathbb {Z}) \twoheadrightarrow SL_r(\mathbb {Z}/p^k \mathbb {Z}) \> \} \> . \end{aligned}$$

Now $G_1$ has finite index in G, and so it has polynomial representation growth from Lemma 20 (i). Moreover, it is not hard to check that $G_1/G_k$ is a p-group for every $k \ge 1$. It follows (from Lemma 17, for example) that $(G_1/G_k)_{k \ge 1}$ is a family of p-groups with polynomial representation growth.

To prove Theorem 14, we need a sharp bound on the number of d-generated groups without abelian composition factors. We quote [6, Corollary 13.4]:

Theorem 30

(Jaikin-Zapirain and Pyber) There exists a constant C such that the number of d-generated finite groups of order n without abelian composition factors is at most $n^{Cd}$.

It is easy to see that, up to replacing C, the same bound is true for the number of d-generated finite groups of order at most n without abelian composition factors.

Proof of Theorem 14

Arguing as in the proof of Proposition 2, we obtain an inequality of the type

$$\begin{aligned} Rep_n(G) \> \le \> \phi _d ( \> \> n! \cdot ab_{(n+1)!}(G) \> \> ) \cdot (n! \cdot ab_{(n+1)!}(G))^{d+1} \> , \end{aligned}$$

where $\phi _d(n)$ is the number of d-generated groups of order at most n and without abelian composition factors. From Theorem 3, we have $ab_n(G) \le n^\alpha$ for some fixed $\alpha >0$. Moreover, Theorem 30 gives $\phi _d(n) \le n^{Cd}$ for some absolute constant $C>0$. Up to replacing with a larger $\alpha '(\alpha ,C)$, the inequality

$$\begin{aligned} Rep_n(G) \le ( n! \cdot ((n+1)!)^\alpha )^{Cd+d+1} \le (n!)^{\alpha ' d} \end{aligned}$$

is true for every $n \ge 1$.

5 Group actions

Let the group G act transitively on the set $\Omega$. We count the irreducible representations which appear in the permutation representation, and measure the abelian sections of G above a stabilizer.

Definition

Let G be a group, and $Y \leqslant G$. For every $n \ge 1$, we define

$$\begin{aligned} ab_n(G,Y) \> := \> \sup _{\begin{array}{c} Y \leqslant H \leqslant G \\ |G:H| \le n \end{array}} |H/H'Y| \> , \end{aligned}$$

where H ranges among the subgroups between Y and G (notice that $H'Y \lhd H$). Moreover, we define $Rep_n(G,Y)$ as the number of (inequivalent) irreducible G-representations having a non-zero Y-fixed vector, degree at most n, and finite image.

We shall use $Irr_n(G,Y)$ to denote the set of the irreducible G-representations (up to isomorphism) having a non-zero Y-fixed vector, degree at most n, and finite image. Here is a simple connection with the matter of the previous sections.

Lemma 31

(Reduction to faithful actions) If $N \leqslant Y$ and $N \lhd G$, then for every $n \ge 1$

$$\begin{aligned} ab_n(G,Y) \> = \> ab_n(G/N,Y/N) , \hbox { and } Rep_n(G,Y) \> = \> Rep_n(G/N,Y/N) . \end{aligned}$$

Proof

Let us recall that the intermediate subgroups $N \leqslant H \leqslant G$ are in bijection (via the correspondence theorem) with the subgroups of the quotient G/N. Let $N \leqslant Y \leqslant H \leqslant G$. We have

$$\begin{aligned} |(H/N):(H/N)'(Y/N)| = |(H/N):(H'Y/N)| = |H/H'Y| . \end{aligned}$$

Moreover, $|(G/N):(H/N)|=|G:H|$, so the proof of the first part follows.

Now the induced representation $(1_Y)^{\uparrow G}$ is contained in $(1_N)^{\uparrow G}$, which is just the regular representation of G/N. So the representations in $Irr_n(G,Y)$ are representations of G/N. Moreover, $(1_Y)^{\uparrow G}$ is equivalent to $(1_{(Y/N)})^{\uparrow (G/N)}$ when seen in the quotient, and the proof follows. $\square$

The following is the generalized version of Proposition 11. Essentially the proof is the same, and we leave it to the reader.

Proposition 32

(Reduction to finite groups) Let G be an arbitrary group, let $\mathcal {G}$ be the family of its finite quotients, and let $Y \leqslant G$. Then

$$\begin{aligned} ab_n(G,Y) \> \ge \> \sup _{G/N \in \mathcal {G}} ab_n \left( \frac{G}{N} , \frac{YN}{N} \right) \end{aligned}$$

for every $n \ge 1$. Equality holds in the following cases:

• (i) $ab_n(G,Y)$ is finite for all $n \ge 1$;

• (ii) G is finitely generated and $\cap _{|G:H|<\infty } H \leqslant Y$.

To not borden unnecessarily the discussion, we do not introduce any notation of the type $ab_n(\mathcal {G})$ or $Rep_n(\mathcal {G})$ in the generalized setting. When G is residually finite and the index of Y is infinite, a remarkable difference with the case $Y \lhd G$ is that there are no non-trivial lower bounds for $ab_n(G,Y)$ and $Rep_n(G,Y)$ in general. This is precisely because there is no such a lower bound for the largest abelian section between Y and G, nor for the number of representations with a non-trivial Y-fixed vector.

Example 33

Let G be a 2-transitive permutation group, and let $Y \leqslant G$ be the stabilizer of a point. Then the permutation representation is composed by two irreducible G-representations, one being the trivial representation. Moreover, since Y is maximal and not normal, there is no non-trivial abelian section between Y and G. It follows that $ab_n(G,Y) \le 1$ and $Rep_n(G,Y) \le 2$ for all $n \ge 1$.

Lemma 34

Let G be residually finite, and let $|G:Y|=\infty$. Then $ab_n(G,Y)$ is bounded by an absolute constant if and only if $|N_G(H):H|$ is bounded by an absolute constant, for every $Y < H \leqslant G$ with $|G:H|<\infty$. In particular, $ab_n(G,Y)=1$ for all $n \ge 1$ if and only if all the subgroups of finite index containing Y are self-normalizing.

Proof

Let $\theta \ge 1$ such that $ab_n(G,Y) \le \theta$ for all $n \ge 1$. This means that every abelian section (of finite index) between Y and G has size at most $\theta$. Now, the size of a finite group is bounded by a function on the size of the largest abelian section (see Theorem 12, for example). Then the size of every section of finite index between Y and G is bounded by a function on $\theta$. But this means exactly that the set $\{|N_G(H):H| \}_{Y< H \leqslant G, |G:H|<\infty }$ is bounded by a function on $\theta$.

The proof of the second part follows, because every non-trivial finite group has a non-trivial abelian section. $\square$

The following are the generalized versions of Lemmas 7 and 20.

Lemma 35

(Relative $ab_n$ and subgroups) Let $Y \leqslant H \leqslant G$, $|G:H|<\infty$, and $n \ge 1$. We have

• (i) $ab_n(H,Y) \le ab_{|G:H|n}(G,Y)$;

• (ii) $ab_n(G,Y) \le |G:H| \cdot ab_n(H,Y)$.

Proof

Choose any $Y \leqslant J \leqslant H$ with $|H:J| \le n$. Since J is also a subgroup of G, and $|G:J|=|G:H||H:J|$, (i) follows.

To prove (ii), choose any $Y \leqslant J \leqslant G$ with $|G:J| \le n$. Since $Y \leqslant H \cap J$, we have

$$\begin{aligned} |J/J'Y|&\> = \> |J:J'(H \cap J)| \cdot |J'(H \cap J):J'Y| \\&\> = \> |J:J'(H \cap J)| \cdot |(J'Y)(H \cap J) : J'Y| \\&\>\le \> |J:H \cap J| \cdot |H \cap J : H \cap (J'Y)| \> . \end{aligned}$$

Now $(H \cap J)'Y \leqslant H \cap (J'Y)$, and so,

$$\begin{aligned} |J/J'Y| \le |G:H| \cdot |H \cap J : (H \cap J)'Y| . \end{aligned}$$

Since $|H:H \cap J| \le |G:J|$, the proof follows. $\square$

Lemma 36

(Relative $Rep_n$ and subgroups) Let G be a finite group, $Y \leqslant H \leqslant G$, and $n \ge 1$. We have

• (i) $Rep_n(H,Y) \le |G:H| \cdot Rep_{n|G:H|}(G,Y)$;

• (ii) $Rep_n(G,Y) \le |G:H| \cdot Rep_n(H,Y)$.

Proof

Let us recover the proof of Lemma 20. Thus, it rests to prove that the two functions $\phi : Irr_n(H) \rightarrow Irr_{n|G:H|}(G)$ and $\varphi :Irr_n(G) \rightarrow Irr(H)$, as defined in that proof, map into $Irr_{n|G:H|}(G,Y)$ and $Irr_n(H,Y)$ when restricted to $Irr_n(H,Y)$ and $Irr_n(G,Y)$, respectively.

For (i), let $\rho \in Irr_n(H,Y)$, and let $\phi (\rho )$ be an irreducible constituent of the induced representation $\rho ^{\uparrow G}$. By Frobenius reciprocity, $\rho$ is a constituent of the restriction $\phi (\rho )_{\downarrow H}$, and so, $\rho _{\downarrow Y}$ is contained in $\phi (\rho )_{\downarrow Y}$. But, again from Frobenius reciprocity, $\rho _{\downarrow Y}$ contains the trivial Y-representation. This implies that $\phi (\rho )_{\downarrow Y}$ contains the trivial Y-representation and, by a last application of Frobenius reciprocity, that $\phi (\rho )$ is a constituent of $(1_Y)^{\uparrow G}$ as desired.

For (ii), let $\rho \in Irr_n(G,Y)$. We remark that, to construct $\varphi$, we can choose $\varphi (\rho )$ to be any irreducible constituent of $\rho _{\downarrow H}$. We use this fact to force $\varphi (\rho )$ to be also an irreducible constituent of $(1_Y)^{\uparrow H}$. This can always be done, in fact, let $\chi :=trace(\rho )$ be the character of $\rho$. Let us use $(1_Y)^{\uparrow H}$ and $(1_Y)^{\uparrow G}$ to denote both the permutation representations and the correspondent characters. If $\langle \cdot , \cdot \rangle _G$ denotes the standard inner product, by Frobenius reciprocity we have

$$\begin{aligned} \langle \chi _{\downarrow H} , (1_Y)^{\uparrow H} \rangle _H = \langle \chi , ((1_Y)^{\uparrow H})^{\uparrow G} \rangle _G = \langle \chi , (1_Y)^{\uparrow G} \rangle _G \ge 1 , \end{aligned}$$

where the last inequality follows because $\rho \in Irr(G,Y)$ by hypothesis. Thus, $\rho _{\downarrow H}$ and $(1_Y)^{\uparrow H}$ share some irreducible components, and the proof is complete. $\square$

When compared to the strong relationship between $ab_n(G)$ and $Rep_n(G)$ (which is settled by (1) and Proposition 2), the one between $ab_n(G,Y)$ and $Rep_n(G,Y)$ is quite milder.

Proposition 37

Let $Y \leqslant G$. The following hold

• (i) $ab_1(G,Y) = Rep_1(G,Y)$;

• (ii) $ab_n(G,Y) \le n \cdot Rep_n(G,Y)$ for every $n \ge 1$.

Proof

It follows from the definition that $ab_1(G,Y)=|G/G'Y|$. Now we notice that

$$\begin{aligned} Rep_1(G,Y) = Rep_1 \left( \frac{G}{G'},\frac{G'Y}{G'} \right) , \end{aligned}$$

and since $(G'Y/G') \lhd (G/G')$, via Lemma 31,

$$\begin{aligned} Rep_1 \left( \frac{G}{G'},\frac{G'Y}{G'} \right) = Rep_1 \left( \frac{(G/G')}{(G'Y/G')} \right) = Rep_1(G/G'Y) = |G/G'Y| , \end{aligned}$$

because $G/G'Y$ is abelian.

Now let $n \ge 1$, and let $Y \leqslant H \leqslant G$ be any intermediate subgroup with $|G:H| \le n$. Lemma 36 (i) applied to $n=1$ provides

$$\begin{aligned} |H/H'Y|&\> = \> ab_1(H,Y) \\&\> = \> Rep_{_1}(H,Y) \\&\>\le \> |G:H| \cdot Rep_{_{|G:H|}}(G,Y) \> , \end{aligned}$$

and (ii) follows. $\square$

Therefore, a bound on $Rep_n(G,Y)$ gives a bound on $ab_n(G,Y)$. The converse is not true, highlighting the absence of a generalized version of Proposition 2 (in the proof, the size of the subgroup $\rho (A_\rho )$, which is provided by the Jordan theorem, cannot be bounded by a function on $ab_n(G,Y)$). Indeed, a robust notion of BAb does not exist in the generalized setting.

Example 38

Let $p \ge 3$ be a prime, and let $G_p:=D_{2p}$ be a dihedral group which acts on the p-gon $\Omega _p$. Let $Y_p \leqslant G_p$ be the stabilizer of a point. Since $Y_p$ is maximal and not normal, we have $ab_n(G_p,Y_p)=1$ for all $n \ge 1$. On the other hand, all the irreducible representations of a dihedral group have degree 1 or 2. It follows that $Rep_2(G_p,Y_p) = \frac{p-1}{2}$ can be arbitrarily large.