Hall classes of groups

Abstract

In 1958, Philip Hall (Ill J Math 2:787–801, 1958) proved that if a group G has a nilpotent normal subgroup N such that $G/N'$ is nilpotent, then G is nilpotent. The scope of Hall’s nilpotency criterion is not restricted to group theory, and in fact similar statements hold for Lie algebras and more generally for algebraically coherent semiabelian categories (see Chao in Math Z 103:40–42, 1968; Gray in Adv Math 349:911–919, 2019; Stitzinger in Ill J Math 22:499–505, 1978). We say that a group class ${\mathfrak {X}}$ is a Hall class if it contains every group G admitting a nilpotent normal subgroup N such that $G/N'$ belongs to ${\mathfrak {X}}$. Thus, Hall’s nilpotency criterion just asserts that nilpotent groups form a Hall class. Many other relevant classes of groups have been proved to be Hall classes; for example, Plotkin (Sov Math Dokl 2:471–474, 1961) and Robinson (Math Z 107:225–231, 1968) proved that locally nilpotent groups and hypercentral groups form Hall classes. Note that these generalizations also hold if groups are replaced by other algebraic structures, for example Lie algebras (see Stitzinger in Ill J Math 22:499–505, 1978). The aim of this paper is to develop a general theory of Hall classes of groups, that could later be reasonably extended to Lie algebras. Among other results, we prove that many natural types of generalized nilpotent groups form Hall classes, and we give examples showing in particular that the class of groups having a finite term in the lower central series is not a Hall class, even if we restrict to the universe of linear groups.

1 Introduction

Let G be a group and let N be a nilpotent normal subgroup of G. A classical result of Philip Hall [13] proves that if $G/N'$ is nilpotent, then G itself is nilpotent; a different proof of this result was given by Boris Plotkin [18], who also proved that, if $G/N'$ is locally nilpotent, then G is locally nilpotent, too.

The scope of Hall’s nilpotency criterion is not restricted to group theory: for example, Chao [4] proved that a similar theorem holds for Lie algebras, and Stitzinger [22] extended this result to almost alternative algebras. Of course, the theorem of Hall can be rephrased in terms of group epimorphisms; in fact, Gray [10] has recently obtained a generalization of Hall’s criterion for algebraically coherent semiabelian categories, and has pointed out that a statement of this type is false for the semiabelian category of non-associative rings. It turns out that tensor products play a relevant role in this kind of problems, and can be used to obtain a common proof of the above theorems; actually, this approach makes it possible to prove these and several further results of the same type for both groups and Lie algebras (see [19] and [22]). In this paper we focus on groups, but some of the methods can be extended, as usual, to Lie algebras (see for instance [16]).

In the wake of the above results, we say that a group class ${\mathfrak {X}}$ is a Hall class if it contains every group G admitting a nilpotent normal subgroup N such that $G/N'$ belongs to ${\mathfrak {X}}$. In particular, Hall’s nilpotency criterion just says that the class ${\mathfrak {N}}$ of all nilpotent groups is a Hall class and Plotkin’s theorem states that locally nilpotent groups form a Hall class L${\mathfrak {N}}$. Moreover, the tensor product method can be used to show that each of the following types of groups determine a Hall class: hypercentral, hypercyclic, (locally) supersoluble, (locally) polycyclic. More recently, it has been proved that this list can be extended by adding paranilpotent, (locally) FC-nilpotent and FC-hypercentral (see [1] and [9]).

On the other hand, there are also many relevant group classes which are not Hall classes, for example the class $\mathfrak {A}$ of all abelian groups; in fact, if a Hall class $\mathfrak {X}$ contains $\mathfrak {A}$, then it must contain all nilpotent groups.

Our aim is to develop a general theory of Hall classes of groups. The preliminary Sect. 2 contains examples and abstract results on Hall classes, such as sufficient conditions for a product of group classes to be a Hall class. In Sect. 3, we prove the following theorem (see there for the necessary definitions); it almost completes the list of generalized nilpotency properties determining Hall classes, the only relevant exception being the class of groups with all subgroups subnormal, for which the problem is still open.

Theorem A

The class consisting of all groups of one of the following 9 types is a Hall class: Engel, bounded Engel, Baer, strongly Baer, Gruenberg, strongly Gruenberg, Fitting, strongly Fitting, k-polynilpotent for a fixed k.

It is easy to see that also the class $\mathfrak {F}$ of finite groups and the class L$\mathfrak {F}$ of locally finite groups are Hall classes. Thus, if $\mathfrak {X}$ is a relevant Hall class made of generalized nilpotent (or soluble) groups, it is natural to ask whether the addition of a finite (or locally finite) obstruction either at the top or at the bottom of groups in $\mathfrak {X}$ still determines a Hall class. The case of a finite or locally finite obstruction at the top is essentially solved by the following general result, which is proved in Sect. 4.

Theorem B

Let $\mathfrak {X}$ be the class consisting of all groups of one of the following 15 types: nilpotent, hypercentral, Engel, bounded Engel, Baer, strongly Baer, Gruenberg, strongly Gruenberg, Fitting, strongly Fitting, locally nilpotent, paranilpotent, hypercyclic, locally supersoluble, k-polynilpotent for a fixed k. Then the product $\mathfrak {XY}$ is a Hall class for any group class $\mathfrak {Y}$.

In the above statement, $\mathfrak {X}$ cannot be chosen to be the class of all supersoluble groups. In fact, let p be a prime number, $R_p$ the ring of all rational numbers whose denominators are powers of p and $G={\text {Tr}}_1(3,R_p)$ the group of all lower unitriangular matrices of degree 3 over $R_p$; then G is nilpotent and $G/G'$ is (finitely generated)-by-periodic, although G is not even polycyclic-by-periodic, because any finitely generated normal subgroup of G is contained in $G'=\zeta _1(G)$. It follows that supersoluble-by-(locally finite) groups do not form Hall classes, as well as polycyclic-by-(locally finite) groups and (finitely generated nilpotent)-by-(locally finite) groups. On the other hand, if $\mathfrak {Y}=\mathfrak {F}$, we can add supersoluble, polycyclic and finitely generated nilpotent groups to the list of possible choices for $\mathfrak {X}$, just because the class ${\text {Max}}$ of all groups satisfying the maximal condition is a Hall class.

In contrast to the fact that finite and locally finite obstructions at the top essentially behave similarly, there is a dichotomy between finite and locally finite obstructions at the bottom. In fact, it is proved in [6] that $({{\textbf {L}}}\mathfrak {F})\mathfrak {X}$ is a Hall class for most Hall classes $\mathfrak {X}$ consisting of generalized nilpotent or soluble groups. On the other hand, our next main result, which is also proved in Sect. 4, shows that the situation of a finite obstruction at the bottom is much worse.

Theorem C

Let $\mathfrak {K}$ be an infinite field of positive characteristic q containing a finite subfield of order $>2$.

• (a) There exists a subgroup G of ${\text {GL}}(3,\mathfrak {K})$ which is not finite-by-(locally nilpotent) but does contain a unipotent normal subgroup N such that $G/N'$ is finite-by-abelian.

• (b) If $q=2$, there exists a subgroup G of ${\text {GL}}(5,\mathfrak {K})$ which is not finite-by-(locally soluble) but does contain a unipotent normal subgroup N such that $G/N'$ is finite-by-abelian.

Although these examples are linear, it turns out that, if G is a linear group of characteristic 0 having a nilpotent normal subgroup N such that $G/N'$ is finite-by-nilpotent, then G itself is finite-by-nilpotent. A detailed study of finite obstructions at the bottom for Hall classes within the universe of linear groups has been made in [7]. Note also that we can give a version of Theorem C for (affine) algebraic groups (see Theorem C$^\#$ in Sect. 4).

The final Sect. 5 deals with finite obstructions at the bottom for Hall classes consisting of groups satisfying the maximal condition. In particular, we prove the following result.

Theorem D

The class consisting of all groups of one of the following 4 types is a Hall class: finite-by-(nilpotent and finitely generated), finite-by-supersoluble, locally (finite-by-nilpotent), locally (finite-by-supersoluble).

Our notation, which is mostly standard, can be found in [21]; we refer to [23] for results and terminology concerning linear groups. In particular, a group class $\mathfrak {X}$ is a class in the usual sense, consisting of groups, with the additional requirement of containing every group which is isomorphic to a group in $\mathfrak {X}$. Moreover, if $\mathfrak {X}$ and $\mathfrak {Y}$ are group classes, the product $\mathfrak {XY}$ is the class of all  $\mathfrak X$-by-$\mathfrak Y$ groups, that is the class of all groups G containing a normal subgroup $N\in \mathfrak {X}$ such that $G/N\in {\mathfrak {Y}}$. For reader’s convenience, we recall some of the natural closure properties of a group class $\mathfrak {X}$:

• $\mathfrak {X}$ is S-closed if it contains all groups which are isomorphic to subgroups of $\mathfrak {X}$-groups, and in this case we write $\mathfrak {X}=\textbf{S}\mathfrak {X}$;

• $\mathfrak {X}$ is ${{\textbf {S}}}_n$-closed if it contains all groups which are isomorphic to subnormal subgroups of $\mathfrak {X}$-groups, and in this case we write $\mathfrak {X}={{\textbf {S}}}_n\mathfrak {X}$;

• $\mathfrak {X}$ is Q-closed if it contains all groups which are isomorphic to factor groups of $\mathfrak {X}$-groups, and in this case we write $\mathfrak {X}={\textbf {Q}}\mathfrak {X}$;

• $\mathfrak {X}$ is ${{\textbf {N}}}_0$-closed if the product of two normal $\mathfrak {X}$-subgroups of an arbitrary group belongs to $\mathfrak {X}$, and in this case we write $\mathfrak {X}={\textbf {N}}_0\mathfrak {X}$;

• $\mathfrak {X}$ is N-closed if the product of any collection of normal $\mathfrak {X}$-subgroups of an arbitrary group belongs to $\mathfrak {X}$, and in this case we write ${\mathfrak {X}}={{\textbf {N}}}\mathfrak {X}$;

• $\mathfrak {X}$ is ${{\textbf {P}}}_n$-closed if it contains every group admitting a finite normal series with factors in ${\mathfrak {X}}$.

2 Examples and abstract results

To ease reader’s comprehension of the topic, we start by giving some easy examples of Hall classes and of classes that do not have such a property. As we mentioned in the introduction, finite groups form a Hall class; actually the tensor product method can be used to show that, if $\kappa$ is any infinite cardinal number, the class of all groups of cardinality strictly less than $\kappa$ is a Hall class; in particular, countable groups form a Hall class. It is also easy to see that any class consisting only of groups with a locally cyclic Fitting subgroup is a Hall class, so for instance cyclic groups form a Hall class. Looking at chain conditions, we have that both the maximal and the minimal conditions on subgroups determine Hall classes. On the other hand, it turns out that the classes ${\text {Max-n}}$ and ${\text {Min-n}}$ of all groups satisfying the maximal or the minimal condition on normal subgroups, respectively, are not Hall classes (see [7], especially Theorem 5.3).

We give here a further example to show that groups with the maximal condition on normal subgroups do not form a Hall class. If $X=\langle x\rangle$ is an infinite cyclic group, the group ring $\mathbb {Z}X$ satisfies the maximal condition on right ideals (see for instance [12]), and so its additive group A is noetherian as an X-module. Of course,

$$\begin{aligned} A\otimes _{\mathbb {Z}} A=\bigoplus _{i,j\in \mathbb {Z}}\mathbb {Z}\bigl (x^i\otimes x^j\bigr ), \end{aligned}$$

and so the tensor product $A\otimes _{\mathbb {Z}}A$ is not noetherian as an X-module, since

$$\begin{aligned} A\otimes _{\mathbb {Z}} A=\bigoplus _{j\in \mathbb {Z}}B_j, \end{aligned}$$

where

$$\begin{aligned} B_j=\bigoplus _{i\in \mathbb {Z}}\mathbb {Z}\bigl (x^{i+j}\otimes x^i\bigr ) \end{aligned}$$

is an X-submodule for each j. It follows that the exterior square

$$\begin{aligned} A\,{\small \wedge }\, A\simeq _G\bigoplus _{j>0}B_j \end{aligned}$$

is not noetherian as an X-module. The (diagonal) action of x on $A\otimes _{\mathbb {Z}}A$ induces an automorphism $\alpha$ on A $\wedge$ A. Consider now the stem cover

of A, so in particular $N'=A\,{\small \wedge }\, A$. It is not difficult to see that there exists an automorphism g of N acting as the multiplication by x on A and as $\alpha$ on $A\,{\small \wedge }\, A$. Put $G=\langle g\rangle \ltimes N$. Then N is a nilpotent normal subgroup of G and $G/N'$ satisfies the maximal condition on normal subgroups, but G is not in ${\text {Max-n}}$. Therefore ${\text {Max-n}}$ is not a Hall class.

Another example of a relevant group class which is not a Hall class is that of linear groups. To see this, let p be any prime number, $R_p$ the ring of rational numbers whose denominators are powers of p, and $R_p^+$ its additive group. Consider now the group ${\text {Tr}}_1(3,R_p)$ of all lower unitriangular matrices of degree 3 over $R_p$, and let G be the factor group of ${\text {Tr}}_1(3,R_p)$ with respect to the central subgroup C generated by the matrix

$$\begin{aligned} \begin{pmatrix}1&{}\quad 0&{}\quad 0\\ 0&{}\quad 1&{}\quad 0\\ 1&{}\quad 0&{}\quad 1 \end{pmatrix} \end{aligned}$$

Clearly, G is nilpotent with $G/G'\simeq R_p^+\times R_p^+$ and $G'$ of type $p^\infty$. Thus $G/G'$ is linear of any characteristic (see [23, Theorem 2.2]), but G is not linear, because the commutator subgroup of a nilpotent linear group is reduced (see for instance [7, Lemma 6.2]). The same example also shows that residually finite groups do not form a Hall class.

Note that the above example can be modified in such a way that $G/G'$ is even profinite. To see this, it is enough to replace $R_p$ by the ring $J_p$ of p-adic integers and C by a central subgroup of ${\text {Tr}}_1(3,J_p)$ such that $G'$ is of type $p^\infty$; here $G/G'\simeq J_p\times J_p$ is profinite, while G is not profinite. Therefore the class of profinite groups is not a Hall class.

Operations on classes of groups can be used to produce more examples of Hall classes. For instance, it is obvious that unions and intersections of Hall classes are likewise Hall classes. Moreover, if $\mathfrak {X}$ is any Q-closed group class, then the class of all groups which are not in $\mathfrak {X}$ is trivially a Hall class; in particular, non-periodic and non-nilpotent groups form Hall classes. Furthermore, it follows from results in [19] that any group class $\mathfrak {X}$ which is ${{\textbf {S}}}_{{{\textbf {n}}}}$- and $\textbf{P}\!_{{{\textbf {n}}}}$-closed is a Hall class, provided that it contains all homomorphic images of the tensor product of any pair of abelian $\mathfrak {X}$-groups. Finally, we deal briefly with the closure operator N. If $\mathfrak {X}$ is any group class, it can be proved that the class of all groups generated by subnormal ${\mathfrak {X}}$-subgroups is the smallest N-closed group class containing $\mathfrak {X}$ (see for instance [21, Part 1, Corollary 1 of Lemma 1.31]); we shall denote this class by N$\mathfrak {X}$. Obviously, the class N$\mathfrak {X}$ is N-closed; moreover, it is known that, if $\mathfrak {X}$ is any group class which is S- and ${{\textbf {N}}}_0$-closed, then N$\mathfrak {X}$ is S-closed (see [17, p. 32]).

Lemma 2.1

Let $\mathfrak {X}=\textbf{S}_n\mathfrak {X}={{\textbf {N}}_0\mathfrak {X}}$ be a group class and let G be a group. If X is a subnormal $\mathfrak {X}$-subgroup and N is a normal ${\mathfrak {X}}$-subgroup of G, then XN belongs to $\mathfrak {X}$.

Proof

Obviously we may suppose that $G=XN$ and X is subnormal but not normal in G. Induction on the defect of X in G gives that $X^G=X(N\cap X^G)$ belongs to $\mathfrak {X}$, so also $G=X^GN$ is an $\mathfrak {X}$-group. $\square$

Note that the above lemma for $\mathfrak {X}=\mathfrak {N}$ is a well known easy consequence of Fitting’s theorem on products of nilpotent normal subgroups.

Theorem 2.2

Let $\mathfrak {X}={\textbf {S}}_n\mathfrak {X}={\textbf {N}}_0\mathfrak {X}$ be a Hall class containing all abelian groups. Then also N$\mathfrak {X}$ is a Hall class.

Proof

Let G be a group having a nilpotent normal subgroup N with $G/N'\in {{\textbf {N}}}\mathfrak {X}$. If $X/N'$ is any subnormal $\mathfrak {X}$-subgroup of $G/N'$, we have that $XN/N'$ belongs to $\mathfrak {X}$ by Lemma 2.1; then XN is a subnormal $\mathfrak {X}$-subgroup of G, because $\mathfrak {X}$ is a Hall class. It follows that G is generated by its subnormal ${\mathfrak {X}}$-subgroups and so $G\in \textbf{N}\mathfrak {X}$. Therefore N$\mathfrak {X}$ is a Hall class. $\square$

The reminder of this section deals with sufficient conditions for a product of group classes $\mathfrak {XY}$ to be a Hall class; here, the key ingredient is essentially that $\mathfrak {X}$ is an ${\textbf {N}}_0$-closed Hall class containing all abelian groups. In particular, the following result is crucial in the proof of Theorem B.

Theorem 2.3

Let $\mathfrak X$ be a Hall class such that in any group the product of an abelian normal subgroup and a normal $\mathfrak {X}$-subgroup is in $\mathfrak {X}$. If $\mathfrak {Y}$ is a group class, and either $\mathfrak {X}=\textbf{S}_n\mathfrak {X}$ or $\mathfrak {Y}={\textbf {Q}}\mathfrak {Y}$, then $\mathfrak {XY}$ is a Hall class.

Proof

Let G be a group having a nilpotent normal subgroup N with $G/N'\in \mathfrak {XY}$. Then there exists a normal $\mathfrak {X}$-subgroup $X/N'$ of $G/N'$ such that $G/X\in {\mathfrak {Y}}$. Since $XN/N'=(X/N')(N/N')$ belongs to the Hall class $\mathfrak {X}$, we have that $XN\in \mathfrak {X}$. If $\mathfrak {X}=\textbf{S}_n\mathfrak {X}$, the subgroup X is in $\mathfrak {X}$ and so G belongs to $\mathfrak {XY}$. On the other hand, if $\mathfrak {Y}={\textbf {Q}}{\mathfrak {Y}}$, then $G/XN\in \mathfrak {Y}$, and hence $G\in \mathfrak {XY}$. Therefore $\mathfrak {XY}$ is a Hall class in both cases. $\square$

Corollary 2.4

Let $\mathfrak {X}=\textbf{S}_n\mathfrak {X}={\textbf {N}}_0\mathfrak {X}$ be a Hall class containing all abelian groups and let $\mathfrak {Y}$ be a Q-closed group class. Then $({\textbf {N}}\mathfrak {X})\mathfrak {Y}$ is a Hall class.

Proof

The class N$\mathfrak {X}$ is a Hall class by Theorem 2.2. Moreover,

$$\begin{aligned} \mathfrak {A}\le {\mathfrak {X}}\le {\textbf {N}}\mathfrak {X}={\textbf {N}}_0({\textbf {N}}\mathfrak {X}) \end{aligned}$$

and so it follows from Theorem 2.3 that $({\textbf {N}}\mathfrak {X})\mathfrak {Y}$ is a Hall class. $\square$

Of course, the existence of ${\textbf {S}}_n$- and ${\textbf {N}}_0$-closed group classes which are not Hall classes, for instance the class $\mathfrak FN$, shows that the condition that $\mathfrak {X}$ is a Hall class cannot be dropped out from the statement of Theorem 2.3. Moreover, the other main assumption of the same statement cannot be omitted. To see this, note first that the class $\mathfrak {X}=\mathfrak {N}\cup \mathfrak {F}$, formed by all groups which are either nilpotent or finite, is a Hall class which is not closed with respect to forming products of an abelian normal subgroup and a normal $\mathfrak {X}$-subgroup. On the other hand, $\mathfrak {X}\mathfrak {N}=\mathfrak {N}^2\cup {\mathfrak {FN}}$ is not a Hall class. In fact, if H is any group which is not finite-by-nilpotent but contains a nilpotent normal subgroup N such that $H/N'\in \mathfrak {FN}$ (for instance any of the groups constructed in Theorem C), the direct product $G=H\times \!{\text {Alt}}(5)$ does not belong to $\mathfrak {N}^2\cup \mathfrak {FN}$ although $G/N'$ does.

Theorem 2.3 provides a number of natural Hall classes, among which we mention that of groups with a (locally) nilpotent commutator subgroup, and that of groups with a (locally) nilpotent finite residual. In general, groups whose commutator subgroup belongs to a given $\textbf{S}_n$- and ${\textbf {N}}_0$-closed Hall class form themselves a Hall class; this is for instance the case of groups whose commutator subgroup is hypercentral/FC-nilpotent/FC-hypercentral/$\ldots$ (see also [21, Part 1, p. 130]).

In this context, we note that Theorem 2.3 also applies to many group classes which are not $\textbf{N}_0$-closed, for example the class of locally soluble groups, showing in particular that groups with a locally soluble commutator subgroup form a Hall class. Moreover, it is known that the product of two hypercyclic (locally supersoluble) normal subgroups need not be hypercyclic (locally supersoluble), but this is at least true in groups with a locally nilpotent commutator subgroup — see for instance [5]. Since the commutator subgroup of any locally supersoluble group is locally nilpotent, it follows that the product of an abelian normal subgroup and a hypercyclic (locally supersoluble) normal subgroup is always hypercyclic (locally supersoluble), and hence the class of hypercyclic groups and that of locally supersoluble groups satisfy the requirements of Theorem 2.3; in particular, the class $\mathfrak {XF}$ of all $\mathfrak {X}$-by-finite groups and the class $\mathfrak {XA}$ of all $\mathfrak {X}$-by-abelian groups are Hall classes, whenever $\mathfrak {X}$ is the class of hypercyclic (locally supersoluble) groups. The same remarks apply to the class of paranilpotent groups; recall here that a group G is paranilpotent if it has a finite normal series

$$\begin{aligned} \{1\}=G_0<G_1<\cdots <G_t=G \end{aligned}$$

such that $G_{i+1}/G_i$ is abelian and all its subgroups are G-invariant, for each $i=0,1,\ldots ,t-1$. Actually, paranilpotent groups are hypercyclic, have a nilpotent commutator subgroup and form a Hall class; moreover, the product of two paranilpotent normal subgroups is paranilpotent, provided that it has a nilpotent commutator subgroup.

Although the class of supersoluble groups does not satisfy the requirements of Theorem 2.3 (because infinitely generated abelian groups are not supersoluble), it turns out that supersoluble-by-finite groups form a Hall class, since they coincide with finitely generated nilpotent-by-finite groups. In any case, the following slight modification of Theorem 2.3 allows us to say that supersoluble-by-finite groups form a Hall class, as well as groups with a supersoluble commutator subgroup; note that the product of two supersoluble normal subgroups of a group with a nilpotent commutator subgroup is supersoluble.

Theorem 2.5

Let $\mathfrak {X}$ be a Hall class such that in any group the product of an abelian normal $\mathfrak {X}$-subgroup and a normal ${\mathfrak {X}}$-subgroup is in $\mathfrak {X}$. If every abelian normal subgroup of an $\mathfrak {XY}$-group belongs to $\mathfrak {X}$, and either $\mathfrak {X}=\textbf{S}_n\mathfrak {X}$ or ${\mathfrak {Y}}=\textbf{Q}\mathfrak {Y}$, then $\mathfrak {XY}$ is a Hall class.

3 Theorem A

This section is devoted to the proof of Theorem A. First we need to recall the relevant role played by tensor products of modules in the study of Hall classes.

If G is a group and A, B are G-modules, the tensor product $A\otimes _{\mathbb {Z}}\! B$ can be made into a G-module via $(a\otimes b)g=(ag)\otimes (bg)$ for all $a\in A$, $b\in B$, $g\in G$. A class ${\mathcal {M}}$ of G-modules is tensorial if, for all A and B in ${\mathcal {M}}$, every G-homomorphic image of $A\otimes _{\mathbb {Z}}\! B$ belongs to ${\mathcal {M}}$. In order to prove that certain kinds of generalized nilpotent groups form Hall classes, one has to prove that certain corresponding classes of modules are tensorial; in fact, it is well known that if N is a normal subgroup of a group G, there exists, for each positive integer i, a G-epimorphism

$$\begin{aligned} (N/N')\otimes _{\mathbb {Z}}\!\bigl (\gamma _i(N)/\gamma _{i+1}(N)\bigr )\longrightarrow \gamma _{i+1}(N)/\gamma _{i+2}(N), \end{aligned}$$

where the factors of the lower central series of N are regarded as G-modules by conjugation. For instance, Hall’s nilpotency criterion rests upon the fact that the class of polytrivial G-modules is tensorial (see for instance [24, p. 10]); recall here that a G-module is polytrivial (of length at most m) if it has a finite series (of length m) of G-submodules whose factors are trivial G-modules.

Recall that a group is Engel if for all its elements x and y there is a positive integer $m=m(x,y)$ such that $[x,_my]=1$. Moreover, if G is a group and A is a G-module, we say that A is an Engel G-module if for all elements a of A and g of G there exists a positive integer $m=m(a,g)$ such that $a(g-1)^m=0$. Of course, the class $\mathcal {E}(G)$ of all Engel G-modules is closed with respect to forming submodules, quotient modules and extensions. The following easy lemma explains why Engel modules arise.

Lemma 3.1

Let G be a group and let A be an abelian normal subgroup of G. Then G is an Engel group if and only if A is an Engel G-module (by conjugation) and G/A is an Engel group.

Lemma 3.2

The class $\mathcal {E}(G)$ is tensorial for any group G.

Proof

Let A and B be Engel G-modules, and consider arbitrary elements $c=\sum _i(a_i\otimes b_i)$ of $A\otimes _{\mathbb {Z}}B$ and g of G. By hypothesis there is a positive integer m such that $a_i(g-1)^m=b_i(g-1)^m=0$ for all i. Denote by $A_0$ and $B_0$ the $\langle g\rangle$-submodules of A and B generated by the $a_i$’s and the $b_i$’s, respectively. Then $A_0$ and $B_0$ are polytrivial $\langle g\rangle$-modules, so $A_0\otimes _{\mathbb {Z}}B_0$ is polytrivial as a $\langle g\rangle$-module, and hence its natural image L in $A\otimes _{\mathbb {Z}} B$ is likewise a polytrivial $\langle g\rangle$-module. Since c lies in L, it follows that $c(g-1)^n=0$ for some large enough n. Therefore $A\otimes _{\mathbb {Z}}B$ is an Engel G-module.  $\square$

Theorem 3.3

The class of Engel groups is a Hall class.

Proof

Let G be a group containing a nilpotent normal subgroup N such that $G/N'$ is an Engel group. Then $N/N'$ is an Engel G-module (by conjugation) and hence by Lemma 3.2 so is each $\gamma _i(N)/\gamma _{i+1}(N)$. In particular, $\gamma _c(N)$ is an Engel G-module, where c is the nilpotency class of N. Moreover, we may assume by induction that the factor group $G/\gamma _c(N)$ is Engel, and so an application of Lemma 3.1 yields that G is an Engel group. $\square$

A group is called bounded Engel if there is a positive integer n such that $[x,_ny]\!=\!1$ for all its elements x and y. Moreover, if G is a group, a G-module A is bounded Engel if there exists a positive integer m such that $a(g-1)^m=0$ for all $a\in A$ and $g\in G$. Of course, a group G containing an abelian normal subgroup A is bounded Engel if and only if G/A is bounded Engel and A is a bounded Engel G-module by conjugation. Since the tensor product of two polytrivial G-modules of length at most m is polytrivial of length at most $2m-1$ (see for instance [21, Part 1, p. 56]), a slight modification of the proof of Lemma 3.2 gives that also the class of bounded Engel G-modules is tensorial. Thus, a proof similar to that of Theorem 3.3 yields the following result.

Theorem 3.4

The class of bounded Engel groups is a Hall class.

Our next step in the proof of Theorem A deals with the classes formed by Baer, Gruenberg, strongly Baer and strongly Gruenberg groups. To this aim, we need a criterion for a subgroup to be subnormal or ascendant. Recall that in any group the product of two hypercentral normal subgroups is likewise hypercentral (see [14, Lemma 1]); the following is a slight extension of this result and corresponds to the generalization of Fitting’s theorem stating that the product of a nilpotent normal subgroup and a nilpotent subnormal subgroup is always nilpotent (see for instance [3, Lemma 1.4.16]).

Lemma 3.5

Let the group $G=XN$ be the product of a hypercentral ascendant subgroup X and a hypercentral normal subgroup N. Then G is hypercentral.

Proof

Since the hypotheses are obviously inherited by homomorphic images, it is enough to prove that a non-trivial G has a non-trivial centre, so without loss of generality we may assume $X\ne G$ and $\zeta _1(X)\cap \zeta _1(N)=\{1\}$. Then $X\cap \zeta _1(N)=\{1\}$ since X is hypercentral and $\zeta _1(N)$ is normal in G. Moreover $X<Y=N_{X\zeta _1(N)}(X)$ and

$$\begin{aligned} \bigl [Y\cap \zeta _1(N),G\bigr ]=\bigl [Y\cap \zeta _1(N),X\bigr ]\le X\cap \zeta _1(N)=\{1\}, \end{aligned}$$

so that $Y\cap \zeta _1(N)\le \zeta _1(G)$. Since $Y=X\bigl (Y\cap \zeta _1(N)\bigr )$, we have $Y\cap \zeta _1(N)\ne \{1\}$ and the statement is proved. $\square$

Corollary 3.6

Let G be a group and let N be a nilpotent normal subgroup of G. If X is a nilpotent (hypercentral) subgroup of G and $XN'$ is subnormal (ascendant) in G, then X is subnormal (ascendant) in G.

Proof

Since $XN'$ is subnormal (ascendant) in G, the subgroup

$$\begin{aligned} XN/N'=(XN'/N')(N/N') \end{aligned}$$

is nilpotent (hypercentral) by Fitting’s theorem (by Lemma 3.5). It follows that XN is likewise nilpotent (hypercentral), so X is subnormal (ascendant) in XN and hence also in G. $\square$

Let G be a group. Then G is called a Baer group if all its cyclic subgroups are subnormal, while G is a Gruenberg group if every cyclic subgroup is ascendant. It is known that G is a Baer group (respectively, Gruenberg group) if and only if it is generated by abelian subnormal (respectively, ascendant) subgroups. Of course, Baer groups are Gruenberg and every Gruenberg group is locally nilpotent. It is also clear that hypercentral groups, as well as countable locally nilpotent groups, are Gruenberg.

Theorem 3.7

The class of Baer groups and the class of Gruenberg groups are Hall classes.

Proof

Let G be a group containing a nilpotent normal subgroup N such that $G/N'$ is Baer (Gruenberg) and let g be any element of G. Then the subgroup $\langle g,N'\rangle$ is subnormal (ascendant) in G and so $\langle g\rangle$ is subnormal (ascendant) in G by Corollary 3.6. Therefore G is a Baer (Gruenberg) group. $\square$

A group G is said to be strongly Baer if all its nilpotent subgroups are subnormal (see [3, Section 7.4]), while G is strongly Gruenberg if every hypercentral subgroup of G is ascendant. Of course, strongly Baer groups are strongly Gruenberg, but the locally dihedral 2-group is a non-Baer group which is strongly Gruenberg; moreover, if p is any prime number, the wreath product of a group of order p by a group of type $p^\infty$ is a Baer group which is not strongly Gruenberg. The argument used in the proof of Theorem 3.7 shows that also strongly Baer groups, as well as strongly Gruenberg groups, form Hall classes.

Of course, the class N$\mathfrak {A}$, which is the smallest N-closed class containing $\mathfrak {A}$, is precisely the class of Baer groups (see [21, Part 1, Lemma 2.34]) and so it is a Hall class. On the contrary, note that the smallest $\textbf{N}_0$-closed group class containing all abelian groups, denoted by $\textbf{N}_0\mathfrak {A}$, is not a Hall class. In fact, it is not difficult to see that the class $\mathfrak {V}$ of all groups generated by finitely many abelian subnormal subgroups is $\textbf{N}_0$-closed, so that it contains $\textbf{N}_0\mathfrak {A}$; on the other hand, the consideration of any of the uncountable extraspecial p-groups (in which all abelian subgroups are countable) constructed by Ehrenfeucht and Faber [8] shows that $\mathfrak {V}$ does not contain $\mathfrak {N}$. Therefore $\textbf{N}_0\mathfrak {A}$ and $\mathfrak {V}$ are subclasses of the class of Baer groups which are not Hall classes. Note finally that $\textbf{N}_0\mathfrak {A}<\mathfrak {V}$, since $\textbf{N}_0\mathfrak {A}\le \mathfrak {N}$ and there exists non-nilpotent groups generated by two abelian subnormal subgroups (see for instance [17, Theorem 2.1.6]).

We now deal with Fitting and strongly Fitting groups. Recall that a group G is said to be a Fitting group if it is covered by its nilpotent normal subgroups, while G is called strongly Fitting if the normal closure of any nilpotent subgroup of G is nilpotent; obviously, strongly Fitting groups are Fitting, while the direct product of dihedral groups of order $2^n$, one for each n, is a Fitting group which is not strongly Fitting. In this case, we need the following piece of information about the behaviour of radicals with respect to suitable Hall classes. Recall first that, if $\mathfrak {X}$ is any class of groups, the $\mathfrak X$-radical of a group G is the subgroup $\rho _{\mathfrak {X}}(G)$ generated by all normal $\mathfrak {X}$-subgroups of G; in particular, if $\mathfrak {X}$ is N-closed, then $\rho _{\mathfrak {X}}(G)$ is the unique maximal normal $\mathfrak {X}$-subgroup of G. Of course, the ${\mathfrak {N}}$-radical is the well known Fitting subgroup.

Lemma 3.8

Let $\mathfrak {X}=\textbf{N}_0\mathfrak {X}$ be a Hall class containing all abelian groups and let N be a nilpotent normal subgroup of a group G. Then $\rho _{{\mathfrak {X}}}(G/N')\le \rho _{\mathfrak {X}}(G)/N'$. Moreover, if $\mathfrak {X}$ is also Q-closed, then $\rho _{\mathfrak {X}}(G/N')=\rho _{\mathfrak {X}}(G)/N'$.

Proof

Recall that $\mathfrak {X}$ contains $\mathfrak {N}$, so $N'\le N\le \rho _{\mathfrak {X}}(G)$. Let $X/N'$ be any normal ${\mathfrak {X}}$-subgroup of $G/N'$. Then $XN/N'$ belongs to $\textbf{N}_0\mathfrak {X}=\mathfrak {X}$ and hence XN is a normal $\mathfrak {X}$-subgroup of G because $\mathfrak {X}$ is a Hall class. Thus $X\le XN\le \rho _{\mathfrak {X}}(G)$ and so $\rho _{\mathfrak {X}}(G/N')\le \rho _{{\mathfrak {X}}}(G)/N'$. Moreover, if $\mathfrak {X}$ is also Q-closed,  $\rho _{\mathfrak {X}}(G)/N'$ is generated by normal $\mathfrak {X}$-subgroups of $G/N'$ and hence it is contained in $\rho _{\mathfrak {X}}(G/N')$. The proof is complete. $\square$

An application of the above statement to the class ${\mathfrak {X}}=\mathfrak {N}$ shows that, if N is a nilpotent normal subgroup of a group G and $F/N'$ is the Fitting subgroup of $G/N'$, then F is the Fitting subgroup of G. Since a group is Fitting if and only it coincides with its Fitting subgroup, we have the following result.

Theorem 3.9

The class of Fitting groups is a Hall class.

Proving that also strongly Fitting groups form a Hall class is very easy: it is enough to combine Fitting’s theorem with the Hall’s nilpotency criterion. The class of strongly Fitting groups properly contains the relevant class consisting of groups in which all subgroups are subnormal, the so-called $N_1$-groups (see [2] and the example on p. 204 of [3]). We leave open the question whether $N_1$-groups form a Hall class. A similar problem arises for the class of groups with only ascendant subgroups (N-groups).

We point out that the scope of Lemma 3.8 is not restricted to the case of nilpotent groups. For instance, it allows us to prove that groups covered by hypercentral normal subgroups form a Hall class (see [14]), as well as groups covered by FC-nilpotent and FC-hypercentral normal subgroups (see [15]). Moreover, it follows from the same lemma that, for each positive integer k, the class of soluble groups of Fitting length at most k is a Hall class. Recall here that a soluble group G has Fitting length at most k if $F_k(G)=G$, where the subgroups $F_i(G)$ are recursively defined by putting $F_0(G)=\{1\}$ and $F_{i+1}(G)/F_i(G)=\rho _{\mathfrak {N}}\bigl (G/F_i(G)\bigr )$. For finite soluble groups, having Fitting length at most k is obviously equivalent to being k-polynilpotent, i.e. to have a series of length at most k with nilpotent factors. This is not the case for arbitrary soluble groups, but it still turns out that the class of k-polynilpotent groups (which is actually $\mathfrak {N}^k$) is a Hall class. In particular, the class $\mathfrak {N}^2$ of metanilpotent groups is a Hall class, a fact that was already clear by Theorem 2.3.

Lemma 3.10

The class of k-polynilpotent groups is $\textbf{N}_0$-closed for any positive integer k.

Proof

It is easy to see that any k-polynilpotent group has a finite characteristic series of length at most k with nilpotent factors, and so it follows from Fitting’s theorem that the class $\mathfrak {N}^k$ is $\textbf{N}_0$-closed. $\square$

Theorem 3.11

$\mathfrak {N}^k$ is a Hall class for each positive integer k.

Proof

Since the class $\mathfrak {N}^k$ is $\textbf{N}_0$-closed by Lemma 3.10, it follows from Theorem 2.3 and induction on k, that ${\mathfrak {N}}^k=\mathfrak {N}^{k-1}\mathfrak {N}$ is a Hall class. $\square$

4 Theorem B and Theorem C

The first part of this section is devoted to the proof of Theorem B. We first note that all group classes considered in its statement are subgroup closed, so that Theorem 2.3, with the subsequent discussion on its scope and Lemma 3.10, allows us to restrict the attention to the following types of groups: Engel, bounded Engel, strongly Baer, strongly Gruenberg, Fitting and strongly Fitting.

It is known that the class of Fitting groups is not $\textbf{N}_0$-closed (see for instance [20, p. 114]), while it seems to be an open question whether the class of (bounded) Engel groups is $\textbf{N}_0$-closed. However, the following result still enables us to use Theorem 2.3 for these group classes, and so to prove Theorem B when $\mathfrak {X}$ is the class consisting of Engel, bounded Engel or Fitting groups.

Lemma 4.1

Let the group $G=HK$ be the product of two normal subgroups H and K.

• (a) If H is Engel and K is hypercentral, then G is an Engel group.

• (b) If H is bounded Engel and K is nilpotent, then G is bounded Engel.

• (c) If H is a Fitting group and K is nilpotent, then G is a Fitting group.

Proof

(a)  Let x and y be arbitrary elements of G. The factor group $G/(H\cap K)$ is isomorphic to a subgroup of the direct product of a hypercentral group and an Engel group, so it is Engel because hypercentral groups are locally nilpotent. It follows that there is a positive integer $m=m(x,y)$ such that $[x,_{_m}y]\in H\cap K$. Let

$$\begin{aligned} \{1\}=Z_0<Z_1<\cdots \, Z_\alpha<Z_{\alpha +1}<\cdots \, Z_\tau =H\cap K \end{aligned}$$

be an ascending G-invariant series whose factors are central in K. Consider the least ordinal $\mu \le \tau$ such that $g=[x,_{_r}y]\in Z_\mu$ for some positive integer r and assume for a contradiction $\mu >0$. Of course, $\mu$ is not a limit and the coset $gZ_{\mu -1}$ centralizes $K/Z_{\mu -1}$. Write $y=kh$, with $h\in H$ and $k\in K$. Since H is Engel, there is a positive integer s such that $[g,_{_s}h]=1$. Thus

$$\begin{aligned} \bigl [gZ_{\mu -1},_{_s}yZ_{\mu -1}\bigr ]=\bigl [gZ_{\mu -1},_{_s}hZ_{\mu -1}\bigr ]=1, \end{aligned}$$

so that $[x,_{_{r+s}}y]=[g,_{_s}y]\in Z_{\mu -1}$ and this contradiction shows that $\mu =0$. Therefore $[x,_{_r}y]=1$ and G is an Engel group.

(b)   Since H is bounded Engel, there exists a positive integer m such that $[a,_{_m}b]=1$ for all $a,b\in H$. Let x and y be arbitrary elements of G and let c be the nilpotency class of K. Of course, the factor group $G/(H\cap K)$ embeds into $(G/H)\times (G/K)$, and so $[x,_{_n}y]\in H\cap K$, where $n=\max \{m,c\}$. Let

$$\begin{aligned} \{1\}=Z_0\le Z_1\le \cdots \le Z_c=H\cap K \end{aligned}$$

be a finite G-invariant series whose factors are central in K, and suppose that $g=[x,_{_{n(i+1)}}y]\in Z_{c-i}$ for some non-negative integer $i<c$. Write $y=kh$, with $h\in H$ and $k\in K$, so $[g,_{_n}h]=[g,_{_m}h]=1$. Since the coset $gZ_{c-(i+1)}$ centralizes $K/Z_{c-(i+1)}$, we have

$$\begin{aligned} \bigl [gZ_{c-(i+1)},_{_n}yZ_{c-(1+1)}\bigr ]=\bigl [gZ_{c-(i+1)},_{_n}hZ_{c-(i+1)}\bigr ]=1, \end{aligned}$$

so that $[x,_{_{n(i+2)}}y]=[g,_{_n}y]\in Z_{c-(i+1)}$. In particular, $[x,_{_{n(c+1)}}y]\in Z_0=\{1\}$. Therefore G is a bounded Engel group.

(c)   Let X be any nilpotent normal subgroup of H. Then X is subnormal in G, so that the product XK is nilpotent, and it is obviously normal in G. Since H is generated by its nilpotent normal subgroups, it follows that G is generated by nilpotent normal subgroups and so it is a Fitting group. $\square$

Our next result completes the proof of Theorem B.

Theorem 4.2

Let $\mathfrak {X}$ be the class consisting of all groups of one of the following types: strongly Fitting, strongly Baer, strongly Gruenberg. If $\mathfrak {Y}$ is any group class, then $\mathfrak {XY}$ is a Hall class.

Proof

Let G be a group containing a nilpotent normal subgroup N such that $G/N'$ has a strongly Fitting normal subgroup $M/N'$ such that $G/M\in \mathfrak {Y}$. If X is any nilpotent subgroup of M, then $X^MN'/N'$ is a nilpotent subnormal subgroup of $G/N'$, so that also $X^MN/N'$ is nilpotent. It follows that $X^MN$ is nilpotent, so in particular $X^M$ is nilpotent and hence M is a strongly Fitting group.

Assume now that the normal subgroup $M/N'$ is strongly Baer (strongly Gruenberg) and let X be a nilpotent (hypercentral) subgroup of M. Then $XN'/N'$ is a nilpotent (hypercentral) subgroup of $M/N'$, so that $XN'$ is subnormal (ascendant) in M and hence also in G. It follows now from Corollary 3.6 that X is subnormal (ascendant) in G. Therefore M is strongly Baer (strongly Gruenberg) and the proof is complete. $\square$

We move now to the proof of Theorem C, i.e. to the study of finite obstructions at the bottom of a Hall class. If $\mathfrak {K}$ is any field and m, n are positive integers, we denote by ${\mathfrak {K}}\,^{m,n}$ the set of all $m\times n$ matrices over ${\mathfrak {K}}$; moreover, $0\,^{m,n}\in \mathfrak {K}\,^{m,n}$ is the zero matrix and when $m=n$ then $I_n$ is the identity matrix of order n. Also the standard set of units in any $\mathfrak {K}\,^{m,n}$ we denote by $\{e_{ij}\}$. Recall finally that ${\text {Tr}}(n,\mathfrak {K})$ is the subgroup of $GL(n,\mathfrak {K})$ consisting of all lower triangular matrices; also ${\text {Tr}}_1(n,\mathfrak {K})$ denotes the unipotent radical of ${\text {Tr}}(n,\mathfrak {K})$.

Proof of Theorem C (a)   Since $\mathfrak {K}$ contains a finite subfield of order $>2$, there is a primitive p-th root $\gamma$ of 1 in $\mathfrak {K}$, where $p\ne q$ is a suitable prime number. Consider now the finite subfield ${\mathfrak {K}}_\gamma$ of $\mathfrak {K}$ generated by $\gamma$, and the matrices

$$\begin{aligned} a_w= \begin{pmatrix} 1&{}\quad 0&{}\quad 0\\ w&{}\quad 1&{}\quad 0\\ 0&{}\quad 0&{}\quad 1 \end{pmatrix}, \quad b= \begin{pmatrix} \gamma &{}\quad 0&{}\quad 0\\ 0&{}\quad 1&{}\quad 0\\ 0&{}\quad 0&{}\quad 1 \end{pmatrix} \quad \text {and}\quad c_{u,v}= \begin{pmatrix} 1&{}\quad 0&{}\quad 0\\ 0&{}\quad 1&{}\quad 0\\ u&{}\quad v&{}\quad 1 \end{pmatrix} \end{aligned}$$

for all $w\in \mathfrak {K}_\gamma$, $u,v\in \mathfrak {K}$; in particular $b^p=1$. Clearly,

$$\begin{aligned} A=\{a_w\;|\; w\in \mathfrak {K}_\gamma \}\quad \text {and}\quad C=\{c_{u,v}\;|\; u,v\in \mathfrak {K}\} \end{aligned}$$

are elementary abelian q-subgroups of ${\text {Tr}}_1(3,\mathfrak {K})$ and A is finite. Then $N=A\ltimes C$ is a unipotent normal subgroup of $G=\langle b,N\rangle \le {\text {Tr}}(3,\mathfrak {K})$ and $N'=\{c_{u,0}\;|\; u\in \mathfrak {K}\}$. It follows that $G/N'\simeq \bigl (\langle b\rangle \ltimes A\bigr )\times {\mathfrak {K}}^+$ is finite-by-abelian. On the other hand, each $a_w$ has infinitely many conjugates in G and so the non-nilpotent subgroup $\langle b\rangle \ltimes A$ has trivial intersection with every finite normal subgroup of G. Therefore G is not finite-by-(locally nilpotent).

(b)   Let $\mathfrak {L}$ be a finite subfield of $\mathfrak {K}$ of order $2^r$, with $r>1$, and let G be the subgroup of ${\text {GL}}(5,\mathfrak {K})$ consisting of all $5\times 5$ matrices of the form

$$\begin{aligned} M(a,b,c,d)=\begin{pmatrix} a &{}\quad 0\,^{2,2} &{}\quad 0\,^{2,1}\\ b &{}\quad I_2 &{}\quad 0\,^{2,1}\\ c &{}\quad d &{}\quad I_1 \end{pmatrix}, \end{aligned}$$

where $a\in SL(2,\mathfrak {L})$, $b\in \mathfrak {L}\,^{2,2}\!\!$, and $\; c,d\in \mathfrak {K}\,^{1,2}$. Put $N=G\cap {\text {Tr}}_1(5,\mathfrak {K})$ and

$$\begin{aligned} S=\bigl \{{\text {diag}}(a,I_2,I_1)\;\bigl |\; a\in {\text {SL}}(2,\mathfrak {L})\bigr \}\simeq {\text {SL}}(2,2^r). \end{aligned}$$

Clearly, N is unipotent, so in particular nilpotent, and $G=S\ltimes N$. Consider now the matrices $a=I_2+e_{12}\in SL(2,\mathfrak {L})$ and $s={\text {diag}}(a,I_2,I_1)\in S$. If w is any element of $\mathfrak {K}$, let $g_w=I_5+we_{5,1}\in G$. It is easy to prove that

$$\begin{aligned} g_wsg_w^{-1}=I_5+e_{12}+we_{52}. \end{aligned}$$

Thus the normal closure $S^G$ is infinite. Since $r>1$, the group S is perfect, and so it is contained in any normal subgroup L of G such that G/L is locally soluble. It follows that G is not finite-by-(locally soluble).

Since

$$\begin{aligned} \bigl [M(I_2,\, 0\,^{2,2},\, 0\,^{1,2},\, c),M(I_2,\, I_2,\, 0\,^{1,2},\, 0\,^{1,2})\bigl ]=M(I_2,\, 0\,^{2,2},\, c,\, 0\,^{1,2}) \end{aligned}$$

for each element c of $\mathfrak {K}\,^{1,2}$, we have that the subgroup

$$\begin{aligned} W=\bigl \{M(I_2,\, 0\,^{2,2},\, c,\, 0\,^{1,2})\;|\; c\in \mathfrak {K}\,^{1,2}\bigr \} \end{aligned}$$

is contained in $N'$. Set

$$\begin{aligned} S_1=\bigl \{ M(a,b,0\,^{1,2},0\,^{1,2})\;\bigl |\; a\in SL(2,\mathfrak {L}), b\in \mathfrak {L}\,^{2,2}\bigr \} \end{aligned}$$

and

$$\begin{aligned} N_1=\bigl \{ M(I_2,0\,^{2,2},c,d)\;\bigl |\; c,d\in {\mathfrak {K}}\,^{1,2}\bigr \}. \end{aligned}$$

In particular, $S_1$ is a finite subgroup of G and $N_1\le N$. Since

$$\begin{aligned} M(a,b,0\,^{1,2},0\,^{1,2})\cdot M(I_2,0\,^{2,2},c,d)=M(a,b,c,d) \end{aligned}$$

for all possible choices of a, b, c, d, we have $G=S_1N_1$. Moreover, an easy calculation shows that $[S_1,N_1]$ is contained in W and so also in $N'$. Therefore the factor group $G/N'$ is finite-by-abelian and the proof is complete. $\square$

It follows from Theorem C that $\mathfrak {FX}$ is not a Hall class for any choice of the group class $\mathfrak {X}$ among the 15 classes considered in the statement of Theorem B; for (bounded) Engel groups one also needs to recall that a locally soluble group is Engel if and only if it is locally nilpotent (see for instance [21, Part 2, Theorem 7.34]).

The final result of this section allows us to extend the scope of Theorem C to the universe of (affine) algebraic groups, i.e. closed subgroups of ${\text {GL}}(n,\mathfrak {K})$. In what follows, if $G\le {\text {GL}}(n,\mathfrak {K})$, we denote by $G^\#$ the Zariski closure of G in ${\text {GL}}(n,\mathfrak {K})$.

Lemma 4.3

Let $\mathfrak {K}$ be an algebraically closed field and let N be a normal subgroupof $G\le {\text {GL}}(n,{\mathfrak {K}})$ such that $G/N'$ is finite-by-(nilpotent of class k) for some positive integer k. Then also $G^{\#}/(N^{\#})'$ is finite-by-(nilpotent of class at most k).

Proof

Since $[N,G]\le N$, we have $[N^{\#},G^{\#}]\le N^{\#}$ (see [23, Lemma 5.10]) and hence $N^{\#}$ is normal in $G^{\#}$. Further $(N^{\#})'$ is closed in ${\text {GL}}(n,\mathfrak {K})$ by Corollary 14.17 of [23], so that $(N')^\#\le (N^\#)'$ and hence $(N')^\#=(N^\#)'$ by Lemma 5.10 of [23]. Moreover, $\gamma _{k+1}(G)N'/N'$ is finite and $N'\le (N^{\#})'$, so that $(N^{\#})'$ has finite index in $M=\gamma _{k+1}(G)(N^{\#})'$ . Thus M is closed in ${\text {GL}}(n,\mathfrak {K})$ and another application of Lemma 5.10 of [23] yields that $\gamma _{k+1}(G^{\#})\le M^{\#}=M$. Therefore the group $\gamma _{k+1}(G^{\#})(N^\#)'/(N^{\#})'$ is finite and the statement is proved. $\square$

Theorem C$^{\#}$ Let $\mathfrak {K}$ be an algebraically closed field of positive characteristic q.

• (a) There exists an algebraic group $G\le {\text {GL}}(3,\mathfrak {K})$ which is not finite-by-(locally nilpotent) but contains a unipotent closed normal subgroup N such that $G/N'$ is finite-by-abelian.

• (b) If $q=2$, there exists an algebraic group $G\le {\text {GL}}(5,\mathfrak {K})$ which is not finite-by-(locally soluble) but contains a unipotent closed normal subgroup N such that $G/N'$ is finite-by-abelian.

Proof

By Theorem C (a) there exists $H\le {\text {GL}}(3,\mathfrak {K})$ which is not finite-by-(locally nilpotent) but contains a unipotent normal subgroup L such that $H/L'\in \mathfrak {FA}$.Put $G=H^\#$ and $N=L^\#$. Then $G/N'$ is finite-by-abelian by Lemma 4.3 and N is unipotent (see for instance [7, Lemma 2.1]). Since $H\le G$, part (a) of the statement is proved. The proof of (b) is similar. $\square$

5 Theorem D

We first note that finite-by-(finitely generated and nilpotent) groups are already known to form a Hall class. Actually, this is a special case of a more general result, which states that if N is a nilpotent normal subgroup of a group G such that $G/N'$ is finite-by-nilpotent, then G itself is finite-by-nilpotent, provided that N satisfies a suitable rank restriction (see [11]). However, we prefer to give here a different type of proof since it can be used as a model for other group classes (such as that of finite-by-supersoluble groups) and is based on characterizations like the following one, which are of an independent interest.

Lemma 5.1

A group G is finite-by-nilpotent if and only if it has a normal series of finite length whose factors are either finite or central.

Proof

Of course, it is enough to prove that G is finite-by-nilpotent, provided that there exists a finite normal series

$$\begin{aligned} \{1\}=G_0\le G_1\le \cdots \le G_t=G \end{aligned}$$

such that, for each non-negative integer $i<t$, $G_{i+1}/G_i$ either is finite or is contained in $\zeta _1(G/G_i)$. The claim is obvious if $t\le 1$. Suppose now $t>1$. By induction the factor group $G/G_1$ is finite-by-nilpotent, so without loss of generality we may assume that $G_1$ is infinite and hence $G_1\le \zeta _1(G)$. Since $\zeta _k(G/G_1)$ has finite index in $G/G_1$ for some non-negative integer k (see [21, Part 1, Theorem 4.25]), it follows that $|G:\zeta _{k+1}(G)|$ is finite and hence G is finite-by-nilpotent (see [21, Part 1, Corollary 2 to Theorem 4.21]). $\square$

The above lemma focuses the attention on poly-(finite or trivial) modules over a group, i.e. modules admitting a finite series of submodules whose factors are either finite or trivial. Since finite-by-nilpotent groups do not form a Hall class, we already know this class cannot be tensorial (see Theorem C). This can also be seen from the following easy example: if $\langle a\rangle$ is a cyclic group of odd prime order, F is a free abelian group of infinite rank and $\langle x\rangle$ is a group of order 2 acting trivially on F and as the inversion on $\langle a\rangle$, the tensor product $\langle a\rangle \otimes _{\mathbb {Z}}F$ is not poly-(finite or trivial) as an $\langle x\rangle$-module.

If G is a group and ${\mathcal {M}}$ is a class of G-modules, we denote by $\mathfrak {P}({{\mathcal {M}}})$ the class of all G-modules admitting a finite series of G-submodules each of whose factors either is finite or belongs to ${\mathcal {M}}$, and by $\textbf{L}{\mathcal {M}}$ the class of all G-modules M such that each finite subset of M is contained in a G-submodule that belongs to ${\mathcal {M}}$.

Although the class of poly-(finite or trivial) G-modules is not tensorial, the first part of the following general lemma shows in particular that the addition of a suitable finiteness condition makes it a tensorial class; the second part will be useful in the last part of the proof of Theorem D.

Lemma 5.2

Let G be a group and let ${\mathcal {M}}$ be a tensorial class of G-modules.

• (a) If all members of ${\mathcal {M}}$ are finitely generated as $\mathbb {Z}$-modules, the class $\mathfrak {P}({{\mathcal {M}}})$ is tensorial.

• (b) If ${\mathcal {M}}$ is closed with respect to forming sections, then the class $\textbf{L}{\mathcal {M}}$ is tensorial.

Proof

(a) Let A and B be G-modules in the class $\mathfrak {P}({{\mathcal {M}}})$. Then there are finite chains of G-submodules

$$\begin{aligned} \{0\}=A_0\le A_1\le \cdots \le A_h=A \quad \text {and}\quad \{0\}=B_0\le B_1\le \cdots \le B_k=B \end{aligned}$$

in which every factor either is finite or belongs to ${\mathcal {M}}$. For each non-negative integer $s\le h+k$, denote by $C_s$ the G-submodule of $A\otimes _{\mathbb {Z}}B$ generated by all elements of the form $a\otimes b$ with $a\in A_i$, $b\in B_j$ and $i+j=s$. Then

is a finite chain of G-submodules of $A\otimes _{\mathbb {Z}}B$ and for $s<h+k$ there exists a G-epimorphism

$$\begin{aligned} \bigoplus _{(i+1)+(j+1)=s+1}\bigl (A_{i+1}/A_i\bigr )\otimes _{\mathbb {Z}}\bigl (B_{j+1}/B_j\bigr )\longrightarrow C_{s+1}/C_s. \end{aligned}$$

Since any tensor product of a finite G-module and a G-module which is finitely generated as a $\mathbb {Z}$-module is finite, it follows that the chain $(\star )$ can be refined to a finite chain of G-submodules in which every factor either is finite or belongs to ${\mathcal {M}}$. Thus  $\mathfrak {P}({{\mathcal {M}}})$ is a tensorial class.

(b) Suppose that A and B are elements of $\textbf{L}{\mathcal {M}}$, and let E be any finitely generated G-submodule of $A\otimes _{\mathbb {Z}}B$. Clearly there exist finitely generated G-submodules $A_0$ of A and $B_0$ of B such that E is isomorphic as a G-module to a section of $A_0\otimes _{\mathbb {Z}}B_0$. Since $A_0$ and $B_0$ belong to the tensorial class ${\mathcal {M}}$, also E is in ${\mathcal {M}}$. Therefore $A\otimes _{\mathbb {Z}}B$ belongs to $\textbf{L}{\mathcal {M}}$ and hence $\textbf{L}{\mathcal {M}}$ is tensorial. $\square$

Let G be a group and ${\mathcal {M}}$ a class of G-modules. We say that a normal section H/K of G is $P_{\mathfrak {F}}({{\mathcal {M}}})$ if it has a finite G-invariant series each of whose factors either is finite or is an abelian group that belongs to ${\mathcal {M}}$ when regarded as a G-module by conjugation. Of course, if an abelian normal subgroup A of G is $P_{\mathfrak {F}}({{\mathcal {M}}})$, then it belongs to $\mathfrak {P}({{\mathcal {M}}})$, when considered as a G-module by conjugation. Moreover, we say that G is $\textbf{L}P_{\mathfrak {F}}({{\mathcal {M}}})$ if every finitely generated subgroup E of G is $P_{\mathfrak {F}}({{\mathcal {M}}}_E)$, where ${{\mathcal {M}}}_E$ is the class ${\mathcal {M}}$ itself, but considering its members as E-modules.

Lemma 5.3

Let G be a group and let ${\mathcal {M}}$ be a tensorial class of G-modules which is closed with respect to forming submodules and whose members are finitely generated as $\mathbb {Z}$-modules. If G contains a nilpotent normal subgroup N such that $G/N'$ is $P_{\mathfrak {F}}({{\mathcal {M}}})$, then G is $P_{\mathfrak {F}}({{\mathcal {M}}})$.

Proof

For every positive integer i, consider the factor group $\gamma _i(N)/\gamma _{i+1}(N)$ as a G-module by conjugation. By hypothesis $N/N'$ is $P_{\mathfrak {F}}({{\mathcal {M}}})$ and so belongs to $\mathfrak {P}({{\mathcal {M}}})$, which is a tensorial class by Lemma 5.2 (a). It follows that each $\gamma _i(N)/\gamma _{i+1}(N)$ lies in $\mathfrak {P}({{\mathcal {M}}})$ and hence is $P_{\mathfrak {F}}({{\mathcal {M}}})$. Therefore G itself is $P_{\mathfrak {F}}({{\mathcal {M}}})$. $\square$

Theorem 5.4

Finite-by-(nilpotent and finitely generated) groups form a Hall class.

Proof

Let G be a group containing a nilpotent normal subgroup N such that $G/N'$ is finitely generated and finite-by-nilpotent. Application of Lemma 5.3 to the tensorial class consisting of all G-modules which are finitely generated as $\mathbb {Z}$-modules and trivial as G-modules yields that G has a normal series of finite length whose infinite factors are central and finitely generated. Thus G is finite-by-nilpotent by Lemma 5.1. $\square$

In order to obtain an analogue of Lemma 5.1, we need the following characterization of finite-by-supersoluble groups, which should be seen in relation to the classical result of Baer and Hall, stating that a group is finite-by-nilpotent if and only if it is finite over some term with finite ordinal type of its upper central series. In what follows, a normal subgroup N of a group G is said to be supersolubly embedded in G if it has a finite G-invariant series with cyclic factors. Recall also that, if G is a polycyclic-by-finite group, the Hirsch number of G is the largest number of infinite cyclic factors among all finite series of G.

Lemma 5.5

A group G is finite-by-supersoluble if and only if it contains a supersolubly embedded subgroup of finite index.

Proof

Suppose first that G has a finite normal series

$$\begin{aligned} \{1\}=N_0\le N_1\le \cdots \le N_t=N\le G, \end{aligned}$$

where $N_{i+1}/N_i$ is cyclic for each $i<t$ and G/N is finite. Since G is polycyclic-by-finite, its largest periodic normal subgroup T is finite and hence the replacement of G by G/T allows us to assume $T=\{1\}$. Thus $N_1$ is infinite cyclic and so by induction on the Hirsch number of G we have that $G/N_1$ is finite-by-supersoluble. If $E/N_1$ is a finite normal subgroup of $G/N_1$ such that G/E is supersoluble, it easily follows that E is either cyclic or infinite dihedral. Thus G is supersoluble.

Suppose conversely that G is finite-by-supersoluble and let T be its largest periodic normal subgroup. Then T is finite and G/T is supersoluble, so G/T contains an infinite cyclic normal subgroup A/T. Of course, A has an infinite cyclic subgroup B which is characteristic, and so normal in G. By induction on the Hirsch number of G, the factor group G/B contains a supersolubly embedded subgroup S/B of finite index, and the proof is complete since S is supersolubly embedded in G. $\square$

Corollary 5.6

A group is finite-by-supersoluble if and only it has a normal series of finite length whose factors are either finite or cyclic.

Proof

The proof is similar to that of Lemma 5.1: one just needs to replace the results quoted from [21] by Lemma 5.5. $\square$

Theorem 5.7

Finite-by-supersoluble groups form a Hall class.

Proof

Let G be a group containing a nilpotent normal subgroup N such that $G/N'$ is finite-by-supersoluble. Application of Lemma 5.3 to the tensorial class consisting of all G-modules which are either finite or cyclic as $\mathbb {Z}$-modules yields that G has a normal series of finite length whose factors are either finite or cyclic. Thus G is finite-by-supersoluble by Corollary 5.6. $\square$

In order to complete the proof of Theorem D, we need the following lemma. If ${\mathcal {M}}$ is a class of modules over a group G and N is a normal subgroup of G, we use the symbol ${{\mathcal {M}}}_{G/N}$ to denote the class of G/N-modules obtained by elements of ${\mathcal {M}}$ upon which N acts trivially.

Lemma 5.8

Let G be a group and let ${\mathcal {M}}$ be a tensorial class of G-modules which is closed with respect to forming sections and whose members are finitely generated as $\mathbb {Z}$-modules. If G contains a nilpotent normal subgroup N such that $G/N'$ is $\textbf{L}P_{\mathfrak {F}}({{\mathcal {M}}}_{G/N'})$, then G is $\textbf{L}P_{\mathfrak {F}}({{\mathcal {M}}})$.

Proof

Let E be any finitely generated subgroup of G. Since the section $EN'/N'$ is $P_{\mathfrak {F}}({{\mathcal {M}}}_{EN'/N'})$, we have that $E/(E\cap N')$ is $P_{\mathfrak {F}}({{\mathcal {M}}}_E)$. Suppose that $E/\bigl (E\cap \gamma _i(N)\bigr )$ is $P_{\mathfrak {F}}({{\mathcal {M}}}_E)$ for some $i\ge 2$. In particular, $E/\bigl (E\cap \gamma _i(N)\bigr )$ is polycyclic-by-finite and so

$$\begin{aligned} \bigl (E\cap \gamma _i(N)\bigr )\gamma _{i+1}(N)/\gamma _{i+1}(N)\simeq _E\bigl (E\cap \gamma _i(N)\bigr )/\bigl (E\cap \gamma _{i+1}(N)\bigr ) \end{aligned}$$

is finitely generated as an E-module (see for instance [21, Part 1, Lemma 1.43]). But $N/N'\in \textbf{L}\mathfrak {P}(\mathcal{M}_E)$, so $\gamma _i(N)/\gamma _{i+1}(N)$ is in $\textbf{L}\mathfrak {P}({{\mathcal {M}}}_E)$ by Lemma 5.2 (b) and hence $\bigl (E\cap \gamma _i(N)\bigr )/\bigl (E\cap \gamma _{i+1}(N)\bigr )\in {\mathfrak {P}}({{\mathcal {M}}}_E)$. It follows that $E/\bigl (E\cap \gamma _{i+1}(N)\bigr )$ is $P_{\mathfrak {F}}({\mathcal {M}}_E)$. Since N is nilpotent, the subgroup E is $P_{\mathfrak {F}}({\mathcal {M}}_E)$ and hence G is $\textbf{L}P_{\mathfrak {F}}({{\mathcal {M}}})$. $\square$

Theorem 5.9

The class of locally (finite-by-nilpotent) groups and the class of locally (finite-by-supersoluble) groups are Hall classes.

Proof

Let G be a group containing a nilpotent normal subgroup N such that $G/N'$ is in $\textbf{L}(\mathfrak {FN})$, and let ${\mathcal {M}}$ be the class of all finitely generated trivial G-modules. Of course, ${\mathcal {M}}$ is tensorial and $G/N'$ is $\textbf{L}P_{\mathfrak {F}}({{\mathcal {M}}}_{G/N'})$. Then it follows from Lemma 5.8 that G is $\textbf{L}P_{{\mathfrak {F}}}({{\mathcal {M}}})$ and so Lemma 5.1 gives that G is locally (finite-by-nilpotent). Therefore $\textbf{L}(\mathfrak {FN})$ is a Hall class.

The proof for the class of locally (finite-by-supersoluble) groups is similar, it suffices to choose ${\mathcal {M}}$ to be the class of G-modules that are cyclic as $\mathbb {Z}$-modules and to replace Lemma 5.1 by Lemma 5.6. $\square$

Finally, note that the class of locally (nilpotent-by-finite) groups, which coincides with the class of locally (supersoluble-by-finite) groups, is a Hall class (see for instance [1]).