1 Introduction

All groups considered in the present paper are supposed to be finite. The solubility graph \(\Gamma _S(G)\) associated with a group G is a simple graph whose vertices are the elements of G,  and there is an edge between two distinct elements x and y if and only if the subgroup \(\langle x,y \rangle \) is soluble. In [15] Thompson proved that a finite group G is soluble if and only if for every \(x, y \in G\) the subgroup \(\langle x,y \rangle \) is soluble. This implies that a finite group G is soluble if and only if the graph \(\Gamma _S (G)\) is complete. We denote by R(G) the soluble radical of a finite group G,  that is, the largest soluble normal subgroup of G. In [8] Guralnick et al. proved that if x is an element of G,  then \(x \in R(G)\) if and only if the subgroup \(\langle x,y \rangle \) is soluble for all \(y \in G.\) This means that \(x \in R(G)\) if and only if x is a universal vertex of \(\Gamma _S(G),\) that is a vertex being adjacent to every other vertex in the graph. The subgraph of \(\Gamma _S(G)\) obtained by removing all vertices in R(G) is connected when G is a finite group (see [1]). Furthermore, it has been proved in [3] that its diameter is at most 5 in general, and at most 3 if G is not almost simple.

For all \(x \in G,\) the neighborhood of x in \(\Gamma _S(G)\) is called the solubilizer of x in G,  and it is denoted by \(\mathrm {Sol}_G(x).\) Then

$$\begin{aligned} \mathrm {Sol}_G(x) = \{y \in G \ \mid \ \langle x,y \rangle \hbox { is soluble}\}. \end{aligned}$$

The aim of this paper is to continue the investigation of properties of the solubilizer of an element in a finite group, started in [1, 9].

In Sect. 2 we collect known properties of the solubilizer of a vertex in the graph \(\Gamma _S(G).\) In general \(\mathrm {Sol}_G(x)\) is just a subset of G and not a subgroup. However, it can happen that \(\mathrm {Sol}_G(x)\) is a subgroup. For instance, the above result in [8] yields that \(x \in R(G)\) if and only if \(\mathrm {Sol}_G(x)=G.\) On the other side, if \(\mathrm {Sol}_G(x)\) is a subgroup of G it does not yield \(x \in R(G).\) Therefore the fact that a single solubilizer is a subgroup does not imply specific restrictions on the structure of the whole group. The situation is quite different when all solubilizers are subgroups. Indeed, in [1] it has been shown that a group G is soluble if and only if \(\mathrm {Sol}_G(x)\) is a subgroup of G for all \(x \in G.\)

It is an interesting problem to find algebraic conditions on the elements of a single solubilizer determining restrictions on the structure of the whole group. In [1] the authors proved that if G is a group having an element x such that the elements of \(\mathrm {Sol}_G(x)\) commute pairwise, then G is abelian. In Sect. 3 we show a generalization of this result. Given an integer \(k \ge 2\) and elements \(x_1, \ldots , x_k\) of a group G,  the long commutator of weight k is inductively defined by the formulae

$$\begin{aligned}{}[x_1,x_2]=x_1^{-1}x_2^{-1}x_1x_2,\qquad [x_1, \ldots , x_k]=[[x_1, \ldots , x_{k-1}], x_k]\quad (k>2). \end{aligned}$$

The subgroup \(\gamma _k(G)\) generated by all long commutators of weight k is the kth term of the lower central series of G. When rephrased in terms of long commutators, the above result in [1] asserts that \(\gamma _2(G)=1\) if and only if there exists an element \(x \in G\) such that \([x_1,x_2]=1\) for all \(x_1,x_2 \in \mathrm {Sol}_G(x).\) Therefore the following question arises naturally.

Question 1

Let G be a group,  \(x \in G\) and \(k \ge 3.\) If \([u_1, \ldots , u_k]=1\) for every \(u_1, \ldots , u_k \in \mathrm {Sol}_G(x),\) is \(\gamma _k(G)=1?\)

Notice that a positive answer would provide a nilpotency criterion for finite groups since the converse is obviously true. Our main result in Sect. 3 is an affirmative answer to Question 1 when \(k=3.\)

Theorem A

Let G be a finite group. Then G is nilpotent of class at most 2 if and only if there exists an element \(x \in G\) such that \([u_1,u_2,u_3]=1\) for every \(u_1,u_2,u_3 \in \mathrm {Sol}_G(x).\)

Furthermore, for \(k \ge 4\) we point out structural restrictions for a minimal (with respect to the cardinality) non-nilpotent group G containing an element x such that \([x_1, \ldots , x_k]=1\) for all \(x_1, \ldots , x_k \in \mathrm {Sol}_G(x).\)

In [1] it has been shown that if x is an element of an insoluble group G then the cardinality of \(\mathrm {Sol}_G(x)\) cannot be a prime. Furthermore, if R(G) is not trivial, then the cardinality of \(\mathrm {Sol}_G(x)\) cannot be a square of a prime. These are the first answers to the following arithmetic question related to the graph \(\Gamma _S(G).\)

Question 2

Let G be a group,  and let x be an element of G. Which positive integers can occur as the cardinality of \(\mathrm {Sol}_G(x)?\)

In Sect. 4 we prove that the above-mentioned result in [1] remains true if the hypothesis \(R(G) \ne 1\) is dropped.

Theorem B

Let G be an insoluble group and x an element of G. Then the cardinality of \(\mathrm {Sol}_G(x)\) cannot be equal to \(p^2\) for any prime p.

Furthermore, we state restrictions for a prime p when the cardinality of \(\mathrm {Sol}_G(x)\) is equal to 3p. Finally, we show that if x is any element of an insoluble group G then the set \(\mathrm {Sol}_G(x)\) has cardinality at least 10.

2 Preliminary Results

In this section, we collect some properties and results which will be useful in the sequel. From now on, if X is any subset of a finite group G,  the cardinality of X will be denoted by |X|. Moreover, for an element \(x\in G,\) o(x) will denote the order of x,  and \(C_G(x)\) the centralizer of x in G. Finally, for a subgroup H of G,  \(N_G(H)\) and \(C_G(H)\) will denote the normalizer and the centralizer of H in G, , respectively. We start by mentioning some results about the solubilizer of an element in a finite group.

Lemma 2.1

[1] Let G be a group and \(x \in G.\) Then : 

  1. (a)

    \( \langle x \rangle \subseteq C_G(x) \subseteq N_G(\langle x \rangle ) \subseteq N_G(\langle x \rangle ) \cup R(G) \subseteq \mathrm {Sol}_G(x);\)

  2. (b)

    \(\mathrm {Sol}_G(x)\) is the union of all soluble subgroups of G containing x; 

  3. (c)

    \(|\mathrm {Sol}_G(x)|\) is divisible by o(x).

Lemma 2.2

[9] Let G be a group and \(x \in G.\) Then \(|C_G(x)|\) divides \(|\mathrm {Sol}_G(x)|.\)

If N is a normal soluble subgroup of G,  we define

$$\begin{aligned} \frac{\mathrm {Sol}_G(x)}{N} = \{yN \mid y \in \mathrm {Sol}_G(x)\} = \{yN \mid \langle x,y \rangle \hbox { is soluble} \}. \end{aligned}$$

In many situations, the following result enables to reduce to the case when the soluble radical is trivial.

Lemma 2.3

[9] If N is a normal soluble subgroup of a group G,  then \(\mathrm {Sol}_G(x)\) is the union of cosets of N,  and \(\mathrm {Sol}_{G/N}(xN)=\mathrm {Sol}_G(x)/N.\) In particular, 

$$\begin{aligned} \left| \frac{\mathrm {Sol}_G(x)}{N}\right| = \frac{|\mathrm {Sol}_G(x)|}{|N|}. \end{aligned}$$

In what follows, some famous results are needed that show the existence of a nilpotent maximal subgroup strongly affects the structure of a finite group. We collect them here for the reader convenience.

Theorem 2.4

[11] Let G be a finite group having a nilpotent maximal subgroup M. If a Sylow 2-subgroup of M has class at most 2,  then G is soluble.

As a consequence, we have the following.

Corollary 2.5

Let G be a finite group and let M be a nilpotent subgroup of G. If the Sylow 2-subgroup of M has class at most 2 and M is not properly contained in any soluble subgroup of G,  then \(G=M.\)

In particular, Corollary 2.5 holds when the subgroup M has odd order. Thus a central role in the description of finite groups with a nilpotent maximal subgroup is played by Sylow 2-subgroups. This is emphasized by the following.

Theorem 2.6

[14] Suppose that G is a finite insoluble group having a nilpotent maximal subgroup M. If \(Z(G)=1,\) then M is a Sylow 2-subgroup of G.

Furthermore, in [2] Baumann showed that only some Sylow 2-subgroups are admitted.

Theorem 2.7

[2] Let G be a finite insoluble group having a nilpotent maximal subgroup. Let \(L = \mathrm {Fit}(G)\) be the Fitting subgroup of G. Then G/L has a unique minimal normal subgroup K/L,  which is a direct product of copies of a simple group with dihedral Sylow 2-subgroups,  and G/K is a 2-group.

On the other side, the groups with dihedral Sylow 2-subgroups have been characterized in [7]. As a consequence, we have the following.

Theorem 2.8

[7] If G is a simple group with dihedral Sylow 2-subgroups, then either G is isomorphic to the projective special linear group \(\mathrm {PSL}(2, q),\) q odd and \(q \ge 5,\) or G is isomorphic to the alternating group \(\mathrm {Alt}(7).\)

3 Proof of Theorem A

We start with the following crucial observation.

Lemma 3.1

Let G be a group,  \(x \in G\) and \(k \ge 2.\) If \([u_1, \ldots , u_k]=1\) for every \(u_1, \ldots , u_k \in \mathrm {Sol}_G(x),\) then \(\mathrm {Sol}_G(x)\) is a subgroup. Moreover, it is nilpotent of class at most \(k-1.\)

Proof

To prove that \(\mathrm {Sol}_G(x)\) is a subgroup, consider elements \(y,z \in \mathrm {Sol}_G(x),\) and write \(L=\langle x,y,z \rangle .\) Then \(\gamma _k(L)\) is generated by long commutators of weight at least k with entry set \(\{x,y,z,x^{-1},y^{-1},z^{-1}\}\) (see, for instance, [12, 2.1.5]), and the latter are trivial by hypothesis. Thus \(\gamma _k(L)=1.\) On the other hand, \(\langle yz, x\rangle \le L\) which is nilpotent of class at most \(k-1.\) Then \(\mathrm {Sol}_G(x)\) is a subgroup and the result follows. \(\square \)

Now we are in a position to prove Theorem A stated in Sect. 1.

Theorem A

Let G be a finite group. Then G is nilpotent of class at most 2 if and only if there exists an element \(x \in G\) such that \([u_1,u_2,u_3]=1\) for every \(u_1,u_2,u_3 \in \mathrm {Sol}_G(x).\)

Proof

By applying Lemma 3.1 for \(k=3,\) we get that \(\mathrm {Sol}_G(x)\) is a nilpotent subgroup of class at most 2. Moreover, \(R(G)=\mathrm {Fit}(G)\) as R(G) is a subgroup of \(\mathrm {Sol}_G(x).\) Since \(\mathrm {Sol}_G(x)\) is not properly contained in any soluble subgroup of G,  Corollary 2.5 yields that \(G=\mathrm {Sol}_G(x),\) and we are done.

It is not clear whether the answer to Question 1 is in the affirmative for \(k>3.\) Nevertheless, if it is not the case, the structure of a minimal counterexample is subject to significant restrictions as shown in the following.

Remark 3.2

Let \(k\ge 4,\) and assume that G is a minimal (with respect to the order) insoluble group having an element x such that \([u_1, \ldots ,u_k]=1\) for every \(u_1,\ldots ,u_k \in \mathrm {Sol}_G(x).\) Then G has a unique minimal normal subgroup \(K = S \times \cdots \times S\) where S is isomorphic either to \(\mathrm {PSL}(2, q),\) q odd and \(q \ge 5,\) or to the alternating group \(\mathrm {Alt}(7).\) Furthermore, \(G = K\langle x \rangle \) and \(\mathrm {Sol}_G(x)\) is a Sylow 2-subgroup of G.

Proof

By Lemma 3.1, \(\mathrm {Sol}_G(x)\) is a subgroup of G and it is nilpotent of class at most \(k-1.\) Hence \(R(G)=\mathrm {Fit}(G).\) Set \(T=\mathrm {Sol}_G(x).\) Since G is not soluble, T is a maximal subgroup by part (b) of Lemma 2.1. If \(R(G) \ne 1\) then G/R(G) is a group of smaller order with \(\mathrm {Sol}_{G/R(G)}(xR(G))\) satisfying our hypotheses. This implies that G/R(G) is soluble, giving the contradiction that G is soluble. Therefore \(R(G)=\mathrm {Fit}(G)=Z(G)=1.\) Now Theorem 2.6 implies that T is a Sylow 2-subgroup of G of order at most \(2^{k},\) and Theorem 2.7 yields the existence of a unique minimal normal subgroup K of G such that \(K= S \times \cdots \times S,\) where S is a non-abelian simple group with dihedral Sylow 2-subgroups, and G/K is a 2-group. Therefore, Theorem 2.8 implies that either S is isomorphic to \(\mathrm {PSL}(2, q),\) q odd and \(q \ge 5,\) or it is isomorphic to the alternating group \(\mathrm {Alt}(7).\)

By [10, Theorem 2.13] x is not an involution. Let \(H= \langle x, K \rangle .\) We claim that \(G=H.\) Indeed, if H is a proper subgroup of G,  then \(\mathrm {Sol}_H(x)=H \cap \mathrm {Sol}_G(x)\) satisfies our hypotheses, so H is soluble by the minimality of G. Therefore \(G= K \langle x \rangle .\) Let \(P= T \cap K\) be a Sylow 2-subgroup of K. By the Dedekind’s Modular Law we have

$$\begin{aligned} T= G \cap T = \langle x \rangle K \cap T = \langle x \rangle (K \cap T)= \langle x \rangle P. \end{aligned}$$

By Theorem 2.7, P is a direct product of dihedral groups. \(\square \)

4 Proof of Theorem B

In this section, we deal with arithmetic questions related to the solubilizer of an element in a finite group.

Lemma 4.1

Let G be an insoluble group and let x be an element of G such that \(\mathrm {Sol}_G(x)\) is a subgroup. Then \(|\mathrm {Sol}_G(x)| \ne p^n\) for all odd primes p and all positive integers n.

Proof

Arguing by contradiction, assume that \(|\mathrm {Sol}_G(x)|= p^n\) for some odd prime p and a positive integer n. Then \(\mathrm {Sol}_G(x)\) is a Sylow p-subgroup of G. Moreover, \(\mathrm {Sol}_G(x)\) is not properly contained in any soluble subgroup of G. From Corollary 2.5 it follows that \(G=\mathrm {Sol}_G(x),\) giving the contradiction that G is soluble. \(\square \)

The following result shows that the solubilizer of an element of prime order has to be large enough when it is not equal to the normalizer.

Lemma 4.2

Let G be a finite group,  and let \(x \in G\) be an element of prime order p. If \(P=\langle x \rangle \) and \(|\mathrm {Sol}_G(x)|\le p^2,\) then \(\mathrm {Sol}_G(x)=N_G(P).\)

Proof

Since \(N_G(P) \subseteq \mathrm {Sol}_G(x)\) by part (a) of Lemma 2.1, we only need to prove the reverse inclusion. As \(\mathrm {Sol}_G(x)\) is the union of all soluble subgroups of G containing x,  it suffices to show that if H is a soluble subgroup containing x,  then \(H \le N_G(P).\) Let H be a soluble subgroup containing x. Then \(|H| \le p^2.\) If H is a p-group, then H is abelian, and so \(H \le N_G(P).\) Therefore assume that H is not a p-group. It follows that \(|H: P| < p,\) and \(|H:N_H(P)|\) is congruent to 1 modulo p because P is a Sylow p-subgroup of H. Hence, \(H = N_H(P) \le N_G(P).\) This yields \(\mathrm {Sol}_G(x) \le N_G(P)\) as desired. \(\square \)

Now we are in a position to prove Theorem B stated in Sect. 1.

Theorem B

Let G be an insoluble group and x an element of G. Then the cardinality of \(\mathrm {Sol}_G(x)\) cannot be equal to \(p^2\) for any prime p.

Proof

Arguing by contradiction, assume that \(|\mathrm {Sol}_G(x)|=p^2\) for some prime p. It follows that o(x) divides \(p^2,\) and \(x \in P\) for some Sylow p-subgroup P of G. Clearly, \(p^2\) does not divide |G|,  otherwise \(P = \mathrm {Sol}_G(x)\) and G is abelian by [1, Theorem 1.2]. Therefore, we can assume that \(P= \langle x \rangle \) has cardinality p. By Lemma 4.2 we get \(\mathrm {Sol}_G(x)=N_G(P),\) which is a contradiction because there are no subgroups of cardinality \(p^2\) in G. The proof is complete. \(\square \)

As a consequence of Theorem B we have the following.

Corollary 4.3

Let G be an insoluble group and x an element of G. If \(R(G) \ne 1,\) then \(|\mathrm {Sol}_G(x)| \ne p^3\) for all primes p.

Proof

Arguing by contradiction, assume that \(|\mathrm {Sol}_G(x)| = p^3\) for some \(x \in G\) and p prime. Then, from Lemma 2.3 it follows that \(\mathrm {Sol}_{G/R(G)}(xR(G))\) has order p or \(p^2,\) which is a contradiction by Theorem B and [1, Corollary 3.2 (b) and Corollary 3.3]. \(\square \)

Actually, we suspect that Corollary 4.3 holds true also when \(R(G)=1.\) As a consequence of Theorem A we are able to prove the above conjecture for \(p=2.\)

Proposition 4.4

Let G be an insoluble group. Then \(|\mathrm {Sol}_G(x)| \ne 8\) for every element \(x \in G.\)

Proof

Assume \(|\mathrm {Sol}_G(x)|=8.\) Since G is not soluble, \(R(G) \ne G.\) Moreover, if \(R(G) \ne 1\) then \(\mathrm {Sol}_{G/R(G)}(xR(G))= \mathrm {Sol}_G(x)/R(G)\) has cardinality 2 or 4,  which is impossible by Theorem B and [1, Theorem 1.2]. Hence we can assume \(R(G)=1.\) As o(x) divides \(|\mathrm {Sol}_G(x)|,\) we can consider a Sylow 2-subgroup of G,  say P,  containing x. Since G is insoluble, it follows that G is not 2-nilpotent. Thus, applying [13, 10.1.9], we can assume that P is not cyclic and \(4 \le |P| \le 8.\)

By Frobenius’ normal p-complement theorem (see, for instance, [6, Theorem 7.4.5]), there exist a 2-subgroup H of G and an element \(b \in N_G(H) \setminus C_G(H)\) of odd order. Then \(|H| \ge 4,\) otherwise \(N_G(H)=C_G(H).\)

As \(H \le P^g \le \mathrm {Sol}_G(x)^g=\mathrm {Sol}_G(x^g)\) for some \(g\in G,\) without loss of generality we can assume that \(H \le P.\) If \(H=P,\) then \(\langle H,b \rangle \subseteq \mathrm {Sol}_G(x),\) which is impossible since \(|\langle H,b \rangle | >8.\) Then \(|H|=4\) and \(|P|=8,\) which implies \(\mathrm {Sol}_G(x)=P.\) Then \(\mathrm {Sol}_G(x)\) has nilpotency class at most 2,  so G is nilpotent of class at most 2 by Theorem A, which is a contradiction. This concludes the proof. \(\square \)

Lemma 4.5

Let G be a finite group with trivial soluble radical and let x be a self-centralizing element of G of order 3. Then \(|\mathrm {Sol}_G(x)| \in \{24,78\}.\)

Proof

Since \(R(G)=1,\) by the main theorem of [4] we deduce that G is isomorphic either to the alternating group \(\mathrm {Alt}(5),\) or to the projective special linear group \(\mathrm {PSL}(2,7).\) Since all elements of order 3 are conjugate in the above groups, using the GAP system [5] one can observe that in the former case \(|\mathrm {Sol}_{G}(x)|=24\) for \(x=(1\ 2\ 3),\) while in the latter \(|\mathrm {Sol}_G(x)|=78\) for

$$\begin{aligned} x=\begin{pmatrix} 0 &{}\quad 1 &{}\quad 0\\ 1 &{}\quad 1 &{}\quad 0\\ 0 &{}\quad 0 &{}\quad 1 \end{pmatrix}. \end{aligned}$$

Proposition 4.6

Let G be an insoluble group. Then \(|\mathrm {Sol}_G(x)| \ne 6\) for every element \(x \in G.\)

Proof

Assume that \(|\mathrm {Sol}_G(x)|=6.\) Then o(x) divides 6. By [1, Theorem 1.2], we can assume \(o(x) \ne 6.\) If \(o(x)=3,\) Lemma 4.2 implies that \(\mathrm {Sol}_G(x)=N_G(x).\) In particular \(C_G(x)=\langle x \rangle \) and x is a self-centralizing element of G. Moreover, the soluble radical R(G) of G is trivial, otherwise \(\mathrm {Sol}_{G/R(G)}(xR(G))\) would have prime order. Then, by Lemma 4.5 we can conclude that \(|\mathrm {Sol}_G(x)| \ne 6,\) which is a contradiction.

Now assume \(o(x)=2.\) Let P be a Sylow 2-subgroup of G containing x. Since G is not 2-nilpotent and \(|\mathrm {Sol}_G(x)|=6,\) P is elementary abelian of order 4. Now, for every \(y \in G\) such that \(o(y)=2,\) the group \(\langle x,y \rangle \) is a dihedral group and thus \(y\in \mathrm {Sol}_G(x).\) Therefore every Sylow 2-subgroup of G is contained in \(\mathrm {Sol}_G(x).\) Let \(n_2\) be the number of Sylow 2-subgroups of G. Of course \(n_2 >1\) because G is not soluble. Hence \(n_2 \ge 3,\) and \(\mathrm {Sol}_G(x)\) contains more than 6 elements, our final contradiction. \(\square \)

As a consequence of Theorem B, Propositions 4.4 and 4.6 we have the following.

Corollary 4.7

Let G be an insoluble group. Then \(|\mathrm {Sol}_G(x)| \ge 10\) for all elements \(x \in G.\)

We point out that if \(G=\mathrm {Alt}(5)\) is the alternating group any element \(x \in G\) of order 5 has \(\mathrm {Sol}_G(x)=N_G(x)\) with \(|N_G(x)|=10.\) Therefore it is possible to have \(|\mathrm {Sol}_G(x)|=pq\) with \(p>q\) primes. However, there are some restrictions, as shown in the following.

Proposition 4.8

Let G be an insoluble group,  and let x be any element of G such that \(|\mathrm {Sol}_G(x)|=3p\) where p is a prime. Then \(o(x)=p,\) \(p \equiv 1 \pmod 3\) and \(\mathrm {Sol}_G(x)=N_G(x).\)

Proof

First of all, we can assume \(R(G)=1,\) otherwise \(\mathrm {Sol}_{G/R(G)}(xR(G))\) is a prime against [1, Corollary 3.2 (b)]. By Proposition 4.6 and Theorem B we can assume \(p \ge 5.\) Assume \(o(x)=3.\) Since \(|C_G(x)|\) divides 3p,  we have \(|C_G(x)|=3\) and we get a contradiction by Lemma 4.5. Therefore \(o(x)=p\) and from Lemma 4.2 it follows that \(\mathrm {Sol}_G(x)=N_G(x).\) Finally, by [1, Theorem 1.2] we obtain \(p \equiv 1 \pmod 3,\) and we are done. \(\square \)