Primitive permutation groups satisfying the small orbit property and a problem of Bourgain and Kalai

Abstract

We classify the primitive but not affine permutation subgroups of $S_n$ with the following property: There exists some unbounded subset X of size at most $\hbox {clog}n$ whose orbit is polynomially bounded (in terms of n). This class of subgroups arises naturally in the study of threshold behavior of G-symmetric properties of Boolean functions. Our result answers a problem of Bourgain and Kalai. Contrary to a suggestion in their paper, we give an example for a primitive permutation group with this property, which contains ${A_{m}}$ as a section for $m \rightarrow \infty$.

1 Introduction

Let $I = \{n_t|t \in \mathbb {N}\}$ be a sequence of the natural numbers $\mathbb {N}$. For a sequence $G_t \le S_{n_t}$$(t \in \mathbb {N})$ of primitive permutation groups, we say that $\{G_t\}$ has the $small \,\ orbit \,\ property$ if there exist some constants b and c and some subsets $X_t \subseteq \{1, \ldots , n_t\}$ such that $\{ |X_t| \}$ is not bounded, for all t$|X_t| \le \hbox {clog}n_t$, and $|G_t(X_t)| \le {n_t}^b$.

Consider the following example:

Let G be $A_n$ or $S_n$ acting on $\varOmega = \{1, \ldots , n \}$ ($n \ge 3$). Then, G acts transitively on the set of all subsets of $\varOmega$ of size k for any k, and therefore, the size of the orbit of any subset X of size k is ${n \atopwithdelims ()k}$. Clearly, the $small \,\ orbit \,\ property$ is not satisfied.

A more interesting example is the case where $S_n$ acts on $\varOmega ={{\{1,2,\dots ,n\}} \atopwithdelims (){2}}$ (namely all 2-subsets of $\{1, \ldots , n \}$), and in this case, the orbits correspond to isomorphism classes of graphs. Note that also here the $small \,\ orbit \,\ property$ is not satisfied.

Let $G_t$ ($t \in \mathbb {N}$) be a sequence of finite groups. Let $I = \{n_t | t \in \mathbb {N} \}$. Bourgain and Kalai posed the question of understanding primitive permutation groups with the $small \,\ orbit \,\ property$ (see [1]). They proposed that the following are equivalent for an infinite set I of natural numbers and a sequence of groups $G_t$ ($t \in I$):

• 1. For any embedding of $G_t$ in $S_{n_t}$ such that the action on $\{1, \ldots , n_t\}$ is primitive, $G_t$ satisfies the $small \,\ orbit \,\ property$.

• 2. There exists some M, such that $G_t$ does not have $A_m$ as a composition factor for m greater than M.

The motivation for studying permutation groups with the $small \,\ orbit \,\ property$ comes from studies of threshold phenomena under symmetry and, in particular, a paper by Bourgain and Kalai [1].

Consider a monotone Boolean function $f(x_1,x_2,\ldots ,x_n)$. Assume that f is invariant under the action of some primitive group G. Let ${\mu }_p(f)$ be the probability that $f=1$ when every variable $x_i$ equals 1 with probability p and equals 0 with probability $1-p$ (independently). Unless f is a constant function, ${\mu }_p(f)$ is a strictly monotone function of p. Define the critical probability $p_\mathrm{c}=p_\mathrm{c}(f)$ by ${\mu }_{p_\mathrm{c}}(f) = 1/2$. Let $\epsilon > 0$ be a fixed tiny real number (say $\epsilon = 1/1000$) and suppose that ${\mu }_{p_1}(f) \epsilon$ and ${\mu }_{p_2}(f) = 1 - \epsilon$. The interval $[p_1,p_2]$ is called the threshold interval for the Boolean function f. Some results of Bourgain, Friedgut and Kalai show that if G satisfies the condition that the orbits of “large” subsets are “large,” then the threshold is sharp (see [2]), i.e., $p_2-p_1=o(1)$ (as a function of n). Friedgut and Kalai [2] described an upper bound for the threshold interval for Boolean functions which is invariant under a transitive group G. Friedgut and Kalai gave an upper bound for the length of the threshold interval when the function is invariant under a transitive permutation group and showed that when $\text {log}p_\mathrm{c}/\text {log}n \rightarrow 0$, then the length of the threshold interval is actually $o(p_\mathrm{c}).$

Bourgain and Kalai developed a method to show stronger results for permutation groups when there are no large sets with small orbits. They observed that no further improvement over the bounds given in [2] is possible when the group satisfies the $small \,\ orbit \,\ property$. This leads naturally to the question considered here.

In this paper, we give a classification of all primitive but not affine groups which satisfy the $small \,\ orbit \,\ property$. This classification shows that the two properties studied by Bourgain and Kalai are incomparable: namely, the first property of Bourgain and Kalai does not imply the second, and the second property does not imply the first.

Theorem

Let $G_t \le S_{n_t}$ be a sequence of primitive groups. Let $G = G_t$ for t large enough. Assuming $G_t$ are not affine groups, the following two conditions hold: $\{G_t\}$ satisfies the $small \,\ orbit \,\ property$ if and only if for all sufficient large t$G_t$ is one of the following :

• 1. $G_t \le S_{k_t} {wr} S_{m_t}$ (product type) where $m_t$ is not bounded.

• 2. $G_t \le H_t {wr} S_{m_t}$ (product type) where $m_t$ is bounded and $H_t$ is a primitive permutation group satisfying the $small \,\ orbit \,\ property$.

• 3. $G_t$ is of diagonal type.

• 4. $G_t$ is of wreath product type.

• 5. $G_t$ is an almost simple exceptional group of Lie type.

• 6. $G_t$ is an almost simple classical group and any point stabilizer of $G_t$ is an irreducible group.

• 7. $G_t$ is an almost simple classical group and $\mathrm{Soc}(G_t) = \hbox {PSL}(k_t,q_t), \hbox {PSp}(2k_t,q_t), \, \hbox {PU}(k_t,{q_t}^2), \,\ \hbox {PO}(2k_t+1,q_t), \,\ \hbox {PO}^+(2k_t,q_t)$ or $\hbox {PO}^-(2k_t,q_t)$, where $q_t$ is not bounded.

Remark

We do not have a full characterization of affine groups that satisfying the $small \,\ orbit \,\ property$.

Remark

In our proof, we show that if $\{G_t\}$ has the $small \,\ orbit \,\ property$, then there exist some subset $X_t \subseteq \{1, \ldots , n_t \}$ such that $\{|X_t|\}$ is not bounded, $|X_t| \le \text {log}{n_t}$ for all t and $|G_t(X_t)| \le {n_t}^5$.

In particular, the two properties of Bourgain and Kalai are incomparable. Indeed, the following are immediate consequences of the theorem.

Corollary 1

The sequence $S_{k_t} {wr} S_{m_t}$, where $m_t$ is not bounded, satisfies the first property of Bourgain and Kalai, but not the second.

Corollary 2

The sequence $\hbox {PSL}(k_t,q_t)$, where $q_t$ is bounded satisfies the second property of Bourgain and Kalai, but not the first.

2 Proof of the theorem

Recall that by the O’Nan Scott Theorem (see [4]) every finite primitive group G is one of the following:

$$\begin{aligned} \begin{array}{ll} (1)&{} G \,\ \hbox {is} \,\ \hbox {an} \,\ \hbox {almost} \,\ \hbox {simple} \,\ \hbox {group.} \\ (2)&{} G \le \hbox {AGL}(V) \,\ \hbox {where} \,\ V \,\ \hbox {is} \,\ \hbox {a} \,\ \hbox {vector} \,\ \hbox {space} \,\ \hbox {of} \,\ \hbox {dimension} \,\ k \\ &{} \hbox {over} \,\ \hbox {a} \,\ \hbox {field} \,\ F \,\ \hbox {of} \,\ \hbox {size} \,\ q. \\ (3)&{} G \,\ \hbox {is} \,\ \hbox {of} \,\ \hbox {simple} \,\ \hbox {diagonal} \,\ \hbox {type.} \\ (4)&{} G \,\ \hbox {is} \,\ \hbox {of} \,\ \hbox {product} \,\ \hbox {type.} \\ (5)&{} G \,\ \hbox {is} \,\ \hbox {of} \,\ \hbox {twisted} \,\ \hbox {wreath} \,\ \hbox {product} \,\ \hbox {type.} \\ \end{array} \end{aligned}$$

Let $G_t \le S_{n_t}$ be a sequence of primitive groups. Assuming $G_t$ is not of type (2), then $G_t$ must be of type (1), (3), (4) or (5). Whenever $G_t$ satisfies the $small \,\ orbit \,\ property$, we construct a subset $X_t$ whose size is not bounded but at most $\text {log}n_t$, but whose orbit size is less than ${n_t}^5$. Otherwise, we give a proof that $G_t$ does not satisfy the $small \,\ orbit \,\ property$. Note that if $|G_t| < {n_t}^5$, then we may take any subset $X_t$ such that $|X_t|$ is not bounded but of size at most $\text {log}n_t$ and obtain $|G_t(X_t)| \le |G_t| < {n_t}^5$. Hence, in what follows, we may assume that $|G_t| > {n_t}^5$.

Since by enlarging the group the size of the orbits cannot go down, then whenever $G_t$ has the $small \,\ orbit \,\ property$, we may replace $G_t$ by a maximal subgroup of $S_{n_t}$ of the same type.

Case 1 Assume that $G_t$ is an almost simple group. Then, by the classification theorem, either $\mathrm{Soc}(G_t) = A_{k_t}$ for some $k_t$, or $\mathrm{Soc}(G_t)$ is a simple group of Lie type, or $\mathrm{Soc}(G_t)$ is a sporadic simple group. By taking t large enough, we may exclude the last case. So suppose now that $\mathrm{Soc}(G_t)$ is a Lie type group. We shall use the following.

Lemma 1

Let $G \le S_n$ be an almost simple exceptional group of Lie type. Suppose G is primitive and n is large enough. Then, $|G| < n^5$.

Proof

Suppose first that G is simple. Denote by m(G) the minimal index of a proper subgroup of G. Estimates for m(G) in the various cases are given in Lemma 6.4 of [6]. Comparing them to the known orders of the groups G, one easily sees that $|G| < m(G)^{5-\epsilon }$ for some fixed $\epsilon > 0$. Since $m(G) \le n$, this yields $|G| < n^{5-\epsilon }$ whenever G is an exceptional simple transitive subgroup of $S_n$.

Now, if G is not simple, let K be its simple socle. Then, $K \le S_n$ is transitive, so we have $|K| \le n^{5-\epsilon }$. It is easy to see that $|\mathrm{Out}(K)| \le \log {|K|}$ and so $|\mathrm{Out}(K)| \le 5 \log {{n}} < n^{\epsilon }$ for large enough n. It follows that $|G| \le |K||\mathrm{Out}(K)| < n^{5-\epsilon } n^{\epsilon } = n^5$, as required. $\square$

Thus, any exceptional almost simple group of Lie type satisfies the $small \,\ orbit \,\ property$. Assume now that $G_t$ is not exceptional. Hence, $G_t$ is an almost simple classical group. Moreover, the values of $|G_t|$ and $m(G_t)$ (see [3] pages 170,175) imply that $k_t$, the rank of $G_t$, is at least 3. Let $H_t \le G_t$ be a point stabilizer. Assume first that $H_t$ is irreducible. By a theorem of M.Liebeck and J.Saxl [5], we have $\vert G_t \vert \le {n_t}^5$, and therefore, $G_t$ satisfies the $small \,\ orbit \,\ property$. Assume now that $H_t$ is a reducible group. Clearly, $H_t$ is a maximal subgroup of $G_t$, since $G_t$ is primitive. The maximal reducible subgroups of the classical groups are well known, and we shall use their description in constructing the required subset $X_t$.

Using the reducibility of the point stabilizer, it follows that there exist some $d_t, q_t$ and $k_t \ge 3$ such that $\mathrm{Soc}(G_t)$ is one of the following:

$$\begin{aligned} \begin{array}{lll} (A) &{} a)&{} \hbox {PSL}(k_t,q_t) \text { acting on all } d_t\text {-dimensional subspaces of} \\ &{} &{} V_t = {(F_{q_t})}^{k_t}. \\ &{}b)&{} \hbox {PSL}(k_t,q_t) \text { acting on pairs of subspaces of} \\ &{} &{} \text { complementary dimensions.} \\ \end{array} \end{aligned}$$

$$\begin{aligned} \begin{array}{lll} (B) &{}a)&{} \hbox {PSp}(2k_t,q_t) \text { acting on all } d_t\text {-dimensional non-degenerate} \\ &{} &{} \text { subspaces of } V_t = {(F_{q_t})}^{2k_t}. \\ &{}b)&{} \hbox {PSp}(2k_t,q_t) \text { acting on all } d_t\text {-dimensional totally singular} \\ &{} &{} \text { subspaces of } V_t = {(F_{q_t})}^{2k_t}.\\ &{}c)&{} \hbox {PSp}(2k_t,q_t) \text { acting on pairs of subspaces of} \\ &{} &{} \text { complementary dimensions.} \\ \end{array} \end{aligned}$$

$$\begin{aligned} \begin{array}{lll} (C) &{}a)&{} \hbox {PU}(k_t,{q_t}^2) \text { acting on all } d_t\text {-dimensional non-degenerate} \\ &{} &{} \text { subspaces of } V_t = {({F_{q_t}}^2)}^{k_t}. \\ &{}b)&{} \hbox {PU}(k_t,{q_t}^2) \text { acting on all } d_t\text {-dimensional totally singular} \\ &{} &{} \text { subspaces of } V_t = {({F_{q_t}}^2)}^{k_t}. \\ &{}c)&{} \hbox {PU}(k_t,{q_t}^2) \text { acting on pairs of subspaces of} \\ &{} &{} \text { complementary dimension.} \\ \end{array} \end{aligned}$$

$$\begin{aligned} \begin{array}{lll} (D) &{}a)(1)&{} \hbox {PO}(2k_t+1,q_t) \text { acting on all } d_t\text {-dimensional non-degenerate }\\ &{} &{} \text { subspaces of } V_t = {(F_{q_t})}^{2k_t+1}. \\ &{}a)(2)&{} \hbox {PO}(2k_t+1,q_t) \text { acting on all } d_t \text {-dimensional totally singular} \\ &{} &{} \text { subspaces of } V_t = {(F_{q_t})}^{2k_t+1}. \\ &{}b)(1)&{} \hbox {PO}^+(2k_t,q_t) \text { acting on all } d_t \text {-dimensional non-degenerate} \\ &{} &{} \text { subspaces of } V_t = {(F_{q_t})}^{2k_t}. \\ &{}b)(2)&{} \hbox {PO}^+(2k_t,q_t) \text { acting on all } d_t \text {-dimensional totally singular} \\ &{} &{} \text { subspaces of } V_t = {(F_{q_t})}^{2k_t}. \\ &{}c)(1)&{} \hbox {PO}^-(2k_t,q_t) \text { acting on all } d_t \text {-dimensional non-degenerate}\\ &{} &{} \text { subspaces of } V_t = {(F_{q_t})}^{2k_t}. \\ &{}c)(2)&{} \hbox {PO}^-(2k_t,q_t) \text { acting on all } d_t \text {-dimensional totally singular} \\ &{} &{} \text { subspaces of } V_t = {(F_{q_t})}^{2k_t}. \\ &{}c)(3)&{} \hbox {PO}(2k_t+1,q_t) \text { where } q_t \text { is even and } G_t\text { acts on all} \\ &{} &{} \text { one-dimensional non-singular subspaces.} \\ &{}d) &{} \hbox {PO}(2k_t,q_t) \text { or } \hbox {PO}(2k_t+1,q_t) \text { acting on pairs of } \\ &{} &{} \text { subspaces of complementary dimensions. } \end{array} \end{aligned}$$

Note that any sequence $G_t$ of almost simple classical groups with the above action has the $small \,\ orbit \,\ property$ if and only if its socle has this property. This follows from the inequality:

$$\begin{aligned} \vert G_t(X_t) \vert \le \vert \mathrm{Soc}(G_t)(X_t) \vert \vert \mathrm{Out}(G_t) \vert \le \vert \mathrm{Soc}(G_t) \vert \cdot n_t. \end{aligned}$$

Therefore, we may restrict to the socle and assume $G_t$ is simple. So let $G_t$ be a sequence of simple classical groups with one of the actions above. We will show that if $q_t$ is not bounded, then $G_t$ satisfies the $small \,\ orbit \,\ property$, and if $q_t$ is bounded, then $G_t$ does not satisfy the $small \,\ orbit \,\ property$. Assume first that $q_t$ is not bounded. Suppose first that $G_t = PGL_{k_t}(q_t)$. Let $\{v_{t,1}, \ldots , v_{t,k_t} \}$ be a basis for $V_t$ and let $H_t \le G_t$ be a point stabilizer. Then, $H_t$ is a maximal subgroup of $G_t$. Assume first that $G_t$ acts on the set of all $d_t$-dimensional subspaces of ${F_{q_t}}^{k_t}$ for some $d_t$ between 1 and ${k_t}/2$. Here $n_t = \frac{({q_t}^{k_t}-1)({q_t}^{k_t}-q_t) \cdots ({q_t}^{k_t}-{q_t}^{d_t-1})}{({q_t}^{d_t}-1)({q_t}^{d_t}-q_t) \cdots ({q_t}^{d_t}-{q_t}^{d_t-1})} \sim {q_t}^{d_t(k_t-d_t)}$ and $\text {log}n_t \sim d_t(k_t-d_t)\text {log}{q_t}$. We now construct the subset $X_t$. For $1 \le i \le \text {log}{q_t}$ and $a_{t,1}, \ldots , a_{t,{\text {log}{q_t}}} \in F_{q_t}$, let

$$\begin{aligned} U_{t,i} = \hbox {Sp} (v_{t,1}, \ldots , v_{t,{d_t-1}}, v_{t,d_t}+a_{t,i}v_{t,{d_t+1}}). \end{aligned}$$

Let

$$\begin{aligned} X_t = \{U_{t,1}, \ldots , U_{t,{\text {log}{q_t}}}\}. \end{aligned}$$

Then, log$q_t -1 \le \vert X_t \vert \le$ log$q_t \le$ log$n_t$, and since $q_t$ is not bounded, $\vert X_t \vert$ is not bounded as well. It is easy to see that any $A_t \in \mathrm{GL}_{k_t}(q_t)$ which stabilizes the subspace $\hbox {Sp}(v_{t,1}, \ldots , v_{t,{d_t-1}})$ and the vectors $v_{t,d_t},v_{t,{d_t+1}}$ also stabilizes $X_t$, and therefore,

$$\begin{aligned} |{G_t}_{X_t}|\ge & {} ({q_t}^{d_t-1}-1)({q_t}^{d_t-1}-q_t) \cdots {q_t}^{d_t-1}-{q_t}^{d_t-2})\\&({q_t}^{k_t-(d_t+1)}-1) \cdots ({q_t}^{k_t-(d_t+1)}-{q_t}^{k_t-d_t}). \end{aligned}$$

It follows that

$$\begin{aligned} |G_t(X_t)| = \frac{({q_t}^{k_t}-1)({q_t}^{k_t}-{q_t}) \cdots ({q_t}^{k_t}-{q_t}^{k_t-1})}{|{G_t}_{X_t}|} \le {n_t}^2. \end{aligned}$$

Thus, $G_t$ satisfies the $small \,\ orbit \,\ property$.

Assume now that $G_t$ acts on the set of all pairs of subspaces $(U_t,W_t)$ such that $U_t \oplus W_t = V_t$ and $dim U_t = d_t$, where $d_t \le {k_t}/2$. Then, $n_t = {q_t}^{2d_t(k_t-d_t)}$ and log${n_t} = 2d_t(k_t-d_t)$ log${q_t}$. We now construct the subset $X_t$. For $1 \le i \le$ log$q_t$ and $a_{t,1}, \ldots , a_{t,{\text {log}{q_t}}} \in F_{q_t}$, let

$$\begin{aligned} U_{t,i} = \hbox {Sp} (v_{t,1}, \ldots , v_{t,{d-1}}, v_{t,d_t}+a_{t,i}v_{t,{d_t+1}}) \end{aligned}$$

and let

$$\begin{aligned} W_t = \hbox {Sp}(v_{t,d_t},v_{t,{d_t+2}}, \ldots , v_{t,k_t}). \end{aligned}$$

Let

$$\begin{aligned} X_t = \{U_{t,1} \oplus W_t, \ldots , U_{t,{\text {log}{q_t}}} \oplus W_t \}. \end{aligned}$$

It is easy to see that any $A_t \in \mathrm{GL}_{k_t}(q_t)$ which stabilizes the subspace $\hbox {Sp}(v_{t,1}, \ldots , v_{t,d_t-1})$, the vectors $v_{t,d_t},v_{t,{d_t+1}}$ and the subspace $\hbox {Sp}( v_{t,{d_t+2}}, \ldots , v_{t,k_t} )$ also stabilizes $X_t$, and therefore,

$$\begin{aligned} |(G_t)_{X_t}|\ge & {} ({q_t}^{d_t-1}-1)({q_t}^{d_t-1}-q_t) \cdots ({q_t}^{d_t-1}-{q_t}^{d_t-2})\\&({q_t}^{k_t-(d_t+1)}-1) \cdots ({q_t}^{k_t-(d_t+1)}-{q_t}^{k_t-d_t}). \end{aligned}$$

It follows that

$$\begin{aligned} |{G_t}(X_t)| = \frac{({q_t}^{k_t}-1)({q_t}^{k_t}-{q_t}) \cdots ({q_t}^{k_t}-{q_t}^{k_t-1})}{|{G_t}_{X_t}|} \le n_t. \end{aligned}$$

Thus, $G_t$ satisfies the $small \,\ orbit \,\ property$.

Assume now that $G_t$ acts on the set of all pairs of subspaces $(U_t,W_t)$ such that $U_t \oplus W_t$, $dim U_t = d_t$, where $d_t \le {k_t}/2$. Then, $n_t = {q_t}^{d_t(2k_t-3d_t)}$ and log${n_t} = d_t(2k_t-3d_t)$log$q_t$. We now construct the subset $X_t$. For $1 \le i \le \text {log}q_t$ and $a_{t,1}, \ldots , a_{t,\text {log}{q_t}}$, let

$$\begin{aligned} U_{t,i} = \hbox {Sp} (v_{t,1}, \ldots , v_{t,{d_t-1}}, v_{t,d_t}+a_{t,i}v_{t,{d+1}}) \end{aligned}$$

and let

$$\begin{aligned} W_t = \hbox {Sp} (v_{t,d_t}, v_{t,{d_t+2}}, \ldots v_{t,k_t}). \end{aligned}$$

Let

$$\begin{aligned} X_t = \{(U_{t,1},W_t), \ldots , (U_{t,{\text {log}q_t}}, W_t) \}. \end{aligned}$$

It is easy to see that any $A_t \in \mathrm{GL}_{k_t}(q_t)$ which stabilizes the subspace $\hbox {Sp}(v_{t,1}, \ldots , v_{t,{d_{t-1}}})$, the vectors $v_{t,d},v_{t,d_{t+1}}$ and the subspace $\hbox {Sp}( v_{t,{d_{t+2}}}, \ldots , v_{t,k_t} )$ also stabilizes $X_t$, and therefore,

$$\begin{aligned} |{(G_t)}_{X_t}|\ge & {} ({q_t}^{d_t-1}-1)({q_t}^{d_t-1}-q_t) \cdots ({q_t}^{d_t-1}-{q_t}^{d_t-2}) \\&({q_t}^{k_t-(d_t+1)}-1) \cdots ({q_t}^{k_t-(d_t+1)}-{q_t}^{k_t-d_t}). \end{aligned}$$

It follows that

$$\begin{aligned} |G_t(X_t)| = \frac{({q_t}^{k_t}-1)({q_t}^{k_t}-q_t) \cdots ({q_t}^{k_t}-{q_t}^{k_t-1})}{|{(G_t)}_{X_t}|} \le {n_t} \end{aligned}$$

Thus, $G_t$ satisfies the $small \,\ orbit \,\ property$.

Assume now that $G_t$ acts on the set of all pairs of subspaces $(U_t,W_t)$ such that $U_t \subseteq W_t$, $dim U_t = d_t$, and $dimW_t = k_t-d_t$, where $d_t \le {k_t}/2$. Then, $n_t = {q_t}^{d_t(2k_t-3d_t)}$ and $\text {log}{n_t} = d_t(2k_t-3d_t) \text {log}q_t$. We now construct the subset $X_t$. Let $\{v_1, \ldots , v_k\}$ be a basis for V. We now construct the subset X. For $1 \le i \le \text {log}q_t$ and $a_{t,1}, \ldots , a_{t,\text {log}{q_t}}$, let

$$\begin{aligned} U_{t,i} = \hbox {Sp} (v_{t,1}, \ldots , v_{t,d_{t-1}}, v_{t,d_t}+a_{t,i}v_{t,d_{t+1}}) \end{aligned}$$

and let

$$\begin{aligned} W_t = \hbox {Sp} (v_{t,1}, \ldots , v_{t,d_t-1},v_{t,d_t},v_{t,d_t+1},v_{t,d_t+2}, \ldots , v_{t,k_t-d_t}). \end{aligned}$$

Let

$$\begin{aligned} X_t = \{(U_{t,1}, W_t), \ldots , (U_{t,\text {log}q_t}, W_t) \}. \end{aligned}$$

It is easy to see that for any $A_t \in \mathrm{GL}_{k_t}(q_t)$ which stabilizes the subspace $\hbox {Sp}(v_{t,1}, \ldots , v_{t,{d_t-1}})$, the vectors $v_{t,d},v_{d_t+1}$ and the subspace $\hbox {Sp}( v_{t,{d_t+2}}, \ldots , v_{t,k_t} )$ also stabilizes $X_t$, and therefore,

$$\begin{aligned} |({G_t)}_{X_t}|\ge & {} ({q_t}^{d_t-1}-1)({q_t}^{d_t-1}-q_t) \cdots ({q_t}^{d_t-1}-{q_t}^{d_t-2})\\&({q_t}^{k_t-(d_t+1)}-1) \cdots ({q_t}^{k_t-(d_t+1)}-{q_t}^{k_t-d_t}). \end{aligned}$$

It follows that

$$\begin{aligned} |G_t(X_t)| = \frac{({q_t}^{k_t}-1)({q_t}^{k_t}-q_t) \cdots ({q_t}^{k_t}-{q_t}^{k_t-1})}{|{(G_t)}_{X_t}|} \le {n_t}^2. \end{aligned}$$

Thus, $G_t$ satisfies the $small \,\ orbit \,\ property$.

Assume now that $G_t$ is any group not of types D(c)(3) or A. Assume first that $G_t$ acts on the set of all $d_t$-dimensional totally singular subspaces for some $d_t$ between 1 and ${k_t}/2-1$. Let $\{e_{t,1},f_{t,1}, \ldots , e_{t,{k_t}/2},f_{t,{k_t}/2} \}$ be a standard basis for $V_t$, i.e., $(e_{t,i},e_{t,j}=(f_{t,i},f_{t,j}=1$, and $(e_{t,i}=f_{t,j}= {\delta }_{i,j}$ (it might be the case that instead of $e_{t,k_t/2},f_{t,k_t/2}$ we will have basis elements $x_t,y_t$, with different relations, or that the dimension is odd, and we will have an additional basis element $x_t$, (see chapter 2 of [3]), but for our purpose it will not make a difference). For $1 \le i \le \text {log}q_t$ and $a_{t,1}, \ldots , a_{t,\text {log}q_t}$, let

$$\begin{aligned} U_{t,i} = \hbox {Sp} (e_{t,1}, \ldots , e_{t,d_t-1}, e_{t,d_t}+a_{t,i}e_{t,d_t+1}). \end{aligned}$$

Let

$$\begin{aligned} X_t = \{U_{t,1}, \ldots , U_{t,\text {log}q_t}\}. \end{aligned}$$

It is easy to see that any $A_t \in G_t$ which stabilizes the subspace $\hbox {Sp}(e_{t,d_t}, e_{t,d_t+1})$ also stabilizes $X_t$. Using the results of chapter 2 of [3], we get that $|G_t(X_t)| \le {n_t}^2$. Assume now that $d_t = k_t/2$. For $1 \le i \le \text {log}q_t$ and $a_{t,1}, \ldots , a_{t,\text {log}q_t} \in F_{q_t}$, let

$$\begin{aligned} U_{t,i} = \hbox {Sp} (e_{t,1}, \ldots , e_{t,{k_t}/2-2}, e_{k/2-1}+a_if_{k/2}, e_{k/2}-a_if_{k/2-1} ) \end{aligned}$$

(in the symplectic case, we take $e_{t,k_t/2}+a_{t,i}f_{t,k_t/2-1}$ instead of $e_{t,k_t/2}-a_{t,i}f_{t,k_t/2-1}$). Let

$$\begin{aligned} X_t = \{U_{t,1}, \ldots , U_{t,\text {log}q_t} \}. \end{aligned}$$

As before, we get that $|G_t(X_t)| \le {n_t}^2$. Assume now that $G_t$ acts on the set of all $d_t$-dimensional non-degenerate subspaces of $V_t$. For $1 \le i \le \text {log}q_t$ and $a_{t,1},\ldots , a_{t,\text {log}q_t} \in F_{q_t}$, let

$$\begin{aligned} U_{t,i} = \{e_{t,1},f_{t,1}, \ldots , e_{t,d_t/2-1},f_{t,d_t/2-1},e_{t,d_t/2}, f_{t,d_t/2}+a_{t,i}e_{t,d_t/2+1}\}. \end{aligned}$$

Let

$$\begin{aligned} X_t = \{U_{t,1}, \ldots , U_{t,\text {log}q_t}\}. \end{aligned}$$

Thus, we get that $|G_t(X_t)| \le {n_t}^2$. Assume now that $G_t$ is of type D(c)(3) above. Let $\{e_{t,1},f_{t,1}, \ldots , e_{t,k_t/2},f_{t,k_t/2} \}$ be a symplectic basis for $V_t$. For $1 \le i \le \text {log}q_t$ and $a_{t,1}, \ldots , a_{t,\text {log}q_t}$, let

$$\begin{aligned} U_{t,i} = \hbox {Sp}(e_{t,1}+a_{t,i}f_{t,1}\}. \end{aligned}$$

Let

$$\begin{aligned} X_t = \{U_{t,1}, \ldots , U_{t,\text {log}q_t}\}. \end{aligned}$$

It follows that $|G_t(X_t)| \le {n_t}^2$. Hence, whenever $q_t$ is not bounded, $G_t$ satisfies the $small \,\ orbit \,\ property$. Assume now that $q_t$ is bounded. Let $G_t$ be any group of types $A-D$ above. We will show that under the action of $G_t$ on the set of all one-dimensional totally singular subspaces of $V_t$, the size of the orbit of any unbounded subset of size at most $c \text {log}n_t$ is not polynomial. Let $X_t$ be any unbounded set of vectors of size at most $c \text {log}n_t$. Let $r_t = dim Span(X_t)$. Clearly, $r_t$ is not bounded (since $q_t$ is bounded). Let $\{x_{t,1}, \ldots , x_{t,r_t} \} \subseteq X_t$ be a base for $Span(X_t)$, and $\{ v_{t,r_t+1}, \ldots , v_{t,k_t} \}$ its complement to a base for $V_t$. Let $g_t \in {(G_t)}_{X_t}$. Then, for all $1 \le i \le r_t$$g_t(x_{t,i}) \in X_t$ and $g_t(v_{t,r_{t+i}}) \notin Span(X_t)$. Therefore,

$$\begin{aligned} |{G_t}_{X_t}|\le & {} |X_t|(|X_t|-1) \cdots (|X_t|-(r_t-1))({q_t}^{k_t}-{q^t}^{r_t}) \cdots ({q_t}^{k_t}-{q_t}^{k_t-1})\\\le & {} {|X_t|}^{r_t}({q_t}^{k_t}-{q_t}^{r_t}) \cdots ({q_t}^{k_t}-{q_t}^{k_t-1}). \end{aligned}$$

It follows that

$$\begin{aligned} |G_t(X_t)|= & {} |G_t : {(G_t)}_{X_t}| \ge \frac{({q_t}^{k_t}-1) \cdots ({q_t}^{k_t}-{q_t}^{k_t-1})}{{|X_t|}^{r_t}({q_t}^{k_t}-{q_t}^{r_t}) \cdots ({q_t}^{k_t}-{q_t}^{k_t-1})}\\= & {} \frac{({q_t}^{k_t}-1) \cdots ({q_t}^{k_t}-{q_t}^{r_t-1})}{{|X_t|}^{r_t}} \ge \frac{{({q_t}^{k_t}-{q_t}^{r_t-1})}^{r_t}}{{(ck_t \text {log} q_t)}^{r_t}} \ge \frac{{({q_t}^{k_t-1})}^{r_t}}{{(ck_t \text {log} q_t)}^{r_t}} \\= & {} {\left( \frac{{({q_t}^{k_t-1})}}{{ck_t \text {log} q_t}}\right) }^{r_t} > {\left( \frac{{({q_t}^{k_t-2})}}{{ck_t }}\right) }^{r_t}. \end{aligned}$$

Since $r_t$ is not bounded, it now follows that $|G_t(X_t)|$ is not polynomial in $n_t={q_t}^{k_t}$, as desired.

Case 3 Assume that $G_t$ is of diagonal type. We will show that $G_t$ satisfies the $small \,\ orbit \,\ property$. We may assume that there exists some integer $k_t$ such that $G_t = {H_t}^{k_t}.(\mathrm{Out}(H_t) \times S_{k_t})$, where $H_t$ is a non-abelian simple group, $k_t$ is a positive integer and $G_t$ acts on the set of all cosets of the diagonal group $D_t = \{(h_t, \ldots , h_t) : h_t \in H_t \}$ Denote this set by $\varOmega$. Here $n_t = |{H_t}^{k_t-1}|$ and log$n_t=(k_t-1)\text {log}|H_t|$. Since $|G_t| = |{H_t}^{k_t}||H_t|k_t! \le {n_t}^2k_t!$, then we may assume that $k_t$ is not bounded. Let $L_t < H_t$ be a nontrivial subgroup of $H_t$ and let $1 \ne l_t \in L_t$. Let

$$\begin{aligned} X_t = \{(l_t,1,\ldots , 1)D_t,(1,l_t,1,\ldots ,1)D_t, \ldots , (1,\ldots ,1,l_t)D_t \}. \end{aligned}$$

Then, $|X_t| = k_t \le \text {log}n_t$. Since $|S_{k_t}(X_t)| = 1$ we have

$$\begin{aligned} |G_t(X_t)| = |{H_t}^{k_t}.(\mathrm{Out}(H_t) \times S_{k_t})(X_t)| \le {n_t}^2|S_{k_t}(X_t)| = {n_t}^2. \end{aligned}$$

Case 4 Assume that $G_t$ is of product type. Let $H_t$ be a primitive permutation group of type (1) or (3) acting on a set ${\Gamma }_t = \{1, \ldots , k_t \}$. For $m_t > 1$, let $W_t = H_t {wr} S_{m_t}$ and take $W_t$ to act on ${\varOmega }_t = {{\Gamma }_t}^{m_t}$ in its natural product action. Then, $G_t \le H_t {wr} S_{m_t}$, and $n_t = {k_t}^{m_t}$. Assume first that $m_t$ is not bounded. We will show that $G_t$ satisfies the $small \,\ orbit \,\ property$. Let $x_t,y_t \in {\Gamma }_t$ and let

$$\begin{aligned} X_t = \{(x_t,y_t, \ldots , y_t), (y_t,x_t,y_t, \ldots , y_t), \ldots , (y_t, \ldots , y_t,x_t) \}. \end{aligned}$$

Then, $|X_t| = m_t \le \text {log}n_t$. It is easy to see that $|{S_{k_t}}^{m_t}(X_t)| \le {({k_t}^{m_t})}^2 = {n_t}^2$. Since $|S_{m_t}(X_t)| = 1$, we have $|G_t(X_t)| \le |{S_{k_t}}^{m_t}.S_{m_t}(X_t)| \le {n_t}^2|S_{m_t}(X_t)| = {n_t}^2$. Assume now that $m_t$ is bounded. We will show that $G_t$ has the $small \,\ orbit \,\ property$ if and only if $H_t$ has the $small \,\ orbit \,\ property$: Assume first that $H_t$ satisfies the $small \,\ orbit \,\ property$. As we have shown (see cases 1 and 3), there exists some subset $Y_t$ which is not bounded but of size at most log$k_t$ such that $|H_t(Y_t)| \le {k_t}^5$. Denote $Y_t = \{{y^t}_1, \ldots , {y^t}_{|Y_t|}\}$. Let

$$\begin{aligned} X_t = \{({y^t}_1, \ldots , {y^t}_1), ({y^t}_2, \ldots , {y^t}_2), \ldots , ({y^t}_{|Y_t|}, \ldots {y^t}_{|Y_t|}) \}. \end{aligned}$$

Then, $X_t$ is not bounded but of size at most log$n_t$. Since $|H_t(Y_t)| \le {k_t}^5$, then $|{H_t}^{m_t}(X_t)| \le {({k_t}^5)}^{m_t} = {({k_t}^{m_t})}^5 = {n_t}^5$. Clearly, $|S_{m_t}(X_t)| = 1$, and therefore, $|{{H_t}^{m_t}}.S_{m_t}(X_t)| \le {n_t}^5$.

Assume now that $H_t$ does not satisfy the $small \,\ orbit \,\ property$. By our classification (see cases 1 and 3), $H_t$ must be an almost simple group. A previous argument (see case 1) implies that $\mathrm{Soc}(H_t)$ does not satisfy the $small \,\ orbit \,\ property$ as well. We will show now that $G_t$ does not satisfy the $small \,\ orbit \,\ property$ as well: Let $X_t$ be any unbounded subset of ${\varOmega }_t$ of size at most $c \text {log}n_t$. Denote by ${X^t}_i$ the projection on the $i'th$ axis. Since $m_t$ is bounded, then there exists some $1 \le i \le m_t$ such that ${X^t}_i$ is an unbounded set of size at most $cm_t \text {log}k_t$. Hence, under our assumption $|(\mathrm{Soc}(H_t))({X^t}_i)|$ is not polynomial in $k_t$, and since $m_t$ is bounded, then $|(\mathrm{Soc}(H_t))({X^t}_i)|$ is not polynomial in $n_t$. It now follows that $|{(\mathrm{Soc}(H_t))}^{m_t}(X_t)|$ is not polynomial in $n_t$. Since ${(\mathrm{Soc}(H_t))}^{m_t} \le G_t$, then we get now that $|G_t(X_t)|$ is not polynomial in $n_t$, as desired.

Case 5 Any group $G_t$ of type 5 satisfies the $small \,\ orbit \,\ property$, since any twisted wreath product group is contained in a group of product type of the form $H_t {wr} S_{m_t}$ where $H_t$ is of diagonal type (see [4]) .