UNIPOTENT SUBGROUPS OF STABILIZERS

We consider semicontinuity of certain dimensions on group schemes.


Introduction
Let G be an algebraic group over a field k.Let d u (G) the maximal dimension of a (smooth) connected unipotent subgroup of G k .Using techniques à la Demazure-Grothendieck, we show the following result.
Theorem 1.1.Let S be a scheme and let G be a separated S-group scheme of finite presentation (for example an affine S-group of finite presentation).Then the function d u on S is upper semi-continuous.
Upper semi-continuous means informally that the function jumps along closed sets.In particular, the function is locally constant at the points of the minimal value locus.This gives the following useful corollary.
Corollary 1.2.Let S be an irreducible scheme and let G be a separated S-group scheme of finite presentation.If for the generic point ξ ∈ S, G κ(ξ) contains a d-dimensional smooth unipotent subgroup, then the same is true for G κ(s) for all s ∈ S.
A case of special interest is the group scheme of stabilizers called also the stabilizer of the diagonal [SGA3,V.10.2].More precisely if G is an S-group scheme acting on a separated S-scheme X of finite presentation (with S noetherian), we consider the fiber product It defines an X-group scheme F which is a closed X-subgroup scheme of G × S X of finite presentation such that for each x ∈ X of image s ∈ S, F κ(x) ⊂ G × κ(s) κ(x) is the stabilizer of the point x for the action of G × κ(s) κ(x) on X × κ(s) κ(x).In case κ(s) = κ(x), the usual notation for F κ(x) is G x .One motivation for this question was related to the base size of finite groups acting primitively on a set and the existence of regular orbits for nontransitive actions.An important Date: November 21, 2022.2020 Mathematics Subject Classification.Primary 14L15; secondary 14L30, 20B15, 20G15.The second author was partially supported by the NSF grant DMS-1901595 and a Simons Foundation Fellowship 609771.case is that of a finite simple group of Lie type over a finite field where the action comes from the algebraic group.See [BGS].Another interesting case is when G is a reductive algebraic group acting linearly on X (or acting on the Grassmanian of a rational module).See [GG, GL, PV] for more on this.We give more details on this in Section 5.
Note that by the Lang-Steinberg theorem, an algebraic group defined over F q has a Borel subgroup defined over F q .We also know that if U is a d-dimensional unipotent subgroup defined over F q , then |U(F q )| = q d [B, GG].We can apply Corollary 1.2 to the stabilizer scheme to obtain the following result.
Corollary 1.3.Let G be a algebraic group acting faithfully on an irreducible variety X and assume that the G, X and the action are defined over a finite field F q .Assume that there is a nonempty open subset X 0 of X such that the stabilizer G x of x ∈ X 0 has a d-dimensional unipotent subgroup.Then for all x ∈ X(F q ), G x (F q ) contains a subgroup of size q d .One can ask more generally what other functions are upper semi-continuous.Of course, dimension is [SGA3,VI B .4.3].In fact, we will show that other such functions are also upper semi-continuous.On the other hand, if we define d 0 (G) to be the dimension of the derived subgroup of the connected component of G, it is not true that d 0 need be upper semicontinuous.We study this in Section 6 and particularly for the smooth case.Smoothness rarely holds in the case of the stabilizer scheme and we give an example to show the failure of upper semi-continuity for d 0 (G) in stabilizer schemes.
We use mostly the terminology of Borel's book [B] and time to time the more general setting Demazure-Gabriel's book [D-G] and the SGA3 seminar [SGA3] where in particular an algebraic group is not supposed smooth.All definitions are coherent.
In the next sections, we prove some preliminary results.We prove Theorem 1.1 and other upper semi-continuity results in Section 4. In Section 6, we consider the dimension of the derived subgroup.In the final section, we present an alternate proof of Theorem 1.1 due to Brian Conrad.
Acknowledgments: We thank Jean-Pierre Serre, Brian Conrad and Matthieu Romagny for valuable comments on a preliminary version of the paper.We also thank Brian Conrad for allowing us to include his alternate proof of Theorem 1.1 and Skip Garibaldi for the applications to the abelianization (Cor.6.2).

Definition of the rank functions
Let k be a field and let G/k be an algebraic group.We remind the reader that a linear algebraic group G is unipotent if for each (or any) faithful k-representation ρ : G k → GL n,k , ρ(G(k)) consists in unipotent elements.This agrees with the general definition given in [D-G, 4.2.2.1], see 4.3.26.b of this reference.In practice we use the equivalent definition that G admits a closed k-embedding in a k-group of strictly upper triangular matrices (ibid, 4.2.2.5).
Similarly a closed smooth k-subgroup G of GL n is trigonalizable if there exists h ∈ GL n (k) such that hGh −1 ⊂ B n where B n is the k-Borel subgroup of GL n consisting in upper triangular matrices; the same holds for any linear representation G → GL r [B, 15.5].That definitions holds more for an arbitrary affine algebraic k-group since it is equivalent to the 4.2.3.4].
2.1.Relative version.We define the following invariants: All these functions are increasing by change of fields.Proof.We do it for d u , the other cases being similar.We can assume that F is algebraically closed so that G F admits a smooth unipotent F -subgroup U of dimension d.There exists a finitely generated k-subextension E of F such that U is defined over E. The field E is function field of a smooth k-variety X.Up to shrinking X, U extends to a closed subgroup scheme U of G × k X which is smooth in view of [SGA3,VI B .10].Up to shrinking one more time, U is a closed subgroup scheme of the X-group scheme of strictly upper triangular matrices.Since k is algebraically closed, we have

Absolute version.
We define now Clearly it does not depend of the choice of k.All these functions are insensitive to change of fields.
Up to considering the connected reduced fiber, we have also and similarly for the other absolute rank functions.Since affine smooth connected solvable k-subgroups trigonalizable by the Lie-Kolchin's theorem [B, 10.5] We have (b) If G is a smooth connected affine algebraic group over an algebraically closed field k, the Borel subgroups are the maximal smooth connected solvable subgroups, they are all conjugate and so d s (G) is nothing but the dimension of a Borel subgroup.The unipotent radicals are then the maximal smooth connected unipotent subgroups, they are all conjugate and so d u (G) is the dimension of a the unipotent radical of a Borel subgroup of G.For d n (G) the situation is more complicated since maximal smooth connected nilpotent subgroups of G do not consist of a single conjugacy class.However there are finitely many conjugacy classes (Platonov,[P,thm 2.13]).

Specialization over a regular local ring
3.1.Group schemes over a DVR.Let A be a discrete valuation ring of fraction field K and of residue field k.If G is an A-group scheme of finite presentation, we would like to list properties on the generic fiber G K which are inherited by the closed fiber G k .For example, if G is flat, then G K and G k share the same dimension [SGA3,VI B .4.3].Also if G is separated and flat and G K is affine, then G is affine (Raynaud, [PY, prop 3.1]) so that G k is affine.Flatness is then an important property, we recall that an A-scheme X is flat if and only if it is torsion free, this second condition being equivalent to the density of the generic fiber X K in X [GW, §14.3].For the study of the function d u , the next statement is the key step.
Lemma 3.1.We assume that G is flat and affine.If G K is trigonalizable (resp.unipotent) so is G k .
Proof.We assume that G K is trigonalizable, that is, G K is an extension of a diagonalizable K-group by an unipotent K-group.According to [BT, 1.4.5],there exists a closed monomorphism ρ : G → GL N .Since G K is trigonalizable, its stabilizes a flag of K N [D-G, prop.4.2.3.4,(i) =⇒ (iv)].In other words ρ factorizes through a Borel subgroup B K of GL N .Since the A-scheme of Borel subgroups of GL N is projective [SGA3,XXII.5.8.3], B K extends uniquely to a Borel A-subgroup scheme B of GL N [a concrete way is to use a filtration We assume furthermore that G K is unipotent.We have an exact sequence of A-group . This is a quite different proof.
(c) One simpler proof of (b) occurs in the alternate proof below of Theorem 1.1 using the smoothness of the scheme of maximal tori of G, see §7.
For dealing the other rank functions, we need more facts.
Lemma 3.2.Let H be a K-subgroup of G K and let H be the schematic closure of H in G.
(1) H is a closed A-subgroup scheme of G which is flat of finite presentation.If H is central, then H is central.
(2) The fppf quotient G/H is representable by a separated A-scheme of finite presentation.
(2) The representability is result by Anantharaman [A,IV,th. 4.C] so that G/H is separated and of finite presentation [SGA3, VI B .9.2.(3) Assume that H is normal in G K , that is, the commutator map According to [SGA3, VI B .9.2.(iv)], it follows that G/H carries a natural structure of A-group scheme.
Proof.If G K is commutative, so is G and G k according to Lemma 3.2.(1).
We assume now that G K is nilpotent, that is, admits a central composition serie this is a flat A-group scheme and all H i 's are normal A-subgroups of G according to Lemma 3.2.(3).Furthermore each quotient G i+1 /G i is central in G/H i .By extending the scalars to k we get then a central composition series for G k .
The argument is similar for the solvable case.
3.2.The regular local ring case.Let A be a regular local ring with fraction field K and residue field k.
Proposition 3.4.Let G be an A-group scheme of finite presentation.
(1) Assume that k is infinite and that G K contains an algebraic subgroup (resp.normal subgroup) of dimension d.Then G k contains an algebraic subgroup (resp.normal subgroup) of dimension d.
(2) Assume that G K contains an algebraic subgroup which is commutative (resp.nilpotent, solvable) of dimension d.Then G k contains an algebraic subgroup which is commutative (resp.nilpotent, solvable) of dimension d.
Proof.According to [EGA4,lemma 15.1.1.6],there exists a discrete valuation ring B which dominates A and such that its residue field is a purely (finitely generated) transcendental extension of k.We denote by L the fraction field of B and by l its residue field.We have l = k or k(t 1 , . . ., t n ).
(1) Our assumption is that G K contains an algebraic subgroup (resp.normal subgroup) H which is of dimension d.We consider the schematic closure of H L in G B , this defines a flat B-group scheme (resp.normal B-subgroup scheme according to Lemma 3.2.(3)) H of closed fiber H l which is a subgroup of (G k ) l .Since k is infinite, one may "specialize" at a rational k-point to obtain a k-subgroup of G k of dimension d.
(2) Our assumption is that G K contains an algebraic subgroup H which is commutative (resp.nilpotent, solvable) of dimension d.We consider the schematic closure of H L in G B , this defines a flat B-group scheme H of closed fiber H l which is a subgroup of (G k ) l .We apply Lemma 3.3 to H and obtain that H l is commutative (resp.nilpotent, solvable) of dimension d.
By induction on n, we may assume that l = k(t).We have that G k((t)) admits the subgroup H k((t)) which is commutative (resp.nilpotent, solvable) of dimension d.By performing the same method as above in the case of the DVR k[[t]], it follows that G k contains a commutative (resp.nilpotent, solvable) subgroup of dimension d.
(3) Though the argument is very similar, we provide all details.Our assumption is that G K contains an algebraic subgroup H which is affine commutative (resp.unipotent, commutative unipotent, affine nilpotent, nilpotent trigonalizable, trigonalizable, affine solvable).We consider the schematic closure of H L in G B , this defines a flat B-group scheme H of closed fiber H l which is a subgroup of (G k ) l .Since G B is separated so is H. Raynaud's affiness criterion [PY, prop 3.1] ensures that H is affine over B.
We combine Lemma 3.1 and 3.3 for H and obtain that H l is affine commutative (resp.unipotent, commutative unipotent, affine nilpotent, nilpotent trigonalizable, trigonalizable, affine solvable) of dimension d.
Once again, by induction on n, we may assume that l = k(t).We have that G k((t)) admits the subgroup M which is affine commutative (resp.unipotent, affine nilpotent, nilpotent trigonalizable, trigonalizable, affine solvable).By performing the same method as above in the case of the DVR k[[t]], it follows that G k contains an affine (resp.unipotent, affine nilpotent, nilpotent trigonalizable, trigonalizable, affine solvable) subgroup of dimension d.

More permanence properties.
Lemma 3.5.Let G be an algebraic group defined over a field k.
(2) If X is a connected smooth k-variety such that X(k) = ∅, we have D(k, G) = D(k(X), G).

Proof. (1) We have the obvious inequalities
(2) We have D(k, G) ≤ D(k(X), G).For proving the converse inequality, we pick a point x ∈ X(k).Let (t 1 , . . ., t d ) be a system of parameters of the regular local ring R = O X,x .Then its completion is k-isomorphic to k[[t 1 , . . ., t d ]] which embeds in the field of iterated Laurent series k((t 1 )) . . .Proof.We prove both statements simultaneously.Let d • one of the function.The problem is local so we can assume that S = Spec(A) for A an integral local ring of fraction field K and residue field k.We have to show that d By using the standard yoga of noetherian reduction [SGA3,VI B .10.2], we can assume that A is furthermore noetherian.If A is a field, we have K = k and this is obvious.We assume that A is not a field.According to [EGA2,prop. 7.1.7],there exists an extension L of K (of finite type) and equipped with a discrete valuation such that its valuation ring B dominates A, that is, A ⊂ B and m B ∩ A = m A .We denote by l the residue field of In other words the problem reduces to the case of a discrete valuation ring A with fraction field K. Our assumption is that G K contains a closed subgroup of dimension d which is commutative (resp.nilpotent, solvable, affine commutative, unipotent, commutative unipotent, nilpotent, nilpotent trigonalizable, nilpotent, nilpotent trigonalizable, nilpotent, trigonalizable, solvable).Then there exists a finite K-subextension K ′ of K such that the same holds for G K ′ .Let v ′ be an extension of the valuation v K to K ′ and denote by B ′ its valuation ring and by k ′ its residue field.Once again we have d 2) and ( 3) shows that G k ′ contains a closed subgroup of dimension d which is commutative (resp.nilpotent, solvable, affine commutative, unipotent, commutative unipotent, nilpotent, nilpotent trigonalizable, nilpotent, nilpotent trigonalizable, nilpotent, trigonalizable, solvable).We get then D

Finite Groups
One motivation for considering this question comes from a problem about finite groups.Let G be a group acting on a set X. A base of G acting on X is a subset Y of X such any element of g ∈ G which fixed Y pointwise acts trivially on X.The base size b(G, X) is the minimal cardinality of a base.In the case of finite groups, this has been classical object of study for more than 150 years.This has had many applications (e.g. in computational group theory).One is also interested in this from a probabilistic point of view; what is the proportion of the subsets of size b which are a base.
In [BGS], this was considered for G a simple algebraic group acting on the homogeneous space G/M with M a maximal closed subgroup and in almost cases b(G, M) was determined exactly (in a few cases, there was a small range of possible values).In this case one can consider two other quantities.We define b 0 (G, X) = c to be the smallest positive integer c so that there is subset Y of X of size c so that the pointwise stabilizer of Y is finite and b 1 (G, X) = e where e is the smallest positive integer such that the pointwise stabilizer of e points is trivial.It is easy to show that b Note that this can be rephrased in terms of G acting on (G/M) e and asking if there is a regular orbit or an orbit of dim G or if the generic orbit is regular.
More generally let G be an algebraic group acting on a variety X and assume that the action is defined over a finite field.We are interested the stabilizers in G(F q ) of a point x ∈ X(F q ) (and more generally we consider Steinberg-Lang endomorphisms with finite group of fixed points).Note that X(F q ) may not be a single orbit for G(F q ) if the stabilizer G x is not connected.
As noted the stabilizer scheme {(g, x)|g ∈ G, x ∈ X, gx = x} satisfies our hypotheses and so our results apply in this case.In particular, if for generic x, G x contains a d-dimensional connected unipotent subgroup, then G y does for all y ∈ X.Then Corollary 1.3 implies that if y ∈ X(F q ) and G y has a smooth d-dimensional connected unipotent subgroup, then G y (F q ) contains a subgroup of order q d .In particular, this gives lower bounds for the base size for G(F q ) in terms of the base size of G (there are examples where the base size of the finite group can be smaller or larger although not by much -see [BGS]).Proof.Using the same kind of argument as in the proof of Theorem 4.1 it is enough to deal with the case S = Spec (A) where A is a DVR with fraction field K and of residue field k.

Derived Subgroups
Let n ≥ 1 be an integer and consider the commutator map c n : G 2n → G. Let C n ⊂ G be the schematic closure of C n,K ; it is flat over A so equidimensional of dimension d according to [EGA4,12.1.1.5] Corollary 6.1.Let S be a scheme and let G be an S-group scheme of finite presentation.
(1) Assume that G is smooth.Then the functions s → d(G κ(s) ) and s → d 0 (G κ(s) ) are lower semi-continuous.
(2) Assume that S is irreducible with generic point ) so that Proposition 6.1 implies that d is lower semi-continuous.For the other function we consider the (smooth) S-group scheme G 0 defined in [SGA3,VI B .3.10] This function d may fail to be upper semi-continuous even in the case of a smooth affine group scheme over a DVR with connected fibers; in that case it would be locally constant according to Corollary 6.1.(1).Lemma 6.2.Let k be a an algebraically closed field and G be a split semisimple simply connected k((t))-group assumed almost simple of rank r.Let B be Bruhat-Tits k[[t]]-group scheme attached to an Iwahori subgroup of G(k((t))).Then we have Such a B is smooth and has connected fibers according to [BT,prop. 4.6.32].
Proof.In this case G k[[t]] is a Bruhat-Tits group scheme attached to the maximal parahoric subgroup G(k[[t]]) of G(k((t))).The Bruhat-Tits correspondence [BT,th. 4.6.35] is a bijection between the k-parabolic subgroups of G k and the parahoric subgroups of G(k ) and a Bruhat-Tits group scheme B such that B k occurs as quotient of B k .In particular B k maps onto a Borel k-subgroup of G k so admits a commutative quotient of dimension r.It follows that d We have proven that for one specific Iwahori subgroup but this is enough by conjugacy reasons.
We conclude by giving an example of a stabilizer scheme where the generic stabilizer is simple of dimension 3 but some stabilizer is abelian (also of dimension 3). Let ) is a generic point, then the stabilizer of x is the subgroup acting trivially on the nondegenerate 2-space spanned by v 1 and v 2 and so G x ∼ = Sp 2 (k).If y = (w 1 , w 2 ) with w 1 and w 2 spanning a totally singular 2-space, then G y is the unipotent radical of the parabolic subgroup stabilizing the space spanned by w 1 and w 2 .In particular, for a generic point x, G x is nonsolvable while G y is abelian.Corollary 6.2.Let S be a scheme and let G be an S-group scheme of finite presentation.
(1) Assume that G is smooth.Then the functions s → d ab (G κ(s) ) and s → d 0 ab (G κ(s) ) are upper semi-continuous.
(2) Assume that S is irreducible with generic point ξ such that G κ(ξ) is smooth.Then d ab (G κ(ξ) ) ≤ d ab (G κ(s) ) for each s ∈ S. neutral component of the algebraic group G K (resp.G k ).Also the A-group scheme G 0 is affine according to Raynaud's criterion.
Lemma 7.1.(1)states that G 0 k is unipotent if and only if all tori of G 0 k are trivial.Let T 0 be a maximal torus of G 0 k .Since A is henselian, T 0 lifts to a subtorus T of G 0 by using Grothendieck's representability theorem [SGA3,XI.4.1].Since G K is unipotent, T K is trivial so that T = 1 and T 0 = 1.It follows that G 0 k is unipotent so that d u (G) = d.We consider now the general case.According to [PY,prop. 3.4] (based on [A, app. II]), there exists a local extension A ′ of A of DVR's such that the normalization G ′ of G ′ = G × A A ′ is smooth over A ′ and such that the fraction field K ′ is finite over K.We denote by k ′ the residue field of A ′ .According to [PY,thm. A.6] the normalization morphism h : G ′ → G ′ is finite.In particular the morphism of smooth affine connected K ′ -groups h 0 K ′ : G ′ ) K ′ 0 → G K ′ is an isogeny between smooth affine connected K ′ -groups.
(2) shows that ( G ′ ) 0 K ′ is unipotent.The smooth case applied to ( G ′ ) 0 over A ′ shows that ( G ′ ) 0 k ′ is unipotent of dimension d.Since the homomorphism h 0

Lemma 2. 1 .
If k is algebraically closed, then D(k, G) = D(F, G) for any field extension F/k and for each function D as above.

Lemma 2. 2 .
We have d(k, G) = d(F, G) for any field extension F/k and for each function d as above.Proof.The function d is the absolute version of a relative rank function D. Let F be an algebraic closure of F containing k. Lemma 2.1 shows that D

2. 3 .
Connections with the literature.(a) In [SGA3, XII.1], Grothendieck defines related rank functions but which are different.For example the Grothendieck unipotent rank ρ u (G) of a smooth connected group G over an algebraically closed field is d u (C) where C is a Cartan subgroup of G 0 .This function ρ u is upper semi-continuous if G is smooth affine over a base [SGA3, XII.2.7.(i)].Our result does not require smoothness (nor flatness).

4.
Upper semi-continuity Theorem 4.1.define d • (s) = d • (G κ(s) ) for each s ∈ S. (1) The functions d ′ c , d ′ n , d ′ s on S arer upper semi-continuous.(2) Assume that G is separated.The functions d c , d u , d cu , d n , d s on S arer upper semicontinuous.

6. 1 .
For an algebraic group G defined over a field k, we define d(G) (resp.d 0 (G)) the dimension of the derived group of the smooth k-group G k,red (resp.the smooth connectedk-group G 0 k,red ).If G is smooth, we have d(G) = dim k (DG) and d 0 (G) = dim k (D(G 0 )).It is convenient to introduce a third dimension function d + (G)which is the supremum of the dimensions of the C n 's where C n stands for the schematic image of the commutator map c n :G 2n → G, c n (x 1 , y 1 , . . ., x n , y n ) = [x 1 , y 1 ] . . .[x n , y n ].Since the formation of the schematic image commutes with flat base change, d + (G) is insensitive to an arbitrary field extension.We haved 0 (G) ≤ d(G) ≤ d + (G) and d(G) = d + (G) for G smooth.Proposition 6.1.Let S be a scheme and let G be a flat S-group scheme of finite presentation.Then the function s → d + (G κ(s) ) is lower semi-continuous.

6. 2 .
Abelianization.A variant is the following.For an algebraic group G defined over a field k, we define d ab (G) (resp.d 0 ab (G)) the dimension of the abelianization of the smooth k-group G k,red (resp.the smooth connected k-group G 0 k,red ).We have d(G) + d ab (G) = dim k (G) and similarly d 0 ab (G) + d 0 ab (G) = dim k (G).