CONSTRUCTIVE NONCOMMMUTATIVE INVARIANT THEORY

The problem of finding generators of the subalgebra of invariants under the action of a group of automorphisms of a finite-dimensional Lie algebra on its universal enveloping algebra is reduced to finding homogeneous generators of the same group acting on the symmetric tensor algebra of the Lie algebra. This process is applied to prove a constructive Hilbert–Nagata Theorem (including degree bounds) for the algebra of invariants in a Lie nilpotent relatively free associative algebra endowed with an action induced by a representation of a reductive group.


Introduction
Let G be a subgroup of the automorphism group of a finite-dimensional Lie algebra L over a field K of characteristic zero. Then there is a natural induced action of G on the universal enveloping algebra U (L) via associative K-algebra automorphisms. First we describe a process by which one can construct generators of the subalgebra U (L) G of G-invariants in U (L) starting from a homogeneous generating system of S(L) G , the commutative algebra of invariants in the symmetric tensor algebra S(L) of L.
Namely, there is a well-known K-vector space isomorphism ω : S(L) → U (L) called the canonical bijection in [5, 2.4.6]. The details of the construction of ω will be given in Section 2, see (3). Note that the map ω is not an algebra homomorphism. However, we have the following: Theorem 1. Suppose that {f λ | λ ∈ Λ} is a homogeneous generating system of the commutative K-algebra S(L) G , where G is a subgroup of the automorphism Theorem 1 will be applied to noncommutative invariant theory in relatively free associative algebras. In this paper by 'algebra' we mean an associative K-algebra with unity, unless explicitly stated otherwise. The base field K is always assumed to have characteristic zero. For a finite-dimensional K-vector space V denote by T (V ) the tensor algebra of V ; that is, T (V ) is the free algebra generated by a basis of V . Given a variety R of algebras we write F (R, V ) for the relatively free algebra in R generated by a basis of V . Recall that R consists of all algebras that satisfy a given set of polynomial identities, so F (R, V ) is the factor algebra of T (V ) modulo a T-ideal (an ideal stable under all algebra endomorphisms of T (V )). Since we deal with algebras with unity and K is infinite, R necessarily contains the variety of commutative algebras, and we have the canonical surjections where S(V ) is the symmetric tensor algebra of V (i.e., the commutative polynomial algebra generated by a basis of V ). The above algebra surjections are GL(V )equivariant. They are homomorphisms of graded algebras, where the algebra T (V ) = ∞ d=0 T d (V ) is endowed with the standard grading: the dth tensor power T d (V ) of V is the degree d homogenerous component of T (V ). Given a non-zero homogeneous element f in any of T (V ), F (R, V ) or S(V ) we shall write deg(f ) for the degree of f . Note that when R is the variety of all algebras we have that F (R, V ) = T (V ), and when R is the variety of commutative algebras we have F (R, V ) = S(V ).
Let G be a linear algebraic group and V a finite-dimensional rational G-module; that is, we are given a group homomorphism ρ : G → GL(V ) that is a morphism of affine algebraic varieties (from now on we shall usually omit the attribute 'rational' and simply say that V is a G-module). The action of G on V induces an action on T (V ), F (R, V ), and S(V ) via automorphisms of graded algebras, and the above surjections are G-equivariant. We are interested in the subalgebra of G-invariants. We refer to [8] and [10] for surveys on results concerning subalgebras of invariants in relatively free algebras.
For an integer p ≥ 1 denote by N p the variety of Lie nilpotent algebras of Lie nilpotency index less or equal to p. In other words, N p is the variety of algebras satisfying the polynomial identity [x 1 , . . . , Remark 1. For a (nonunitary) associative algebra nilpotence of index ≤ p means that the algebra satisfies the polynomial identity x 1 · · · x p = 0. By analogy with group theory a Lie algebra is nilpotent of index ≤ p, if it satisfies the polynomial identity [x 1 , . . . , x p+1 ] = 0.
Our starting point is the following non-commutative generalization of the Hilbert-Nagata theorem (see for example [

])
Suppose that G is reductive, and R ⊆ N p for some p ≥ 1. Then F (R, V ) G is a finitely generated algebra for any G-module V . Remark 2. The assumption R ⊆ N p for some p ≥ 1 above is necessary, as the following converse of Theorem 2 is also shown in [6]: If dim K (V ) ≥ 2 and F (R, V ) G is finitely generated for all reductive subgroups G of GL(V ), then R ⊆ N p for some p ≥ 1.
The proof of Theorem 2 in [6] is non-constructive, since it uses the Noetherian property of F (R, V ) for R ⊂ N p (proved in [17]) similarly to the fundamental paper of Hilbert [13] on the commutative case p = 1, where the Hilbert Basis Theorem was proved. Hilbert gave a more constructive proof of the commutative case in [14] (explicit degree bounds for the generators were first proved by Popov [20], [21], and stronger bounds more recently by Derksen [2]). For constructive (commutative) invariant theory of reductive groups see also [1], [4].
In the present paper we make Theorem 2 constructive. In algorithms for computing generators of algebras of invariants a crucial role is played by degree bounds. For example, an a priori degree bound for the generators of the algebra F (R, V ) G implies an algorithm to find explicit generators (by solving systems of linear equations). From a different perspective, sometimes there is a qualitatively known process that yields generators of the algebra of invariants, and the aim is to derive from it a degree bound for a minimal generating system, to have some quantitative information on the algebra of invariants.
In order to discuss degree bounds we need to introduce some notation. Given a graded algebra A = ∞ d=0 A d we denote by β(A) the minimal non-negative integer d such that A is generated by homogeneous elements of degree at most d (and write It is a classical theorem of E. Noether [19] that for G finite we have β(G) ≤ |G|. On the other hand Derksen and Kemper [3, Thm. 2.1] proved that β(G) = ∞ for any infinite group G. So for an infinite group G, finite degree bounds may hold only for restricted classes of G-modules.
Given a set τ of isomorphism classes of simple G-modules, denote by add(τ ) the class of all G-modules that are finite direct sums of simple modules whose isomorphism class belongs to τ . This definition is particularly natural when the group G is reductive, because in that case any G-module decomposes as a direct sum of simple G-modules. By slight abuse of notation we write V ∈ τ if the isomorphism class of the G-module V belongs to τ . Moreover, write τ ⊗p for the isomorphism classes of simple summands of all G-modules V 1 ⊗ · · · ⊗ V q , where V i ∈ τ and q ≤ p. Set Weyl's theorem [24] on polarizations implies the following: Proposition 3. Let G be a reductive group and let τ be a finite set of isomorphism classes of simple G-modules. Then the number β τ is finite.
Proof. Let V 1 , . . . , V q be simple G-modules representing the isomorphism classes in τ . Let V be an arbitrary G-module in add(τ ).
where the m i are non-negative integers and m i V i stands for the direct sum of m i copies of V i . Weyl's theorem on polarizations (the special case h = 1 of Theorem 9 below) implies that (1) (The second inequality above follows from the fact that if A is a submodule of the G-module B, then there is a G-equivariant graded K-algebra surjection S(B) → S(A) mapping some of the variables to zero.) Since (1) holds for any V ∈ add(τ ), we conclude the equality The assumption on G guarantees that the number on the right-hand side above is finite.
Turning to a variety R of associative algebras, we write is the supremum of the degrees of the elements in a minimal homogeneous generating system of the algebra F (R, V ) G . Having established this notation we are in position to state the following corollary of the results of the present paper: Theorem 4. Suppose that the group G is reductive and the variety R is contained in N p for some p ≥ 1. Let τ be a finite set of isomorphism classes of simple Gmodules. Then we have the inequality In fact the results of this paper give more: for any (not necessarily reductive) group G ≤ GL(V ) the construction of an explicit generating system of F (R, V ) G (where R is contained in N p for some p ≥ 1) is reduced to finding a generating system of a commutative algebra of G-invariants S(W ) G for a G-module W associated canonically to V . For the precise statement see Theorem 7.
The paper is organized as follows. In Section 2 we prove Theorem 1, which is then applied in Section 3 to prove Theorem 7. Finally, Theorem 4 is obtained as a consequence of Theorem 7.
Since the technique of polarization in commutative invariant theory is fundamental for Proposition 3, in Section 4 we investigate to what extent it works in our noncommutative setup. Theorem 9 is a noncommutative generalization of Weyl's theorem on polarization; however, it applies for a class of varieties different from those appearing in Theorem 2, Theorem 4, or Theorem 7. Section 4 is logically independent from the previous sections.
Acknowledgements. We are very grateful to the referees for their suggestions. In particular, it was the referee's insight that our results -originally dealing only with relatively free algebras -should be treated as a consequence of a statement on universal enveloping algebras.

Universal enveloping algebras
Returning to the setup of the first paragraph of the Introduction, denote by π U (L) : T (L) → U (L) the defining surjection onto U (L) from the tensor algebra . Moreover, consider the linear map is a K-vector space isomorphism called the canonical bijection in [5, 2.4.6]. Furthermore, we have the following: where Aut(L) denotes the automorphism group of the Lie algebra L. In particular, for any subgroup G of Aut(L), the canonical bijection restricts to a K-vector space isomorphism Proof. The universal properties of T (L) and U (L) imply that the action of Aut(L) on L extends uniquely to an action via K-algebra automorphisms on T (L) and U (L), and the algebra homomorphism π U (L) is Aut(L)-equivariant. The map ι d : is even GL(L)-equivariant, and clearly Aut(L) ≤ GL(L). Therefore the composition ω d = π U (L) • ι d is Aut(L)-equivariant for all d, implying in turn that ω is Aut(L)-equivariant. Consequently, the K-linear isomorphism ω : S(L) → U (L) is in fact an isomorphism of G-modules for any subgroup G in Aut(L), and thus ω restricts to a linear isomorphism between the subspaces S(V ) G → U (L) G of G-invariants.
As ω is not an algebra homomorphism in general, further considerations are needed to prove Theorem 1. Write η d : So µ d is an isomorphism of Aut(L)-modules, and restricting to G-invariants (where G is a subgroup of Aut(L)) we obtain the following commutative diagram: Proof of Theorem 1. Since the space S d (L) G is mapped by µ d isomorphically onto Now let {f λ | λ ∈ Λ} be a homogeneous system of generators of S(L) G . The maps ι d : S d (L) → T d (L) defined by (2) are GL(L)-equivariant, hence they are G-equivariant as well. In particular, Denote by M the subalgebra of T (L) generated by { f λ | λ ∈ Λ}. By construction, M is generated by homogeneous G-invariants, (4) and (5) we have the commutative diagram The restriction of π S(L) to M is a homomorphism of graded algebras M → S(L) G . By construction of M the generators of S(L) G are contained in π S(L) (M ). It follows that π S(L) (M ) = S(L) G , and since π S(L) preserves the grading, we have

and by the induction hypothesis
In the above proof we pointed out that the natural surjection η d : (5)). For later use we state a general lemma guaranteeing that a surjective map restricts to a surjection between the corresponding subspaces of invariants.
Given a reductive group H, recall that by a rational H-module we mean a not necessarily finite-dimensional vector space X together with an action of H via linear transformations, such that any x ∈ X is contained in a finite-dimensional subspace Y of X which is stable under the action of H, i.e., H · Y = Y , and the group homomorphism H → GL(Y ) giving the action of H on Y is a morphism of affine algebraic varieties. We shall denote by X H the subspace of H-invariants in X.

Lemma 6.
Let H be a reductive group and ϕ : X → Z a surjective homomorphism of rational H-modules. Then for any subgroup G of H we have Z G = ϕ(X G ).
Proof. Since H acts completely reducibly on X, the H-submodule ker(ϕ) has an Hmodule direct complement Y in X, so X = ker(ϕ)⊕Y . The restriction ϕ| Y : Y → Z is an H-module isomorphism, hence it is a G-module isomorphism as well, thus it restricts to a K-vector space isomorphism Y G → Z G . Clearly Y G ⊆ X G , so we derive ϕ(X G ) ⊇ ϕ(Y G ) = Z G . The reverse inclusion ϕ(X G ) ⊆ Z G holds for any G-module homomorphism.

Lie nilpotent relatively free algebras
Fix an integer p ≥ 1, a variety R contained in N p , and a finite-dimensional rational G-module V (where G is an arbitrary linear algebraic group). Without loss of generality we may assume that G ≤ GL(V ) (i.e., V is a faithful G-module). Set F := F (R, V ). Consider the subspaces in T (V ) given by 1, 2, . . . , and V [1] := V . As a consequence of the Jacobi identity we have The subspace and let T := T (V [≤p] ) be the tensor algebra generated by V [≤p] . Denote by ε i the ith standard basis vector ε i := (0, . . . , 0, 1, 0, . . . , 0) ∈ N p 0 (the coordinate 1 is in the ith position). The algebra T has an N p 0 -grading Note that GL(V ) acts on T via N p 0 -graded algebra automorphisms. The symmetric tensor algebra S := S(V [≤p] ) is endowed with the analogous N p 0 -grading, so the natural surjection π S(L) : T → S is a GL(V )-equivariant homomorphism of N p 0graded algebras.
The tautological linear embedding of V [≤p] into T (V ) extends to a GL(V )equivariant algebra surjection π : T → T (V ). Composing this with the natural surjection ν : T (V ) → F (R, V ) we obtain A homogeneous element in T may not be mapped to a homogeneous element of T (V ) by π (or of F by π F ). However, the multihomogeneous component T α ⊂ T is mapped under π (respectively π F ) into the homogeneous component of Denote by ι α : S α → T α the linear map given by (where Sym d stands for the symmetric group on {1, . . . , d} and v 1 , . . . , v |α| ∈ V [≤p] ). Obviously ι α is GL(V )-equivariant (as it is even GL(V [≤p] )-equivariant), and hence ι α is G-equivariant (just like π S(L) ). Thus ι α (S G α ) ⊆ T G α . Moreover, π S(L) • ι α is the identity map on S α , consequently π S(L) (ι α (S G α )) = S G α . In parti- Theorem 7. Let R be a variety contained in N p for some p ≥ 1. Let V be a Gmodule, and S, T , F the algebras defined above. Take a multihomogeneous (with respect to the N p 0 -grading (7)) K-algebra generating system {f λ | λ ∈ Λ} of S G , denote by α λ the multidegree of f λ , and set Then the K-algebra F G is generated by the homogeneous elements Remark 3. The algebra S G is known to be finitely generated (in addition to the case when G is reductive) also when G is a maximal unipotent subgroup of a reductive group (see [12] or [11,Thm. 9.4]), and consequently when G is a Borel subgroup of a reductive group. In these cases Theorem 7 reduces the construction of a finite generating system of F G to the construction of a finite generating system in the commutative algebra of invariants S G .
Proof of Theorem 7. The subspace is a Lie subalgebra (in fact it is the free Lie algebra generated by dim K (V ) elements). It contains the Lie ideal d>p V [d] , and the corresponding factor Lie algebra L p (V ) is the relatively free nilpotent Lie algebra of nilpotency index p generated by dim K (V ) elements. We identify the underlying vector space of L p (V ) with p d=1 V [d] in the obvious way. Then T is identified with T (L p (V )) and S is identified with S(L p (V )). Applying Theorem 1 for the Lie algebra L p (V ) and the group G ≤ GL(V ) (note that GL(V ) is naturally a subgroup of the automorphism group Aut(L p (V )) of the Lie algebra L p (V )) we obtain that is a generating system of U (L p (V )) G . By relative freeness of L p (V ) in the variety N p , the identity map V → V extends to a (surjective) Lie algebra homomorphism from L p (V ) onto the Lie subalgebra of F generated by V . Now the universal property of the universal enveloping algebra implies that this Lie algebra homomorphism extends to an associative algebra homomorphism γ : U (L p (V )) → F . The image of γ contains the K-algebra generating system V ⊂ F , hence γ is surjective onto F . It is also GL(V )-equivariant, therefore by Lemma 6 we have γ(U (L p (V )) G ) = F G . It follows that the generating system (9) of U (L p (V )) G is mapped by γ to a generating system of F G . On the other hand, by construction π F and γ • π U (Lp(V )) agree on the subspace V [≤p] = L p (V ) of T generating T as a K-algebra, hence Thus π F ( f λ ) = γ(π U (Lp(V )) ( f λ )), and these elements generate F G as λ varies over Λ.
Proof of Theorem 4. Let V be a G-module in add(τ ). Now G is reductive, so we may take a finite minimal multihomogeneous (with respect to the N p 0 -grading (7)) generating system {f λ | λ ∈ Λ} of S(V [≤p] ) G . Consider the generating system {π F ( f λ ) | λ ∈ Λ} of F (R, V ) G given by Theorem 7. Note that {f λ | λ ∈ Λ} is a minimal homogeneous generating system of S G with respect to the standard grading, where V [≤p] is the degree 1 homogeneous component of S. Since G is reductive, all G-modules are completely reducible. Moreover, V [q] is a GL(V )module (hence G-module) direct summand in V ⊗q . It follows that V [≤p] belongs to add(τ ⊗p ), implying that deg(f λ ) = p i=1 α λ,i ≤ β τ ⊗p for each λ ∈ Λ (recall that the multidegree of f λ is α λ = (α λ,1 , . . . , α λ,p ), and deg(f λ ) here stands for the degree of f λ with respect to the standard grading on S, for which the generators have degree 1). So we have In other words, F (R, V ) G is generated by elements of degree at most pβ τ ⊗p . This holds for any G-module V ∈ add(τ ), therefore the desired inequality β τ,R ≤ pβ τ ⊗p holds.
Remark 4. In the proof of Theorem 4 we used that all simple G-submodules of V [q] are contained in V ⊗q , because V [q] is a GL(V )-submodule of V ⊗q . We mention that it was proved by Klyachko [16] that for q = 4, 6, all simple GL(V )-submodules of V ⊗q different from the qth exterior or symmetric powers of V are contained in V [q] . Following [7] for a finite group G we set Corollary 8. For a finite group G we have Remark 5. The results of [7] provide an upper bound for β(G, N p ) of different nature.

A noncommutative generalization of Weyl's theorem
In this section we assume that the variety R is generated (in the sense of universal algebra) by a finitely generated algebra. Kemer [15] proved that then R satisfies a Capelli identity. This means that there exists a positive integer h = h(R) such that all algebras in R satisfy the polynomial identity π∈Sym h+1 (−1) π x π(1) y 1 x π(2) y 2 · · · y h x π(h+1) = 0.
Our focus is on the case when V = U + mW , where mW = W + · · · + W is the direct sum of m copies of a G-module W , and U is a G-module. We shall identify mW with W ⊗ K m , which is naturally a GL(W ) × GL(K m )-module (here K m stands for the space of column vectors of length m over K). Also V = U +mW is naturally a GL(U ) × GL(W ) × GL(K m )-module. Moreover, for l ≤ m we shall identify K l with the subspace of K m consisting of column vectors whose last m − l coordinates are zero. Accordingly lW is identified with the subspace of mW consisting of m-tuples of elements of W with the zero vector as the last m − l component. In this way F (R, U + lW ) becomes a subalgebra of F (R, U + mW ).
Theorem 9. Let G be a group, let U and W be G-modules, n = dim(W ), and let B be a set of generators of the algebra F (R, U + nhW ) G , where h = h(R) as above. Then for any m ≥ nh the algebra F (R, U + mW ) G is generated by Recall that the relatively free algebra F (R, V ) is graded such that the subspace V is the degree 1 homogeneous component. Remark 6. In the special case when R is the variety of commutative algebras we have h(R) = 1, and hence Theorem 9 in this special case recovers Weyl's theorem on polarization, which is a main theme in [24].
In order to prove Theorem 9 we need to recall some facts about polynomial representations of the general linear group. A partition λ of d (we write λ d) is a finite non-increasing sequence λ = (λ 1 , λ 2 , . . . ) of non-negative integers (with the convention that zeros can be freely appended to or removed from the end) such that i λ i = d. Write ht(λ) for the number of non-zero components of λ. Denote by S λ (.) the Schur functor (see for example [22]), so for a finite-dimensional Kvector space E and a partition λ with ht(λ) ≤ dim(E), we have that S λ (E) is a simple polynomial GL(E)-module.
Lemma 11. If the simple GL(E)-module S µ (E) occurs as a direct summand in S λ (U + nE), where n is a positive integer and GL(E) acts trivially on the finitedimensional K-vector space U , then ht(µ) ≤ n · ht(λ).
Proof. Suppose that S µ (E) occurs as a direct summand in S λ (V ), where V = U + nE. It follows from Pieri's rules (see [18]) that S λ (V ) is a direct summand in where for the partition (d) with only one non-zero part, S (d) (V ) is the dth symmetric tensor power of V . Therefore S µ (E) is a direct summand in ht(λ) i=1 S (λi) (U + nE). Any simple GL(E)-module summand of S (λi) (U + nE) is isomorphic to S ν i (E) for some partition ν i = (ν i 1 , . . . , ν i n ) with ht(ν i ) ≤ n by the Cauchy identity (see [22]). Therefore again by (11), S µ (E) is a direct summand in S ν 1 (E) ⊗ · · · · · · ⊗ S ν ht(λ) (E), hence S µ (E) is a direct summand in We conclude by Pieri's rules that ht(µ) is bounded by the number nht(λ) of tensor factors in the above expression.
Proof of Theorem 9. Set V = U + mW = U + W ⊗ K m . Since F (R, V ) satisfies the Capelli identity (10), by a theorem of Regev [23] (see also [9,Thm. 2.3.4]) no S λ (V ) with ht(λ) > h = h(R) occurs as a summand of F (R, V ). So the degree d homogeneous component of F (R, V ) has the GL(V )-module decomposition with some non-negative integers m λ . Setting E = K m and n = dim(W ), Lemma 11 shows that as a GL(E)-module, F (R, U + W ⊗ E) d decomposes as with some non-negative integers r µ . Since the actions of G and GL(E) commute, the algebra F (R, U + W ⊗ E) G is a GL(E)-submodule in F (R, U + W ⊗ E). Thus each simple GL(E)-module direct summand of F (R, U + W ⊗ E) G is isomorphic to S µ (E) for some partition µ with ht(µ) ≤ nh. Such a summand is generated by a highest weight vector w, having the property that for an element g = diag(z 1 , . . . , z m ) ∈ GL(K m ) = GL(E) we have g · w = (z µ1 1 · · · z µm m )w. Since µ nh+1 = · · · = µ m = 0, we conclude that w belongs to the subalgebra F (R, U +W ⊗ K nh ) = F (R, U +nhW ) of F (R, U +W ⊗E). Thus F (R, U +W ⊗E) G is contained in the GL(E)-submodule of F (R, U + W ⊗ E) generated by F (R, U + W ⊗ K nh ) G . Now {g · f | g ∈ GL(K m ), f ∈ B} spans a GL(E)-submodule, hence it generates a GL(E)-stable subalgebra of F (R, U + W ⊗ E) G . By construction this subalgebra contains F (R, U + W ⊗ K nh ) G , so it contains F (R, U + W ⊗ E) G .