On Classical groups detected by the triple tensor product and the Littlewood–Richardson semigroup

Abstract

Langlands’ beyond endoscopy proposal for establishing functoriality motivates the study of irreducible subgroups of $\mathrm {GL}_n$ that stabilize a line in a given repesentation of $\mathrm {GL}_n$. Such subgroups are said to be detected by the representation. In this paper we continue our study of the important special case where the representation of $\mathrm {GL}_n$ is the triple tensor product representation $\otimes ^3$. We prove a family of results describing when subgroups isomorphic to classical groups of type $B_{n}$, $C_n$, $D_{2n}$ are detected.

1 Background

Let F be a number field and let $\mathbb {A}_F$ be the adeles of F. Let H be a reductive group over F with Langlands dual group ${}^LH$. Given a representation

$$\begin{aligned} {}^LH\longrightarrow \mathrm {GL}_N(\mathbb {C}), \end{aligned}$$

(1.1)

Langlands’ functorial conjectures [20] predict there should be a corresponding transfer of L-packets of automorphic representations of $H(\mathbb {A}_F)$ to isomorphism classes of automorphic representations of $\mathrm {GL}_N(\mathbb {A}_F)$.

One can ask for a characterization of those automorphic representations in the image. To explain Langlands’ conjectural criterion for an automorphic representation to be in the image, we recall the following definition from [11].

Definition 1.1

Let H be an irreducible reductive subgroup of $\mathrm {GL}_N$. We say a representation r : $\mathrm {GL}_N \longrightarrow \mathrm {GL}(V)$ detects H if H stabilizes a line in V.

Remark

If H is connected then r detects H if and only if it detects $H^{\mathrm {der}}$.

Let ${}^{\lambda }H$ denote the Zariski closure in $\mathrm {GL}_N(\mathbb {C})$ of the image of ${}^LH$ under the map (1.1). The following conjecture is the crux of Langlands’ beyond endoscopy proposal [21], which aims to prove Langlands functoriality in general:

Conjecture 1.2

Let $\pi$ be a unitary cuspidal automorphic representation of $\mathrm {GL}_N(\mathbb {A}_F).$ If $\pi$ is a functorial transfer from H , then $L(s,\pi ,r \otimes \chi )$ has a pole at $s=1$ for some character $\chi \in F^{\times } \backslash \mathbb {A}_F^{\times } \rightarrow \mathbb {C}^\times$ whenever r detects ${}^{\lambda }H$.

Motivated by Langlands’ proposal, in the recent paper [11], the author proposed the following concrete question in algebraic group theory:

Question 1.3

Given a representation

$$\begin{aligned} r:\mathrm {GL}_N \longrightarrow \mathrm {GL}(V) \end{aligned}$$

which algebraic subgroups of $\mathrm {GL}_N$ are detected by r?

If $r=\mathrm {Sym}^2$, one knows that every irreducible reductive subgroup of $\mathrm {GL}_N$ detected by r is conjugate to a subgroup of $\mathrm {GO}_N$. Moreover, in this case Conjecture 1.2 is proven by work of Arthur [2], work of Cogdell et al. [5] and work of Ginzburg et al. [10]. There is a similar statement for $r=\Lambda ^2$. Thus the case $r=\otimes ^2$ is relatively well-understood.

Apart from this special case, explicit results are hard to come by (but see [9]). In [11] we initiated the study of the subgroups of $\mathrm {GL}_N$ detected by $\otimes ^3$ by studying irreducible simple subgroups of type $A_n$. In this paper we continue this investigation for the series of classical groups.

For the remainder of the paper we make the following assumption on an algebraic group G over $\mathbb {C}$:

• (A1) The algebraic group G is one of the classical groups $\mathrm {SO}(2n+1)$, $\mathrm {Sp}(2n)$ and $\mathrm {SO}(2n)$, where for $\mathrm {SO}(2n)$, we further assume n is even.

It is well-known [25, Lemma 4.2] that all irreducible modules of G are then self-dual. This is false for $\mathrm {SO}(2n)$ when n is odd, which is why we assume that n is even. We investigate which irreducible subgroups of $\mathrm {GL}_N$ isomorphic to some G as above are detected by the representation $\otimes ^3: \mathrm {GL}_N \longrightarrow \mathrm {GL}_{N^3}$.

Throughout the paper, for $n\geqslant 1$, we let

$$\begin{aligned} P_n=\{\lambda : \lambda =(\lambda _1, \lambda _2, \dots , \lambda _n)\in \mathbb {Z}^n,\quad \lambda _1\geqslant \lambda _2\geqslant \dots \geqslant \lambda _n\geqslant 0\} \end{aligned}$$

be the semigroup of partitions with at most n parts. Let $\mathbb {S}_{\parallel \lambda \parallel }(G)$ be the irreducible subgroup of $\mathrm {GL}_N$ obtained by applying the Schur functor associated to the highest weight $\lambda$. The precise construction of $\mathbb {S}_{\parallel \lambda \parallel }(G)$ depends on the type of G and is reviewed in Sect. 2 below. We make the following additional assumption on $\lambda \in P_n$:

• (A2) For $G=\mathrm {SO}(2n)$, we assume that $\lambda _n=0$.

We make this assumption to ensure that the representation $\mathbb {S}_{\parallel \lambda \parallel }$ of G is irreducible.

We now state our results on detection. We start by observing that the property of $\mathbb {S}_{\parallel \lambda \parallel }(G)$ being detected is essentially stable under replacing $\lambda$ by its conjugate partition $\lambda '$ (see Sect. 2.1 for the definition):

Theorem 1.4

Let $\lambda \in P_n$ be a partition such that its conjugate $\lambda '$ is also in $P_n$. Then $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is detected by $\otimes ^3$ if and only if $\mathbb {S}_{\parallel \lambda '\parallel }(G)$ is detected by $\otimes ^3$.

If $\lambda$ is a partition of an odd number, then $\mathbb {S}_\lambda (G)$ is never detected:

Theorem 1.5

Let $\lambda \in P_n$ be a partition of an odd number. Then the representation $\otimes ^3$ does not detect $\mathbb {S}_{\parallel \lambda \parallel }(G)$.

Conversely, we prove the following result which constructs families of subgroups that are detected by $\otimes ^3$.

Theorem 1.6

Let $\lambda =(\lambda _1, \lambda _2, \dots , \lambda _n)\in P_n$ be a partition of an even number. If

• (a) all $\lambda _i$ are even, or

• (b) it has even number of non-zero $\lambda _i$ and all non-zero $\lambda _i$ are distinct and odd, or

• (c) $\lambda$ is a hook partition,

then the representation $\otimes ^3$ detects $\mathbb {S}_{\parallel \lambda \parallel }(G)$.

To experts in algebraic combinatorics and representation theory it is clear that the results above ought to have something to do with the famous Littlewood–Richardson semigroup $\mathrm {LR}_n$ of order n (see Sect. 3 for definition) or equivalently with the Littlewood–Richardson coefficients $c^\lambda _{\mu \nu }$ (see Sect. 2.2 for the definition). Studying $c^{\lambda }_{\mu \nu }$ or $\mathrm {LR}_n$ is a central topic in the representation theory (see [7, 22, 23] and [27], for example), in combinatorics of symmetric functions ([23, 24, 26]), in the topology Grassmann varieties and hence the theory of vector bundles and K-theory [14]. Also there has been a long history in connection to Hermitian eigenvalues and Horn’s conjecture ([12] and see [16] for example). It is well-known that $\mathrm {LR}_n$ has saturation property, namely, if $(k\lambda , k\mu , k\nu )\in \mathrm {LR}_n$ for some $k>0$, then $(\lambda , \mu , \nu )\in \mathrm {LR}_n$. This is a famous theorem of Knudson and Tao [17] (see also [18]). For various descriptions of $\mathrm {LR}_n$ we refer to [4, 13, 28]. The precise connection between $\mathrm {LR}_n$ and detection via $\otimes ^3$ is given in the following theorem, which is essentially a reformulation of the Newell–Littlewood formula (2.6):

Theorem 1.7

The subgroup $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is detected by $\otimes ^3$ if and only if there are $\alpha , \beta , \gamma \in P_n$ such that all triples $(\lambda , \alpha , \beta )$, $(\lambda , \alpha , \gamma )$ and $(\lambda , \beta , \gamma )$ are elements in $\mathrm {LR}_n$.

Theorems 1.4 and 1.5 follow formally from Theorem 1.7. However, in general, even given Theorem 1.7, it is not clear how to use the descriptions of the Littlewood–Richardson coefficients given in the references above to describe which subgroups $\mathbb {S}_{\parallel \lambda \parallel }(G)$ are detected by $\otimes ^3$. In particular our proof of Theorem 1.6 requires some “hands on” combinatorics with partitions.

Remark

Given the saturation theorem for the Littlewood–Richardson semigroup $\mathrm {LR}_n$, it is natural to ask whether $\mathbb {S}_{\parallel k\lambda \parallel }(G)$ is detected by $\otimes ^3$ for some $k>0$ implies that $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is detected. This is evidently false. Indeed, Theorem 1.5 implies that if $\lambda$ is a partition of an odd number then $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is not detected. However, $\mathbb {S}_{\parallel k\lambda \parallel }(G)$ is always detected if k is even by Theorem 1.6 (a).

Before outlining the paper we comment on these results from the perspective of Langlands’ beyond endoscopy proposal. Essentially they give some feeling of how much more complicated the structure of the beyond endoscopy proposal is than the theory of endoscopy. In the theory of endoscopy one writes the trace formula in terms of stable orbital integrals on endoscopic groups. In the beyond endoscopy proposal Langlands proposes that one writes limiting forms of the trace formula in terms of groups that are detected by a particular representation. It is evident from the theorems above that this set is much more complicated than the set of endoscopic groups of a given group. On the other hand there may be simplifications that can be made in certain situations. For example, all of the groups $\mathbb {S}_{\parallel \lambda \parallel }(G) \leqslant \mathrm {GL}_N$ considered above are conjugate to subgroups of $\mathrm {O}_N$ or $\mathrm {Sp}_N$ since the representations $\mathbb {S}_{\parallel \lambda \parallel }$ are self-dual. Thus for some purposes it might be possible to sieve them all out at the outset by restricting to non-self dual representations.

We close our introduction by outlining the paper. In Sect. 2, we review some basic facts on partitions, the Littlewood–Richardson coefficients and the groups $\mathrm {SO}(2n+1)$, $\mathrm {Sp}(2n)$ and $\mathrm {SO}(2n)$. In Sect. 3, we prove a key proposition showing a necessary and sufficient condition for detection by $\otimes ^3$ and discuss the connection to the Littlewood–Richardon semigroup $\mathrm {LR}_n$ and prove Theorems 1.4, 1.5 and 1.7. In Sect. 4, we prove Theorem 1.6 by constructing $\lambda$ explicitly such that $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is detected by $\otimes ^3$.

2 Preliminaries

2.1 Partitions

In this section, we recall some basic notion in the theory of partitions. For $\lambda \in P_n$, we let $|\lambda |=\sum _i\lambda _i$ be the number partitioned by $\lambda$. Moreover, denote by $\ell (\lambda )$ the number of non-zero $\lambda _i$ for a partition $\lambda$.

The Young diagram of the partition $\lambda =(\lambda _1, \lambda _2, \dots , \lambda _n)$ is an array of boxes arranged in left-justified horizontal rows where for each i, $\lambda _i$ is the number of boxes in the ith row [7]. For example, the Young diagram of the partition $\lambda =(5,3,1)$ is

Note that in this example, $|\lambda |=9$ and $\ell (\lambda )=3$ which corresponds the number of rows of its Young diagram.

The conjugate of a partition $\lambda$ is the partition of $|\lambda |$ whose Young diagram is obtained by reflecting the Young diagram of $\lambda$ about the diagonal so that rows become columns and columns become rows. We write it as $\lambda '$. In above example $\lambda =(5,3,1)$, the conjugate partition of $\lambda$ is $\lambda '=(3,2,2, 1,1)$ (see [1] for instance).

2.2 Littlewood–Richardson coefficients

In this section, we follow the exposition of [8] as we recall basic facts: For any n dimensional vector space V over $\mathbb {C}$ and any partition $\lambda \in P_n$, we can apply the Schur functor $\mathbb {S}_{\lambda }$ to V to obtain a representation $\mathbb {S}_{\lambda }(V)$ for $\mathrm {GL}_n$. It remains irreducible when restrict to $\mathrm {SL}_n$. In particular it determines an irreducible representation of the Lie algebra $\mathfrak {sl}_n$ (see [8, Proposition 15.15]).

By the Littlewood–Richardson formula (compare with [8, Exercise 15.23]), one knows the decomposition of a tensor product of any two irreducible representations of $\mathfrak {sl}_n$, namely

$$\begin{aligned} \mathbb {S}_{\mu }(V) \otimes \mathbb {S}_{\nu }(V)=\bigoplus _{\lambda }c^{\lambda }_{\mu \nu }\mathbb {S}_{\lambda }(V). \end{aligned}$$

(2.1)

Here $\lambda$ is a partition of $|\mu |+|\nu |$ and the coefficient $c^{\lambda }_{\mu \nu }$ are given by the Littlewood–Richardson rule. The constant $c^{\lambda }_{\mu \nu }$ is the number of ways to obtain the partition $\lambda$ from the partition $\mu$ by “adding” the partition $\nu$ following the Littlewood–Richardson rule.

As our proofs for Theorem 1.6 rely heavily on this rule, we will briefly state it following the exposition of [13, §4]. Let $\mu =(\mu _1, \mu _2, \dots , \mu _n),\,\lambda =(\lambda _1, \lambda _2, \dots , \lambda _n)\in P_n$. One writes $\mu \subset \lambda$ if the Young diagram of $\mu$ sits inside the Young diagram of $\lambda$. In other words, $\mu _i\leqslant \lambda _i$ for all i. If $\mu \subset \lambda$, put the Young diagram of $\mu$ on the Young diagram of $\lambda$ with the same top-left corner and remove $\mu$ out of $\lambda$. That way, we obtain the skew diagram $\lambda -\mu$. Put a positive number in each box $\lambda -\mu$, then it becomes a skew tableau with the shape $\lambda -\mu$. If the entries of this skew tableau are taken from $\{1, 2, \dots , n\}$ and $\nu _j$ of them are j for each $j=1, 2, \dots , m$, then the content of this skew tableau becomes $\nu =(\nu _1, \dots , \nu _n)$. For a skew tableau T, we define the word of T by the sequence w(T) of positive integers obtained by reading the entries of T from top to bottom and right to left in each row.

For example,

(2.2)

is a skew tableau of shape $\lambda -\mu$, where $\mu =(3,2,1)$ and $\lambda =(6, 4,4,2)$ and the content is $\nu =(4,3,2,1)$. Its word is

$$\begin{aligned} w(T)=(1,1,1,2,1,3,2,2,4,3). \end{aligned}$$

Definition 2.1

A Littlewood–Richardson tableau is a skew tableau T with the following properties:

• (i) The numbers in each row of T weakly increase from left to right and the numbers in each column of T strictly increase from top to bottom.

• (ii) For each positive integer j, starting from the first entry of w(T) to any place in w(T), there are at least as many as j’s as $(j+1)$’s.

Then the Littlewood–Richardson coefficient $c^{\lambda }_{\mu \nu }$ is the number of the Littlewood–Richardson tableau of shape $\lambda -\mu$ and content $\nu$. Note that for conjugate pairs $\lambda , \lambda '$, $\mu , \mu '$ and $\nu , \nu '$, it is well-known that

$$\begin{aligned} c^{\lambda }_{\mu \nu }=c^{\lambda '}_{\mu '\nu '} \end{aligned}$$

(2.3)

(see [3] for example). The skew tableau T in (2.2) is indeed a Littlewood–Richardson tableau.

2.3 Classical groups $\mathrm {SO}(2n+1)$, $\mathrm {Sp}(2n)$ and $\mathrm {SO}(2n)$

In this section, we briefly go over the basic facts on orthogonal and symplectic groups which we will be using mainly for the purpose of fixing notations. We follow the expositions of [8] in part.

Let V be a complex vector space equipped with a nondegenerate symplectic or orthogonal form Q such that

$$\begin{aligned} \dim V=2n+\delta , \quad \delta \in \{0, 1\}. \end{aligned}$$

(2.4)

Denote by

$$\begin{aligned} \{e_1, e_2, \ldots , e_{2n+\delta }\} \end{aligned}$$

a basis such that

$$\begin{aligned} Q(e_i, e_{2n+1+\delta -i})=\pm Q(e_{2n+1+\delta -i}, e_i)=1 \end{aligned}$$

for all $i=1, 2, \dots , n+\delta$ and such that all other pairings are 0.

Let G be the subgroup of $\mathrm {SL}(V)$ which preserves this form. Then by using the Cartan-Killing classification, one knows that G is of type $B_n$ (resp. type $D_n$) if Q is orthogonal and $\dim V=2n+1$ (resp. $\dim V=2n$). In the former case we write $G=\mathrm {SO}(2n+1)$ and in the latter we write $G=\mathrm {SO}(2n)$. If Q is symplectic which forces $\dim V=2n$, we say that G is of type $C_n$ and write $G=\mathrm {Sp}(2n)$.

Given any partition $\lambda$ with at most $2n+\delta$ parts, one obtains an irreducible representation $\mathbb {S}_{\lambda }(V)$ of $\mathrm {SL}(V)$ of highest weight $\lambda$ inside of $V^{\otimes |\lambda |}$. Given the form Q and integers $1\leqslant i< j\leqslant |\lambda |$, one has a contraction map

$$\begin{aligned} V^{\otimes |\lambda |}\longrightarrow V^{\otimes (|\lambda |-2)} \end{aligned}$$

defined by

$$\begin{aligned} v_1\otimes v_2\otimes \cdots \otimes v_{|\lambda |}\mapsto Q(v_i, v_j)v_1\otimes \cdots \otimes \hat{v}_i\otimes \cdots \otimes \hat{v}_j\otimes \cdots \otimes v_{|\lambda |}, \end{aligned}$$

where the hat means we have removed those two vectors. Define $\mathbb {S}_{\parallel \lambda \parallel }(V)$ to be the intersection of $\mathbb {S}_\lambda (V)$ with the kernels of all possible contraction maps with $1 \leqslant i \le j \leqslant n$. In the standard notation,

$$\begin{aligned} \mathbb {S}_{\parallel \lambda \parallel }(V)={\left\{ \begin{array}{ll}\mathbb {S}_{[\lambda ]}(V) &{}\text{ if } \text{ G } \text{ is } \text{ of } \text{ type } B_n\text{, } D_n\\ \mathbb {S}_{\langle \lambda \rangle }(V)&{}\text{ if } \text{ G } \text{ is } \text{ oftype } C_n\text{. } \end{array}\right. } \end{aligned}$$

It is well-known that for type $B_n$ and $C_n$, $\mathbb {S}_{\parallel \lambda \parallel }(V)$ is the irreducible representation of G with highest weight $\lambda =(\lambda _1, \lambda _2, \dots , \lambda _{n-1}, \lambda _n)$. For type $D_n$, the same is true if $\lambda _n=0$. For type $D_n$ with $\lambda _n>0$, then $\mathbb {S}_{\parallel \lambda \parallel }(V)$ is the direct sum of two irreducible representations of G, one with highest weight $\lambda$ and the other with highest weight $\lambda ^{-}=(\lambda _1, \dots , \lambda _{n-1}, -\lambda _n)$. For the proofs, see [8, Theorem 17.11] for type $C_n$ and see [8, Theorem 19.22] for type $B_n$ and $D_n$, for example.

There is common structure constant $N^{\lambda }_{\mu \nu }$ [19, Corollary 2.5.3] such that

$$\begin{aligned} \mathbb {S}_{\parallel \mu \parallel }(V) \otimes \mathbb {S}_{\parallel \nu \parallel }(V)=\bigoplus _{\begin{array}{c} \lambda \\ \ell (\lambda )\leqslant n \end{array}}N^{\lambda }_{\mu \nu }\mathbb {S}_{\parallel \lambda \parallel }(V), \end{aligned}$$

(2.5)

where the coefficients $N^{\lambda }_{\mu \nu }$ are given by the Newell–Littlewood formula:

$$\begin{aligned} N^{\lambda }_{\mu \nu }=\sum _{\alpha , \beta , \gamma }c^{\mu }_{\alpha \beta }c^{\nu }_{\alpha \gamma }c^{\lambda }_{\beta \gamma }, \end{aligned}$$

(2.6)

where c’s denote the usual Littlewood–Richardson coefficients. The sum is over all partitions $\alpha$, $\beta$ and $\gamma$ (see [15] for example).

3 Connections to the Littlewood–Richardson semigroup

In this section, we discuss the connection between the detection by $\otimes ^3$ and the Littlewood–Richardson semigroup $\mathrm {LR}_n$ of order n. We begin this section by recalling the definition of $\mathrm {LR}_n$ (see [28] for instance).

For partitions $\lambda , \mu , \nu \in P_n$, the Littlewood–Richardson semigroup $\mathrm {LR}_n$ of order n is defined by

$$\begin{aligned} \mathrm {LR}_n=\{(\lambda , \mu , \nu ): c^{\lambda }_{\mu \nu }>0 \}. \end{aligned}$$

It is known that $\mathrm {LR}_n$ is finitely generated subsemigroup of the additive semigroup $P_n^3\subset \mathbb {Z}^{3n}$ of tuples of integers with positive entries [6].

We prove the following key proposition:

Proposition 3.1

The representation $\otimes ^3$ detects $\mathbb {S}_{\parallel \lambda \parallel }(G)$ if and only if $N^{\lambda }_{\lambda \lambda } > 0$, where $N^{\lambda }_{\lambda \lambda }$ is the common structure constant as in (2.6).

Proof

Let V be the standard representation of G. Then as described in Sect. 2, one obtains irreducible representations $\mathbb {S}_{\parallel \lambda \parallel }(V)$ of G with highest weight $\lambda$.

One has isomorphisms of G-modules

$$\begin{aligned} \mathbb {S}_{\parallel \lambda \parallel }(V)^{\otimes 3}\cong & {} \mathrm {Hom}(\mathbb {S}_{\parallel \lambda \parallel }(V)^{\vee }, \mathbb {S}_{\parallel \lambda \parallel }(V)\otimes \mathbb {S}_{\parallel \lambda \parallel }(V))\nonumber \\\cong & {} \bigoplus _{\begin{array}{c} \nu \\ \ell (\nu )\leqslant n \end{array}}N^{\nu }_{\lambda \lambda }~ \mathrm {Hom}(\mathbb {S}_{\parallel \lambda \parallel }(V)^{\vee },\mathbb {S}_{\parallel \nu \parallel }(V))\nonumber \\\cong & {} \bigoplus _{\begin{array}{c} \nu \\ \ell (\nu )\leqslant n \end{array}}N^{\nu }_{\lambda \lambda }~ \mathrm {Hom}(\mathbb {S}_{\parallel \lambda \parallel }(V), \mathbb {S}_{\parallel \nu \parallel }(V)), \end{aligned}$$

(3.1)

where we employ (2.6) and use the fact $\mathbb {S}_{\parallel \lambda \parallel }(V)$ is self-dual (see [25]). Therefore $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is detected by $\otimes ^3$ if and only if for some $\lambda$ and $\nu$ there is a line stabilized by G in

$$\begin{aligned} \mathrm {Hom}(\mathbb {S}_{\parallel \lambda \parallel }(V), \mathbb {S}_{\parallel \nu \parallel }(V)) \end{aligned}$$

(3.2)

and

$$\begin{aligned} N^{\nu }_{\lambda \lambda }\ne 0. \end{aligned}$$

Schur’s lemma implies (3.2) has a line fixed by G if and only if $\lambda =\nu$. On the other hand since G is semisimple $\mathbb {S}_{\parallel \nu \parallel }(G)$ stabilizes a line if and only if it fixes a line. The proposition follows. $\square$

By Proposition 3.1, the detection by $\otimes ^3$ is equal to nonvanishing of $N^{\lambda }_{\lambda \lambda }$. Hence the Newell–Littlewood formula of (2.6) implies the following corollary:

Theorem 3.2

The subgroup $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is detected by $\otimes ^3$ if and only if there are $\alpha , \beta , \gamma \in P_n$ such that all triples $(\lambda , \alpha , \beta )$, $(\lambda , \alpha , \gamma )$ and $(\lambda , \beta , \gamma )$ are elements in $\mathrm {LR}_n$. $\square$

This gives obvious constraints on the partitions on $\alpha$, $\beta$ and $\gamma$ in order for $\mathbb {S}_{\parallel \lambda \parallel }(G)$ to be detected by $\otimes ^3$. For example, one must have

$$\begin{aligned} |\alpha |=|\beta |=|\gamma |=\tfrac{1}{2}|\lambda |. \end{aligned}$$

For odd $|\lambda |$ this allows us to prove the following result:

Theorem 3.3

Let $\lambda$ be a partition of an odd number. Then the representation $\otimes ^3$ does not detect $\mathbb {S}_{\parallel \lambda \parallel }(G)$.

Proof

One has $c^{\lambda }_{\alpha \beta }=0$ unless $|\alpha |+|\beta |=|\lambda |$. Thus $N^{\lambda }_{\mu \nu }$ is zero unless $|\lambda |+|\mu |+|\nu |$ is even. Hence $N^{\lambda }_{\lambda \lambda }=0$ unless $|\lambda |$ is even. $\square$

Since the Littlewood–Richardson constant $c^{\lambda }_{\mu \nu }$ is the same as the conjugate Littlewood–Richardson constant $c^{\lambda '}_{\mu '\nu '}$ as in (2.3), we also obtain the following result:

Theorem 3.4

Let $\lambda \in P_n$ be such that its conjugate $\lambda '$ is also in $P_n$. Then $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is detected by $\otimes ^3$ if and only if $\mathbb {S}_{\parallel \lambda '\parallel }(G)$ is detected by $\otimes ^3$.

Proof

Let $\lambda \in P_n$ be a partition such that its conjugate $\lambda '$ is in $P_n$ as well. Suppose $\mathbb {S}_{\parallel \lambda \parallel }(G)$ is detected by $\otimes ^3$. Then $N^{\lambda }_{\lambda \lambda }>0$ which implies, by Theorem 3.2, there exist $\alpha , \beta , \gamma \in P_n$ so that $c^{\lambda }_{\alpha \beta }>0$, $c^{\lambda }_{\alpha \gamma }>0$ and $c^{\lambda }_{\beta \gamma }>0$.

On the other hand, by (2.3), one knows that $c^{\lambda '}_{\alpha '\beta '}=c^{\lambda }_{\alpha \beta }>0$, $c^{\lambda '}_{\alpha '\gamma '}=c^{\lambda }_{\alpha \gamma }>0$ and $c^{\lambda '}_{\beta '\gamma '}=c^{\lambda }_{\beta \gamma }>0$, which in turn implies $N^{\lambda '}_{\lambda '\lambda '}>0$ as well. The fact $\lambda ' \in P_n$ together with the definition of the Littlewood–Richardson coefficient forces $\alpha ', \beta ', \gamma '$ to be elements in $P_n$. This completes the proof. $\square$

4 Explicit constructions of subgroups detected by $\otimes ^3$

As mentioned briefly in the introduction, describing $\mathrm {LR}_n$ explicitly is not obvious at all in general (see [28]). In this section, we will explore the combinatrical contructions on $\lambda$ explicitly such that $N^{\lambda }_{\lambda \lambda }>0$. The constructions are purely based on the Littlewood–Richardson rule summarized in Sect. 2.2.

We restate Theorem 1.6 for the reader’s convenience:

Theorem 4.1

Let $\lambda =(\lambda _1, \lambda _2, \dots , \lambda _n) \in P_n$ be a partition of an even number. If

• (a) all $\lambda _i$ are even, or

• (b) it has even $\ell (\lambda )$ and all non-zero $\lambda _i$ are distinct and odd, or

• (c) $\lambda$ is a hook partition,

then the representation $\otimes ^3$ detects $\mathbb {S}_{\parallel \lambda \parallel }(G)$.

We will consider all cases separately in following subsections. Note that rectangular partition is a special case of (a).

4.1 All parts are even

In this section, we consider the first case of Theorem 4.1.

Proposition 4.2

Let $\lambda \in P_n$ be a partition such that all parts $\lambda _i$ are even. Then the representation $\otimes ^3$ detects $\mathbb {S}_{\parallel \lambda \parallel }(G)$.

Proof

Let $\lambda =(\lambda _1, \lambda _2, \dots , \lambda _n)$ be the partition such that all $\lambda _i$ are even. One needs to construct partitions $\alpha$, $\beta$ and $\gamma$ such that $c^{\lambda }_{\alpha \beta }>0$, $c^{\lambda }_{\alpha \gamma }>0$ and $c^{\lambda }_{\beta \gamma }>0$, thus in turn $N^{\lambda }_{\lambda \lambda }>0$. Let $\alpha$, $\beta$ and $\gamma$ be the partitions give by

$$\begin{aligned} \alpha =\beta =\gamma =\tfrac{1}{2}\lambda =\left( \tfrac{\lambda _1}{2}, \tfrac{\lambda _2}{2}, \dots , \tfrac{\lambda _n}{2}\right) . \end{aligned}$$

(4.1)

Note that the assumption that all $\lambda _i$ are even guarantees $\alpha , \beta , \gamma \in P_n$. Then it is not hard to obtain, for example, a skew tableau of shape $\lambda -\alpha$ with content $\beta$ in an obvious way: Put 1’s into $\tfrac{\lambda _1}{2}$ boxes in the first row of $\beta$ and add them to the right side of the $\tfrac{\lambda _1}{2}$ empty boxes in the first row of $\alpha$. This becomes then the first row of $\lambda$ with $\lambda _1$ boxes where the first half of them are empty and the remaining half of them have 1’s in them. Likewise, put 2’s into $\tfrac{\lambda _2}{2}$ boxes in the second row of $\beta$ and add them to the right side of the $\tfrac{\lambda _2}{2}$ empty boxes in the second row of $\alpha$. Again then this becomes the second row of $\lambda$ with $\lambda _2$ boxes where the first half of them are empty and the remaining half of boxes have 2’s in them. We repeat this process for all i. This gives us a skew tableau of shape $\lambda -\alpha$ with content $\beta$. This implies that $c^{\lambda }_{\alpha \beta }\geqslant 1$. Similarly, with the exact same method, one can show that $c^{\lambda }_{\alpha \gamma }$ and $c^{\lambda }_{\beta \gamma }$ are both at least 1. All together, we conclude that $N^{\lambda }_{\lambda \lambda }>0$, which completes the proof. $\square$

The following example shows the skew tableau of shape $\lambda -\alpha$ with content $\beta$:

Example 1

Let $\lambda =(6,4,4,2,2)$, $\alpha =(3,2,2,1,1)$, $\beta =(3, 2,2,1,1)$ be partitions as in the proof of Theorem 4.1. The following Young diagrams explain how to obtain the skew-tableau of shape $\lambda -\alpha$ with content $\beta$ in the proof of Proposition 4.2:

Then $w(T)=(1,1,1,2,2,3,3,4,5)$ and T is clearly a Littlewood–Richardson tableau.

4.2 Even $\ell (\lambda )$ and all parts are distinct and odd

In this section, we consider the partitions $\lambda$ such that $\ell (\lambda )$ is even and all parts are distinct and odd.

Proposition 4.3

Let $\lambda \in P_n$ be a partition such that $\ell (\lambda )$ even and all non-zero $\lambda _i$ are distinct and odd. Then the representation $\otimes ^3$ detects $\mathbb {S}_{\parallel \lambda \parallel }(G)$.

Proof

Let $\lambda$ be a partition such that $\ell (\lambda )=2k\leqslant n$ and all non-zero $\lambda _i$ are distinct and odd. As before, we need to construct partitions $\alpha$, $\beta$ and $\gamma$ such that $c^{\lambda }_{\alpha \beta }>0$, $c^{\lambda }_{\alpha \gamma }>0$ and $c^{\lambda }_{\beta \gamma }>0$. Let $\alpha$, $\beta$ and $\gamma$ be such that

$$\begin{aligned} \alpha =\gamma =\left( \tfrac{\lambda _1+1}{2}, \dots , \tfrac{\lambda _k+1}{2}, \tfrac{\lambda _{k+1}-1}{2},\dots , \tfrac{\lambda _{2k}-1}{2}\right) \end{aligned}$$

and

$$\begin{aligned} \beta =\left( \tfrac{\lambda _1-1}{2}, \dots , \tfrac{\lambda _k-1}{2}, \tfrac{\lambda _{k+1}+1}{2},\dots , \tfrac{\lambda _{2k}+1}{2}\right) . \end{aligned}$$

Then clearly $\alpha$ and $\gamma$ are partitions in $P_n$. We claim that $\beta$ is a partition in $P_n$ as well. Indeed, we assumed that $\lambda$ has distinct parts, so $\lambda _k>\lambda _{k+1}$ and hence one has

$$\begin{aligned} \tfrac{\lambda _k-1}{2}\geqslant \tfrac{\lambda _{k+1}+1}{2}. \end{aligned}$$

In fact this is the only place in the proof where we use the fact that $\lambda$ has distinct parts. Note that $|\alpha |=|\beta |=|\gamma |=\tfrac{|\lambda |}{2}$ and the last part $\tfrac{\lambda _{2k}-1}{2}$ of $\alpha$ and $\gamma$ could be zero if $\lambda _{2k}=1$.

Showing $c^{\lambda }_{\alpha \beta }\geqslant 1$ will be similar to the proof of Proposition 4.2: For $1\leqslant i\le k$, put i’s into $\tfrac{\lambda _i-1}{2}$ boxes in the ith row of $\beta$ and add them to the right side of the $\tfrac{\lambda _i+1}{2}$ empty boxes in the ith row of $\alpha$. This becomes then the ith row of $\lambda$ with $\lambda _i$ boxes where the first $\tfrac{\lambda _i+1}{2}$ boxes are empty and the remaining $\tfrac{\lambda _i-1}{2}$ boxes have i’s in them. For $k+1\leqslant i\le 2k$, put i’s into $\tfrac{\lambda _i+1}{2}$ boxes in the ith row of $\beta$ and add them to the right side of the $\tfrac{\lambda _i-1}{2}$ empty boxes in the ith row of $\alpha$. This becomes then the ith row of $\lambda$ with $\lambda _i$ boxes where the first $\tfrac{\lambda _i-1}{2}$ boxes are empty and the remaining $\tfrac{\lambda _i+1}{2}$ boxes have i’s in them. If $\lambda _{2k}>1$, then the resulting tableau is just the skew tableau of shape $\lambda -\alpha$ with content $\beta$. If $\lambda _{2k}=1$, then the last part $\tfrac{\lambda _{2k}-1}{2}$ of $\alpha$ becomes zero, so $\alpha$ will have only $2k-1$ nonzero parts. In this case, we add the last box of $\beta$ containing 2k to the bottom of the first column of $\alpha$ (compare with the Young diagrams in the first case of Example 2 below). This becomes then the last row of $\lambda$. This gives us the skew tableau of shape $\lambda -\alpha$ with content $\beta$. This implies that $c^{\lambda }_{\alpha \beta }\geqslant 1$ in either case.

Similarly, we obtain the skew-tableau of shape $\lambda -\beta$ with content $\gamma$: For $1\leqslant i\le k$, put i’s into $\tfrac{\lambda _i+1}{2}$ boxes in the ith row of $\gamma$ and add them to the right side of the $\tfrac{\lambda _i-1}{2}$ empty boxes in the ith row of $\beta$. This becomes then the ith row of $\lambda$ with $\lambda _i$ boxes where the first $\tfrac{\lambda _i-1}{2}$ boxes are empty and the remaining $\tfrac{\lambda _i+1}{2}$ boxes have i’s in them. For $k+1\leqslant i\le 2k$, put i’s into $\tfrac{\lambda _i-1}{2}$ boxes in the ith row of $\gamma$ and add them to the right side of the $\tfrac{\lambda _i+1}{2}$ empty boxes in the ith row of $\beta$. This becomes then the ith row of $\lambda$ with $\lambda _i$ boxes where the first $\tfrac{\lambda _i+1}{2}$ boxes are empty and the remaining $\tfrac{\lambda _i-1}{2}$ boxes have i’s in them. If the last part $\tfrac{\lambda _{2k}-1}{2}$ of $\gamma$ is zero, we don’t have any box to be added. In that case, the last row of $\beta$ will be just the last row of $\lambda$ (compare with the Young diagrams in the second case of Example 2 below). This implies that $c^{\lambda }_{\beta \gamma }\geqslant 1$.

Now, in order to prove that $c^{\lambda }_{\alpha \gamma }\geqslant 1$, we will need to modify the above process slightly: For each $1\leqslant j\le 2k$, put j’s into all boxes in the jth row of $\gamma$. To right of the first row of $\alpha$, add only $\tfrac{\lambda _1-1}{2}$ boxes with 1’s in them. This ensures $\lambda _1$ boxes in the first row of $\lambda$. The last box with 1 in it should be added to right to the second row of $\alpha$. After that, we add the boxes with 2 in them in the second row until we reach to $\lambda _2$ boxes all together. Whatever the remaining boxes with 2 should be added into the third row. We repeat this process until we add all the boxes of $\gamma$ with numbers in them (compare with the Young diagrams in the third case of Example 2 below). In this way, we obtain the skew tableau of shape $\lambda -\alpha$ with content $\gamma$. Therefore we prove that $c^{\lambda }_{\alpha \gamma }\geqslant 1$ which completes the proof. $\square$

The following example shows how to obtain skew tableau of shape $\lambda -\alpha$ with content $\beta$, skew tableau of shape $\lambda -\beta$ with content $\gamma$, and skew tableau of shape $\lambda -\alpha$ with content $\gamma$, respectively, by the process given in the proof of Proposition 4.3.

Example 2

Let $\lambda =(7,5,3,1)$, $\alpha =(4,3,1,0)$, $\beta =(3,2,2,1)$ and $\gamma =(4,3,1,0)$. The following Young diagrams explain how to obtain the skew-tableau of shape $\lambda -\alpha$ with content $\beta$ in the proof of Proposition 4.3:

Then $w(T_1)=(1,1,1,2,2,3,3,4)$ and $T_1$ is clearly a Littlewood–Richardson tableau.

For the same $\lambda$, $\alpha$, $\gamma$ as above, the following Young diagrams explain the method for obtaining the skew-tableau of shape $\lambda -\beta$ with content $\gamma$:

Then $w(T_2)=(1,1,1,1,2,2,2,3)$ and $T_2$ is again a Littlewood–Richardson tableau.

For the same $\lambda$, $\alpha$, $\gamma$ as above, the following Young diagrams explain the method for obtaining the skew-tableau of shape $\lambda -\alpha$ with content $\gamma$:

Then $w(T_3)=(1,1,1,2,1,2,2,3)$ and $T_3$ is once again a Littlewood–Richardson tableau.

4.3 Hook partitions

In this section, we consider partitions $\lambda$ with at most n parts whose shape is like a hook, namely

$$\begin{aligned} \lambda =(1+a, 1^b):=(1+a, \underbrace{1, \dots , 1}_{b \text { times }}), \quad b\in \{0, 1, 2, \dots , n-1\}. \end{aligned}$$

(4.2)

Here a is any nonnegative integer. We call such a partition $\lambda$ hook partition with the arm length a and the leg length b. Note that $|\lambda |=1+a+b$ and $\ell (\lambda )=b+1\leqslant n$. For example, the partition $\lambda =(1+3, 1^2)$ is a hook partition with arm length 3 and the leg length 2 and its Young diagram is

Clearly hook partitions are not fit to the cases considered in Propositions 4.2 and 4.3.

Proposition 4.4

Let $\lambda \in P_n$ be a hook partition of an even number. Then the representation $\otimes ^3$ detects $\mathbb {S}_{\parallel \lambda \parallel }(G)$.

Proof

Fix $b\in \{0, 1, 2, \dots , n-1\}$ and let $\lambda =(1+a, 1^b)$ be the partition such that $1+a+b$ is even. Note that for $\mathrm {SO}(2n)$, b ranges only up to $n-2$ due to the constraint on the $\lambda _n$ being zero.

Again, one needs to construct partitions $\alpha$, $\beta$ and $\gamma$ such that $c^{\lambda }_{\alpha \beta }>0$, $c^{\lambda }_{\alpha \gamma }>0$ and $c^{\lambda }_{\beta \gamma }>0$. Note that in order for $\lambda$ to be hook partition, all $\alpha$, $\beta$, $\gamma$ must be hook partitions as well. Since $|\lambda |=1+a+b$ is even, we have two cases to consider: one is where a is odd and b is even and the other is where a is even but b is odd.

Assume first that a is odd and b is even. Choose partitions $\alpha$, $\beta$ and $\gamma$ as

$$\begin{aligned} \alpha =\beta =\gamma =\left( 1+\tfrac{a-1}{2}, 1^{b/2}\right) . \end{aligned}$$

Then it is easy to obtain the skew tableau of shape $\lambda -\alpha$ with content $\beta$, for example: Put 1’s into the $(1+\tfrac{a-1}{2})$ boxes in the first row of the Young diagram $\beta$ and add them to right side of the $(1+\tfrac{a-1}{2})$ empty boxes in the first row of $\alpha$. Then this becomes the first row of $\lambda$ with $1+a$ boxes, where the half $\tfrac{1+a}{2}$ boxes are empty and the remaining half $\tfrac{1+a}{2}$ boxes have 1’s in them. Likewise, put each j, $2\leqslant j\le (\tfrac{b}{2}+1)$, into the each box in the leg of $\beta$. The the leg length $\tfrac{b}{2}$ of $\alpha$ can be extended to the leg length b by adding $\tfrac{b}{2}$ boxes in the leg of $\beta$, where each box contains each j’s. This gives the skew tableau with shape $\lambda -\alpha$ with content $\beta$. Therefore $c^{\lambda }_{\alpha \beta }\geqslant 1$. Similar arguments show that the same is true for $c^{\lambda }_{\alpha \gamma }$ and $c^{\lambda }_{\beta \gamma }$. Hence we complete the proof for the first case.

Next, assume that a is even and b is odd. Then the conjugate partition $\lambda '$ belongs to the case just mentioned. By Theorem 3.4, we obtain the desired result. $\square$

Now we provide an example to explain the proof above.

Example 3

Let $\lambda =(1+5, 1^4)$ and let $\alpha =\beta =\gamma =(1+2, 1^2).$ The Young diagrams below explain how to obtain the skew tableau of shape $\lambda -\alpha$ with content $\beta$ following the proof of Proposition 4.4:

Then $w(T)=(1,1,1,2,3)$ and T is clearly a Littlewood–Richardson tableau.