Homological branching law for $({\mathrm {GL}}_{n+1}(F), {\mathrm {GL}}_n(F))$: projectivity and indecomposability

Abstract

Let F be a non-Archimedean local field. This paper studies homological properties of irreducible smooth representations restricted from ${\mathrm {GL}}_{n+1}(F)$ to ${\mathrm {GL}}_n(F)$. A main result shows that each Bernstein component of an irreducible smooth representation of ${\mathrm {GL}}_{n+1}(F)$ restricted to ${\mathrm {GL}}_n(F)$ is indecomposable. We also classify all irreducible representations which are projective when restricting from ${\mathrm {GL}}_{n+1}(F)$ to ${\mathrm {GL}}_n(F)$. A main tool of our study is a notion of left and right derivatives, extending some previous work joint with Gordan Savin. As a by-product, we also determine the branching law in the opposite direction.

1 Introduction

1.1

Let F be a non-Archimedean local field. Let $G_n={\mathrm {GL}}_n(F)$. Let ${\mathrm {Alg}}(G_n)$ be the category of smooth representations of $G_n$. This paper is a sequel of [15] in studying homological properties of smooth representations of $G_{n+1}$ restricted to $G_n$, which is originally motivated from the study of D. Prasad in his ICM proceeding [28]. In [15], we show that for generic representations $\pi$ and $\pi '$ of $G_{n+1}$ and $G_n$ respectively, the higher Ext-groups

$$\begin{aligned} {\mathrm {Ext}}^i_{G_n}(\pi , \pi ')=0, \text{ for } i \ge 1, \end{aligned}$$

which was previously conjectured in [28]. This result gives a hope that there is an explicit homological branching law, generalizing the multiplicity one theorem [2, 31] and the local Gan–Gross–Prasad conjectures [17, 18].

The main techniques in [15] are utilizing Hecke algebra structure developed in [13, 14] and simultaneously applying left and right Bernstein–Zelevinsky derivatives, based on the classical approach of using Bernstein–Zelevinsky filtration on representations of $G_{n+1}$ restricted to $G_n$ [27, 28]. We shall extend these methods further, in combination of other things, to obtain new results in this paper.

In [15], we showed that an essentially square-integrable representation $\pi$ of $G_{n+1}$ is projective when restricted to $G_n$. However, those representations do not account for all irreducible representations whose restriction is projective. The first goal of the paper is to classify such representations:

Theorem 1.1

(=Theorem 3.7) Let $\pi$ be an irreducible smooth representation of $G_{n+1}$. Then $\pi |_{G_n}$ is projective if and only if

• (1) $\pi$ is essentially square-integrable, or

• (2) $n+1$ is even, and $\pi \cong \rho _1 \times \rho _2$ for some cuspidal representations $\rho _1, \rho _2$ of $G_{(n+1)/2}$.

There are also recent studies of the projectivity under restriction in other settings [1, 16, 22].

It is a well-known result that an irreducible representation of a reductive group G is projective (modulo center) if and only if it is supercuspidal, giving a tight connection between homological algebra and harmonic analysis. The classification theorem above for $(G_{n+1},G_n)$ suggests that one may still expect such connection in a relative setting. It would be interesting to see the interplay of our study with the harmonic analysis study in the relative Langlands program [30].

A main step in our classification is the following projectivity criteria:

Theorem 1.2

(=Theorem 3.3) Let $\pi$ be an irreducible smooth representation of $G_{n+1}$. Then $\pi |_{G_n}$ is projective if and only if the following two conditions hold:

• (1) $\pi$ is generic; and

• (2) ${\mathrm {Hom}}_{G_n}(\pi |_{G_n}, \omega ) =0$ for any irreducible non-generic representation $\omega$ of $G_n$.

Theorem 1.2 turns the projectivity problem into a Hom-branching problem and is a consequence of two results: (1) the Euler-Poincaré pairing formula of D. Prasad [28] and (2) the Hecke algebra argument used in [15] by G. Savin and the author. Roughly speaking, (1) is used to show non-projectivity while (2) is used to show projectivity.

The second part of the paper studies indecomposability of a restricted representation. It is clear that an irreducible representation (except one-dimensional ones) restricted from $G_{n+1}$ to $G_n$ cannot be indecomposable as it has more than one non-zero Bernstein components. However, the Hecke algebra realization in [13, 15] of the projective representations in Theorem 1.1 immediately implies that each Bernstein component of those restricted representation is indecomposable. This is a motivation of our study in general case, and precisely we prove:

Theorem 1.3

(=Theorem 6.1) Let $\pi$ be an irreducible representation of $G_{n+1}$. Then for each Bernstein component $\tau$ of $\pi |_{G_n}$, any two non-zero $G_n$-submodules of $\tau$ have non-zero intersection.

As a consequence, we have:

Corollary 1.4

Let $\pi$ be an irreducible smooth representation of $G_{n+1}$. Then any Bernstein component of $\pi |_{G_n}$ is indecomposable.

In Sect. 6.2, we explain how to determine which Bernstein component of $\pi |_{G_n}$ is non-zero in terms of Zelevinsky segments, and hence Theorem 1.3 essentially parametrizes the indecomposable components of $\pi |_{G_n}$.

For a mirabolic subgroup $M_n$ of $G_{n+1}$, it is known [35] that $\pi |_{M_n}$ is indecomposable for an irreducible representation $\pi$ of $G_{n+1}$. The approach in [35] uses the Bernstein–Zelevinsky filtration of $\pi$ to $M_n$ and that the bottom piece of the filtration is irreducible. We prove that the Bernstein component of a bottom piece is indecomposable as a $G_n$-module, and then make use of left and right derivatives, developed and used to prove main results in [15]. The key fact is that left and right derivatives of an irreducible representation are asymmetric. We now make more precise the meaning of ‘asymmetric’. We say that an integer i is the level of an irreducible representation $\pi$ of $G_n$ if the left derivative $\pi ^{(i)}$ is the highest derivative of $\pi$.

Theorem 1.5

(=Theorem 6.2) Let $\pi$ be an irreducible smooth representation of $G_{n}$. Let $\nu (g)=|{\mathrm {det}}(g)|_F$. Suppose i is not the level of $\pi$. Then $\nu ^{1/2}\cdot \pi ^{(i)}$ and $\nu ^{-1/2}\cdot {}^{(i)}\pi$ have no isomorphic irreducible quotients whenever $\nu ^{1/2}\cdot \pi ^{(i)}$ and $\nu ^{-1/2}\cdot {}^{(i)}\pi$ are non-zero. The statement also holds if one replaces quotients by submodules.

We remark that $\nu ^{1/2}\cdot \pi ^{(i)}$ and $\nu ^{-1/2}\cdot {}^{(i)}\pi$ are shifted derivatives in the sense of [4], which have been used in loc. cit. to study unitary representations of $G_n$.

As a by-product of Theorem 1.5, we give a complete answer to the branching law in another direction:

Theorem 1.6

(=Corollary 7.7) Let $\pi _1, \pi _2$ be smooth irreducible representations of $G_{n+1}$ and $G_n$ respectively. Then

$$\begin{aligned} {\mathrm {Hom}}_{G_n}(\pi _2, \pi _1|_{G_n}) \ne 0 \end{aligned}$$

if and only if both $\pi _1$ and $\pi _2$ are one-dimensional and $\pi _2 \cong \pi _1|_{G_n}$.

Computing the structure of a derivative of an arbitrary representation is a difficult question in general. Our approach is to approximate the information of derivatives of irreducible ones by some parabolically induced modules, whose derivatives can be computed via geometric lemma. On the other hand, the Speh representations behave more symmetrically for left and right derivatives, which motivates our proof to involve Speh representations.

Another key ingredient in proving Theorem 1.3 is a study on the submodule structure between left and right Bernstein–Zelevinsky inductions. We explain in Sect. 5 how the submodule structure of an induced module can be partly reflected from the module which induced from (see Proposition 5.6 for a precise statement). The study relies on the Hecke algebra structure of the Gelfand–Graev representation.

A related question to Theorem 1.3 is that the Bernstein–Zelevinsky induction functor preserves indecomposability at the level of each Bernstein component, which is shown in Theorem 8.3. This is in contrast with that the (usual) parabolic induction does not preserve indecomposability in general. In [12], we study some special situations of the parabolic induction that one can obtain certain indecomposability-preserving results, which have applications to the local nontempered Gan–Gross–Prasad conjectures [18].

1.1 Organization of the paper

Section 2 studies derivatives of generic representations, which simplifies some computations for Theorem 1.1. The results also give some guiding examples in the study of this paper and [15].

Section 3 firstly proves a criteria for an irreducible representation to be projective under restriction, and then apply this criteria to give a classification of such class of modules.

Section 4 discusses some results on Gelfand–Graev representations such as their indecomposability while Sect. 5 develops a theory of the submodule relation between left and right Bernstein–Zelevinsky inductions.

Section 6 proves a main result on the indecomposability of an irreducible representation under restriction, which uses results in Sect. 5 and the asymmetric property of derivatives proved in Sect. 7.

Section 8 proves that the Bernstein–Zelevinsky induction preserves indecomposability. This partly generalizes Sect. 5.

In the first appendix, we explain how an irreducible representation appears as the unique submodule of a product of some Speh representations. In the second appendix, we provide some preliminaries on module theory.

2 Bernstein–Zelevinsky derivatives of generic representations

2.1 Notations

Let $G_n={\mathrm {GL}}_n(F)$. All representations in this paper are smooth and over ${\mathbb {C}}$, and we usually omit the adjective ’smooth’. Let ${\mathrm {Irr}}(G_n)$ be the set of all irreducible representations of $G_n$ and let ${\mathrm {Irr}}=\sqcup _{n} {\mathrm {Irr}}(G_n)$. For an admissible representation $\omega \in {\mathrm {Alg}}(G_n)$, denote by ${\mathrm {JH}}(\omega )$ the set of (isomorphism classes of) irreducible composition factors of $\omega$. For $\pi \in {\mathrm {Alg}}(G_n)$, let $\pi ^{\vee }$ be the smooth dual of $\pi$.

Let $\rho$ be an (irreducible) cuspidal representation of $G_l$. Let $a, b\in {\mathbb {C}}$ with $b-a \in {\mathbb {Z}}_{\ge 0}$. We have a Zelevinsky segment $\Delta =[\nu ^a\rho , \nu ^b \rho ]$, which we may simply call a segment. Denote $a(\Delta )=\nu ^a\rho$ and $b(\Delta )=\nu ^b\rho$. The relative length of $\Delta$ is defined as $b-a+1$ and the absolute length of $\Delta$ is defined as $l(b-a+1)$. We can truncate $\Delta$ from each side to obtain two segments of absolute length $r(b-a)$:

$$\begin{aligned} {}^{-}\Delta = [\nu ^{a+1}\rho , \ldots , \nu ^{b}\rho ] \text { and } \Delta ^-= [ \nu ^{a}\rho , \ldots , \nu ^{b-1}\rho ]. \end{aligned}$$

Moreover, if we perform the truncation k-times, the resulting segments will be denoted by ${}^{(kl)}\Delta$ and $\Delta ^{(kl)}$. We remark that the convention here is different from the previous paper [15] for convenience later. If i is not an integer divisible by l, then we set $\Delta ^{(i)}$ and ${}^{(i)}\Delta$ to be empty sets. We also denote $\Delta ^{\vee }=[\nu ^{-b}\rho ^{\vee },\nu ^{-a}\rho ^{\vee }]$. For a singleton segment $[\rho , \rho ]$, we abbreviate as $[\rho ]$. For $\pi \in {\mathrm {Alg}}(G_l)$, define $n(\pi )=l$. For $c \in {\mathbb {C}}$ and a segment $\Delta =[\nu ^a\rho , \nu ^b \rho ]$, define $\nu ^c\Delta =[\nu ^{a+c}\rho , \nu ^{b+c}\rho ]$.

For a Zelevinsky segment $\Delta$, define $\langle \Delta \rangle$ and ${\mathrm {St}}(\Delta )$ to be the (unique) irreducible submodule and quotient of $\nu ^a\rho \times \ldots \times \nu ^b\rho$ respectively. We have

$$\begin{aligned} {\mathrm {St}}(\Delta )^{\vee } \cong {\mathrm {St}}(\Delta ^{\vee }) \quad \text{ and } \quad \langle \Delta \rangle ^{\vee } \cong \langle \Delta ^{\vee } \rangle . \end{aligned}$$

For two cuspidal representations $\rho _1, \rho _2$ of $G_m$, we say that $\rho _1$ precedes $\rho _2$, denoted by $\rho _1 < \rho _2$, if $\nu ^c\rho _1 \cong \rho _2$ for some $c \in {\mathbb {Z}}_{>0}$. We say two segments $\Delta$ and $\Delta '$ are linked if $\Delta \not \subset \Delta$, $\Delta \not \subset \Delta '$ and $\Delta \cup \Delta '$ is still a segment. We say that a segment $\Delta$ precedes $\Delta '$, denoted by $\Delta < \Delta '$, if $b(\Delta )$ precedes $b(\Delta ')$; and $\Delta$ and $\Delta '$ are linked. If $\Delta$ does not precede $\Delta '$, write $\Delta \not < \Delta '$.

A multisegment is a multiset of finite numbers of segments. Let ${\mathrm {Mult}}$ be the set of multisegments. Let ${\mathfrak {m}}=\left\{ \Delta _1,\ldots , \Delta _r \right\} \in {\mathrm {Mult}}$. We relabel the segments in ${\mathfrak {m}}$ such that for $i<j$, $\Delta _i$ does not precede $\Delta _j$. The modules defined below are independent of the labeling (up to isomorphisms) [35]. Define $\zeta ({\mathfrak {m}})=\langle \Delta _1 \rangle \times \ldots \times \langle \Delta _r \rangle$. Denote by $\langle {\mathfrak {m}} \rangle$ the unique irreducible submodule of $\zeta ({\mathfrak {m}})$.¹ Similarly, define $\lambda ({\mathfrak {m}})={\mathrm {St}}(\Delta _1)\times \ldots \times {\mathrm {St}}(\Delta _r)$. Denote by ${\mathrm {St}}({\mathfrak {m}})$ the unique quotient of $\lambda ({\mathfrak {m}})$. Both notions $\langle {\mathfrak {m}} \rangle$ and ${\mathrm {St}}({\mathfrak {m}})$ give a classification of irreducible smooth representations of $G_n$ ([35, Proposition 6.1], also see [24]) i.e. both the maps

$$\begin{aligned} {\mathfrak {m}} \mapsto \langle {\mathfrak {m}} \rangle , \quad \text{ and } \quad {\mathfrak {m}} \mapsto {\mathrm {St}}({\mathfrak {m}}) \end{aligned}$$

determine bijections from ${\mathrm {Mult}}$ to ${\mathrm {Irr}}$. For example, when ${\mathfrak {m}}=\left\{ [\nu ^{-1/2}], [\nu ^{1/2}] \right\}$, $\langle {\mathfrak {m}} \rangle$ is the Steinberg representation and ${\mathrm {St}}({\mathfrak {m}})$ is the trivial representation. The two notions $\langle {\mathfrak {m}} \rangle$ and ${\mathrm {St}}({\mathfrak {m}})$ are related by the so-called Aubert-Zelevinsky duality, and Mœglin-Waldspurger algorithm.

We shall use the following results several times (see [35, Theorems 1.9, 4.2 and 9.7]): if two segments $\Delta , \Delta '$ are not linked, then

$$\begin{aligned}&\langle \Delta \rangle \times \langle \Delta ' \rangle \cong \langle \Delta ' \rangle \times \langle \Delta \rangle , \end{aligned}$$

(2.1)

$$\begin{aligned}&{\mathrm {St}}(\Delta ) \times {\mathrm {St}}(\Delta ') \cong {\mathrm {St}}(\Delta ') \times {\mathrm {St}}(\Delta ) . \end{aligned}$$

(2.2)

Let $\pi \in {\mathrm {Irr}}(G_n)$. Then $\pi$ is a subquotient of $\rho _1\times \ldots \times \rho _r$ for some cuspidal representations $\rho _i$ of $G_{n_i}$. The multiset $(\rho _1, \ldots , \rho _r)$, denoted by ${\mathrm {csupp}}(\pi )$, is called the cuspidal support of $\pi$. We also set

$$\begin{aligned} {\mathrm {csupp}}_{{\mathbb {Z}}}(\pi )=\left\{ \nu ^{c} \rho : \rho \in {\mathrm {csupp}}(\pi ), c \in {\mathbb {Z}} \right\} , \end{aligned}$$

which is regarded as a set (rather than a multiset).

2.2 Derivatives and Bernstein–Zelevinsky inductions

Let $U_n$ (resp. $U_n^-$) be the group of unipotent upper (resp. lower) triangular matrices in $G_n$. For $i\le n$, let $P_i$ be the parabolic subgroup of $G_n$ containing the block diagonal matrices ${\mathrm {diag}}(g_1, g_2)$ ($g_1 \in G_i$, $g_2\in G_{n-i}$) and the upper triangular matrices. Let $P_i=M_iN_i$ with the Levi $M_i$ and the unipotent $N_i$. Let $N_i^-$ be the opposite unipotent subgroup of $N_i$. Let $\nu :G_n\rightarrow {\mathbb {C}}$ given by $\nu (g)=|{\mathrm {det}}(g)|_F$. Let

$$\begin{aligned} R_{n-i} =\left\{ \begin{pmatrix} g &{}\quad x\\ 0 &{}\quad u \end{pmatrix} \in G_n : g \in {\mathrm {GL}}_{n-i}(F), u \in U_i, x \in {\mathrm {Mat}}_{n-i,i}(F) \right\} . \end{aligned}$$

Let $R_{n-i}^-$ be the transpose of $R_{n-i}$.

We shall use ${\mathrm {Ind}}$ for normalized induction and ${\mathrm {ind}}$ for normalized induction with compact support. Let $\psi _i$ be a character on $U_i$ given by $\psi _i(u)={\overline{\psi }}(u_{1,2}+\ldots +u_{i-1,i})$, where ${\overline{\psi }}$ is a nondegenerate character on F and $u_{k,k+1}$ is the value in the $(k,k+1)$-entry of u. For $\tau \in {\mathrm {Irr}}(G_{n-i})$, we extend trivially the $G_{n-i}\times U_i$-representation $\tau \boxtimes \psi _i$ to a $R_{n-i}$-representation. This defines functors from ${\mathrm {Alg}}(G_{n-i})$ to ${\mathrm {Alg}}(G_n)$ given by

$$\begin{aligned} \pi \mapsto {\mathrm {Ind}}_{R_{n-i}}^{G_n}\pi \boxtimes \psi _i, \quad \text{ and } \pi \mapsto {\mathrm {ind}}_{R_{n-i}}^{G_n} \pi \boxtimes \psi _i , \end{aligned}$$

both of which will be called (right) Bernstein–Zelevinsky inductions. Similarly, one has left Bernstein–Zelevinsky inductions by using $R_{n-i}^-$ instead of $R_{n-i}$.

Let $\pi \in {\mathrm {Alg}}(G_n)$. Following [15], define $\pi ^{(i)}$ to be the left adjoint functor of ${\mathrm {Ind}}_{R_{n-i}}^{G_n} \pi \boxtimes \psi _i$. Let $\theta _n:G_n \rightarrow G_n$ given by $\theta _n(g)=g^{-T}$, the inverse transpose of g. Define the left derivative

$$\begin{aligned} {}^{(i)}\pi :=\theta _{n-i}(\theta _n(\pi )^{(i)}), \end{aligned}$$

(2.3)

which is left adjoint to ${\mathrm {Ind}}_{R_{n-i}^-}^{G_n}\pi \boxtimes \psi _i'$, where $\psi _i'(u)=\psi _i(u^T)$ for $u \in U_i^-$. The level of an admissible representation $\pi$ is the largest integer $i^*$ such that $\pi ^{(i^*)}\ne 0$ and $\pi ^{(j)}=0$ for all $j>i^*$. It follows from (2.3) that if $i^*$ is the level of $\pi$, then ${}^{(i^*)}\pi \ne 0$ and ${}^{(j)}\pi =0$ for all $j>i^*$. When $i^*$ is the level for $\pi$, we shall call $\pi ^{(i^*)}$ and ${}^{(i^*)}\pi$ to be the highest right and left derivatives of $\pi$ respectively, where we usually drop the term of left and right if no confusion.

We now define the shifted derivatives as follow: for any i,

$$\begin{aligned} \pi ^{[i]} = \nu ^{1/2} \cdot \pi ^{(i)} , \quad \text{ and } \quad {}^{[i]}\pi = \nu ^{-1/2} \cdot {}^{(i)}\pi . \end{aligned}$$

For the details of Bernstein–Zelevinsky filtrations, see [15].

The following result [7, Corollary 4.6] will be used several times, which is a consequence of the geometric lemma: for $\pi _k \in {\mathrm {Alg}}(G_{n_k})$ ($k=1,\ldots , r$), for any i,

$$\begin{aligned} (\pi _1 \times \ldots \times \pi _r)^{(i)} \end{aligned}$$

admits a filtration whose successive quotients isomorphic to

$$\begin{aligned} (\pi _1)^{(i_1)}\times \ldots \times (\pi _r)^{(i_1)}, \end{aligned}$$

(2.4)

where $i_1,\ldots , i_r$ run for all integers satisfying $i_1+\ldots +i_r=i$.

2.3 On computing derivatives

Let $\Pi _i={\mathrm {ind}}_{U_i}^{G_i}\psi _i$ be the Gelfand–Graev representation. Using inductions by stages, we have that

$$\begin{aligned} {\mathrm {ind}}_{R_{n-i}}^{G_n} \pi \boxtimes \psi _i \cong \pi \times {\mathrm {ind}}_{U_i}^{G_i} \psi _i =\pi \times \Pi _i. \end{aligned}$$

Let ${\dot{w}}_0 \in G_n$ whose antidiagonal entries are 1 and other entries are 0. By the left translation of ${\dot{w}}_0$ on ${\mathrm {ind}}_{R_{n-i}^-}^{G_n} \pi \boxtimes \psi _i$, we have that

$$\begin{aligned} {\mathrm {ind}}_{R_{n-i}^-}^{G_n} \pi \boxtimes \psi _i \cong \left( {\mathrm {ind}}_{U_i}^{G_i} \psi _i \right) \times \pi =\Pi _i \times \pi . \end{aligned}$$

Since the right derivative is left adjoint to ${\mathrm {Ind}}_{R_{n-i}}^{G_n}$, we consequently have:

$$\begin{aligned} \pi ^{(i)} \cong (\pi _{N_i})_{U_i, \psi _i} , \end{aligned}$$

where $U_i$ is regarded as the subgroup $G_{n-i}\times G_i$ via the embedding $g \mapsto {\mathrm {diag}}(1,g)$. We shall use the later expression when computing derivatives in Sect. 3.2. We also have an analogous expression for left derivatives.

We shall often use the following lemma:

Lemma 2.1

Let $\pi \in {\mathrm {Alg}}(G_{n+1})$ and let $\pi '$ be an admissible representation of $G_n$. Suppose there exists i such that the following conditions hold:

$$\begin{aligned} {\mathrm {Hom}}_{G_{n+1-i}}( \pi ^{[i]}, {}^{(i-1)}\pi ') \ne 0 ; \end{aligned}$$

and

$$\begin{aligned} {\mathrm {Ext}}^k_{G_{n+1-j}}( \pi ^{[j]}, {}^{(j-1)}\pi '))=0 \end{aligned}$$

for all $j=1,\ldots , i-1$ and all k. Then ${\mathrm {Hom}}_{G_n}(\pi ,\pi ')\ne 0$.

Proof

The Bernstein–Zelevinsky filtration of $\pi$ gives that there exists

$$\begin{aligned} 0 \subset \pi _n \subset \ldots \subset \pi _0 =\pi \end{aligned}$$

such that $\pi _{i-1}/\pi _{i} \cong {\mathrm {ind}}_{R_{n-i+1}}^{G_n} \pi ^{[i]} \boxtimes \psi _i$. Now, by the second adjointness property of derivatives [15, Lemma 2.2], we have

$$\begin{aligned} {\mathrm {Ext}}^k_{G_n}({\mathrm {ind}}_{R_{n-j+1}}^{G_{n}}\pi ^{[j]} \boxtimes \psi _{j-1} , \pi ') \cong {\mathrm {Ext}}^k_{G_{n-j+1}}( \pi ^{[j]}, {}^{(j-1)}\pi ') =0 \end{aligned}$$

for all k and j. Now a long exact sequence argument gives that

$$\begin{aligned} {\mathrm {dim}}~{\mathrm {Hom}}_{G_{n}}(\pi , \pi ')&\ge {\mathrm {dim}}~{\mathrm {Hom}}_{G_{n}}({\mathrm {ind}}_{R_{n+1-i}}^{G_{n}}\pi ^{[i]} \boxtimes \psi _{i-1} , \pi ' ) \end{aligned}$$

(2.5)

$$\begin{aligned}&= {\mathrm {dim}}~{\mathrm {Hom}}_{G_{n+1-i}}( \pi ^{[i]}, {}^{(i-1)}\pi ') \ne 0 . \end{aligned}$$

(2.6)

$\square$

2.4 Subrepresentation of a standard representation

Lemma 2.2

Let ${\mathfrak {m}} \in {\mathrm {Mult}}$. Suppose all segments in ${\mathfrak {m}}$ are singletons i.e. of relative length 1. Then $\lambda ({\mathfrak {m}})$ has unique irreducible submodule and quotient. Moreover, the unique irreducible submodule is generic.

Proof

By definitions, $\lambda ({\mathfrak {m}})=\zeta ({\mathfrak {m}})$ and hence has unique submodule and quotient. Since all segments in ${\mathfrak {m}}$ are singletons, the submodule is generic [35]. $\square$

It is known (see [21]²) that $\lambda ({\mathfrak {m}})$ always has a generic representation as the unique submodule. We shall prove a slightly stronger statement, using Zelevinsky theory:

Proposition 2.3

Let ${\mathfrak {m}} \in {\mathrm {Mult}}$. Then $\lambda ({\mathfrak {m}})$ can be embedded to $\lambda ({\mathfrak {m}}')$ for some ${\mathfrak {m}}' \in {\mathrm {Mult}}$ whose segments are singletons. In particular, $\lambda ({\mathfrak {m}})$ has a unique irreducible submodule and moreover, the submodule is generic.

Proof

Let $\rho$ be a cuspidal representation in ${\mathfrak {m}}$ such that for any cuspidal representation $\rho '$ in ${\mathfrak {m}}$, $\rho \not < \rho '$. Let $\Delta$ be a segment in ${\mathfrak {m}}$ with the shortest relative length among all segments $\Delta '$ in ${\mathfrak {m}}$ with $b(\Delta ') \cong \rho$.

By definition of $\lambda ({\mathfrak {m}})$, we have that

$$\begin{aligned} \lambda ({\mathfrak {m}}) \cong {\mathrm {St}}(\Delta ) \times \lambda ({\mathfrak {m}}{\setminus } \left\{ \Delta \right\} ) . \end{aligned}$$

On the other hand, we have that

$$\begin{aligned} {\mathrm {St}}(\Delta ) \hookrightarrow b(\Delta ) \times {\mathrm {St}}(\Delta ^-) . \end{aligned}$$

Thus we have that

$$\begin{aligned} \lambda ({\mathfrak {m}})&\hookrightarrow b(\Delta ) \times {\mathrm {St}}(\Delta ^-)\times \lambda ({\mathfrak {m}}{\setminus } \left\{ \Delta \right\} ) \end{aligned}$$

(2.7)

$$\begin{aligned}&\cong b(\Delta ) \times \lambda ({\mathfrak {m}}{\setminus } \left\{ \Delta \right\} +\Delta ^-) \end{aligned}$$

(2.8)

We now explain the last isomorphism (2.8), and it suffices to show

$$\begin{aligned} \lambda ({\mathfrak {m}} {\setminus } \left\{ \Delta \right\} +\Delta ^-) \cong {\mathrm {St}}(\Delta ^-) \times \lambda ({\mathfrak {m}} {\setminus } \left\{ \Delta \right\} ) . \end{aligned}$$

To this end, we write ${\mathfrak {m}}=\left\{ \Delta , \Delta _1, \ldots , \Delta _k, \Delta _{k+1}, \ldots , \Delta _s \right\}$ such that

$$\begin{aligned} b(\Delta ) \cong b(\Delta _1) \cong \ldots \cong b(\Delta _k) \end{aligned}$$

and $b(\Delta _j) \not \cong b(\Delta )$ for $j \ge k+1$. Then we have that

$$\begin{aligned} {\mathrm {St}}(\Delta ^-)\times \lambda ({\mathfrak {m}} {\setminus } \left\{ \Delta \right\} ) \cong&{\mathrm {St}}(\Delta ^-)\times {\mathrm {St}}(\Delta _1)\times \ldots \times {\mathrm {St}}(\Delta _s) \\ \cong&{\mathrm {St}}(\Delta _1)\times \ldots \times {\mathrm {St}}(\Delta _k)\times {\mathrm {St}}(\Delta ^-)\\&\times {\mathrm {St}}(\Delta _{k+1})\times \ldots \times {\mathrm {St}}(\Delta _s) \\ \cong&\lambda ({\mathfrak {m}}{\setminus } \left\{ \Delta \right\} +\Delta ^-) \end{aligned}$$

where the second isomorphism follows from applying (2.2) few times. Here we need to use our choice of $\Delta$, which guarantees that $\Delta ^-$ and $\Delta _j$ are unlinked for $j=1, \ldots , k$.

Now $\lambda ({\mathfrak {m}}{\setminus } \left\{ \Delta \right\} +\Delta ^-)$ embeds to $\lambda ({\mathfrak {m}}')$ by induction for some ${\mathfrak {m}}'\in {\mathrm {Mult}}$ with all segments to be singletons. Thus this gives that $b(\Delta ) \times \lambda ({\mathfrak {m}}{\setminus } \left\{ \Delta \right\} +\Delta ^-)$ embeds to $b(\Delta )\times \lambda ({\mathfrak {m}}') \cong \lambda ({\mathfrak {m}}'+b(\Delta ))$, and so does $\lambda ({\mathfrak {m}})$ by (2.7).

The second assertion follows from Lemma 2.2. $\square$

2.5 Derivatives of generic representations

Recall that the socle (resp. cosocle) of an admissible representation $\pi$ of $G_n$ is the maximal semisimple submodule (resp. quotient) of $\pi$.

Lemma 2.4

Let $\pi \in {\mathrm {Irr}}(G_n)$. Then the cosocle of $\pi ^{(i)}$ (resp. ${}^{(i)}\pi$) is isomorphic to the socle of $\pi ^{(i)}$ (resp. ${}^{(i)}\pi$).

Proof

It follows from that for an irreducible $G_n$-representation $\pi$,

$$\begin{aligned} {}^{(i)}\pi \cong \theta _{n-i}(\theta _n(\pi )^{(i)}) \cong \theta _{n-i} ((\pi ^{\vee })^{(i)})\cong \theta _{n-i}(({}^{(i)}\pi )^{\vee } ) \end{aligned}$$

and the fact [6] that $\theta _{n-i}(\tau )\cong \tau ^{\vee }$ for any irreducible $G_{n-i}$-representation $\tau$. The last isomorphism follows from [15, Lemma 2.2]. $\square$

Proposition 2.5

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. Both socle and cosocle of $\pi ^{(i)}$ (and ${}^{(i)}\pi$) are multiplicity-free.

Proof

Let $\pi _0$ be an irreducible quotient of $\pi ^{(i)}$. Let $\pi _1$ be a cuspidal representation of $G_{i-1}$ which is not in ${\mathrm {cupp}}_{{\mathbb {Z}}}(\pi _0)$. Then, by comparing cuspidal supports,

$$\begin{aligned} {\mathrm {Ext}}^j_{G_{n+1-k}}( \pi ^{[k]}, {}^{(k-1)}(\pi _0 \times \pi _1))=0 \end{aligned}$$

for all j and $k<i$. With a long exact sequence argument using Bernstein–Zelevinsky filtration (similar to the proof of Lemma 2.1), we have that

$$\begin{aligned} \dim ~ {\mathrm {Hom}}_{G_n}(\pi , \pi _0\times \pi _1)&\ge {\mathrm {dim}}~ {\mathrm {Hom}}_{G_{n+1-i}}(\pi ^{[i]},{}^{(i-1)}(\pi _0\times \pi _1)) \end{aligned}$$

Now one applies the geometric lemma to obtain a filtration on ${}^{(i-1)}(\pi _0\times \pi _1)$, and then by comparing cuspidal supports, the only possible layer that can contribute the above non-zero Hom is $\pi _0$. Hence,

$$\begin{aligned} \dim ~ {\mathrm {Hom}}_{G_n}(\pi , \pi _0\times \pi _1)&\ge {\mathrm {dim}}~{\mathrm {Hom}}_{G_{n+1-i}}( \pi ^{[i]}, \pi _0). \end{aligned}$$

The first dimension is at most one by [2] and so is the second dimension. This implies the cosocle statement by Lemma 2.1 and the socle statement follows from Lemma 2.4. $\square$

A representation $\pi$ of $G_n$ is called generic if $\pi ^{(n)}\ne 0$. The Zelevinsky classification of irreducible generic representations is in [35], that is, ${\mathrm {St}}({\mathfrak {m}})$ is generic if and only if any two segments in ${\mathfrak {m}}$ are unlinked. With Proposition 2.5, the following result essentially gives a combinatorial description on the socle and cosocle of the derivatives of a generic representation.

Corollary 2.6

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$ be generic. Then any simple quotient and submodule of $\pi ^{(i)}$ (resp. ${}^{(i)}\pi$) is generic.

Proof

By Lemma 2.4, it suffices to prove the statement for quotient. Let ${\mathfrak {m}}=\left\{ \Delta _1, \ldots , \Delta _r \right\}$ be the Zelevinsky segment ${\mathfrak {m}}$ such that

$$\begin{aligned} \pi \cong {\mathrm {St}}({\mathfrak {m}})= \lambda ({\mathfrak {m}})={\mathrm {St}}(\Delta _1)\times \ldots \times {\mathrm {St}}(\Delta _r) . \end{aligned}$$

Since any two segments in ${\mathfrak {m}}$ are unlinked, we can label in any order and so we shall assume that for $i<j$, $b(\Delta _j)$ does not precede $b(\Delta _i)$. Then geometric lemma (2.4) produces a filtration on $\pi ^{(i)}$ whose successive subquotient is isomorphic to

$$\begin{aligned} {\mathrm {St}}({}^{(i_1)}\Delta _1)\times \ldots \times {\mathrm {St}}({}^{(i_r)}\Delta _r) , \end{aligned}$$

where $i_1+\ldots +i_r=i$. The last module is isomorphic to $\lambda ({\mathfrak {m}}')^{\vee }$, where

$$\begin{aligned} {\mathfrak {m}}'=\left\{ ({}^{(i_1)}\Delta _1)^{\vee }, \ldots , ({}^{(i_r)}\Delta _r)^{\vee } \right\} . \end{aligned}$$

If $\pi '$ is a simple quotient of $\pi ^{(i)}$, then $\pi '$ is a simple quotient of one successive subquotient in the filtration, or in other words, $\pi '$ is a simple submodule of $\lambda ({\mathfrak {m}}')$ for a multisegment ${\mathfrak {m}}'$. Now the result follows from Proposition 2.3. $\square$

Remark 2.7

One can formulate the corresponding statement of Proposition 2.5 for affine Hecke algebra level by using the sign projective module in [13]. Then it might be interesting to ask for an analogue result for affine Hecke algebras over fields of positive characteristics.

Here we give a consequence to branching law (c.f. [12, 18, 19]):

Corollary 2.8

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$ be generic. Let $\pi ' \in {\mathrm {Irr}}(G_n)$ and let ${\mathfrak {m}} \in {\mathrm {Mult}}$ with $\pi '\cong \langle {\mathfrak {m}}\rangle$. If ${\mathrm {Hom}}_{G_n}(\pi ,\langle {\mathfrak {m}} \rangle ) \ne 0$, then each segment in ${\mathfrak {m}}$ has relative length at most 2.

Proof

Write ${\mathfrak {m}}=\left\{ \Delta _1,\ldots , \Delta _r\right\}$ such that $\Delta _i$ does not precede $\Delta _j$ if $i <j$. Let $\pi _0=\langle {\mathfrak {m}} \rangle$. By using the Bernstein–Zelevinsky filtration, ${\mathrm {Hom}}_{G_n}(\pi , \pi _0 ) \ne 0$ implies that

$$\begin{aligned} {\mathrm {Hom}}_{G_{n+1-i}}( \pi _1^{[i]}, {}^{(i-1)}\pi _0) \ne 0 \end{aligned}$$

for some $i \ge 1$ [15] (c.f. Lemma 2.1). Hence Corollary 2.6 implies that ${}^{(i-1)}\pi _0$ is generic for some $i \ge 1$. Since

$$\begin{aligned} \pi _0 \hookrightarrow \zeta ({\mathfrak {m}}) \end{aligned}$$

we have ${}^{(i-1)}\pi _0 \hookrightarrow {}^{(i-1)}\zeta ({\mathfrak {m}})$ and so ${}^{(i-1)}\zeta ({\mathfrak {m}})$ has a generic composition factor. On the other hand, geometric lemma gives that ${}^{(i-1)}\pi _0$ admits a filtration whose successive quotients are isomorphic to ${}^{(i_1)}\langle \Delta _1 \rangle \times \ldots \times {}^{(i_r)} \langle \Delta _r \rangle$ where $i_1, \ldots , i_r$ run for all sums equal to $i-1$. Then at least one such quotient is non-degenerate and so in that quotient, all ${}^{(i_k)}\langle \Delta _k \rangle$ are cuspidal. Following from the derivatives on $\langle \Delta \rangle$, $\Delta$ can have at most of relative length 2. $\square$

Another consequence is on the indecomposability of derivatives of generic representations.

Corollary 2.9

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$ be generic. Then the projections of $\pi ^{(i)}$ and ${}^{(i)}\pi$ to any cuspidal support component have unique simple quotient and submodule. In particular, the projections of $\pi ^{(i)}$ and ${}^{(i)} \pi$ to any cuspidal support component are indecomposable.

Proof

For a fixed cuspidal support, there is a unique (up to isomorphism) irreducible smooth generic representation. Now the result follows from Proposition 2.5 and Corollary 2.6. $\square$

The analogous statement is not true in general if one replaces an irreducible generic representation by an arbitrary irreducible representation. We give an example below.

Example 2.10

Let ${\mathfrak {m}}=\left\{ [1,\nu ],[\nu ^{-1},1], [1], [\nu ^{-1},\nu ] \right\}$. We note that

$$\begin{aligned} \langle {\mathfrak {m}} \rangle =\langle [1,\nu ], [\nu ^{-1},1]\rangle \times \langle [1] \rangle \times \langle [\nu ^{-1},\nu ]\rangle . \end{aligned}$$

We compute the components of $\langle {\mathfrak {m}} \rangle ^{(2)}$, at the cuspidal support $\left\{ \nu ^{-1},\nu ^{-1}, 1,1, 1, \nu \right\}$. In this case, there are four composition factors whose Zelevinsky multisegments are:

$$\begin{aligned} {\mathfrak {m}}_1= & {} \left\{ [1], [\nu ^{-1}] , [1], [\nu ^{-1}, \nu ] \right\} , \quad {\mathfrak {m}}_2 =\left\{ [1,\nu ], [\nu ^{-1}], [1], [\nu ^{-1},1] \right\} \\ {\mathfrak {m}}_3= & {} \left\{ [1,\nu ], [\nu ^{-1},1], [\nu ^{-1},1] \right\} \text{(multiplicity } \text{2) } \end{aligned}$$

Note that the socle and cosocle coincide by Lemma 2.4. Note $\langle {\mathfrak {m}}_1\rangle$ and $\langle {\mathfrak {m}}_3 \rangle$ are in socle (and so cosocle). $\langle {\mathfrak {m}}_2\rangle$ cannot be a submodule or quotient. Thus the only possible structure is two indecomposable modules. One of them has composition factors of $\langle {\mathfrak {m}}_3 \rangle$ (with multiplicity 2) and $\langle {\mathfrak {m}}_2 \rangle$. Another one is a simple module isomorphic to $\langle {\mathfrak {m}}_1 \rangle$.

3 Projectivity

3.1 Projectivity criteria

We need the following formula of D. Prasad:

Theorem 3.1

[28] Let $\pi _1$ and $\pi _2$ be admissible representations of ${\mathrm {GL}}_{n+1}(F)$ and ${\mathrm {GL}}_n(F)$ respectively. Then

$$\begin{aligned} \sum _{i \in {\mathbb {Z}}} (-1)^i{\mathrm {dim}}~ {\mathrm {Ext}}^i_{G_n}(\pi _1, \pi _2) = {\mathrm {dim}}~ {\mathrm {Wh}}(\pi _1) \cdot {\mathrm {dim}}~{\mathrm {Wh}}(\pi _2) , \end{aligned}$$

where ${\mathrm {Wh}}(\pi _1)=\pi _1^{(n+1)}$ and ${\mathrm {Wh}}(\pi _2)= \pi _2^{(n)}$.

Lemma 3.2

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. If $\pi ^{(i)}$ has a non-generic irreducible submodule or quotient for some i, then there exists a non-generic representation $\pi '$ of $G_n$ such that ${\mathrm {Hom}}_{G_n}(\pi , \pi ')\ne 0$. The statement still holds if we replace $\pi ^{(i)}$ by ${}^{(i)}\pi$.

Proof

By Lemma 2.4, it suffices just to consider that $\pi ^{(i)}$ has a non-generic irreducible quotient, say $\lambda$. Now let

$$\begin{aligned} \pi '=(\nu ^{1/2}\lambda ) \times \tau , \end{aligned}$$

where $\tau$ is a cuspidal representation such that $\tau$ is not an unramified twist of a cuspidal representation appearing in any segment in ${\mathfrak {m}}$. Here ${\mathfrak {m}}$ is a multisegment with $\pi \cong \langle {\mathfrak {m}} \rangle$. Now

$$\begin{aligned} {\mathrm {Ext}}^k_{G_{n+1-j}}(\pi ^{[j]}, {}^{(j-1)}\pi ')=0 \end{aligned}$$

for $j< n(\tau )$ and any k since $\tau$ is in ${\mathrm {csupp}}({}^{(j-1)}\pi ')$ whenever it is nonzero while $\tau \not \in {\mathrm {csupp}}( \pi ^{[j]})$. Moreover, ${}^{(n(\tau )-1)}\pi '$ has a simple quotient isomorphic to $\nu ^{1/2}\lambda$. This checks the Hom and Ext conditions in Lemma 2.1 and hence proves the lemma. The proof for ${}^{(i)}\pi$ is almost identical with switching left and right derivatives in suitable places. $\square$

Theorem 3.3

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. Then the following conditions are equivalent:

• (1) $\pi |_{G_n}$ is projective

• (2) $\pi$ is generic and any irreducible quotient of $\pi |_{G_n}$ is generic.

Proof

For (2) implying (1), it is proved in [15]. We now consider $\pi$ is projective. All higher Exts vanish and so ${\mathrm {EP}}(\pi , \pi ')={\mathrm {dim}}~{\mathrm {Hom}}_{G_n}(\pi ,\pi ')$ for any irreducible $\pi '$ of $G_n$. If $\pi '$ is an irreducible quotient of $\pi$, then ${\mathrm {EP}}(\pi ,\pi ')\ne 0$ and hence $\pi$ is generic by Theorem 3.1. But Theorem 3.1 also implies $\pi '$ is generic. This proves (1) implying (2). $\square$

3.2 Classification

Definition 3.4

We say that an irreducible representation $\pi$ of $G_{n+1}$ is relatively projective if either one of the following conditions holds:

• (i) $\pi$ is essentially square-integrable;

• (ii) $\pi$ is isomorphic to $\pi _1 \times \pi _2$ for some irreducible cuspidal representations $\pi _1, \pi _2$ of $G_{(n+1)/2}$ with $\pi _1 \not \cong \nu ^{\pm 1}\pi _2$.

In particular, a relatively-projective representation is generic. The condition $\pi _1 \not \cong \nu ^{\pm 1}\pi _2$ is in fact automatic from $\pi$ being irreducible.

We can formulate the conditions (i) and (ii) combinatorially as follows. Let $\pi \cong {\mathrm {St}}({\mathfrak {m}})$ for a multisegment ${\mathfrak {m}}=\left\{ \Delta _1, \ldots , \Delta _r\right\}$. Then (i) is equivalent to $r=1$; and (ii) is equivalent to that $r=2$, and $\Delta _1$ and $\Delta _2$ are not linked, and the relative lengths of $\Delta _1$ and $\Delta _2$ are both 1.

Lemma 3.5

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$ be not relatively projective. Then there exists an irreducible non-generic representation $\pi '$ of $G_{n}$ such that ${\mathrm {Hom}}_{G_n}(\pi , \pi ') \ne 0$.

Proof

It suffices to construct an irreducible non-generic representation $\pi '$ satisfying the Hom and Ext properties in Lemma 2.1.

Let ${\mathfrak {m}}=\left\{ \Delta _1,\ldots , \Delta _r \right\}$ be a multisegment such that $\pi \cong {\mathrm {St}}({\mathfrak {m}})$. We divide into few cases.

Case 1 $r\ge 3$; or when $r=2$, each segment has relative length at least 2; or when $r=2$, $\Delta _1 \cap \Delta _2=\emptyset$. We choose a segment $\Delta '$ in ${\mathfrak {m}}$ with the shortest absolute length. Now we choose a maximal segment $\Delta$ in ${\mathfrak {m}}$ with the property that $\Delta ' \subset \Delta$. By genericity, $\nu ^{-1}a(\Delta ) \notin \Delta _k$ for any $\Delta _k\in {\mathfrak {m}}$. Let

$$\begin{aligned} {\mathfrak {m}}'=\left\{ \nu ^{1/2}\Delta , [\nu ^{-1/2}a(\Delta )], [\tau ] \right\} , \end{aligned}$$

where $\tau$ is a cuspidal representation so that ${\mathrm {St}}({\mathfrak {m}}')$ is a representation of $G_n$ and $\tau$ is not an unramified twist of any cuspidal representation appearing in a segment of ${\mathfrak {m}}$. To make sense of the construction, it needs the choices and the assumptions on this case. Let $k=n(a(\Delta )), l=n(\tau )$. Let

$$\begin{aligned} \pi '={\mathrm {St}}({\mathfrak {m}}') . \end{aligned}$$

Now as for $i <k+l+1$, either $\nu ^{-1/2}a(\Delta )$ or $\tau$ appears in the cuspidal support of ${}^{(i-1)}\pi$, but not in that of $\nu ^{1/2}\pi ^{(i)}$,

$$\begin{aligned} {\mathrm {Ext}}^j_{G_{n+1-i}}(\nu ^{1/2}\cdot \pi ^{(i)}, {}^{(i-1)}\pi ') = 0 \end{aligned}$$

and all j. By Corollary 2.6, $\nu ^{1/2}\cdot \pi ^{(k+l+1)}$ has a simple generic quotient isomorphic to $\nu ^{1/2}{\mathrm {St}}({}^-\Delta )$. On the other hand, a submodule structure of ${}^{(k+l)}\pi '$ can be computed as follows:

$$\begin{aligned} 0 \ne&{\mathrm {Hom}}_{G_n}(\lambda ({\mathfrak {m}}'), {\mathrm {St}}({\mathfrak {m}}')) \end{aligned}$$

(3.9)

$$\begin{aligned} \cong&{\mathrm {Hom}}_{G_{n-k-l} \times G_{k+l}}( {\mathrm {St}}(\nu ^{1/2}\cdot {}^-\Delta ) \boxtimes (\nu ^{-1/2}a(\Delta ) \times \tau ), {\mathrm {St}}({\mathfrak {m}}')_{N_{k+l}^-}) , \end{aligned}$$

(3.10)

Here the non-zeroness comes from the fact that ${\mathrm {St}}({\mathfrak {m}}')$ is the unique quotient of $\lambda ({\mathfrak {m}}')$, and the isomorphism follows from Frobenius reciprocity. Since taking the derivative is an exact functor, we have that ${\mathrm {St}}(\nu ^{1/2}\cdot {}^-\Delta )$ is a subrepresentation of ${}^{(k+l)}{\mathrm {St}}({\mathfrak {m}}')$ (see Sect. 2.3). Thus we have

$$\begin{aligned} {\mathrm {Hom}}_{G_{n-k-l}}( \pi ^{[k+l+1]}, {}^{(k+l)}\pi ') \ne 0 . \end{aligned}$$

Case 2 $r=2$ with $\Delta _1 \cap \Delta _2 \ne \emptyset$ and one segment having relative length 1 (and not both having relative length 1 by the definition of relatively-projective type). By switching the labeling on segments if necessary, we assume that $\Delta _1 \subset \Delta _2$. Let p and let l be the absolute and relative length of $\Delta _2$ respectively. Let

$$\begin{aligned} {\mathfrak {m}}'=\left\{ [\nu ^{1/2}a(\Delta _1)], [\nu ^{3/2-l}a(\Delta _1), \nu ^{-1/2}a(\Delta _1)], [\tau ] \right\} , \pi '={\mathrm {St}}({\mathfrak {m}}') , \end{aligned}$$

where $\tau$ is a cuspidal representation of $G_{k}$ (here k is possibly zero) so that ${\mathrm {St}}({\mathfrak {m}}')$ is a $G_n$-representation. Note that $\pi '$ is non-generic. By Corollary 2.6 and geometric lemma, a simple quotient $\nu ^{1/2}\cdot \pi ^{(p)}$ is isomorphic to $\nu ^{1/2}a(\Delta _1)$. Similar computation as in (3.9) gives that a simple module of ${}^{(p-1)}\pi '$ is isomorphic to $\nu ^{1/2}a(\Delta _1)$. This implies the non-vanishing ${\mathrm {Hom}}$ between those two $G_{n+1-p}$-representations.

We now prove the vanishing ${\mathrm {Ext}}$-groups in order to apply Lemma 2.1. Now applying the Bernstein–Zelevinsky derivatives $(j=1,\ldots , p-1)$, unless $a(\Delta _1)\cong b(\Delta _2)$, we have that $\nu ^{3/2-l}a(\Delta _1)$ is a cuspidal support for ${}^{(j-1)}\pi '$ whenever ${}^{(j-1)}\pi '$ is nonzero and is not a cuspidal support for $\nu ^{1/2}\pi ^{(j)}$. It remains to consider $a(\Delta _1)\cong b(\Delta _2)$. We can similarly consider the cuspidal support for $\nu ^{-1/2}a(\Delta _1)$ and $\nu ^{1/2}a(\Delta _1)$ to make conclusion. $\square$

Lemma 3.6

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. If $\pi$ is (generic) relatively projective, then $\pi |_{G_n}$ is projective.

Proof

When $\pi$ is essentially square-integrable, it is proved in [15]. We now assume that $\pi$ is in the case (2) of Definition 3.4. It is equivalent to prove the condition (2) in Theorem 3.3. Let $\pi ' \in {\mathrm {Irr}}(G_n)$ with ${\mathrm {Hom}}_{G_n}(\pi ,\pi ')\ne 0$. We have to show that $\pi '$ is generic. Note that the only non-zero derivative of $\pi ^{(i)}$ can occur when $i=n+1$ and $\frac{n+1}{2}$.

Case 1 ${\mathrm {Hom}}_{G_{(n+1)/2}}(\pi ^{[(n+1)/2]}, {}^{((n-1)/2)}\pi ') \ne 0$. For (1), let

$$\begin{aligned} \pi = \rho _1 \times \rho _2 \end{aligned}$$

for some cuspidal representations $\rho _1, \rho _2$ of $G_{(n+1)/2}$ with $\rho _1 \not \cong \nu ^{\pm 1}\rho _2$, and

$$\begin{aligned} {\mathfrak {m}}'=\left\{ \Delta _1',\ldots , \Delta _s' \right\} \quad \text{ for } \pi '\cong {\mathrm {St}}({\mathfrak {m}}') . \end{aligned}$$

By a simple count on dimensions, we must have $\Delta _k' \cong \nu ^{1/2}\rho _1$ or $\cong \nu ^{1/2}\rho _2$ for some k. Using dimensions again, we have for $l\ne k$, $[\rho _1]$ and $[\rho _2]$ are unlinked to $\Delta _l'$ and so

$$\begin{aligned} \pi ' \cong (\nu ^{1/2}\cdot \rho _r)\times {\mathrm {St}}({\mathfrak {m}}'{\setminus } \left\{ \Delta _k \right\} ) ,\end{aligned}$$

for $r=1$ or 2. Then

$$\begin{aligned} {}^{((n-1)/2)}\pi '\cong \nu ^{1/2}\cdot \rho _r , \end{aligned}$$

which implies

$$\begin{aligned} {}^{((n-1)/2)}{\mathrm {St}}({\mathfrak {m}}'{\setminus } \left\{ \Delta _k' \right\} )\ne 0 . \end{aligned}$$

Thus ${\mathrm {St}}({\mathfrak {m}}'{\setminus } \left\{ \Delta _k' \right\} )$ is generic and so is $\pi '$.

Case 2 ${\mathrm {Hom}}_{G_{(n+1)/2}}( \pi ^{[(n+1)/2]}, {}^{(n-1)/2}\pi ') = 0$. We must have

$$\begin{aligned} {\mathrm {Hom}}_{G_{n+1-i}}(\pi ^{[n+1]}, {}^{(n)}\pi ')\ne 0 \end{aligned}$$

and so ${}^{(n)}\pi '\ne 0$. Hence $\pi '$ is generic. $\square$

We now achieve the classification of irreducible representations which are projective when restricted from $G_{n+1}$ to $G_n$:

Theorem 3.7

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. Then $\pi |_{G_n}$ is projective if and only if $\pi$ is relatively projective in Definition 3.4.

Proof

The if direction is proved in Lemma 3.6. The only if direction follows from Lemma 3.5 and Theorem 3.1. $\square$

One advantage for such classification is that those restricted representations admit a more explicit realization as shown in [15]:

Theorem 3.8

Let $\pi , \pi ' \in {\mathrm {Irr}}(G_{n+1})$. If $\pi$ and $\pi '$ are relatively projective, then $\pi |_{G_n} \cong \pi '|_{G_n}$. In particular, $\pi |_{G_n}$ is isomorphic to the Gelfand–Graev representation ${\mathrm {ind}}_{U_n}^{G_n} \psi _n$.

Proof

This follows from that $\pi$ and $\pi '$ are projective in ${\mathrm {Alg}}(G_n)$ and [15]. $\square$

4 Gelfand–Graev representations and affine Hecke algebras

Several insights come from the affine Hecke algebra realization of Gelfand–Graev representations. We shall first recall those results.

4.1 Affine Hecke algebras

Definition 4.1

The affine Hecke algebra ${\mathcal {H}}_{l}(q)$ of type A is an associative algebra over ${\mathbb {C}}$ generated by $\theta _1,\ldots , \theta _{l}$ and $T_w$ ($w \in S_{l}$) satisfying the relations:

• (1) $\theta _i\theta _j=\theta _j\theta _i$;

• (2) $T_{s_k}\theta _k-\theta _{k+1}T_{s_k}=(q-1)\theta _k$, where q is a certain prime power and $s_k$ is the transposition between the numbers k and $k+1$;

• (3) $T_{s_k}\theta _i=\theta _i T_{s_k}$, where $i \ne k, k+1$

• (4) $(T_{s_k}-q)(T_{s_k}+1)=0$;

• (5) $T_{s_k}T_{s_{k+1}}T_{s_k}=T_{s_{k+1}}T_{s_k}T_{s_{k+1}}$.

Let ${\mathcal {A}}_l(q)$ be the (commutative) subalgebra generated by $\theta _1, \ldots , \theta _l$. Let ${\mathcal {H}}_{W,l}(q)$ be the subalgebra generated by $T_{s_1},\ldots , T_{s_{l-1}}$. Let ${\mathrm {sgn}}$ be the 1-dimensional ${\mathcal {H}}_{W,l}(q)$-module characterized by $T_{s_k}$ acting by $-1$.

It is known from [25, Proposition 3.11] that the center ${\mathcal {Z}}_l$ of ${\mathcal {H}}_l(q)$ has a basis $\left\{ z_M =\sum _{w \in S_l} \theta _1^{i_{w(1)}}\ldots \theta _l^{i_{w(l)}} \right\} _M$, where $M=(i_1, \ldots , i_l)$ runs for all l-tuples in ${\mathbb {Z}}^l/S_l$.

Bernstein decomposition asserts that

$$\begin{aligned} {\mathfrak {R}}(G_n) \cong \prod _{{\mathfrak {s}} \in {\mathfrak {B}}(G_n)} {\mathfrak {R}}_{{\mathfrak {s}}}(G_n) , \end{aligned}$$

where ${\mathfrak {R}}(G_n)$ is the category of smooth $G_n$-representations, ${\mathfrak {B}}(G_n)$ is the set of inertial equivalence classes of $G_n$ and ${\mathfrak {R}}_{{\mathfrak {s}}}(G_n)$ is the full subcategory of ${\mathfrak {R}}(G_n)$ associated to ${\mathfrak {s}}$ (see [10]). For a smooth representation $\pi$ of $G_n$, define $\pi _{{\mathfrak {s}}}$ to be the projection of $\pi$ to the component ${\mathfrak {R}}_{{\mathfrak {s}}}(G_n)$.

For each ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$, [9] and [10] associate with a compact group $K_{{\mathfrak {s}}}$ and a finite-dimensional representation $\tau$ of $K_{{\mathfrak {s}}}$, such that the convolution algebra

$$\begin{aligned} {\mathcal {H}}(K_{{\mathfrak {s}}}, \tau ) := \left\{ f : G_n \rightarrow {\mathrm {End}}(\tau ^{\vee }) : f(k_1gk_2) = \tau ^{\vee }(k_1)\circ f(g) \circ \tau ^{\vee }(k_2) \text{ for } k_1,k_2 \in K_{{\mathfrak {s}}} \right\} \end{aligned}$$

is isomorphic to the product ${\mathcal {H}}_{n_1}(q_1) \otimes \ldots \otimes {\mathcal {H}}_{n_r}(q_r)$ of affine Hecke algebra of type A, denoted by ${\mathcal {H}}_{{\mathfrak {s}}}$. For a smooth representation $\pi$ of $G_n$, the algebra ${\mathcal {H}}(K_{{\mathfrak {s}}},\tau )$ acts naturally on the space ${\mathrm {Hom}}_{K_{{\mathfrak {s}}}}(\tau , \pi ) \cong (\tau ^{\vee }\otimes \pi )^{K_{{\mathfrak {s}}}}$. This defines an equivalence of categories:

$$\begin{aligned} {\mathfrak {R}}_{{\mathfrak {s}}}(G_n) \cong \text{ category } \text{ of } {\mathcal {H}}_{{\mathfrak {s}}}\text{-modules } . \end{aligned}$$

(4.11)

By abuse notation, we shall identify ${\mathrm {Hom}}_{K_{{\mathfrak {s}}}}(\tau , \pi )$ with $\pi _{{\mathfrak {s}}}$ under (4.11) and consider $\pi _{{\mathfrak {s}}}$ as ${\mathcal {H}}_{{\mathfrak {s}}}$-module.

Let ${\mathcal {A}}_{{\mathfrak {s}}} ={\mathcal {A}}_{n_1}(q_1)\otimes \ldots \otimes {\mathcal {A}}_{n_r}(q_r)$. Let ${\mathcal {H}}_{W,{\mathfrak {s}}}={\mathcal {H}}_{W,n_1}(q_1)\otimes \ldots \otimes {\mathcal {H}}_{W, n_r}(q_r)$. Let

$$\begin{aligned} {\mathrm {sgn}}_{{\mathfrak {s}}} ={\mathrm {sgn}}\boxtimes \ldots \boxtimes {\mathrm {sgn}} \end{aligned}$$

as an ${\mathcal {H}}_{W,{\mathfrak {s}}}$-module. Note that in [13], we proved when ${\mathfrak {s}}$ is a simply type, but the generalization to all types follows from [10] and a simple generalization of [13, Theorem 2.1]. The center of ${\mathcal {H}}_{{\mathfrak {s}}}$ is equal to ${\mathcal {Z}}_1 \otimes \ldots \otimes {\mathcal {Z}}_r$, where each ${\mathcal {Z}}_k$ is the center of ${\mathcal {H}}_{n_k}(q_k)$.

Recall that $\Pi _n={\mathrm {ind}}_{U_n}^{G_n} \psi _n$. We may simply write $\Pi$ for $\Pi _n$ if there is no confusion.

Theorem 4.2

[13] For any ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$, the Bernstein component of the Gelfand–Graev representation $\Pi _{{\mathfrak {s}}}$ is isomorphic to ${\mathcal {H}}_{{\mathfrak {s}}} \otimes _{{\mathcal {H}}_{W,{\mathfrak {s}}}} {\mathrm {sgn}}_{{\mathfrak {s}}}$.

Using Theorem 4.2, we have that $\Pi _{{\mathfrak {s}}}$ is isomorphic to ${\mathcal {A}}_{{\mathfrak {s}}}$, as ${\mathcal {A}}_{{\mathfrak {s}}}$-module. This observation has the following consequence:

Lemma 4.3

For any ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$, any two non-zero submodules of $\Pi _{{\mathfrak {s}}}$ have non-zero intersections. In particular, $\Pi _{{\mathfrak {s}}}$ is indecomposable.

We remark that for general quas-split group G, the indecomposability of $\Pi _{{\mathfrak {s}}}$ can be deduced from results of [8], which show that endomorphism algebra of $\Pi _{{\mathfrak {s}}}$ is isomorphic to the Bernstein center at the corresponding (indecomposable) Bernstein block ${\mathfrak {R}}_{{\mathfrak {s}}}(G)$.

We shall also need:

Lemma 4.4

Fix ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$. Let ${\mathcal {J}}$ be a maximal ideal in ${\mathcal {Z}}_{{\mathfrak {s}}}$ that annihilates an irreducible module in ${\mathfrak {R}}_{{\mathfrak {s}}}(G_n)$. Let $\pi$ be a non-zero submodule of $\Pi _{{\mathfrak {s}}}$. Let ${\widehat{\pi }}$ be the ${\mathcal {J}}$-adic completion of $\pi$. Then ${\widehat{\pi }} \ne 0$.

Proof

Note that ${\mathcal {A}}_{{\mathfrak {s}}}$ is an integral domain and is finitely-generated over ${\mathcal {Z}}_{{\mathfrak {s}}}$. Hence, by Krull’s intersection theorem, $\cap _k {\mathcal {J}}^k{\mathcal {A}}_{{\mathfrak {s}}}=0$. Thus $\cap _k {\mathcal {J}}^k \pi =0$. This implies that $\pi /{\mathcal {J}}^k \pi \ne 0$ for some k. Hence ${\widehat{\pi }}\ne 0$. $\square$

4.2 Jacquet functors on Gelfand–Graev representations

Lemma 4.5

Let $P=LN$ be the parabolic subgroup containing upper triangular matrices and block-diagonal matrices ${\mathrm {diag}}(g_1,\ldots , g_r)$ with $g_k \in G_{i_k}$, where $i_1+\ldots +i_r=n$. Then $(\Pi _n )_N \cong \Pi _{i_1} \boxtimes \ldots \boxtimes \Pi _{i_r}$.

Proof

Let w be a permutation matrix in $G_n$. Then $w(N) \cap U_n$ contains a unipotent subgroup $\left\{ I_n +tu_{k,k+1} : t \in F \right\}$ for some k if and only if $w(N) \not \subset U_n^-$. Here $u_{k,k+1}$ is a matrix with $(k,k+1)$-entry 1 and other entries 0. For any such w, it gives that PwB is the same unique open orbit in $G_n$. Now the geometric lemma in [7, Theorem 5.2] gives the lemma. $\square$

5 Submodule structure of Bernstein–Zelevinsky layers

5.1 Inertial equivalence classes

We give more discussions on inertial equivalence classes, relating the parabolic induction. See, for example, [10]. An inertial equivalence class ${\mathfrak {s}}$ of $G_n$ can be represented by a pair $[G_{m_1}\times \ldots \times G_{m_r}, \rho _1 \boxtimes \ldots \boxtimes \rho _r]$, where $m_1+\ldots +m_r=n$ and each $\rho _k$ is a cuspidal $G_{m_k}$-representation. Two pairs

$$\begin{aligned} {[}G_{m_1}\times \ldots \times G_{m_r}, \rho _1\boxtimes \ldots \boxtimes \rho _r], [G_{m_1'}\times \ldots \times G_{m_s'}, \rho _1'\boxtimes \ldots \boxtimes \rho _s'] \end{aligned}$$

represent the same inertial equivalence class if and only if $r=s$ and there exists a permutation $\sigma \in S_r$ such that

$$\begin{aligned} G_{m_1}=G_{m_{\sigma (1)}'}, \ldots , G_{m_r}=G_{m_{\sigma (r)}'} \end{aligned}$$

and

$$\begin{aligned} \rho _1\cong \chi _1 \otimes \rho _{\sigma (1)}', \ldots , \rho _r \cong \chi _r \otimes \rho _{\sigma (r)}' \end{aligned}$$

for some unramified character $\chi _k$ of $G_{m_k}$ ($k=1,\ldots , r$).

Let

$$\begin{aligned} {\mathfrak {s}}_1= & {} [G_{m_1}\times \ldots \times G_{m_r}, \rho _1\boxtimes \ldots \boxtimes \rho _r] \in {\mathfrak {B}}(G_{n_1}), \\ {\mathfrak {s}}_2= & {} [G_{m_1'}\times \ldots \times G_{m_s'}, \rho _1' \boxtimes \ldots \boxtimes \rho _s'] \in {\mathfrak {B}}(G_{n_2}) . \end{aligned}$$

Then $\pi _1 \times \pi _2$ lies in ${\mathfrak {R}}_{{\mathfrak {s}}}(G_{n_1+n_2})$, where

$$\begin{aligned} {\mathfrak {s}}=[G_{m_1}\times \ldots \times G_{m_r}\times G_{m_1'}\times \ldots \times G_{m_s'}, \rho _1\boxtimes \ldots \boxtimes \rho _r \boxtimes \rho _1'\boxtimes \ldots \boxtimes \rho _s'] \end{aligned}$$

From this, one deduces the following lemma:

Lemma 5.1

Let $\pi _1 \in {\mathfrak {R}}_{{\mathfrak {s}}}(G_{n_1})$ for some ${\mathfrak {s}}\in {\mathfrak {B}}(G_{n_1})$ and let $\pi _2\in {\mathrm {Alg}}(G_{n_2})$. Fix ${\mathfrak {t}} \in {\mathfrak {B}}(G_{n_1+n_2})$ with $(\pi _1 \times \pi _2)_{{\mathfrak {t}}}\ne 0$. There exists a unique ${\mathfrak {s}}' \in {\mathfrak {B}}(G_{n_1+n_2})$ such that $\pi _1 \times (\pi _2)_{{\mathfrak {s}}'} \cong (\pi _1 \times \pi _2)_{{\mathfrak {t}}}$.

5.2 Bernstein center

The Bernstein center of a category ${\mathfrak {R}}$ is defined as the endomorphism ring of the identity functor in ${\mathfrak {R}}$. Denote by ${\mathfrak {Z}}_n$ the Bernstein center of ${\mathfrak {R}}(G_n)$. For ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$, denote by ${\mathfrak {Z}}_{{\mathfrak {s}}}$ the Bernstein center of ${\mathfrak {R}}_{{\mathfrak {s}}}(G_n)$, which is a finitely-generated commutative algebra [3, Theorem 2.13]. Explicitly, for

$$\begin{aligned} {\mathfrak {s}}=[G_{n_1}\times \ldots \times G_{n_k}, \rho _1 \boxtimes \ldots \boxtimes \rho _k ] \in {\mathfrak {B}}(G_n), \end{aligned}$$

set, the Bernstein variety, to be

$$\begin{aligned} {\mathfrak {X}}_{{\mathfrak {s}}}=({\mathbb {C}}^{\times })^k/W_{{\mathfrak {s}}}, \end{aligned}$$

where each copy of ${\mathbb {C}}^{\times }$ comes from the group of unramified characters of $G_{n_p}$ and $W_{{\mathfrak {s}}}$ is the subgroup of $N_G(M)/M$, where $M=G_{n_1}\times \ldots \times G_{n_k}$, which stabilizes $\rho _1\boxtimes \ldots \boxtimes \rho _k$, and $W_{{\mathfrak {s}}}$ permutes the factors in $({\mathbb {C}}^{\times })^k$ via its action on M. Here $N_G(M)$ is the normalizer of M in G. According to [3],

$$\begin{aligned} {\mathfrak {Z}}_{{\mathfrak {s}}} \cong {\mathbb {C}}[{\mathfrak {X}}_{{\mathfrak {s}}}] , \end{aligned}$$

and we also have

$$\begin{aligned} {\mathfrak {Z}}_n \cong \prod _{{\mathfrak {s}} \in {\mathfrak {B}}(G_n)} {\mathfrak {Z}}_{{\mathfrak {s}}} . \end{aligned}$$

It follows from [3, Corollaire 3.4] that for any maximal ideal ${\mathcal {J}}$ of ${\mathfrak {Z}}_n$ and any finitely-generated $\pi$ of $G_n$, $\pi /{\mathcal {J}}(\pi )$ is admissible. For more properties of the Bernstein center, see, e.g. [3, 5, 8, 20].

For a maximal ideal ${\mathcal {J}}$ in ${\mathfrak {Z}}_n$ and $\pi \in {\mathrm {Alg}}(G_n)$, define ${\widehat{\pi }}$ to be the ${\mathcal {J}}$-adic completion of $\pi$ i.e. the inverse limit

$$\begin{aligned} {\widehat{\pi }} ={\mathrm {lim}}_i~ \pi /({\mathcal {J}}^i\pi ) , \end{aligned}$$

which has a natural G-module structure from $\pi$. The notion ${\widehat{\pi }}$ depends on ${\mathcal {J}}$, and it should be clear from the context. Note that ${\mathcal {J}}$ comes from some maximal ideal in ${\mathfrak {Z}}_{{\mathfrak {s}}}$ for some unique ${\mathfrak {s}}\in {\mathfrak {B}}(G_n)$ and this gives that

$$\begin{aligned} {\widehat{\pi }} \in {\mathfrak {R}}_{{\mathfrak {s}}}(G_n) \end{aligned}$$

Lemma 5.2

Let $\omega \in {\mathrm {Alg}}(G_{n_1})$ be admissible with $n_1\ne 0$. Fix $\tau \in {\mathrm {Irr}}(G_{n_1+n_2})$ such that ${\mathrm {csupp}}(\omega ) \cap {\mathrm {csupp}}(\tau ) =\emptyset$. Let $\pi =\omega \times \Pi _{n_2}$ and $\pi '=\Pi _{n_2}\times \omega$. Let ${\mathcal {J}}$ be the maximal ideal in ${\mathfrak {Z}}_{n_1+n_2}$ which annihilates $\tau$. Then

• (1) ${\widehat{\pi }}=0$; and

• (2) $\widehat{\pi '}=0$.

Proof

We only prove (1) and a proof for (2) is similar. Each Bernstein component of $\Pi$ is finitely generated [8]. Thus $\pi /{\mathcal {J}} (\pi )$ is admissible [3] and is annihilated by ${\mathcal {J}}$. If $\pi /{\mathcal {J}}(\pi )$ is non-zero, then $\pi$ has a subquotient with irreducible composition factors isomorphic to $\tau '$ with ${\mathrm {csupp}}(\tau ')={\mathrm {csupp}}(\tau )$ [3, Proposition 2.11, Theorem 2.13]. However, using Jacquet functor, one shows that the cuspidal support of any irreducible subquotients of $\omega \times \Pi _{n_2}$ has non-zero intersection with ${\mathrm {csupp}}(\omega )$. This gives a contradiction. Hence $\pi /{\mathcal {J}}(\pi )=0$ and so ${\widehat{\pi }}=0$. $\square$

5.3 Intersection properties of Bernstein–Zelevinsky layers

The main tool of this section is the exactness of Jacquet functor.

Lemma 5.3

For $i\ne j$, let $\omega , \tau$ be an admissible $G_{n-i}$-representation and $G_{n-j}$-representation respectively. There is no non-zero isomorphic submodule of ${\mathrm {ind}}^{G_n}_{R_{n-i}} \omega \boxtimes \psi _i$ and ${\mathrm {ind}}_{R_{n-j}^-}^{G_n} \tau \boxtimes \psi _j$.

Proof

We only prove for $i>j$ and the case for $j<i$ is similar. Suppose there exists a non-zero isomorphic submodule $\lambda$ of $\omega \times \Pi _i$ and $\Pi _j \times \tau$. By Frobenius reciprocity on $\lambda \hookrightarrow \omega \times \Pi _i$,

$$\begin{aligned} \lambda _{N_i} \ne 0 . \end{aligned}$$

(5.12)

We have that $({\mathrm {ind}}^{G_n}_{R_{n-i}} \omega \otimes \psi _i)_{N_i}$ admits a filtration, from geometric lemma [7, Theorem 5.2], that successive quotients take the form

$$\begin{aligned} ( \omega ' \times \Pi _k) \boxtimes (\omega '' \times \Pi _{i-k} ). \end{aligned}$$

(5.13)

Similarly, $({\mathrm {ind}}^{G_n}_{R_{n-i}^-} \tau \boxtimes \psi _j)_{N_i}$ admits a filtration that successive quotients take the form

$$\begin{aligned} (\Pi _l \times \tau ') \boxtimes (\Pi _{j-l}\times \tau ''), \end{aligned}$$

(5.14)

where $\tau '$ is an admissible $G_{n-i-l}$-representation and $\tau ''$ is an admissible $G_{i-j+l}$ representation. Since Jacquet functor is exact, it reduces, by Lemma 10.3 and (5.12), to see that there is no submodule for

$$\begin{aligned} (\Pi _l \times \tau ') \boxtimes (\Pi _{j-l}\times \tau ''), \quad \text{ and } ( \omega ' \times \Pi _k) \boxtimes (\omega '' \times \Pi _{i-k} ). \end{aligned}$$

for all k, l.

In the case that $k=0$ and $l=j$; or $k=n-i$ and $l=0$, a certain element in the Bernstein center annihilates $\omega ''$ or $\omega '$, but not on $\Pi _i$ or $\Pi _{n-i}$ respectively (by [8]). Now to prove other cases, we notice that either $k>l$ or $i-k>j-l$ (which happens when $l\ge k$). In the first case, we apply $N_k \subset G_{n-i}$ on the first factor, and in the second case, apply $N_{i-k}$ on the second factor. Then repeat the process and the process terminates as each process the value on i decreases. $\square$

A special case of Lemma 5.3 is that $i=0$ and $j=n \ge 1$. In such case, Lemma 5.3 says that the Gelfand–Graev representation $\Pi _n$ does not admit an irreducible submodule. We also remark that the last fact also holds for a more general quasi-split connected reductive group with a non-compact center,³ which can be deduced from a combination of Ext-duality (see [11, 26, 29]) and the projectivity of a Gelfand–Graev representation (see [13]).

For convenience, an admissible representation $\pi$ of $G_n$ is said to be uniform if all its composition factors have the same cuspidal support. We shall set ${\mathrm {csupp}}(\pi )={\mathrm {csupp}}(\pi ')$ for any irreducible $\pi ' \in {\mathrm {JH}}(\pi )$. It is well-known that any admissible representation can be written as direct sum of uniform representations. For a uniform representation $\pi$ of $G_n$, we associate the maximal ideal ${\mathcal {J}}_{\pi }$ in ${\mathfrak {Z}}_n$ such that some power of ${\mathcal {J}}_{\pi }$ annihilates $\pi$.

Lemma 5.4

Let $\omega \in {\mathrm {Alg}}(G_{n-i})$ be uniform. Let $\tau \in {\mathrm {Irr}}(G_i)$ such that ${\mathrm {csupp}}(\tau ) \cap {\mathrm {csupp}}(\omega )=\emptyset$. Set ${\mathcal {I}}={\mathcal {J}}_{\omega } \otimes 1 +1 \otimes {\mathcal {J}}_{\tau }$ in ${\mathfrak {Z}}_{n-i}\otimes {\mathfrak {Z}}_i$. Then, as $G_{n-i}\times G_i$-representations,

$$\begin{aligned} \widehat{(\omega \times \Pi _i)_{N_i}} = \omega \boxtimes {\widehat{\Pi }}_i. \end{aligned}$$

(Here $\widehat{(\omega \times \Pi _i)_{N_i}}$ is the ${\mathcal {I}}$-adic completion for the $G_{n-i}\times G_i$-representation $(\omega \times \Pi _i)_{N_i}$. The ${\widehat{\Pi }}_i$ is the ${\mathcal {J}}_{\tau }$-adic completion of $\Pi _i$.)

Proof

We have observed that $(\omega \times \Pi _i)_{N_i}$ admits a filtration with successive quotients in (5.13). We now apply the ${\mathcal {I}}$-adic completion on each subquotients in the form of (5.13). By using Lemma 5.2 and the condition that ${\mathrm {csupp}}(\tau ) \cap {\mathrm {csupp}}(\omega )=\emptyset$, the only successive quotient which is non-zero after taking the ${\mathcal {I}}$-adic completion is the factor $\omega \boxtimes \Pi _i$. Since ${\mathcal {I}}$-adic completion is exact on finitely-generated $G_{n-i}\times G_i$-representations, we obtain that

$$\begin{aligned} \widehat{(\omega \times \Pi _i)_{N_i}} \cong \omega \boxtimes \widehat{\Pi _i} . \end{aligned}$$

We make two remarks on the above proof:

• In order to get finitely-generated modules, one has to consider each Bernstein component of $\omega \times \Pi _i$ and those successive quotients.

• To show the exactness on the ${\mathcal {I}}$-adic completion, one can pass to the Hecke algebra so that a finitely-generated $G_{n-i}\times G_i$-representation will give rise a corresponding Hecke algebra module which is finitely-generated over ${\mathfrak {Z}}_{n-i}\otimes {\mathfrak {Z}}_i$.$\square$

Lemma 5.5

Keep using the notation in the previous lemma. We also have that

$$\begin{aligned} \widehat{(\Pi _i \times \omega )_{N_i}} \cong \omega \boxtimes {\widehat{\Pi }}_i . \end{aligned}$$

Proof

The proof is the same as that of Lemma 5.4 except that we use the filtration (5.14) instead of (5.13). $\square$

Proposition 5.6

Let $\omega _1, \omega _2 \in {\mathrm {Alg}}(G_{n-i})$ be admissible and non-zero. If ${\mathrm {ind}}_{R_{n-i}}^{G_n} \omega _1 \boxtimes \psi _i$ and ${\mathrm {ind}}_{R_{n-i}^-}^{G_n} \omega _2 \boxtimes \psi _i$ have isomorphic non-zero submodules, then $\omega _1$ and $\omega _2$ have isomorphic non-zero submodules.

Proof

Let $0 \ne \pi \in {\mathrm {Alg}}(G_n)$ such that

$$\begin{aligned} \pi {\mathop {\hookrightarrow }\limits ^{\iota _1}} \omega _1 \times \Pi _i, \quad \text{ and } \quad \pi {\mathop {\hookrightarrow }\limits ^{\iota _2}} \Pi _i \times \omega _2 . \end{aligned}$$

We now write $\omega _1$ as direct sum of uniform representations $\lambda _1, \ldots , \lambda _k$. Then $\omega _1 \times \Pi _i$ admits a filtration whose successive quotients of the form $\lambda _r \times \Pi _i$. By Lemma 10.2, we have that

$$\begin{aligned} \pi ' \hookrightarrow \lambda _r \times \Pi _i \end{aligned}$$

(5.15)

for some r and some non-zero submodule $\pi '$ of $\pi$. It suffices to show that $\lambda _r$ and $\omega _2$ share an isomorphic irreducible submodule. To this end, now set ${\mathcal {I}}={\mathcal {J}}_{\lambda _r}\otimes 1+1\otimes {\mathcal {J}}_{\tau }$, where $\tau \in {\mathrm {Irr}}(G_i)$ satisfies ${\mathrm {csupp}}(\lambda _r)\cap {\mathrm {csupp}}(\tau )=\emptyset$.

Frobenius reciprocity gives a non-zero map from $\pi '_{N_i}$ to $\lambda _r \boxtimes \Pi _i$ and so the image is a non-zero submodule of $\lambda _r \boxtimes \Pi _i$, which is still non-zero after taking ${\mathcal {I}}$-adic completion by Lemma 4.4. As taking ${\mathcal {I}}$-adic completion is exact, we have that $\widehat{\pi '}_{N_i} \ne 0$.

Now taking exact functors on $\pi ' \hookrightarrow \Pi _i \times \omega _2$, we obtain:

$$\begin{aligned} 0\ne \widehat{\pi '_{N_i}} \hookrightarrow \widehat{(\Pi _i \times \omega _2)}_{N_i} \end{aligned}$$

(5.16)

Now we regard $\widehat{\pi '_{N_i}}$ as $G_{n-i}$-submodule via the embedding $g \mapsto {\mathrm {diag}}(g,I_i)$. Using (5.15) and Lemma 5.4, any $G_{n-i}$-submodule of $\widehat{\pi '_{N_i}}$ is a submodule of $\lambda _r$. Similarly, using (5.16) and Lemma 5.5, any $G_{n-i}$-submodule of $\widehat{\pi '_{N_i}}$ is a submodule of $\omega _2$. This concludes that $\lambda _r$ and $\omega _2$ share isomorphic irreducible $G_{n-i}$-submodules. $\square$

5.4 Strong indecomposability

We first prove a preparation lemma.

Lemma 5.7

Let $\pi _1, \pi _2, \pi$ be in ${\mathrm {Alg}}(G_n)$ such that $\pi _1 \hookrightarrow \pi$ and $\pi _2 \hookrightarrow \pi$. Let $N=N_i$ for some i. Then $(\pi _1 \cap \pi _2)_N \cong (\pi _1)_N \cap (\pi _2)_N$. Here the later intersection is taken in $\pi _N$.

Proof

We have the natural projection $p:\pi _1 \cap \pi _2 \rightarrow \pi _N$ as linear spaces. Since the image of the projection lies in both $(\pi _1)_N$ and $(\pi _2)_N$, the projection factors through the embedding $(\pi _1)_N \cap (\pi _2)_N$ to $\pi _N$. Now taking the Jacquet functor on p gives an isomorphism from $(\pi _1\cap \pi _2)_N$ onto $(\pi _1\cap \pi _2)_N \subset \pi _N$. Thus the map from $(\pi _1)_N \cap (\pi _2)_N$ to $(\pi _1 \cap \pi _2)_N$ is also surjective. $\square$

We shall prove a weak version on the strong indecomposability of the Bernstein–Zelevinsky induction. The stronger version of Lemma 5.8 is to only assume the embedding on either left or right Bernstein–Zelevinsky induction. Proving such statement requires more work and we will not do it here. Some special cases can be more easily achieved by using Theorem 6.1 below. In Sect. 8, we shall prove a variation, which says that the Bernstein–Zelevinsky induction preserves indecomposability (but not strong indecomposability).

Lemma 5.8

Let $\omega \in {\mathrm {Irr}}(G_{n-i})$. Let ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$. Suppose there exists non-zero $\pi _1, \pi _2 \in {\mathrm {Alg}}(G_n)$ with the following embeddings:

$$\begin{aligned} \pi _1 \hookrightarrow (\omega \times \Pi _i)_{{\mathfrak {s}}}, \quad \pi _2 \hookrightarrow (\omega \times \Pi _i)_{{\mathfrak {s}}} \end{aligned}$$

and

$$\begin{aligned} \pi _1 \hookrightarrow (\Pi _i \times \omega )_{{\mathfrak {s}}}, \quad \pi _2 \hookrightarrow (\Pi _i \times \omega )_{{\mathfrak {s}}} . \end{aligned}$$

Then the images of the embeddings of $\pi _1$ and $\pi _2$ in $(\omega \times \Pi _i)_{{\mathfrak {s}}}$ have non-zero intersections. Similarly, the images of the embeddings of $\pi _1$ and $\pi _2$ in $(\Pi _i \times \omega )_{{\mathfrak {s}}}$ have non-zero intersections.

Proof

We shall prove the latter statement. The former statement can be proved similarly. We shall identify $\pi _k$ ($k=1,2$) with the image of the embedding of $\pi _k$ in $(\Pi _i\times \omega )_{{\mathfrak {s}}}$.

Let $\tau \in {\mathrm {Irr}}(G_i)$ with ${\mathrm {csupp}}(\tau )\cap {\mathrm {csupp}}(\omega )=\emptyset$. Set ${\mathcal {I}}={\mathcal {J}}_{\omega }\otimes 1+1\otimes {\mathcal {J}}_{\tau }$. Using a similar argument to the one in the proof of Proposition 5.6 (which uses Frobenius reciprocity and Lemma 4.4), we have that

$$\begin{aligned} \widehat{(\pi _1)_{N_i}} \ne 0 \end{aligned}$$

(5.17)

(We recall that we need the embedding from $\pi _1$ to $\omega \times \Pi _i$ to prove the non-zero part.)

On the other hand, as observed in Proposition 5.6, among all the successive quotients in the filtration of $(\Pi _i \times \omega )_{N_i}$ of the form (5.15), the only one that does not vanish after taking ${\mathcal {I}}$-adic completion is $\omega \boxtimes \Pi _i$.

Moreover, the successive quotient $\omega \boxtimes \Pi _i$ lies in the bottom layer of $(\Pi _i \times \omega )_{N_i}$ [7, Theorem 5.2]. Thus, the non-vanishing (5.17) implies that $(\pi _1)_{N_i} \cap (\omega \boxtimes \Pi _i) \ne 0$. Moreover, the Bernstein component ${\mathfrak {s}}$ uniquely determines a ${\mathfrak {t}} \in {\mathfrak {B}}(G_i)$ such that $(\pi _1)_{N_i} \cap (\omega \boxtimes (\Pi _i)_{{\mathfrak {t}}}) \ne 0$ (c.f. Lemma 5.1).

Similarly, we have that $(\pi _2)_{N_i} \cap (\omega \boxtimes (\Pi _i)_{{\mathfrak {t}}}) \ne 0$. Thus since $\omega \boxtimes (\Pi _i)_{{\mathfrak {t}}}$ is strongly indecomposable $G_{n-i}\times G_i$-representation (by Lemma 4.3 and $\omega$ is irreducible), we have that

$$\begin{aligned} (\pi _1)_{N_i} \cap (\pi _2)_{N_i} \ne 0 . \end{aligned}$$

By Lemma 5.7, $(\pi _1 \cap \pi _2)_{N_i} \ne 0$ and so $\pi _1 \cap \pi _2 \ne 0$. $\square$

6 Indecomposability of restricted representations

6.1 Indecomposability of restriction

We now prove our main result:

Theorem 6.1

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. Then for each ${\mathfrak {s}} \in {\mathfrak {R}}(G_n)$, $\pi _{{\mathfrak {s}}}$ is strongly indecomposable whenever it is nonzero i.e. for any two non-zero submodules $\tau , \tau '$ of $\pi _{{\mathfrak {s}}}$, $\tau \cap \tau ' \ne 0$.

Proof

There exists a Bernstein–Zelevinsky $G_n$-filtration on $\pi$ with

$$\begin{aligned} \pi _n \subset \pi _{n-1} \subset \ldots \subset \pi _1 \subset \pi _0 =\pi \end{aligned}$$

(6.18)

such that

$$\begin{aligned} \pi _{i}/\pi _{i+1} \cong {\mathrm {ind}}_{R_{n-i}}^{G_n} \pi ^{[i+1]} \boxtimes \psi _{i} . \end{aligned}$$

(6.19)

We also have a $G_n$-filtration on $\pi$ with

$$\begin{aligned} {}_n\pi \subset {}_{n-1} \pi \subset \ldots \subset {}_1\pi \subset {}_0\pi =\pi \end{aligned}$$

(6.20)

such that

$$\begin{aligned} {}_{i}\pi /{}_{i+1}\pi \cong {\mathrm {ind}}_{R_{n-i}^-}^{G_n} {}^{[i+1]}\pi \boxtimes \psi _{i}. \end{aligned}$$

(6.21)

Let $i^*$ be the level of $\pi$. For ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$ such that $\pi _{{\mathfrak {s}}} \ne 0$, we also have $(\pi _{i^*-1})_{{\mathfrak {s}}} \ne 0$ (see Sect. 6.2 for the detail). For notation simplicity, we set $\tau =(\pi _{i^*-1})_{{\mathfrak {s}}}$ and $\tau '=({}_{i^*-1}\pi )_{{\mathfrak {s}}}$, both regarded as subspaces of $\pi$.

Let $\omega$ and $\gamma$ be two non-zero submodules of $\pi _{{\mathfrak {s}}}$.

Claim $\omega \cap \tau \ne 0$.

Proof of the Claim Suppose not. Then, the natural projection gives an injection

$$\begin{aligned} \omega \hookrightarrow \pi _{{\mathfrak {s}}}/\tau \end{aligned}$$

(6.22)

By Lemma 5.3, there is no isomorphic submodules between $\pi _{{\mathfrak {s}}}/\tau$ and $\tau '$. This implies $\omega \cap \tau '=0$. Hence, we also have an injection:

$$\begin{aligned} \omega \hookrightarrow \pi _{{\mathfrak {s}}}/\tau ' \end{aligned}$$

(6.23)

By (6.18), (6.20), (6.22), (6.23) and Lemma 10.3, there exists a $G_n$-representation which is isomorphic to submodules of

$$\begin{aligned} ( \pi ^{[j]} \times \Pi _{j-1})_{{\mathfrak {s}}}, \quad \text{ and } \quad (\Pi _{k-1} \times {}^{[k]}\pi )_{{\mathfrak {s}}} \end{aligned}$$

for some $j,k< i^*$. By Lemma 5.3 again, we must have $j=k$. However, Proposition 5.6 contradicts to the following Theorem 6.2 below (whose proof does not depend on this result). This proves the claim.

Since $\omega$ is an arbitrary submodule of $\pi$, we also have $\gamma \cap \tau \ne 0$. Now we refine our $G_n$-modules and set:

$$\begin{aligned} 0\ne \omega ' =\omega \cap \tau \subset \pi _{{\mathfrak {s}}} , \quad 0\ne \gamma ' = \gamma \cap \tau \subset \pi _{{\mathfrak {s}}}. \end{aligned}$$

Now using similar argument as above, we also have that

$$\begin{aligned} \omega ' \cap \tau ' \ne 0, \quad \gamma ' \cap \tau ' \ne 0 . \end{aligned}$$

Now we further refine our $G_n$-modules and set

$$\begin{aligned} \omega '' = \omega '\cap \tau ' \subset \pi _{{\mathfrak {s}}}, \quad \gamma ''=\gamma ' \cap \tau ' \subset \pi _{{\mathfrak {s}}} . \end{aligned}$$

Now $\omega ''$ and $\gamma ''$ have desired embeddings as in Lemma 5.8. Hence Lemma 5.8 implies that $\omega '' \cap \gamma '' \ne 0$ and so we have $\omega \cap \gamma \ne 0$ as desired. $\square$

We remark that one can use Theorem 6.1 to show that for $\omega \in {\mathrm {Irr}}(G_{m})$ and for i large enough (e.g. $i\ge m$), $(\omega \times \Pi _i)_{{\mathfrak {s}}}$ (resp. $(\Pi _i \times \omega )_{{\mathfrak {s}}}$) are strongly indecomposable for any ${\mathfrak {s}} \in {\mathfrak {B}}(G_{m+i})$ (c.f. Lemma 5.8). This is done by realizing $\omega \times \Pi _i$ (resp. $\Pi _i \times \omega$) as the bottom layer of the Bernstein–Zelevinsky filtration of some irreducible module $\pi$ of $G_{m+i+1}$ and then embed the submodules of $\omega \times \Pi _i$ (resp. $\Pi _i \times \omega$) to $\pi$.

Theorem 6.2

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. If i is not the level of $\pi$, then $\pi ^{[i]}$ and ${}^{[i]}\pi$ do not have an isomorphic irreducible quotient, and also do not have an isomorphic irreducible submodule whenever the two derivatives are not zero.

The proof of Theorem 6.2 will be carried out in Sect. 7. Note that the converse of the above theorem is also true, which follows directly from the well-known highest derivative due to Zelevinsky [35, Theorem 8.1].

6.2 Non-zero Bernstein components

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. In order to give an explicit parametrization of indecomposable components of $\pi |_{G_n}$, we also have to determine when $\pi _{{\mathfrak {s}}}\ne 0$ for ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$. Indeed this can be done as follows. Write $\pi \cong \langle {\mathfrak {m}} \rangle$ for a multisegment ${\mathfrak {m}}=\left\{ \Delta _1, \ldots , \Delta _k \right\}$. Let $\pi '$ be the (right) highest derivative of $\pi$. Then we obtain a multiset

$$\begin{aligned} {\mathrm {csupp}}(\pi ')=( \rho _1, \ldots , \rho _p ) . \end{aligned}$$

This multiset determines a cuspidal $G_{k_1}\times \ldots \times G_{k_p}$-representation $\rho _1\boxtimes \ldots \boxtimes \rho _p$. Now we pick positive integers $k_{p+1}, \ldots , k_q$ such that $k_1+\ldots +k_q=n$, and pick cuspidal representations $\rho _{p+1}, \ldots , \rho _q$ of $G_{k_{p+1}}, \ldots , G_{k_q}$ respectively. Then for the inertial equivalence class

$$\begin{aligned} {\mathfrak {s}}=[G_{k_1}\times \ldots \times G_{k_q}, \rho _1\boxtimes \ldots \boxtimes \rho _q], \end{aligned}$$

we have that $\pi _{{\mathfrak {s}}}\ne 0$, which follows from that the bottom Bernstein–Zelevinsky layer $( \pi ^{[i^*]}\times \Pi _{i^*-1})_{{\mathfrak {s}}} \subset \pi _{{\mathfrak {s}}}$ is non-zero. Here $i^*$ is the level of $\pi$.

Indeed for any ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$ with $\pi _{{\mathfrak {s}}} \ne 0$, ${\mathfrak {s}}$ arises in the above way. To see this, we need the following lemma:

Lemma 6.3

Let $\pi \in {\mathrm {Irr}}(G_{n+1})$. Let $i^*$ be the level of $\pi$. Then, for any $i\le i^*$ with $\pi ^{[i]}\ne 0$ and for any $\omega \in {\mathrm {JH}}(\pi ^{[i]})$, ${\mathrm {csupp}}(\pi ^{[i^*]}) \subset {\mathrm {csupp}}(\omega )$ (counting multiplicities).

Proof

Let ${\mathfrak {m}} \in {\mathrm {Mult}}$ such that $\pi \cong \langle {\mathfrak {m}} \rangle$. Now by definition, we have that

$$\begin{aligned} \langle {\mathfrak {m}} \rangle \hookrightarrow \zeta ({\mathfrak {m}}) \end{aligned}$$

and so $\langle {\mathfrak {m}} \rangle ^{[i]} \hookrightarrow \zeta ({\mathfrak {m}})^{[i]}$.

For any segment $\Delta$, set $\Delta ^{[0]}=\nu ^{1/2}\Delta$ and set $\Delta ^{[-]}=\nu ^{1/2}\Delta ^{-}$. We also set ${\mathfrak {m}}=\left\{ \Delta _1, \ldots , \Delta _k \right\}$ and set

$$\begin{aligned} {\mathcal {M}}=\left\{ \left\{ \Delta _1^{\#},\ldots , \Delta _k^{\#} \right\} : \#=[0],[-] \right\} . \end{aligned}$$

Now the geometric lemma gives a filtration on $\zeta ({\mathfrak {m}})^{[i]}$ whose successive quotients are isomorphic to $\zeta ({\mathfrak {n}})$ for some ${\mathfrak {n}} \in {\mathcal {M}}$ (see Lemma 7.3 below). Since $\langle {\mathfrak {m}} \rangle ^{[i]} \hookrightarrow \zeta ({\mathfrak {m}})^{[i]}$ as discussed before, any $\omega \in {\mathrm {JH}}(\langle {\mathfrak {m}} \rangle ^{[i]})$ is a composition factor of $\zeta ({\mathfrak {n}})$ for some ${\mathfrak {n}} \in {\mathcal {M}}$. Hence, ${\mathrm {csupp}}(\omega ) =\cup _{\Delta \in {\mathfrak {n}}}\Delta$ (counting multiplicities).

On the other hand, $\pi ^{[i^*]}=\langle \left\{ \Delta _1^{[-]}, \ldots , \Delta _k^{[-]} \right\} \rangle$. Hence ${\mathrm {csupp}}(\pi ^{[i^*]}) \subset {\mathrm {csupp}}(\omega )$ for any $\omega \in {\mathrm {JH}}(\pi ^{[i]})$. $\square$

Now we go back to consider that ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$ with $\pi _{{\mathfrak {s}}} \ne 0$. By the Bernstein–Zelevinsky filtration,

$$\begin{aligned} \pi _{{\mathfrak {s}}} \ne 0 \Rightarrow (\pi ^{[i]}\times \Pi _{i-1})_{{\mathfrak {s}}} \ne 0 \end{aligned}$$

for some $i \le i^*$. Now, as a similar manner to what we did for the bottom layer above, we could determine (abstractly) all possible ${\mathfrak {s}}' \in {\mathfrak {B}}(G_n)$ with $( \pi ^{[i]}\times \Pi _{i-1})_{{\mathfrak {s}}'}\ne 0$. Then by Lemma 6.3, one sees that,

$$\begin{aligned} (\pi ^{[i]} \times \Pi _{i-1})_{{\mathfrak {s}}} \ne 0 \Rightarrow ( \pi ^{[i^*]}\boxtimes \Pi _{i^*-1})_{{\mathfrak {s}}} \ne 0 . \end{aligned}$$

Remark 6.4

We use notations in Sect. 5.2. Fix ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$. Let $\pi \in {\mathrm {Irr}}(G_{n+1})$ with $\pi _{{\mathfrak {s}}}\ne 0$. We consider the set of points $\sigma \in {\mathfrak {X}}_{{\mathfrak {s}}}$ such that the corresponding maximal ideal ${\mathcal {J}}^{\sigma }$ satisfies $\pi /{\mathcal {J}}^{\sigma }\pi \ne 0$. The above discussion with Lemma 4.4 gives a description on such set, and in particular, such set forms an irreducible closed subvariety in ${\mathfrak {X}}_{{\mathfrak {s}}}$.

Indeed, for any submodule $\pi '$ of $\pi |_{G_n}$, the analogous set for $\pi '$ defines precisely the same variety as $\pi |_{G_n}$. This can be seen by refining to a submodule $\omega$ of $\pi '$ that embeds to the bottom layer $\pi ^{[i^*]}\times \Pi _{i^*-1}$ (as what we saw in the proof of Theorem 6.1). Then one shows, as in the proof of Proposition 5.6, that $\omega _{N_{i^*-1}}/{\mathcal {I}}\omega _{N_{i^*-1}}\ne 0$ for any ideal ${\mathcal {I}}$ of the form ${\mathcal {J}}_{\pi ^{[i^*]}}\otimes 1+1 \otimes {\mathcal {J}}_{\tau }$. (Here $\tau \in {\mathrm {Irr}}(G_{i^*-1})$ without any conditions.)

6.3 Indecomposability of Zelevinsky induced modules

Let ${\mathfrak {m}} \in {\mathrm {Mult}}$. In [7], it is shown that the restriction of $\zeta ({\mathfrak {m}})$ to the mirabolic subgroup is strongly indecomposable. One may expect the following conjecture, which is a stronger statement of Theorem 6.1:

Conjecture 6.5

Let ${\mathfrak {m}} \in {\mathrm {Mult}}$ with sum of absolute lengths of all its segments equal to $n+1$. Then any Bernstein component of $\zeta ({\mathfrak {m}})|_{G_n}$ is strongly indecomposable.

A variation of the above conjecture is to replace $\zeta ({\mathfrak {m}})$ in Conjecture 6.5 by $\lambda ({\mathfrak {m}})$. However, it is not true in general for the smooth dual $\zeta ({\mathfrak {m}})^{\vee }|_{G_n}$. An example is ${\mathfrak {m}}=\left\{ [\nu ^{-1/2}],[\nu ^{1/2}] \right\}$. In this case, the short exact sequence:

$$\begin{aligned} 0 \rightarrow \langle \Delta \rangle \rightarrow \zeta ({\mathfrak {m}})^{\vee }|_{G_1} \rightarrow {\mathrm {St}}(\Delta ) \rightarrow 0 , \end{aligned}$$

where $\Delta =[\nu ^{-1/2}, \nu ^{1/2}]$, gives a split sequence since ${\mathrm {St}}(\Delta )|_{G_1}$ is projective (see Theorem 3.7). Hence the Iwahori component of $\zeta ({\mathfrak {m}})^{\vee }|_{G_1}$ is not indecomposable.

7 Asymmetric property of left and right derivatives

We are going to prove Theorem 6.2 in this section. The idea lies in two simple cases. The first one is a generic representation. Since an irreducible generic representation is isomorphic to $\lambda ({\mathfrak {m}}) \cong {\mathrm {St}}({\mathfrak {m}})$ for ${\mathfrak {m}} \in {\mathrm {Mult}}$ (with the property that any two segments in ${\mathfrak {m}}$ are unlinked), a simple counting on cuspidal supports of derivatives can show Theorem 6.2 for that case. The second one is an irreducible representation whose Zelevinsky multisegment has all segments with relative length strictly greater than 1. In such case, one can narrow down the possibility of irreducible submodules of the derivatives via the embedding $\langle {\mathfrak {m}}\rangle ^{(i)} \hookrightarrow \zeta ({\mathfrak {m}})^{(i)}$ and ${}^{(i)}\langle {\mathfrak {m}} \rangle \hookrightarrow {}^{(i)}\zeta ({\mathfrak {m}})$, and use geometric lemma to compute the possible submodules of derivatives of $\zeta ({\mathfrak {m}})^{(i)}$ and ${}^{(i)}\zeta ({\mathfrak {m}})$. The combination of these two cases seems to require some extra work. The strategy is to use Speh representations, which can be viewed as a generalization of generalized Steinberg representations, and then apply Lemma 9.4 to obtain information on submodules.

7.1 Union–intersection operation

Let ${\mathfrak {m}}=\left\{ \Delta _1, \ldots , \Delta _r \right\}$. For two segments $\Delta$ and $\Delta '$ in ${\mathfrak {m}}$ which are linked, the process of replacing $\Delta$ and $\Delta '$ by $\Delta \cap \Delta '$ and $\Delta \cup \Delta '$ is called the union–intersection process. It follows from [35, Chapter 7] that the Zelevinsky multisegment of any irreducible composition factor in

$$\begin{aligned} \langle \Delta _1 \rangle \times \ldots \times \langle \Delta _r \rangle \end{aligned}$$

can be obtained by a chain of intersection-union process. For a positive integer l, define $N({\mathfrak {m}}, l)$ to be the number of segments in ${\mathfrak {m}}$ with relative length l.

Lemma 7.1

Let $k \ge 1$. Let ${\mathfrak {m}}_0, {\mathfrak {m}}_1, \ldots {\mathfrak {m}}_k \in {\mathrm {Mult}}$ such that for $i=1,\ldots , k$, each ${\mathfrak {m}}_i$ is obtained from ${\mathfrak {m}}_{i-1}$ by one-step of union–intersection operation. Suppose $\Delta _i, \Delta _i'$ are two segments in ${\mathfrak {m}}_{i-1}$ involved in the union–intersection operation to obtain ${\mathfrak {m}}_i$. Let $l_i$ be the relative length of $\Delta _i \cup \Delta _i'$. Then there exists l such that $l \ge l_i$ for all i, and

$$\begin{aligned} N({\mathfrak {m}}_k, l) > N({\mathfrak {m}}_0, l) \end{aligned}$$

and, for any $l' >l$,

$$\begin{aligned} N({\mathfrak {m}}_k, l') \ge N({\mathfrak {m}}_0, l') . \end{aligned}$$

Proof

We shall prove inductively on k. The basic case that $k=1$ can be proved by a similar argument that will be used to prove the inductive case, and so we omit the details. We assume the targeted statement is true for $X \ge 1$ number of union–intersection operations. Let ${\mathfrak {m}}_{X+1}$ be a new multisegment obtained from ${\mathfrak {m}}_0$ by $X+1$ number of union–intersection operations. Then we have a multisegment ${\mathfrak {m}}_X$ obtained from ${\mathfrak {m}}_0$ by X number of union–intersection operations and ${\mathfrak {m}}_{X+1}$ is obtained from ${\mathfrak {m}}_X$ by one union–intersection operation.

By inductive hypothesis, we can find a positive number $l_X$ such that

$$\begin{aligned} N({\mathfrak {m}}_X, l_X) > N({\mathfrak {m}}_0, l_X) , \end{aligned}$$

and for any $l' > l_X$,

$$\begin{aligned} N({\mathfrak {m}}_X, l') \ge N({\mathfrak {m}}_0, l') . \end{aligned}$$

Now let $\Delta _i, \Delta _j$ be the segments in ${\mathfrak {m}}_X$ involved in the union–intersection operation to obtain ${\mathfrak {m}}_{X+1}$. In particular, $\Delta _i$ and $\Delta _j$ are linked. Let $l_0$ be the relative length of $\Delta _i \cup \Delta _j$. If $l_0 \ge l_X$, set $l=l_0$, and otherwise set $l=l_X$. Now it is straightforward to check that such l satisfies the required properties. $\square$

7.2 Speh multisegments

Definition 7.2

Let $\Delta$ be a segment. Let

$$\begin{aligned} {\mathfrak {m}}(m,\Delta )= \left\{ \nu ^{-(m-1)/2}\Delta ,\nu ^{1-(m-1)/2}\Delta ,\ldots , \nu ^{(m-1)/2}\Delta \right\} . \end{aligned}$$

We shall call ${\mathfrak {m}}(m,\Delta )$ to be a Speh multisegment. Define

$$\begin{aligned} u(m,\Delta ) = \langle {\mathfrak {m}}(m,\Delta ) \rangle , \end{aligned}$$

which will be called a Speh representation. In the literature, it is sometimes called an essentially Speh representation, reflecting that it is not necessarily unitary. Denote by $L({\mathfrak {m}}(m,\Delta ))$ the relative length of $\Delta$.

We similarly define

$$\begin{aligned} u_r(m,i,\Delta )=\langle \nu ^{-(m-1)/2}\Delta ^-,\ldots , \nu ^{-(m-2i+1)/2}\Delta ^-, \nu ^{-(m-2i-1)/2}\Delta , \ldots , \nu ^{(m-1)/2}\Delta \rangle , \end{aligned}$$

and

$$\begin{aligned} u_l(m,i,\Delta )=\langle \nu ^{-(m-1)/2}\Delta , \ldots , ,\nu ^{(m-2i-1)/2}\Delta , \nu ^{(m-2i+1)/2}({}^-\Delta ), \ldots , \nu ^{(m-1)/2}({}^-\Delta ) \rangle . \end{aligned}$$

Let $l=n(\rho )$. It follows from [23, 33] (also see [13]) that

$$\begin{aligned} u(m,\Delta )^{(li)} \cong u_r(m, i, \Delta ) , \end{aligned}$$

(7.24)

and $u(m,\Delta )^{(k)}$ is zero if l does not divide k. Applying (2.3), we have that

$$\begin{aligned} {}^{(li)}u(m,\Delta ) \cong u_l(m,i, \Delta ) , \end{aligned}$$

(7.25)

and ${}^{(k)}u(m,\Delta ) =0$ if l does not divide k.

7.3 Notations for multisegments

For a multisegment ${\mathfrak {m}}=\left\{ \Delta _1, \ldots , \Delta _r \right\}$, define

$$\begin{aligned} {\mathfrak {m}}^{(i_1, \ldots , i_r)}= & {} \left\{ \Delta _1^{(i_1)}, \ldots , \Delta _r^{(i_r)}\right\} , \quad {\mathfrak {m}}^{(i)}= \left\{ {\mathfrak {m}}^{(i_1, \ldots , i_r)} : i_1+\ldots +i_r=i \right\} ,\\ {}^{(i_1, \ldots , i_r)}{\mathfrak {m}}= & {} \left\{ {}^{(i_1)}\Delta _1, \ldots , {}^{(i_r)}\Delta _r \right\} , \quad {}^{(i)}{\mathfrak {m}} =\left\{ {}^{(i_1, \ldots , i_r)}{\mathfrak {m}} : i_1+\ldots +i_r=i \right\} . \end{aligned}$$

We shall need the following lemma:

Lemma 7.3

Let ${\mathfrak {m}} \in {\mathrm {Mult}}$. Then $\zeta ({\mathfrak {m}})^{(i)}$ (resp. ${}^{(i)}\zeta ({\mathfrak {m}})$) admits a filtration whose successive subquotients are isomorphic to $\zeta ({\mathfrak {n}})$ for some ${\mathfrak {n}} \in {\mathfrak {m}}^{(i)}$ (resp. ${\mathfrak {n}} \in {}^{(i)}{\mathfrak {m}}$).

Proof

Write ${\mathfrak {m}}=\left\{ \Delta _1, \ldots , \Delta _r \right\}$. We shall assume that for $i<j$, $b(\Delta _i) \not < b(\Delta _j)$ and if $b(\Delta _i) \cong b(\Delta _j)$, then $a(\Delta _j) < a(\Delta _i)$. By definition,

$$\begin{aligned} \zeta ({\mathfrak {m}}) \cong \langle \Delta _1 \rangle \times \ldots \times \langle \Delta _r \rangle . \end{aligned}$$

By (2.4), we have that $\zeta ({\mathfrak {m}})^{(i)}$ admits a filtration whose successive quotients are:

$$\begin{aligned} \langle \Delta _1 \rangle ^{(i_1)} \times \ldots \times \langle \Delta _r \rangle ^{(i_r)} \end{aligned}$$

for $i_1+\ldots +i_r =i$. Now the lemma follows from that the product $\langle \Delta _1 \rangle ^{(i_1)} \times \ldots \times \langle \Delta _r \rangle ^{(i_r)}$ is isomorphic to $\zeta ({\mathfrak {n}})$ for ${\mathfrak {n}}=\left\{ \Delta _1^{(i_1)}, \ldots , \Delta _r^{(i_r)} \right\}$. We remark that in order to check the isomorphism, we have to use (2.1) and our choice of labelling for ${\mathfrak {m}}$. $\square$

7.4 Proof of Theorem 6.2

By Lemma 2.6, it suffices to prove the statement for submodules of the derivatives.

Let ${\mathfrak {m}}$ be the Zelevinsky multisegment with $\pi \cong \langle {\mathfrak {m}} \rangle$. We shall assume that any cuspidal representation in each segment of ${\mathfrak {m}}$ is an unramified twist of a fixed cuspidal representation $\rho$, i.e.

$$\begin{aligned} {\mathrm {csupp}}(\langle {\mathfrak {m}} \rangle ) \subset \left\{ \nu ^c \rho : c\in {\mathbb {C}} \right\} . \end{aligned}$$

We shall prove Theorem 6.2 for such $\pi$. The general case follows from this by writing an irreducible representation as a product of irreducible representations of such specific form.

Step 1: First approximation using Lemma 7.3 Let $\pi '$ be a common isomorphic irreducible quotient of $\pi {}^{[i]}$ and ${}^{[i]}\pi$. (Here we assume that $\pi {}^{[i]}$ and ${}^{[i]}\pi$ are non-zero.) Recall that we have that

$$\begin{aligned} \pi \hookrightarrow \zeta ({\mathfrak {m}}) . \end{aligned}$$

Since taking derivatives is an exact functor, $\nu ^{1/2} \cdot \pi ^{(i)}$ embeds to $\nu ^{1/2}\cdot \zeta ({\mathfrak {m}})^{(i)}$ and so does $\pi '$.

By Lemma 7.3, there is a filtration on $\zeta ({\mathfrak {m}})^{(i)}$ given by $\zeta ( {\mathfrak {m}}^{(i_1,\ldots , i_r)} )$ for $i_1+\ldots +i_r=i$, where $i_k=0$ or $n(\rho )$. Then $\pi '$ is isomorphic to the unique submodule of $\nu ^{1/2}\zeta ({\mathfrak {m}}^{(i_1',\ldots , i_r')})$ for some $(i_1',\ldots , i_r')$ and so $\pi ' \cong \nu ^{1/2}\langle {\mathfrak {m}}^{(i_1', \ldots , i_r')} \rangle$. Similarly, $\pi '$ is isomorphic to $\nu ^{-1/2} \langle {}^{(j_1',\ldots , j_r')}{\mathfrak {m}} \rangle$ for some $j_1'+\ldots +j_r'=i$.

Suppose i is not the level of $\pi$. Then there exists at least one $i_{k}'=0$ and at least one $j_k'=0$. Among all those segment $\Delta _k$ with either $i_k'=0$ or $j_k'=0$, we shall choose $\Delta _{k^*}$ to have the largest relative (and absolute) length. Denote the relative length of $\Delta _{k^*}$ by L.

Step 2: Second approximation using Lemma 9.4

We write ${\mathfrak {m}}$ as the sum of Speh multisegments

$$\begin{aligned} {\mathfrak {m}}= {\mathfrak {m}}_1'+\ldots + {\mathfrak {m}}_s ' \end{aligned}$$

(7.26)

satisfying properties in Proposition 9.3.

Let

$$\begin{aligned} {\mathfrak {m}}_1 , \ldots , {\mathfrak {m}}_r \end{aligned}$$

be all the Speh multisegments appearing in the sum (7.26) such that $L({\mathfrak {m}}_k)=L$. For each ${\mathfrak {m}}_k$, write ${\mathfrak {m}}_k={\mathfrak {m}}(m_k, \Delta _k)$ and define $b({\mathfrak {m}}_k)=\nu ^{(m_k-1)/2}b(\Delta _k)$. We shall label ${\mathfrak {m}}_k$ in the way that $b({\mathfrak {m}}_k) \not < b({\mathfrak {m}}_l)$ for $k<l$. Furthermore, the labelling satisfies the properties that

($\Diamond$):

for any ${\mathfrak {m}}_p$ and $p<q$, ${\mathfrak {m}}_p +\Delta$ is not a Speh multisegment for any $\Delta \in {\mathfrak {m}}_q$.

($\Diamond \Diamond$):

if ${\mathfrak {m}}_p \cap {\mathfrak {m}}_q \ne \emptyset$ and $p \le q$, then ${\mathfrak {m}}_q \subset {\mathfrak {m}}_p$.

Let ${\mathfrak {n}}_1$ be the collection of all segments $\Delta '$ in ${\mathfrak {m}} {\setminus } ({\mathfrak {m}}_1+\ldots +{\mathfrak {m}}_r)$ which satisfies either (1) $b({\mathfrak {m}}_1) <b(\Delta ')$ or (2) $b({\mathfrak {m}}_1) \cong b(\Delta ')$. Define inductively that ${\mathfrak {n}}_k$ is the collection of all segments $\Delta '$ in ${\mathfrak {m}} {\setminus } ({\mathfrak {m}}_1+\ldots +{\mathfrak {m}}_{r}+{\mathfrak {n}}_1+\ldots {\mathfrak {n}}_{k-1})$ that satisfies the property that either (1) $b({\mathfrak {m}}_k)< b(\Delta ')$, or (2) $b({\mathfrak {m}}_k) \cong b(\Delta ')$. (It is possible that some ${\mathfrak {n}}_k$ is empty.)

By Lemma 9.4, we have a series of embeddings:

$$\begin{aligned} \langle {\mathfrak {m}} \rangle \hookrightarrow&\zeta ({\mathfrak {n}}_1)\times \langle {\mathfrak {m}}_1 \rangle \times \ldots \zeta ({\mathfrak {n}}_r) \times \langle {\mathfrak {m}}_r \rangle \times \zeta ( {\mathfrak {n}}_{r+1} )\\ \hookrightarrow&\ldots \\ \hookrightarrow&\zeta ({\mathfrak {n}}_1) \times \langle {\mathfrak {m}}_1 \rangle \times \zeta ({\mathfrak {n}}_2) \times \langle {\mathfrak {m}}_2 \rangle \times \zeta ({\mathfrak {n}}_3+\ldots +{\mathfrak {n}}_r+{\mathfrak {m}}_3+{\mathfrak {m}}_r+{\mathfrak {n}}_{r+1}) \\ \hookrightarrow&\zeta ({\mathfrak {n}}_1) \times \langle {\mathfrak {m}}_1 \rangle \times \zeta ({\mathfrak {n}}_2+\ldots +{\mathfrak {n}}_r+{\mathfrak {m}}_2+{\mathfrak {m}}_r+{\mathfrak {n}}_{r+1}) \\ \hookrightarrow&\zeta ({\mathfrak {n}}_1 +\ldots {\mathfrak {n}}_{r+1}+{\mathfrak {m}}_1+\ldots +{\mathfrak {m}}_r)=\zeta ({\mathfrak {m}}) \end{aligned}$$

For simplicity, define, for $k\ge 0$,

$$\begin{aligned} \lambda _k&=\zeta ({\mathfrak {n}}_1) \times \langle {\mathfrak {m}}_1 \rangle \times \ldots \times \zeta ({\mathfrak {n}}_{k})\times \langle {\mathfrak {m}}_{k} \rangle \times \zeta ({\mathfrak {n}}_{k+1}+\ldots \\&\qquad +{\mathfrak {n}}_{r+1} +{\mathfrak {m}}_{k+1}+\ldots +{\mathfrak {m}}_{r}) \end{aligned}$$

Step 3: Approximation on right derivatives Then for each k, we again have an embedding:

$$\begin{aligned} \pi ' \hookrightarrow \nu ^{1/2}\cdot \pi ^{(i)} \hookrightarrow \nu ^{1/2} \cdot \lambda _k^{(i)} . \end{aligned}$$

As $\lambda _k$ is a product of representations, we again have a filtration on $\lambda _k^{(i)}$ by (2.4). This gives that $\pi '$ embeds to a successive quotient of the filtration:

$$\begin{aligned} \pi ' \hookrightarrow \nu ^{1/2}\cdot ( \zeta ({\mathfrak {n}}_1)^{(p^k_1)} \times \langle {\mathfrak {m}}_1 \rangle ^{(q^k_1)} \times \ldots \times \zeta ({\mathfrak {n}}_{k})^{(p^k_{k})} \times \langle {\mathfrak {m}}_k \rangle ^{(q^k_k)} \times \zeta ({\mathfrak {o}}_{k+1})^{(s^k)}) \end{aligned}$$

with $p^k_1+\ldots +p^k_k+q_1^k+\ldots +q^k_k+s^k=i$,

$$\begin{aligned} {\mathfrak {o}}_{k+1} = {\mathfrak {n}}_{k+1}+\ldots +{\mathfrak {n}}_{r+1}+{\mathfrak {m}}_{k+1}+\ldots +{\mathfrak {m}}_r .\end{aligned}$$

Lemma 7.4

• (1) $i_k= 0$ for some $\Delta _k$ in ${\mathfrak {m}}$ with relative length L (see the choice of L in Step 1).

• (2) Following above notations, there exists a $k' \ge 1$ such that at least one of $q^{k'}_l$ is not equal to the level of $\langle {\mathfrak {m}}_l \rangle$.

Remark 7.5

Similarly, we have $j_k=0$ for some $\Delta _k$ in ${\mathfrak {m}}$ with relative length L.

Proof

We first assume (1) to prove (2). It suffices to show that when $k'=r$, at least one of $q^{k'}_l$ is not equal to the level of $\langle {\mathfrak {m}}_l \rangle$. Suppose not. Let $\langle {\mathfrak {m}}_i \rangle ^-$ be the highest derivative of $\langle {\mathfrak {m}}_i \rangle$. Then we obtain an embedding:

$$\begin{aligned} \pi ' \hookrightarrow \nu ^{1/2}\cdot (\zeta ({\mathfrak {n}}_1)^{(p_1^k)}\times \langle {\mathfrak {m}}_1 \rangle ^- \times \ldots \times \zeta ({\mathfrak {n}}_r)^{(p_r^k)}\times \langle {\mathfrak {m}}_r \rangle ^- \times \zeta ({\mathfrak {n}}_{r+1})^{(s_r)}) . \end{aligned}$$

Now set ${\mathfrak {m}}_i^-$ to be the multisegment such that $\langle {\mathfrak {m}}_i^- \rangle \cong \langle {\mathfrak {m}}_i \rangle ^-$. By definitions, ${\mathfrak {m}}_i^-$ is still a Speh multisegment. Now, by Lemma 7.3, we have that

$$\begin{aligned} \pi ' \hookrightarrow \nu ^{1/2}\cdot (\zeta (\widetilde{{\mathfrak {n}}}_1)\times \langle {\mathfrak {m}}_1^- \rangle \times \ldots \times \zeta (\widetilde{{\mathfrak {n}}}_r)\times \langle {\mathfrak {m}}_r^- \rangle \times \zeta (\widetilde{{\mathfrak {n}}}_{r+1})) , \end{aligned}$$

where $\widetilde{{\mathfrak {n}}}_a \in {\mathfrak {n}}_a^{(p_i^k)}$. From the construction of ${\mathfrak {n}}_a$, we can check that those $\widetilde{{\mathfrak {n}}}_i$ satisfies the conditions in Lemma 9.4. Hence, by Lemma 9.4,

$$\begin{aligned} \pi ' \cong \nu ^{1/2} \cdot \langle \widetilde{{\mathfrak {n}}}_1+{\mathfrak {m}}_1^- +\ldots +\widetilde{{\mathfrak {n}}}_k+{\mathfrak {m}}_k^-+\widetilde{{\mathfrak {n}}}_{k+1} \rangle . \end{aligned}$$

Now we write ${\mathfrak {n}}_a=\left\{ \Delta _{a,1}, \ldots , \Delta _{a,r(a)} \right\}$ for each a. Then we have that

$$\begin{aligned} \widetilde{{\mathfrak {n}}}_a = \left\{ \Delta _{a,1}^{(p_{a,1})}, \ldots , \Delta _{a,r(a)}^{(p_{a,r(a)})} \right\} , \end{aligned}$$

where each $p_{a,k}=0$ or $n(\rho )$.

Claim For any segment $\Delta _{a,k}$ with relative length at least $L+1$, $p_{a,k}=n(\rho )$.

Proof of Claim Recall that we also have $\pi ' \cong \nu ^{1/2} \cdot \langle {\mathfrak {m}}^{(i_1, \ldots , i_r)} \rangle$ in Step 1. Hence,

$$\begin{aligned} {\mathfrak {m}}^{(i_1, \ldots , i_r)} = \widetilde{{\mathfrak {n}}}_1+{\mathfrak {m}}_1^- +\ldots +\widetilde{{\mathfrak {n}}}_k+{\mathfrak {m}}_k^-+\widetilde{{\mathfrak {n}}}_{k+1} . \end{aligned}$$

(7.27)

Let $L^{**}$ be the largest relative length among all the relative length of segments in ${\mathfrak {m}}$. If $L^{**}=L$, then there is nothing to prove (for the claim). We assume $L^{**}>L$. In such case, there is no segment in ${\mathfrak {m}}^{(i_1, \ldots , i_r)}$ has relative length $L^{**}$. This implies that all segments $\Delta _{a,k}$ with relative length $L^{**}$ have $p_{a,k}=n(\rho )$. With the use of (7.27) and

$$\begin{aligned} {\mathfrak {m}}={\mathfrak {n}}_1 +{\mathfrak {m}}_1^- +\ldots +{\mathfrak {n}}_k+{\mathfrak {m}}_k^-+{\mathfrak {n}}_{k+1}, \end{aligned}$$

we can proceed to the length $L^{**}-1$ in a similar fashion. Inductively, we obtain the claim.

Now we go back to the proof of the lemma. By using the claim (and the definitions of ${\mathfrak {m}}_i$),

$$\begin{aligned} N(\widetilde{{\mathfrak {n}}}_1 +{\mathfrak {m}}_1^- +\ldots +\widetilde{{\mathfrak {n}}}_k+{\mathfrak {m}}_k^-+\widetilde{{\mathfrak {n}}}_{k+1},L) = N({\mathfrak {m}}, L+1). \end{aligned}$$

On the other hand,

$$\begin{aligned} N({\mathfrak {m}}^{(i_1,\ldots , i_r)}, L)\ge N({\mathfrak {m}}, L+1)+1 \end{aligned}$$

by using the hypothesis in the lemma. However, ${\mathfrak {m}}=\widetilde{{\mathfrak {n}}}_1 +{\mathfrak {m}}_1^- +\ldots +\widetilde{{\mathfrak {n}}}_k+{\mathfrak {m}}_k^-+\widetilde{{\mathfrak {n}}}_{k+1}$ and this gives a contradiction. This proves (2) modulo (1).

It remains to prove (1). From our choice of L, either

$$\begin{aligned} N({}^{(j_1, \ldots , j_r)}{\mathfrak {m}},L) \ge N({\mathfrak {m}}, L+1)+1 \end{aligned}$$

or

$$\begin{aligned} N({\mathfrak {m}}^{(i_1, \ldots , i_r)}, L) \ge N({\mathfrak {m}}, L+1)+1 . \end{aligned}$$

however, since $\nu ^{1/2}\cdot {\mathfrak {m}}^{(i_1, \ldots , i_r)} \cong \nu ^{-1/2}\cdot {}^{(j_1, \ldots , j_r)}{\mathfrak {m}}$, we have that

$$\begin{aligned} N({\mathfrak {m}}^{(i_1, \ldots , i_r)}, L)=N({}^{(j_1,\ldots , j_r)}{\mathfrak {m}}, L). \end{aligned}$$

Thus, we must have that $N({\mathfrak {m}}^{(i_1, \ldots , i_r)}, L) \ge N({\mathfrak {m}}, L+1)+1$. But this forces (1). $\square$

Now let ${\bar{k}}$ be the smallest number such that at least one of $q^{{\bar{k}}}_l$ is not equal to the level of $\langle {\mathfrak {m}}_l \rangle$. Now we shall denote such l by $l^*$, and so ${\bar{k}}\ge l^*$.

We use similar strategy to further consider the filtrations on each ${\mathfrak {n}}_a$ by geometric lemma. For that we write, for each a,

$$\begin{aligned} {\mathfrak {n}}_a =\left\{ \Delta _{a,1} ,\ldots , \Delta _{a, r(a)} \right\} \end{aligned}$$

and

$$\begin{aligned} {\mathfrak {o}}_{{\bar{k}}+1}=\left\{ \Delta _{{\bar{k}}+1,1 }, \ldots , \Delta _{{\bar{k}}+1 , r({\bar{k}}+1) }\right\} . \end{aligned}$$

Here r(a) is an index counting the number of segments in ${\mathfrak {n}}_a$ depending on a.

Then we again have an embedding

$$\begin{aligned} \pi ' \hookrightarrow \nu ^{1/2} \cdot (\zeta (\widetilde{{\mathfrak {n}}}_1) \times \langle {\mathfrak {m}}_1 \rangle ^{(q_1^{{\bar{k}}})} \times \ldots \times \zeta (\widetilde{{\mathfrak {n}}}_{{\bar{k}}}) \times \langle {\mathfrak {m}}_{{{\bar{k}}}} \rangle ^{(q_{{{\bar{k}}}}^{{\bar{k}}})} \times \zeta ({\mathfrak {o}}_{{{\bar{k}}}+1})^{(s_{{{\bar{k}}}})} ), \end{aligned}$$

where, for $a=1,\ldots , {\bar{k}}$,

$$\begin{aligned} \widetilde{{\mathfrak {n}}}_a = \left\{ \Delta _{a,1}^{(p_{a,1})}, \ldots , \Delta _{a, r(a)}^{(p_{a,r(a)})} \right\} \in {\mathfrak {n}}_a^{(p_{a}^{{\bar{k}}})} \end{aligned}$$

with $p_{a,1}+\ldots +p_{a,r(a)}=p_a^{{\bar{k}}}$ and each $p_{a,b}=0$ or $n(\rho )$, and

$$\begin{aligned} \widetilde{{\mathfrak {o}}}_{{\bar{k}}+1}=\left\{ \Delta _{{\bar{k}}+1,1}^{(p_{{\bar{k}}+1,1})}, \ldots , \Delta _{{\bar{k}}+1,r({\bar{k}}+1)}^{(p_{{{\bar{k}}}+1,r({\bar{k}}+1)})} \right\} \in {\mathfrak {o}}_{{\bar{k}}+1}^{(s_{{\bar{k}}})} \end{aligned}$$

with $p_{{\bar{k}}+1,1}+\ldots +p_{{\bar{k}}+1, r({\bar{k}}+1)}=s_{{\bar{k}}}$, with each $p_{{\bar{k}}+1,b}=0$ or $n(\rho )$.

Step 4: Computing some indexes $p_{a,b}$ and approximating the number of special segments by union–intersection operations We claim (*) that if $\Delta _{a,b}$ has a relative length at least $L+1$, then $p_{a,b}=n(\rho )$. This is indeed similar to the proof of Lemma 7.4 and the main difference is that we do not have an analogous form of (7.27) (since we cannot apply Lemma 9.4). Instead, we can obtain this from Lemma 7.1 (and its proof of Lemma 7.1).

Now from our choice of ${\bar{k}}$, we have that $\langle {\mathfrak {m}}_{l^*} \rangle ^{(q^{{{\bar{k}}}}_{l^*})}$ is not a Speh representation. We can write

$$\begin{aligned} {\mathfrak {m}}_{l^*} =\left\{ \nu ^{-x+1}\Delta ^*, \ldots , \nu ^{-1}\Delta ^*, \Delta ^* \right\} \end{aligned}$$

for a certain $\Delta ^*$ with relative length L and some x. By ($\Diamond$), $\nu \Delta ^* \notin {\mathfrak {m}}_l$ for any $l \ge l^*$ from our labelling on ${\mathfrak {m}}_l$. Rephrasing the statement, we get the following statement:

$$\begin{aligned} (^{**}) \,{\nu ^{1/2}\Delta ^* \notin \nu ^{-1/2}{\mathfrak {m}}_l\, \textit{ for any }l \ge l^* . } \end{aligned}$$

Now with (*), we have that $\pi '$ is a composition factor of $\zeta ({\mathfrak {m}}''')$, where ${\mathfrak {m}}'''$ is some multisegment containing all the segments $\Delta ^-$ (counting multiplicities) with $\Delta$ in ${\mathfrak {m}}$ that has relative length at least $L+1$ and containing an additional segment $\nu ^{1/2}\Delta ^*$ (from $\langle {\mathfrak {m}}_{l^*} \rangle ^{(q^{{{\bar{k}}}}_{l^*})}$ by Sect. 7.2), and we shall call the former segments (i.e. the segment in the form of $\Delta ^-$) to be special for convenience.

We can apply the intersection-union process to obtain the Zelevinsky multisegment for $\pi '$ from ${\mathfrak {m}}'''$. However in each step of the process, any one of the two segments involved in the intersection-union cannot be special. Otherwise, by Lemma 7.1, there exists $l \ge L+1$ such that the number of segments in the resulting multisegment with relative length l is more than the number of segments in ${\mathfrak {m}}^{(i_1,\ldots , i_r)}$. (Here we also used that $L({\mathfrak {m}}^{(i_1,\ldots , i_r)}, l)=L({\mathfrak {m}}''',l)$ for any $l\ge L+1$.) Hence we obtain the following:

(***) the number of segments $\Delta$ in $\nu ^{1/2}\cdot {\mathfrak {m}}^{(i_1,\ldots , i_r)}$ such that $\nu ^{1/2}\Delta ^* \subset \Delta$ is at least equal to one plus the number special segments in ${\mathfrak {m}}'''$ satisfying the same property

Step 5: Approximation on left derivatives Now we come to the final part of the proof. We now consider $\nu ^{-1/2}\cdot {}^{(i)}\pi$. Following the strategy for right derivatives, we have that for each k

$$\begin{aligned} \pi ' \hookrightarrow \nu ^{-1/2}\cdot {}^{(i)}\pi \hookrightarrow \nu ^{-1/2}\cdot {}^{(i)}\lambda _k . \end{aligned}$$

This gives that $\pi '$

$$\begin{aligned} \pi ' \hookrightarrow \nu ^{-1/2} \cdot ({}^{(u^k_1)}\zeta ({\mathfrak {n}}_1) \times {}^{(v^k_1)} \langle {\mathfrak {m}}_1 \rangle \times \ldots \times {}^{(u^k_{k})}\zeta ({\mathfrak {n}}_{k}) \times {}^{(v^k_{k})}\zeta ({\mathfrak {m}}_{k})\times {}^{(w^{k})}\zeta ({\mathfrak {o}}_{k+1})) \end{aligned}$$

with $u^k_1+\ldots +u^k_k+v^k_1+\ldots +v^k_k+w^k=i$. Let ${\widetilde{k}}$ be the smallest integer such that at least one of $v^{{\widetilde{k}}}_l$ is not equal to the level of $\langle {\mathfrak {m}}_l \rangle$. (For the existence of ${\widetilde{k}}$, see Lemma 7.4.) We shall denote such l by ${\widetilde{l}}$.

Step 6: the case that ${\widetilde{k}} \ge {\bar{k}}$ and determining the number of special segments

We firstly consider the case that ${\widetilde{k}} \ge {\bar{k}}$. We abbreviate $v_p=v_{p}^{{\bar{k}}-1}$. Similar to right derivatives, we have that

$$\begin{aligned} \pi ' \hookrightarrow \nu ^{-1/2} \cdot \zeta (\widehat{{\mathfrak {n}}}_1) \times {}^{(v_1)}\langle {\mathfrak {m}}_1 \rangle \times \ldots \times \zeta (\widehat{{\mathfrak {n}}}_{{\bar{k}}-1}) \times {}^{(v_{{{\bar{k}}}-1})}\langle {\mathfrak {m}}_{{{\bar{k}}}-1} \rangle \times {}^{(w_{{{\bar{k}}}-1})} \zeta (\widehat{{\mathfrak {o}}}_{{{\bar{k}}}}), \end{aligned}$$

where

$$\begin{aligned} \widehat{{\mathfrak {n}}}_a = \left\{ {}^{(u_{a,1})}\Delta _{a,1}, \ldots , {}^{(u_{a,r(a)})}\Delta _{a, r(a)} \right\} . \end{aligned}$$

with $u_{a,1}+\ldots +u_{a,r(a)}=u_a^{{\bar{k}}-1}$ and each $u_{a,b}=0$ or $n(\rho )$. Since we assume that ${\widetilde{k}}\ge {\bar{k}}$, we have that ${}^{(v_l)}\langle {\mathfrak {m}}_l \rangle$ is a highest derivative for any $l\le {\bar{k}}-1$ and so is a Speh representation, and we can apply Lemma 9.4(1). Hence the unique subrepresentation of

$$\begin{aligned} \nu ^{-1/2}\cdot \zeta (\widehat{{\mathfrak {n}}}_1) \times {}^{(v_1)}\langle {\mathfrak {m}}_1 \rangle \times \ldots \times \zeta (\widehat{{\mathfrak {n}}}_{{\bar{k}}-1}) \times {}^{(v_{{{\bar{k}}}-1})}\langle {\mathfrak {m}}_{{{\bar{k}}}-1} \rangle \times \zeta (\widehat{{\mathfrak {o}}}_{{\bar{k}}}) \end{aligned}$$

is isomorphic to

$$\begin{aligned} \nu ^{-1/2} \cdot \langle \widehat{{\mathfrak {n}}}_1+\widehat{{\mathfrak {m}}}_1+\ldots + \widetilde{{\mathfrak {n}}}_{{{\bar{k}}}-1}+\widehat{{\mathfrak {m}}}_{{{\bar{k}}}-1}+\widehat{{\mathfrak {o}}}_{{{\bar{k}}}} \rangle , \end{aligned}$$

(7.28)

where $\langle \widehat{{\mathfrak {m}}}_l \rangle ={}^{(v_l)}\langle {\mathfrak {m}}_l \rangle$. Similar to (*) for right derivatives (but the proof could be easier here), we obtain the analogous statement for those $\widehat{{\mathfrak {n}}}_a$. Now if

$$\begin{aligned} \Delta \in \nu ^{-1/2}( \widehat{{\mathfrak {n}}}_1+\widehat{{\mathfrak {m}}}_1+\ldots + \widetilde{{\mathfrak {n}}}_{{{\bar{k}}}-1}+\widehat{{\mathfrak {m}}}_{{{\bar{k}}}-1} +\widehat{{\mathfrak {o}}}_k )\end{aligned}$$

such that $\Delta =\nu ^{1/2}\Delta ^*$, then we must have that $\Delta =\nu ^{-1/2}\cdot {}^-\Delta _0$ or $\nu ^{-1/2}\cdot \Delta _0$ for some segment $\Delta _0$ in ${\mathfrak {m}}$. For the latter case to happen, we must have that $\Delta _0 \in {\mathfrak {m}}_l \subset {\mathfrak {o}}_{{\bar{k}}}$ with $l \ge {\bar{k}}\ge l^*$, but, by (**), the possibility $\Delta =\nu ^{-1/2}\Delta _0$ cannot happen. Thus we must have that $\Delta$ is a special in the same sense as the discussion in right derivatives. This concludes the following:

(****) The number of segments $\Delta$ in $\nu ^{-1/2}( \widehat{{\mathfrak {n}}}_1+\widehat{{\mathfrak {m}}}_1+\ldots + \widetilde{{\mathfrak {n}}}_{{{\bar{k}}}-1}+\widehat{{\mathfrak {m}}}_{{{\bar{k}}}-1} +\widehat{{\mathfrak {o}}}_{{{\bar{k}}}})$ with the property that $\nu ^{1/2}\Delta ^* \subset \Delta$ is equal to the number of special segments satisfying the same property.

Now the above statement contradicts to (***) since both Zelevinsky multisegments give an irreducible representation isomorphic to $\pi '$.

Remark 7.6

The isomorphism (7.28) is a key to obtain the equality in (****), in contrast with the inequality in (***).

Step 7: the case that ${\bar{k}} >{\widetilde{k}}$ Now the way to get contradiction in the case ${\bar{k}} \ge {\widetilde{k}}$ is similar by interchanging the role of left and right derivatives. We remark that to prove the analogue of (**), one uses ($\diamond$). And to obtain the similar isomorphism as (7.28), one needs to use Lemma 9.4(2). We can argue similarly to get an analogue of (***) and (****). Hence the only possibility that $\nu ^{1/2}\cdot \pi ^{(i)}$ and so $\nu ^{-1/2}\cdot {}^{(i)}\pi$ have an isomorphic irreducible quotient only if i is the level for $\pi$. This completes the proof of Theorem 6.2.

7.5 Branching law in opposite direction

Here is another consequence of the asymmetric property on the Hom-branching law in another direction:

Corollary 7.7

Let $\pi ' \in {\mathrm {Irr}}(G_n)$. Let $\pi \in {\mathrm {Irr}}(G_{n+1})$.

• (1) Suppose $\pi$ is not 1-dimensional. Then

  $$\begin{aligned} {\mathrm {Hom}}_{G_n}(\pi ', \pi |_{G_n}) = 0 . \end{aligned}$$

• (2) Suppose $\pi$ is 1-dimensional. Then

  $$\begin{aligned} {\mathrm {Hom}}_{G_n}(\pi ', \pi |_{G_n}) \ne 0 \end{aligned}$$

  if and only if $\pi '$ is also 1-dimensional and $\pi '=\pi |_{G_n}$.

Proof

The second statement is trivial. We consider the first one. Since $\pi$ is not one-dimensional, the level of $\pi$ is not 1 by Zelevinsky classification. By Theorem 6.2 and Proposition 2.5, $\pi ^{[1]}$ and ${}^{[1]}\pi$ have no common irreducible submodule if $\pi ^{(1)} \ne 0$ and ${}^{(1)}\pi \ne 0$. Then at least one of

$$\begin{aligned} {\mathrm {Hom}}_{G_n}(\pi ', \pi ^{[1]})=0 \quad \text{ or } \quad {\mathrm {Hom}}_{G_n}(\pi ', {}^{[1]}\pi )=0 . \end{aligned}$$

On the other hand, we have that for all $i \ge 2$, by Frobenius reciprocity,

$$\begin{aligned} {\mathrm {Hom}}_{G_n}(\pi ', {\mathrm {ind}}_{R_{n-i+1}}^{G_n} \pi ^{[i]} \boxtimes \psi _{i-1}) \cong {\mathrm {Hom}}_{G_{n+1-i}\times G_{i-1}}( \pi '_{N_{i-1}}, \pi ^{[i]} \boxtimes \Pi _{i-1}) , \end{aligned}$$

and

$$\begin{aligned} {\mathrm {Hom}}_{G_n}(\pi ', {\mathrm {ind}}_{R_{n-i+1}^-}^{G_n} {}^{[i]}\pi \boxtimes \psi _{i-1}) \cong {\mathrm {Hom}}_{G_{i-1} \times G_{n+1-i} }(\pi '_{N_{n+1-i}}, \Pi _{i-1}\boxtimes {}^{[i]}\pi ) \end{aligned}$$

However, in the last two equalities, the Hom in the right hand side is always zero (see Lemma 5.3). Now applying a Bernstein–Zelevinsky filtration to obtain the corollary. $\square$

8 Preserving indecomposability of Bernstein–Zelevinsky induction

We shall use the following criteria of indecomposable representations:

Lemma 8.1

Let G be a connected reductive p-adic group. Let $\pi$ be a smooth representation of G. The only idempotents in ${\mathrm {End}}_G(\pi )$ are 0 and the identity (up to automorphism) if and only if $\pi$ is indecomposable.

Proof

If $\pi$ is not indecomposable, then any projection to a direct summand gives a non-identity idempotent. On the other hand, if $\sigma \in {\mathrm {End}}_G(\pi )$ is a non-identity idempotent (i.e. $\sigma (\pi ) \ne \pi$), then $\pi \cong {\mathrm {im}}(\sigma ) \oplus {\mathrm {im}}(1-\sigma )$. $\square$

Lemma 8.2

Set $\Pi =\Pi _k$. Let ${\mathfrak {s}} \in {\mathfrak {B}}(G_k)$. Then ${}^{(k)}(\Pi _{{\mathfrak {s}}}) \cong {\mathfrak {Z}}_{{\mathfrak {s}}}$, where ${\mathfrak {Z}}_{{\mathfrak {s}}}$ is the Bernstein center of ${\mathfrak {R}}_{{\mathfrak {s}}}(G_k)$.

Proof

We have

$$\begin{aligned} {}^{(k)}(\Pi _{{\mathfrak {s}}}) \cong {\mathrm {Hom}}_{{\mathbb {C}}}({\mathbb {C}}, {}^{(k)}(\Pi _{{\mathfrak {s}}})) \cong {\mathrm {Hom}}_{G_k}(\Pi , \Pi _{{\mathfrak {s}}}) \cong {\mathrm {End}}_{G_k}(\Pi _{{\mathfrak {s}}}) , \end{aligned}$$

where the second isomorphism follows from the second adjointness theorem in [15], and the third isomorphism follows from the Bernstein decomposition. Now the lemma follows from that the final endomorphism is isomorphic to ${\mathfrak {Z}}_{{\mathfrak {s}}}$ by [8]. $\square$

Theorem 8.3

Let $\pi$ be an admissible indecomposable smooth representation of $G_{n-i}$. For each ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$, the Bernstein component $({\mathrm {ind}}_{R_{n-i}}^{G_n} \pi \boxtimes \psi _i)_{{\mathfrak {s}}}$ is indecomposable whenever it is nonzero.

Proof

We first prove the following:

Lemma 8.4

Let $\pi \in {\mathrm {Alg}}(G_{n-i})$ be admissible. Then, as algebras,

$$\begin{aligned} {\mathrm {End}}_{G_n}({\mathrm {ind}}_{R_{n-i}}^{G_n} \pi \boxtimes \psi _i ) \cong {\mathrm {End}}_{G_{n-i}}(\pi ) \otimes {\mathfrak {Z}}_i ,\end{aligned}$$

where ${\mathfrak {Z}}_i=\prod _{{\mathfrak {s}} \in {\mathfrak {B}}(G_{i})} {\mathfrak {Z}}_{{\mathfrak {s}}}$ is the Bernstein center.

Proof

Let ${\mathfrak {s}}_1, {\mathfrak {s}}_2 \in {\mathfrak {B}}(G_{n-i})$. Let $\pi _1, \pi _2$ be two admissible $G_{n-i}$ representations in ${\mathfrak {R}}_{{\mathfrak {s}}_1}(G_{n-i})$ and ${\mathfrak {R}}_{{\mathfrak {s}}_2}(G_{n-i})$ respectively. Then

$$\begin{aligned}&{\mathrm {Hom}}_{G_n}({\mathrm {ind}}_{R_{n-i}}^{G_n}\pi _1\boxtimes \psi _i, {\mathrm {ind}}_{R_{n-i}}^{G_n}\pi _2 \boxtimes \psi _i) \\&\quad \cong \prod _{{\mathfrak {s}} \in {\mathfrak {R}}(G_n)}{\mathrm {Hom}}_{G_n}({\mathrm {ind}}_{R_{n-i}}^{G_n} \pi _1 \boxtimes \psi _i, ({\mathrm {ind}}_{R_{n-i}}^{G_n}\pi _2 \boxtimes \psi _i)_{{\mathfrak {s}}}) \\&\quad \cong \prod _{{\mathfrak {s}} \in {\mathfrak {R}}(G_n)}{\mathrm {Hom}}_{G_n}(\pi _1, {}^{(i)}({\mathrm {ind}}_{R_{n-i}}^{G_n}\pi _2 \boxtimes \psi _i)_{{\mathfrak {s}}}) \quad \text{[16, } \text{ Lemma } \text{2.4] }\\&\quad \cong \prod _{{\mathfrak {u}} \in {\mathfrak {R}}(G_i)}{\mathrm {Hom}}_{G_{n-i}}(\pi _1, {}^{(i)}(\pi _2 \times (\Pi _i)_{{\mathfrak {u}}})) \end{aligned}$$

We remark that in order to apply the second adjointness theorem [15, Lemma 2.4], we need to get the first isomorphism. For the third isomorphisms, see discussions in Sect. 5.1.

Now we apply the geometric lemma on ${}^{(i)}(\pi _2 \times (\Pi _i)_{{\mathfrak {u}}})$ and we obtain a filtration on ${}^{(i)}(\pi _2 \times (\Pi _i)_{{\mathfrak {u}}})$ whose successive quotients are of the form

$$\begin{aligned} {}^{(j)}(\pi _2) \times {}^{(k)}((\Pi _i)_{{\mathfrak {u}}}) , \end{aligned}$$

where $j+k=i$. Using Lemma 5.3,

$$\begin{aligned} {\mathrm {Hom}}_{G_{n-i}}(\pi _1, {}^{(j)}(\pi _2) \times {}^{(k)}((\Pi _i)_{{\mathfrak {u}}})) =0 \end{aligned}$$

unless $j=0$ and $k=i$. Combining the above isomorphisms, we now have

$$\begin{aligned}&{\mathrm {Hom}}_{G_n}({\mathrm {ind}}_{R_{n-i}}^{G_n}\pi _1\boxtimes \psi _i, {\mathrm {ind}}_{R_{n-i}}^{G_n}\pi _2 \boxtimes \psi _i) \\&\quad \cong \prod _{{\mathfrak {u}} \in {\mathfrak {B}}(G_i)} {\mathrm {Hom}}_{G_{n-i}}(\pi _1, \pi _2 \otimes {}^{(i)}((\Pi _i)_{{\mathfrak {u}}})) \\&\quad \cong \prod _{{\mathfrak {u}} \in {\mathfrak {B}}(G_i)} {\mathrm {Hom}}_{G_{n-i}}(\pi _1, \pi _2\otimes {\mathfrak {Z}}_{{\mathfrak {u}}}) \\&\quad \cong \prod _{{\mathfrak {u}} \in {\mathfrak {B}}(G_i)} {\mathrm {Hom}}_{G_{n-i}}(\pi _1,\pi _2) \otimes {\mathfrak {Z}}_{{\mathfrak {u}}} \\&\quad \cong {\mathrm {Hom}}_{G_{n-i}}(\pi _1,\pi _2) \otimes {\mathfrak {Z}} \end{aligned}$$

We remark that the third and last isomorphisms require the admissibility of $\pi _1$ and $\pi _2$. The second isomorphism follows from Lemma 8.2. One further traces the above isomorphisms to see that they give an isomorphism preserving the algebra structure. Now we specialize to $\pi _1=\pi _2=\pi$ and we obtain the lemma when each of $\pi _1, \pi _2$ lying in one Bernstein component.

The general case follows by writing $\pi =\oplus _{{\mathfrak {s}}} \pi _{{\mathfrak {s}}}$. $\square$

Now the theorem follows from Lemmas 8.1 and 8.3 since the right hand side has only 0 and 1 as idempotents if and only if the left hand side does. $\square$

Let $M_n$ be the mirabolic subgroup in $G_{n+1}$, that is the subgroup containing all matrices with the last row of the form $(0,\ldots , 0,1)$. We have the following consequence restricted from $M_n$ to $G_n$. Here $G_n$ is viewed as the subgroup of $M_n$ via the embedding $g \mapsto {\mathrm {diag}}(g,1)$.

Corollary 8.5

Let $\pi$ be an irreducible smooth representation of $M_n$. Then for any ${\mathfrak {s}} \in {\mathfrak {B}}(G_n)$, $\pi _{{\mathfrak {s}}}$ is indecomposable.

Proof

This follows from [7, Corollary 3.5] and Theorem 8.3. $\square$

Appendices

Appendix: Speh representation approximation

1.1 Two lemmas

Some studies on parabolic induction can be found in [24, 32, 34]. We give a proof of the following specific cases, using the theory of derivatives. We use notations in Sect. 7.2.

Lemma 9.1

Let $m\in {\mathbb {Z}}$. Let $\Delta =[\nu ^a \rho , \nu ^b \rho ]$. If $\Delta ' = [\nu ^k \rho , \nu ^{l}\rho ]$ such that

$$\begin{aligned} -\frac{m-1}{2}+a \le k \le l \le \frac{m-1}{2}+b , \end{aligned}$$

then

$$\begin{aligned} u(m,\Delta ) \times \langle \Delta ' \rangle \cong \langle \Delta ' \rangle \times u(m,\Delta ) , \end{aligned}$$

and is irreducible.

Proof

When $-(m-1)/2+a-k \notin {\mathbb {Z}}$, this case is easier by using (2.1) and we omit the details. We now consider $-(m-1)/2+a-k \in {\mathbb {Z}}$.

The statement is not difficult when $\Delta$ is a singleton $[\rho ]$ because

$$\begin{aligned} u(m,\Delta ) \cong {\mathrm {St}}([\nu ^{-(m-1)/2}\rho , \nu ^{(m-1)/2}\rho ]) \end{aligned}$$

and ${\mathrm {St}}(\Delta ') \times \langle \Delta '' \rangle \cong \langle \Delta '' \rangle \times {\mathrm {St}}(\Delta ')$ whenever $\Delta '' \subset \Delta '$. (Indeed, one can also prove the latter fact by similar arguments as below by noting that the Zelevinsky multisegment of any simple composition factor in ${\mathrm {St}}(\Delta ')\times \langle \Delta ''\rangle$ contains a segment ${\widetilde{\Delta }}$ with $b({\widetilde{\Delta }}) \cong b(\Delta ')$ and at least one segment ${\widehat{\Delta }}$ with $b({\widehat{\Delta }})\cong b(\Delta '')$.)

We now assume $\Delta$ is not a singleton. We consider two cases:

• (1) Case 1: $l=b+\frac{m-1}{2}$. Suppose $\tau$ is a composition factor of $a(m,\Delta ) \times \langle \Delta ' \rangle$ with the associated Zelevinsky multisegment ${\mathfrak {m}}$. Then we know that at least two segments $\Delta _1, \Delta _2$ in ${\mathfrak {m}}$ takes the form $b(\Delta _1)\cong b(\Delta _2) \cong \nu ^{\frac{m-1}{2}+b}\rho$. If $\tau ^{(i^*)}$ is the highest derivative of $\tau$, then we know that the cuspidal support of $\tau ^{(i^*)}$ does not contain $\nu ^{\frac{m-1}{2}}\rho$. We also have that $\tau ^{(i^*)}$ is a composition factor of $(u(m,\Delta ) \times \langle \Delta \rangle )^{(i^*)}$. The only possibility is that $i^*=m+1$ i.e $(u(m,\Delta )\times \langle \Delta ' \rangle )^{(i^*)}=u(m,\Delta ^-) \times \langle (\Delta ')^- \rangle$, which the latter one is irreducible by induction. This proves the lemma. Since taking derivative is an exact functor, we have that $u(m,\Delta ) \times \langle \Delta ' \rangle$ is irreducible. Using the Gelfand–Kazhdan involution [6, Section 7], we have that $\langle \Delta ' \rangle \times u(m,\Delta ) \cong u(m,\Delta ) \times \langle \Delta ' \rangle$.

• (2) Case 2: $l<b+\frac{m-1}{2}$. The argument is similar. Again suppose $\tau$ is a composition factor of $u(m,\Delta ) \times \langle \Delta ' \rangle$. Using an argument similar to above, we have that the level of $\tau$ is either $m+1$ or m. However, if the level of $\tau$ is m, then $\tau ^{(m)}$ would be $u(m,\Delta ^-) \times \langle \Delta ' \rangle$, which is irreducible by induction. Then it would imply that the number of segments in Zelevinsky multisegment of $\tau ^{(m)}$ is m and contradicts that the number of segments for the Zelevinsky multisegment of the highest derivative of an irreducible representation $\pi$ must be at most that for $\pi$. Hence, the level of $\tau$ must be $m+1$. Now repeating a similar argument as in (1) and using the induction, we obtain the statements.$\square$

Lemma 9.2

Let $\Delta$ be a segment. Then

$$\begin{aligned} u(m,\Delta ^-) \times \nu ^{(m-1)/2}\langle \Delta \rangle \cong \nu ^{(m-1)/2}\langle \Delta \rangle \times u(m, \Delta ^-) \end{aligned}$$

is irreducible.

Proof

The statement is clear if $\Delta$ is a singleton. For the general case, we note by simple counting that the Zelevinsky multisegment of any composition factor must contain a segment $\Delta _1$ with $b(\Delta _1) \cong \nu ^{(m-1)/2}b(\Delta )$ and at least one segment $\Delta _2$ with $b(\Delta _2)\cong \nu ^{(m-1)/2}b(\Delta ^-)$. Then one proves the statement by a similar argument using the highest derivative as in the previous lemma. $\square$

1.2 Speh representation approximation

For a Speh multisegment

$$\begin{aligned} {\mathfrak {m}}=\left\{ \Delta , \nu ^{-1}\Delta , \ldots , \nu ^{-k}\Delta \right\} , \end{aligned}$$

define $b({\mathfrak {m}})=b(\Delta )$.

Proposition 9.3

Let ${\mathfrak {m}}=\left\{ \Delta _1,\ldots , \Delta _k \right\}$. Then there exists Speh multisegments ${\mathfrak {m}}_1, \ldots ,{\mathfrak {m}}_r$ satisfying the following properties:

• (1) ${\mathfrak {m}}={\mathfrak {m}}_1+\ldots +{\mathfrak {m}}_r$;

• (2) For each Speh multisegment ${\mathfrak {m}}_i$ and any $j >i$, there is no segment $\Delta$ in ${\mathfrak {m}}_i$ such that ${\mathfrak {m}}_j +\left\{ \Delta \right\}$ is a Speh multisegment;

• (3) $\langle {\mathfrak {m}} \rangle$ is the unique submodule of $\langle {\mathfrak {m}}_1 \rangle \times \ldots \times \langle {\mathfrak {m}}_r \rangle$;

• (4) $b({\mathfrak {m}}_i) \not < b({\mathfrak {m}}_j)$ if $i<j$;

• (5) if ${\mathfrak {m}}_i \cap {\mathfrak {m}}_j \ne \emptyset$ and $i\le j$, then ${\mathfrak {m}}_j \subset {\mathfrak {m}}_i$.

Proof

We shall label the segments ${\mathfrak {m}}$ in the way that for $i <j$, either (i) $b(\Delta _i) \not < b(\Delta _j)$ and (ii) if $b(\Delta _i)=b(\Delta _j)$, then $a(\Delta _i) \not < a(\Delta _j)$. Let $\Delta =\Delta _1$. Let k be the largest integer ($k \ge 0$) such that $\Delta , \nu ^{-1}\Delta , \ldots , \nu ^{-k}\Delta$ are segments in ${\mathfrak {m}}$. We claim that

$$\begin{aligned} \langle {\mathfrak {m}} \rangle \hookrightarrow \langle {\mathfrak {m}}' \rangle \times \langle {\mathfrak {m}} {\setminus } {\mathfrak {m}}' \rangle \end{aligned}$$

(9.29)

and moreover $\langle {\mathfrak {m}} \rangle$ is the unique submodule of $\langle {\mathfrak {m}}' \rangle \times \langle {\mathfrak {m}} {\setminus } {\mathfrak {m}}' \rangle$. By induction, the claim proves (1) and (3), and the remaining ones follow from the inductive construction.

We now prove the claim. We shall prove by induction that for $i=0,\ldots , k$,

$$\begin{aligned} \langle {\mathfrak {m}} \rangle \hookrightarrow \langle {\mathfrak {m}}_i \rangle \times \zeta ({\mathfrak {m}} {\setminus } {\mathfrak {m}}_i ) \hookrightarrow \zeta ({\mathfrak {m}}), \end{aligned}$$

where ${\mathfrak {m}}_i = \left\{ \Delta , \nu ^{-1} \Delta , \ldots , \nu ^{-i}\Delta \right\}$. When $i=0$ the statement is clear from the definition. Now suppose that we have the inductive statement for i. To prove the statement for $i+1$. We now set ${\mathfrak {n}}$ to be the collection of all segments $\Delta '$ in ${\mathfrak {m}}$ such that $b(\Delta ')=b(\Delta ), \nu ^{-1}b(\Delta ), \ldots , \nu ^{-i}b(\Delta )$ and $\nu ^{-i}a(\Delta )$ proceeds $a(\Delta ')$. We also set $\bar{{\mathfrak {n}}}$ to be the collection of all segments $\Delta '$ in ${\mathfrak {m}}$ such that $b(\Delta ')=b(\Delta ), \nu ^{-1}b(\Delta ),\ldots , \nu ^{-i}b(\Delta )$ and $a(\Delta ')$ precedes $\nu ^{-i-1}a(\Delta )$. By using (2.1) several times, we have that $\zeta ({\mathfrak {n}})\times \zeta (\bar{{\mathfrak {n}}}) \cong \zeta (\bar{{\mathfrak {n}}}+{\mathfrak {n}})$ and

$$\begin{aligned} \zeta ({\mathfrak {m}} {\setminus } {\mathfrak {m}}_i) \cong \zeta ({\mathfrak {n}}) \times \zeta (\bar{{\mathfrak {n}}}) \times \zeta ({\mathfrak {m}}{\setminus } ({\mathfrak {n}}+\bar{{\mathfrak {n}}}+{\mathfrak {m}}_i)). \end{aligned}$$

(9.30)

From our construction (and $i \ne k$), we have that $\nu ^{-i-1}\Delta$ is a segment in ${\mathfrak {m}}{\setminus } ({\mathfrak {n}}+\bar{{\mathfrak {n}}}+{\mathfrak {m}}_i)$ and

$$\begin{aligned} \zeta ({\mathfrak {m}}{\setminus } ({\mathfrak {n}}+\bar{{\mathfrak {n}}}+{\mathfrak {m}}_i)) =\langle \nu ^{-i-1}\Delta \rangle \times \zeta ({\mathfrak {m}}{\setminus } ({\mathfrak {n}}+\bar{{\mathfrak {n}}}+{\mathfrak {m}}_{i+1}) )\end{aligned}$$

On the other hand

$$\begin{aligned}&\langle {\mathfrak {m}}_i \rangle \times \zeta ({\mathfrak {n}}) \times \zeta (\bar{{\mathfrak {n}}}) \times \langle \nu ^{-i-1}\Delta \rangle \end{aligned}$$

(9.31)

$$\begin{aligned}&\quad \cong \zeta ({\mathfrak {n}}) \times \langle {\mathfrak {m}}_i \rangle \times \zeta (\bar{{\mathfrak {n}}}) \times \langle \nu ^{-i-1}\Delta \rangle \end{aligned}$$

(9.32)

$$\begin{aligned}&\quad \cong \zeta ({\mathfrak {n}}) \times \langle {\mathfrak {m}}_i \rangle \times \langle \nu ^{-i-1}\Delta \rangle \times \zeta (\bar{{\mathfrak {n}}}) \end{aligned}$$

(9.33)

$$\begin{aligned}&\quad \hookleftarrow \zeta ({\mathfrak {n}}) \times \langle {\mathfrak {m}}_{i+1} \rangle \times \zeta (\bar{{\mathfrak {n}}})\end{aligned}$$

(9.34)

$$\begin{aligned}&\quad \cong \langle {\mathfrak {m}}_{i+1} \rangle \times \zeta ({\mathfrak {n}}) \times \zeta (\bar{{\mathfrak {n}}}) \end{aligned}$$

(9.35)

The first and last isomorphisms follow from Lemma 9.3. The injectivity in the forth line comes from the uniqueness of submodule in $\zeta ({\mathfrak {m}}_{i+1})$. The second isomorphism follows from again by (2.1). Combining (9.30), the above series of isomorphisms and the inductive case, we have:

$$\begin{aligned} \zeta ({\mathfrak {m}}) \hookleftarrow \langle {\mathfrak {m}}_{i+1} \rangle \times \zeta ({\mathfrak {n}}) \times \zeta (\bar{{\mathfrak {n}}}) \times \zeta ({\mathfrak {m}}{\setminus } ({\mathfrak {n}}+\bar{{\mathfrak {n}}}+{\mathfrak {m}}_{i+1})). \end{aligned}$$

This gives the desired injectivity by using the uniqueness of submodule of $\zeta ({\mathfrak {m}})$ and proves the claim. $\square$

We shall need a variation which is more flexible in our application.

Lemma 9.4

Let ${\mathfrak {m}}={\mathfrak {m}}(m,\Delta )$ be a Speh multisegment. Let ${\mathfrak {n}}_1$ be a Zelevinsky multisegment such that for any segment $\Delta '$ in ${\mathfrak {n}}_1$ satisfying $b(\Delta ) < b(\Delta ')$ or $b(\Delta ) \cong b(\Delta ')$. Let ${\mathfrak {n}}_2$ be a Zelevinsky multisegment such that for any segment $\Delta '$ in ${\mathfrak {n}}_2$ satisfying either one of the following properties:

• (1) $b(\Delta ) \not <b(\Delta ')$; or

• (2) $b(\Delta )\not < b(\Delta ')$, or if $b(\Delta )< b(\Delta ')$, then $(\Delta ')^-=\Delta$.

Then

$$\begin{aligned} \langle {\mathfrak {n}}_1+{\mathfrak {m}}+{\mathfrak {n}}_2 \rangle \hookrightarrow \zeta ({\mathfrak {n}}_1) \times \langle {\mathfrak {m}} \rangle \times \zeta ({\mathfrak {n}}_2) \hookrightarrow \zeta ({\mathfrak {n}}_1+{\mathfrak {m}}+{\mathfrak {n}}_2) . \end{aligned}$$

In particular, $\zeta ({\mathfrak {n}}_1) \times \langle {\mathfrak {m}} \rangle \times \zeta ({\mathfrak {n}}_2)$ has unique submodule isomorphic to $\langle {\mathfrak {n}}_1 +{\mathfrak {m}}+{\mathfrak {n}}_2 \rangle$.

Remark 9.5

Case (2) covers case (1). But for the purpose of clarity of an argument used in Sect. 7.4, we divide into two cases. We also recall that the case $b(\Delta ) \not < b(\Delta ')$ includes the possibility $b(\Delta ) \cong b(\Delta ')$.

Proof

For all cases, we have that

$$\begin{aligned} \zeta ({\mathfrak {n}}_1) \times \zeta ({\mathfrak {m}}+{\mathfrak {n}}_2) \cong \zeta ({\mathfrak {n}}_1+{\mathfrak {m}}+{\mathfrak {n}}_2) . \end{aligned}$$

Using (9.29) for (1) we obtain the lemma. For (2), let ${\mathfrak {n}}'$ be all the segments in ${\mathfrak {n}}_2$ with the property that $(\Delta ')^-\cong \Delta$. Then we have that

$$\begin{aligned} \zeta ({\mathfrak {n}}') \times \zeta ({\mathfrak {m}})\times \zeta ({\mathfrak {n}}_2 {\setminus } {\mathfrak {n}}') \hookrightarrow \zeta ({\mathfrak {n}}') \times \zeta (({\mathfrak {m}}+{\mathfrak {n}}_2){\setminus } {\mathfrak {n}}')\hookrightarrow \zeta ({\mathfrak {m}}+{\mathfrak {n}}_2) . \end{aligned}$$

By Lemma 9.2, we have that

$$\begin{aligned} \zeta ({\mathfrak {n}}') \times \zeta ({\mathfrak {m}})\times \zeta ({\mathfrak {n}}_2 {\setminus } {\mathfrak {n}}') \cong \zeta ({\mathfrak {m}})\times \zeta ({\mathfrak {n}}')\times \zeta ({\mathfrak {n}}_2 {\setminus } {\mathfrak {n}}') \cong \zeta ({\mathfrak {m}})\times \zeta ({\mathfrak {n}}_2), \end{aligned}$$

which proves the lemma. $\square$

Appendix

Let G be a connected reductive group over a non-Archimedean local field.

Lemma 10.1

Let $\pi$ be a smooth representation of G. Let $\pi$ admits a filtration

$$\begin{aligned} 0=\pi _0' \subset \pi _1' \subset \ldots \subset \pi _r' =\pi \end{aligned}$$

Suppose $\pi$ admits an irreducible subquotient $\tau$. Let $\pi _k=\pi _k'/\pi _{k-1}'$ for $k=1,\ldots , r$. Then there exists s such that $\pi _s$ contains an irreducible subquotient isomorphic to $\tau$.

Proof

Let $\lambda _1, \lambda _2$ be submodules of $\pi$ such that $\lambda _2/\lambda _1 =\tau$. Now we have a filtration on $\lambda _p$ ($p=1,2$):

$$\begin{aligned} 0 \subset (\pi _1' \cap \lambda _p) \subset (\pi _2'\cap \lambda _p) \subset \ldots \subset (\pi _r'\cap \lambda _p) =\lambda _p . \end{aligned}$$

Now $\lambda _2/\lambda _1$ admits a filtration

$$\begin{aligned} 0 \subset \gamma _1 \subset \gamma _2 \subset \ldots \subset \gamma _r , \end{aligned}$$

where $\gamma _k = (\pi _{k}'\cap \lambda _2) /(\pi _k'\cap \lambda _1)$ and so

$$\begin{aligned} \gamma _{k+1}/\gamma _k \cong \frac{(\pi _{k+1}' \cap \lambda _2)/(\pi _{k+1}'\cap \lambda _1)}{(\pi _{k}'\cap \lambda _2) /(\pi _k'\cap \lambda _1)} . \end{aligned}$$

This implies $\pi _{k+1}' \cap \lambda _2$ and so $\pi _{k+1}'$ has irreducible subquotient isomorphic to $\tau$. $\square$

Lemma 10.2

Let $\pi$ be a non-zero smooth representation of G. Let $\tau$ be a non-zero G-submodule of $\pi$. Suppose $\pi$ admits a filtration on

$$\begin{aligned} 0=\pi _0 \subset \pi _1 \subset \ldots \subset \pi _r=\pi . \end{aligned}$$

Then there exists a non-zero G-submodule $\tau '$ of $\tau$ such that

$$\begin{aligned} \tau ' \hookrightarrow \pi _{k+1}/\pi _k. \end{aligned}$$

for some k.

Proof

Let k be the smallest positive integer such that

$$\begin{aligned} \tau \cap \pi _k \ne 0 . \end{aligned}$$

Define $\tau '=\tau \cap \pi _k$. The non-zero G-submodule $\tau '$ embeds to $\pi _k/\pi _{k-1}$ as desired. $\square$

Lemma 10.3

Let $0 \ne \pi \in {\mathrm {Alg}}(G)$. Suppose $\pi$ admits two G-filtrations:

$$\begin{aligned} 0=\pi _0 \subset \pi _1\subset \pi _2\subset \ldots \subset \pi _r =\pi \end{aligned}$$

and

$$\begin{aligned} 0= \pi _0' \subset \pi _1' \subset \pi _2' \subset \ldots \subset \pi _s'=\pi . \end{aligned}$$

Then there exists a non-zero $\tau \in {\mathrm {Alg}}(G)$ such that for some i, j,

$$\begin{aligned} \tau \hookrightarrow \pi _{i+1}/\pi _i, \quad \tau \hookrightarrow \pi _{j+1}'/\pi _j' . \end{aligned}$$

Proof

When $\pi _1\ne 0$, we consider the smallest integer k such that $\pi _1 \cap \pi '_k \ne 0$. The general case is similar. $\square$