Advertisement

Research in Number Theory

, 4:31 | Cite as

Extension of Whittaker functions and test vectors

  • R. Kurinczuk
  • N. Matringe
Open Access
Research
  • 372 Downloads

Abstract

We show that certain products of Whittaker functions and Schwartz functions on a general linear group extend to Whittaker functions on a larger general linear group. This generalizes results of Cogdell and Piatetski-Shapiro (Representation Theory, Number Theory, and Invariant Theory. Progress in Mathematics, Birkhäuser/Springer, Cham, 2017) and Jacquet et al. (Am J Math 105(2):367–464, 1983). As a consequence, we prove that the Rankin–Selberg L-factor of the product of a discrete series representation and the Zelevinsky dual of a discrete series representation is given by a single Rankin–Selberg integral.

Keywords

Automorphic L-functions Rankin–Selberg method Whittaker models Representations of p-adic groups 

Mathematics Subject Classification

11F70 11F66 

1 Introduction

In their seminal work, Jacquet–Piatetski-Shapiro–Shalika developed the Rankin–Selberg method for automorphic representations, treating the local theory in [14]. The local Euler factors, or L-factors, are defined as greatest common divisors of families of local Rankin–Selberg integrals. As a consequence of the definition, each L-factor can be written as a finite sum of Rankin–Selberg integrals, but it is not clear whether one can find test vectors expressing the L-factor as a single Rankin–Selberg integral. When the local representations are cuspidal, this is possible by an explicit computation [16].

In this article, we prove a local result on the extension of Whittaker and Schwartz functions (stated precisely at the end of this introduction). We show that this result simultaneously generalizes results of [6] and [14], both of which have proved useful in the theory of integral representations of L-factors. As a consequence of our result we answer the test vector question in the affirmative for the L-factor of the product of a discrete series representation and the Zelevinsky dual of a discrete series representation.

Let F be a locally compact non-archimedean local field, \(\psi \) be a nontrivial character of F, and put \(G_n={\text {GL}}_n(F)\). Let \(\pi \) and \(\pi '\) be irreducible smooth representations of \(G_n\) and \(G_m\) with \(n\geqslant m\), and let \({\mathcal {S}}(\pi )\) and \({\mathcal {S}}(\pi ')\) be the standard modules whose irreducible quotients are \(\pi \) and \(\pi '\) respectively. If \(n>m\), the L-factor \(L(s,\pi ,\pi ')\) attached to the pair \((\pi ,\pi ')\) by Jacquet et al. [14] is a finite sum \(\sum _i I(s,W_i,W'_i)\) of Rankin-Selberg integrals for Whittaker functions \(W_i\in W({\mathcal {S}}(\pi ),\psi )\) and \(W'_i\in W({\mathcal {S}}(\pi '),\psi ^{-1})\) in the Whittaker models of the standard modules. If \(n=m\), it is a finite sum \(\sum _i I(s,W_i,W'_i,\phi _i)\) of Rankin–Selberg integrals with \(W_i,W'_i\) as before and \(\phi _i\in {\mathcal {C}}_c^\infty (F^n)\) a Schwartz function. A tuple \((W,W')\) or \((W,W',\phi )\) whose Rankin–Selberg integral equals the L-factor is called a test vector. The systematic study of test vectors for local Rankin–Selberg integrals was initiated in the doctoral thesis [15], where many interesting partial results were obtained.

Let \(\delta \) and \(\delta '\) be discrete series representations of \(G_n\) and \(G_m\) respectively, and \({}^{t}\delta '\) denote the Zelevinsky dual of \(\delta '\). In this article we prove the following test vector result:

Theorem

(Remark 4.2, Proposition 4.5 and Theorem 4.6) There exist \(W\in W(\delta ,\psi )\) and \(W'\in W({\mathcal {S}}({}^{t}\delta '),\psi ^{-1})\), such that,
  1. i.
    If \(n>m\)
    $$\begin{aligned} L(s,\delta ,{}^{t}\delta ')=I(s,W,W'); \end{aligned}$$
     
  2. ii.
    If \(n=m\), in addition there exists a Schwartz function \(\phi \) on \(F^n\), such that
    $$\begin{aligned} L(s,\delta ,{}^{t}\delta ')=I(s,W,W',\phi ). \end{aligned}$$
    Moreover one can always choose \(\phi ={\mathbf {1}}_{({{\mathfrak {p}}}^f)^{n-1} \times (1+{{\mathfrak {p}}}^f)}\) for f large enough.
     

In fact, \(L(s,\delta ,{}^{t}\delta ')=L(s,\delta ,\delta ')\), so this also shows that the Euler factor \(L(s,\delta ,\delta ')\) is given by a single integral \(I(s,W,W')\) or \(I(s,W,W',\phi )\); however with \(W'\) inside \(W({\mathcal {S}}({}^{t}\delta '),\psi ^{-1})\) rather than inside \(W(\delta ',\psi ^{-1})\). Indeed, in this case, this makes the test vector question simpler as \(W({\mathcal {S}}({}^{t}\delta '),\psi ^{-1})\) contains \(W(\delta ',\psi ^{-1})\) as a proper subspace. An easy, yet already instructive, example is given in Sect. 4.2, where we take \(\delta \) and \(\delta '\) to be Steinberg representations. In the general case, we do not address the question of finding explicit test vectors, which would require other techniques, for example Bushnell–Kutzko type theory in the spirit of [16, 20]. The techniques of this paper are entirely different to [16]; here we use Bernstein and Zelevinsky’s theory of derivatives, in particular Cogdell–Piatetski-Shapiro’s interpretation of derivatives [6], to reduce to the cuspidal case.

To obtain our test vector result, the key step is Theorem 3.10, which generalizes both [14, Proposition 9.1] and part of [6, Proposition 1.7]. To state Theorem 3.10, first we explain a consequence of the results of [6] which we recall and expand in Sects. 3.2 and 3.3. Denote by \(P_n\) the mirabolic subgroup of \(G_n\) consisting of matrices with final row \(\eta _n=\begin{pmatrix}0&\cdots&0&1\end{pmatrix}\), and by \(N_n\) its standard maximal unipotent subgroup. If \(\tau \) is a submodule of \({\text {Ind}}_{N_n}^{P_n}(\psi )\) such that the k-th Bernstein–Zelevinsky derivative \(\tau ^{(k)}\) has a central character, then there is a natural embedding
$$\begin{aligned} {\overline{S}}:\tau ^{(k)}\rightarrow {\text {Ind}}_{N_{n-k}}^{G_{n-k}}(\psi ), \end{aligned}$$
and we put \(W(\tau ^{(k)},\psi )={\overline{S}}(\tau ^{(k)})\). We can now state Theorem 3.10 as:

Theorem 3.10 Let \(n >k \geqslant 1\). For any Schwartz function \(\phi \) on \(F^{n-k}\) and any \(W_0\in W(\tau ^{(k)},\psi )\), the map \(|\mathrm {det}(g)|^{k/2}W_0(g)\phi (\eta _{n-k} g)\) extends to a function in \(\tau \subseteq {\text {Ind}}_{N_n}^{P_n}(\psi )\).

In particular, this applies to the case where \(\tau \) is a submodule of the space of restrictions to \(P_n\) of functions in the Whittaker model of a representation of Whittaker type of \(G_n\).

2 Notation and preliminaries

As in the introduction, F denotes a locally compact non-archimedean local field, we put \(G_n={\text {GL}}_n(F)\), and \(\psi \) denotes a nontrivial character of F. We denote by \(|\,\cdot \,|\) the normalized absolute value of F, by \({\mathfrak {o}}\) its ring of integers, and by \({{\mathfrak {p}}}\) the maximal ideal of \({\mathfrak {o}}\). We let q denote the order of the residue field \({\mathfrak {o}}/{{\mathfrak {p}}}\), \(Z_n\) denote the centre of \(G_n\), and \(K_n={\text {GL}}_n({\mathfrak {o}})\). We denote by \(\nu \) the character of \(G_n\) given by \(\nu (g)=\left| {\det (g)}\right| \) for \(g\in G_n\). Let \(P_n\) be the mirabolic subgroup of \(G_n\), consisting of all matrices in \(G_n\) with bottom row \(\eta _n=(0,\dots ,0,1)\). Any \(p\in P_n\) can be written in a uniquely as \(p=g(p)u(p)\) for \(g(p)\in G_{n-1}\), and \(u(p)\in U_n\) the unipotent radical of the standard (block upper triangular) parabolic subgroup of \(G_n\) of type \((n-1,1)\). For \(p\in P_n\), we put \(\delta _{P_n}(p)=\nu (g(p))\). We have the Iwasawa decomposition
$$\begin{aligned}G_n=P_nZ_nK_n,\end{aligned}$$
and if \(g=pzk\) in this decomposition, then \(\nu (z)\) only depends on g, and we shall write \(\nu (z(g))\) for \(\nu (z)\). We consider \(G_k\) as a subgroup of \(G_n\) for \(1\leqslant k \leqslant n\) via the embedding \(g\mapsto {\text {diag}}(g,I_{n-k})\).

Let G be a locally compact totally disconnected group. By a representation of G we mean a smooth representation on a complex vector space. We denote by \({\mathfrak {S}}(G)\) the category of (smooth) representations of G, and by \({\mathfrak {R}}(G)\) the category of finite length representations of G. When practical, we use the same notation for the collection of objects in a category and the underlying category; so, for example, \(\pi \in {\mathfrak {S}}(G)\) will mean \(\pi \) is an object of \({\mathfrak {S}}(G)\).

We let \(G_0\) denote the trivial group. For \(G=G_n\) we set \({\mathfrak {S}}(n)={\mathfrak {S}}(G_n)\), \({\mathfrak {R}}(n)={\mathfrak {R}}(G_n)\), \({\mathfrak {S}}=\bigsqcup _{n=0}^\infty {\mathfrak {S}}(n)\) and \({\mathfrak {R}}=\bigsqcup _{n=0}^\infty {\mathfrak {R}}(n)\). We also denote by \({\text {Irr}}(n)\) the collection of irreducible representations of \(G_n\), and set \({\text {Irr}}=\bigsqcup _{n=0}^\infty {\text {Irr}}(n)\).

For H a closed subgroup of a locally compact totally disconnected group G, we use the notation \({\text {Ind}}_H^G:{\mathfrak {S}}(H)\rightarrow {\mathfrak {S}}(G)\) for the functor of normalized induction. For representations \(\pi _i\) of \(G_{n_i}\), \(i=1,\dots ,t\) we denote by \(\pi _1\times \cdots \times \pi _t\) the representation of \(G_{n_1+\cdots +n_t}\) obtained from \(\pi _1\otimes \cdots \otimes \pi _t\) by normalized parabolic induction for the standard parabolic of type \((n_1,\cdots ,n_t)\). For a representation \(\pi \) and a character \(\chi \) of \(G_n\) let \(\chi \pi \) be the representation on the space of \(\pi \) given by \((\chi \pi )(g)=\chi (g)\pi (g)\) for \(g\in G_n\).

In [3, Sect. 3], the authors define functors
$$\begin{aligned}&{\Phi }^{-}:{\mathfrak {S}}(P_n)\rightarrow {\mathfrak {S}}(P_{n-1});\\&{\Psi }^{-}:{\mathfrak {S}}(P_n)\rightarrow {\mathfrak {S}}(G_{n-1}). \end{aligned}$$
(in fact, such functors originally appear in [2], but we use the modified definition of [3]). It is shown in [3] that \(\Phi ^-,\Psi ^-\) are exact, and restrict as functors to
$$\begin{aligned} \Phi ^{-}:{\mathfrak {R}}(P_n)&\rightarrow {\mathfrak {R}}(P_{n-1});\\ \Psi ^{-}:{\mathfrak {R}}(P_n)&\rightarrow {\mathfrak {R}}(G_{n-1}). \end{aligned}$$
It is also shown for a representation \(\pi \in {\mathfrak {S}}(G_n)\), that \(\pi \in {\mathfrak {R}}(G_n)\) if and only if \(\pi \vert _{P_n}\in {\mathfrak {R}}(P_n)\). Following [3], for \(\pi \in {\mathfrak {S}}(P_n)\), and \(k\geqslant 1\), we set
$$\begin{aligned} \pi ^{(k)}=\Psi ^-(\Phi ^-)^{k-1} \pi , \qquad \pi ^{(0)}=\pi . \end{aligned}$$
For \(\pi \in {\mathfrak {S}}(G_n)\), and \(k\geqslant 0\), we set \(\pi ^{(k)}=(\pi \mid _{P_n})^{(k)}\).
Let \(N_n\) be the group of upper triangular unipotent matrices in \(G_n\) and, by abuse of notation, let \(\psi \) also denote the character on \(N_n\) defined by
$$\begin{aligned} \psi (u)=\psi (u_{1,2}+\cdots +u_{n-1,n}), \end{aligned}$$
for \(u\in N_n\) with (ij)-th entry \(u_{i,j}\).
By Frobenius reciprocity and [3, Proposition 3.2], for \(\pi \in {\mathfrak {S}}(P_n)\), we have
$$\begin{aligned} {\text {Hom}}_{N_n}(\pi ,\psi )\simeq {\text {Hom}}_{P_n}(\pi ,{\text {Ind}}_{N_n}^{P_n}(\psi )) \simeq {\text {Hom}}_{{\mathbb {C}}}(\pi ^{(n)},{\mathbb {C}}). \end{aligned}$$
In particular when \(\pi \) belongs to \({\mathfrak {R}}(P_n)\), the (finite) dimension of \(\pi ^{(n)}\) is precisely that of the space \({\text {Hom}}_{N_n}(\pi ,\psi )\) of Whittaker functionals on \(\pi \). Following the authors of [14], we introduce the following classes of representations.

Definition 2.1

We say that a representation \(\pi \in {\mathfrak {R}}(P_n)\) is of Whittaker type if \({\text {Hom}}_{N_n}(\pi ,\psi )\) is of dimension one. We say that a representation \(\pi \in {\mathfrak {R}}(G_n)\) is of Whittaker type if \(\pi \mid _{P_n}\) is of Whittaker type, i.e. if \({\text {Hom}}_{N_n}(\pi ,\psi )\) is of dimension one.

In either case, a finite length representation \(\pi \) is of Whittaker type if and only if \(\pi ^{(n)}\simeq {\mathbb {C}}\). In particular, if \(\pi _1\) and \(\pi _2\) are both representations of Whittaker type of \(G_{n_1}\) and \(G_{n_2}\) respectively, then the representation \(\pi _1\times \pi _2\) is also of Whittaker type according to [3, Corollary 4.14, (c)]. Let \(\pi \) be a representation of Whittaker type of \(P_n\) (resp. \(G_n\)). By Frobenius reciprocity, there is a unique up to scalar nonzero intertwining operator from \(\pi \) to \({\text {Ind}}_{N_n}^{P_n}(\psi )\) (resp. \({\text {Ind}}_{N_n}^{G_n}(\psi )\)), and we denote by \(W(\pi ,\psi )\) the image of \(\pi \) and call it the Whittaker model of \(\pi \), though it is not always a model of \(\pi \), i.e. in general \(W(\pi ,\psi )\) is a quotient of \(\pi \), but not isomorphic to it.

An irreducible representation of Whittaker type of \(G_n\) is called generic. In fact, by [8], the generic representations of \(G_n\) are those irreducible representations \(\pi \) such that \({\text {Hom}}_{N_n}(\pi ,\psi )\ne 0\). By exactness of the n-th derivative functor, a representation of \(G_n\) of Whittaker type has a unique generic subquotient. For the group \(G_n\), it follows from [3, Lemma 4.5] and [23, Sect. 9], that if \(\delta _1,\dots ,\delta _t\) are irreducible essentially square integrable, which we call discrete series, representations, then \(\delta _1\times \cdots \times \delta _t\) is a representation of Whittaker type.

If \(\delta \) is an irreducible discrete series representation, we denote by \(e(\delta )\) the unique real number such that \(\nu ^{-e(\delta )}\delta \) is unitary. We say that a representation in \({\mathfrak {R}}\) is a standard module if it is of the form \(\delta _1\times \cdots \times \delta _t\), with \(\delta _i\)’s irreducible discrete series such that \(e(\delta _1)\geqslant \dots \geqslant e(\delta _t)\) (these representations are called induced of Langlands type in [14] and [12]). If all the \(\delta _i\)’s are cuspidal, we say that the standard module \(\delta _1\times \cdots \times \delta _t\) is cuspidally induced. By [22], a standard module \({\mathcal {S}}\) has a unique irreducible quotient \(\pi ({\mathcal {S}})\), and the map
$$\begin{aligned}{\mathcal {S}}\in ({\mathfrak {R}}_{{\text {stand}}}(n)/\simeq ) \mapsto \pi ({\mathcal {S}})\in ({\text {Irr}}(n)/\simeq )\end{aligned}$$
is a bijection; the inverse of this bijection we denote by \(\pi \mapsto {\mathcal {S}}(\pi )\). We call \({\mathcal {S}}(\pi )\) the standard module over \(\pi \).
We will use the following lower index notation. If A is one of the collections \({\mathfrak {S}}, {\mathfrak {R}}\) or \({\text {Irr}}\), and \(\bullet \) is an abbreviation of a type of representation, we will denote by \(A_\bullet \) the collection of representations of type \(\bullet \) inside A. If a representation \(\pi \) in A has type \(\bullet _1\) and \(\bullet _2\) together, we will write \(\pi \in A_{\bullet _1,\bullet _2}\). We will denote by \(A_{\bullet }(n)\) the representations of \(G_n\) in \(A_\bullet \). We use the following abbreviations of types of representations:
  • \(=~{{\text {cusp}}}\): cuspidal,

  • \(=~{{\text {disc}}}\): essentially square integrable/discrete series,

  • \(=~{{\text {gen}}}\): generic,

  • \(=~{{\text {stand}}}\): standard module,

  • \(=~{{\text {stand}}}-{{\text {cusp}}}\): cuspidally induced standard module,

  • \(=~{{\text {Whitt}}}\): Whittaker type,

  • \(=~{{\text {cent}}}\): with central character.

For example, we have the well known inclusions
$$\begin{aligned}&{\text {Irr}}\subseteq {\mathfrak {S}}_{{\text {cent}}},&{\text {Irr}}_{{\text {cusp}}}\subseteq {\text {Irr}}_{{\text {disc}}}\subseteq {\text {Irr}}_{{\text {gen}}}\subseteq {\mathfrak {R}}_{{{\text {Whitt}}},{{\text {cent}}}}, \\&{\mathfrak {R}}_{{\text {stand}}}\subseteq {\mathfrak {R}}_{{{\text {Whitt}}},{{\text {cent}}}}&{\text {Irr}}\cap {\mathfrak {R}}_{{\text {stand}}}={\text {Irr}}\cap {\mathfrak {R}}_{{\text {Whitt}}}= {\text {Irr}}_{{{\text {gen}}}}. \end{aligned}$$
Notice that \({\mathfrak {R}}_{{{\text {stand}}}-{{\text {cusp}}}}\) is different from \({\mathfrak {R}}_{{{\text {stand}}},{{\text {cusp}}}}={\text {Irr}}_{{\text {cusp}}}\). Recall that
$$\begin{aligned} ({\text {Irr}}_{{\text {disc}}}/\simeq )=\{{\text {St}}_k(\rho ):\rho \in ({\text {Irr}}_{{\text {cusp}}}/\simeq ),\,k\in {\mathbb {N}}\} \end{aligned}$$
where \({\text {St}}_k(\rho )\) is the up to nonzero scalar unique isomorphism unique irreducible quotient of \(\nu ^{(1-k)/2}\rho \times \nu ^{(3-k)/2}\rho \times \cdots \times \nu ^{(k-1)/2}\rho \) (see [23, Theorem 9.3]). By [23, Sect. 9.1], the representation \({\text {St}}_k(\rho )\) is also the unique irreducible submodule of
$$\begin{aligned} {\mathcal {S}}_k(\rho )=\nu ^{(k-1)/2}\rho \times \nu ^{(k-3)/2}\rho \times \cdots \times \nu ^{(1-k)/2}\rho \in {\mathfrak {R}}_{{{\text {stand}}}}, \end{aligned}$$
and we denote by \({\text {Sp}}_k(\rho )\) its irreducible quotient, i.e. \({\mathcal {S}}_k(\rho )={\mathcal {S}}({\text {Sp}}_k(\rho ))\). The author of [23] attaches to \(\delta ={\text {St}}_k(\rho )\) the cuspidal segment \(\Delta =[\nu ^{\frac{1-k}{2}}\rho ,\dots ,\nu ^{\frac{k-1}{2}}\rho ]\) (see [23, Sects. 3 and 9]), and defines being linked and preceding relations on the set of cuspidal segments in [23, Sect. 4.1]. According to [23, Sect. 9], if \(\delta ={\text {St}}_k(\rho )\), then its Zelevinsky dual is \(^t\delta ={\text {Sp}}_k(\rho )\) (see [23, Sect. 9] for the definition of the Zelevinsky dual). An easy induction on the number of segments using the fact that discrete series corresponding to unlinked segments commute in the sense of parabolic induction shows the well known fact that \(\pi \) is in \({\mathfrak {R}}_{{\text {stand}}}\) if and only if it is isomorphic to a representation of the form \(\delta _1\times \dots \times \delta _t\) with \(\delta _i\) discrete series representations such that \(\Delta _i\) does not precede \(\Delta _j\) for \(i<j\). By [23, Theorem 9.7], \(\pi \) belongs to \({\text {Irr}}_{{{\text {gen}}}}\) if and only if it can be written as a (necessarily commutative) product of discrete series \(\pi =\delta _1\times \dots \times \delta _t\) where the corresponding \(\Delta _i\)’s are unlinked, the \(\delta _i\)’s being unique up to reordering. In particular \({\text {Irr}}\cap {\mathfrak {R}}_{{\text {stand}}}= {\text {Irr}}_{{{\text {gen}}}}\) as we mentioned before and \({\mathcal {S}}(\pi )=\pi \) when \(\pi \) is generic.

We denote by \({\mathcal {C}}_c^\infty (F^n)\) the space of smooth functions on \(F^n\) with compact support.

3 Whittaker models and derivatives

Here we recall some useful but not very well-known facts from [7] and [6] about Whittaker models and the interpretation of derivatives in the space of Whittaker functions. Then we push the techniques of [6] in the spirit of [18] to obtain our first main result, which is Theorem 3.10 about extending Whittaker functions to larger linear groups.

3.1 The Whittaker model of representations of Whittaker type

Here we highlight a result of [7] on Whittaker models of representations of Whittaker type. We shall only need it in order to prove that our first main result Theorem 3.10 extends [14, Proposition 9.1]. However, the result was not known to us until recently, and it is a quite striking result on representations of Whittaker type. It says that the Whittaker model of such a representation is a submodule of the Whittaker model of a standard module. Notice that by the main result of [12], the Whittaker model of a standard module is isomorphic to this standard module and it moreover has a Kirillov model. The result follows from [7, Lemma 3.2.4] and [7, Lemma 4.3.9]. The setting of [7] being much more general and the result being stated there with a different terminology, we give a largely self-contained proof here, which in any case uses the ideas of [7].

If \(\pi \) belongs to \({\mathfrak {R}}_{{\text {Whitt}}}\), we denote by \(\pi ^{{\text {gen}}}\) its unique generic subquotient. Notice that
$$\begin{aligned}\pi ^{{\text {gen}}}\simeq W(\pi ^{{\text {gen}}},\psi ) \subseteq W(\pi ,\psi ),\end{aligned}$$
and that \(W(\pi ^{{\text {gen}}},\psi )\) is the unique irreducible submodule of \(W(\pi ,\psi )\). We give a different classification of generic representations, due to Zelevinsky as well.

Proposition 3.1

Let \(\pi \in {\text {Irr}}_{{\text {gen}}}(n)\), then it is the unique irreducible submodule of a cuspidally induced standard module \({\mathcal {S}}_c(\pi )\in {\mathfrak {R}}_{{{\text {stand}}}-{{\text {cusp}}}}(n)\). All cuspidally induced standard modules containing \(\pi \) are isomorphic.

Proof

By [23, Theorem 6.1], with the same notations, \(\pi \) is of the form \(\langle a\rangle \) for a a sequence of cuspidal segments satisfying the non-preceding ordering condition (see [23] for the precise statement). However, the representation \(\pi \) being generic, its highest derivative is \(\pi ^{(n)}\simeq {\mathbb {C}}\). By [23, Theorem 6.1], this implies that the cuspidal segments occurring in a are all cuspidal representations, and this exactly says that \(\pi \) is the unique irreducible submodule of a cuspidally induced standard module. It is also a consequence of [23, Theorem 6.1] that all cuspidally induced standard modules containing \(\pi \) are isomorphic \(\square \)

If \(\pi \) is of Whittaker type, we set \({\mathcal {S}}_c(\pi )={\mathcal {S}}_c(\pi ^{{\text {gen}}})\), we will soon show that \(W(\pi ,\psi )\subseteq W({\mathcal {S}}_c(\pi ),\psi )\), as a consequence of the following proposition.

Lemma 3.2

Let \(\pi \in {\mathfrak {R}}_{{\text {Whitt}}}(n)\), if \(\tau \in {\text {Irr}}(n)\) admits a nontrivial extension by \({\mathcal {S}}_c(\pi )\), then \(\tau =\pi ^{{\text {gen}}}\).

Proof

Set \(G=G_n\). Write \({\mathcal {S}}_c(\pi )=\rho _1\times \dots \times \rho _t\) with the cuspidal segments \([\rho _i]\) satisfy the non-preceding condition described at the end of Sect. 2, we set M to be the standard Levi subgroup of G, such that
$$\begin{aligned} \rho =\rho _1\otimes \dots \otimes \rho _t \end{aligned}$$
is a representation of M. We denote by \(\mathrm {J}^G_M\) the normalized Jacquet functor from \({\mathfrak {S}}(G)\) to \({\mathfrak {S}}(M)\). Suppose that \(\tau \) admits a nontrivial extension by \({\mathcal {S}}_c(\pi )\), this means that
$$\begin{aligned} \mathrm {Ext}_G^1(\tau ,{\mathcal {S}}_c(\pi ))\ne 0, \end{aligned}$$
(see [5] for the basic definitions concerning \(\mathrm {Ext}\)-functors in the categories \({\mathfrak {S}}(G)\) and \({\mathfrak {S}}(M)\)). Then by [5, Theorem A.12], this is equivalent to
$$\begin{aligned} \mathrm {Ext}_M^1(\mathrm {J}_G^M(\tau ),\rho )\ne 0. \end{aligned}$$
We claim that this implies there is an irreducible subquotient \(\mu \) of \(\mathrm {J}_G^M(\tau )\) (which has finite length) which admits a nontrivial extension by \(\rho \). Let’s justify this claim by writing an exact sequence
$$\begin{aligned} 0\rightarrow \mathrm {J}_1 \rightarrow \mathrm {J}_G^M(\tau ) \rightarrow \mathrm {J}_2\rightarrow 0 \end{aligned}$$
with \(\mathrm {J}_2\) irreducible. Then writing the long exact sequence
this implies that \(\mathrm {Ext}_M^1(\mathrm {J}_2,\rho )\) and \(\mathrm {Ext}_M^1(\mathrm {J}_1,\rho )\) can not be zero together because \(\mathrm {Ext}_M^1(\mathrm {J}_G^M(\tau ),\rho )\) is not. If \(\mathrm {Ext}_M^1(\mathrm {J}_2,\rho )\ne 0\) then we set \(\mu =\mathrm {J}_2\), otherwise we repeat the same operation with \(\mathrm {J}_1\) instead of \(\mathrm {J}_G^M(\pi )\) (as \(\mathrm {J}_G^M(\pi )\) is of finite length, this process terminates to find an irreducible \(\mu \)). Now by [1], we must have \(\mu \simeq \chi \rho \) for \(\chi \) an unramified character of G because \(\rho \) is cuspidal, but then \(\mu \simeq \rho \) by cuspidality of \(\tau \) and \(\rho \) (otherwise they would be in direct sum by [3, Theorem 2.4. (b)]). Hence \(\rho \) is a subquotient of \(\mathrm {J}_G^M(\tau )\), so it is a quotient of it as well ( [3, Theorem 2.4. (b)]). Thus \({\text {Hom}}_G(\tau ,{\mathcal {S}}_c(\pi ))\simeq {\text {Hom}}_M(\mathrm {J}_G^M(\tau ),\rho )\ne 0\), and hence \(\tau \) is the unique irreducible submodule of \({\mathcal {S}}_c(\pi )\), i.e. \(\tau \simeq \pi ^{{\text {gen}}}\). \(\square \)

We denote by \({\text {Res}}_{P_n}\) the map from \({\text {Ind}}_{N_n}^{G_n}(\psi )\) to \({\text {Ind}}_{N_n}^{P_n}(\psi )\) which is the restriction of functions to \(P_n\). By the main result of [12], if \(\pi \in {\mathfrak {R}}_{{{\text {stand}}}}(n)\), then \(\pi \simeq W(\pi ,\psi )\) and \({\text {Res}}_{P_n}\) is injective on \(W(\pi ,\psi )\). We will now show that the last part of this result remains true for \(\pi \in {\mathfrak {R}}_{{\text {Whitt}}}\), i.e. that if \(\pi \in {\mathfrak {R}}_{{\text {Whitt}}}\), then \(W(\pi ,\psi )\) has a Kirillov model.

Proposition 3.3

[7, Lemma 3.2.4, Lemma 4.3.9] Let \(\pi \in {\mathfrak {R}}_{{\text {Whitt}}}(n)\) with \(n\geqslant 2\), then \(W(\pi ,\psi )\subseteq W({\mathcal {S}}_c(\pi ),\psi )\), in particular \({\text {Res}}_{P_n}\) is injective on \(W(\pi ,\psi )\).

Proof

Consider \(W({\mathcal {S}}_c(\pi ),\psi )\subseteq W({\mathcal {S}}_c(\pi ),\psi )+W(\pi ,\psi )\subseteq {\text {Ind}}_{N_n}^{G_n}(\psi )\). If \(W(\pi ,\psi )\) was not contained in \(W({\mathcal {S}}_c(\pi ),\psi )\), considering a Jordan-Hölder sequence of \(\frac{W({\mathcal {S}}_c(\pi ),\psi )+W(\pi ,\psi )}{W({\mathcal {S}}_c(\pi ),\psi )}\) , there would exist a \(G_n\)-module V such that \(W({\mathcal {S}}_c(\pi ),\psi )\subseteq V \subseteq W({\mathcal {S}}_c(\pi ),\psi )+W(\pi ,\psi )\), and such that \(\tau =\frac{V}{W({\mathcal {S}}_c(\pi ),\psi )}\) is irreducible. If the extension V of \(\tau \) by \({\mathcal {S}}_c(\pi )\) was trivial, then \(\tau \subseteq V \subseteq {\text {Ind}}_{N_n}^{G_n}(\psi )\) would be generic. If nontrivial, by Lemma 3.2, this would also imply that \(\tau \) is generic. This is absurd as \(\tau \) would be a generic subquotient of \(W(\pi ,\psi )\) different from \(W(\pi ^{{{\text {gen}}}},\psi )\), because \(W(\pi ^{{{\text {gen}}}},\psi )\) is contained in \(W({\mathcal {S}}_c(\pi ),\psi )\), contradicting multiplicity 1 for \(W(\pi ,\psi )\). As \({\mathcal {S}}_c(\pi )\) is a standard module, the restriction map \({\text {Res}}_{P_n}\) is injective of on \(W({\mathcal {S}}_c(\pi ),\psi )\), hence on \(W(\pi ,\psi )\). \(\square \)

3.2 The Cogdell–Piatetski–Shapiro interpretation of derivatives

We first recall two important results from [6, Sect. 1]. Then, thanks to those results and those of Sect. 3.1, we can interpret [14, Proposition 9.1] as the inclusion of the Whittaker model of a representation of Whittaker type \(\pi _2\) in the appropriate derivative of the Whittaker model of \(\pi _1\times \pi _2\), for \(\pi _1\) another representation of Whittaker type. Notice that according to [3, Proposition 3.2, (f)], the map \(\Phi ^-\) sends the representation \({\text {Ind}}_{N_n}^{P_n}(\psi )\) surjectively onto \({\text {Ind}}_{N_{n-1}}^{P_{n-1}}(\psi )\). The first result from [6] that we shall need is the following observation, which is an immediate consequence of the proof of [6, Proposition 1.1].

Proposition 3.4

For \(n\geqslant 3\), the map \(\Phi ^{-}:{\text {Ind}}_{N_n}^{P_n}(\psi )\rightarrow {\text {Ind}}_{N_{n-1}}^{P_{n-1}}(\psi )\) identifies with the twisted restriction map \(\nu ^{-\frac{1}{2}}{\text {Res}}_{P_{n-1}}\), where \({\text {Res}}_{P_{n-1}}\) is the restriction of functions to \(P_{n-1}\).

The second result is deeper. It is a consequence of the proof of [6, Proposition 1.6] (see [18, Corollary 2.1] for the precise statement and its proof).

Proposition 3.5

Take \(n\geqslant 2\), and let \(\tau \) be a submodule of \({\text {Ind}}_{N_n}^{P_n}(\psi )\) such that \(\tau ^{(1)}\) has a central character c. If \(W\in \tau \), then for all \(g\in G_{n-1}\), the following limit
$$\begin{aligned}S(W)(g)=\lim _{z\in F^\times ,\ z\rightarrow 0} \ c^{-1}(z)|z|^{\frac{(1-n)}{2}}\nu (g)^{-\frac{1}{2}}W\begin{pmatrix} zg &{} \\ &{} 1\end{pmatrix}\end{aligned}$$
exists (in fact the function of z the limit of which is considered above is constant when z tends to zero). The linear map \(S:\tau \rightarrow {\text {Ind}}_{N_{n-1}}^{G_{n-1}}(\psi )\) descends to \(\tau ^{(1)}\) (i.e. the kernel of S contains that of \(\Psi ^-\), hence induces a linear map on \(\tau ^{(1)}\)), and its descent \({\overline{S}}\) induces an isomorphism of \(G_{n-1}\)-modules between \(\tau ^{(1)}\) and \({\overline{S}}(\tau ^{(1)})\subseteq {\text {Ind}}_{N_{n-1}}^{G_{n-1}}(\psi )\).

In fact, we shall use the above result later. For the moment we rather need the following, which is part of the proof of [18, Corollary 2.1], and is a kind of converse to Proposition 3.5.

Proposition 3.6

Let c be a character of \(Z_{n-1}\) with \(n\geqslant 2\). Define \({\text {Ind}}_{N_n}^{P_n}(\psi )_c\subseteq {\text {Ind}}_{N_n}^{P_n}(\psi )\) to be the \(P_n\)-submodule of \({\text {Ind}}_{N_n}^{P_n}(\psi )\) the elements of which are the functions W such that for all \(g\in G_{n-1}\), the quantity
$$\begin{aligned} f_{c,W,g}(z)=c^{-1}(z)|z|^{\frac{(1-n)}{2}}\nu (g)^{-\frac{1}{2}}W \begin{pmatrix} zg &{} \\ &{} 1\end{pmatrix} \end{aligned}$$
becomes constant for z in a punctured neighbourhood of zero in \(F^\times \). The map
$$\begin{aligned} S:\tau \rightarrow {\text {Ind}}_{N_{n-1}}^{G_{n-1}}(\psi ) \end{aligned}$$
defined by \(S(W)(g)=\underset{z\in F^\times ,\ z \rightarrow 0}{\lim }\ f_{c,W,g}(z)\) descends to \(({\text {Ind}}_{N_n}^{P_n}(\psi )_c)^{(1)}\), and its descent \({\overline{S}}\) induces an isomorphism of \(G_{n-1}\)-modules between \(({\text {Ind}}_{N_n}^{P_n}(\psi )_c)^{(1)}\) and
$$\begin{aligned} S({\text {Ind}}_{N_n}^{P_n}(\psi )_c)={\overline{S}}(({\text {Ind}}_{N_n}^{P_n}(\psi )_c)^{(1)})\subseteq {\text {Ind}}_{N_{n-1}}^{G_{n-1}}(\psi ). \end{aligned}$$

Notice that by [3, Proposition 4.13, (a) and (c)] (which is more precise than the more commonly used [3, Corollary 4.14] describing the subquotients of the Bernstein–Zelevinsky filtration), if \(\pi _1\) and \(\pi _2\) are representations of \(G_{n_1}\) and \(G_{n_2}\) such that \(\pi _1^{(n_1)}\simeq {\mathbb {C}}\), then \(\pi _2\subseteq (\pi _1\times \pi _2)^{(n_1)}\). Here we give another result of this type, which by [12] is equivalent to it if \(\pi _1\) and \(\pi _1\times \pi _2\) are standard modules. It is essentially a reformulation of [14, Proposition 9.1] using the above interpretation of derivatives.

Proposition 3.7

Let \(\pi _1\) and \(\pi _2\) be representations in \({\mathfrak {R}}_{{\text {Whitt}}}(n_1)\) and \({\mathfrak {R}}_{{\text {Whitt}}}(n_2)\) respectively (with \(n_1,n_2\geqslant 1\)), then
$$\begin{aligned} W(\pi _2,\psi )\subseteq W(\pi _1\times \pi _2,\psi )^{(n_1)}. \end{aligned}$$

Proof

Set \(n=n_1+n_2\), the representation \(\pi _1\times \pi _2\) is of Whittaker type, and we denote by \(W(\pi _1\times \pi _2,\psi )\mid _{P_n}\) the representation \(W(\pi _1\times \pi _2,\psi )\) considered as a \(P_n\)-module. By the second part of Proposition 3.3, one has
$$\begin{aligned}W(\pi _1\times \pi _2,\psi )\mid _{P_n}\simeq {\text {Res}}_{P_n}(W(\pi _1\times \pi _2,\psi ))\subseteq {\text {Ind}}_{N_n}^{P_n}(\psi ).\end{aligned}$$
Take \(W_2\in W(\pi _2,\psi )\). By [14, Proposition 9.1], for any \(\phi \in {\mathcal {C}}^\infty _c(F^{n_2})\), there is \(W\in W(\pi ,\psi )\) such that, for any \(p\in P_{n_2+1}\), we have
$$\begin{aligned} W\begin{pmatrix}p &{} \\ &{} I_{n_1-1} \end{pmatrix}=W_2(g(p))\nu (g(p))^{\frac{n_1}{2}}\psi (u(p))\phi (\eta _{n_2}g(p)). \end{aligned}$$
By Proposition 3.4, this means that the map
$$\begin{aligned} W':p\mapsto W_2(g(p))\nu (g(p))^{\frac{1}{2}}\psi (u(p))\phi (\eta _{n_2}g(p)) \end{aligned}$$
belongs to the space
$$\begin{aligned} (\Phi ^-)^{n_1-1}W(\pi _1\times \pi _2,\psi )\subseteq {\text {Ind}}_{N_{n_2+1}}^{P_{n_2+1}}(\psi ). \end{aligned}$$
Notice that by the first part of Proposition 3.3, the representation \(W(\pi _2,\psi )\) has a central character c. If we choose \(\phi \) to be equal to 1 in a neighbourhood of 0 (whatever this neighbourhood is), then \(W'\) belongs to \({\text {Ind}}_{N_{n_2+1}}^{P_{n_2+1}}(\psi )_c\): indeed for \(g\in G_{n_2}\), and \(z\in F^\times \), one has
$$\begin{aligned} W'\begin{pmatrix}zg &{} \\ &{} 1\end{pmatrix}=W_2(zg)\nu (zg)^{\frac{1}{2}}\phi (\eta _{n_2}zg)= c(z)|z|^{\frac{n_2}{2}} W_2(g)\nu (g)^{\frac{1}{2}}\phi (\eta _{n_2}zg), \end{aligned}$$
and the assertion follows by definition of \({\text {Ind}}_{N_{n_2+1}}^{P_{n_2+1}}(\psi )_c\) thanks to our hypothesis on \(\phi \). Moreover, by definition of S, one has \(S(W')=W_2\). According to Proposition 3.6, this means that \(W_2\in W(\pi _1\times \pi _2,\psi )^{(n_1)}\). \(\square \)

3.3 Extension of Whittaker functions

In this section we prove one of the main results of the paper, which is simultaneously a generalization of one part of [6, Corollary of Proposition 1.7] and of [14, Proposition 9.1]. Both these technical results have proved very useful in the study of Rankin-Selberg L-factors. Our generalization will be used in Sect. 4.3 to prove the existence of test vectors for the L-factors that we are interested in.

We recall that if \(\tau \) is a \(P_{r+1}\)-submodule of \({\text {Ind}}_{N_{r+1}}^{P_{r+1}}(\psi )\) with \(r\geqslant 1\), such that \(\tau ^{(1)}\) has a central character, we defined in Proposition 3.5 a map S from \(\tau \) to \({\text {Ind}}_{N_r}^{G_r}(\psi )\), inducing an injection \({\overline{S}}\) of \(\tau ^{(1)}\) in \({\text {Ind}}_{N_r}^{G_r}(\psi )\). We set
$$\begin{aligned} W(\tau ^{(1)},\psi )=S(\tau )={\overline{S}}(\tau ^{(1)}). \end{aligned}$$
This is consistent with our previous notations, as when \(\tau ^{(1)}\) is of Whittaker type, \(W(\tau ^{(1)},\psi )\) is indeed the Whittaker model of \(\tau ^{(1)}\). More generally, for \(1\leqslant k \leqslant r\), then \((\Phi ^{-})^{k-1}\tau \) is naturally a subspace of \({\text {Ind}}_{N_{r+2-k}}^{P_{r+2-k}}(\psi )\) thanks to Proposition 3.4, and if \(\tau ^{(k)}\) has a central character, then \({\overline{S}}\) induces an embedding from \(\tau ^{(k)}= \Psi ^-(\Phi ^{-})^{k-1}\tau \) into \({\text {Ind}}_{N_{r+1-k}}^{G_{r+1-k}}(\psi )\), and we set
$$\begin{aligned} W(\tau ^{(k)},\psi )=S((\Phi ^{-})^{k-1}\tau )={\overline{S}}(\tau ^{(k)}). \end{aligned}$$

Proposition 3.8

Let \(\tau \) be a \(P_{r+1}\)-submodule of \({\text {Ind}}_{N_{r+1}}^{P_{r+1}}(\psi )\) with \(r\geqslant 1\), such that \(\tau ^{(1)}\) has a central character. Then for any \(W_0\in W(\tau ^{(1)},\psi )\), and any \(\phi \in {\mathcal {C}}_c^\infty (F^r)\), there is \(W\in \tau \) such that, for any \(g\in G_r\),
$$\begin{aligned} W\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}\phi (\eta _r g)=\delta _{P_{r+1}}^{\frac{1}{2}} (g)W_0(g)\phi (\eta _r g). \end{aligned}$$

Proof

The proof is inspired by the proof of [18, Proposition 3.2]. We denote by c the central character of \(\tau ^{(1)}\) and set \(\rho =\tau ^{(1)}\). Take \(W_1\in \tau \) such that \(S(W_1)=W_0\), in particular \(S(\tau (g)W_1)=\rho (g)W_0\) for all g in \(G_r\) (notice that both \(\rho \) and \(\tau \) act by right translation). For \(t_1,\dots ,t_r \in F^\times \), we will denote by t the element
$$\begin{aligned} t=\begin{pmatrix} t_1\dots t_r &{} &{} &{} \\ &{} t_2\dots t_r &{} &{} \\ &{} &{} \ddots &{} \\ &{} &{} &{} t_r \end{pmatrix}\in G_r, \end{aligned}$$
and set
$$\begin{aligned} t'=t_r^{-1}t \in G_r. \end{aligned}$$
We choose representatives \(k_1,\dots ,k_l\) of \(K_r/U\) with \(k_1=1\), where U is a compact open subgroup of \(K_r\) fixing \(W_1\) on the right. By the claim in Theorem 2.1 of [17] (or the arguments in the claim of Proposition 1.6 of [6]), there is \(N_i\in {\mathbb {N}}\) such that
$$\begin{aligned} \tau (k_i)W_1\begin{pmatrix} ta &{} \\ &{} 1 \end{pmatrix}=c(a)\nu (a)^{\frac{1}{2}}\tau (k_i)W_1\begin{pmatrix} t &{} \\ &{} 1 \end{pmatrix} \end{aligned}$$
for any \(t_1,\dots , t_{r-1} \in F^\times \), \(a\in F^\times \) such that \(|a|\leqslant 1\) and \(t_r\in F^\times \) such that \(|t_r| \leqslant q^{-N_i}\). We can choose all \(N_i\)’s independent of i, say equal to an integer N, as there is a finite number of them. We set \(z_b={\text {diag}}(b,\dots ,b,1)\in P_{r+1}\) and
$$\begin{aligned} W_b=\frac{\tau (z_b)W_1}{c(b)|b|^{\frac{r}{2}}}. \end{aligned}$$
Notice that for all \(i=1,\dots ,l\), the functions
$$\begin{aligned} \frac{(\tau (k_i)W_b)\begin{pmatrix} t &{} \\ &{} 1 \end{pmatrix}}{c(t_r)|t_r|^{\frac{r}{2}}} =\frac{\tau (k_i)W_1\begin{pmatrix} t't_rb &{} \\ &{} 1 \end{pmatrix}}{c(t_rb)|t_rb|^{\frac{r}{2}}} \end{aligned}$$
are constant with respect to \(t_r\) for \(|t_rb|\leqslant 1 \Leftrightarrow |t_r|\leqslant q^{-N}/|b|\). We write
$$\begin{aligned} \frac{1}{c(t_r)|t_r|^{\frac{r}{2}}}\tau (k_i)W_b\begin{pmatrix} t &{} \\ &{} 1 \end{pmatrix} = \frac{\nu (t')^{1/2}}{c(t_rb)|t_rb|^{\frac{r}{2}}\nu (t')^{1/2}}\tau (k_i)W_1\begin{pmatrix} t't_rb &{} \\ &{} 1 \end{pmatrix} \end{aligned}$$
and observe that \(S(\tau (k_i)W_1)=\rho (k_i)W_0.\) Hence by Proposition 3.5, making \(z=t_rb\rightarrow 0\) we get:
$$\begin{aligned} \frac{(\tau (k_i)W_b)\begin{pmatrix} t &{} \\ &{} 1 \end{pmatrix}}{c(t_r)|t_r|^{\frac{r}{2}}}&= \nu (t')^{1/2}\rho (k_i)W_0(t'), \end{aligned}$$
this equality being valid for all \(t_r\) such that \(|t_r|\leqslant q^{-N}/|b|\). So for such \(t_r\):
$$\begin{aligned} (\tau (k_i)W_b)\begin{pmatrix} t &{} \\ &{} 1 \end{pmatrix}=|t_r|^{\frac{r}{2}}\nu (t')^{1/2}c(t_r)\rho (k_i)W_0(t') =\nu (t)^{1/2}\rho (k_i)W_0(t). \end{aligned}$$
By the Iwasawa decomposition of \(G_r\), we deduce that
$$\begin{aligned} W_b\begin{pmatrix}g &{} \\ &{} 1 \end{pmatrix}=\nu (g)^{1/2}W_0(g) \end{aligned}$$
for all \(g\in G_r\) such that \(\nu (z(g))^{1/r}\leqslant q^{-N}/|b| \). But we can take b such that \(q^{-N}/|b|\) is as large as we need, i.e. such that \(\eta _r g\) belongs to \(({{\mathfrak {p}}}^{-l})^r\) for l as large as we want. In particular for \(\phi \in {\mathcal {C}}_c^\infty (F^r)\), one can take l large enough for \(({{\mathfrak {p}}}^{-l})^r\) to contain the support of \(\phi \), and this proves the claim. \(\square \)

After fixing a Haar measure dx on \(F^r\), we denote by \({\widehat{\phi }}\) the Fourier transform of \(\phi \in {\mathcal {C}}_c^\infty (F^r)\) with respect to dx and the character \(\psi \otimes \dots \otimes \psi \) of \(F^n\).

Corollary 3.9

With same notations as in the Proposition 3.8, for any \(W_0\in W(\tau ^{(1)},\psi )\), and any \(\phi \in {\mathcal {C}}_c^\infty (F^r)\), there is \(W'\in \tau \) such that, for all \(g\in G_r\),
$$\begin{aligned} W'\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}=\nu (g)^{\frac{1}{2}}W_0(g)\phi (\eta _r g). \end{aligned}$$

Proof

Take W as in the statement of Proposition 3.8. Let \(u(x)=\left( {\begin{matrix} I_r &{} x \\ 0 &{} 1\end{matrix}}\right) \). Let \(\alpha \) be the element of \({\mathcal {C}}_c^\infty (F^r)\) is such that \({\widehat{\alpha }}=\phi \), then if one sets \(W'(p)=\int _{x\in F^n} \alpha (x)W(pu(x))dx\) for \(p\in P_{r+1}\), the map \(W'\) belongs to \(\tau \) and
$$\begin{aligned} W'\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}=W\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}{\widehat{\alpha }}(\eta _r g) \end{aligned}$$
for \(g\in G_r\). The statement now follows from Proposition 3.8. \(\square \)

We thus obtain the following important result. As we said before, it is obviously a generalization of one part of [6, Corollary of Proposition 1.7], but it is also a generalization of [14, Proposition 9.1] thanks to Proposition 3.7 and Proposition 3.3.

Theorem 3.10

Let \(\tau \) be a submodule of \({\text {Ind}}_{N_n}^{P_n}(\psi )\), with \(n\geqslant 2\), and let \(1\leqslant k \leqslant n-1\) be such that \(\rho =\tau ^{(k)}\) has a central character. Then for any \(W_0\in W(\rho ,\psi )\) and \(\phi \in {\mathcal {C}}_c^\infty (F^{n-k})\), there is \(W\in \tau \) such that
$$\begin{aligned} W\begin{pmatrix} g &{} \\ &{} I_k \end{pmatrix}= \nu (g)^{\frac{k}{2}} W_0(g)\phi (\eta _{n-k} g) \end{aligned}$$
for all \(g\in G_{n-k}\).

Proof

According to Corollary 3.9, there is \(W'\in (\Phi ^-)^{(k-1)}(\tau )\) such that for \(g\in G_{n-k}\):
$$\begin{aligned} W'\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}=\nu (g)^{\frac{1}{2}} W_0(g)\phi (\eta _{n-k} g). \end{aligned}$$
Now, Proposition 3.4 repeated \(k-1\) times gives the existence of \(W\in \tau \) such that
$$\begin{aligned} W\begin{pmatrix} g &{} \\ &{} I_{k} \end{pmatrix}= \nu (g)^{\frac{k-1}{2}}W'\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix} \end{aligned}$$
for \(g\in G_{n-k}\), the statement follows. \(\square \)

4 Test vectors for L-factors of pairs of discrete series

The aim of the second part of this paper is to show that \(L(s,{\text {St}}_l(\rho ),{\text {Sp}}_k(\rho '))=L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho '))\) is given by a single Rankin-Selberg integral. We first recall their definitions.

4.1 L-factors for pairs of discrete series

All results of this section are fundamental facts from [14]. We normalize the Haar measure on \(G_n\) to give volume 1 to \(K_n\), and on any closed subgroup H of \(G_n\) we normalize the Haar measure to give volume 1 to \(H\cap K_n\). If H is unimodular, this then defines a unique nonzero right invariant measure on \(H\backslash G_n\). We consider \(\pi \in {\mathfrak {R}}_{{\text {Whitt}}}(n)\) and \(\pi '\in {\mathfrak {R}}_{{\text {Whitt}}}(m)\) with \(n\geqslant m\geqslant 1\).

If \(n=m\), then for \(\phi \in {\mathcal {C}}_c^\infty (F^n)\), \(W\in W(\pi ,\psi )\) and \(W'\in W(\pi ',\psi ^{-1})\), we define for \(s\in {\mathbb {C}}\) the Rankin-Selberg integral:
$$\begin{aligned} I_n(s,W,W',\phi )=\int _{N_n\backslash G_n} W(g)W'(g)\phi (\eta _n g)\nu (g)^sdg. \end{aligned}$$
It is absolutely convergent for \(\mathrm {Re}(s)\) larger than a real number depending only on \(\pi \) and \(\pi '\), and extends to an element of \({\mathbb {C}}(q^{-s})\). Moreover, letting \(W,W'\) and \(\phi \) vary, the subspace of \({\mathbb {C}}(q^{-s})\) spanned by the integrals \(I_n(s,W,W',\phi )\) is equal to \(L(s,\pi ,\pi '){\mathbb {C}}[q^{\pm s}]\) for a unique Euler factor \(L(s,\pi ,\pi ')\), which is called the L-factor of \(\pi \) and \(\pi '\).
If \(n>m\), then for \(W\in W(\pi ,\psi )\) and \(W'\in W(\pi ',\psi ^{-1})\), we define for \(s\in {\mathbb {C}}\) the Rankin-Selberg (or Hecke) integral:
$$\begin{aligned} I_{n,m}(s,W,W')=\int _{N_m\backslash G_m} W\begin{pmatrix}g &{} \\ &{} I_{n-m} \end{pmatrix}W'(g)\nu (g)^{s-\frac{n-m}{2}}dg. \end{aligned}$$
It is absolutely convergent for \(\mathrm {Re}(s)\) larger than a real number depending only on \(\pi \) and \(\pi '\), and extends to an element of \({\mathbb {C}}(q^{-s})\). Moreover, letting W and \(W'\) vary, the subspace of \({\mathbb {C}}(q^{-s})\) spanned by the integrals \(I_{n,m}(s,W,W')\) is equal to \(L(s,\pi ,\pi '){\mathbb {C}}[q^{\pm s}]\) for a unique Euler factor \(L(s,\pi ,\pi ')\), which is again called the L-factor of \(\pi \) and \(\pi '\). We set \(L(s,\pi ',\pi )=L(s,\pi ,\pi ')\).

By definition, following the authors of [14] again, if \(\pi \) and \(\pi '\) belong respectively to \({\text {Irr}}(n)\) and \({\text {Irr}}(m)\), then the Whittaker models \(W({\mathcal {S}}(\pi ),\psi )\) and \(W({\mathcal {S}}(\pi '),\psi ^{-1})\) are uniquely determined by \(\pi \) and \(\pi '\). We set \(L(s,\pi ,\pi ')=L(s,{\mathcal {S}}(\pi ),{\mathcal {S}}(\pi '))\) where \({\mathcal {S}}(\pi )\) and \({\mathcal {S}}(\pi ')\) are the standard modules over \(\pi \) and \(\pi '\).

As explained in the introduction, we will only be interested in the L-factors of the form \(L(s,\delta ,{}^{t}\delta ')\) for \(\delta \) and \(\delta '\) in \({\text {Irr}}_{{{\text {disc}}}}\). We recall from Sect. 2 that \(\delta \) is of the form \({\text {St}}_l(\rho )\) for \(\rho \in {\text {Irr}}_{{{\text {cusp}}}}\), similarly \(\delta '\) is of the form \({\text {St}}_k(\rho ')\) for \(\rho '\in {\text {Irr}}_{{{\text {cusp}}}}\), hence \({}^{t}\delta '={\text {Sp}}_k(\rho ')\) and \({\mathcal {S}}({\text {Sp}}_k(\rho '))={\mathcal {S}}_k(\rho ')\). As \(\delta ={\mathcal {S}}(\delta )\) and \(\delta '={\mathcal {S}}(\delta ')\), we have
$$\begin{aligned} L(s,\delta ,\delta ')=L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho ')) \end{aligned}$$
whereas
$$\begin{aligned} L(s,\delta ,{}^{t}\delta ')=L(s,{\text {St}}_l(\rho ),{\mathcal {S}}_k(\rho ')). \end{aligned}$$
We now notice that
$$\begin{aligned} L(s,\delta ,{}^{t}\delta ')=L(s,\delta ,\delta '), \end{aligned}$$
so our integral will also compute \(L(s,\delta ,\delta ')\). This observation is a consequence of [14, Proposition 8.1, Theorem 8.2 and Proposition 9.4].

Proposition 4.1

For \(1\leqslant k\leqslant l\) and \(\rho \) and \(\rho '\) in \({\text {Irr}}_{{{\text {cusp}}}}\), the L factor \(L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho '))\) is equal to 1 unless there is \(s_0\in {\mathbb {C}}\) such that \(\rho '\simeq \nu ^{s_0}\rho ^\vee \). In the case \(\rho '\simeq \nu ^{s_0}\rho ^\vee \), one has the equalities
$$\begin{aligned} L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho '))&=\prod _{i=1}^k L(s+s_0,\nu ^{\frac{l-1}{2}}\rho ,\nu ^{\frac{k+1-2i}{2}}\rho ^\vee )\\&=L(s,{\text {St}}_l(\rho ),{\mathcal {S}}_k(\rho ')) \\~&=L(s,{\text {St}}_l(\rho ),{\text {Sp}}_k(\rho ')). \end{aligned}$$

Remark 4.2

If \(\rho \) and \(\rho '\) are not equal up to an unramified twist, then it is not new that the Rankin-Selberg L-factor \(L(s,{\text {St}}_l(\rho ),{\mathcal {S}}_k(\rho '))=1\) is given by a single integral: this follows from the proof of [14, Theorem 2.7]. If they are equal up to unramified twist, it is enough to show the test vector result for \(\rho '=\rho ^\vee \).

Following Remark 4.2, henceforth we can and will assume that \(\rho '=\rho ^\vee \). We will also suppose that \(\psi \) has conductor 0, i.e. is trivial on \({\mathfrak {o}}\), but not on \({{\mathfrak {p}}}^{-1}\).

Let r be the integer such that \(\rho \in {\text {Irr}}_{{{\text {cusp}}}}(r)\). If \(l=k\) we will find \(W\in W({\text {St}}_l(\rho ),\psi )\), \(W'\in W({\mathcal {S}}_k(\rho ^\vee ),\psi ^{-1})\), and \(\phi \in {\mathcal {C}}_c^\infty (F^n)\) such that \(I_{lr}(s,W,W',\phi )=L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho ^\vee ))\), whereas if \(l>k\), we will find \(W\in W({\text {St}}_l(\rho ),\psi )\) and \(W'\in W({\mathcal {S}}_k(\rho ^\vee ),\psi ^{-1})\) such that \(I_{lr,kr}(s,W,W')=L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho ^\vee ))\). Hence it is fair to say that we find test vectors for \(L(s,{\text {St}}_l(\rho ),{\text {Sp}}_k(\rho ^\vee ))\) rather then \(L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho ^\vee ))\) although the L-factors are equal.

Taking \(W'\in W({\mathcal {S}}_k(\rho ^\vee ),\psi ^{-1})\) makes things simpler, as the space \(W({\mathcal {S}}_k(\rho ^\vee ),\psi ^{-1})\) contains \(W({\text {St}}_k(\rho ^\vee ),\psi ^{-1})\) as a proper subspace. We believe that it is possible to take \(W'\in W({\text {St}}_k(\rho ^\vee ),\psi ^{-1})\), but it seems much more difficult to us. To justify and motivate the fact that allowing to take \(W\in W({\mathcal {S}}_k(\rho ),\psi ^{-1})\) simplifies matters, we start with the toy example \(\rho ={\mathbf {1}}\).

4.2 The case of Steinberg representations

Here \(\rho ={\mathbf {1}}\) (the trivial character of \(G_1\)). We set \({\mathcal {S}}_k={\mathcal {S}}_k({\mathbf {1}})\), \({\text {St}}_k={\text {St}}_k({\mathbf {1}})\), and \({\text {Sp}}_k={\text {Sp}}_k({\mathbf {1}})={\mathbf {1}}_{G_k}\). For \(k=2\), by [15, Proposition 5.3.7], there are \(W\in W({\text {St}}_2,\psi )\) and \(W'\in W({\text {St}}_2,\psi ^{-1})\), and \(\phi \in {\mathcal {C}}_c^\infty (F^2)\) such that \(I_2(s,W,W',\phi )=L(s,{\text {St}}_2,{\text {St}}_2)\), and it is in fact shown in [ibid.] that one can take W and \(W'\) to be the essential vectors in \(W({\text {St}}_2,\psi )\) and \(W({\text {St}}_2,\psi ^{-1})\). The proof of this result is already quite technical. Notice that \(W({\mathcal {S}}_k,\psi ^{-1})\) is spherical (contains \(K_n\)-fixed vectors), whereas the smaller space \(W({\text {St}}_k,\psi ^{-1})\) is not spherical when \(k\geqslant 2\). Hence we choose \(W_k^0\) to be the normalized spherical vector in \(W({\mathcal {S}}_k,\psi ^{-1})\). On the other hand, in \(W({\text {St}}_k,\psi )\), we choose the essential vector \(W_k^{{\text {ess}}}\) of [13] (see also [9, 18] and [19]). We fix \(l\geqslant k \geqslant 1\), and moreover as for \(l=k=1\) one has \(W_1^{{\text {ess}}}=W_1^0={\mathbf {1}}\), Tate’s thesis gives us the equality
$$\begin{aligned} I_1(s,W_1^{{\text {ess}}}, W_1^0,{\mathbf {1}}_{\mathfrak {o}})=L(s,{\text {St}}_1,{\text {St}}_1), \end{aligned}$$
we suppose that \(l\geqslant 2\).

If \(l>k\), then Corollary 3.3 of [18] immediately gives the existence of test vectors.

Proposition 4.3

If \(l>k\geqslant 1\) then
$$\begin{aligned}I_{l,k}(s,W_l^{{\text {ess}}}, W_k^0)=L(s,{\text {St}}_l,{\text {Sp}}_k)=L(s,{\text {St}}_l,{\text {St}}_k).\end{aligned}$$
We now consider the case \(l=k\geqslant 2\). Take \(\phi \) a Schwartz function on \(F^l\) of the form
$$\begin{aligned} \phi =\lambda {\mathbf {1}}_{({{\mathfrak {p}}}^f)^{l-1} \times (1+{{\mathfrak {p}}}^f)}, \end{aligned}$$
with f sufficiently large, and \(\lambda =\mu (1+{{\mathfrak {p}}}^f)^{-1}\) for \(d\mu \) the normalized (see the beginning of Sect. 4.1) Haar measure on \(F^\times \), then
$$\begin{aligned} I_l(s,W_l^{{\text {ess}}},W_l^0,\phi )= \int _{N_{l-1}\backslash G_{l-1}}W_l^{{\text {ess}}}\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix} W_l^0\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}\nu (g)^{s-1}dg. \end{aligned}$$
The integral on the right above (a formula which makes sense only for \(\mathrm {Re}(s)\) large) is also considered as a rational function of \(q^{-s}\).
In [18, Definition 1.3], to each \(\pi \in {\text {Irr}}_{{{\text {gen}}}}\), one attaches an unramified standard module \(\pi _u\) as follows: let \(\{{\text {St}}_{k_i}(\mu _i)\}_{i=1,\dots ,b}\) be the multiset of Steinberg representations with \(\mu _i\) an unramified character of \(F^\times \) occurring as factors of the product of discrete series \(\pi \) (see Sect. 2) numbered such that
$$\begin{aligned} e(\nu ^{\frac{k_1-1}{2}}\mu _1)\ge \dots \ge e(\nu ^{\frac{k_b-1}{2}}\mu _b), \end{aligned}$$
then
$$\begin{aligned} \pi _u:=\nu ^{\frac{k_1-1}{2}}\mu _1\times \dots \times \nu ^{\frac{k_b-1}{2}}\mu _b. \end{aligned}$$
For \(\pi ={\text {St}}_l\), one gets: \(({\text {St}}_l)_u=\nu ^{\frac{l-1}{2}}\). We write \(W_{({\text {St}}_l)_u}^0=\nu ^{\frac{l-1}{2}}\) for its normalized spherical vector. We recall the formula for \(W_l^{{\text {ess}}}\) given in [18, Corollary 3.2]. Take \(n\in N_{l-1}\), \(a={\text {diag}}(a_1,\dots ,a_{l-1})\in G_{l-1}\subseteq G_l\), and \(k\in K_{l-1}\subseteq G_{l-1}\subseteq G_l\):
$$\begin{aligned} W_l^{{\text {ess}}}\begin{pmatrix} nak &{} \\ &{} 1 \end{pmatrix}=\psi (n)\nu (a_1)^{l-1}{\mathbf {1}}_{\mathfrak {o}}(a_1)\prod _{i=2}^{l-1}{\mathbf {1}}_{{\mathfrak {o}}^\times }(a_i). \end{aligned}$$
For H a closed subgroup of \(G_l\), we denote by \(\delta _H\) its modulus character which satisfies that if dh is a right invariant Haar measure on H, then for any \(f\in {\mathcal {C}}_c^\infty (H)\), and any \(x\in H\): \(dh (f(x^{-1} \ .))=\delta _H(x)dh(f)\). Now we do computations similar to those in the proof of [18, Corollary 3.3, Case \(m=r\)].
$$\begin{aligned} \int _{N_{l-1}\backslash G_{l-1}}&W_l^{{\text {ess}}}\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}W_l^0\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}\nu (g)^{s-1}dg\\&=\int _{(F^\times )^{l-1}}W_l^0\begin{pmatrix} a &{} \\ &{} 1 \end{pmatrix}|a_1|^{s+l-2}{\mathbf {1}}_{\mathfrak {o}}(a_1)\prod _{i=2}^{l-1}{\mathbf {1}}_{{\mathfrak {o}}^\times }(a_i)|a_i|^{s-1}\delta _{B_{l-1}}^{-1}(a)\prod _{i=1}^{l-1} d_{F^\times }a_i\\&=\int _{G_1}W_l^0\begin{pmatrix} a_1 &{} \\ &{} I_{l-1} \end{pmatrix}|a_1|^{s+l-2}{\mathbf {1}}_{\mathfrak {o}}(a_1)\delta _{B_{l-1}}^{-1}\begin{pmatrix} a_1 &{} \\ &{} I_{l-2}\end{pmatrix}d_{F^\times }a_1 \\&=\int _{G_1}W_l^0\begin{pmatrix} a_1 &{} \\ &{} I_{l-1} \end{pmatrix}|a_1|^{s}d_{F^\times }a_1\\&\text {(because }W_l^0(a_1)=W_l^0(a_1){\mathbf {1}}_{\mathfrak {o}}(a_1)\text { by }[21])\\&=I_{l,1}(s,W_l^0,\nu ^{\frac{l-1}{2}})=\prod _{i=1}^{l} L\left( s,\nu ^{\frac{1}{2}},\nu ^{\frac{l+1-2i}{2}}\right) \\ {}&=L(s,{\text {St}}_l,{\text {Sp}}_l)=L(s,{\text {St}}_l,{\text {St}}_l), \end{aligned}$$
the antepenultimate equality follows from [11, Proposition 2.3], and [10, Sect. 1, equality (3)] (or more precisely their immediate extensions to standard modules as explained in the discussion of [18, p. 1201]), and the two last equalities by Proposition 4.1. Hence we just proved:

Proposition 4.4

For \(l\geqslant 2\), then for f large enough, and a well-chosen multiple \(\phi \) of \({\mathbf {1}}_{({{\mathfrak {p}}}^f)^{l-1} \times (1+{{\mathfrak {p}}}^f)}\), we have
$$\begin{aligned} I_l(s,W_l^{{\text {ess}}},W_l^0,\phi )= L(s,{\text {St}}_l,{\text {Sp}}_l)= L(s,{\text {St}}_l,{\text {St}}_l). \end{aligned}$$

4.3 The general case

In this section, for \(l\geqslant k\geqslant 1\), we find test vectors for \(L(s,{\text {St}}_l(\rho ),{\text {Sp}}_k(\rho ^{\vee }))\) by reducing the problem to the known case of pairs of cuspidal representations ( [16, Theorem 9.1]) thanks to Theorem 3.10.

We fix \(\rho \in {\text {Irr}}_{{{\text {cusp}}}}(r)\) for \(r\geqslant 1\). We set \(n=lr\) and \(m=kr\). By [3, Theorem 4.4 and Lemma 4.5] and [23, Proposition 9.6], using moreover that representations of \(G_r\) with different central characters are always in direct sum, we have the formulae
$$\begin{aligned} {\mathcal {S}}_k(\rho ^\vee )^{((k-1)r)}&\simeq \nu ^{\frac{1-k}{2}}\rho ^\vee \oplus \cdots \oplus \nu ^{\frac{k-1}{2}}\rho ^\vee , \end{aligned}$$
(1)
$$\begin{aligned} {\text {St}}_k(\rho ^\vee )^{((k-1)r)}&\simeq \nu ^{\frac{k-1}{2}}\rho ^\vee . \end{aligned}$$
(2)
Let us first treat separately the easy case \(k=1\). Then \({\mathcal {S}}_1(\rho ^\vee )={\text {St}}_1(\rho ^\vee )=\rho ^\vee \). In this case, by [16, Theorem 9.1], there is \(W^{{{\text {cusp}}}}\in W(\nu ^{\frac{l-1}{2}}\rho ,\psi )\), \(V^{{{\text {cusp}}}}\in W(\rho ^{\vee },\psi ^{-1})\), and \(\phi \in {\mathcal {C}}_c^\infty (F^r)\) such that
$$\begin{aligned} I_r(s,W^{{{\text {cusp}}}},V^{{{\text {cusp}}}},\phi )=L(s,\nu ^{\frac{l-1}{2}}\rho ,\rho ^\vee )=L(s,{\text {St}}_l(\rho ),\rho ^\vee ) \end{aligned}$$
(3)
Now by Theorem 3.10 applied to \({\text {Res}}_{P_n}(W({\text {St}}_l(\rho ),\psi ))\) ( [14, Proposition 9.1] is in fact sufficient here), there is \(W_l\in W({\text {St}}_l(\rho ),\psi )\) such that
$$\begin{aligned}W_l\begin{pmatrix} g &{} \\ &{} I_{n-r}\end{pmatrix}=\nu (g)^{\frac{n-r}{2}}W^{{{\text {cusp}}}}(g)\phi (\eta _r g)\end{aligned}$$
for all \(g\in G_r\). The following proposition follows at once from Eq. (3).

Proposition 4.5

For \(l\geqslant 1\), there is \(W_l\in W({\text {St}}_l(\rho ),\psi )\) and \(V^{{{\text {cusp}}}}\in W(\rho ^\vee ,\psi ^{-1})\) (as above), such that
$$\begin{aligned} I(s,W_l,V^{{{\text {cusp}}}})=L(s,{\text {St}}_l(\rho ),\rho ^\vee ). \end{aligned}$$
Hence now we suppose that \(k\geqslant 2\) until the end of the paper. According to [16, Theorem 9.1] again, one can find \(W^{{{\text {cusp}}}}\in W(\nu ^{\frac{l-1}{2}}\rho ,\psi )\), \(V_i^{{{\text {cusp}}}}\in W(\nu ^{\frac{1-k+2i}{2}}\rho ^\vee ,\psi ^{-1})\) (one can in fact take \(V_i^{{{\text {cusp}}}}=\nu ^{\frac{1-k+2i}{2}}V^{{{\text {cusp}}}}\) for \(V^{{{\text {cusp}}}}\) as above) for each i between 0 and \(k-1\), and \(\phi \in {\mathcal {C}}_c^\infty (F^r)\) such that
$$\begin{aligned} I_r(s,W^{{{\text {cusp}}}},V_i^{{{\text {cusp}}}},\phi )=L(s,\nu ^{\frac{l-1}{2}}\rho ,\nu ^{\frac{1-k+2i}{2}}\rho ^\vee ). \end{aligned}$$
(4)
By Theorem 3.10 and Eq. (1), we deduce that for each i between 0 and \(k-1\), there is \(V_i\in W({\mathcal {S}}_k(\rho ^\vee ),\psi ^{-1})\) such that
$$\begin{aligned} I_r(s,W^{{{\text {cusp}}}},V_i^{{{\text {cusp}}}},\phi )=I_{kr,r}(s,V_i,W^{{{\text {cusp}}}}), \end{aligned}$$
which together with Eq. (4) yields
$$\begin{aligned} I_{kr,r}(s,V_i,W^{{{\text {cusp}}}})=L\left( s,\nu ^{\frac{l-1}{2}}\rho ,\nu ^{\frac{1-k+2i}{2}}\rho ^\vee \right) . \end{aligned}$$
(5)
We recall that if d is the cardinality of the group \(R(\rho )\) of unramified characters fixing \(\rho \), and denoting by \(\varpi \) a uniformizer of F, then \(\chi \mapsto \chi (\varpi )\) is an isomorphism between \(R(\rho )\) and the group of d-th roots of unity in \({\mathbb {C}}^\times \) (notice that [4, Lemma 6.2.5] gives an “arithmetical” description of d as \(d=r/e\) where e is the ramification index of \(\rho \) in the sense of type theory). In particular
$$\begin{aligned} \prod _{\chi \in R(\rho )}(1-\chi (\varpi )X)=1-X^d. \end{aligned}$$
Setting \(X=q^{-s}\), we have
$$\begin{aligned} L\left( s,\nu ^{\frac{l-1}{2}}\rho ,\nu ^{\frac{1-k+2i}{2}}\rho ^\vee \right) = L\left( s+\frac{l-k+2i}{2},\rho ,\rho ^\vee \right) , \end{aligned}$$
which according to [14, Proposition 8.1] and the discussion above is equal to:
$$\begin{aligned} \frac{1}{1-q^{\frac{(k-l-2i)d}{2}}X^d}. \end{aligned}$$
But as a function of \(Y=X^d\), one has
$$\begin{aligned} L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho ^\vee ))=\prod _{0=1}^{k-1} \frac{1}{1-q^{\frac{(k-l-2i)d}{2}}Y}, \end{aligned}$$
and, in particular, it has simple poles. We can thus write its partial fraction decomposition, and find (explicit) \(\lambda _i\in {\mathbb {C}}\) such that
$$\begin{aligned} \nonumber L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho ^\vee ))&=\sum _{i=0}^{k-1}\frac{\lambda _i}{1-q^{\frac{(k-l-2i)d}{2}}Y}= \sum _{i=0}^{k-1}\lambda _iL\left( s,\nu ^{\frac{l-1}{2}}\rho ,\nu ^{\frac{1-k+2i}{2}}\rho ^\vee \right) \\&=\sum _{i=0}^{k-1} \lambda _i I_{kr,r}(s, V_i,W^{{{\text {cusp}}}})=I_{kr,r}\left( s,\sum _{i=0}^{k-1} \lambda _i V_i,W^{{{\text {cusp}}}}\right) , \end{aligned}$$
(6)
the second to last equality thanks to Eq. (5).

Set \(n=lr\) and \(m=kr\), we are now in position to prove the second and last main result of the paper.

Theorem 4.6

If \(l>k\geqslant 2\), there is \(W_l\) in \(W({\text {St}}_l(\rho ),\psi )\), and \(W'_k\in W({\mathcal {S}}_k((\rho ^\vee )),\psi ^{-1})\), such that
$$\begin{aligned} I_{n,m}(s,W_l,W'_k)=L(s,{\text {St}}_l(\rho ),{\text {Sp}}_k(\rho ^\vee ))=L(s,{\text {St}}_l(\rho ),{\text {St}}_k(\rho ^\vee )). \end{aligned}$$
If \(l=k\geqslant 2\), there is \(W_l\) in \(W({\text {St}}_l(\rho ),\psi )\), \(W'_l\in W({\mathcal {S}}_l(\rho ^\vee ),\psi ^{-1})\), and \(\phi \in {\mathcal {C}}_c^\infty (F^n)\), such that
$$\begin{aligned} I_{n}(s,W_l,W'_l,\phi )=L(s,{\text {St}}_l(\rho ),{\text {Sp}}_l(\rho ^\vee ))=L(s,{\text {St}}_l(\rho ),{\text {St}}_l(\rho ^\vee )). \end{aligned}$$
Moreover one can always choose \(\phi ={\mathbf {1}}_{({{\mathfrak {p}}}^f)^{n-1} \times (1+{{\mathfrak {p}}}^f)}\) for f large enough.

Proof

Let’s first deal with the \(l>k\) case. Set \(W^{{\text {cusp}}}\in W(\nu ^{\frac{l-1}{2}}\rho ,\psi )\) and \(\phi \in {\mathcal {C}}_c^\infty (F^r)\) as in Eq. (3). By Theorem 3.10, we can find \(W\in W({\text {St}}_l(\rho ),\psi )\) such that
$$\begin{aligned} W\begin{pmatrix} g &{} \\ &{} I_{n-r}\end{pmatrix}=\nu (g)^{\frac{n-r}{2}} W^{{{\text {cusp}}}}(g)\phi (\eta _r g) \end{aligned}$$
(7)
for \(g\in G_r\). We write \({\mathcal {M}}_{m-r,r}\) for the additive group of \(m-r\times m\) matrices with coefficients in F, and \(B_{m-r}^-\) for the subgroup of lower triangular matrices in \(G_{m-r}\). Then by [14, Lemma 9.2], there is \(W'\in W({\text {St}}_l(\rho ),\psi )\) such that
$$\begin{aligned} W'\begin{pmatrix} g &{} &{} \\ x &{} h &{} \\ &{} &{} I_{n-m} \end{pmatrix}=W\begin{pmatrix} g &{} \\ &{} &{} I_{n-r} \end{pmatrix} \end{aligned}$$
(8)
for x and h in open compact subgroups \(U_1\)and \(U_2\) of \({\mathcal {M}}_{m-r,r}\) and \(B_{m-r}^-\) respectively that we can choose as small as we like, and is equal to zero for x or h outside of those subgroups. We choose \(U_1\) and \(U_2\) small enough such that matrices \(\left( {\begin{matrix} I_r &{} \\ x &{} h \end{matrix}}\right) \) for \(x\in U_1\) and \(h\in U_2\) fix all functions \(V_i\) by right translation. The following integration formula for positive measurable functions F on \(N_m\backslash G_m\)
$$\begin{aligned} \int _{N_m \backslash G_m} F(g) dg = \int _{g\in N_r\backslash G_r, x\in {\mathcal {M}}_{m-r,r}, h\in B_{m-r}^-} F\begin{pmatrix} g &{} \\ x &{} h \end{pmatrix}\nu (g)^{(r-m)} dx d_rh dg \end{aligned}$$
(9)
is valid for an appropriate right Haar measure \(d_rh\) on \(B_{m-r}^-\). In combination with Eqs. (7) and (8), it gives the equality
$$\begin{aligned} I_{n,m}(s,W',V_i)=I_{m,r}(s,V_i,W^{{{\text {cusp}}}}). \end{aligned}$$
(10)
Hence setting
$$\begin{aligned} V'=\sum _{i=0}^{k-1} \lambda _i V_i\in W({\mathcal {S}}_k(\rho ^\vee ),\psi ^{-1}), \end{aligned}$$
Equations (6) and (10) together imply the equality
$$\begin{aligned} I_{n,m}(s,W',V')= L(s,{\text {St}}_l(\rho ),{\text {Sp}}_k(\rho ^\vee )). \end{aligned}$$
We are thus done in this case. Notice that our choices for \(W'\) and \(V'\) are highly non-canonical.
It remains to deal with the case \(l=k\geqslant 2\) (i.e. \(n=m\geqslant 2r\)), and thanks to Sect. 4.2, we suppose that \(r\geqslant 2\). We take \(V'\) as above, and thanks to [14, Lemma 9.2] again, we take \(W'\in W({\text {St}}_l(\rho ),\psi )\) such that for \(g\in G_r\):
$$\begin{aligned} W'\begin{pmatrix} g &{} &{} \\ x &{} h \\ &{} &{} I_{n-m} \end{pmatrix}=W\begin{pmatrix} g &{} \\ &{} I_{n-r} \end{pmatrix} \end{aligned}$$
(11)
for x and h in open compact subgroups \(U_1\) and \(U_2\) of respectively \({\mathcal {M}}_{m-1-r,r}\) and \(B_{m-1-r}^-\) and is equal to zero for x or h outside of those subgroups. Again we choose \(U_1\) and \(U_2\) small enough such that matrices \(\left( {\begin{matrix} I_r &{} &{} \\ x &{} h &{} \\ &{} &{} 1\end{matrix}}\right) \) for \(x\in U_1\) and \(h\in U_2\) fix all functions \(V_i\) by right translation. If \(\phi \) is a Schwartz function of the form \(\lambda {\mathbf {1}}_{({{\mathfrak {p}}}^f)^{n-1} \times (1+{{\mathfrak {p}}}^f)}\) for f large enough and \(\lambda =\mu (1+{{\mathfrak {p}}}^f)^{-1}\)for \(d\mu \) the normalized Haar measure on \(F^\times \), we have:
$$\begin{aligned} I_m(s,W',V',\phi )=\int _{N_{m-1}\backslash G_{m-1}}W'\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}V'\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}\nu (g)^{s-1}dg. \end{aligned}$$
The the integration formula (9) with \(m-1\) instead of m together with our choices of \(W'\) and \(V'\) gives the equality
$$\begin{aligned} \int _{N_{m-1}\backslash G_{m-1}}W'\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}V'\begin{pmatrix} g &{} \\ &{} 1 \end{pmatrix}\nu (g)^{s-1}dg=I_{m,r}(s,\sum _{i=0}^{k-1} \lambda _i V_i,W^{{{\text {cusp}}}}), \end{aligned}$$
and we conclude again, by appealing to Equality (6), that
$$\begin{aligned} I_m(s,W',V',\phi )=L(s,{\text {St}}_l(\rho ),{\text {Sp}}_l(\rho ^\vee )). \end{aligned}$$
\(\square \)

Notes

Acknowledgements

This work was started during a research visit of the second author (N.M.) to Imperial College London and he would like to thank them for their hospitality. The visit was supported by the LMS (Research in Pairs Grant) and by GDRI: Representation Theory (2016-2020). The authors thank David Helm, Gil Moss, Dipendra Prasad and Shaun Stevens for fruitful conversations. Most importantly, we thank the referee for pointing out a mistake in a computation of a previous version, his precise reading and accurate corrections, and his very useful suggestions concerning the presentation of the paper. The second author was supported by the Grant ANR-13-BS01-0012 FERPLAY.

References

  1. 1.
    Bernstein, J.N.: Le “centre”de Bernstein. In: Deligne, P. (ed.) Representations of Reductive Groups Over a Local Field, Travaux en Cours, pp. 1–32. Hermann, Paris (1984)Google Scholar
  2. 2.
    Bernstein, I.N., Zelevinsky, A.V.: Representations of the group \(\text{ GL }(n, F)\) where \(F\) is a non-Archimedean local field. Russ. Math. Surv. 31(3), 1–68 (1976)CrossRefGoogle Scholar
  3. 3.
    Bernstein, I.N., Zelevinsky, A.V.: Induced representations of reductive \({ p}\)-adic groups. I. Ann. Sci. École Norm. Sup. (4) 10(4), 441–472 (1977)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bushnell, C., Kutzko, J., Philip C.: The Admissible Dual of GL(\(N\)) via Compact Open Subgroups. Annals of Mathematics Studies, vol. 129. Princeton University Press, Princeton (1993)Google Scholar
  5. 5.
    Casselman, W.: A new nonunitarity argument for \(p\)-adic representations. J. Fac. Sci. Univ. Tokyo Sect. IA Math. 28(3):907–928 (1982). 1981Google Scholar
  6. 6.
    Cogdell, J.W., Piatetski-Shapiro, I.I.: Derivatives and L-functions for \(GL\_n\). In: Representation Theory, Number Theory, and Invariant Theory. Progress in Mathematics, vol. 323, pp. 115–173. Birkhäuser/Springer, Cham (2017)CrossRefGoogle Scholar
  7. 7.
    Emerton, M., Helm, D.: The local Langlands correspondence for \({\text{ GL }}\_n\) in families. Ann. Sci. Éc. Norm. Supér. (4) 47(4), 655–722 (2014)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Gelfand, I.M., Kajdan, D.A.: Representations of the group \({\rm GL}(n,K)\) where \(K\) is a local field. In: Lie Groups and Their Representations (Proc. Summer School, Bolyai János Math. Soc., Budapest, 1971). pp. 95–118. Halsted, New York (1975)Google Scholar
  9. 9.
    Jacquet, H.: A correction to conducteur des représentations du groupe linéaire. Pac. J. Math. 260(2), 515–525 (2012)CrossRefGoogle Scholar
  10. 10.
    Jacquet, H., Shalika, J.A.: On Euler products and the classification of automorphic forms. II. Am. J. Math. 103(4), 777–815 (1981)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Jacquet, H., Shalika, J.A.: On Euler products and the classification of automorphic representations. I. Am. J. Math. 103(3), 499–558 (1981)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Jacquet, H., Shalika, J.: The Whittaker models of induced representations. Pac. J. Math. 109(1), 107–120 (1983)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Jacquet, H., Piatetski-Shapiro, I.I., Shalika, J.: Conducteur des représentations du groupe linéaire. Math. Ann. 256(2), 199–214 (1981)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Jacquet, H., Piatetski-Shapiro, I.I., Shalika, J.A.: Rankin–Selberg convolutions. Am. J. Math. 105(2), 367–464 (1983)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kim, K.-M.: Test vectors of Rankin–Selberg convolutions for general linear groups. PhD thesis, The Ohio State University (2010). https://etd.ohiolink.edu/
  16. 16.
    Kurinczuk, R., Matringe, N.: Test vectors for local cuspidal Rankin-Selberg integrals. Nagoya Math. J. 1–23 (2017)Google Scholar
  17. 17.
    Matringe, N.: Derivatives and asymptotics of Whittaker functions. Represent. Theory 15, 646–669 (2011)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Matringe, N.: Essential Whittaker functions for \(GL(n)\). Doc. Math. 18, 1191–1214 (2013)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Miyauchi, M.: Whittaker functions associated to newforms for \(GL(n)\) over \(p\)-adic fields. J. Math. Soc. Jpn. 66(1), 17–24 (2014)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Paskunas, V., Stevens, S.: On the realization of maximal simple types and epsilon factors of pairs. Am. J. Math. 130(5), 1211–1261 (2008)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Shintani, T.: On an explicit formula for class-\(1\) “Whittaker functions” on \(GL\_{n}\) over \(P\)-adic fields. Proc. Jpn. Acad. 52(4), 180–182 (1976)CrossRefGoogle Scholar
  22. 22.
    Silberger, A.J.: The Langlands quotient theorem for \(p\)-adic groups. Math. Ann. 236(2), 95–104 (1978)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Zelevinsky, A.V.: Induced representations of reductive \({ p}\)-adic groups. II. On irreducible representations of \({\rm GL}(n)\). Ann. Sci. École Norm. Sup. (4) 13(2), 165–210 (1980)MathSciNetCrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of MathematicsImperial College LondonLondonUK
  2. 2.Laboratoire de Mathématiques et ApplicationsUniversité de PoitiersFuturoscope Chasseneuil CedexFrance

Personalised recommendations