Abstract
We consider a special class of unipotent periods for automorphic forms on a finite cover of a reductive adelic group \(\mathbf {G}(\mathbb {A}_\mathbb {K})\), which we refer to as Fourier coefficients associated to the data of a ‘Whittaker pair’. We describe a quasiorder on Fourier coefficients, and an algorithm that gives an explicit formula for any coefficient in terms of integrals and sums involving higher coefficients. The maximal elements for the quasiorder are ‘Levidistinguished’ Fourier coefficients, which correspond to taking the constant term along the unipotent radical of a parabolic subgroup, and then further taking a Fourier coefficient with respect to a \({\mathbb K}\)distinguished nilpotent orbit in the Levi quotient. Thus one can express any Fourier coefficient, including the form itself, in terms of higher Levidistinguished coefficients. In companion papers we use this result to determine explicit Fourier expansions of minimal and nexttominimal automorphic forms on split simplylaced reductive groups, and to obtain Euler product decompositions of certain Fourier coefficients.
Introduction
Main results
In this paper we establish a reduction principle for unipotent periods of an automorphic form on a reductive algebraic group \(\mathbf{G}\) defined over a number field \({\mathbb K}\). Such periods generalize Fourier coefficients for classical modular forms, and are important because they often encode quantities of arithmetic interest. Our motivation comes in part from string theory, where certain socalled nonperturbative effects can be expressed in terms of unipotent periods [23, 24, 35]. This is explored further in a companion paper [21] that uses the results of the present paper in an essential way.
We work in the useful generality of a pair \((G,\Gamma ),\) where G is a finite central extension of the adele group \(\mathbf{G}(\mathbb {A}) = \mathbf{G}(\mathbb {A}_{\mathbb K})\), and \(\Gamma \subset G\) is a discrete subgroup on which the covering map restricts to an isomorphism with \(\mathbf{G}({\mathbb K})\). This class includes the important central extensions defined in [8]. Also in this case, by [32, Appendix I], the covering map has a canonical splitting over every unipotent subgroup \(N = \mathbf{N}(\mathbb {A})\), which we will use to identify N as a subgroup of G. If \(\eta \) is a left \(\Gamma \)invariant function on G, and \(\chi \) is a character of N that is trivial on \(N\cap \Gamma \), then the \((N,\chi )\)unipotent period of \(\eta \) is the integral
Since [N] is compact, the period integral converges under mild conditions on \(\eta \); although it is quite unlikely that one can say anything useful in the generality of any N and \(\chi \).
In this paper we consider a special class of unipotent periods \({\mathcal {F}}_{S,\varphi }\) associated to certain pairs \((S,\varphi )\in {\mathfrak {g}}\times {\mathfrak {g}}^*\), where \({\mathfrak {g}}={\mathfrak {g}}({\mathbb K})\) is the Lie algebra of \({\mathbf {G}}({\mathbb K})\). We refer to these as \((S,\varphi )\)Fourier coefficients, or simply as Fourier coefficients. As we explain in the next section, this class includes the unipotent periods studied in the theory of automorphic forms [14,15,16,17, 29], which we call neutral Fourier coefficients, and also those arising in string theory [13, 24, 35], which we call parabolic Fourier coefficients.
Our main result is a reduction algorithm that allows us to express a given Fourier coefficient in terms of others, which are higher with respect to a natural quasiorder \(\succcurlyeq \). This is the more important direction for applications. It is much easier to write a Fourier coefficient in terms of lower coefficients; such expressions tend to be simpler but less useful — see Proposition 4.3.3 and its proof below, which uses the techniques of [19].
We now give a quick sketch of the essential ideas. We say \(S\in {\mathfrak {g}}\) is \({\mathbb Q}\)semisimple if \({\text {ad}}(S)\) acts semisimply on \({\mathfrak {g}}\) with eigenvalues in \({\mathbb Q}\), and we write \({\mathfrak {g}}^S_{{\lambda }}\) for the \(\lambda \)eigenspace. We set
with similar notation for \({\mathfrak {g}}^*\). We say \((S,\varphi )\in {\mathfrak {g}}\times {\mathfrak {g}}^*\) is a Whittaker pair if S is \({\mathbb Q}\)semisimple and \(\varphi \in ({\mathfrak {g}}^*)^S_{2}\). In this case \(\varphi \) is given by the Killing form pairing with a unique nilpotent element \(f_\varphi \) in \({\mathfrak {g}}^S_{2}\). We write \({\mathfrak {g}}_{\varphi } \subset {\mathfrak {g}}\) for the stabilizer of \(\varphi \), and we set
Then \( {\mathfrak {n}}_{S,\varphi }\) is a nilpotent Lie algebra and \(\varphi \vert _{{\mathfrak {n}}_{S,\varphi }}\) is a Lie algebra character. By exponentiation we get a unipotent subgroup \(N_{S,\varphi }\subset G\) and a character \(\chi _{S,\varphi }\), and we define
We now define a \(\Gamma \)invariant quasiorder on Whittaker pairs. If \(\varpi =(H,\varphi )\) and \(\varpi '=(H',\varphi ')\) are Whittaker pairs, then we write \(\varpi \succcurlyeq \varpi '\), first in the following two basic cases:

(i)
if \( \varphi '=\varphi \), H and \(H'\) commute, and we have
$$\begin{aligned} {\mathfrak {g}}_{\varphi }\cap {\mathfrak {g}}^H_{\ge 1} \subseteq {\mathfrak {g}}^{HH'}_{\ge 0} \end{aligned}$$(1.4) 
(ii)
if \(H=H'\), and there is a parabolic subalgebra \(\mathfrak {p}=\mathfrak {l}+{\mathfrak {n}}\) defined over \({\mathbb K}\) such that
$$\begin{aligned} H, f_{\varphi '} \in \mathfrak {l},\quad f_\varphi f_{\varphi '} \in {\mathfrak {n}}. \end{aligned}$$(1.5)
More generally, we write \(\varpi \succcurlyeq \varpi '\) if there is a sequence \(\varpi _1,\ldots ,\varpi _n\) with \(\varpi _1=\varpi \), \(\varpi _n=\varpi '\), and elements \(\gamma _1,\ldots ,\gamma _{n1}\) in \(\Gamma \) such that \(\varpi _i \succcurlyeq \gamma _i\cdot \varpi _{i+1}\) in the sense of (i) or (ii).
We will also consider separately the quasiorders given by (i) and (ii). Thus we will write
We say that a pair \((H,\varphi )\) is Levidistinguished if H is maximal with respect to \(\succcurlyeq _\varphi \). In this case, \({\mathcal {F}}_{H,\varphi }\) reduces to a \(\mathbb {K}\)distinguished coefficient on a Levi quotient of a parabolic subgroup, by first taking the constant term along the unipotent radical—see Sect. 2.3 for details.
Algorithm A, our main result, described in Sect. 4, relates the Fourier coefficients for two pairs \((S,\varphi )\succcurlyeq (H,\varphi )\) and gives an explicit formula of the form
Here the “main” term \({\mathcal {M}}_H^S\) is a discrete sum of integral transforms that we describe shortly. The “higher” terms are similar expressions, detailed in Sect. 4, involving coefficients \({\mathcal {F}}_{H',\varphi '}\) such that \({(H',\varphi ')} \succcurlyeq {(H,\varphi )}\) but \(\varphi '\) is not in the \(\Gamma \)orbit of \(\varphi \), which implies that the closure of the \(\mathbf{G}({\mathbb C})\)orbit of \(\varphi '\) properly contains that of \(\varphi \). By iteration, we deduce that

(a)
any \({\mathcal {F}}_{H,\varphi }\) can be expressed via Levidistinguished \({\mathcal {F}}_{S,\psi }\) satisfying \((S,\psi )\succcurlyeq (H,\varphi )\).

(b)
in particular, \(\eta ={\mathcal {F}}_{0,0}[\eta ]\) can be expressed in terms of Levidistinguished \({\mathcal {F}}_{S,\psi }\).

(c)
if \(\eta \) is cuspidal (see Sect. 4.4) it can be expressed in terms of \({\mathbb K}\)distinguished \(\mathcal {F}_{S,\psi }\).
We now define \({\mathcal {M}}_H^S\). For this we write \((\mathfrak {g}^H_{>\lambda })^S_\mu ={\mathfrak {g}}^H_{>{\lambda }}\cap {\mathfrak {g}}^S_\mu \) etc., and we set
We regard these as nilpotent subquotients of the Lie algebra \({\mathfrak {g}}\) via the identifications
and we define corresponding subquotients of the group G as follows:
We note that \(\Omega \) is a discrete group since \({\mathfrak {w}}\subset {\mathfrak {g}}= {\mathfrak {g}}({\mathbb K})\) is a lattice. We now set
There is an important special case where the higher terms in (1.7) vanish. We write \(\psi \succ \varphi \) if \(\psi \succcurlyeq _H \varphi \) for some H, but \(\psi \not \in \Gamma \cdot \varphi \), and we say \(\varphi \) is in the Whittaker support \(\mathrm{WS}(\eta )\) of \(\eta \) if \({\mathcal {F}}_{S,\varphi }[\eta ]\ne 0\) for some S, but if \(\psi \succ \varphi \) then \( {\mathcal {F}}_{S,\psi }[\eta ]= 0\) for all S.
Theorem B
(see Sect. 4.2) If \(\varphi \) is in \(\mathrm{WS}(\eta )\) and \((S,\varphi )\succcurlyeq (H,\varphi )\) then \({\mathcal {F}}_{H,\varphi }[\eta ] ={\mathcal {M}}_H^S({\mathcal {F}}_{S,\varphi }[\eta ]).\)
Classes of Fourier coefficients and further results
The most natural way to complete a nilpotent \(\varphi \in {\mathfrak {g}}^*\) to a Whittaker pair is provided by the Jacobson–Morozov theorem. Specifically, \(\varphi \) is given by the Killing pairing with a unique nilpotent \(f\in {\mathfrak {g}},\) and f in turn can be completed to an \(\mathfrak {sl_2}\)triple (e, h, f). Then \((h,\varphi )\) is a Whittaker pair. We will call such pairs, and the corresponding coefficients, neutral. Such coefficients were extensively studied in [14,15,16,17, 29], where the name “Fourier coefficient” was reserved exclusively for this case.
Many applications use the class of parabolic Fourier coefficients, which are period integrals with respect to a character of the unipotent radical U of a parabolic subgroup \(P=LU\). Despite their prevalence in the literature, these have in general not been evaluated in terms of known functions on \(\mathbf {G}(\mathbb {A})\).
An important subclass of parabolic Fourier coefficients are those with respect to maximal unipotent subgroups, which we call Whittaker coefficients. When the automorphic form is spherical and the unipotent character is nondegenerate these factorize over primes and the nonarchimedean factors can be explicitly computed using the Casselman–Shalika formula [10]. By Lemma 2.3.9 below these Whittaker coefficients are precisely the Levidistinguished Fourier coefficients corresponding to nilpotent orbits that are principal in Levi subalgebras defined over \({\mathbb K}\). We will call such orbits PLorbits (see Sect. 2.2 below for more details).
It follows from [19] that for any Whittaker pair \((H,\varphi )\) there exists \(h\in {\mathfrak {g}}\) such that \((h,\varphi )\) is a neutral pair and \(H\succcurlyeq _\varphi h\) (see Corollary 3.2.2 below). Moreover, by [19, Theorem C], if \({\mathcal {F}}_{h,\varphi }[\eta ]\equiv 0\) for some neutral pair \((h,\varphi )\) then \({\mathcal {F}}_{S,\varphi }[\eta ]\equiv 0\) for any Whittaker pair \((S,\varphi )\) with the same nilpotent element \(\varphi \). Thus, neutral coefficients determine the Whittaker support \({\text {WS}}(\eta )\) and are often used to define it. More generally, if \(S\succcurlyeq _\varphi H\) then \({\mathcal {F}}_{S,\varphi }\) can be obtained from \({\mathcal {F}}_{H,\varphi }\) by an integral transform (that can also involve discrete summation), see Proposition 4.3.3 below.
As an illustrative example we may take the coefficient \(B={\mathcal {F}}_{S,\varphi }\) to be a Whittaker coefficient with a character supported only on the exponentiated root space of a single simple root, and the coefficient \(C={\mathcal {F}}_{H,\varphi }\) to be a Fourier coefficient with respect to the unipotent radical of the standard maximal parabolic subgroup obtained from the same root, together with a restriction of the same character. Then, B can be obtained as a period integral of C over the quotient of the unipotent groups. Theorem B and Algorithm A allow us to go in the opposite direction and obtain C from B and from Levidistinguished coefficients corresponding to higher orbits.
Let us discuss the relations between these classes of Fourier coefficients referring to Sect. 2 for more details. By Corollary 4.4.6 below, for any Whittaker pair \((H,\varphi )\), there exists an S such that \((S,\varphi )\) is a Levidistinguished Whittaker pair, and \(S\succcurlyeq _\varphi H\). In Lemma B.7 we show that Levidistinguished Whittaker pairs have maximal dimension of \({\mathfrak {n}}_{S,\varphi }\) among all Whittaker pairs with the same \(\varphi \). By Lemma 2.3.9 below, if \(\varphi \) is a principal nilpotent in L then \({\mathcal {F}}_{H,\varphi }\) is a Whittaker coefficient. To summarize, for any orbit \(\mathcal {O}\) we have that
In particular, for the zero orbit \(\{0\}\), the neutral coefficient is the identity map \({\mathcal {F}}_{0,0}[\eta ]=\eta \), while the Whittaker coefficient is the constant term map. From this summary we conclude that if all Fourier coefficients corresponding to nonPL orbits identically vanish then Algorithm A allows us to express \(\eta \), and all its Fourier coefficients, through its Whittaker coefficients.
In the case \(G={\text {GL}}_n(\mathbb {A})\) this generalizes the classical result by PiatetskiShapiro and Shalika [34, 37] that expresses every cuspidal form through its Whittaker coefficients with respect to nondegenerate characters. We explain how Algorithm A allows us to reproduce this result in Sect. 5.3 below.
Related works
Our main tool is a generalization of the deformation technique of [18, 19], which in turn builds on the rootexchange method of [15, 17].
There are three crucial differences between the approach in the current paper and that of [18]: we consider automorphic forms (rather than mostly local representations), we consider the general case of the relation \(S\succcurlyeq _{\varphi } H\) (rather than requiring H to be neutral), and we give explicit formulas relating various Fourier coefficients, while [18] concentrates on vanishing properties. A special case of Algorithm A in which \((H,\varphi )\) is neutral can be established using the technique of [18]. One could combine this with a similar, but easier, statement (Proposition 4.3.3 below) to relate any \({\mathcal {F}}_{H,\varphi }\) to any \({\mathcal {F}}_{S,\varphi }\). However this would be less useful than Algorithm A, which provides more compact expressions since it proceeds directly from \((H,\varphi )\) to \((S,\varphi )\) without a detour through neutral coefficients.
One can show that for split simplylaced groups the socalled minimal and nexttominimal orbits are always PL. Thus Algorithm A and Theorem B give formulas for automorphic forms attached to the minimal and nexttominimal representations of simplylaced groups, as well as all their Fourier coefficients, in terms of their Whittaker coefficients. We develop these formulas in a companion paper [21]. Another application of Theorem B is to deduce that certain Fourier coefficients are Eulerian [20].
A Fourier expansion for the discrete spectrum of \({\text {GL}}_n(\mathbb {A})\) is provided in [28], generalizing [34, 37]. In Sect. 5.3 below we apply Algorithm A to provide a further generalization.
Structure of the paper
In Sect. 2 we give the definitions of the notions mentioned above, as well as of Whittaker triples and quasiFourier coefficients. These are technical notions defined in [18] and widely used in the current paper as well.
In Sect. 3 we relate Fourier and quasiFourier coefficients corresponding to different Whittaker pairs and triples. To do that we further develop the deformation technique of [18, 19], making it more general, more explicit, and better adapted to the global case.
Section 4 contains the main results of the paper. In Sect. 4.1 we describe Algorithm A. The algorithm proceeds by deforming a Whittaker pair \((H,\varphi )\) to a bigger pair \((S,\varphi )\) along a straight line \(H+t(SH)\) with \(t\in [0,1]\) in the Cartan subalgebra. At certain critical values t, additional quasiFourier coefficients (associated with higher orbits) are generated. These can be rewritten in terms of higher Fourier coefficients by proceeding in a straight line away from a neutral element, see Algorithm 4.1.1. The final result consists of the main term for \((S,\varphi )\) together with these higher Fourier coefficients. In Sect. 4.2 we prove that the algorithm is correct and terminates in a finite number of steps. We also derive Theorem B. In Sect. 4.3 we prove additional results, including a formula for \({\mathcal {F}}_{S,\varphi }\) in terms of \({\mathcal {F}}_{H,\varphi }\) (this direction is opposite to that of Theorem B). In Sect. 4.4 we give a constructive proof that for any Whittaker pair \((H,\varphi )\), there exists an S such that \((S,\varphi )\) is a Levidistinguished Whittaker pair, and \(S\succcurlyeq _\varphi H\).
In Sect. 5 we provide explicit examples in the cases \({\text {SL}}_4, {\text {GL}}_n, {\text {Sp}}_4\) and Heisenberg parabolics of arbitrary simplylaced groups.
Two appendices contain proofs of geometric lemmas on PLorbits and on our order relation on rational nilpotent orbits.
Preliminaries on nilpotent orbits and Whittaker pairs
In this section, we fix basic notation for Whittaker pairs and Fourier coefficients. We also introduce some new preparatory results.
Whittaker pairs
As in the introduction, let \({\mathbb K}\) be a number field, \(\mathfrak {o}\) its ring of integers, and let \(\mathbb {A}=\mathbb {A}_{{\mathbb K}}\) be its ring of adeles. Let \(\mathbb {T} = \{z \in \mathbb {C} : \left {z}\right = 1\}\) and fix a nontrivial additive character \(\chi : \mathbb {A}\rightarrow \mathbb {T}\), which is trivial on \({\mathbb K}\). Then \(\chi \) defines an isomorphism between \(\mathbb {A}\) and the character group \(\hat{\mathbb {A}} := {\text {Hom}}(\mathbb {A}, \mathbb {T})\) via the map \(a\mapsto \chi _{a}\), where \(\chi _{a}(b)=\chi (ab)\) for \(a,b\in \mathbb {A}\). Furthermore, this isomorphism restricts to an isomorphism
which means that we may parametrize characters on \(\mathbb {A}\) trivial on \(\mathbb {K}\) by elements in \(\mathbb {K}\).
Let \(\mathbf{G}\) be a reductive group defined over \({\mathbb K}\), \(\mathbf{G}(\mathbb {A})\) the group of adelic points of \(\mathbf{G}\) and let G be a finite central extension of \(\mathbf{G}(\mathbb {A})\). That is,
for some finite group C.
We assume that there exists a section \(\mathbf{G}({\mathbb K})\rightarrow G\) of the projection \({\text {pr}} : G {\twoheadrightarrow }\mathbf{G}(\mathbb {A})\). Fix such a section and denote its image in G by \(\Gamma \). By [32, Appendix I], the cover \(G{\twoheadrightarrow }\mathbf{G}(\mathbb {A})\) canonically splits over unipotent subgroups, and thus we will consider such subgroups as subgroups of G. Let \({\mathfrak {g}}({\mathbb K})\) denote the Lie algebra of \(\mathbf{G}({\mathbb K}) \cong \Gamma \) which we will often abbreviate to \(\mathfrak {g}\). Let \(\mathfrak {v}\) be a nilpotent subalgebra of \({\mathfrak {g}}\) and let \({\mathfrak {v}}(\mathbb {A}):={\mathfrak {v}}\otimes _{{\mathbb K}}\mathbb {A}\) be its adelization. As in the introduction, we denote by \({\text {Exp}}({\mathfrak {v}})\) the unipotent subgroup of \(\Gamma \) obtained by exponentiation of \({\mathfrak {v}}\) using the above split over unipotent subgroups, and we denote by \(V:={\text {Exp}}({\mathfrak {v}}(\mathbb {A}))\) the unipotent subgroup of G obtained by exponentiation of \({\mathfrak {v}}(\mathbb {A})\). We note that \({\text {Exp}}(\mathfrak {v}) = V \cap \Gamma \) and for convenience we will denote by [V] the compact quotient \((V\cap \Gamma )\backslash V\).
To conveniently describe different unipotent subgroups of G and characters on these subgroups we introduce the following notion.
Definition 2.1.1
A Whittaker pair is an ordered pair \((S,\varphi )\in {\mathfrak {g}}\times {\mathfrak {g}}^*\) such that S is a rational semisimple element (that is, a semisimple element for which the eigenvalues of the adjoint action are in \({\mathbb Q}\)), and \({\text {ad}}^*(S)\varphi =2\varphi \).
We will often identify \(\varphi \in {\mathfrak {g}}^*\) with its dual nilpotent element \(f = f_\varphi \in \mathfrak {g}\) with respect to the Killing form \(\langle \, , \, \rangle \). We will say that \(\varphi \) is nilpotent if \(f_\varphi \) is a nilpotent element of \({\mathfrak {g}}\). Equivalently, \(\varphi \in {\mathfrak {g}}^*\) is nilpotent if and only if the Zariski closure of its coadjoint orbit includes zero. For example, if \((S,\varphi )\) is a Whittaker pair then \(\varphi \) is nilpotent.
For any rational semisimple \(S\in {\mathfrak {g}}\) and \(\lambda \in {\mathbb Q}\) we introduce the following notation
and analogously for \({\mathfrak {g}}_{< \lambda }^{S}\) and \({\mathfrak {g}}_{\le \lambda }^{S}\), with a similar use of notation for \({\mathfrak {g}}^*\).
For any \(\varphi \in {\mathfrak {g}}^*\) let \(\mathfrak {g}_\varphi \) be the centralizer of \(\varphi \) in \(\mathfrak {g}\) under the coadjoint action and define an antisymmetric form \(\omega _{\varphi } : \mathfrak {g} \times \mathfrak {g} \rightarrow \mathbb {K}\) by \(\omega _{\varphi }(X,Y)=\varphi ([X,Y])\). We extend \(\varphi \) and \(\omega _\varphi \) to a functional and an antisymmetric form on \(\mathfrak {g}(\mathbb {A})\) respectively by linearity. Given a Whittaker pair \((S,\varphi ) \in {\mathfrak {g}}\times {\mathfrak {g}}^*\), we let \(\mathfrak {u} := \mathfrak {g}^S_{\ge 1}\) and define
which, by Lemma 3.2.5 below, can also be written as
Note that \({\mathfrak {n}}_{S,\varphi }\) is an ideal in \(\mathfrak {u}\) with abelian quotient, and that \(\varphi \) defines a character of \({\mathfrak {n}}_{S,\varphi }\).
We define a corresponding character \(\chi _\varphi \) on \(N_{S,\varphi }\), trivial on \(N_{S,\varphi } \cap \Gamma \), by
More generally, let \(\mathfrak {r}\subseteq {\mathfrak {u}}\) be any isotropic subspace (not necessarily maximal) with respect to \(\omega _\varphi _{{\mathfrak {u}}}\), that includes \(\mathfrak {n}_{S,\varphi }\). Note that \({\mathfrak {n}}_{S,\varphi } \subseteq \mathfrak {r} \subseteq {\mathfrak {u}}\), and that \({\mathfrak {n}}_{S,\varphi }\) and \(\mathfrak {r}\) are ideals in \({\mathfrak {u}}\). Let \(R={\text {Exp}}\bigl (\mathfrak {r}(\mathbb {A})\bigr )\). Then \(\chi _{\varphi }^R : R \rightarrow \mathbb {T}\) defined by \(\chi _\varphi ^R(r)=\chi (\varphi (\log r))\) is a character of R trivial on \(R \cap \Gamma \). Indeed, since \(\mathfrak {r}\) is isotropic, we have that \(\omega _\varphi _{\mathfrak {r(\mathbb {A})}} = 0\) and thus \(\chi _\varphi ^R \in {\text {Hom}}(R, \mathbb {T})\), and \(\varphi (X) \in \mathbb {K}\) for \(X \in \mathfrak {r}(\mathbb {K})\).
Definition 2.1.2
We call a function on G an automorphic function if it satisfies the following properties:

(i)
invariant under the left action of \(\Gamma \),

(ii)
finite under the right action of the preimage in G of \(\prod _{\text {finite }\nu }{} \mathbf{G}(\mathfrak {o}_\nu ),\) and

(iii)
smooth when restricted to the preimage in G of \(\prod _{\text {infinite }\nu }{} \mathbf{G}({\mathbb K}_\nu )\).
We denote the space of all automorphic functions by \(C^{\infty }(\Gamma \backslash G)\).
Definition 2.1.3
Let \((S,\varphi )\) be a Whittaker pair for \({\mathfrak {g}}\) and let \(R, N_{S,\varphi },\chi _{\varphi }\) and \(\chi _{\varphi }^{R}\) be as above. For an automorphic function \(\eta \), we define the Fourier coefficient of \(\eta \) with respect to the pair \((S,\varphi )\) to be
We also define its RFourier coefficient to be the function
Observe that if \(\pi \) denotes a subrepresentation of \(C^{\infty }(\Gamma \backslash G)\) that contains \(\eta \) then \({\mathcal {F}}_{S,\varphi }[\eta ]\) and \({\mathcal {F}}_{S,\varphi }^{R}[\eta ]\) are matrix coefficients corresponding to the vector \(\eta \in \pi \) and the functional on the space of automorphic functions defined by the integrals above.
Note that \(\mathcal {F}_{0,0}[\eta ]=\eta \). For a general Whittaker pair \((S,\varphi ),\) \(\mathcal {F}_{S,\varphi }[\eta ](g)\) is a smooth function on G in the above sense, but is not invariant under \(\Gamma \) any more. On the other hand, its restriction to the joint centralizer \(G_{S,\varphi }\) of S and \(\varphi \) is left \(G_{S,\varphi }\cap \Gamma \)invariant. As shown in [14], if \(\eta \) is also \(\mathfrak {z}\)finite and has moderate growth, then the restriction of \(\eta \) to \(G_{S,\varphi }\) still has moderate growth, but may stop being \(\mathfrak {z}\)finite.
Remark 2.1.4
In [19, Sect. 6] the integrals (2.7) and (2.8) above are called Whittaker–Fourier coefficients, but in this paper we call them Fourier coefficients for short.
Definition 2.1.5
A Whittaker pair \((h,\varphi )\) is called neutral if either \((h,\varphi )=(0,0)\), or h and the Lie algebra element \(f = f_\varphi \) that corresponds to \(\varphi \) under the Killing form pairing can be completed to an triple (e, h, f). Equivalently, \((h, \varphi )\) is called neutral if the map \(X \mapsto {\text {ad}}^*(X)\varphi \) defines an epimorphism \({\mathfrak {g}}^h_{0}{\twoheadrightarrow }({\mathfrak {g}}^*)^h_{2} \), and h can be completed to an triple. For more details on triples over arbitrary fields of characteristic zero see [7, Sect. 11].
Definition 2.1.6
We say that \((S,\varphi ,\varphi ')\) is a Whittaker triple if \((S,\varphi )\) is a Whittaker pair and \(\varphi '\in ({\mathfrak {g}}^*)^{S}_{>2}\).
For a Whittaker triple \((S,\varphi ,\varphi ')\), let U, R, and \(N_{S,\varphi }\) be as in Definition 2.1.3. Note that \(\varphi +\varphi '\) defines a character of \(\mathfrak {r}\). Extend it by linearity to a character of \(\mathfrak {r}(\mathbb {A})\) and define an automorphic character \(\chi _{\varphi +\varphi '}\) of R by \(\chi _{\varphi +\varphi '}^{R}(\exp X):=\chi (\varphi (X)+\varphi '(X))\). For an example for this notation see Sect. 5.2 below.
Definition 2.1.7
For an automorphic function \(\eta \), we define its \((S,\varphi ,\varphi ')\)quasi Fourier coefficient to be the function
We also define its \((S,\varphi ,\varphi ',R)\)quasi Fourier coefficient to be the function
Definition 2.1.8
We call a \({\mathbb K}\)subgroup of \(\mathbf{G}\) a split torus of rank m if it is isomorphic as a \({\mathbb K}\)subgroup to \({\text {GL}}_1^m\). We call a Lie subalgebra \(\mathfrak {l}\subseteq {\mathfrak {g}}\) a \({\mathbb K}\)Levi subalgebra if it is the centralizer of a split torus.
Remark 2.1.9
We note that the Lie algebra of any split torus is spanned by rational semisimple elements. Consequently, a subalgebra of \(\mathfrak {l}\subseteq {\mathfrak {g}}\) is a \({\mathbb K}\)Levi subalgebra if and only if it is the centralizer of a rational semisimple element of \({\mathfrak {g}}\). Another equivalent condition is that \(\mathfrak {l}\) is the Lie algebra of a Levi subgroup of a parabolic subgroup of G defined over \({\mathbb K}\).
For convenience, we fix a complex embedding \(\sigma :{\mathbb K}{\hookrightarrow }{\mathbb C}\), which allows us to map a \(\Gamma \)orbit \(\mathcal {O}\) in \(\mathfrak {g}\) to a \(\mathbf {G}(\mathbb {C})\)orbit in \(\mathfrak {g}(\mathbb {C}) := \mathfrak {g} \otimes _{\sigma (\mathbb {K})} \mathbb {C}\). One can show, using [12], that the complex orbit corresponding to \({\mathcal {O}}\) does not depend on \(\sigma \). However, we will not need this fact.
Definition 2.1.10
Let \((H,\varphi )\) and \((S,\varphi )\) be two Whittaker pairs with the same \(\varphi \). We say that \((H,\varphi )\) dominates \((S,\varphi )\) if H and S commute and
This relation is denoted \(S \succcurlyeq _{\varphi } H\) in the introduction.
Principal nilpotent elements, PL elements and standard Whittaker pairs
We introduce some notions for coadjoint nilpotent orbits under the action of \(\Gamma \). For general results on nilpotent orbits over algebraically closed fields see [9, 11].
Definition 2.2.1
We say that a nilpotent orbit under \(\Gamma \) in \({\mathfrak {g}}^*\) is principal if it is Zariski dense in the nilpotent cone \(\mathcal {N}({\mathfrak {g}}^*)\). We say that \(\varphi \in {\mathfrak {g}}^*\) is a principal nilpotent element if its orbit is principal.
We say that a nilpotent \(\varphi \in {\mathfrak {g}}^*\) is principal in a Levi (or PL for short) if there exists a \({\mathbb K}\)Levi subalgebra \(\mathfrak {l}\subset {\mathfrak {g}}\) and a nilpotent element \(f\in \mathfrak {l}\) such that the Killing form pairing with f defines \(\varphi \) in \({\mathfrak {g}}^*\), and a principal nilpotent element of \(\mathfrak {l}^*\). We call a nilpotent \(\Gamma \)orbit in \({\mathfrak {g}}^*\) a PLorbit if it consists of PL elements.
We remark that if G is quasisplit then a nilpotent element \(\varphi \in {\mathfrak {g}}^*\) is principal if and only if it is regular, i.e. the dimension of its centralizer equals the rank of \({\mathfrak {g}}\).
Lemma 2.2.2
Let \({\mathfrak {n}}\) be the nilpotent radical of the Lie algebra of a minimal parabolic subgroup \(P_0\). Then \({\mathfrak {n}}\) intersects any nilpotent orbit under \(\Gamma \) in \({\mathfrak {g}}\).
Proof
Let \(f\in {\mathfrak {g}}\) be nilpotent, and complete to an \({\mathfrak {sl}}_2\)triple (e, h, f). Then h defines a parabolic subgroup P which then includes a minimal parabolic \(Q_0\). Then \(Q_0\) is conjugate to \(P_0\) under \(\Gamma \) (see [5, Thm. 4.13(b)]). Since f lies in the nilpotent radical of the Lie algebra of P, its conjugate will lie in \({\mathfrak {n}}\). \(\square \)
Definition 2.2.3
We say that a Whittaker pair \((S,\varphi )\) is standard if \({\mathfrak {n}}_{S,\varphi }\) is the nilpotent radical of the Lie algebra of a minimal parabolic subgroup of G. In this case we will call the Fourier coefficient \({\mathcal {F}}_{S,\varphi }\) a Whittaker coefficient.
Remark 2.2.4
In [19, Sect. 6] the Whittaker coefficients are called principal degenerate Whittaker–Fourier coefficients.
Corollary 2.2.5
A nilpotent element \(\varphi \in {\mathfrak {g}}^*\) is principal if and only if it can be completed to a neutral standard Whittaker pair.
Proof
Let h complete \(\varphi \) to a neutral standard Whittaker pair. Then \({\mathfrak {n}}_{h,\varphi }={\mathfrak {g}}^h_{>1}={\mathfrak {g}}^h_{>0}\) is the nilpotent radical of the Lie algebra of a minimal parabolic subgroup, and thus so is \({\mathfrak {n}}:={\mathfrak {g}}^h_{<1}\). Let \(f\in {\mathfrak {n}}\) define \(\varphi \) through the Killing form pairing. Then we have \([f,{\mathfrak {g}}^h_{\le 0}]={\mathfrak {n}}\) and thus \(\Gamma f\cap {\mathfrak {n}}\) is Zariski open, and thus Zariski dense, in \({\mathfrak {n}}\). The statement follows now from Lemma 2.2.2.
Conversely, let \(\varphi \in {\mathfrak {g}}^*\) be a principal nilpotent, and let (e, h, f) be an \({\mathfrak {sl}}_2\)triple such that f defines \(\varphi \) via the Killing form. Let \({\mathcal {O}}\) denote the complex orbit of f and \(\bar{\mathcal {O}}\) denote its Zariski closure. Then \(\bar{\mathcal {O}}= {\mathcal {N}}({\mathfrak {g}})\supset {\mathfrak {g}}^h_{<0}\). Thus \({\mathcal {O}}\) is the Richardson orbit for \({\mathfrak {g}}^h_{\le 0}\), and thus \(\dim {\mathcal {O}}=2\dim {\mathfrak {g}}^h_{<0}\). Now suppose by way of contradiction that the pair \((h,\varphi )\) is not standard. Then \({\mathfrak {g}}^h_{\le 0}\) is not a minimal \({\mathbb K}\)parabolic subalgebra, i.e. there exists a smaller \({\mathbb K}\)parabolic subalgebra \(\mathfrak {p}\) with nilpotent radical \({\mathfrak {n}}\supsetneq {\mathfrak {g}}^h_{<0}\). But \({\mathfrak {n}}\subset {\mathcal {N}}= \bar{\mathcal {O}}\), and thus \({\mathcal {O}}\) is a Richardson orbit for \(\mathfrak {p}\), thus \(\dim {\mathcal {O}}= 2\dim {\mathfrak {n}}>2\dim {\mathfrak {g}}^h_{<0}=\dim {\mathcal {O}}\) which is a contradiction. \(\square \)
Corollary 2.2.6
A nilpotent \(\varphi \in {\mathfrak {g}}^*\) is PL if and only if it can be completed to a standard Whittaker pair \((S,\varphi )\).
Proof
Let \((S,\varphi )\) be a standard Whittaker pair. Then \(S=h+Z\) where \((h,\varphi )\) is neutral and commutes with Z. Then Z defines a Levi subalgebra \(\mathfrak {l}\), and the Whittaker pair \((h,\varphi )\) is neutral and standard in \(\mathfrak {l}\). By Corollary 2.2.5, \(\varphi \) is principal in \(\mathfrak {l}\).
Conversely, if \(\varphi \) is principal in \(\mathfrak {l}\) and Z defines \(\mathfrak {l}\) we let \(S:=TZ+h\) for \(T\in {\mathbb Q}_{>0}\) big enough. Then \((S,\varphi )\) is a standard Whittaker pair. \(\square \)
Let us remark that in [19] a different definition of principal and PL elements was given. The following lemma states the equivalence of the definitions.
Lemma 2.2.7
Let \(\varphi \in {\mathfrak {g}}^*\) be nilpotent. Then

(i)
\(\varphi \) is PL if and only if there exist a maximal split toral subalgebra \({\mathfrak {a}}\) of \(\mathfrak {g}\) and a choice of associated simple roots \(\Pi \) such that \(\varphi \in \bigoplus _{\alpha _i \in \Pi } \mathfrak {g}^*_{\alpha _i}\), where \(\mathfrak {g}^*_{\alpha _i}\) denotes the dual of the root space \({\mathfrak {g}}_{{\alpha }_i}\).

(ii)
If \(\varphi \in \bigoplus _{\alpha _i \in \Pi } \mathfrak {g}^*_{\alpha _i}\) then \(\varphi \) is principal in the Levi subalgebra given by those simple roots \({\alpha }_i\) for which the projection of \(\varphi \) to \(\mathfrak {g}^*_{\alpha _i}\) is nonzero.
Proof
Let \(\sum _{{\alpha }_i\in \Pi } {\mathfrak {g}}^{\times }_{{\alpha }_i}\subset \bigoplus _{\alpha _i \in \Pi } \mathfrak {g}^*_{\alpha _i}\) denote the subset of vectors with all projections nonzero. It is enough to show that \(\varphi \in {\mathfrak {g}}^*\) is principal if and only if there exist \(({\mathfrak {a}},\Pi )\) as above such that \(\varphi \in \sum _{{\alpha }_i\in \Pi } {\mathfrak {g}}^{\times }_{{\alpha }_i}\).
To show that, assume first that \(\varphi \) is principal. Then, by Corollary 2.2.5, \(\varphi \) can be completed to a neutral standard pair \((h,\varphi )\). Then h defines a torus and simple roots, and we have \(\varphi \in \sum _{{\alpha }_i\in \Pi } {\mathfrak {g}}^{\times }_{{\alpha }_i}\). Conversely, give \({\mathfrak {a}}\) and \(\Pi \) as above, we let \(h:=\sum c_i {\alpha }_i^\vee \), where \({\alpha }_i^\vee \) are the coroots given by scalar product with \({\alpha }_i\), and \(c_i\) are chosen such that \(\varphi \in ({\mathfrak {g}}^*)^h_{2}\). Then \((h,\varphi )\) is a standard Whittaker pair. Moreover, \(\varphi \) is a generic element of \(({\mathfrak {g}}^*)^h_{2}\) and thus the Jacobson–Morozov theory implies that \((h,\varphi )\) is a neutral pair. \(\square \)
Remark 2.2.8
Note that for \(G={\text {GL}}_n(\mathbb {A})\) all orbits \(\mathcal {O}\) are PLorbits. In general this is, however, not the case, see Appendix A for details.
Levidistinguished Fourier coefficients
Definition 2.3.1
We say that a nilpotent \(f \in {\mathfrak {g}}\) is \({\mathbb K}\)distinguished, if it does not belong to a proper \({\mathbb K}\)Levi subalgebra \(\mathfrak {l}\subsetneq {\mathfrak {g}}\). In this case we will also say that \(\varphi \in {\mathfrak {g}}^*\) given by the Killing form pairing with f is \({\mathbb K}\)distinguished. We will also say that the orbit of \(\varphi \) is \({\mathbb K}\)distinguished.
Example 2.3.2
The nilpotent orbits for \(\mathrm {Sp}_{2n}({\mathbb C})\) are given by partitions of 2n such that odd parts have even multiplicity. Each such orbit, except the zero one, decomposes to infinitely many \(\mathrm {Sp}_{2n}({\mathbb Q})\)orbits — one for each collection of equivalence classes of quadratic forms \(Q_1,\dots ,Q_k\) of dimensions \(m_1,\dots m_k\) where k is the number of even parts in the partition and \(m_1,\dots m_k\) are the multiplicities of these parts. A complex orbit is distinguished over \(\mathbb {C}\) (i.e. does not intersect a proper Levi subalgebra) if and only if all parts have multiplicity one (and thus there are no odd parts). To see the “only if” part note that if the partition includes a part k with multiplicity two then the orbit intersects the Levi \({\text {GL}}_k \times \mathrm {Sp}_{2(nk)}\). If k is odd then this Levi is defined over \({\mathbb Q}\) and thus all \({\mathbb Q}\)distinguished orbits correspond to totally even partitions. If k is even then this Levi is defined over \({\mathbb Q}\) if and only if the quadratic form on the multiplicity space of k is not anisotropic. Thus, we obtain that a necessary condition for an orbit \({\mathcal {O}}\) to be \({\mathbb Q}\)distinguished is that its partition \({\lambda }({\mathcal {O}})\) is totally even, a sufficient condition is that \({\lambda }({\mathcal {O}})\) is multiplicity free, and for totally even partitions with multiplicities there are infinitely many \({\mathbb Q}\)distinguished orbits and at least one not \({\mathbb Q}\)distinguished. For example, for the partition (4, 2) all orbits in \(\mathfrak {sp}_{6}({\mathbb Q})\) are \({\mathbb Q}\)distinguished, for the partition \(2^3\) some orbits are \({\mathbb Q}\)distinguished and some are not, and all other partitions do not correspond to \({\mathbb Q}\)distinguished orbits.
Lemma 2.3.3
Every principal nilpotent element is \({\mathbb K}\)distinguished.
Proof
Let \(f\in {\mathfrak {g}}\) define a principal nilpotent element via the Killing form. Suppose the contrary: f lies in a proper \({\mathbb K}\)Levi subalgebra \(\mathfrak {l}\) of \({\mathfrak {g}}\). Let \(Z\in {\mathfrak {g}}\) be a rational semisimple element such that \(\mathfrak {l}= \mathfrak {g}^Z_0\). Complete f to an \({\mathfrak {sl}}_2\)triple \(\gamma :=(e,h,f)\) in \(\mathfrak {l}\). Then \({\text {ad}}(Z)\) acts by a scalar on every irreducible submodule of the adjoint action of \(\gamma \) on \({\mathfrak {g}}\). Since \(\mathfrak {l}\ne {\mathfrak {g}}\), there exists an irreducible submodule V on which \({\text {ad}}(Z)\) acts by a negative scalar \(c\). Let v be a highest weight vector of V of weight d, and let \(S:=h+c^{1}(d+2)Z\). Then \(v+f\in {\mathfrak {g}}^S_{2}\) and thus \(v+f\) is nilpotent. Since f is principal, \(v+f\) lies in the Zariski closure of \(\Gamma f\). On the other hand, \(v+f\) belongs to the affine space \(f+{\mathfrak {g}}^e\), which is called the Slodowy slice to \(\Gamma f\) at f, and is transversal to \(\Gamma f\), contradicting the assumption that \(v+f\) lies in the Zariski closure of \(\Gamma f\). \(\square \)
Lemma 2.3.4
Let \(f\in {\mathfrak {g}}\) be nilpotent. Then all \({\mathbb K}\)Levi subalgebras \(\mathfrak {l}\subseteq {\mathfrak {g}}\) such that \(f \in \mathfrak {l}\) and f is \({\mathbb K}\)distinguished in \(\mathfrak {l}\) are conjugate by the centralizer of f.
Proof
Complete f to an triple \(\gamma :=(e,h,f)\) and denote its centralizer by \(G_{\gamma }\). Let us show that all \({\mathbb K}\)Levi subalgebras \(\mathfrak {l}\) of \({\mathfrak {g}}\) that contain \(\gamma \) and in which f is distinguished are conjugate by \(G_\gamma \). Let \(\mathfrak {l}\) be such a subalgebra, \(L\subseteq G\) be the corresponding Levi subgroup, and let C denote the maximal split torus of the center of L. Then C is a split torus in \(G_{\gamma }\). Let us show that it is a maximal split torus. Let \(T\supseteq C\) be a larger split torus in \(G_{\gamma }\). Then, the centralizer of T in \({\mathfrak {g}}\) is a \({\mathbb K}\)Levi subalgebra that lies in \(\mathfrak {l}\) and includes \(\gamma \), and thus is equal to \(\mathfrak {l}\). Thus \(T=C\).
Since \(\mathfrak {l}\) is the centralizer of T in G, T is a maximal split torus of \(G_{\gamma }\), and all maximal split tori of reductive groups are conjugate (see [3, 15.14]), we get that all the choices of L are conjugate.
Since all the choices of \(\gamma \) are conjugate by the centralizer of f, the lemma follows. \(\square \)
Definition 2.3.5
Let \(Z\in {\mathfrak {g}}\) be a rationalsemisimple element and \(\mathfrak {l}\) denote its centralizer. Let \((h,\varphi )\) be a neutral Whittaker pair for \(\mathfrak {l}\), such that the orbit of \(\varphi \) in \(\mathfrak {l}^*\) is \({\mathbb K}\)distinguished. We say that the Whittaker pair \((h+Z,\varphi )\) is Levidistinguished if
In this case we also say that the Fourier coefficient \({\mathcal {F}}_{h+Z,\varphi }\) is Levidistinguished.
Remark 2.3.6
Let \((h,\varphi )\) be a neutral Whittaker pair for \({\mathfrak {g}}\). If \(\varphi \) is \({\mathbb K}\)distinguished then \({\mathcal {F}}_{h,\varphi }\) is a Levidistinguished Fourier coefficient. If a rational semisimple Z commutes with h and with \(\varphi \), and \(\varphi \) is \({\mathbb K}\)distinguished in \(\mathfrak {l}:={\mathfrak {g}}^Z_0\) then \({\mathcal {F}}_{h+TZ,\varphi }\) is a Levidistinguished Fourier coefficient for any T bigger than \(m/M+1\), where m is the maximal eigenvalue of h and M is the minimal positive eigenvalue of Z. See also Lemma 4.4.5 for further discussion.
Lemma 2.3.7
([19, Lemma 3.0.2]) For any Whittaker pair \((H,\varphi )\) there exists \(Z\in {\mathfrak {g}}^{H}_0\) such that \((HZ,\varphi )\) is a neutral Whittaker pair.
Remark 2.3.8
In [19] the lemma is proven over a local field, but the proof only used the Jacobson–Morozov theorem, that holds over arbitrary fields of characteristic zero.
Lemma 2.3.9
For any Whittaker pair \((H,\varphi )\), the following are equivalent:

(i)
\((H,\varphi )\) is standard pair

(ii)
\((H,\varphi )\) is a Levidistinguished Fourier coefficient, and \(\varphi \) is a PL nilpotent.
Proof
First let \((H,\varphi )\) be a standard pair. Then by Lemma 2.3.7, H can be decomposed as \(H=h+Z\) where \((h,\varphi )\) is a neutral pair and Z commutes with h and with \(\varphi \). Let \(\mathfrak {l}\) and L denote the centralizers of Z in \({\mathfrak {g}}\) and G, and \(N:=N_{H,\varphi }\). Then N is the unipotent radical of a minimal parabolic subgroup of G, and L is a Levi subgroup of G. Thus, \(N\cap L\) is the unipotent radical of a minimal parabolic subgroup of L. The Lie algebra of \(N\cap L\) is \({\mathfrak {n}}_{H,\varphi }\cap {\mathfrak {g}}^Z_0={\mathfrak {g}}^{h}_{\ge 1}\cap {\mathfrak {g}}^Z_0\). Thus, \({\text {Exp}}({\mathfrak {g}}^h_{\le 1}\cap {\mathfrak {g}}^Z_0)\) is the unipotent radical of a minimal parabolic subgroup of L. Since \(\varphi \) is given by Killing form pairing with \(f\in {\mathfrak {g}}^h_{\le 1}\cap {\mathfrak {g}}^Z_0\), we get by Corollary 2.2.5 that \(\varphi \) is principal in \(\mathfrak {l}\). Replacing Z by tZ with t large enough, we obtain that \((H,\varphi )\) is a Levidistinguished pair.
Now, assume that \(\varphi \) is a PL nilpotent, and let \((H, \varphi ) = (h+Z,\varphi )\) be a Levidistinguished pair. Let \(\mathfrak {l}={\mathfrak {g}}^Z_0\) be the corresponding Levi subalgebra, and let \(f=f_{\varphi }\) be the element of \({\mathfrak {g}}\) that defines \(\varphi \). Since f is distinguished in \(\mathfrak {l}\), and principal in some Levi subalgebra, Lemmas 2.3.3 and 2.3.4 imply that f is principal in \(\mathfrak {l}\). Thus, \({\mathfrak {n}}_{H,\varphi }\cap \mathfrak {l}\) is the nilpotent radical of the Lie algebra of a minimal parabolic subgroup of L and thus \({\mathfrak {n}}_{H,\varphi }={\mathfrak {n}}_{H,\varphi }\cap \mathfrak {l}\oplus {\mathfrak {g}}^Z_{>0}\) the nilpotent radical of the Lie algebra of a minimal parabolic subgroup of G. Thus \((H,\varphi )\) is standard Whittaker pair. \(\square \)
Lemma 2.3.10
Let \((H,\varphi )\) and \((S,\varphi )\) be Levidistinguished Whittaker pairs with the same \(\varphi \). Then \(\dim {\mathfrak {n}}_{S,\varphi }=\dim {\mathfrak {n}}_{H,\varphi }\)
Proof
By the definition of Levidistinguished Whittaker pair, there exists a decomposition \(H=h+Z\) such that \(\varphi \) is given by the Killing form pairing with a distinguished element \(f\in \mathfrak {l}={\mathfrak {g}}^Z_0\) and (2.12) holds. Let \(S=h'+Z'\) be a decomposition for S satisfying the same properties, and let \(\mathfrak {l}':={\mathfrak {g}}^{Z'}_0\). By Lemma 2.3.4, \(\mathfrak {l}\) and \(\mathfrak {l}'\) are conjugate by the centralizer of f, and thus we can assume that \(\mathfrak {l}=\mathfrak {l}'\). By the Jacobson–Morozov theorem, h and \(h'\) are conjugate by the centralizer of f in \(\mathfrak {l}\). Now,
\(\square \)
In Lemma B.7 (ii) below we show that Levidistinguished pairs have maximal dimension of \({\mathfrak {n}}_{S,\varphi }\) among all Whittaker pairs with nilpotent element \(\varphi \).
Order on nilpotent orbits and Whittaker support
Definition 2.4.1
We define a partial order on nilpotent orbits in \({\mathfrak {g}}^* = {\mathfrak {g}}^*({\mathbb K})\) to be the transitive closure of the following relation R: \(({\mathcal {O}},{\mathcal {O}}')\in R\) if there exist \(\varphi \in {\mathcal {O}},\) rational semisimple \(H,Z\in {\mathfrak {g}}\), and \(\varphi '\in ({\mathfrak {g}}^*)^Z_{>0 } \cap ({\mathfrak {g}}^*)^H_{2}\) such that \(\varphi \in ({\mathfrak {g}}^*)^Z_0 \cap ({\mathfrak {g}}^*)^H_{2 }\), \([H,Z]=0,\) and \(\varphi +\varphi '\in {\mathcal {O}}'\).
In the notation of the introduction, the conditions in the definition read \(\varphi +\varphi '\succcurlyeq _H \varphi \), the parabolic subalgebra in (1.5) being \({\mathfrak {g}}^Z_{\ge 0}\). In Appendix B, we study these rational orbits in more detail. In particular, in Corollary B.3 we prove that this is indeed a partial order, i.e. that R is antisymmetric. We will thus denote \({\mathcal {O}}\le {\mathcal {O}}'\) (or \({\mathcal {O}}'\ge {\mathcal {O}}\)) if \(({\mathcal {O}},{\mathcal {O}}')\) lies in the transitive closure of R, and \({\mathcal {O}}<{\mathcal {O}}'\) if \({\mathcal {O}}\le {\mathcal {O}}'\) and \({\mathcal {O}}\ne {\mathcal {O}}'\). By Corollary B.2 below this implies an inequality on the dimensions of complexifications: \(\dim {\mathcal {O}}_{{\mathbb C}}<\dim {\mathcal {O}}'_{{\mathbb C}}\).
Lemma 2.4.2
If \({\mathcal {O}}'\ge {\mathcal {O}}\), then for any place \(\nu \) of \({\mathbb K}\), the closure of \({\mathcal {O}}'\) in \({\mathfrak {g}}({\mathbb K}_{\nu }) := {\mathfrak {g}}\otimes _\mathbb {K} \mathbb {K}_\nu \) (in the local topology) contains \({\mathcal {O}}\).
Proof
Fix a place \(\nu \) of \({\mathbb K}\). Since \(\mathbf{G}({\mathbb K})\) is dense in \(\mathbf{G}({\mathbb K}_{\nu })\), the closure of \({\mathcal {O}}'\) equals the closure of the saturation \(\mathbf{G}({\mathbb K}_{\nu }){\mathcal {O}}'\). Thus it is enough to show that for any \(Z\in {\mathfrak {g}}\), \(\varphi \in {\mathfrak {g}}^Z_0\) and \(\psi \in {\mathfrak {g}}^Z_{>0}\), \(\varphi \) lies in the closure of \(\mathbf{G}({\mathbb K}_{\nu })(\varphi +\psi )\). By [18, Lemma 2.18], there exist a central element \(Z'\in {\mathfrak {g}}\), a positive integer k and an algebraic group homomorphism \(\mu :{\mathbb K}_{\nu }^{\times }\rightarrow \mathbf{G}^{\mathrm {ad}}({\mathbb K}_{\nu })\) such that \(k(ZZ')=d\mu (1)\), where \(\mathbf{G}^{\mathrm {ad}}\) denotes the adjoint group of \(\mathbf{G}\). Let \(\lambda \in {\mathbb K}_\nu ^{\times }\) be small, and let \(g':=\mu (\lambda )\). Since the natural projection \(\mathbf{G} \rightarrow G^{\mathrm {ad}}\) is onto, the image of the map between the \({\mathbb K}_{\nu }\)points \(\mathbf{G}({\mathbb K}_{\nu })\rightarrow \mathbf{G}^{\mathrm {ad}}({\mathbb K}_{\nu })\) has finite index in \(\mathbf{G}^{\mathrm {ad}}({\mathbb K}_{\nu })\) (by [4, Corollary 6.4]), and thus for some power m, \((g')^m\) lies in this image. Let \(g\in \mathbf{G}({\mathbb K}_{\nu })\) be a preimage of \((g')^{m}\). Since \(\varphi \in {\mathfrak {g}}^Z_0\) and \(\psi \in {\mathfrak {g}}^Z_{>0}\), we have \(g^n\varphi =\varphi \), while \(g^n\psi \rightarrow 0\) when \(n\rightarrow \infty \). Thus \(g^n(\varphi +\psi )\rightarrow \varphi \). \(\square \)
Let us use this opportunity to mention that a similar argument in the 4th paragraph in the proof of [18, Proposition 4.12] is imprecise, and should be replaced by the argument above.
Definition 2.4.3
For an automorphic function \(\eta \), we define \(\mathrm {WO}(\eta )\) to be the set of nilpotent orbits \({\mathcal {O}}\) in \({\mathfrak {g}}^*\) under the coadjoint action of \(\mathbf{G}({\mathbb K})\) such that \({\mathcal {F}}_{h,\varphi }[\eta ]\ne 0\) for some neutral Whittaker pair \((h,\varphi )\) with \(\varphi \in {\mathcal {O}}\). Using the partial order of Definition 2.4.1, we define the Whittaker support \({\text {WS}}(\eta )\) to be the set of maximal elements in \(\mathrm {WO}(\eta )\).
Remark 2.4.4
Applying Lemma 2.4.2 to Archimedean places, we obtain that our order is stronger than the closure (Bruhat) order on complex orbits, i.e. if \({\mathcal {O}}'\ge {\mathcal {O}}\) then \(\overline{{\mathcal {O}}'_{{\mathbb C}}}\supseteq {\mathcal {O}}_{{\mathbb C}}\). It is in general strictly stronger; for example one can show that the maximal elements with respect to our order are the \({\mathbb K}\)distinguished orbits. For \(\mathbf{G}= {\text {GL}}_n,\) our order coincides with the Bruhat order on complex orbits by [18, Proposition 7.5].
The definition of the Whittaker support \({\text {WS}}\) of automorphic representations given in [18, Sect. 8] uses the coarser Bruhat order on complex orbits. Thus, the \({\text {WS}}\) defined in Definition 2.4.3 includes the \({\text {WS}}\) used in [18, Sect. 8] but is frequently not equal to it.
Relating different Fourier coefficients
In this section we will introduce some standard tools used to relate different types of Fourier coefficients. In the subsequent subsection we first relate Fourier coefficients for different isotropic subspaces for a single Whittaker pair. Then we relate Fourier coefficients along deformations of Whittaker pairs, and lastly we explain the relationship between the conjugation of a Whittaker pair and the translation of the argument of a Fourier coefficient.
The statements in this section have partial local analogues in [18, 19]. However, the statements we give here are global and more explicit.
Relating different isotropic subspaces
We will now see how \({\mathcal {F}}_{S,\varphi ,\varphi '}\) and \({\mathcal {F}}_{S,\varphi ,\varphi '}^R\) can be expressed through each other.
Lemma 3.1.1
(cf. [19, Lemma 6.0.2] for a slightly weaker statement) Let \(\eta \in C^{\infty }(\Gamma \backslash G)\), let \((S,\varphi ,\varphi ')\) be a Whittaker triple, \({\mathfrak {n}}_{S,\varphi }\) be as in (2.4), and \({\mathfrak {u}}:= \mathfrak {g}^S_{\ge 1}\).
Let \({\mathfrak {n}}_{S,\varphi } \subseteq \mathfrak {i}\subseteq \mathfrak {r}\) be isotropic subspaces of \({\mathfrak {u}}\), and let \(\mathfrak {i}^\perp \supseteq \mathfrak {r}^{\perp }\) be their orthogonal complements with respect to \(\omega _\varphi _{\mathfrak {u}}\). Let also \(I: = {\text {Exp}}(\mathfrak {i}(\mathbb {A}))\), \(R = {\text {Exp}}(\mathfrak {r}(\mathbb {A}))\), \(I^\bot :={\text {Exp}}( \mathfrak {i}^\bot (\mathbb {A}))\) and \(R^\bot :={\text {Exp}}(\mathfrak {i}^\bot (\mathbb {A}))\). Then,
and
We will mostly use this lemma in the case \(\mathfrak {i}={\mathfrak {n}}_{S,\varphi }\) for which \(\mathfrak {i}^{\bot }={\mathfrak {u}}\). The convergence in (3.2) is absolute convergence, uniform in compact subsets of G.
Proof
We assume that \(\varphi \) is nonzero since otherwise \(R=I=N_{S,\varphi }\). We have that \(I \subseteq R\) with R/I abelian which means that (3.1) follows immediately from the definitions of \({\mathcal {F}}^I_{S,\varphi ,\varphi '}\) and \({\mathcal {F}}_{S,\varphi ,\varphi '}^R\). For (3.2) observe that the function \((\chi _{\varphi }^{R})^{1}\cdot {\mathcal {F}}^I_{S,\varphi ,\varphi '}[\eta ]\) on R is leftinvariant under the action of \(I\cdot (R\cap \Gamma )\). In other words, we can identify it with a function on
where the equality follows from the fact that R/I is abelian. Therefore, we have a Fourier series expansion
where \([R/I]^{\wedge }\) denotes the Pontryagin dual group of [R/I] and
The Fourier series (3.4) converges absolutely since the function \(\eta \) is smooth.
In particular, denoting by \({\text {Id}}\in G\) the identity element we obtain
Now observe that the map \(X \mapsto \omega _{\varphi }(X,\cdot )=\varphi \circ {\text {ad}}(X)\) induces an isomorphism between \(\mathfrak {i}^{\bot }/\mathfrak {r}^\bot \) and the dual space \((\mathfrak {r}/\mathfrak {i})^*\). Hence, according to equations (2.1) and (3.3), we can use the character \(\chi \) to define a group isomorphism
where
Hence, for all \(u\in I^{\bot }\cap \Gamma \) and \(r\in R\) we have
Here we are taking again \(u=\exp X\), \(r=\exp Y\) and the middle equality follows from the vanishing of \(\varphi \) on \({\mathfrak {g}}^{S}_{>2}\). But now, from formula (3.5) and the fact that \(\eta \) is automorphic, we have
for all \(u\in U\cap \Gamma \). Combining this with (3.6) and (3.7) we obtain
Applying this to \(\eta \) and its right shifts we obtain (3.2). The convergence is uniform when g varies in compact subsets of G since \(\eta \) is infinitely smooth. \(\square \)
Corollary 3.1.2
Let \(\eta \in C^{\infty }(\Gamma \backslash G)\), let \((S,\varphi ,\varphi ')\) be a Whittaker triple, and \({\mathfrak {n}}_{S,\varphi }\), and \({\mathfrak {u}}\) be as above. Let \(\mathfrak {r},\mathfrak {r}'\subseteq {\mathfrak {u}}\) be two isotropic subspaces that include \(\mathfrak {n}_{S,\varphi }\). Assume \(\dim \mathfrak {r} =\dim \mathfrak {r}'\) and \(\mathfrak {r}\cap (\mathfrak {r}')^{\bot }\subseteq \mathfrak {r}'\). Then
Note that this is a noncompact, adelic, integral. It converges absolutely.
Proof
Let \(\mathfrak {i}:=\mathfrak {r}\cap \mathfrak {r}'\). We claim that the natural map \(p:\mathfrak {r}/\mathfrak {i} \rightarrow \mathfrak {i}^{\bot }/(\mathfrak {r}')^{\bot }\) is an isomorphism. Indeed, from the assumption \(\mathfrak {r}\cap (\mathfrak {r}')^{\bot }\subseteq \mathfrak {r}'\) we have \(\mathfrak {i} = \mathfrak {r}\cap (\mathfrak {r}')^{\bot }\) and thus p is an embedding. Further, from the assumption \(\dim \mathfrak {r}=\dim \mathfrak {r}'\) we obtain that the source and the target spaces of p have the same dimension. Indeed,
Now, p defines a natural isomorphism . Let \(I:={\text {Exp}}(\mathfrak {i}(\mathbb {A}))=R\cap R'\). From Lemma 3.1.1 we obtain
The convergence of the integral follows from the convergence of the Fourier series in (3.2). \(\square \)
This corollary can be seen as a version of the root exchange lemma in [15].
Relating different Whittaker pairs
Let \((H,\varphi )\) be a Whittaker pair.
Lemma 3.2.1
Let Z be as in Lemma 2.3.7. Then \((HZ,\varphi )\) dominates \((H,\varphi )\).
Proof
Denote \(h:=HZ\). We have to show that (2.11) holds, i.e.
Since \({\mathfrak {g}}_{\varphi }\) is spanned by lowest weight vectors, we have \({\mathfrak {g}}_{\varphi }\subseteq {\mathfrak {g}}^h_{\le 0}\) and thus \({\mathfrak {g}}_{\varphi }\cap {\mathfrak {g}}^h_{\ge 1}=\{0\}\). \(\square \)
Corollary 3.2.2
Any Whittaker pair is dominated by a neutral Whittaker pair with the same character \(\varphi \).
Another example of domination is provided by the following proposition, that immediately follows from [19, Proposition 3.3.3].
Proposition 3.2.3
If \(\varphi \) is a PL nilpotent then there exists \(Z\in {\mathfrak {g}}\) such that \((H+Z,\varphi )\) is a standard Whittaker pair and \((H,\varphi )\) dominates \((H+Z,\varphi )\).
From now till the end of the section let \((H,\varphi )\) be a Whittaker pair and \(Z\in {\mathfrak {g}}_0^H\) be a rational semisimple element such that \((H,\varphi )\) dominates \((H+Z,\varphi )\). We will now consider the deformation of the former Whittaker pair to the latter. For any rational number \(t \ge 0\) define
Definition 3.2.4
We call \(t \ge 0\) regular if \({\mathfrak {u}}_t = {\mathfrak {u}}_{t+\varepsilon }\) for any small enough \(\varepsilon \in {\mathbb Q}\), or in other words \({\mathfrak {w}}_t\subset {\mathfrak {g}}^Z_0\). If t is not regular we call it critical. Equivalently, t is critical if \({\mathfrak {g}}^{H_t}_{1}\nsubseteq {\mathfrak {g}}^{Z}_{0}\) which we may interpret as something new has entered the 1eigenspace of H. For convenience, we will say that \(t=0\) is critical.
We also say that \(t \ge 0\) is quasicritical if either \({\mathfrak {g}}^{H_t}_{1}\nsubseteq {\mathfrak {g}}^{Z}_{0}\) or \({\mathfrak {g}}^{H_t}_{2}\nsubseteq {\mathfrak {g}}^{Z}_{0}\). We may interpret this as something new has entered either the 1eigenspace or the 2eigenspace. The latter is related to new characters being available in the Whittaker pairs. Note that there are only finitely many (quasi)critical numbers.
Recall the antisymmetric form \(\omega _\varphi \) on \({\mathfrak {g}}\) given by \(\omega _\varphi (X,Y)=\varphi ([X,Y])\) and the definition \(\mathfrak {n}_{H_t,\varphi } := {\text {Ker}}(\omega _\varphi _{{\mathfrak {u}}_t})\).
Lemma 3.2.5
([19, Lemma 3.2.6])

(i)
The form \(\omega _\varphi \) is \({\text {ad}}(Z)\)invariant.

(ii)
\({\text {Ker}}\omega _\varphi = {\mathfrak {g}}_\varphi \).

(iii)
\({\text {Ker}}(\omega _\varphi _{{\mathfrak {w}}_t})={\text {Ker}}(\omega _\varphi )\cap {\mathfrak {w}}_t\).

(iv)
\({\text {Ker}}(\omega _\varphi _{{\mathfrak {u}}_t})={\mathfrak {v}}_t\oplus {\text {Ker}}(\omega _\varphi _{{\mathfrak {w}}_t}) \).

(v)
\({\mathfrak {w}}_s\cap {\mathfrak {g}}_\varphi \subseteq {\mathfrak {u}}_{t}\) for any \(s<t\).
We will often suppress the deformation \(H_t\) and character \(\varphi \) and simply write \({\mathfrak {n}}_t = {\mathfrak {n}}_{H_t,\varphi }\). Similarly, define
We note that \(\mathfrak {l}_t\) and \(\mathfrak {r}_t\) are nilpotent subalgebras. The choice of notation for them comes from ‘left’ and ‘right’.
Lemma 3.2.6
For any \(t\ge 0\) we have

(i)
\(\mathfrak {l}_t\) and \(\mathfrak {r}_t\) are ideals in \({\mathfrak {u}}_t\) and \([\mathfrak {l}_t,\mathfrak {r}_t]\subseteq \mathfrak {l}_t\cap \mathfrak {r}_t={\mathfrak {n}}_t\).

(ii)
\(\mathfrak {l}_t\) and \(\mathfrak {r}_t\) are isotropic subspaces of \({\mathfrak {u}}_t\) of the same dimension, and the natural projections \(\mathfrak {l}_t/{\mathfrak {n}}_t\rightarrow {\mathfrak {u}}_t/\mathfrak {r}_t^\bot \) and \(\mathfrak {r}_t/{\mathfrak {n}}_{t} \rightarrow {\mathfrak {u}}_t/\mathfrak {l}_t^\bot \) are isomorphisms. Furthermore, \(\mathfrak {l}_t = \mathfrak {g}^{H_t}_1 \cap \mathfrak {g}^Z_{<0} \oplus \mathfrak {n}_t\).

(iii)
Suppose that \(0\le s< t\), and all the elements of the interval (s, t) are regular. Then
$$\begin{aligned}&{\mathfrak {v}}_{t}\oplus ({\mathfrak {w}}_{t}\cap {\mathfrak {g}}^Z_{<0})={\mathfrak {v}}_{s}\oplus ({\mathfrak {w}}_{s}\cap {\mathfrak {g}}^Z_{>0}) \end{aligned}$$(3.17)$$\begin{aligned}&\mathfrak {l}_{t}=\mathfrak {r}_{s}+ ({\mathfrak {w}}_{t}\cap {\mathfrak {g}}_{\varphi }) \text { and }\mathfrak {r}_{s}\cap ({\mathfrak {w}}_{t}\cap {\mathfrak {g}}_{\varphi })={\mathfrak {w}}_{0}\cap {\mathfrak {g}}^Z_0\cap {\mathfrak {g}}_{\varphi }. \end{aligned}$$(3.18)Moreover, \(\mathfrak {r}_{s}\) is an ideal in \(\mathfrak {l}_{t}\) and the quotient is commutative.
Proof
It is easy to see that \({\mathfrak {v}}_t\) is an ideal in \({\mathfrak {u}}_t\) with commutative quotient, and that \({\mathfrak {v}}_t\subseteq \mathfrak {l}_t\cap \mathfrak {r}_t={\mathfrak {n}}_t\). This proves (i). For the first part of (ii), note that \((\mathfrak {l}_t+\mathfrak {r}_t)/{\mathfrak {n}}_t\) is a symplectic space in which the projections of \(\mathfrak {l}_t\) and \(\mathfrak {r}_t\) are complementary Lagrangians.
For the second part, we have by Lemma 3.2.5 that \(\mathfrak {g}^{H_t}_1 \cap \mathfrak {g}_\varphi \subseteq \mathfrak {g}^Z_{\ge 0}\) and thus,
For part (iii) note that
This implies (3.17). By Lemma 3.2.5 we have
and thus
Hence, (3.17) and (3.19) imply (3.18), and the rest is straightforward. \(\square \)
Using Lemma 2.3.7, choose an triple \((e_\varphi ,h,f_\varphi )\) in \({\mathfrak {g}}^Z_0\) such that h commutes with H and with Z, and \(\varphi \) is given by the Killing form pairing with \(f = f_\varphi \). Define \(L_{t} = L_{H_t,\varphi } :={\text {Exp}}(\mathfrak {l}_{H_t,\varphi }(\mathbb {A})),\, R_{t}=R_{H_t,\varphi }:={\text {Exp}}(\mathfrak {r}_{H_t,\varphi }(\mathbb {A}))\) and \(V_t := {\text {Exp}}({\mathfrak {v}}_t(\mathbb {A}))\). From Lemmas 3.2.6 and 3.1.1 we get the following.
Lemma 3.2.7
Let \((H,\varphi )\) and \((S,\varphi )\) be Whittaker pairs such that \((H,\varphi )\) dominates \((S,\varphi )\). Let \(Z = S  H\), \(H_t = H + tZ\) and define \(R_t\), \(L_t\) and \(V_t\) as above.

(i)
Let \(t \ge 0\) and \(\varphi '\in ({\mathfrak {g}}^*)^{H_t}_{>2}\). Let also \(I = L_t\) and \(\mathfrak {j} = \mathfrak {r}_t\), or \(I = R_t\) and \(\mathfrak {j} = \mathfrak {l}_t\) respectively. Let \(A:={\text {Exp}}(({\mathfrak {u}}_t/{\mathfrak {v}}_t)^Z_{<0})(\mathbb {A})\). Then,
(3.26)(3.27)(3.28) 
(ii)
Now, let \(0 \le s < t\) such that there are no critical values in the interval (s, t) and let \(\varphi '\in ({\mathfrak {g}}^*)^{H_t}_{>2}\cap ({\mathfrak {g}}^*)^{H_s}_{>2}\). Then,
$$\begin{aligned} {\mathcal {F}}_{H_t,\varphi ,\varphi '}^{L_t}[\eta ](g)&= \mathop \int \limits _{[L_t/R_s]} {\mathcal {F}}_{H_s,\varphi ,\varphi '}^{R_s}[\eta ](ug) \, du \end{aligned}$$(3.29)$$\begin{aligned} {\mathcal {F}}_{H_s,\varphi ,\varphi '}^{R_s}[\eta ](g)&= \sum _{\psi ' \in \Psi '} {\mathcal {F}}_{H_t,\varphi ,\varphi '+\psi '}^{L_t}[\eta ](g), \end{aligned}$$(3.30)where \(\Psi ' := ({\mathfrak {g}}^*)^{H_t}_{1}\cap ({\mathfrak {g}}^*)^{e_\varphi }\cap ({\mathfrak {g}}^*)^Z_{<0}\).

(iii)
Let \(0 \le s < t\) and \(\varphi '\) as in (ii), and let \(\psi \in ({\mathfrak {g}}^*)^{H_s}_{>2}\cap ({\mathfrak {g}}^*)^{H_t}_{ 2}\). Then
$$\begin{aligned} {\mathcal {F}}_{H_s,\varphi ,\psi +\varphi '}^{R_s}[\eta ](g) = \mathop \int \limits _{[R_s/V_t]} \sum _{\psi ' \in \Psi ''} {\mathcal {F}}_{H_t,\varphi +\psi ,\varphi '+\psi '}[\eta ](ug) \, du, \end{aligned}$$(3.31)where \(\Psi '' := (\mathfrak {g}^*)^{H_t}_{1} \cap (\mathfrak {g}^*)^{e_{\varphi +\psi }}\).
Proof
For part (i) we note that \(\mathfrak {l}_t\) and \(\mathfrak {r}_t\) are isotropic subspaces of \({\mathfrak {u}}_t\) containing \({\mathfrak {n}}_t\) by Lemma 3.2.6(ii). Thus, the statement follows from Lemma 3.1.1 and Corollary 3.1.2. The domain of summation for \(\gamma \) in (3.26) follows from Lemma 3.2.6(ii). Note that for Corollary 3.1.2 we have that \(\mathfrak {l}_t \cap \mathfrak {r}_t^\perp = \mathfrak {l}_t \cap \mathfrak {r}_t={\mathfrak {n}}_t\) and \(\mathfrak {l}_t/{\mathfrak {n}}_t\cong ({\mathfrak {u}}^{t}/{\mathfrak {v}}^t)^Z_{<0}\) by Lemma 3.2.6.
For part (ii) we note from Lemma 3.2.6(iii) that \({\mathfrak {r}_s} \subseteq \mathfrak {l}_t\) and that \(\mathfrak {l}_t/\mathfrak {r}_s\) is commutative with a set of representatives \({\mathfrak {g}}^{H_t}_1 \cap {\mathfrak {g}}_\varphi \cap {\mathfrak {g}}^Z_{>0}\). Indeed, \(\mathfrak {l}_t/\mathfrak {r}_s\) projects naturally and isomorphically to \(({\mathfrak {g}}^{H_t}_1 \cap {\mathfrak {g}}_{\varphi })/({\mathfrak {g}}^{H_t}_1 \cap {\mathfrak {g}}_\varphi \cap {\mathfrak {g}}^Z_0)\), and \({\mathfrak {g}}^{H_t}_1 \cap {\mathfrak {g}}_\varphi \subseteq {\mathfrak {g}}^Z_{\ge 0}\) by Lemma 3.2.5. Thus, (3.29) follows directly by integration, while (3.30) follows from a Fourier expansion in \(L_t/R_s\) with characters given by \((\mathfrak {l}_t/{\mathfrak {r}_s})^*\) for which we have the set of representatives \(\Psi ' := ({\mathfrak {g}}^*)^{H_t}_{1}\cap ({\mathfrak {g}}^*)^{e_\varphi }\cap ({\mathfrak {g}}^*)^Z_{<0}\).
For part (iii), note first that \(N'_t:=N_{H_t, \varphi +\psi }\) and \(R_s = R_{H_s, \varphi }\) depend on different characters which means that we cannot directly relate them using Lemma 3.2.6(iii) as we did in part (ii). Instead, we will relate them both to \(V_t = {\text {Exp}}({\mathfrak {v}}_t(\mathbb {A}))\) where \({\mathfrak {v}}_t := {\mathfrak {g}}^{H_t}_{>1}\) which does not depend on any character.
We have that \({\mathfrak {v}}_t\) is an ideal in \(\mathfrak {l}_t := \mathfrak {l}_{H_t, \varphi }\) with commutative quotient. Since \(\mathfrak {l}_t\) and \({\mathfrak {r}_s} := {\mathfrak {r}}_{H_s, \varphi }\) are part of the same deformation with the same character we can then use (3.18) in Lemma 3.2.6(iii) to see that \({\mathfrak {v}}_t\) is an ideal in \({\mathfrak {r}_s}\) with commutative quotient. Thus,
Furthermore, \({\mathfrak {v}}_t\) is an ideal in \({\mathfrak {n}}_t' := \mathfrak {l}_{H_t,\varphi +\psi }\) and \({\mathfrak {n}}_t'/{\mathfrak {v}}_t\) is commutative with a set of representatives \(\mathfrak {g}^{H_t}_1 \cap \mathfrak {g}_{\varphi +\psi }\) by (2.5). The statement now follows from a Fourier expansion of \({\mathcal {F}}_{V_t}[\eta ](g)\) in \(N_t'/V_t\) with characters given by \(({\mathfrak {n}}_t'/{\mathfrak {v}}_t)^*\) for which we have the representatives \(\Psi '' := (\mathfrak {g}^*)^{H_t}_{1} \cap (\mathfrak {g}^*)^{e_{\varphi +\psi }}\). \(\square \)
Lemma 3.2.8
Let \((S, \varphi , \psi )\) be a Whittaker triple, \(\eta \) an automorphic function and \(\gamma \in \Gamma \). Then,
Proof
The proof is straightforward. We have that \(\chi _{\varphi +\psi }(u) = \chi _{{\text {Ad}}^*(\gamma )(\varphi + \psi )}(\gamma u \gamma ^{1})\). Indeed, the righthand side equals
We also have that \( \mathfrak {g}^{{\text {Ad}}(\gamma )S}_\lambda = {\text {Ad}}(\gamma ) \mathfrak {g}^S_\lambda \) since, for \(x \in \mathfrak {g}\), \([{\text {Ad}}(\gamma ) S, {\text {Ad}}(\gamma ) x] = {\text {Ad}}(\gamma )[S,x]\). Similarly, \( \mathfrak {g}_{{\text {Ad}}^*(\gamma )\varphi } = {\text {Ad}}(\gamma ) \mathfrak {g}_\varphi \) and thus, \(\mathfrak {n}_{{\text {Ad}}(\gamma ) S,{\text {Ad}}^*(\gamma ) \varphi } = {\text {Ad}}(\gamma ) \mathfrak {n}_{S,\varphi }\).
Hence, using the automorphic invariance of \(\eta \), the righthand side of (3.33) equals
By the arguments above, this equals \({\mathcal {F}}_{S, \varphi , \psi }[\eta ](g)\). \(\square \)
The reduction algorithm
In this section we will present Algorithm A which details the steps for expressing a Fourier coefficient \(\mathcal {F}_{H,\varphi }\) for a Whittaker pair \((H,\varphi )\) in terms of Fourier coefficients \(\mathcal {F}_{S,\psi }\) with pairs \((S,\psi ) \succcurlyeq (H,\varphi )\) as described in Sect. 1.1 and detailed below in Proposition 4.1.3.
The proof of this proposition, which also shows that Algorithm A terminates in a finite number of steps and produces the stated result, is given in Sect. 4.2.
After stating and proving some additional results in Sect. 4.3 that follow from the algorithm (and are used in [20]) we show in Corollary 4.4.7 in Sect. 4.4 that \(\mathcal {F}_{H, \varphi }\) can, by iteration, be expressed in terms of Levidistinguished Fourier coefficients associated to orbits which are equal to or bigger than \(\Gamma \varphi \). In the same subsection we also discuss some further results in Remark 4.4.8.
Statement of algorithm
Before stating Algorithm A, we first start with an algorithm that expresses any quasiFourier coefficient \({\mathcal {F}}_{H,\varphi ,\varphi '}\) in terms of Fourier coefficients \({\mathcal {F}}_{S,\psi }\) such that either \((H,\varphi )\) dominates \((S,\psi )\) or \(\Gamma \psi >\Gamma \varphi \).
Algorithm 4.1.1
Given a Whittaker triple \((H,\varphi ,\varphi ')\), choose h that commutes with H such that \((h, \varphi )\) is a neutral pair. This is possible by Lemma 2.3.7. Let \(Z = H  h\), and let \(H_t:=H+tZ\) for any \(t\in {\mathbb Q}\). Let \(m_Z\) denote the minimal positive eigenvalue of Z and \(M_h\) denote the maximal positive eigenvalue of h. Let \(T:=(M_h+2)/m_Z\). If Z is central we set \(m_Z\) to 1 for definiteness. Then for every \(i\ge 0\) we have
Since \({\mathfrak {g}}_{\varphi }\subset {\mathfrak {g}}^h_{\le 0}\), and since \({\text {ad}}(h)\) has integer eigenvalues, we obtain
We get two cases depending on whether there are any quasicritical t in the interval (0, T). Recall that the number of quasicritical t is in any case finite.

(i)
If there are no quasicritical \(t\in (0,T)\) then, using Lemma 3.2.7(i) and (ii), we may express \({\mathcal {F}}_{H,\varphi ,\varphi '}\) linearly in terms of the set of all \({\mathcal {F}}_{H_T,\varphi ,\varphi '+\psi '}\) with \(\psi ' \in \Psi ' = (\mathfrak {g}^*)^{H+TZ}_{1} \cap (\mathfrak {g}^*)^{e_\varphi } \cap (\mathfrak {g}^*)^{Z}_{<0}\). By (4.1) we have \(\Psi '=\{0\}\) and \(L_T=N_{H_T,\varphi }\). By (4.2), \(\varphi '\) vanishes on \({\mathfrak {n}}_{H_T,\varphi }\), thus \({\mathcal {F}}^{L_T}_{H_T,\varphi ,\varphi '}={\mathcal {F}}_{H_T,\varphi ,\varphi '}={\mathcal {F}}_{H_T,\varphi }\). Altogether we have
(4.3) 
(ii)
Now assume that there are quasicritical numbers in (0, T) and let s be the smallest one. Since s is the first quasicritical value we have that \((\mathfrak {g}^*)^{H}_{>2} \subseteq (\mathfrak {g}^*)^{H_s}_{\ge 2}\) because this is the first point where something new may enter the \(2\)eigenspace. Decompose \(\varphi ' = \psi + \varphi '' \) where \(\psi \in (\mathfrak {g}^*)^{H_s}_{2}\) and \(\varphi '' \in (\mathfrak {g}^*)^{H_s}_{>2}\). Using Lemma 3.2.7(i) and (iii), we may express \({\mathcal {F}}_{H,\varphi ,\varphi '}\) in terms of the Fourier coefficients \({\mathcal {F}}_{H_{s},\varphi +\psi ,\varphi '' + \psi ''}\) with \(\psi ''\in \Psi '' = (\mathfrak {g}^*)^{H_s}_{1} \cap (\mathfrak {g}^*)^{e_{\varphi +\psi }}\) in the following way:
(4.4)Now, we repeat the procedure for each triple \({\mathcal {F}}_{H_{s},\varphi +\psi ,\varphi ''+\psi ''}\) and so on until we reach an expression that includes only Fourier coefficients.
\(\square \)
Lemma 4.1.2
Algorithm 4.1.1 terminates in a finite number of steps, and expresses \({\mathcal {F}}_{H,\varphi ,\varphi '}\) in terms of Fourier coefficients \({\mathcal {F}}_{S_i,\Phi _i}\) such that for each i, either \(\Phi _i=\varphi \) and \((H,\varphi )\) dominates \((S_i,\Phi _i)\), or \(\Gamma \Phi _i >\Gamma \varphi \).
We postpone the proof of the above lemma to the next subsection. Now let \((H,\varphi )\) and \((S,\varphi )\) be Whittaker pairs such that \((H,\varphi )\) dominates \((S,\varphi )\). The following algorithm expresses \({\mathcal {F}}_{H,\varphi }\) in terms of \({\mathcal {F}}_{S,\varphi }\) and Fourier coefficients \({\mathcal {F}}_{S',\psi }\) such that \(\Gamma \psi >\Gamma \varphi \).
Algorithm A
Let \(Z:=SH\) and \(H_t:=H+tZ\). Let first \(s = 0\) and t be the first quasicritical point in the interval (s, 1). If there are no critical points in this interval we let \(t = 1\) which will be the end point of this algorithm. Using Lemma 3.2.7(i) and (ii), we have that \({\mathcal {F}}_{H, \varphi }[\eta ](g)\) equals
where \({\Psi }' = (\mathfrak {g}^*)^{H_t}_{1} \cap (\mathfrak {g}^*)^{e_\varphi } \cap (\mathfrak {g}^*)^{Z}_{<0}\). The last step follows from Lemma 3.2.6(ii). For all nonzero \(\psi '\in \Psi '\) we run Algorithm 4.1.1 to express the quasiFourier coefficient \({\mathcal {F}}_{H_t,\varphi ,\psi '}\) in terms of Fourier coefficients, that, as we will show, correspond to higher orbits. For the \(\psi '=0\) term we iterate the same step as above using (4.5) and Algorithm 4.1.1 but now with \(s = t\) and t being the next quasicritical point, until we reach \(t = 1\). \(\square \)
Proposition 4.1.3
Algorithm A terminates in a finite number of steps and produces an expression of the form
where
which is called the “main term” in the introduction,
and \(\mathcal {A}\) is a countable absolutely convergent sum of integral transforms of Fourier coefficients corresponding to orbits bigger than that of \(\varphi \).
We prove this proposition in the next subsection.
Proofs of Theorem B and the validity of Algorithm A
We prove the following lemma in Appendix B.
Lemma 4.2.1
Let \((H,\varphi )\) be a Whittaker pair, and let Z be a rational semisimple element that commutes with H and with \(\varphi \). Let \(\varphi '\in ({\mathfrak {g}}^*)^Z_{>0}\cap ({\mathfrak {g}}^*)^{S}_{2}\). Then either \(\dim \mathbf{G}({\mathbb C})(\varphi +\varphi ')>\dim \mathbf{G}({\mathbb C})(\varphi )\), or \(\varphi \) is conjugate to \(\varphi +\varphi '\) by the stabilizer of H in \(\Gamma \).
Let us now present the proofs that were postponed from the last subsection.
Proof of Lemma 4.1.2
In case (i) of the algorithm it terminates in one step, and (4.3) expresses \({\mathcal {F}}_{H,\varphi ,\varphi '}\) in terms of \({\mathcal {F}}_{H_T,\varphi }\), and \((H,\varphi )\) dominates \((H_T,\varphi )\). Thus assume that we are in case (ii), and let \(s\in (0,T]\) be the smallest critical number. Decompose \(\varphi ' = \psi + \varphi '' \) where \(\psi \in (\mathfrak {g}^*)^{H_s}_{2}\) and \(\varphi '' \in (\mathfrak {g}^*)^{H_s}_{>2}\). Note that \(\psi \in ({\mathfrak {g}}^*)^{Z}_{<0}\) and thus the orbit \(\Gamma (\varphi +\psi )\) of \(\varphi +\psi \) is bigger than or equal to the orbit \(\Gamma \varphi \) of \(\varphi \).
If \(\Gamma (\varphi +\psi )=\Gamma \varphi \) then, by Lemma 4.2.1, \(\varphi +\psi \) is conjugate to \(\varphi \) under the stabilizer of \(H_s\) in \(\Gamma \). Then, by shifting the argument, we express each \({\mathcal {F}}_{H_s,\varphi +\psi ,\varphi ''+\psi ''}\) in terms of \({\mathcal {F}}_{H_s,\varphi ,\varphi '''}\) for some \(\varphi '''\). Then we run the algorithm on \({\mathcal {F}}_{H_s,\varphi ,\varphi '''}\) and it terminates by induction on the finite number of critical values in the interval (0, T).
If \(\Gamma (\varphi +\psi )>\Gamma \varphi \) then, by Lemma 4.2.1, \(\dim \mathbf{G}({\mathbb C})(\varphi +\psi )>\dim \mathbf{G}({\mathbb C})(\varphi )\), and thus the algorithm terminates by induction on \(\dim \mathbf{G}({\mathbb C})(\varphi )\). \(\square \)
Lemma 4.2.2
Let \((H,\varphi )\) be a Whittaker pair, and let \((e_{\varphi },h,f_{\varphi })\) be an \({\mathfrak {sl}}_2\)triple such that \(\varphi \) is given by Killing form pairing with \(f_{\varphi },\) and h commutes with H. Let \(0 \ne \varphi ' \in ({\mathfrak {g}}^*)^{H}_{>2}\cap ({\mathfrak {g}}^*)^{H}_{\le 1} \cap ({\mathfrak {g}}^*)^{e_\varphi }\). Then Algorithm 4.1.1 expresses the quasiFourier coefficient \({\mathcal {F}}_{H,\varphi ,\varphi '}\) in terms of Fourier coefficients \((S_i,\Phi _i)\) with \(\Gamma \Phi _i>\Gamma \varphi \).
Proof
As in the algorithm, we let \(Z:=Hh\), and for any \(t\ge 0\) denote \(H_t:=H+tZ\). Then \(\varphi '\) decomposes to a sum of eigenvectors \(\psi _i\) of Z, and each of these lies in \((({\mathfrak {g}}^*)^{e_{\varphi }})^H_{\le 1}\). Since \(({\mathfrak {g}}^*)^{e_{\varphi }}\subset ({\mathfrak {g}}^*)^{h}_{\ge 0}\), we obtain \(\psi _i\in (({\mathfrak {g}}^*)^{h}_{\ge 0})^{Z}_{<0}\). Hence, while running Algorithm 4.1.1, some of the eigenvectors \(\psi _i\) will join the space \(({\mathfrak {g}}^*)^{H_t}_{2}\) for some \(t>0\), leading to an orbit that is greater than, or equal to, that of \(\varphi \). To show that it is not equal to \(\Gamma \varphi \) note that \(\varphi +\psi _i\) lies in the Slodowy slice \(\varphi +({\mathfrak {g}}^*)^{e_{\varphi }}\), and thus its complex orbit has bigger dimension than that of \(\varphi \). More Whittaker triples will be produced while running Algorithm 4.1.1, but their third elements will lie in \(({\mathfrak {g}}^*)^{H_t}_{1} \cap ({\mathfrak {g}}^*)^{e_\varphi }\) for some \(t>0\) and thus lead to bigger orbits using the argument as above. \(\square \)
Proof of Proposition 4.1.3
As in the statement of Algorithm A let \(Z:=SH\) and \(H_t:=H+tZ\). If there are no quasicritical points in the interval (0, 1) then \(t=1\) and the algorithm terminates in one step. In this case we can decompose the expression in the righthand side of (4.5) as
where
By Lemma 4.2.2, for every nonzero \(\psi '\in \Psi '=(\mathfrak {g}^*)^{S}_{1} \cap (\mathfrak {g}^*)^{e_\varphi } \cap (\mathfrak {g}^*)^{Z}_{<0}\) Algorithm 4.1.1 expresses \({\mathcal {F}}_{S,\varphi ,\varphi +\psi '}\) in terms of Fourier coefficients \((S_i,\Phi _i)\) with \(\Gamma \Phi _i>\Gamma \varphi \). Thus the term \(\mathcal {A}\) satisfies the conditions of the proposition.
Let us now assume that there exist quasicritical points \(0<t_1<\cdots<t_{n}<1\). Lemma 4.2.2 again implies that all nonzero \(\psi '\in \Psi ''_i=(\mathfrak {g}^*)^{H_{t_i}}_{1} \cap (\mathfrak {g}^*)^{e_\varphi } \cap (\mathfrak {g}^*)^{Z}_{<0}\) will lead to Fourier coefficients corresponding to bigger orbits, that will be accumulated in \(\mathcal {A}\). It is left to track down the formula for the term that we get when we take all \(\psi '\) in all the steps to be zero. Let
By Lemma 3.2.7(i) we have
Using Lemma 3.2.7(ii) (retaining only \(\psi ' = 0\)) we obtain
Since \({\mathfrak {v}}=\bigoplus _{i=1}^n({\mathfrak {g}}^{H_{t_i}}_1\cap {\mathfrak {g}}^{Z}_{<0})\), and as a commutative Lie algebra \({\mathfrak {g}}^{H_{t_i}}_1\cap {\mathfrak {g}}^{Z}_{<0}\) is naturally isomorphic to \({\mathfrak {v}}_i\), the group V is glued from \(V_i\). Thus
The proposition now follows from Lemma 3.2.7(i). \(\square \)
Proof of Theorem B
Under the assumption of Theorem B, for any \(\Phi \) with \(\Gamma \Phi >\Gamma \varphi ,\) the neutral coefficient \({\mathcal {F}}_{h,\Phi }[\eta ]\) vanishes. By [19, Theorem C], this implies the vanishing of \({\mathcal {F}}_{S,\Phi }[\eta ]\) for any pair \((S,\Phi )\) with \(\Gamma \Phi >\Gamma \varphi \). Thus, the term \(\mathcal {A}\) in Proposition 4.1.3 also vanishes. The theorem follows. \(\square \)
Additional results
Let us prove for future use some additional results that utilise the technique of this section. For the entire subsection we fix Whittaker pairs \((H,\varphi )\) and \((S,\varphi )\) such that \((H,\varphi )\) dominates \((S,\varphi )\). Let \(Z:=SH\) and let \(H_t:=H+tZ\). Let \(0<t_1<\dots<t_{n}<1\) be all the critical values between 0 and 1. Let \(t_0:=0\) and \(t_{n+1}:=1\). Lastly, for each \(t_i\), let \(R:=R_{t_i}\) and \(L:=L_{t_i}\) be defined as in (3.16).
Lemma 4.3.1
Let \(\eta \) be an automorphic function such that \(\Gamma \varphi \in {\text {WS}}(\eta )\). Then we have \({\mathcal {F}}^R_{H_{t_i},\varphi }[\eta ]={\mathcal {F}}^L_{H_{t_{i+1}},\varphi }[\eta ]\).
Proof
Denote \(H_{j}:=H_{t_{j}}\) for any j, and \({\mathfrak {c}}:=({\mathfrak {g}}^*)^{H_{i+1}}_{1}\cap ({\mathfrak {g}}^*)^{e_{\varphi }}\cap (\mathfrak {g}^*)^Z_{<0}\). Arguing as in the proof of Lemma 3.2.7(ii), we obtain
We have to show that for any nonzero \(\varphi '\in {\mathfrak {c}}\), we have \({\mathcal {F}}^L_{H_{i+1},\varphi ,\varphi '}[\eta ]=0\). But \({\mathcal {F}}^L_{H_{i+1},\varphi ,\varphi '}[\eta ]\) is an integral of \({\mathcal {F}}_{H_{i+1},\varphi ,\varphi '}[\eta ]\), which by Lemma 4.2.2 is expressed through coefficients \({\mathcal {F}}_{S_j,\Phi _j}\) with \(\Gamma \Phi _j>\Gamma \varphi \). Since \(\varphi \in {\text {WS}}(\eta ),\) \({\mathcal {F}}_{S_j,\Phi _j}[\eta ]\equiv 0\) for all j, and thus \({\mathcal {F}}^L_{H_{i+1},\varphi ,\varphi '}[\eta ]\equiv 0\). \(\square \)
Corollary 4.3.2
Let \(\eta \) be an automorphic function such that \(\Gamma \varphi \in {\text {WS}}(\eta )\). Let \({\mathfrak {v}}:= {\mathfrak {g}}^H_{> 1}/({\mathfrak {g}}^H_{> 1}\cap {\mathfrak {g}}^{S}_{\ge 1})\), \({\mathfrak {v}}':= {\mathfrak {g}}^S_{> 1}/({\mathfrak {g}}^S_{> 1}\cap ({\mathfrak {g}}^{H}_{\ge 1}+{\mathfrak {g}}_{\varphi }))\), \(V:={\text {Exp}}({\mathfrak {v}}(\mathbb {A}))\), and \(V':={\text {Exp}}({\mathfrak {v}}'(\mathbb {A}))\). Then,
Proof
With the notation introduced above, let
By Lemma 3.2.7(i) we have
By Lemma 4.3.1 we have \({\mathcal {F}}^R_{H_{t_i},\varphi }[\eta ]={\mathcal {F}}^L_{H_{t_{i+1}},\varphi }[\eta ]\). Thus
Similarly, let
Then the nilpotent group \(V'\) is glued from the commutative groups \(V'_i\) and
\(\square \)
Proposition 4.3.3
(cf. [19, Theorem A] for a local analogue) For any \(\eta \in C^{\infty }(\Gamma \backslash G)\) with \({\mathcal {F}}_{H,\varphi }[\eta ]\equiv 0\) we have \({\mathcal {F}}_{S,\varphi }[\eta ]\equiv 0\).
Proof
Let \(Z:=SH\), and for any \(t\ge 0\) let \(H_t:=H+tZ\). Let \(0< t_1< \dots< t_k < 1\) be all the critical values of t between 0 and 1. Let \(t_0:=0\) and \(t_{k+1}:=1\). By Lemma 3.2.7(i) and (ii), for any \(0\le i\le k\), \({\mathcal {F}}_{H_{t_{i+1}},\varphi }\) is expressed in terms of \({\mathcal {F}}_{H_{t_i},\varphi }\). Since \(H_{t_0}=H\), we obtain by induction that \({\mathcal {F}}_{H_{t_i},\varphi }[\eta ]\equiv 0\) for all i. Since \(H_{t_{k+1}}=S\), the proposition follows. \(\square \)
Levidistinguished coefficients
Let us show that any Whittaker pair \((H,\varphi )\) dominates a Levidistinguished Whittaker pair. Using Lemma 2.3.7, decompose \(H=h+Z\), where \((h,\varphi )\) is a neutral pair, and Z commutes with h and \(\varphi \).
Notation 4.4.1
Let \(C\subseteq \Gamma \) denote the centralizer of \((h,\varphi )\). Let A denote a maximal split torus of C such that its Lie algebra \({\mathfrak {a}}\) includes Z, and let M denote the centralizer of \({\mathfrak {a}}\) in G. Then M is a Levi subgroup of G, \(\mathfrak {m}\) includes h, Z and \(\varphi \). Let z be a rational semisimple element of \({\mathfrak {a}}\) that is generic in the sense that its centralizer is M.
Lemma 4.4.2
As an element of \(\mathfrak {m}\), \(\varphi \) is \({\mathbb K}\)distinguished.
Proof
Let \(\mathfrak {l}\) be the Lie algebra of a Levi subgroup L of M defined over \({\mathbb K}\) such that \(\varphi \in \mathfrak {l}^*\). We have to show that \(L=M\). By replacing L by its conjugate by an element of \(\Gamma \) we can assume \(h\in \mathfrak {l}\), and that there exists a rational semisimple element \(z'\in \mathfrak {m}\) such that \(\mathfrak {l}\) is the centralizer of \(z'\). Then \(z'\) commutes with h and \(\varphi \) and we have to show that \(z'\) is central in \(\mathfrak {m}\).
Indeed, \(z'\in \mathfrak {m}\cap {\mathfrak {c}}={\mathfrak {a}}\). Now, any \(X\in \mathfrak {m}\) commutes with z, and thus with any element of \({\mathfrak {a}}\), since z is generic in \({\mathfrak {a}}\). Thus \({\mathfrak {a}}\) lies in the center of \(\mathfrak {m}\) and thus \(z'\) is central. \(\square \)
Note that the eigenvalues of the adjoint action of any Lie algebra element are symmetric around zero.
Notation 4.4.3
Let N be a positive integer that is bigger than the ratio of the maximal eigenvalue of \({\text {ad}}(z)\) by the minimal positive eigenvalue of \({\text {ad}}(Z)\). Let
From our choice of N we have
That \(\mathfrak {m} \subseteq \mathfrak {g}^Z_0\) follows from the fact that M is the centralizer of z which equals the centralizer of \(\mathfrak {a}\) and \(\mathfrak {a}\) includes Z.
Lemma 4.4.4
For rational \(T > 0\), \((H, \varphi )\) dominates \((H+TZ', \varphi )\), that is, \(H,\varphi \) and \(T Z'\) commute, and satisfy (2.11).
Proof
By construction \(H = h + Z\), \(\varphi \) and Z commute, and since \(h, Z, \varphi \in \mathfrak {m}\) they commute with z. Thus, \(Z'\) commutes with H and \(\varphi \). Furthermore, \({\mathfrak {g}}_{\varphi }\cap {\mathfrak {g}}^H_{\ge 1}\subseteq {\mathfrak {g}}^h_{\le 0}\cap {\mathfrak {g}}^H_{\ge 1}\subseteq {\mathfrak {g}}^Z_{>0} \subseteq {\mathfrak {g}}^{Z'}_{>0} = {\mathfrak {g}}^{TZ'}_{>0}.\) \(\square \)
Lemma 4.4.5
For a fixed \(\lambda \in {\mathbb Q}\), and a rational \(T > 0\) large enough,
The Fourier coefficient \(\mathcal {F}_{H+TZ', \varphi }\) is then Levidistinguished.
Proof
For large enough T, we have that \(\mathfrak {g}^{H+TZ'}_{>1} \cap \mathfrak {g}^{Z'}_{<0} = \{0\}\) and \(\mathfrak {g}^{H+TZ'}_{>1} \cap \mathfrak {g}^{Z'}_{>0} = \mathfrak {g}^{Z'}_{>0}\). Thus \(\mathfrak {g}^{H+TZ'}_{>1} = \mathfrak {g}^{H+TZ'}_{>1} \cap \bigl ( \mathfrak {g}^{Z'}_{<0} \oplus \mathfrak {g}^{Z'}_0 \oplus \mathfrak {g}^{Z'}_{>0} \bigr ) = \mathfrak {g}^{Z'}_{>0} \oplus \bigl ( \mathfrak {g}^{Z'}_0 \cap \mathfrak {g}^{H+TZ'}_{>1} \bigr )\). Since \(H = h + Z\) and \(\mathfrak {g}^{Z'}_0 = \mathfrak {m} \subseteq \mathfrak {g}^Z_0\) we have that \(\mathfrak {g}^{Z'}_0 \cap \mathfrak {g}^{H+TZ'}_{>1} = \mathfrak {g}^{Z'}_0 \cap \mathfrak {g}^{h}_{>1}\) and since h is neutral \(\mathfrak {g}^{h}_{>1} = \mathfrak {g}^{h}_{\ge 2}\). Now \(\mathfrak {g}^{Z'}_0 = \mathfrak {m}\) and thus, \(\mathfrak {g}^{H+TZ'}_{>1} = \mathfrak {g}^{Z'}_{>0} \oplus \bigl ( \mathfrak {g}^{Z'}_0 \cap \mathfrak {g}^{h}_{\ge 2} \bigr ) = \mathfrak {g}^{Z'}_{>0} \oplus \mathfrak {m}^h_{\ge 2}\). Doing the same manipulations for \(\mathfrak {g}^{H+TZ'}_{\ge 2}\) one ends up with the same result, proving the equality \(\mathfrak {g}^{H+TZ'}_{>1} = \mathfrak {g}^{H+TZ'}_{\ge 2}\).
Now, for any fixed \(\lambda \in {\mathbb Q}\) and a large enough T, we have that \(\mathfrak {g}^{H+TZ'}_\lambda = \mathfrak {g}^{H}_\lambda \cap \mathfrak {g}^{Z'}_0 = \mathfrak {g}^{H}_\lambda \cap \mathfrak {m} = \mathfrak {m}^H_\lambda \). Again, since \(H = h + Z\) and \(\mathfrak {m} \subseteq \mathfrak {g}^Z_0\), we get that \(\mathfrak {m}^H_\lambda = \mathfrak {m}^h_\lambda \).
Since \(H + TZ' = h + Z + TZ'\), the semisimple element denoted by Z in Definition 2.3.5 is here \(Z + TZ'\), which, for large enough T has the centralizer \(\mathfrak {g}^Z_0 \cap \mathfrak {g}^{Z'}_0 = \mathfrak {g}^{Z'}_0 = \mathfrak {m}\). By Lemma 4.4.2, \(\varphi \) is \({\mathbb K}\)distinguished in \(\mathfrak {m}\). Since \(\mathfrak {g}^Z_{>0} \subseteq \mathfrak {g}^{Z'}_{>0}\) we have that \(\mathfrak {g}^{Z+TZ'}_{>0} = \mathfrak {g}^{Z'}_{>0}\) and thus (4.24) implies (2.12) which means that \({\mathcal {F}}_{H+TZ', \varphi }\) is Levidistinguished. \(\square \)
Corollary 4.4.6
Any Whittaker pair \((H,\varphi )\) dominates a Levidistinguished Whittaker pair.
Corollary 4.4.7
Algorithm A allows us to express any Fourier coefficient \({\mathcal {F}}_{H,\varphi }\) in terms of Levidistinguished Fourier coefficients with characters in orbits which are equal to or bigger than \(\Gamma \varphi \).
Proof
Choose a Levidistinguished Whittaker pair \((S,\varphi )\) dominated by \((H,\varphi )\). Then Algorithm A expresses \({\mathcal {F}}_{H,\varphi }\) in terms of \({\mathcal {F}}_{S,\varphi }\), and Fourier coefficients \({\mathcal {F}}_{H'_i,\Phi _i}\) corresponding to higher orbits. Each of the pairs \((H'_i,\Phi _i)\) dominates a Levidistinguished Whittaker pair \((S_i,\Phi _i)\). We repeat the procedure for each pair. The process terminates in a finite number of steps since the dimension of each complex orbit \(\mathbf{G}({\mathbb C})\Phi _i\) is bigger than that of \(\mathbf{G}({\mathbb C})\varphi \). \(\square \)
Remark 4.4.8

(i)
Note that \(\eta ={\mathcal {F}}_{0,0}[\eta ]\). Thus, the algorithm allows us to express any automorphic function in terms of its Levidistinguished Fourier coefficients.

(ii)
Algorithm A produces a general formula, that holds for all automorphic functions \(\eta \in C^{\infty }(\Gamma \backslash G)\). However, if we put additional assumptions on \(\eta \) the algorithm might terminate earlier and produce a shorter expression.

(iii)
By Lemma 2.3.9, the Levidistinguished Fourier coefficients of PL elements are Whittaker coefficients. This implies that if all Levidistinguished Fourier coefficients of some automorphic function \(\eta \) corresponding to nonPL orbits vanish, then the algorithm allows us to express \(\eta \) in terms of its Whittaker coefficients.

(iv)
Vanishing as in (iii) happens in two important cases. One is the case of \({\text {GL}}_n\) in which all orbits are PL orbits. We explain the results in this case in Sect. 5.3 below. Another case is the case when G is simplylaced, and \(\eta \) is minimal or nexttominimal. In this case the output of the algorithm is analyzed in great detail in [21].
Let us now go to the other extreme and consider cuspidal \(\eta \).
Lemma 4.4.9
Let \(\eta \in C^{\infty }(\Gamma \backslash G)\), and assume that the constant term \(c_U[\eta ]:=\int _{[U]}\eta (u)du\) vanishes for any \(U\subset G\) which is a unipotent radical of a proper parabolic subgroup. Let \({\mathcal {F}}_{S,\varphi }[\eta ]\) be a nonvanishing Levidistinguished Fourier coefficient. Then the orbit \(\Gamma \varphi \in {\mathfrak {g}}^*\) is \({\mathbb K}\)distinguished.
Proof
Recall that by Definition 2.3.5, there is a decomposition \(S=h+Z\) such that \((h,\varphi )\) be a neutral Whittaker pair for \(\mathfrak {l}:={\mathfrak {g}}^Z\), the orbit of \(\varphi \) in \(\mathfrak {l}^*\) is \({\mathbb K}\)distinguished, and
Let \(\mathfrak {p}:={\mathfrak {g}}^Z_{\ge 0}\), P be the corresponding parabolic subgroup, and U be the unipotent radical of P. By (4.25), \({\mathcal {F}}_{S,\varphi }[\eta ]={\mathcal {F}}_{h,\varphi }(c_U[\eta ])\), where we view \(c_U[\eta ]\) as an element of \(C^{\infty }(\Gamma \backslash G)\). Since \({\mathcal {F}}_{S,\varphi }[\eta ]\) does not vanish, neither does \(c_U[\eta ]\) and thus \(P=G\). Thus \(L=G\) and thus the orbit \(\Gamma \varphi \in {\mathfrak {g}}^*\) is \({\mathbb K}\)distinguished. \(\square \)
Applications and examples
In this section we will illustrate how to apply the framework introduced in this paper to compute certain Fourier coefficients in detail. We begin in Sect. 5.1 to consider the case when G is split and simplylaced and \(P\subset G\) is a parabolic subgroup with unipotent radical U isomorphic to a Heisenberg group. We use Algorithm A to express any automorphic function on G in terms of its Fourier coefficients with respect to U. In Sect. 5.2 we then give an example of a Whittaker triple and a quasiFourier coefficient for the group \(G={\text {SL}}_4\). In Sect. 5.3 we demonstrate Algorithm A, Corollary 4.4.7 and Remark 4.4.8 for \(G= {\text {GL}}_n\). In Sect.5.4 we demonstrate them for \(G= {\text {Sp}}_4\).
In [21] we apply Theorem B, Algorithm A, and Proposition 5.1.5 below to automorphic forms in small automorphic representations of all simple split simplylaced groups.
As many examples below are built on classical groups, we shall use matrix notation and denote by \(e_{ij}\) the elementary matrix with a 1 at position (i, j) and zeroes elsewhere.
Fourier expansions along Heisenberg parabolics
Let G be split and simplylaced, and let \({\mathfrak {h}}\subset {\mathfrak {g}}\) be the Lie algebra of a maximal split torus. Fix a choice of positive roots. For any simple root \({\alpha }\) define \(S_{{\alpha }}\in {\mathfrak {h}}\) by \({\alpha }(S_{{\alpha }})=2\) and \(\beta (S_{{\alpha }})=0\) for any other simple root \(\beta \).
Definition 5.1.1
We say that a simple root \(\alpha \) is a Heisenberg root if \(\mathfrak {g}^{S_\alpha }_{>0}\) is a Heisenberg Lie algebra, or, equivalently, if \({\mathfrak {g}}^{S_\alpha }_{4}\) has dimension one.
Lemma 5.1.2
If \({\mathfrak {g}}\) is simple of type \(A_n\), there are no Heisenberg roots. If \({\mathfrak {g}}\) is simple of type \(D_n\) or \(E_n\) then there exists a unique Heisenberg root, and this is the unique simple root \(\alpha \) satisfying \(\langle {\alpha }, {\alpha }_{\max }\rangle =1,\) where \({\alpha }_{\max }\) denotes the highest root.
Proof
Let \({\mathfrak {g}}\) be simple and let \({\alpha }\) be a Heisenberg root. Then \({\mathfrak {g}}^{S_\alpha }_{4}\) has to be the highest weight space of the adjoint representation, i.e. the root space of \({\alpha }_{\max }\). Since \({\mathfrak {g}}^{S_\alpha }_{4}\) is onedimensional, \({\alpha }_{\max }\beta \) is not a root for any simple root \(\beta \ne {\alpha }\). Thus \({\alpha }_{\max }{\alpha }\) is a root, and thus \(\langle {\alpha }, {\alpha }_{\max }\rangle =1\). The roots \({\alpha }\) with this property are precisely the nodes in the affine Dynkin diagram that are connected to the affine node (corresponding to \({\alpha }_{\max }\)).
Checking the affine Dynkin diagrams (see [6, Tables IVVII]), we see that there is a unique simple root \({\alpha }\) with this property in types \(D_n\) and \(E_n\), and these roots are indeed Heisenberg. In the Bourbaki notation, these roots are \({\alpha }_2\) for \(D_n\) and \(E_6\), \({\alpha }_1\) for \(E_7\) and \({\alpha }_8\) for \(E_8\). In type \(A_n\) there are two roots with this property but none of them is Heisenberg. In fact, \(\mathfrak {g}^{S_\beta }_{>0}\) is abelian for any simple root \(\beta \) in type \(A_n\). This is so, since in type \(A_n\), \({\alpha }_{\max }\) is the sum of all simple roots (with all coefficients being 1). \(\square \)
Notation 5.1.3
Let \({\alpha }\) be a root. Define \(h_{{\alpha }}:={\alpha }^{\vee }\in {\mathfrak {h}}\) by requiring for all roots \(\beta \)
Denote also by \({\mathfrak {g}}^{\times }_{{\alpha }}\) the set of nonzero covectors in the dual root space \({\mathfrak {g}}^{*}_{{\alpha }}\).
Note that for \(\beta \ne {\pm } {\alpha },\) \(\beta (h_{{\alpha }})\in \{1,0,1\}\). By [25, Proposition II.8.3], \((h_{{\alpha }},\varphi )\) is a neutral pair for any \(\varphi \in {\mathfrak {g}}^{\times }_{{\alpha }}\).
Notation 5.1.4
For any Heisenberg root \({\alpha }\), let \(\Omega _{{\alpha }}\subset \Gamma \) be the abelian subgroup obtained by exponentiation of the abelian Lie algebra given by the direct sum of the root spaces of negative roots \(\beta \) satisfying \(\langle {\alpha }, \beta \rangle =1\). Let
Note that all the roots in \(\Psi _{{\alpha }}\) have to be positive.
In this subsection we use Algorithm A to deduce the following proposition, that will be used in the sequel paper [21].
Proposition 5.1.5
Let \({\alpha }\) be a Heisenberg root. Let \(\gamma _{{\alpha }}\in \Gamma \) be a representative of a Weyl group element that conjugates \({\alpha }\) to \({\alpha }_{\max }\), where \({\alpha }_{\max }\) denotes the maximal root of the component of \({\mathfrak {g}}\) corresponding to \({\alpha }\). Then we have
For the proof we will need the following lemma.
Lemma 5.1.6
Let \({\alpha }\) be a Heisenberg root. Then for any \(\varphi \in {\mathfrak {g}}^{\times }_{{\alpha }}\) we have
Proof
Following Algorithm A we consider the deformation \((1t)h_{{\alpha }}+tS_{{\alpha }}\). By Lemma 3.1.1, we have
Then, the critical values are 1/2 and 2/3, and the quasicritical values are 1/3 and 1. At 1/3, we have no Whittaker triple entries yet and thus nothing moves into the \(2\)eigenspace. At 1/2, we get contributions in the third component of the Whittaker triple from the root spaces of all the roots \(\varepsilon \) with \(\langle \varepsilon , {\alpha }\rangle = 0\) and \(\varepsilon (S_{\alpha }) = 2\). At \(t=2/3\) we also get all the negative roots with \( \langle \varepsilon , {\alpha }\rangle = 1\) and \(\varepsilon (S_{\alpha }) = 2\). This means that we would get contributions from all these root spaces in the third component of the Whittaker triple. At \(t=1\) the Whittaker triple becomes a Whittaker pair and thus we obtain
\(\square \)
Since \(\eta ={\mathcal {F}}_{0,0}[\eta ],\) in order to express \(\eta \) in terms of Fourier coefficients of the form \({\mathcal {F}}_{S_{{\alpha }},\varphi }\) we need to consider the deformation \(S_t:=tS_{{\alpha }}\). To simplify the exposition we do that in more elementary terms.
Proof of Proposition 5.1.5
By the conditions, the Lie algebra \({\mathfrak {g}}^{S_{{\alpha }}}_{>0}\) is a Heisenberg Lie algebra, with center \({\mathfrak {g}}^{S_{{\alpha }}}_{4}\), and abelian quotient \({\mathfrak {g}}^{S_{{\alpha }}}_{2}\). We restrict \(\eta \) to the exponential of the center and decompose to Fourier series. The constant term with respect to the center \({\mathfrak {g}}^{S_{{\alpha }}}_{4}\) is \({\mathcal {F}}_{S_{{\alpha }}/3,0}[\eta ]\), and the other terms are \({\mathcal {F}}_{S_{{\alpha }}/2,\varphi }[\eta ]\) for \(\varphi \ne 0\in ({\mathfrak {g}}^*)^{S_{{\alpha }}}_{4} =({\mathfrak {g}}^*)^{S_{{\alpha }}/2}_{2}\). We remark that this constant term can be denoted \({\mathcal {F}}_{cS_{{\alpha }},0}[\eta ]\) for any \(1/4 \le c<1/2\) but not for \(c=1/2\) since 0 defines a zero form on the 1eigenspace, and thus \({\mathfrak {n}}_{cS_{{\alpha }},0}={\mathfrak {g}}^{cS_{{\alpha }}}_{\ge 1}={\mathfrak {g}}^{S_{{\alpha }}}_{\ge c^{1}}\) and \({\mathfrak {n}}_{S_{{\alpha }}/2,0}={\mathfrak {g}}^{S_{{\alpha }}}_{\ge 2}\). Note also that \(({\mathfrak {g}}^*)^{S_{{\alpha }}/2}_{2}={\mathfrak {g}}^{\times }_{{\alpha }_{\max }}\). Altogether we have
Note that \(\gamma _{{\alpha }}\) conjugates \(S_{{\alpha }}/2\) to \(h_{{\alpha }}\). Thus, by Lemma 3.2.8, we have
and
We restrict the constant term of (5.7) to the maximal abelian quotient of \({\text {Exp}}({\mathfrak {g}}^{S_{{\alpha }}}_{>0})\), decompose to Fourier series and obtain
Formula (5.3) follows now from (5.7), (5.8), (5.4) and (5.9). \(\square \)
Remark 5.1.7
Let us explain why we chose to use \(S_{{\alpha }}\) in Proposition 5.1.5. In types \(E_6,E_7,\) and \(E_8\) this choice follows Sect. 4.4. Indeed, the starting point is \(\eta ={\mathcal {F}}_{0,0}[\eta ]\). In the first step we choose a generic element z in the Cartan. Choose z to be 2 on the Heisenberg root \({\alpha }\), and to be very small positive rational numbers on other roots. This obtained deformation gives the same results as the deformation with \(Z=S_{{\alpha }}\). The first critical value is 1/4, at which we obtain the decomposition described in (5.7). With the constant term we can proceed to the next critical value 1/2, at which we obtain the decomposition in (5.9).
Then we conjugate the nonconstant terms obtained in (5.7) by \(\gamma _{{\alpha }}\). This is not part of the algorithm, but we do that for convenience. Now we need to choose a generic z that commutes with the root space of \({\alpha }\). We choose it to be 1 on the only simple root nonorthogonal to \({\alpha }\), and very small positive rational numbers on other simple roots. The resulting decomposition appears in Lemma 5.1.6. Altogether, this gives Proposition 5.1.5.
To express \(\eta \) in terms of its Levidistinguished coefficients one should continue with each of the terms in the righthand side of (5.3). However, the obtained expression would be very long and complicated. In [21] we provide the expression under the assumption that the Whittaker support of \(\eta \) consists of orbits with Bala–Carter labels \(A_1\) and \(2A_1\). Since these orbits are PL orbits, in this case we obtain an expression for \(\eta \) in terms of its Whittaker coefficients.
For groups of type \(D_n\), Sect. 4.4 would provide us with a different formula, but we still prove formula (5.3) for all cases for its uniformity, beauty, and future applications.
Remark 5.1.8
Here we elaborate a little on the structure of the Fourier expansion (5.3) and comment on the relation to previous works on Heisenberg expansions. The semisimple element \(S_{\alpha }\) defines a Heisenberg parabolic subgroup \(P_{\alpha }\subset G\) with Levi decomposition \(P_{\alpha }=LU\). The Lie algebra \(\mathfrak {p}_{\alpha }\subset \mathfrak {g}\) of \(P_{\alpha }\) exhibits the following grading
where the subscripts indicate the values of the inner products \(\langle \,\cdot \, , {\alpha }_{\max }\rangle \). Thus \(\mathfrak {g}_1\) is spanned by all roots \(\varepsilon \) such that \(\langle \varepsilon , {\alpha }_{\max }\rangle =1\). Equivalently, these are all roots \(\varepsilon \) such that \({\alpha }_{\max }\varepsilon \) is also a root. Notice that all roots in \(\mathfrak {g}_1\) are positive and the only simple root satisfying the condition \(\langle \varepsilon , {\alpha }_{\max }\rangle =1\) is \(\alpha \) itself. Since only \(\varepsilon =\alpha _{\max }\) satisfies \(\langle \varepsilon , {\alpha }_{\max }\rangle =2\) the space \(\mathfrak {g}_2\) is onedimensional and equals \(\mathfrak {g}_{\alpha _{\max }}\). The zeroth subspace \(\mathfrak {g}_0\) is the Lie algebra of the Levi \(L\subset P\). The subspace \(\mathfrak {g}_1\oplus \mathfrak {g}_2\) is thus the Heisenberg nilpotent subalgebra with center \(\mathfrak {g}_2\). Notice that \(\sum _{\gamma \in \Psi _\alpha }\mathfrak {g}_\gamma \) is a Lagrangian subspace of \(\mathfrak {g}_1\). Indeed, \(\mathfrak {g}_1\) has a canonical Lagrangian decomposition
where \(\Psi _{\alpha }^{\perp }\) is the orthogonal complement
Note that the root \(\alpha \) belongs to \(\Psi _{\alpha }^{\perp }\). The Fourier expansion (5.3) thus corresponds to the standard nonabelian Fourier expansion along the Heisenberg unipotent U, which exhibits a sum over the center \(\mathfrak {g}_2\) along with a sum over a Lagrangian subspace \(\Psi _{\alpha }\) of \(\mathfrak {g}_1\). The choice of Lagrangian decomposition is usually referred to as a choice of “polarization”. Similar kinds of expansions have been treated in several places in the literature; see [2, 13, 30, 31, 36] for a sample. In the notation of the original paper by Kazhdan and Savin [31], the space \(\Psi _{\alpha }\) corresponds to \(\Pi _o^{*}\) while \(\Psi _{\alpha }^{\perp }\) corresponds to \(\Pi _o\).
Whittaker triples
We will now illustrate what type of quasiFourier coefficients we are able to describe using Whittaker triples that are not captured by Whittaker pairs in an example for \(G = {\text {SL}}_4\).
Let \((S, \varphi , \psi )\) be the Whittaker triple with \(S = \tfrac{1}{3}{\text {diag}}(3,1,1,3)\), \(\varphi = e_{41}\) and \(\psi = m e_{31} + n e_{42}\), where \(m,n \in {\mathbb K}\) and \(e_{ij}\) denote elementary matrices. The Seigenvalues for the different elementary matrices can be illustrated by the following matrix
from which we may read out that \(\varphi \) has eigenvalue \(2\) while \(\psi \) has eigenvalue \(4/3\).
As seen from this matrix we get the following unipotent subgroup (independent of \(\psi \))
and the corresponding Fourier coefficient of an automorphic function \(\eta \) can be expressed as
where we recall that \(\chi \) is a fixed nontrivial character on \(\mathbb {A}\) trivial on \(\mathbb {K}\).
From this example we see that we require Whittaker triples in addition to Whittaker pairs if we want to construct Fourier coefficients with characters that are not only supported on \(x_1\) but also on \(x_2\) and \(x_3\).
The case of \({\text {GL}}_n\)
Let \(\mathbf {G}:={\text {GL}}_n\), \(G:={\text {GL}}_n(\mathbb {A})\), and \(\Gamma :={\text {GL}}_n({\mathbb K})\). In this section we will follow Algorithm A and Sect. 4.4 to present any automorphic function \(\eta \in C^{\infty }(\Gamma \backslash G)\) as a countable linear combination of its Whittaker coefficients. We will show that our proof amounts in this case morally to the same decomposition as in [28, 34, 37].
In [34, 37], \(\eta \) is first restricted to the mirabolic nilradical, i.e.
and decomposed into Fourier series with respect to U. Our algorithm does the same thing, but in several steps. First let \((h,\varphi )=(0,0)\). Let \(N\gg 0\),
and consider the deformation \(S_t:=tz_{1}\). Under this deformation, the first thing that happens is that the highest root space (spanned by \(e_{1n}\)) enters \({\mathfrak {g}}^{S_t}_1\). At this point \(\eta \) decomposes into a sum of quasiFourier coefficients. At the next step \(e_{1n}\) enters \({\mathfrak {g}}^{S_t}_2\), and the quasiFourier coefficients become Fourier coefficients. For the constant term, we continue with the same deformation, until \(e_{2n}\) enters. For the nonconstant term we have to change the deformation into something that will commute with the corresponding new \(\varphi \). This \(\varphi \) can be identified with \(ae_{n1}\) under the trace form, for some \(a\in {\mathbb K}^{\times }\). We take the deformation by
and continue in the same way. Eventually, all of U enters and all possible characters (including the trivial one) appear.
Let us now analyze the summands. The constant term is \({\mathcal {F}}_{tz_1,0}\) for \(t=2/(N^{n2}N^{n3})\), and we can continue the deformation along \(z_1\). Any nontrivial character of U can be conjugated using \({\text {GL}}_{n1}\) (embedded into the upper left corner) to the one given by \(e_{n,n1}\). We can now choose the deformation
In the same way as above, it will give a decomposition of \({\mathcal {F}}_{tz_1,0}\) into Fourier series with respect to the column \(n1\), i.e.
Continuing in this way we obtain
where \([n1]\) denotes the set \(\{1,\dots ,n1\}\), \(2^{[n1]}\) denotes the set of all its subsets, \(S={\text {diag}}(n1,n3,\dots , 3n,1n)\), and for any \(x\in 2^{[n1]}\), \(\psi _x:=\sum _{i\in x}e_{i+1,i}\) and \(\Gamma _x\) is a certain subset of \(\Gamma \).
For cuspidal \(\eta \) and \(x\ne [n1]\), we have \({\mathcal {F}}_{S,\varphi _x}[\eta ]=0\) and (5.21) becomes the formula in [34, 37]. For \(\eta \) in the discrete spectrum, \({\mathcal {F}}_{S,\varphi _x}[\eta ]\) vanishes for many x by [28, Lemma 3.2], and (5.21) reflects the formula in [28, Theorem 3.3]. If \(\eta \) is minimal then \({\mathcal {F}}_{S,\varphi _x}[\eta ]=0\) for \(x>1\) and if \(\eta \) is nexttominimal then \({\mathcal {F}}_{S,\varphi _x}[\eta ]=0\) for \(x>2\). These cases were computed in [1], motivated by applications in string theory.
Examples for \({\text {Sp}}_{4}\)
Let \(G:={\text {Sp}}_4(\mathbb {A})\), \(\Gamma :={\text {Sp}}_4({\mathbb K})\) and let \(\eta \in C^{\infty }(\Gamma \backslash G)\). In this section we express \(\eta \) in terms of its Levidistinguished Fourier coefficients, providing an example for Algorithm A and Remark 4.4.8. Let \({\mathfrak {g}}:={\text {Lie}}(\Gamma )\), realized in \({\mathfrak {gl}}_4\) by the \(2\times 2\) block matrices
Let \({\mathfrak {n}}\subset {\mathfrak {g}}\) be the maximal unipotent subalgebra spanned by the matrices \(e_{12}e_{43}\), \(e_{13}\), \(e_{24}\), \(e_{14}+e_{23}\) and let \(N:={\text {Exp}}({\mathfrak {n}}(\mathbb {A}))\). For any \(a,b\in {\mathbb K}\) denote by \(\chi _{a,b}\) the character of \({\mathfrak {n}}\) given by \(\chi _{a,b}(e_{12}e_{43})=a\) and \(\chi _{a,b}(e_{24})=b\), and let \(\mathcal {W}_{a,b}\) denote the corresponding Whittaker coefficient. Let \({\mathfrak {u}}\subset {\mathfrak {n}}\) be the Siegel nilradical, i.e. the normal commutative subalgebra spanned by the matrices \(e_{13}\), \(e_{24}\), \(e_{14}+e_{23}\) and let \(U:={\text {Exp}}({\mathfrak {u}}(\mathbb {A}))\). Let L denote the Siegel Levi subgroup of \(\Gamma \) given by \({\text {diag}}(g,(g^t)^{1})\), where \(g\in {\text {GL}}_2(\mathbb {A})\). Using the trace form on \({\mathfrak {g}}\), we can identify \({\mathfrak {u}}^*\) with the nilradical \(\bar{\mathfrak {u}}\) of the opposite parabolic, i.e. with the space of matrices of the form (5.22) with \(A=B=0\). Note that \(\bar{\mathfrak {u}}\cong {\text {Sym}}^2({\mathbb K}^2)\), and L acts on it by the standard action on symmetric forms. For any \(\varphi \in {\mathfrak {u}}^*\cong \bar{\mathfrak {u}}\cong {\text {Sym}}^2({\mathbb K}^2)\), denote by \({\mathcal {F}}_{{\mathfrak {u}},\varphi }\) the corresponding parabolic Fourier coefficient.
Let us now outline the strategy for this subsection. According to Sect. 4.4 we should choose a generic element z of the Cartan. Choose it to be 1 on the long simple root \({\alpha }_2=2\varepsilon _2\) that defines U and a small positive rational number on the short simple root. Then, the first steps of the algorithm will provide us with decomposition of \(\eta \) into Fourier series along the abelian unipotent radical U. The coefficients will be parameterized by characters of \({\mathfrak {u}}\), that can be identified with quadratic forms on \({\mathbb K}^2\). For the zero form we can just continue the deformation all the way to a Whittaker coefficient. All forms of rank one are conjugate, so we can conjugate them to a convenient form and again continue the deformation, obtaining a Whittaker coefficient. The same goes for split forms of rank two. The nonsplit forms of rank two belong to \({\mathbb K}\)distinguished orbits, and thus the corresponding Fourier coefficients are already Levidistinguished.
We will now give all the details of the decompositions we just mentioned. We will do this in elementary terms in a selfcontained way.
Since U is abelian, the Fourier decomposition on it gives
We now decompose this sum into three different terms, by the rank of \(\varphi \), viewed as a quadratic form. Let us first analyze the constant term \({\mathcal {F}}_{{\mathfrak {u}},0}[\eta ]\). We restrict it to L, and decompose to Fourier series on the abelian group \(N\cap L\). We obtain
Next, any \(\varphi \) of rank one is conjugate under L to \(\varphi _1:= \left( \begin{array}{c@{\quad }c} 1 &{} 0 \\ 0 &{} 0\\ \end{array}\right) \). This \(\varphi _1\) is normalized by N, and thus we can again decompose \({\mathcal {F}}_{{\mathfrak {u}},\varphi }[\eta ]\) on \(N\cap L\). We obtain
The nondegenerate forms (i.e. those of rank two) can be divided into two subsets: split and nonsplit. All the split ones are conjugate under L to . Let \(w\in \Gamma \) denote a representative for the Weyl group element given by the simple reflection with respect to the long simple root \({\alpha }_2=2\varepsilon _2\), e.g. \(w={\text {diag}}(1,1,1,1)\sigma _{24}\), where \(\sigma _{24}\) is the permutation matrix on indices 2 and 4. Then \({\mathfrak {u}}^w={{\text {Span}}}(e_{12}e_{43}, e_{13}, e_{42})\), and \(\varphi _2^w\) equals the restriction to \({\mathfrak {u}}^w\) of \(\chi _{1,0}\). Using Corollary 3.1.2, we can express \({\mathcal {F}}_{{\mathfrak {u}}^w,\chi _{1,0}}\) through \({\mathcal {F}}_{{\mathfrak {u}}',\chi _{1,0}}\), where \({\mathfrak {u}}'={{\text {Span}}}(e_{12}e_{43}, e_{13}, e_{24})\subset {\mathfrak {n}}\). The integration will be over matrices of the form \(v_x:={\text {Id}}+xe_{24}\in G\). Using Fourier expansion by the remaining coordinate of \(e_{14}+e_{23}\in {\mathfrak {n}}\), we obtain
Finally, let \(X\subset \bar{u} \cong {\mathfrak {u}}^*\) denote the set of anisotropic nondegenerate forms. For \(\varphi \in X\), we have no expression of \({\mathcal {F}}_{{\mathfrak {u}},\varphi }[\eta ]\) in terms of Whittaker coefficients. However, any \(\varphi \in X\) is \({\mathbb K}\)distinguished. Indeed, let \(h:={\text {Id}}\in \mathfrak {l}\). Then \((h,\varphi )\) is a neutral pair, and its centralizer is anisotropic. By Lemma 4.4.2 applied to \((h,\varphi )\) and \(Z:=0\), \(\varphi \) is \({\mathbb K}\)distinguished.
Combining (5.23)–(5.26) we obtain the following theorem, that exemplifies Algorithm A and Remark 4.4.8.
Theorem 5.4.1
For any \(\eta \in C^{\infty }(\Gamma \backslash G)\) and \(g\in G\), \(\eta (g)\) equals
where \(O(1,1)\subset L\) denotes the stabilizer of the split form \(\varphi _2\).
If \(\eta \) is cuspidal then \(\mathcal {W}_{0,a}[\eta ]=\mathcal {W}_{a,0}[\eta ]=0\). If \(\eta \) is nongeneric, then \(\mathcal {W}_{1,a}[\eta ]=\mathcal {W}_{a,1}[\eta ]=0\), unless \(a=0\). Thus Theorem 5.4.1 implies the following corollary.
Corollary 5.4.2
Let \(\eta \in C^{\infty }(\Gamma \backslash G)\) and \(g\in G\).

(i)
If \(\eta \) is cuspidal then
(5.28) 
(ii)
If \(\eta \) is nongeneric then
(5.29) 
(iii)
If \(\eta \) is cuspidal and nongeneric then \(\eta =\sum _{\varphi \in X}{\mathcal {F}}_{{\mathfrak {u}},\varphi }[\eta ]\).
References
Ahlén, O., Gustafsson, H.P.A., Kleinschmidt, A., Liu, B., Persson, D.: Fourier coefficients attached to small automorphic representations of \(\text{SL}_n({\mathbb{A}})\), J. Number Theory 192 (2018) 80–142, arXiv:1707.08937 [math.RT]
Bao, L., Kleinschmidt, A., Nilsson, B.E.W., Persson, D., Pioline, B.: Instanton corrections to the universal hypermultiplet and automorphic forms on SU(2,1). Commun. Num. Theor. Phys. 4, 187–266 (2010). arXiv:0909.4299 [hepth]
Borel, A.: Linear algebraic groups. Graduate Texts in Mathematics, vol. 126, 2nd edn. SpringerVerlag, New York (1991)
Borel, A., Serre, J.P.: Théorèmes de finitude en cohomologie galoisienne. Comment. Math. Helv. 39, 111–164 (1964)
Borel, A., Tits, J.: Groupes réductifs. Inst. Hautes Études Sci. Publ. Math. 27, 55–150 (1965)
Bourbaki, N.: Éléments de mathématique. Fasc. XXXIV. Groupes et algèbres de Lie. Chapitre IV: Groupes de Coxeter et systèmes de Tits. Chapitre V: Groupes engendrés par des réflexions. Chapitre VI: systèmes de racines. Actualités Scientifiques et Industrielles, No. 1337. Hermann, Paris, (1968)
Bourbaki, N.: Éléments de mathématique. Fasc. XXXVIII: Groupes et algèbres de Lie. Chapitre VII: Sousalgèbres de Cartan, éléments réguliers. Chapitre VIII: Algèbres de Lie semisimples déployées. Actualités Scientifiques et Industrielles, No. 1364. Hermann, Paris, (1975)
Brylinski, J.L., Deligne, P.: Central extensions of reductive groups by \(\mathbf{K}_2\). Publ. Math. Inst. Hautes Études Sci. 94, 5–85 (2001)
Carter, R.W.: Finite groups of Lie type. Conjugacy classes and complex characters. Wiley, New York (1985)
Casselman, W., Shalika, J.: The unramified principal series of \(p\)adic groups. II. The Whittaker function. Compos. Math. 41(2), 207–231 (1980)
Collingwood, D.H., McGovern, W.M.: Nilpotent orbits in semisimple Lie algebras. Van Nostrand Reinhold Mathematics Series. Van Nostrand Reinhold Co., New York (1993)
Đoković, D. Ž.: Note on rational points in nilpotent orbits of semisimple groups, Indag. Math. (N.S.)9(1), 31–34 (1998)
Fleig, P., Gustafsson, H.P.A., Kleinschmidt, A., Persson, D.: Eisenstein series and automorphic representations, vol. 176 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 2018. arXiv:1511.04265 [math.NT]
Ginzburg, D., Hundley, J.: Constructions of global integrals in the exceptional groups, arXiv:1108.1401 [math.RT]
Ginzburg, D., Rallis, S., Soudry, D.: The descent map from automorphic representations of \({\rm GL}(n)\) to classical groups. World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, (2011)
Ginzburg, D.: Certain conjectures relating unipotent orbits to automorphic representations. Israel J. Math. 151, 323–355 (2006)
Ginzburg, D., Rallis, S., Soudry, D.: On the automorphic theta representation for simply laced groups. Israel J. Math. 100, 61–116 (1997)
Gomez, R., Gourevitch, D., Sahi, S.: Whittaker supports for representations of reductive groups, Annales de l’Institut Fourier (2021). https://doi.org/10.5802/aif.3372
Gomez, R., Gourevitch, D., Sahi, S.: Generalized and degenerate Whittaker models. Compos. Math. 153(2), 223–256 (2017)
Gourevitch, D., Gustafsson, H.P.A., Kleinschmidt, A., Persson, D., Sahi, S.: Eulerianity of Fourier coefficients of automorphic forms. Represent Theor. (2021). https://doi.org/10.1090/ert/565
Gourevitch, D., Gustafsson, H.P.A., Kleinschmidt, A., Persson, D., Sahi, S.: Fourier coefficients of minimal and nexttominimal automorphic representations of simplylaced groups, Can. J. Math. (2020) 1–48, 1908.08296 [math.NT]
Gourevitch, D., Sahi, S.: Degenerate Whittaker functionals for real reductive groups. Am. J. Math. 137(2), 439–472 (2015)
Green, M.B., Gutperle, M.: Effects of D instantons. Nucl. Phys. B 498, 195–227 (1997). arXiv:hepth/9701093
Green, M.B., Miller, S.D., Vanhove, P.: Small representations, string instantons, and Fourier modes of Eisenstein series. J. Number Theor. 146, 187–309 (2015). arXiv:1111.2983 [hepth]
Humphreys, J.E.: Introduction to Lie algebras and representation theory, vol. 9 of Graduate Texts in Mathematics. SpringerVerlag, New YorkBerlin, 1978. Second printing, revised
Hundley, J., Sayag, E.: Descent construction for GSpin groups, Mem. Am. Math. Soc. 243 no. 1148, v+124 (2016)
Hundley, J.: Email correspondence
Jiang, D., Liu, B.: On Fourier coefficients of automorphic forms of \({\rm GL}(n)\). Int. Math. Res. Not. IMRN 17, 4029–4071 (2013)
Jiang, D., Liu, B., Savin, G.: Raising nilpotent orbits in wavefront sets. Represent. Theory 20, 419–450 (2016)
Kazhdan, D., Pioline, B., Waldron, A.: Minimal representations, spherical vectors, and exceptional theta series, Commun. Math. Phys. 226 (2002) 1–40, arXiv:hepth/0107222 [hepth]
Kazhdan, D., Savin, G.: The smallest representation of simply laced groups, In: Festschrift in honor of I. I. PiatetskiShapiro on the occasion of his sixtieth birthday, Part I (Ramat Aviv, 1989), vol. 2 of Israel Math. Conf. Proc., pp. 209–223. Weizmann, Jerusalem, (1990)
Mœglin, C., Waldspurger, J.L.: Spectral decomposition and Eisenstein series, vol. 113 of Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, 1995. Une paraphrase de l’Écriture [A paraphrase of Scripture]
Mœglin, C., Waldspurger, J.L.: Modèles de Whittaker dégénérés pour des groupes \(p\)adiques. Math. Z. 196(3), 427–452 (1987)
PiatetskiShapiro, I.I.: Multiplicity one theorems, In: Automorphic forms, representations and \(L\)functions (Proc. Sympos. Pure Math., Oregon State Univ., Corvallis, Ore., 1977), Part 1, Proc. Sympos. Pure Math., XXXIII, pp. 209–212. Amer. Math. Soc., Providence, R.I., (1979)
Pioline, B.: \(R^4\) couplings and automorphic unipotent representations. JHEP 03, 116 (2010). arXiv:1001.3647 [hepth]
Pioline, B., Persson, D.: The automorphic NS5brane. Commun. Num. Theor. Phys. 3, 697–754 (2009). arXiv:0902.3274 [hepth]
Shalika, J.A.: The multiplicity one theorem for \({\rm GL}_{n}\). Ann. Math. 2(100), 171–193 (1974)
Acknowledgements
The authors are grateful for helpful discussions with Ben Brubaker, David Ginzburg and Stephen D. Miller. We are particularly thankful to Joseph Hundley for sharing with us his insights on nilpotent orbits for exceptional groups. We wish to thank the anonymous referees for very useful comments on an earlier version of this paper. We also thank the Banff International Research Station for Mathematical Innovation and Discovery and the Simons Center for Geometry and Physics for their hospitality during different stages of this project. D.G. was partially supported by ERC StG grant 637912 and BSF grant 2019724. During his time at Stanford University, H.G. was supported by the Knut and Alice Wallenberg Foundation. Later, H.G. was supported by the Swedish Research Council (Vetenskapsrådet), grant no. 201806774. S.S. was partially supported by NSF grants DMS1939600 and DMS2001537, and Simons’ foundation grant 509766. D.P. was supported by the Swedish Research Council (Vetenskapsrådet), grant no. 201804760.
Funding
Open access funding provided by University of Gothenburg.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Dr. Henrik P. A. Gustafsson was affiliated to Department of Mathematics, Stanford University, Stanford, CA 94305–2125, USA (Until September 15, 2019).
Appendices
Appendix A. On PLorbits
A complex orbit is a PLorbit if and only if its Bala–Carter label has no parenthesis. In particular, all complex minimal and nexttominimal orbits are PL. The classification of PL orbits of complex classical groups in terms of the corresponding partitions is given in [22, Sect. 6].
The classification of rational PLorbits is a more complicated task. In this subsection we discuss the PL property for small \({\mathbb K}\)rational orbits of simple split groups. A complex orbit \({\mathcal {O}}_{{\mathbb C}}\) may include several or even infinitely many rational orbits. If \({\mathcal {O}}_{{\mathbb C}}\) is nonPL then all its rational orbits are nonPL. If \({\mathcal {O}}_{{\mathbb C}}\) is PL then it includes at least one rational PLorbit, but can also include nonPL rational orbits. In type \(A_n\), all rational orbits are PL. Let us now describe the PL properties of minimal and nexttominimal orbits. Here, minimal and nexttominimal refers to the closure order on the complex orbits, which might be coarser than the order defined in Definition 2.4.1.
All minimal rational orbits are PL. Indeed, for classical groups it is easy to establish the Levi in which they are principal: for \({\text {SO}}_{n+1,n}\) it is \({\text {SO}}_{2,1}\times ({\text {GL}}_1)^{n1}\), for \({\text {Sp}}_{2n}\) it is \({\text {Sp}}_2\times ({\text {GL}}_1)^{n1}\) and for \({\text {SO}}_{n,n}\) it is \({\text {SO}}_{2,2}\times ({\text {GL}}_1)^{n2}\). For exceptional groups, the rational minimal orbit is unique and thus PL. This uniqueness was explained to us by Joseph Hundley. Let us now deal with the nexttominimal orbits.
Lemma A.1
All nexttominimal rational orbits for \({\text {SO}}_{n,n}\) and \({\text {SO}}_{n+1,n}\) are PL.
Proof
One can give a classification of the rational orbits in the spirit of the classification of real orbits given in [11, Sect. 9.3]. Namely, a \({\mathbb K}\)rational orbit with a given partition is defined by a collection of quadratic forms \(Q_{2i+1}\) on multiplicity spaces of the odd parts. If we add a hyperbolic form to the direct sum of these forms we get the initial form, which is also hyperbolic. Here, a hyperbolic form is a direct multiple of the 2dimensional quadratic form given by \(H(x,y)=xy\). By Witt’s cancellation theorem this implies that the direct sum of the forms on multiplicity spaces of the odd parts is hyperbolic.
An orbit for \({\text {SO}}_{n,n}\) is PL if and only if all \(Q_{2i+1}\) are hyperbolic, except \(Q_{2j+1}\) for a single index \(j\ge 1\), which is a direct sum of a hyperbolic form and a onedimensional quadratic form. For \({\text {SO}}_{n,n}\) there are two nexttominimal partitions. One of them is \(2^41^{2n8}\). For it, \(Q_1\) has to be hyperbolic. The other nexttominimal partition is \(3 1^{2n3}\). Thus \(Q_3\) is onedimensional. Now, note that \(H^n=Q_3\oplus Q_3 \oplus H^{n1}\). Thus, \(Q_3\oplus Q_1=Q_3\oplus Q_3 \oplus H^{n1}\) and thus \(Q_1=(Q_3) \oplus H^{n1}\), i.e. \(Q_1\) is a direct sum of a hyperbolic form and a onedimensional quadratic form.
Similarly, it is easy to see that the nexttominimal orbits for \({\text {SO}}_{n+1,n}\) are principal in Levis isomorphic to \(({\text {GL}}_2)^2\times ({\text {GL}}_1)^{n4}\) or \({\text {SO}}_{2,1}\times ({\text {GL}}_1)^{n1}\). \(\square \)
However, \({\text {Sp}}_{2n}({\mathbb K})\) has infinitely many rational nexttominimal orbits, already for \(n=2\). Moreover, there exist cuspidal nexttominimal representations of \({\text {Sp}}_4(\mathbb {A})\). Note that cuspidal nongeneric automorphic forms cannot expressed through their Whittaker coefficients, since the latter coefficients have to vanish on such forms. See [16, Sect. 4] for a discussion of cuspidal representations, in particular those of \({\text {Sp}}_4(\mathbb {A})\).
As for the exceptional groups, Joseph Hundley showed that the nexttominimal orbit is unique, and thus PL, for \(E_6,E_7,E_8\) and \(G_2\) [27].
We prove the following lemma in AppendixThe group \(F_4\) has infinitely many rational nexttominimal orbits. We expect that infinitely many of them are not PL.
Appendix B. Some geometric lemmas
In this appendix we show some properties of the partial order on rational nilpotent orbits defined in Definition 2.4.1.
Lemma B.1
Let \(Z\in {\mathfrak {g}}\) be rational semisimple, let \(\varphi \in {\mathfrak {g}}^Z_0\) and \(\varphi '\in {\mathfrak {g}}^Z_{>0}\). Assume that \(\varphi \) is conjugate to \(\varphi +\varphi '\) by \(\mathbf{G}({\mathbb C})\). Then there exist \(X\in {\mathfrak {g}}^{Z}_{>0}\) such that \({\text {ad}}^*(X)(\varphi )=\varphi '\) and \(v\in {\text {Exp}}({\mathfrak {g}}^{Z}_{>0})\) such that \({\text {Ad}}^*(v)(\varphi )=\varphi +\varphi '\).
Proof
Decompose \(\varphi '=\sum _{i=1}^k\varphi '_i\) where \(\varphi '_i\in ({\mathfrak {g}}^*)^{Z}_{{\lambda }_i}\) and \({\lambda }_1<{\lambda }_2<\dots <{\lambda }_k\in {\mathbb Q}_{>0}\) are all the positive eigenvalues of Z.
Let us first construct X using the fixed complex embedding \(\sigma : \mathbb {K} {\hookrightarrow }\mathbb {C}\) from Sect. 2. For any \(t\in {\mathbb R}\), we have the following identity in \({\mathfrak {g}}^{*}({\mathbb C}) := \mathfrak {g}^* \otimes _{\sigma (\mathbb {K})} \mathbb {C}\):
Thus, \(\varphi + \sum _i {\text {Ad}}^*(\exp (t{\lambda }_i))\varphi '_i\in G({\mathbb C})\varphi \). Differentiating by t at 0 we obtain that \(\sum _i {\lambda }_i\varphi '_i\) lies in the tangent space to the orbit \(\mathbf{G}({\mathbb C})\varphi \) at \(\varphi \). This tangent space is the image of \(\varphi \) under the coadjoint action. Thus there exists \(Y_{{\mathbb C}}\in {\mathfrak {g}}({\mathbb C})\) with \({\text {ad}}^*(Y_\mathbb {C})(\varphi )=\sum _i {\lambda }_i\varphi '_i\). Since both \(\varphi \) and \(\sum _i {\lambda }_i\varphi '_i\) lie in the \({\mathbb K}\)points \({\mathfrak {g}}^*\), there exists \(Y\in {\mathfrak {g}}\) satisfying the same linear equation. Decompose \(Y=Y'+\sum _i Y_i\) with \(Y_i\in {\mathfrak {g}}^Z_{{\lambda }_i}\). Since \(\varphi \) commutes with Z, we obtain \({\text {ad}}^*(Y_i)(\varphi )={\lambda }_i\varphi '_i\). Now we take \(X:=\sum _i {\lambda }_i^{1}Y_i \in \mathfrak {g}^Z_{>0}\).
We now prove the existence of v by descending induction on the maximal index i such that \(\varphi '\in {\mathfrak {g}}^Z_{>{\lambda }_i}\). The base case \(i=k\) has \(\varphi '=0\). For the induction step, let \(i<k\) such that \(\varphi '\in {\mathfrak {g}}^Z_{>{\lambda }_i}\). Then \({\text {Ad}}^*(\exp (X))(\varphi +\varphi ')=\varphi +\psi \), where \(\psi \in {\mathfrak {g}}^Z_{>{\lambda }_{i+1}}\). By the induction hypothesis, \(\varphi +\psi \in {\text {Ad}}^*({\text {Exp}}({\mathfrak {g}}^{Z}_{>0}))\varphi \). \(\square \)
Corollary B.2
Let \({\mathcal {O}}\ne {\mathcal {O}}'\) be two nilpotent \(\Gamma \)orbits with \(({\mathcal {O}}, {\mathcal {O}}')\in R\) (see Definition 2.4.1), and let \({\mathcal {O}}_{{\mathbb C}},{\mathcal {O}}'_{{\mathbb C}}\) denote their complexifications. Then \(\dim {\mathcal {O}}_{{\mathbb C}}<\dim {\mathcal {O}}'_{{\mathbb C}}\).
Proof
By Lemma 2.4.2 we have \({\mathcal {O}}_{{\mathbb C}}\subset \overline{{\mathcal {O}}'_{{\mathbb C}}}\). Thus either \(\dim {\mathcal {O}}_{{\mathbb C}}<\dim {\mathcal {O}}'_{{\mathbb C}}\) or \( {\mathcal {O}}_{{\mathbb C}}= {\mathcal {O}}'_{{\mathbb C}}\). If \( {\mathcal {O}}_{{\mathbb C}}= {\mathcal {O}}'_{{\mathbb C}}\) then, by the definition of R, there exist a rational semisimple \(Z\in {\mathfrak {g}}\), \(\varphi \in {\mathcal {O}}\cap {\mathfrak {g}}^Z_0\), and \(\psi \in {\mathfrak {g}}^Z_{>0}\) such that \(\varphi +\psi \in {\mathcal {O}}_{{\mathbb C}}\), but \(\varphi +\psi \notin {\mathcal {O}}\). This contradicts Lemma B.1. \(\square \)
Corollary B.3
Definition 2.4.1 indeed defines an order relation.
Proof
We have to show that if \({\mathcal {O}}\ge {\mathcal {O}}'\) and \({\mathcal {O}}\le {\mathcal {O}}'\) then \({\mathcal {O}}={\mathcal {O}}'\). By Lemma 2.4.2 the complexifications \({\mathcal {O}}'_{{\mathbb C}}\) and \({\mathcal {O}}_{{\mathbb C}}\) coincide. Moreover, there exist orbits \({\mathcal {O}}^i\) such that \({\mathcal {O}}^1={\mathcal {O}}\), \({\mathcal {O}}^n={\mathcal {O}}'\) and \(({\mathcal {O}}^{i+1}, {\mathcal {O}}^i) \in R\) with the same complexification. By Lemma B.1, all these orbits are the same. \(\square \)
Lemma B.4
Let \(Z,S\in {\mathfrak {g}}\) be commuting rational semisimple elements, let \(q\in {\mathbb Q}\) and let \(\varphi \in {\mathfrak {g}}^Z_0\cap {\mathfrak {g}}^{S}_{q}\) and \(\varphi '\in {\mathfrak {g}}^Z_{>0}\cap {\mathfrak {g}}^{S}_{q}\). Assume that \(\varphi \) is conjugate to \(\varphi +\varphi '\) by \(\mathbf{G}({\mathbb C})\). Then there exist \(X\in {\mathfrak {g}}^{Z}_{>0}\cap {\mathfrak {g}}^{S}_{0}\) such that \({\text {ad}}^*(X)(\varphi )=\varphi '\) and \(v\in {\text {Exp}}({\mathfrak {g}}^{Z}_{>0}\cap {\mathfrak {g}}^{S}_{0})\) such that \({\text {Ad}}^*(v)(\varphi )=\varphi +\varphi '\).
Proof
To construct X we proceed in the same way as in the proof of Lemma B.1, and then decompose it with respect to eigenspaces of S and take projection on the 0 eigenspace. Then we construct v in the same way as in the proof of Lemma B.1. \(\square \)
Lemma 4.2.1 follows now from Lemma B.4 (with \(q=2\)) and Corollary B.2.
We would now like to relate the notion of dominance to dimensions. For any Whittaker pair \((S,\varphi )\) define
Example B.5
If \((S,\varphi )\) is a neutral pair then \(d(S,\varphi )=(\dim \Gamma \varphi )/2\). If \((S,\varphi )\) is a Levidistinguished pair corresponding to a nilpotent orbit \({\mathcal {O}}\) in a Levi subalgebra \(\mathfrak {l}\subset {\mathfrak {g}}\) then \(d(S,\varphi )=(\dim {\mathfrak {g}} \dim \mathfrak {l}+\dim {\mathcal {O}})/2\). Furthermore, if \({\mathfrak {g}}^{S}_1 = \{0\}\), as in for example a parabolic Fourier coefficient, then \(d(S,\varphi ) = \dim {\mathfrak {n}}_{S,\varphi }\).
Lemma B.6
The number \(d(S,\varphi )\) equals the dimension of any maximal isotropic subspace of \({\mathfrak {u}}_S:={\mathfrak {g}}^S_{\ge 1}\).
Proof
Any such subspace includes \({\mathfrak {n}}_{S,\varphi }\), and the quotient is Lagrangian in the symplectic space \( {\mathfrak {u}}_S/{\mathfrak {n}}_{S,\varphi }\). \(\square \)
We study Fourier coefficients with respect to maximal isotropic subspaces, socalled Fourier–Jacobi coefficients, in [20].
Lemma B.7
Let \((H,\varphi )\) and \((S,\varphi )\) be Whittaker pairs with the same \(\varphi \).

(i)
If \((H,\varphi )\) dominates \((S,\varphi )\) then \(d(H,\varphi )\le d(S,\varphi )\).

(ii)
If \((S,\varphi )\) is Levidistinguished then \(\dim {\mathfrak {n}}_{H,\varphi }\le \dim {\mathfrak {n}}_{S,\varphi }\).
Proof
For part (i) let \(Z:=SH\) and choose a Lagrangian subspace \(a\subset ({\mathfrak {g}}^H_1\cap {\mathfrak {g}}^Z_{0})/( {\mathfrak {g}}^H_1\cap {\mathfrak {g}}^Z_{0}\cap {\mathfrak {g}}_{\varphi })\). Let \(a'\) denote the preimage of a in \({\mathfrak {g}}^H_1\cap {\mathfrak {g}}^Z_{0}\). For any rational \(t\in [0,1]\) denote \(H_t:=H+tZ,\) and define \(\mathfrak {l}_t\) and \(\mathfrak {r}_t\) as in (3.16). Define also \(\mathfrak {l}_t^\text {max}:=\mathfrak {l}_t+a'\) and \(\mathfrak {r}_t^\text {max}:=\mathfrak {r}_t+a'\). Then both \(\mathfrak {l}^\text {max}_t\) and \(\mathfrak {r}^\text {max}_t\) are maximal isotropic subspaces in \({\mathfrak {u}}_{H_t} := {\mathfrak {g}}^{H_t}_{\ge 1}\) with respect to the antisymmetric form \(\omega _\varphi : {\mathfrak {g}}\times {\mathfrak {g}}\rightarrow \mathbb {K}\) defined by \(\omega _\varphi (X, Y) = \varphi ([X,Y])\). Indeed, the symplectic space \( {\mathfrak {u}}_{H_t}/{\mathfrak {n}}_{H_t,\varphi }\) is naturally isomorphic to \({\mathfrak {w}}_t/({\mathfrak {w}}_t\cap {\mathfrak {g}}_\varphi )\), where \({\mathfrak {w}}_t:={\mathfrak {g}}^{H_t}_1\). Note that \(({\mathfrak {w}}_t)^Z_0={\mathfrak {g}}^H_1\cap {\mathfrak {g}}^Z_{0}\) for all t. Now, \({\mathfrak {w}}_t=({\mathfrak {w}}_t)^{Z}_0\oplus ({\mathfrak {w}}_t)^{Z}_{<0}\oplus ({\mathfrak {w}}_t)^Z_{>0}\), with \(({\mathfrak {w}}_t)^Z_{<0}\) and \(({\mathfrak {w}}_t)^Z_{>0}\) both isotropic and orthogonal to \(({\mathfrak {w}}_t)^Z_{0}\) with respect to \(\omega _\varphi \). Let \({\mathfrak {w}}_t^+\) and \({\mathfrak {w}}_t^\) denote the images of \(({\mathfrak {w}}_t)^Z_{<0}\) and \(({\mathfrak {w}}_t)^Z_{>0}\) in \({\mathfrak {w}}_t/({\mathfrak {w}}_t\cap {\mathfrak {g}}_\varphi )\).
Since \(\mathfrak {l}^\text {max}_t\) projects onto \({\mathfrak {w}}_t^{} \oplus a\) and \(\mathfrak {r}^\text {max}_t\) projects onto \({\mathfrak {w}}_t^{+} \oplus a\), both project to Lagrangian subspaces of \({\mathfrak {u}}_{H_t}/{\mathfrak {n}}_{H_t,\varphi }\cong {\mathfrak {w}}_t/({\mathfrak {w}}_t\cap {\mathfrak {g}}_\varphi ),\) and thus are maximal isotropic. Hence, by Lemma B.6, we have
Now let \(0=t_0< t_1< \cdots < t_n=1\) be all the critical numbers in the interval [0, 1]. Then by (3.18) for every i we have \(\mathfrak {r}_{t_i}\subseteq \mathfrak {l}_{t_{i+1}},\) thus \(\mathfrak {r}^\text {max}_{t_i}\subseteq \mathfrak {l}^\text {max}_{t_{i+1}},\) and thus \(d(H_{t_i},\varphi )\le d(H_{t_{i+1}},\varphi )\). Since \(H_{t_0}=H\) and \(H_{t_n}=S\) part (i) follows.
For part (ii), let \(Z'\) be as in Notation 4.4.3, and let \(S':=H+TZ'\) with T large enough as in Lemma 4.4.5. Then \({\mathfrak {g}}^{S'}_1 \subseteq {\mathfrak {g}}^H_1\) and thus
Furthermore, by Lemma 4.4.4, \((H,\varphi )\) dominates \((S',\varphi )\). Part (i) implies now that \(\dim {\mathfrak {n}}_{(H,\varphi )}\le \dim {\mathfrak {n}}_{(S',\varphi )}\). Finally, by Lemma 4.4.5 the pair \((S',\varphi )\) is Levidistinguished, and thus, by Lemma 2.3.10, we have \(\dim {\mathfrak {n}}_{S,\varphi }= \dim {\mathfrak {n}}_{S',\varphi }\). \(\square \)
Remark B.8
In a previous arXiv preprint version of this paper we claimed that if \((H,\varphi )\) dominates \((S,\varphi )\) then \(\dim {\mathfrak {n}}_{H,\varphi }\le \dim {\mathfrak {n}}_{S,\varphi }\), and that therefore \(\dim {\mathfrak {n}}_{H,\varphi }\) is minimal for a neutral pair \((H, \varphi )\). Unfortunately, these statements are wrong. Indeed, it is possible that two pairs \((H,\varphi )\) and \((S,\varphi )\) dominate each other, with \((H,\varphi )\) neutral, and \(\dim {\mathfrak {n}}_{H,\varphi } > \dim {\mathfrak {n}}_{S,\varphi }\).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Gourevitch, D., Gustafsson, H.P.A., Kleinschmidt, A. et al. A reduction principle for Fourier coefficients of automorphic forms. Math. Z. 300, 2679–2717 (2022). https://doi.org/10.1007/s0020902102784w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s0020902102784w
Keywords
 Automorphic function
 Automorphic representation
 Fourier expansion on covers of reductive groups
 Whittaker support
 Nilpotent orbit
 Wavefront set
Mathematics Subject Classification
 11F30
 11F70
 22E55
 20G45