The Fourier Transform on the Group $\mathrm {GL}_2({\mathbb R })$ and the Action of the Overalgebra $\mathfrak {gl}_4$

Abstract

We define a kind of ’operational calculus’ for the Fourier transform on the group $\mathrm {GL}_2({\mathbb R })$. Namely, $\mathrm {GL}_2({\mathbb R })$ can be regarded as an open dense chart in the Grassmannian of 2-dimensional subspaces in ${\mathbb R }^4$. Therefore the group $\mathrm {GL}_4({\mathbb R })$ acts in $L^2$ on $\mathrm {GL}_2({\mathbb R })$. We transfer the corresponding action of the Lie algebra $\mathfrak {gl}_4$ to the Plancherel decomposition of $\mathrm {GL}_2({\mathbb R })$, the algebra acts by differential-difference operators with shifts in an imaginary direction. We also write similar formulas for the action of $\mathfrak {gl}_4\oplus \mathfrak {gl}_4$ in the Plancherel decomposition of $\mathrm {GL}_2({\mathbb C })$.

The Statement of the Paper

The Group $\mathrm {GL}_n({\mathbb R })$

Let $\mathrm {GL}_n({\mathbb R })$ be the group of invertible real matrices of order n. We denote elements of $\mathrm {GL}_2({\mathbb R })$ by

$$\begin{aligned} X=\begin{pmatrix} x_{11}&{}x_{12}\\ x_{21}&{}x_{22} \end{pmatrix}. \end{aligned}$$

The Haar measure on $\mathrm {GL}_2({\mathbb R })$ is given by

$$\begin{aligned} (\det X)^{-2}\,dX= (\det X)^{-2} \,dx_{11} \,dx_{12}\,\,dx_{21}\,dx_{22}. \end{aligned}$$

Recall some basic facts on representations of the group $\mathrm {GL}_2({\mathbb R })$, for systematic exposition of the representation theory of $\mathrm {SL}_2({\mathbb R })$, see, e.g., [4, 6]. Denote by ${\mathbb R }^\times$ the multiplicative group of ${\mathbb R }$. Let $\mu \in {\mathbb C }$ and $\varepsilon \in {\mathbb Z}/2{\mathbb Z}$. We define the function $x^{\mu /\!\!/\varepsilon }$ on ${\mathbb R }^\times$ by

$$\begin{aligned} x^{\mu /\!\!/\varepsilon }:=|x|^\mu \mathop {\mathrm {sgn}}\nolimits (x)^\varepsilon , \end{aligned}$$

these functions are precisely all homomorphisms from ${\mathbb R }^\times$ to the multiplicative group of ${\mathbb C }$. Denote by $\Lambda$ the set of all collections

$$\begin{aligned} (\mu _1,\varepsilon _1;\mu _2,\varepsilon _2), \end{aligned}$$

i.e.,

$$\begin{aligned} \Lambda \simeq {\mathbb C }\times {\mathbb Z}_2\times {\mathbb C }\times {\mathbb Z}_2. \end{aligned}$$

For each element of $\Lambda$ we define a representation $T_{\mu ,\varepsilon }$ of $\mathrm {GL}_2({\mathbb R })$ in the space of functions on ${\mathbb R }$ by

$$\begin{aligned}&T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}\begin{pmatrix} x_{11}&{}x_{12}\\ x_{21}&{}x_{22} \end{pmatrix} \varphi (t)\nonumber \\&\quad =\; \varphi \Bigl (\frac{x_{12}+t x_{22}}{x_{11}+t x_{21}}\Bigr ) \cdot (x_{11}+t x_{21})^{-1+\mu _1-\mu _2/\!\!/\varepsilon _1-\varepsilon _2} \det \begin{pmatrix} x_{11}&{}x_{12}\\ x_{21}&{}x_{22} \end{pmatrix}^{1/2+\mu _2/\!\!/\varepsilon _2} . \end{aligned}$$

(1.1)

A space of functions can be specified in various ways, it is convenient to consider the space $C^\infty _{\mu _1-\mu _2,\varepsilon _1-\varepsilon _2}$ of $C^\infty$-functions on ${\mathbb R }$

such that¹

$$\begin{aligned} \varphi (-1/t) (-t)^{-1+\mu _1-\mu _2/\!\!/\varepsilon _1-\varepsilon _2} \end{aligned}$$

also is $C^\infty$-smooth.

Thus, for any fixed $X\in \mathrm {GL}_2({\mathbb R })$ we get an operator-valued function $(\mu _1,\varepsilon _1;\mu _2,\varepsilon _2) \mapsto T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}$ on $\Lambda$ holomorphic in the variables $\mu _1$, $\mu _2$.

Generators of the Lie algebra $\mathfrak g\mathfrak l_2({\mathbb R })$ act by formulas

$$\begin{aligned} L_{11}&=-t\frac{d}{dt}+(-1/2+\mu _1),\qquad&L_{12}&=\frac{d}{dt}, \end{aligned}$$

(1.2)

$$\begin{aligned} L_{21}&=-t^2\frac{d}{dt}+t(-1+\mu _1-\mu _2), \qquad&L_{22}&=t\frac{d}{dt}+(1/2+\mu _2). \end{aligned}$$

(1.3)

The expressions for the generators do not depend on $\varepsilon _1$, $\varepsilon _2$. However the space of the representation depends on $\varepsilon _1-\varepsilon _2$.

Consider integral operators

$$\begin{aligned} A_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}: C^\infty _{\mu _1-\mu _2,\varepsilon _1-\varepsilon _2} \rightarrow C^\infty _{\mu _2-\mu _1,\varepsilon _1-\varepsilon _2} \end{aligned}$$

defined by

$$\begin{aligned} A_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2} f(t):= \int _{{\mathbb R }} (t-s)^{-1-\mu _1+\mu _2/\!\!/\varepsilon _1-\varepsilon _2} f(s)\,ds. \end{aligned}$$

The integral is convergent if $\mathop {\mathrm {Re}}\nolimits (\mu _1-\mu _2)<0$ and determines a function holomorphic in $\mu _1$, $\mu _2$. As usual (see, e.g., [3], §I.3), the integral admits a meromorphic continuation to the whole plane $(\mu _1,\mu _2)\in {\mathbb C }^2$ with poles at $\mu _1-\mu _2=0$, 1, 2, .... The operators $A_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}$ are intertwining,

$$\begin{aligned} A_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2} = T_{\mu _2,\varepsilon _2;\mu _1,\varepsilon _1}A_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2} \end{aligned}$$

(1.4)

(parameters $(\mu _1,\varepsilon _1)$ and $(\mu _2,\varepsilon _2)$ of T are transposed). If $\mu _1-\mu _2\ne \pm 1$, $\pm 2$, ..., then representations $T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}$ are irreducible and operators $T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}$ are invertible.

Representations $T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}$ form a so-called (nonunitary) principal series of representations. Recall description of some unitary irreducible representations of $\mathrm {GL}_2({\mathbb R })$.

Unitary principal series. If $\mu _1$, $\mu _2\in i{\mathbb R }$, then the representation is unitary in $L^2({\mathbb R })$. Denote by $\Lambda _{\mathrm {principal}}\subset \Lambda$ the subset consisting of tuples $(is_1,\varepsilon _1;is_2,\varepsilon _2)$ with $s_1\geqslant s_2$.

Discrete series. Notice that the group $\mathrm {GL}_2({\mathbb R })$ acts on the Riemann sphere $\overline{C}={\mathbb C }\cup \infty$ by linear-fractional transformations

$$\begin{aligned} X:\,z\mapsto \frac{x_{12}+z x_{22}}{x_{11}+z x_{21}}, \end{aligned}$$

(1.5)

these transformations preserve the real projective line ${\mathbb R }\cup \infty$ and therefore preserves its complement ${\mathbb C }\setminus {\mathbb R }$. If $\det X>0$, then (1.5) leave upper and lower half-planes invariant, if $\det X<0$, then (1.5) permutes the half-planes.

Let $n=1$, 2, 3, .... Consider the space $H_n$ of holomorphic functions $\varphi$ on ${\mathbb C }\setminus {\mathbb R }$ satisfying

$$\begin{aligned} \int _{{\mathbb C }\setminus {\mathbb R }} |\varphi (z)|^2 |\mathop {\mathrm {Im}}\nolimits z|^{-n-1} \,d\mathop {\mathrm {Re}}\nolimits z\,d\mathop {\mathrm {Im}}\nolimits z<\infty . \end{aligned}$$

In fact, $\varphi$ is a pair of holomorphic functions $\varphi _+$ and $\varphi _-$ determined on half-planes $\mathop {\mathrm {Im}}\nolimits z>0$ and $\mathop {\mathrm {Im}}\nolimits z<0$. These functions have boundary values on ${\mathbb R }$ in distributional sense (we omit a precise discussion since it is not necessary for our purposes). The space $H_n$ is a Hilbert space with respect to the inner product

$$\begin{aligned} \langle \varphi _1, \varphi _2\rangle =\int _{{\mathbb C }\setminus {\mathbb R }} \varphi _1(z)\overline{\varphi _2(z)} \,|\mathop {\mathrm {Im}}\nolimits z|^{-n-1} \,d\mathop {\mathrm {Re}}\nolimits z\,d\mathop {\mathrm {Im}}\nolimits z. \end{aligned}$$

For $s\in {\mathbb R }$, $\delta \in {\mathbb Z}_2$ we define a unitary representation $D_{n,s}$ of $\mathrm {GL}_2({\mathbb R })$ in $H_n$ by

$$\begin{aligned} D_{n,s}\begin{pmatrix} x_{11}&{}x_{12}\\ x_{21}&{}x_{22} \end{pmatrix}\varphi (z)&= \varphi \Bigl (\frac{x_{12}+z x_{22}}{x_{11}+z x_{21}}\Bigr ) (x_{11}+z x_{21})^{-1-n} \\&\quad \times \det \begin{pmatrix} x_{11}&{}x_{12}\\ x_{21}&{}x_{22} \end{pmatrix}^{1/2+n/2+is/\!\!/\delta } . \end{aligned}$$

In fact, we have operators (1.1) for

$$\begin{aligned} \mu _1=-n/2-is,\, \varepsilon _1=-1+n+\delta ;\,\, \mu _2=n/2+is,\, \varepsilon _2=\delta \end{aligned}$$

restricted to the subspace generated by boundary values of functions $f_+$ and $f_-$.

We denote by $\Lambda _{\mathrm {discrete}}$ the set of all parameters of discrete series.

The Fourier Transform

Let F be contained in the space $C_0^\infty (\mathrm {GL}_2({\mathbb R }))$ of compactly supported function on $\mathrm {GL}_2({\mathbb R })$. We consider a function that sent each $\widetilde{\mu }=(\mu _1,\varepsilon _1;\mu _2,\varepsilon _2)$ to an operator in $C^\infty _{\mu _1-\mu _2,\varepsilon _1-\varepsilon _2}$ given by

$$\begin{aligned} T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}(F)= \int _{\mathrm {GL}_2({\mathbb R })} F(X)\, T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}(X)\,\frac{dX}{\det (X)^2} . \end{aligned}$$

By definition the Fourier transform is the map $F\mapsto T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2} (F)$, for each F we get a function on the space of parameters $(\mu _1,\varepsilon _1;\mu _2,\varepsilon _2)$.

Next, we define a subset $\Lambda _{\mathrm {tempered}}\subset \Lambda$ by

$$\begin{aligned}\Lambda _{\mathrm {tempered}}:=\Lambda _{\mathrm {principal}}\cup \Lambda _{\mathrm {discrete}}.\end{aligned}$$

Let us define Plancherel measure$d\mathcal P(\widetilde{\mu })$ on $\Lambda _{\mathrm {tempered}}$. On the piece $\Lambda _{\mathrm {principal}}$ it is given by

$$\begin{aligned} d\mathcal P(is_1,0;is_2,\varepsilon _2) =\frac{1}{16\pi ^3}(s_1-s_2) \tanh \pi (s_1-s_2)/2\,ds_1\,ds_2; \\ d\mathcal P(is_1,1;is_2,\varepsilon _2) = \frac{1}{16\pi ^3} (s_1-s_2) \coth \pi (s_1-s_2)/2\,ds_1\,ds_2. \end{aligned}$$

On n-th piece of $\Lambda _{\mathrm {discrete}}$ it is given by

$$\begin{aligned} d\mathcal P=\frac{n}{8\pi ^3} ds. \end{aligned}$$

Consider the space $\mathcal L^2$ of functions Q on $\Lambda _{\mathrm {tempered}}$ taking values in the space of Hilbert–Schmidt operators in the corresponding Hilbert spaces and satisfying the condition

$$\begin{aligned} \int _{\Lambda _{\mathrm {tempered}}} \mathop {\mathrm {tr}}\nolimits \Bigl ( Q(\widetilde{\mu })Q^*(\widetilde{\mu })\Bigr )\,d\mathcal P(\widetilde{\mu })<\infty . \end{aligned}$$

This is a Hilbert space with respect to the inner product

$$\begin{aligned} \langle Q_1,Q_2\rangle _{\mathcal L^2}:= \int _{\Lambda _{\mathrm {tempered}}} \mathop {\mathrm {tr}}\nolimits \Bigl (Q_1(\widetilde{\mu })Q_2^*(\widetilde{\mu })\Bigr )\,\,d\mathcal P(\widetilde{\mu }). \end{aligned}$$

According the Plancherel theorem, for any$F_1$, $F_2\in C^\infty _0(\mathrm {GL}_2({\mathbb R }))$we have

$$\begin{aligned} \langle F_1, F_2\rangle _{L^2(\mathrm {GL}_2({\mathbb R }))} =\langle T(F_1), T(F_2)\rangle _{\mathcal L^2(\Lambda _{\mathrm {tempered}})}. \end{aligned}$$

Moreover, the map$F\rightarrow T(F)$extends to a unitary operator$L^2(\mathrm {GL}_2({\mathbb R }))\rightarrow \mathcal L^2$.

Overgroup

Let $\mathrm {Mat}_2({\mathbb R })$ be the space of all real matrices of order 2. By $\mathrm {Gr}_{4,2}({\mathbb R })$ we denote the Grassmannian of 2-dimensional subspaces in ${\mathbb R }^2\oplus {\mathbb R }^2$. For any operator ${\mathbb R }^2\rightarrow {\mathbb R }^2$ its graph is an element $\mathrm {Gr}_{4,2}({\mathbb R })$. The set $\mathrm {Mat}_2({\mathbb R })$ of such operators is an open dense chart in $\mathrm {Gr}_{4,2}({\mathbb R })$.

The group $\mathrm {GL}_4({\mathbb R })$ acts in a natural way in ${\mathbb R }^4$ and therefore on the Grassmannian. In the chart $\mathrm {Mat}_2({\mathbb R })$ the action is given by the formula (see, e.g., [18], Theorem 2.3.2)

$$\begin{aligned} \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix}:\,X\mapsto (A+XC)^{-1}(B+XD), \end{aligned}$$

where $\begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix}$ is an element of $\mathrm {GL}_4({\mathbb R })$ written as a block matrix of size $2+2$. The Jacobian of this transformation is (see, e.g., [18], Theorem 2.3.2)

$$\begin{aligned} \det (A+XC)^{-4}\det \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix}^2. \end{aligned}$$

For $\sigma \in i{\mathbb R }$ we define a unitary representation of $\mathrm {GL}_4({\mathbb R })$ in $L^2(\mathrm {Mat}_2({\mathbb R }))$ by

$$\begin{aligned}&R_\sigma \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix} F(X)=F\bigl ( (A+XC)^{-1}(B+XD)\bigr )\\&\qquad \qquad \qquad \qquad \qquad \qquad \quad \qquad \qquad \,\times \,|\det (A+XC)|^{-2+2\sigma } \left| \det \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix} \right| ^{1-\sigma } \end{aligned}$$

(these representations are contained in degenerate principal series).

The group $\mathrm {GL}_2({\mathbb R })$ is an open dense subset in $\mathrm {Mat}_2({\mathbb R })$. Therefore, we can identify the spaces $L^2$ on $\mathrm {GL}_2({\mathbb R })$ and $\mathrm {Mat}_2({\mathbb R })$. For this we consider a unitary operator

$$\begin{aligned} J:\, L^2\bigl (\mathrm {Mat}_2({\mathbb R })\bigr )\rightarrow L^2\bigl (\mathrm {GL}_2({\mathbb R }) \bigr ) \end{aligned}$$

given by

$$\begin{aligned} J_\sigma F(X)=F(X)\cdot |\det X|^{1-\sigma } \end{aligned}$$

This determines a unitary representation $U_\sigma :=J_\sigma R_\sigma J_\sigma ^{-1}$ of $\mathrm {GL}_4({\mathbb R })$ in $L^2\bigl (\mathrm {GL}_2({\mathbb R }) \bigr )$, the explicit expression is

$$\begin{aligned} U_\sigma \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix} F(X)&=F\bigl ( (A+XC)^{-1}(B+XD)\bigr ) \nonumber \\&\quad \times \Biggl |\frac{\det (A+XC)\det (B+XD)}{\det X \det \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix}} \Biggr |^{-1+\sigma } . \end{aligned}$$

(1.6)

Consider a subgroup $\mathrm {GL}_2({\mathbb R })\times \mathrm {GL}_2({\mathbb R })\subset \mathrm {GL}_4({\mathbb R })$ consisting of matrices $\begin{pmatrix} A&{}0\\ 0&{}D \end{pmatrix}$. For this subgroup we get the usual left-right action in $L^2\bigl (\mathrm {GL}_2({\mathbb R })\bigr )$,

$$\begin{aligned} U_\sigma \begin{pmatrix} A&{}0\\ 0&{}D \end{pmatrix} F(X)=F(A^{-1}X D). \end{aligned}$$

Formula (1.6) extends this formula to the whole group $\mathrm {GL}_4({\mathbb R })$. The Lie algebra $\mathfrak g\mathfrak l_4$ acts in the space of functions on $\mathrm {Mat}_2({\mathbb R })$ by first order differential operators, which can be easily written; a list of formulas for all generators $e_{kl}$, where k, $l=1$, 2, 3, 4, is given below in Sect. 2.5. We restrict this action to the space of smooth compactly supported functions on $\mathrm {GL}_2({\mathbb R })$. Notice that the operators $i\cdot e_{kl}$ are symmetric on this domain, but some of them are not essentially self-adjoint.

Our purpose is to write explicitly the images $E_{kl}$ of operators $e_{kl}$ under the Fourier transform.

Formulas

Operators $T_{\mu _1,\varepsilon _1;\mu _2;\varepsilon _2}(F)$ have the form

$$\begin{aligned} T_{\mu _1,\varepsilon _1;\mu _2;\varepsilon _2}(F)\varphi (t)= \int _{-\infty }^{\infty } K(t,s|\mu _1,\varepsilon _1;\mu _2,\varepsilon _2)\, \varphi (t)\,ds. \end{aligned}$$

(1.7)

Recall that functions K are holomorphic in $\mu _1$, $\mu _2$.

We wish to write operators $E_{kl}$ on kernels K. The complete list is contained below in Sect. 2.5, here we present two basic expressions.

The algebra $\mathfrak {gl}_{4}({\mathbb R })$ can be decomposed as a linear space into a direct sum of four subalgebras $\mathfrak a$, $\mathfrak b$, $\mathfrak c$, $\mathfrak d$ consisting of matrices of the form

$$\begin{aligned} \begin{pmatrix} *&{}0\\ 0&{}0 \end{pmatrix}, \qquad \begin{pmatrix} 0&{}0\\ 0&{}* \end{pmatrix}, \qquad \begin{pmatrix} 0&{}*\\ 0&{}0 \end{pmatrix}, \qquad \begin{pmatrix} 0&{}0\\ 0&{}* \end{pmatrix} \end{aligned}$$

The subalgebras $\mathfrak a$ and $\mathfrak d$ are isomorphic to $\mathfrak {gl}_2({\mathbb R })$, subalgebras $\mathfrak b$ and $\mathfrak c$ are Abelian.

Formulas for the action of $\mathfrak a$ and $\mathfrak d$ immediately follow from the definition of the Fourier transform. To obtain formulas for the whole $\mathfrak {gl}_4$, it is sufficient to write expressions for one generator of $\mathfrak b$, say $E_{14}$, and one generator of $\mathfrak c$, say $E_{32}$. After this other generators can be obtained by evaluation of commutators.

We define shift operators $V_1^+$, $V_1^-$, $V_2^+$, $V_2^-$ by

$$\begin{aligned} V_1^\pm K(t,s|\mu _1,\varepsilon _1;\mu _2,\varepsilon _2)= K(t,s|\mu _1\pm 1,\varepsilon _1+1;\mu _2,\varepsilon _2); \end{aligned}$$

(1.8)

$$\begin{aligned} V_2^\pm K(t,s|\mu _1,\varepsilon _1;\mu _2,\varepsilon _2)= K(t,s|\mu _1,\varepsilon _1;\mu _2\pm 1,\varepsilon _2+1). \end{aligned}$$

(1.9)

Theorem 1.1

The operators $E_{14}$, $E_{32}$ are given by the formulas

$$\begin{aligned} E_{14}=\frac{-1/2 - \sigma + \mu _1}{\mu _1 - \mu _2}\, \frac{\partial }{\partial s}\, V_1^- +\frac{-1/2 - \sigma + \mu _2}{\mu _1 - \mu _2}\,\frac{\partial }{\partial t}\, V_2^-; \end{aligned}$$

$$\begin{aligned} E_{32}=\frac{1/2 + \mu _1 + \sigma }{\mu _1 - \mu _2}\, \frac{\partial }{\partial t}\, V_1^+ +\frac{1/2 + \mu _2 + \sigma }{\mu _1 - \mu _2} \, \frac{\partial }{\partial s}\, V_2^+. \end{aligned}$$

Remark

We emphasize that these formulas determining unbounded skew-symmetric operators in $\mathcal L$ include shifts transversal to the contour of integration in the Plancherel formula (the contour corresponds to pure imaginary $\mu _1$, $\mu _2$ and (1.8), (1.9) are a shift operators in real directions). $\square$

Remarks on a General Problem

In [17] the author formulated the following question: Assume that we know the explicit Plancherel formula for the restriction of a unitary representation$\rho$of a group G to a subgroup H. Is it possible to write the action of the Lie algebra of G in the direct integral of representations of H?

Now it seems that an answer to this question is affirmative.

The initial paper [17] contains a solution for a tensor product² of a highest and lowest weight representations of $\mathrm {SL}_2({\mathbb R })$. In this case the overalgebra acts by differential-difference operators in the space $L^2(S^1\times {\mathbb R }_+)$ having the form

$$\begin{aligned} Lf(\varphi ,s)=D_1 f(\varphi , s+i)+ D_2 f(\varphi ,s) +D_3 f(\varphi ,s-i), \end{aligned}$$

where $D_1$, $D_2$, $D_3$ are differential operators in the variable $\varphi$ of orders 0, 1, 2 respectively.

In [9,10,11,12,13] Molchanov solved several rank 1 problems of this type, expressions are similar, but there appear differential operators of order 4. In [20] there was obtained the action of the overalgebra in restrictions from $\mathrm {GL}_{n+1}({\mathbb C })$ to $\mathrm {GL}_n({\mathbb C })$, in this case differential operators have order n. In all the cases examined by now formulas include shift operators in the imaginary direction.

In the present paper, we write the action of the overalgebra in the restriction of a degenerate principal series of the group $\mathrm {GL}_4({\mathbb R })$ to $\mathrm {GL}_2({\mathbb R })$. Notice that canonical overgroups exist for all 10 series of real classical groups.³ Moreover overgroups exist for all 52 series of classical semisimple symmetric spaces G / H, see [8, 15], see also [18], Addendum D.6. So the problem makes sense for all classical symmetric spaces.

Sturm–Liouville problems for difference operators in $L^2({\mathbb R })$ in the imaginary direction

$$\begin{aligned} \lambda f(s)=a(s) f(s+i)+b(s) f(s)+ c(s) f(s-i) \end{aligned}$$

arise in a natural way in the theory of hypergeometric orthogonal polynomials, see, e.g., [1, 7], apparently a first example (the Meixner–Pollaczek system) was discovered by J. Meixner in 1930s. On such operators with continuous spectra see [5, 16, 19]. See also a multi-rank work of Cherednik [2] on Harish-Chandra spherical transforms.

The Fourier Transform on $\mathrm {GL}_2({\mathbb C })$

For a detailed exposition of representations of the Lorentz group, see [14]. For $\nu$, $\nu '\in {\mathbb C }$ satisfying $\nu -\nu '\in {\mathbb Z}$ we define the function $z^{\nu \Vert \nu '}$ on the multiplicative group of ${\mathbb C }$ by

$$\begin{aligned} z^{\nu \Vert \nu '}:=z^\nu z^{\nu '}:=|z|^{2\nu }\, \overline{z}^{\,\nu '-\nu }. \end{aligned}$$

Denote by $\Delta$ the set of all $(\mu _1,\mu _1';\mu _2,\mu _2')\in {\mathbb C }^4$ such that $\mu _1-\mu _1'\in {\mathbb Z}$, $\mu _2-\mu _2'\in {\mathbb Z}$. For $(\mu _1,\mu _1';\mu _2,\mu _2')\in \Delta$ we define a representation of $\mathrm {GL}_2({\mathbb C })$ in the space of functions on ${\mathbb C }$ by

$$\begin{aligned}&T_{\mu _1,\mu _1';\mu _2,\mu _2'} \begin{pmatrix} x_{11}&{}x_{12}\\ x_{21}&{}x_{22} \end{pmatrix} \varphi (t)\\&\quad =\varphi \Bigl (\frac{x_{12}+t x_{22}}{x_{11}+t x_{21}}\Bigr ) (x_{11}+t x_{21})^{-1+\mu _1-\mu _2\Vert -1+\mu _1'-\mu _2'} \det \begin{pmatrix} x_{11}&{}x_{12}\\ x_{21}&{}x_{22} \end{pmatrix}^{1/2+\mu _2\Vert 1/2+\mu _2'}. \end{aligned}$$

We consider the space of $C^\infty$-smooth functions $\varphi$ on ${\mathbb C }$ such that

$$\begin{aligned} \varphi (-t^{-1}) t^{-1+\mu _1-\mu _2\Vert -1+\mu '_1-\mu '_2} \end{aligned}$$

is $C^\infty$-smooth at 0. These representations form the (nonunitary) principal series.

It is convenient to complexify the Lie algebra of $\mathrm {GL}_2({\mathbb C })$,

$$\begin{aligned} \mathfrak {gl}_2({\mathbb C })_{\mathbb C }\simeq \mathfrak {gl}_2({\mathbb C })\oplus \mathfrak {gl}_2({\mathbb C }). \end{aligned}$$

Under this isomorphism, the operators of the Lie algebra act in our representation by

$$\begin{aligned} L_{11}&=-t\frac{d}{dt}+(-1/2+\mu _1),&L_{12}&=\frac{d}{dt}, \end{aligned}$$

(1.10)

$$\begin{aligned} L_{21}&=-t^2\frac{d}{dt}+t(-1+\mu _1-\mu _2),&L_{22}&=t\frac{d}{dt}+(1/2+\mu _2), \end{aligned}$$

(1.11)

$$\begin{aligned} \overline{L}_{11}&=-\overline{t}\frac{d}{ d\overline{t}}+(-1/2+\mu _1'),&\overline{L}_{12}&=\frac{d}{d\overline{t}}, \end{aligned}$$

(1.12)

$$\begin{aligned} \overline{L}_{21}&=-\overline{t}^2\frac{d}{d\overline{t}}+\overline{t}(-1+\mu _1'-\mu _2'),&\overline{L}_{22}&=\overline{t}\frac{d}{d\overline{t}}+(1/2+\mu _2'). \end{aligned}$$

(1.13)

Formally, we have duplicated expressions (1.2)–(1.3).

If

$$\begin{aligned} \mathop {\mathrm {Re}}\nolimits (\mu _1+\mu _1')=0,\qquad \mathop {\mathrm {Re}}\nolimits (\mu _2+\mu _2')=0, \end{aligned}$$

(1.14)

then the representation $T_{\mu _1,\mu _1';\mu _2,\mu _2'}$ is unitary in $L^2$. Denote by $\Delta _{\mathrm {tempered}}$ the set of such tuples $(\mu _1,\mu _1';\mu _2,\mu _2')$, it is a union of a countable family of parallel 2-dimensional real planes in ${\mathbb C }^4$, we equip it by a natural Lebesgue measure $d\lambda (\mu )$.

For any compactly supported smooth function F on $\mathrm {GL}_2({\mathbb C })$ we define its Fourier transform as an operator-valued function on $\Delta$ given by

$$\begin{aligned} T_{\mu _1,\mu _1';\mu _2,\mu _2'} (F) =\int _{\mathrm {GL}_2({\mathbb C })} F(X)\, |\det X|^{-4}\, \prod _{k,l=1,2} d\mathop {\mathrm {Re}}\nolimits x_{kl}\,d\mathop {\mathrm {Im}}\nolimits x_{kl}. \end{aligned}$$

The Plancherel formula is the following identity

$$\begin{aligned}&\langle F_1,F_2\rangle _{L^2\bigl (\mathrm {GL}_2({\mathbb C })\bigr )} =-C\cdot \int _{\Delta _{\mathrm {tempered}}} \mathop {\mathrm {tr}}\nolimits \bigl ( T_{\mu _1,\mu _1';\mu _2,\mu _2'} (F_1) T_{\mu _1,\mu _1';\mu _2,\mu _2'} (F_2)\bigr )^* \\&\times (\mu _1-\mu _2) (\mu _1'-\mu _2') d\lambda (\mu ), \end{aligned}$$

where C is an explicit constant.

Denote by $K(t,s|\mu _1,\mu _1';\mu _2,\mu _2')$ the kernel of the operator $T_{\mu _1,\mu _1';\mu _2,\mu _2'} (F)$,

$$\begin{aligned} T_{\mu _1,\mu _1';\mu _2,\mu _2'} (F) \varphi (t)= \int _{\mathbb C }K(t,s|\mu _1,\mu _1';\mu _2,\mu _2')\varphi (s)\,ds. \end{aligned}$$

Overgroup for $\mathrm {GL}_2({\mathbb C })$

Consider the complex Grassmannian $\mathrm {Gr}_{4,2}({\mathbb C })$ of 2-dimensional planes in ${\mathbb C }^4$, again the set $\mathrm {Mat}_2({\mathbb C })$ is an open dense set on $\mathrm {Gr}_{4,2}({\mathbb C })$. For $\sigma$, $\sigma '\in i {\mathbb R }$ consider a unitary representation $R_{\sigma ,\sigma '}$ of $\mathrm {GL}_4({\mathbb C })$ in $L^2\bigl (\mathrm {Mat}_2({\mathbb C })\bigr )$ given by

$$\begin{aligned}&R_{\sigma ,\sigma '} F(X) =\begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix} F(X)=F\bigl ( (A+XC)^{-1}(B+XD)\bigr ) \\&\qquad \qquad \quad \qquad \times \,|\det (A+XC)|^{-2+2\sigma \Vert -2+2\sigma '} \left| \det \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix} \right| ^{1-\sigma \Vert 1-\sigma '} . \end{aligned}$$

We define a unitary operator $J:L^2\bigl (\mathrm {Mat}_2({\mathbb C })\bigr ) \rightarrow L^2\bigl (\mathrm {Mat}_2({\mathbb C })\bigr )$ by

$$\begin{aligned} J_{\sigma ,\sigma '} F(X)=F(X)(\det X)^{-1+\sigma \Vert -1+\sigma '}. \end{aligned}$$

In this way we get a unitary representation $U_{\sigma ,\sigma '}=J_{\sigma ,\sigma '} R_{\sigma ,\sigma '} J_{\sigma ,\sigma '}^{-1}$ of $\mathrm {GL}_4({\mathbb C })$ in $L^2\bigl (\mathrm {Mat}_2({\mathbb C })\bigr )$:

$$\begin{aligned}&U_{\sigma \Vert \sigma '} \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix} F(X)=F\bigl ( (A+XC)^{-1}(B+XD)\bigr ) \\&\qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \times \Biggl |\frac{\det (A+XC)\det (B+XD)}{\det X \det \begin{pmatrix} A&{}B\\ C&{}D \end{pmatrix}} \Biggr |^{-1+\sigma \Vert -1+\sigma '} . \end{aligned}$$

Formulas for $\mathrm {GL}_2({\mathbb C })$

We wish to write the action of the Lie algebra

$$\begin{aligned} \mathfrak {gl}_4({\mathbb C })_{\mathbb C }\simeq \mathfrak {gl}_4({\mathbb C })\oplus \mathfrak {gl}_4({\mathbb C }) \end{aligned}$$

in the Plancherel decomposition of $\mathrm {GL}_2({\mathbb C })$. Denote the standard generators of $\mathfrak {gl}_4({\mathbb C })\oplus 0$ and $0\oplus \mathfrak {gl}_4({\mathbb C })$ by $E_{kl}$ and $\overline{E}_{kl}$ respectively. Define the following shift operators

$$\begin{aligned} V_1 K(t,s;\mu _1,\mu _1',\mu _2,\mu _2')= K(t,s;\mu _1+1,\mu _1',\mu _2,\mu _2'); \\ V_1' K(t,s;\mu _1,\mu _1',\mu _2,\mu _2')= K(t,s;\mu _1,\mu _1'+1,\mu _2,\mu _2'), \end{aligned}$$

and similar operators $V_2$ and $V_2'$ shifting $\mu _2$ and $\mu _2'$.

Theorem 1.2

The operators $E_{14}$, $\overline{E}_{14}$, $E_{23}$, $\overline{E}_{23}$ are given by the formulas

$$\begin{aligned} E_{14}&=\frac{-1/2 - \sigma + \mu _1}{\mu _1 - \mu _2}\, \frac{\partial }{\partial s} \, V_1^{-1} -\frac{-1/2 - \sigma + \mu _2}{\mu _1 - \mu _2}\,\frac{\partial }{\partial t}\, V_2^{-1} ; \\ \overline{E}_{14}&=\frac{-1/2 - \sigma ' + \mu '_1}{\mu '_1 - \mu '_2} \, \frac{\partial }{\partial \overline{s}} \,(V_1')^{-1} -\frac{-1/2 - \sigma ' + \mu '_2}{\mu '_1 - \mu '_2}\, \frac{\partial }{\partial \overline{t}}\, (V_2')^{-1} ; \\ E_{32}&=\frac{1/2 + \mu _1 + \sigma }{\mu _1 - \mu _2} \, \frac{\partial }{\partial t}\, V_1 +\frac{1/2 + \mu _2 + \sigma }{\mu _1 - \mu _2} \,\frac{\partial }{\partial s} \, V_2 ;\\ \overline{E}_{32}&=\frac{1/2 + \mu _1' + \sigma '}{\mu '_1 - \mu '_2}\, \frac{\partial }{\partial \overline{t}}\, V_1' +\frac{1/2 + \mu '_2 + \sigma '}{\mu '_1 - \mu '_2} \, \frac{\partial }{\partial \overline{s}}\, V_2' . \end{aligned}$$

Calculations

The Expression for Kernel

Lemma 2.1

The kernel $K(\cdot )$ of an integral operator $T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}(F)$ is given by the formula

$$\begin{aligned}&K(t,s|\mu _1,\varepsilon _1;\mu _2,\varepsilon _2)\nonumber \\&\quad = \iiint \limits _{{\mathbb R }^3} F\bigl ( u - t v, s u - s t v - t w, v, s v + w \bigr )\,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw.\nonumber \\ \end{aligned}$$

(2.1)

For $F\in C_0^\infty \bigl (\mathrm {GL}_2({\mathbb R })\bigr )$ the integration is actually taken over a bounded domain.

The integral converges if $\mathop {\mathrm {Re}}\nolimits \mu _1>1/2$, $\mathop {\mathrm {Re}}\nolimits \mu _2>1/2$. For fixed $\varepsilon _1$, $\varepsilon _2$ it has a meromorphic continuation to the whole complex plane $(\mu _1,\mu _2)$ with poles on the hyperplanes $\mu _1=-1/2-k$, $\mu _2=-1/2-k$, where $k=0$, 1, 2, ...(see, e.g., [3], §I.3).

Proof

By the definition

$$\begin{aligned}&\int \limits _{{\mathbb R }} T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}(F)\varphi (t)\,\,\psi (t)\,dt \\&\qquad = \int \limits _{{\mathbb R }}\int \limits _{\mathrm {Mat}_2({\mathbb R })} F(x_{11},x_{12},x_{21},x_{22})\varphi \Bigl (\frac{x_{12}+t x_{22}}{x_{11}+t x_{21}}\Bigr ) (x_{11}+t x_{21})^{-1+\mu _1-\mu _2/\!\!/\varepsilon _1-\varepsilon _2} \\&\qquad \quad \times (x_{11}x_{22}-x_{12}x_{21})^{1/2+\mu _2/\!\!/\varepsilon _2} \,\frac{dx_{11}\,dx_{12}\,dx_{21}\,dx_{22}}{(x_{11}x_{22}-x_{12}x_{21})^2}\,dt. \end{aligned}$$

In the interior integral, we pass from the variables $x_{11}$, $x_{12}$, $x_{21}$, $x_{22}$ to new variables u, v, w, s defined by

$$\begin{aligned} \begin{pmatrix} x_{11}&{}x_{12}\\ x_{21}&{}x_{22} \end{pmatrix} = \begin{pmatrix} 1&{}-t\\ 0&{}1 \end{pmatrix} \begin{pmatrix} u&{}0\\ v&{}w \end{pmatrix} \begin{pmatrix} 1&{}s\\ 0&{}1 \end{pmatrix} \end{aligned}$$

(2.2)

or

$$\begin{aligned} x_{11}=u - t v,\qquad x_{12}= s u - s t v - t w,\qquad x_{21}= v,\qquad x_{22}= s v + w . \end{aligned}$$

The Jacobi matrix of this transformation is triangular, and the Jacobian is |u|. The inverse transformation is

$$\begin{aligned} u=x_{11}+t x_{21},\quad v=x_{21}, \quad w=\frac{x_{11}x_{22}-x_{12}x_{21}}{x_{11}+t x_{21}}, \qquad s= \frac{x_{12}+t x_{22}}{x_{11}+t x_{21}} . \end{aligned}$$

We also have $x_{11}x_{22}-x_{12}x_{21}=uw$.

After the change of variables we come to

$$\begin{aligned} \iint _{{\mathbb R }^2} K(t,s|\mu _1,\varepsilon _1;\mu _2,\varepsilon _2)\, \varphi (s)\,\psi (t)\,ds\,dt, \end{aligned}$$

where $K(\cdot )$ is given by (2.1).

A function F has a compact support in ${\mathbb R }^4\setminus \{x_{11}x_{22}-x_{12}x_{21}=0\}$. So, actually, $x_{21}=v$, $x_{11}=u-tv$, $x_{22}=w+sv$ are contained in a bounded domain. This implies the second claim of the lemma. $\square$

Preliminary Remarks

Below F denotes

$$\begin{aligned} F:=F(u - t v, s u - s t v - t w, v, s v + w). \end{aligned}$$

Also, $\partial _{11}F$, $\partial _{12}F$, etc. denote

$$\begin{aligned} \partial _{11}F:=\frac{\partial }{\partial x_{11}} F(x_{11}, x_{12}, x_{21}, x_{22})\Biggr |_{\begin{matrix} x_{11}= u - t v&{} x_{12}=s u - s t v - t w ,\\ x_{21}=v,&{} x_{22}=s v + w \end{matrix} } \end{aligned}$$

etc. Partial derivatives of F are

$$\begin{aligned} \frac{\partial }{\partial u} F&=\partial _{11} F+s\, \partial _{12} F; \end{aligned}$$

(2.3)

$$\begin{aligned} \frac{\partial }{\partial v} F&=-t\, \partial _{11} F-st\, \partial _{12} F+ \partial _{21} F+s\, \partial _{22} F; \end{aligned}$$

(2.4)

$$\begin{aligned} \frac{\partial }{\partial w} F&=-t\,\partial _{12}F+ \partial _{22} F, \end{aligned}$$

(2.5)

and

$$\begin{aligned} \frac{\partial }{\partial s} F&= (u-vt)\, \partial _{12}F+v\, \partial _{22}F; \end{aligned}$$

(2.6)

$$\begin{aligned} \frac{\partial }{\partial t} F&=-v\, \partial _{11}F-(w+sv)\,\partial _{12}F. \end{aligned}$$

(2.7)

Also, notice that

$$\begin{aligned} \left( y^{\nu /\!\!/\delta }\right) '=\nu y^{\nu -1/\!\!/\delta +1} . \end{aligned}$$

A Verification of the Formula for $E_{14}$

It is easy to verify that the operator $e_{14}$ in $C^\infty _0\bigl (\mathrm {GL}_2({\mathbb R })\bigr )$ is given by

$$\begin{aligned} e_{14}= \frac{\partial }{\partial x_{12}} - (-1 + \sigma )\frac{x_{21}}{x_{11}x_{22}-x_{12}x_{21}} . \end{aligned}$$

Therefore,

$$\begin{aligned}&T_{\mu _1,\varepsilon _1;\mu _2,\varepsilon _2}(e_{14}F)\nonumber \\&\quad = \iiint \partial _{12} F \, \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw\nonumber \\&\quad \quad -\; (-1 + \sigma ) \iiint F \frac{v}{uw} \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw, \end{aligned}$$

(2.8)

(the integration is taken over ${\mathbb R }^3$ on default).

We must verify that (2.8) coincides with

$$\begin{aligned} E_{14} K= \Bigg (\frac{-1/2 - \sigma + \mu _1}{\mu _1 - \mu _2}\, \frac{\partial }{\partial s} \, V_1^- +\frac{-1/2 - \sigma + \mu _2}{\mu _1 - \mu _2}\,\frac{\partial }{\partial t}\, V_2^-\Bigg ) K \end{aligned}$$

(2.9)

Below we establish two formulas

$$\begin{aligned}&\iiint \partial _{12} F\, \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw \,\, -\,\, \frac{\partial }{\partial s} V_1^- K \nonumber \\&\quad = \; (-3/2+\mu _2) \iiint F \,\, \frac{v}{uw} \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw, \end{aligned}$$

(2.10)

$$\begin{aligned}&\iiint \partial _{12} F\,\,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw \,\, +\,\, \frac{\partial }{\partial t} V_2^- K \nonumber \\&\quad = \; (-3/2+\mu _1) \iiint F\,\, \frac{v}{uw} \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw. \end{aligned}$$

(2.11)

Considering the sum of (2.10) and (2.11) with coefficients $\frac{-1/2 - \sigma + \mu _1}{\mu _1 - \mu _2}$ and $\frac{-1/2 - \sigma + \mu _2}{\mu _1 - \mu _2}$ we get coincidence of (2.8) and (2.9); for this, we use he identities

$$\begin{aligned}&\frac{-1/2 - \sigma + \mu _1}{\mu _1 - \mu _2}-\frac{-1/2 - \sigma + \mu _2}{\mu _1 - \mu _2}=1; \\&(-3/2+\mu _2)\cdot \frac{-1/2 - \sigma + \mu _1}{\mu _1 - \mu _2}- (-3/2+\mu _1)\cdot \frac{-1/2 - \sigma + \mu _2}{\mu _1 - \mu _2}=1-\sigma . \end{aligned}$$

Now let us check (2.10). The following identity can be verified by a straightforward calculation (with (2.6) and (2.5)):

$$\begin{aligned} \partial _{12}F-\frac{1}{u} \frac{\partial }{\partial s} F =-\frac{v}{u} \frac{\partial }{\partial w} F. \end{aligned}$$

Therefore the left-hand side of (2.10) equals to

$$\begin{aligned} \iiint \Bigl [ -\frac{v}{u} \frac{\partial }{\partial w} F \Bigr ]\cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw. \end{aligned}$$

We integrate this expression by parts in the variable w and come to (2.10).

To check (2.11), we verify the identity

$$\begin{aligned} \partial _{12} F +\frac{1}{v} \frac{\partial }{\partial t} F =\frac{v}{w} \frac{\partial }{\partial u} F \end{aligned}$$

and after this integrate by parts as above.

A Verification of the Formula for $E_{32}$

We have

$$\begin{aligned} e_{32} = -\Bigg (x_{11} x_{21} \frac{\partial }{\partial x_{11}} + x_{11} x_{22} \frac{\partial }{\partial x_{12}} + x_{21}^2 \frac{\partial }{\partial x_{21}} + x_{21} x_{22} \frac{\partial }{\partial x_{22}}\Bigg ) + (-1 + \sigma )x_{21} . \end{aligned}$$

Therefore,

$$\begin{aligned} T(e_{32}K)= \iiint \bigl (G+ (-1+\sigma )v F\bigr ) \cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw,\nonumber \\ \end{aligned}$$

(2.12)

where

$$\begin{aligned} G=(u-vt) v\,\partial _{11} F+ (u-vt)(w+vs) \,\partial _{12} F +v^2\, \partial _{21} F +v (w+vs) \,\partial _{22} F. \end{aligned}$$

On the other hand,

$$\begin{aligned} E_{32}K&=\frac{1/2 + \mu _1 + \sigma }{\mu _1 - \mu _2} \iiint u\frac{\partial }{\partial t}F \cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw \nonumber \\&\quad \quad +\; \frac{1/2 + \mu _2 + \sigma }{\mu _1 - \mu _2} \iiint w\frac{\partial }{\partial s} F \cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw. \nonumber \\ \end{aligned}$$

(2.13)

We must verify that (2.12) and (2.13) are equal. As in the previous subsection, this statement is reduced to a pair of identities

$$\begin{aligned} \iiint \bigl (G+ (-1+\sigma )v F-u\frac{\partial }{\partial t} F\bigr ) \cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw \nonumber \\ = (1/2+\mu _2+\sigma ) \iiint v F \cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw; \nonumber \\ \end{aligned}$$

(2.14)

$$\begin{aligned} \iiint \bigl (G+ (-1+\sigma )v F+w\frac{\partial }{\partial s} F\bigr ) \cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw \nonumber \\ = (1/2+\mu _1+\sigma ) \iiint v F \cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw. \nonumber \\ \end{aligned}$$

(2.15)

Let us verify (2.14). It can be easily checked (with (2.7), (2.4), (2.5)) that

$$\begin{aligned} G -u\frac{\partial }{\partial t} F=-v^2 \frac{\partial }{\partial v}F-vw \frac{\partial }{\partial w} F. \end{aligned}$$

We substitute this to the left-hand side of (2.14) and come to

$$\begin{aligned} \iiint \bigl (-v^2 \frac{\partial }{\partial v}F-vw \frac{\partial }{\partial w} F + (-1+\sigma )v F\bigr )\nonumber \\ \times \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw. \end{aligned}$$

(2.16)

Integrating by parts, we get

$$\begin{aligned}&\iiint F \cdot \frac{\partial }{\partial v} \Bigl (v^2\Bigr ) \cdot u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw \\&\quad + \iiint vF \cdot u^{-3/2+\mu _1/\!\!/\varepsilon _1} \frac{\partial }{\partial v} \Bigl (w^{-1/2+\mu _2/\!\!/\varepsilon _2+1}\Bigr )\,du\,dv\,dw\\&\quad + \; (-1+\sigma ) \iiint v F \cdot \,u^{-3/2+\mu _1/\!\!/\varepsilon _1} w^{-3/2+\mu _2/\!\!/\varepsilon _2}\,du\,dv\,dw. \end{aligned}$$

After a summation we come to the right-hand side of (2.14).

A proof of (2.15) is similar, we use the identity

$$\begin{aligned} G+v\frac{\partial }{\partial s}F=-v^2\frac{\partial }{\partial v}F- -uv \frac{\partial }{\partial u}F \end{aligned}$$

and repeat the same steps.

Table of Formulas

First, we present formulas for the action of the Lie algebra $\mathfrak g\mathfrak l_4$ corresponding to $U_\sigma$, see (1.6). Denote generators of $\mathfrak g\mathfrak l_4$ by $e_{kl}$, where $1\leqslant k,l\leqslant 4$. Denote by $\partial _{pq}$ the partial derivatives $\frac{\partial }{\partial x_{pq}}$, where $p,q=1,2$. The generators $e_{kl}$ naturally split into 4 groups corresponding to blocks A, B, C, D in (1.6).

• (a) Generators corresponding to the block A form a Lie algebra $\mathfrak g\mathfrak l_2$:

  $$\begin{aligned} e_{11}= -x_{11} \partial _{11} - x_{12}\partial _{12},\qquad e_{12}= -x_{21} \partial _{11} - x_{22}\partial _{12},\\ e_{21}= -x_{11} \partial _{21} - x_{12}\partial _{22},\qquad e_{22}= -x_{21} \partial _{21} - x_{22} \partial _{22}. \end{aligned}$$

• (b) Generators corresponding to the block D also form a Lie algebra $\mathfrak g\mathfrak l_2$:

  $$\begin{aligned} e_{33}= x_{11} \partial _{11} + x_{21} \partial _{21},\qquad e_{34}= x_{11} \partial _{12} + x_{21} \partial _{22},\\ e_{43}= x_{12} \partial _{11} + x_{22} \partial _{21},\qquad e_{44}= x_{12} \partial _{12} + x_{22} \partial _{22}. \end{aligned}$$

• (c) Elements corresponding to the block B form a 4-dimensional Abelian Lie algebra:

  $$\begin{aligned} e_{13}= \partial _{11} + (-1 + \sigma )\frac{x_{22}}{\det X},\qquad e_{14}= \partial _{12} - (-1 + \sigma )\frac{x_{21}}{\det X},\\ e_{23}= \partial _{21} - (-1 + \sigma )\frac{x_{12}}{\det X},\qquad e_{24}= \partial _{22} + (-1 + \sigma )\frac{x_{11}}{\det X}. \end{aligned}$$

• (d) Elements corresponding to the block C also form a 4-dimensional Abelian Lie algebra:

  $$\begin{aligned} e_{31} = -(x_{11}^2 \partial _{11} + x_{11} x_{12} \partial _{12} + x_{11} x_{21} \partial _{21} + x_{12} x_{21} \partial _{22}) + (-1 + \sigma )x_{11}, \\ e_{32} = -(x_{11} x_{21}\partial _{11} + x_{11} x_{22} \partial _{12} + x_{21}^2 \partial _{21} + x_{21} x_{22} \partial _{22}) + (-1 + \sigma )x_{21}, \\ e_{41} = -(x_{11}x_{12} \partial _{11} + x_{12}^2 \partial _{12} + x_{11} x_{22} \partial _{21} + x_{12} x_{22} \partial _{22}) + (-1 + \sigma )x_{12}, \\ e_{42} = -(x_{12}x_{21} \partial _{11} + x_{12} x_{22} \partial _{12} + x_{21} x_{22} \partial _{21} + x_{22}^2 \partial _{22}) + (-1 + \sigma )x_{22}. \end{aligned}$$

Denote by $E_{kl}$ the corresponding operators $E_{kl}$ on kernels K. Formulas for operators of groups (a), (b) immediately follow from the definition of the Fourier transform,

$$\begin{aligned} E_{11}&= -t \frac{\partial }{\partial t}-(1/2-\mu _1), \qquad&E_{12}&=\frac{\partial }{\partial t},\\ E_{21}&=-t^2 \frac{\partial }{\partial t}+ (-1+\mu _1-\mu _2)t, \qquad&E_{22}&= t \frac{\partial }{\partial t}+(1/2 + \mu _2), \end{aligned}$$

and

$$\begin{aligned} E_{33}&=-s\frac{\partial }{\partial s}- (1/2+\mu _1), \qquad&E_{34}&=\frac{\partial }{\partial s},\\ E_{43}&=-s^2\frac{\partial }{\partial s}+(-1-\mu _1+\mu _2)s, \qquad&E_{44}&= s\frac{\partial }{\partial s}+(1/2-\mu _2). \end{aligned}$$

Next,

$$\begin{aligned}&E_{13} := \frac{1/2 - \mu _2 + \sigma }{\mu _1 - \mu _2} \,s \frac{\partial }{\partial t}\, V_2^- + \frac{1/2 - \mu _1 + \sigma }{\mu _1 - \mu _2} \Bigl (\mu _1 - \mu _2+ s \frac{\partial }{\partial s}\Bigr )\, V_1^- , \\&E_{14}=-\frac{1/2+ \sigma - \mu _1}{\mu _1 - \mu _2}\, \frac{\partial }{\partial s}\, V_1^- -\frac{1/2 + \sigma - \mu _2}{\mu _1 - \mu _2}\,\frac{\partial }{\partial t}\, V_2^- , \\&E_{23}:= \frac{1/2 - \mu _2 + \sigma }{\mu _1 - \mu _2}\,s\,\Bigl (-(\mu _1-\mu _2)+t \frac{\partial }{\partial t}\Bigr )\, V_2^- \\&\qquad \qquad +\frac{1/2 - \mu _1 + \sigma }{\mu _1 - \mu _2}\,t\,\Bigl (\mu _1-\mu _2+ s \frac{\partial }{\partial s}\Bigr )\, V_1^- , \\&E_{24}= -\frac{1/2+ \sigma - \mu _1 }{\mu _1 - \mu _2} t\, \frac{\partial }{\partial s}\, V_1^- + \frac{1/2 + \mu _2 - \sigma }{\mu _1 - \mu _2}\, \Bigl (\mu _1-\mu _2+ t \frac{\partial }{\partial s}\Bigr )\, V_2^-. \end{aligned}$$

and

$$\begin{aligned} E_{31}&:= \frac{1/2 + \mu _1 + \sigma }{\mu _1 - \mu _2}\Bigl (\mu _1-\mu _2-t \frac{\partial }{\partial t}\Bigr ) \, V_1^+ - \frac{1/2 + \mu _2 + \sigma }{\mu _1 - \mu _2}\, t \frac{\partial }{\partial s} \, V_2^+ ,\\ E_{32}&=\frac{1/2 + \mu _1 + \sigma }{\mu _1 - \mu _2} \frac{\partial }{\partial t}\, V_1^+ +\frac{1/2 + \mu _2 + \sigma }{\mu _1 - \mu _2} \frac{\partial }{\partial s}\, V_2^+ ,\\ E_{41}&= \frac{1/2 + \mu _1 + \sigma }{\mu _1 - \mu _2}\, s \Bigl (\mu _1-\mu _2- t \frac{\partial }{\partial t}\Bigr ) V_1^+\\&\quad - \frac{1/2 + \mu _2 + \sigma }{\mu _1 - \mu _2}\, t\, \Bigl (\mu _1-\mu _2+ s \frac{\partial }{\partial s}\Bigr ) V_2^+ , \\ E_{42}&= \frac{1/2 + \mu _1 + \sigma }{\mu _1 - \mu _2} s \frac{\partial }{\partial t} V_1^+ +\frac{1/2 + \mu _2 + \sigma }{\mu _1 - \mu _2} \Bigr (\mu _1-\mu _2+s\frac{\partial }{\partial s}\Bigr ) V_2^+ . \end{aligned}$$

The Case of $\mathrm {GL}_2({\mathbb C })$

Notice that formulas in Sects. 1.1–1.4 for $\mathrm {SL}_2({\mathbb R })$ and in Sects. 1.6–1.8 are very similar, except the Plancherel formulas.

The analog of formula (2.1) is

$$\begin{aligned}&K(t,s|\mu _1,\mu _1';\mu _2,\mu _2')\nonumber \\&\quad = \iiint \limits _{{\mathbb C }^3} F\bigl ( u - t v, s u - s t v - t w, v, s v + w \bigr )\,u^{-3/2+\mu _1\Vert -3/2+\mu _1'} w^{-3/2+\mu _2\Vert -3/2+\mu _2'} \nonumber \\&\qquad \times \,d \mathop {\mathrm {Re}}\nolimits u\,d \mathop {\mathrm {Im}}\nolimits u \,d \mathop {\mathrm {Re}}\nolimits v\,d \mathop {\mathrm {Im}}\nolimits v \,d \mathop {\mathrm {Re}}\nolimits w\,d \mathop {\mathrm {Im}}\nolimits w . \end{aligned}$$

(2.17)

Its derivation is based on the same change of variables (2.2), its real Jacobian is $u\overline{u}$. A further calculation one-to-one follows the calculation for $\mathrm {GL}_2({\mathbb R })$.