1 Background

In this article we derive an explicit formula for the average

$$\begin{aligned} I(t) := \int _K Z(t,k)\, dk \end{aligned}$$
(1.1)

where K is one of the classical compact Lie groups \({\mathrm {O}}_N\,\), \({\mathrm {SO}}_N\,\), or \({\mathrm {USp}}_N\) equipped with Haar measure dk of unit mass \(\int _K dk = 1\) and

$$\begin{aligned} Z(t,k) := \prod _{j=1}^n \frac{\mathrm {Det}( \mathrm {e}^{\frac{\mathrm {i}}{2}\psi _j} \mathrm {Id}_N - \mathrm {e}^{-\frac{\mathrm {i}}{2}\psi _j} \,k)}{\mathrm {Det}(\mathrm {e}^{\frac{1}{2}\phi _j}\mathrm {Id}_N - \mathrm {e}^{- \frac{1}{2}\phi _j} \, k)} \end{aligned}$$
(1.2)

depends on a set of complex parameters \(t := (\mathrm {e}^{ \mathrm {i}\psi _1} , \ldots , \mathrm {e}^{\mathrm {i}\psi _n} , \mathrm {e}^{\phi _1} , \ldots , \mathrm {e}^{\phi _n})\), which satisfy \(\mathfrak {Re}\, \phi _j > 0\) for all \(j = 1, \ldots , n\,\). The case of \(K = \mathrm {U}_N\) is handled in [7]. Note that

$$\begin{aligned} Z(t,k) = \mathrm {e}^{\lambda _N} \prod _{j=1}^n \frac{\mathrm {Det} (\mathrm {Id}_N-\mathrm {e}^{-\mathrm {i}\psi _j}\,k)}{\mathrm {Det} (\mathrm {Id}_N - \mathrm {e}^{-\phi _j}\, k)} \end{aligned}$$

with \(\lambda _N = \frac{N}{2} \sum _{j=1}^n (\mathrm {i} \psi _j - \phi _j)\). This means that Z(tk) is a product of ratios of characteristic polynomials, which explains the title of the article.

The Haar average I(t) can be regarded as the (numerical part of the) character of an irreducible representation of a Lie supergroup \((\mathfrak {g}\, ,G)\) restricted to a suitable subset of a maximal torus of G. The Lie superalgebra \(\mathfrak {g}\) is the Howe dual partner of the compact group K in an orthosymplectic Lie superalgebra \(\mathfrak {osp} \,\). It is naturally represented on a certain infinite-dimensional spinor–oscillator module \(\mathfrak {a} (V)\) —more concretely, the complex of holomorphic differential forms on the vector space \(\mathbb {C}^n \otimes \mathbb {C}^N\)—and the irreducible representation is that on the subspace \(\mathfrak {a}(V)^K\) of K-equivariant forms.

To even define the character, we must exponentiate the representation of the Lie algebra part of \(\mathfrak {osp}\) on \(\mathfrak {a}(V)\). This requires going to a completion \(\fancyscript {A}_V\) of \(\mathfrak {a}(V)\), and can only be done partially. Nevertheless, the represented semigroup contains enough structure to derive Laplace-Casimir differential equations for its character.

Our explicit formula for I(t) looks exactly like a classical Weyl formula and is derived in terms of the roots of the Lie superalgebra \(\mathfrak {g}\) and the Weyl group W. Let us state this formula for \(K = \mathrm {O}_N\,\), \(\mathrm {USp}_N\) without going into the details of the \(\lambda \)-positive even and odd roots \(\Delta ^+_{\lambda ,0}\) and \(\Delta ^+_{\lambda ,1}\) and the Weyl group W (see Sect. 5.3 for precise formulas). If \(W_\lambda \) is the isotropy subgroup of W fixing the highest weight \(\lambda = \lambda _N\,\), then

$$\begin{aligned} I(t)=\sum _{[w] \in W\!\! / W_\lambda } \mathrm {e}^{ w(\lambda _N)} \frac{\prod _{\beta \in \Delta _{\lambda ,1}^+}(1-\mathrm {e}^{ -w(\beta )})}{\prod _{\alpha \in \Delta _{\lambda ,0}^+} (1 - \mathrm {e}^{-w(\alpha )})} (\ln t). \end{aligned}$$
(1.3)

To prove this formula we establish certain properties of I(t) which uniquely characterize it and are satisfied by the right-hand side. These are a weight expansion of I(t) (see Corollary 4.1), restrictions on the set of weights (see Corollary 2.3), and the fact that I(t) is annihilated by certain invariant differential operators (see Corollary 5.1).

As was stated above, the case \(K = \mathrm {U}_N\) is treated in [7]. Here, we restrict to the compact groups \(K = \mathrm {O}_N \,\), \(K = \mathrm {USp}_N\,\), and \(K = \mathrm {SO}_N\,\). The cases \(K = \mathrm {O}_N\) and \(K = \mathrm {USp}_N\) can be treated simultaneously. Having established formula (1.3) for \(K = \mathrm {O}_N\,\), the following argument gives a similar result for \(K = \mathrm {SO}_N\,\). Let \(dk_\mathrm {O}\) and \(dk_\mathrm {SO}\) be the unit mass Haar measures for \(\mathrm {O}_N\) and \(\mathrm {SO}_N\,\), respectively. The \(\mathrm {O}_N\)-measure \((1 + \mathrm {Det}\, k)\, dk_\mathrm {O}\) has unit mass on \(\mathrm {SO}_N\) and zero mass on \(\mathrm {O}_N^- = \mathrm {O}_N {\setminus} \mathrm {SO}_N\,\). It is \(\mathrm {SO}_N\)-invariant. Now,

$$\begin{aligned} I_{\mathrm {SO}_N}(t)&= \int _{\mathrm {SO}_N} Z(t,k)\,dk_\mathrm {SO} = \int _{\mathrm {O}_N} Z(t,k) (1+\mathrm {Det}\, k)\, dk_\mathrm {O} \\&= \int _{\mathrm {O}_N} Z(t,k)\, dk_\mathrm {O} + \int _{\mathrm {O}_N} Z(t,k) \mathrm {Det}(k) \, dk_\mathrm {O} = I_{\mathrm {O}_N}(t) + (-1)^N I_{\mathrm {O}_N}(t') \end{aligned}$$

with \(t' = (\mathrm {e}^{- \mathrm {i}\psi _1}, \mathrm {e}^{ \mathrm {i} \psi _2} , \ldots , \mathrm {e}^{\mathrm {i}\psi _n}, \mathrm {e}^{\phi _1}, \ldots , \mathrm {e}^{\phi _n})\), since \(\mathrm {Det}(k)\, Z(t,k) = (-1)^N Z(t',k)\).

1.1 Comparison with results of other approaches

To facilitate the comparison with related work, we now present our final results in the following explicit form. Let \(x_j := \mathrm {e} ^{- \mathrm {i}\psi _j}\) and \(y_l := \mathrm {e}^{-\phi _l}\). Consider first the case of the unitary symplectic group \(K = \mathrm {USp}_N\) (where \(N \in 2\mathbb {N}\)). Then for any pair of non-negative integers pq in the range \(q - p \le N+1\) one directly infers from (1.3) the formula

$$\begin{aligned} \int \limits _{\mathrm {USp}_N} \frac{\prod _{k=1}^p \mathrm {Det}(1 - x_k \, u)}{\prod _{l=1}^q \mathrm {Det} (1 - y_l \, u)} \, du = \sum _{\epsilon \in \{\pm 1\}^p} \, \frac{\prod _{k=1}^p x_k^{\frac{N}{2}(1-\epsilon _k)} \prod _{l=1}^q (1 - x_k^{\epsilon _k} y_l)}{\prod _{k\le k^\prime } (1 - x_k^{\epsilon _k} x_{k^\prime }^{\epsilon _{k^\prime }}) \prod _{l < l^\prime }(1 - y_l y_{l^\prime })}. \end{aligned}$$

The sum on the right-hand side is over sign configurations \(\epsilon \equiv (\epsilon _1, \ldots , \epsilon _p) \in \{ \pm 1 \}^p\). The proof proceeds by induction in \(p\,\), starting from the result (1.3) for \(p = q\) and sending \(x_p \rightarrow 0\) to pass from p to \(p-1\). In published recent work [3, 8] the same formula was derived under the more restrictive condition \(q \le N/2\,\). In [3] this unwanted restriction on the parameter range came about because the numerator and denominator on the left-hand side were expanded separately, ignoring the super-symmetric Howe duality (see Sect. 2 of the present paper) of the problem at hand.

For \(K = \mathrm {SO}_N\) the same induction process starting from (1.3) yields the result

$$\begin{aligned} \int \limits _{\mathrm {SO}_N} \frac{\prod _{k=1}^p \mathrm {Det}(1 - x_k \, u)}{\prod _{l=1}^q \mathrm {Det} (1 - y_l \, u)} \, du = \sum _{\epsilon \in \{ \pm 1\}^p} \frac{\prod _{k=1}^p (\epsilon _k x_k)^{\frac{N}{2}(1-\epsilon _k)} \prod _{l=1}^q (1 - x_k^{\epsilon _k} y_l)}{\prod _{k < k^\prime } \left( 1 - x_k^{\epsilon _k} x_{k^\prime }^{\epsilon _{k^\prime }}\right) \prod _{l \le l^\prime }\left( 1 - y_l y_{l^\prime }\right) } \end{aligned}$$

as long as \(q-p \le N-1\,\). Please note that this includes even the case of the trivial group \(K = \mathrm {SO}_1 = \{ \mathrm {Id} \}\) with any \(p = q > 0\,\). For \(K = \mathrm {O}_N\) one has an analogous result where the sum on the right-hand side is over \(\epsilon \) with an even number of sign reversals.

The very same formulas for \(\mathrm {SO}_N\) and \(\mathrm {O}_N\) were derived in the recent literature [3, 8] but, again, only in the much narrower range \(q \le \mathrm {Int}[N/2]\). There exist a number of other interesting recent works which emphasize the Lie superalgebraic and combinatorial side of the picture (see, e.g., [46]).

1.2 Howe duality and weight expansion

To find an explicit expression for the integral I(t), we first of all observe that the integrand Z(tk) is the supertrace of a representation \(\rho \) of a semigroup \((T_1 \times T_+) \times K\) on the spinor–oscillator module \(\mathfrak {a}(V)\) (cf. Lemma 4.1). More precisely, we start with the standard K-representation space \(\mathbb {C}^N\), the \(\mathbb {Z}_2\)-graded vector space \(U = U_0 \oplus U_1\) with \(U_s \simeq \mathbb {C}^n\), and the abelian semigroup

$$\begin{aligned} T_1 \times T_+ := \{ (\mathrm {diag}(\mathrm {e}^{\mathrm {i}\psi _1}, \ldots , \mathrm {e}^{\mathrm {i} \psi _n}), \mathrm {diag} (\mathrm {e}^{\phi _1}, \ldots , \mathrm {e}^{\phi _n})) \mid \mathfrak {Re}\, \phi _j > 0 \;, j = 1, \ldots , n \} \end{aligned}$$

of diagonal transformations in \(\mathrm {GL}(U_1) \times \mathrm {GL} (U_0)\). We then consider the vector space \(V := U \otimes \mathbb {C}^N\) which is \(\mathbb {Z}_2\)-graded by \(V_s = U_s \otimes \mathbb {C}^N\), the infinite-dimensional spinor–oscillator module \(\mathfrak {a}(V) := \wedge (V_1^*) \otimes \mathrm {S} (V_0^*)\), and a representation \(\rho \) of \((T_1 \times T_+) \times K\) on \(\mathfrak {a}(V)\). We also let \(V \oplus V^*=: W = W_0 \oplus W_1\) (not the Weyl group).

Averaging the product of ratios Z(tk) with respect to the compact group K corresponds to the projection from \(\mathfrak {a}(V)\) onto the vector space \(\mathfrak {a}(V)^K\) of K-invariants (Corollary 4.1). Now, Howe duality (Proposition 2.2) implies that \(\mathfrak {a}(V)^K\) is the representation space for an irreducible highest-weight representation \(\rho _*\) of the Howe dual partner \(\mathfrak {g}\) of K in the orthosymplectic Lie superalgebra \(\mathfrak {osp}(W)\). This representation \(\rho _*\) is constructed by realizing \(\mathfrak {g} \subset \mathfrak {osp}(W)\) as a subalgebra in the space of degree-two elements of the Clifford–Weyl algebra \(\mathfrak {q}(W)\). Precise definitions of these objects, their relationships, and the Howe duality statement can be found in Sect. 2.

Using the decomposition \(\mathfrak {a}(V)^K = \oplus _{\gamma \in \Gamma }\, V_\gamma \) into weight spaces, Howe duality leads to the weight expansion \(I(\mathrm {e}^H) = \mathrm {STr}_{\mathfrak {a}(V)^K} \, \mathrm {e}^{\rho _*(H)} = \sum _{\gamma \in \Gamma } B_\gamma \, \mathrm {e}^{\gamma (H)}\) for \(t = \mathrm {e}^H \in T_1 \times T_+\,\). Here \(\mathrm {STr}\) denotes the supertrace. There are strong restrictions on the set of weights \(\Gamma \). Namely, if \(\gamma \in \Gamma \), then \(\gamma = \sum _{j=1}^n (\mathrm {i} m_j \psi _j - n_j \phi _j)\) and \(- \frac{N}{2}\le m_j \le \frac{N}{2} \le n_j\) for all \(j\,\). The coefficients \(B_\gamma = \mathrm {STr}_{V_\gamma } (\mathrm {Id})\) are the dimensions of the weight spaces (multiplied by parity). Note that the set of weights of the representation \(\rho _*\) of \(\mathfrak {g}\) on \(\mathfrak {a}(V)^K\) is infinite.

1.3 Group representation and differential equations

Before outlining the strategy for computing our character in the infinite-dimensional setting of representations of Lie superalgebras and groups, we recall the classical situation where \(\rho _*\) is an irreducible finite-dimensional representation of a reductive Lie algebra \(\mathfrak {g}\) and \(\rho \) is the corresponding Lie group representation of the complex reductive group G. In that case the character \(\chi \) of \(\rho \), which automatically exists, is the trace \(\mathrm {Tr}\, \rho \), which is a radial eigenfunction of every Laplace-Casimir operator. These differential equations can be completely understood by their behavior on a maximal torus of G.

In our case we must consider the infinite-dimensional irreducible representation \(\rho _*\) of the Lie superalgebra \(\mathfrak {g} = \mathfrak {osp}\) on \(\mathfrak {a}(V)^K\). Casimir elements, Laplace-Casimir operators of \(\mathfrak {osp}\), and their radial parts have been described by Berezin [1]. In the situation \(U_0 \simeq U_1\) at hand we have the additional feature that every \(\mathfrak {osp}\)-Casimir element I can be expressed as a bracket \(I = [ \partial , F ]\) where \(\partial \) is the holomorphic exterior derivative when we view \(\mathfrak {a} (V)^K\) as the complex of K-equivariant holomorphic differential forms on \(V_0\,\).

To benefit from Berezin’s theory of radial parts, we construct a radial superfunction \(\chi \) which is defined on an open set containing the torus \(T_1 \times T_+\) such that its numerical part satisfies \(\mathrm {num} \chi (t) = I(t)\) for all \(t \in T_1\times T_+\,\). If we had a representation \((\rho _*,\rho )\) of a Lie supergroup \((\mathfrak {osp},G)\) at our disposal, we could define \(\chi \) to be its character, i.e.,

$$\begin{aligned} \chi (g) \mathop {=}\limits ^{?} \mathrm {STr}_{\mathfrak {a}(V)^K} \rho (g)\, \mathrm {e}^{\sum _j \xi _j \, \rho _*(\Xi _j)} . \end{aligned}$$

Since we do not have such a representation, our idea is to define \(\chi \) as a character on a totally real submanifold M of maximal dimension which contains a real form of \(T_1 \times T_+\) and is invariant with respect to conjugation by a real form \(G_\mathbb {R}\) of G, and then to extend \(\chi \) by analytic continuation.

Thinking classically we consider the even part of the Lie superalgebra \(\mathfrak {osp}(W_0 \oplus W_1)\), which is the Lie algebra \(\mathfrak {o}(W_1) \oplus \mathfrak {sp}(W_0)\). The real structures at the Lie supergroup level come from a real form \(W_\mathbb {R}\) of W. The associated real forms of \(\mathfrak {o} (W_1)\) and \(\mathfrak {sp}(W_0)\) are the real orthogonal Lie algebra \(\mathfrak {o}(W_{1,\mathbb {R}})\) and the real symplectic Lie algebra \(\mathfrak {sp}(W_{0,\mathbb {R}})\). These are defined in such a way that the elements in \(\mathfrak {o}(W_{1,\mathbb {R}}) \oplus \mathfrak {sp} (W_{0,\mathbb {R}})\) and \(\mathrm {i} W_\mathbb {R}\) are mapped as elements of the Clifford–Weyl algebra via the spinor–oscillator representation to anti-Hermitian operators on \(\mathfrak {a}(V)\) with respect to a compatibly defined unitary structure. In this context we frequently use the unitary representation of the real Heisenberg group \(\exp ( \mathrm {i} W_{0, \mathbb {R}}) \times \mathrm {U}_1\) on the completion \(\fancyscript {A}_V\) of the module \(\mathfrak {a}(V)\).

Since \(\wedge (V^*_1)\) has finite dimension, exponentiating the spinor representation of \(\mathfrak {o}(W_{1,\mathbb {R}})\) causes no difficulties. This results in the spinor representation of \(\mathrm {Spin}(W_{1,\mathbb {R}})\), a 2:1 covering of the compact group \(\mathrm {SO}(W_{1,\mathbb {R}})\). So in this case one easily constructs a representation \(R_1 : \, \mathrm {Spin} (W_{1, \mathbb {R}}) \rightarrow \mathrm {U} (\mathfrak {a}(V))\) which is compatible with \(\rho _*|_{\mathfrak {o} (W_{1,\mathbb {R}})}\).

Exponentiating the oscillator representation of \(\mathfrak {sp} (W_{0,\mathbb {R}})\) on the infinite-dimensional vector space \(\mathrm {S}(V_0^*)\) requires more effort. In Sect. 3.4, following Howe [11], we construct the Shale–Weil–Segal representation \(R' : \, \mathrm {Mp}(W_{0, \mathbb {R}}) \rightarrow \mathrm {U} (\fancyscript {A}_V)\) of the metaplectic group \(\mathrm {Mp}(W_{0, \mathbb {R}})\) which is the 2:1 covering group of the real symplectic group \(\mathrm {Sp} (W_{ 0,\mathbb {R}})\). This is compatible with \(\rho _*\vert _{\mathfrak {sp} (W_{0, \mathbb {R}})}\). Altogether we see that the even part of the Lie superalgebra representation integrates to \(G_\mathbb {R} = \mathrm {Spin} (W_{1 ,\mathbb {R}}) \times _{\mathbb {Z}_2} \mathrm {Mp}(W_{0,\mathbb {R}})\,\).

The construction of \(R^\prime \) uses a limiting process coming from the oscillator semigroup \(\widetilde{\mathrm {H}}(W_0^s)\), which is the double covering of the contraction semigroup \(\mathrm {H}(W_0^s) \subset \mathrm {Sp}(W_0)\) and has \(\mathrm {Mp}(W_{0,\mathbb {R}})\) in its boundary. Furthermore, we have \(\widetilde{ \mathrm {H}} (W_0^s) = \mathrm {Mp}(W_{0,\mathbb {R}}) \times M\) where M is an analytic totally real submanifold of maximal dimension which contains a real form of the torus \(T_+\) (see Sect. 3.2). The representation \(R_0 : \,\widetilde{\mathrm {H}}(W_0^s)\rightarrow \mathrm {End}(\fancyscript {A}_V)\) constructed in Sect. 3.3 facilitates the definition of the representation \(R^\prime \) and of the character \(\chi \) in Sect. 4.2 and Sect. 5.1. It should be underlined that Proposition 3.24 ensures convergence of the superfunction \(\chi (h)\), which is defined as a supertrace and exists for all \(h \in \widetilde{\mathrm {H}}(W_0^s)\).

On that basis, the key idea of our approach is to exploit the fact that every Casimir invariant \(I \in \mathsf {U}(\mathfrak {g})\) is exact in the sense that \(I = [ \partial , F]\). By a standard argument, this exactness property implies that every such invariant I vanishes in the spinor–oscillator representation. This result in turn implies for our character \(\chi \) the differential equations \(D(I) \chi = 0\) where D(I) is the Laplace-Casimir operator representing I. By drawing on Berezin’s theory of radial parts, we derive a system of differential equations which in combination with certain other properties ultimately determines \(\chi \).

In the case of \(K = \mathrm {O}_N\) the Lie group associated to the even part of the real form of the Howe partner \(\mathfrak {g}\) is embedded in a simple way in the full group \(G_\mathbb {R}\) described above. It is itself just a lower-dimensional group of the same form. In the case of \(K = \mathrm {USp}_N\) a sort of reversing procedure takes places and the analogous real form is \(\mathrm {USp}_{2n} \times \mathrm {SO}_{2n}^*\). Nevertheless, the precise data which are used as input into the series developments, the uniqueness theorem, and the final calculations of \(\chi \) are essentially the same in the two cases. Therefore there is no difficulty handling them simultaneously.

2 Howe dual pairs in the orthosymplectic Lie superalgebra

In this chapter we collect some foundational information from representation theory. Basic to our work is the orthosymplectic Lie superalgebra, \(\mathfrak {osp}\,\), in its realization as the space spanned by super-symmetrized terms of degree two in the Clifford–Weyl algebra. Representing the latter by its fundamental representation on the spinor–oscillator module, one gets a representation of \(\mathfrak {osp}\) and of all Howe dual pairs inside of \(\mathfrak {osp} \,\). Roots and weights of the relevant representations are described in detail.

2.1 Notion of Lie superalgebra

A \(\mathbb {Z}_2\) -grading of a vector space V over \(\mathbb {K} = \mathbb {R}\) or \(\mathbb {C}\) is a decomposition \(V = V_0\oplus V_1\) of V into the direct sum of two \(\mathbb {K}\)-vector spaces \(V_0\) and \(V_1\,\). The elements in \((V_0 \cup V_1) {\setminus} \{ 0\}\) are called homogeneous. The parity function \(|\, |:(V_0 \cup V_1) {\setminus} \{ 0\} \rightarrow \mathbb {Z}_2\,\), \(v\in V_s \mapsto |v| = s\,\), assigns to a homogeneous element its parity. We write \(V \simeq \mathbb {K}^{p| q}\) if \(\dim _\mathbb {K} V_0 = p\) and \(\dim _\mathbb {K} V_1 = q\,\).

A Lie superalgebra over \(\mathbb {K}\) is a \(\mathbb {Z}_2\)-graded \(\mathbb {K}\)-vector space \(\mathfrak {g} = \mathfrak {g}_0 \oplus \mathfrak {g}_1\) equipped with a bilinear map \([\, , \,]: \, \mathfrak {g} \times \mathfrak {g} \rightarrow \mathfrak {g}\) satisfying

  1. 1.

    \([\mathfrak {g}_s , \mathfrak {g}_{s^\prime } ] \subset \mathfrak {g}_{s + s^\prime }\,\), i.e., \(|[X,Y]|= |X| + |Y|\) (mod 2) for homogeneous elements XY.

  2. 2.

    Skew symmetry: \([X,Y] = -(-1)^{|X||Y|}[Y,X]\) for homogeneous XY.

  3. 3.

    Jacobi identity, which means that \(\mathrm {ad}(X) = [X, \; ] : \, \mathfrak {g} \rightarrow \mathfrak {g}\) is a (super-)derivation:

    $$\begin{aligned} \mathrm {ad}(X)\, [Y,Z] = [\mathrm {ad}(X)Y , Z] + (-1)^{|X||Y|}[Y,\mathrm {ad}(X)Z]. \end{aligned}$$

Example 2.1

(\(\mathfrak {gl}(V)\)) Let \(V = V_0 \oplus V_1\) be a \(\mathbb {Z}_2\)-graded \(\mathbb {K}\)-vector space. There is a canonical \(\mathbb {Z}_2\)-grading \(\mathrm {End}(V) = \mathrm {End}(V)_0 \oplus \mathrm {End}(V)_1\) induced by the grading of V:

$$\begin{aligned} \mathrm {End}(V)_s := \{X \in \mathrm {End}(V) \mid \forall s^\prime \in \mathbb {Z}_2 : \, X(V_{s^\prime }) \subset V_{s + s^\prime }\}. \end{aligned}$$

The bilinear extension of \([X,Y] := XY - (-1)^{|X||Y|}YX\) for homogeneous elements \(X , Y \in \mathrm {End}(V)\) to a bilinear map \([\, , \,] : \, \mathrm {End}(V) \times \mathrm {End}(V) \rightarrow \mathrm {End}(V)\) gives \(\mathrm {End}(V)\) the structure of a Lie superalgebra, namely \(\mathfrak {gl}(V)\). The Jacobi identity in this case is a direct consequence of the associativity \((XY)Z = X (YZ)\) and the definition of [XY].

In fact, for every \(\mathbb {Z}_2\)-graded associative algebra \(\fancyscript {A}\) the bracket \([\, , \,] : \, \fancyscript {A} \times \fancyscript {A} \rightarrow \fancyscript {A}\) defined by \([X,Y] = XY - (-1)^{|X| |Y|} YX\) satisfies the Jacobi identity.

Example 2.2

(\(\mathfrak {osp}(V \oplus V^*)\)) Let \(V = V_0 \oplus V_1\) be a \(\mathbb {Z}_2\)-graded \(\mathbb {K}\)-vector space and put \(W := V \oplus V^*\). The \(\mathbb {Z}_2\)-grading of V induces a \(\mathbb {Z}_2\)-grading \(W = W_0 \oplus W_1\) in the obvious manner: \(W_0^{} = V_0^{} \oplus V_0^*\) and \(W_1^{} = V_1^{} \oplus V_1^*\,\). Then consider the canonical alternating bilinear form A on \(W_0\,\),

$$\begin{aligned} A : \,\, W_0 \times W_0 \rightarrow \mathbb {K} \;, \quad (v + \varphi \, , v' + \varphi ') \mapsto \varphi '(v) - \varphi (v'), \end{aligned}$$

and the canonical symmetric bilinear form S on \(W_1\,\),

$$\begin{aligned} S : \,\, W_1 \times W_1 \rightarrow \mathbb {K} \;, \quad (v + \varphi \, , v' + \varphi ') \mapsto \varphi '(v) + \varphi (v'). \end{aligned}$$

The orthosymplectic form of W is the non-degenerate bilinear form \(Q : \, W \times W \rightarrow \mathbb {K}\) defined as the orthogonal sum \(Q = A + S{:}\)

$$\begin{aligned} Q(w_0^{} + w_1^{} , w'_0 + w'_1) = A(w_0^{} , w'_0) + S(w_1^{} , w'_1) \quad (w_s^{} \, , w_s^\prime \in W_s^{}). \end{aligned}$$

Note the exchange symmetry \(Q(w,w^\prime ) = -(-1)^{|w| |w^\prime |} Q(w^\prime , w)\) for \(w, w^\prime \in W_0 \cup W_1\,\).

Given \(Q\,\), define a complex linear bijection \(\tau : \, \mathrm {End}(W) \rightarrow \mathrm {End}(W)\) by the equation

$$\begin{aligned} Q(\tau (X) w , w') + (-1)^{|X||w|} Q(w, X w^\prime ) = 0 \end{aligned}$$
(2.1)

for all \(w, w^\prime \in W_0 \cup W_1\,\). It is easy to check that \(\tau \) has the property

$$\begin{aligned} \tau (XY) = - (-1)^{|X| |Y|} \tau (Y) \tau (X) \;, \end{aligned}$$

which implies that \(\tau \) is an involutory automorphism of the Lie superalgebra \(\mathfrak {gl}(W)\) with bracket \([X,Y] = XY - (-1)^{|X| |Y|} YX\,\). Hence the subspace \(\mathfrak {osp}(W) \subset \mathrm {End}(W)\) of \(\tau \)-fixed points is closed w.r.t. that bracket; it is called the (complex) orthosymplectic Lie superalgebra of W.

Example 2.3

(Jordan–Heisenberg algebra) Using the notation of Example 2.2, consider the vector space \(\widetilde{W} := W \oplus \mathbb {K}\) and take it to be \(\mathbb {Z}_2\)-graded by \(\widetilde{W}_0 = W_0 \oplus \mathbb {K}\) and \(\widetilde{W}_1 = W_1\,\). Define a bilinear mapping \([ \, , \,] : \, \widetilde{W} \times \widetilde{W} \rightarrow \widetilde{W}\) by

$$\begin{aligned} \,[\mathbb {K}\, ,\widetilde{W}] = [\widetilde{W},\mathbb {K} ] = 0 \;, \quad [W , W] \subset \mathbb {K} \;, \quad [w , w^\prime ] = Q(w,w^\prime ) \quad (w,w^\prime \in W). \end{aligned}$$

By the basic properties of the orthosymplectic form \(Q\,\), the vector space \(\widetilde{W}\) equipped with this bracket is a Lie superalgebra—the so-called Jordan–Heisenberg algebra. Note that \(\widetilde{W}\) is two-step nilpotent, i.e., \([\widetilde{W} , [ \widetilde{W} , \widetilde{W} ] ] = 0\,\).

2.1.1 Supertrace

Let \(V = V_0\oplus V_1\) be a \(\mathbb {Z}_2\)-graded \(\mathbb K\)-vector space, and recall the decomposition \(\mathrm {End}(V) = \bigoplus _{ s,\, t} \mathrm {Hom}(V_s \, , V_t)\). For \(X \in \mathrm {End}(V)\), we denote by \(X = \sum _{s,\, t} X_{\, t s}\) the corresponding decomposition of an operator. The supertrace on V is the linear function

$$\begin{aligned} \mathrm {STr} : \,\, \mathrm {End}(V) \rightarrow \mathbb {K} \;, \quad X \mapsto \mathrm {Tr}\, X_{00} - \mathrm {Tr}\, X_{11} = \sum \nolimits _s (-1)^s \mathrm {Tr}\, X_{s s}. \end{aligned}$$

(If \(\mathrm {dim}\, V = \infty \,\), then usually the domain of definition of \(\mathrm {STr}\) must be restricted.)

An \(\mathrm {ad}\)-invariant bilinear form on a Lie superalgebra \(\mathfrak {g} = \mathfrak {g}_0 \oplus \mathfrak {g}_1\) over \(\mathbb {K}\) is a bilinear mapping \(B : \, \mathfrak {g} \times \mathfrak {g} \rightarrow \mathbb {K}\) with the properties

  1. 1.

    \(\mathfrak {g}_0\) and \(\mathfrak {g}_1\) are B-orthogonal to each other;

  2. 2.

    B is symmetric on \(\mathfrak {g}_0\) and skew on \(\mathfrak {g}_1\,\);

  3. 3.

    \(B([X,Y],Z) = B(X,[Y,Z])\) for all \(X, Y, Z \in \mathfrak {g}\,\).

We will repeatedly use the following direct consequences of the definition of \(\mathrm {STr}\).

Lemma 2.1

If \(\mathfrak {g}\) is a Lie superalgebra in \(\mathrm {End}(V)\) , the trace form \(B(X,Y) = \mathrm {STr}\, (XY)\) is an \(\mathrm {ad} \) -invariant bilinear form. One has \(\mathrm {STr} \, [X,Y] = 0\,\).

Recalling the setting of Example 2.2, note that the supertrace for \(W = V \oplus V^*\) is odd under the \(\mathfrak {gl}\)-automorphism \(\tau \) fixing \(\mathfrak {osp}(W)\), i.e., \(\mathrm {STr}_W \circ \tau = - \mathrm {STr}_W\,\). It follows that \(\mathrm {STr}_W X = 0\) for any \(X \in \mathfrak {osp}(W)\). Moreover, \(\mathrm {STr}_W (X_1 X_2 \cdots X_{2n+1}) = 0\) for any product of an odd number of \(\mathfrak {osp}\)-elements.

2.1.2 Universal enveloping algebra

Let \(\mathfrak {g}\) be a Lie superalgebra with bracket \([ \, , \, ]\). The universal enveloping algebra \(\mathsf {U}(\mathfrak {g})\) is defined as the quotient of the tensor algebra \(\mathsf {T}(\mathfrak {g}) = \oplus _{n = 0}^\infty \mathsf {T}^n (\mathfrak {g})\) by the two-sided ideal \(\mathsf {J} (\mathfrak {g})\) generated by all combinations

$$\begin{aligned} X \otimes Y - (-1)^{|X| |Y|} Y \otimes X - [X,Y] \end{aligned}$$

for homogeneous \(X , Y \in \mathsf {T}^1( \mathfrak {g}) \equiv \mathfrak {g}\,\). If \(\mathsf {U}_n(\mathfrak {g})\) is the image of \(\mathsf {T}_n(\mathfrak {g}) := \oplus _{k = 0}^n \mathsf {T}^k (\mathfrak {g})\) under the projection \(\mathsf {T} (\mathfrak {g}) \rightarrow \mathsf {U}(\mathfrak {g}) = \mathsf {T}(\mathfrak {g}) / \mathsf {J} (\mathfrak {g})\), the algebra \(\mathsf {U}(\mathfrak {g})\) is filtered by \(\mathsf {U}( \mathfrak {g}) = \cup _{n=0}^\infty \mathsf {U}_n (\mathfrak {g})\). The \(\mathbb {Z}_2\)-grading \(\mathfrak {g} = \mathfrak {g}_0 \oplus \mathfrak {g}_1\) gives rise to a \(\mathbb {Z}_2\)-grading of \(\mathsf {T}(\mathfrak {g})\) by

$$\begin{aligned} | X_1 \otimes X_2 \otimes \cdots \otimes X_n | = \sum \nolimits _{i = 1}^n | X_i | \quad (\text {for homogenous } X_i \in \mathfrak {g}), \end{aligned}$$

and this in turn induces a canonical \(\mathbb {Z}_2\)-grading of \(\mathsf {U} (\mathfrak {g})\).

One might imagine introducing various bracket operations on \(\mathsf {T}(\mathfrak {g})\) and/or \(\mathsf {U}(\mathfrak {g})\). However, in view of the canonical \(\mathbb {Z}_2\)-grading, the natural bracket operation to use is the supercommutator, which is the bilinear map \(\mathsf {T}(\mathfrak {g}) \times \mathsf {T}(\mathfrak {g}) \rightarrow \mathsf {T}(\mathfrak {g})\) defined by \(\{ a , b \} := a b - (-1)^{|a||b|} b a\) for homogeneous elements \(a, \, b \in \mathsf {T}(\mathfrak {g})\). (For the time being, we use a different symbol \(\{ \, , \, \}\) for better distinction from the bracket \([\, , \,]\) on \(\mathfrak {g}\,\).) Since by the definition of \(\mathsf {J} (\mathfrak {g})\) one has

$$\begin{aligned} \{ \mathsf {T}(\mathfrak {g}),\mathsf {J}(\mathfrak {g}) \} = \{ \mathsf {J}(\mathfrak {g}), \mathsf {T}(\mathfrak {g}) \} \subset \mathsf {J}(\mathfrak {g}) , \end{aligned}$$

the supercommutator descends to a well-defined map \(\{\,,\,\} : \, \mathsf {U}(\mathfrak {g})\times \mathsf {U}(\mathfrak {g}) \rightarrow \mathsf {U}(\mathfrak {g})\).

Lemma 2.2

If \(\mathfrak {g}\) is a Lie superalgebra, the supercommutator \(\{\,,\,\}\) gives \(\mathsf {U}(\mathfrak {g})\) the structure of another Lie superalgebra in which \(\{ \mathsf {U}_n(\mathfrak {g}) , \mathsf {U}_{n^\prime }(\mathfrak {g})\} \subset \mathsf {U}_{n + n^\prime - 1} (\mathfrak {g})\).

Proof

Compatibility with the \(\mathbb {Z}_2\)-grading, skew symmetry, and Jacobi identity are properties of \(\{ \, , \, \}\) that are immediate at the level of the tensor algebra \(\mathsf {T}(\mathfrak {g})\). They descend to the corresponding properties at the level of \(\mathsf {U} (\mathfrak {g})\) by the definition of the two-sided ideal \(\mathsf {J} (\mathfrak {g})\). Thus \(\mathsf {U}(\mathfrak {g})\) with the bracket \(\{ \, , \, \}\) is a Lie superalgebra.

To see that \(\{\mathsf {U}_n(\mathfrak {g}),\mathsf {U}_{n^\prime } (\mathfrak {g})\}\) is contained in \(\mathsf {U}_{n + n^\prime - 1} (\mathfrak {g})\), notice that this property holds true for \(n = n^\prime = 1\) by the defining relations \(\mathsf {J}(\mathfrak {g}) \equiv 0\) of \(\mathsf {U} (\mathfrak {g})\). Then use the associative law for \(\mathsf {U}(\mathfrak {g})\) to verify the formula

$$\begin{aligned} \{ a , bc \} = a b c - (-1)^{|a| (|b| + |c|)} b c a = \{ a , b \} c + (-1)^{|a| |b|} b \{ a , c \} \end{aligned}$$

for homogeneous \(a, b, c \in \mathsf {U}(\mathfrak {g})\). The claim now follows by induction on the degree of the filtration \(\mathsf {U} (\mathfrak {g}) = \cup _{n = 0}^\infty \mathsf {U}_n (\mathfrak {g})\). \(\square \)

By definition, the supercommutator of \(\mathsf {U}(\mathfrak {g})\) and the bracket of \(\mathfrak {g}\) agree at the linear level: \(\{X,Y\} \equiv [X,Y]\) for \(X, Y \in \mathfrak {g}\,\). It is therefore reasonable to drop the distinction in notation and simply write \([ \, , \,]\) for both of these product operations. This we now do.

For future use, note the following variant of the preceding formula: if \(Y_1, \ldots , Y_k\, , X\) are any homogeneous elements of \(\mathfrak {g}\,\), then

$$\begin{aligned}{}[Y_1 \cdots Y_k \, , \, X] = \sum _{i=1}^k (-1)^{|X|\sum _{j = i+1}^k |Y_j|}\, Y_1 \cdots Y_{i-1}\, [Y_i\, , X] \, Y_{i+1} \cdots Y_k, \end{aligned}$$
(2.2)

which expresses the supercommutator in \(\mathsf {U}(\mathfrak {g})\) by the bracket in \(\mathfrak {g}\,\).

2.2 Structure of \(\mathfrak {osp}(W)\)

For a \(\mathbb {Z}_2\)-graded \(\mathbb {K}\)-vector space \(V = V_0 \oplus V_1\) let \(W = V \oplus V^*= W_0 \oplus W_1\) as in Example 2.2. The orthogonal Lie algebra \(\mathfrak {o}(W_1)\) is the Lie algebra of the Lie group \(\mathrm {O}(W_1)\) of \(\mathbb {K}\)-linear transformations of \(W_1\) that leave the non-degenerate symmetric bilinear form S invariant. This means that \(X \in \mathrm {End}(W_1)\) is in \(\mathfrak {o}(W_1)\) if and only if

$$\begin{aligned} \forall w, w^\prime \in W_1 : \,\, S(Xw,w') + S(w,Xw') = 0. \end{aligned}$$

Similarly, the symplectic Lie algebra \(\mathfrak {sp}(W_0)\) is the Lie algebra of the automorphism group \(\mathrm {Sp}(W_0)\) of \(W_0\) equipped with the non-degenerate alternating bilinear form A

$$\begin{aligned} \mathfrak {sp}(W_0) = \{ X\in \mathrm {End}(W_0) \mid \forall w ,w' \in W_0 : \,\, A(Xw ,w') + A(w,Xw') = 0 \}. \end{aligned}$$

For the next statement, recall the definition of the orthosymplectic Lie superalgebra \(\mathfrak {osp}(W)\) and the decomposition \(\mathfrak {osp}(W) = \mathfrak {osp}(W)_0 \oplus \mathfrak {osp}(W)_1\,\).

Lemma 2.3

As Lie algebras resp. vector spaces,

$$\begin{aligned} \mathfrak {osp}(W)_0 \simeq \mathfrak {o}(W_1)\oplus \mathfrak {sp}(W_0) \;, \quad \mathfrak {osp}(W)_1 \simeq W_1^{} \otimes W_0^*. \end{aligned}$$

Proof

The first isomorphism follows directly from the definitions. For the second isomorphism, decompose \(X \in \mathfrak {osp}(W)_1\) as \(X = X_{01}+ X_{10}\) where \(X_{01} \in \mathrm {Hom}(W_1,W_0)\) and \(X_{10} \in \mathrm {Hom}(W_0\, ,W_1)\). Then

$$\begin{aligned} \mathfrak {osp}(W)_1 = \{ X_{10} + X_{01} \in \mathrm {End}(W)_1 \mid \forall w_s \in W_s : \, S(X_{10}w_0\, ,w_1) + A(w_0\, ,X_{01}w_1) = 0 \}. \end{aligned}$$

Since both S and A are non-degenerate, the component \(X_{01}\) is determined by the component \(X_{10}\,\), and one therefore has \(\mathfrak {osp}(W)_1 \simeq \mathrm {Hom}(W_0\, , W_1) \simeq W_1^{} \otimes W_0^*\,\). \(\square \)

We now review how \(\mathfrak {sp}(W_0)\) and \(\mathfrak {o}(W_1)\) decompose for our case \(W_s = V_s^{} \oplus V_s^{*} \,\). For that purpose, if U is a vector space with dual vector space \(U^*\), let \(\mathrm {Sym}(U,U^*)\) and \(\mathrm {Alt}(U, U^*)\) denote the symmetric resp. alternating linear maps from U to \(U^*\).

Lemma 2.4

As vector spaces,

$$\begin{aligned} \mathfrak {o}(W_1)&\simeq \mathrm {End}(V_1) \oplus \mathrm {Alt} (V_1^{}, V_1^*) \oplus \mathrm {Alt} (V_1^*, V_1^{}) \;, \\ \mathfrak {sp}(W_0)&\simeq \mathrm {End}(V_0) \oplus \mathrm {Sym}(V_0^{}\, , V_0^*)\oplus \mathrm {Sym}(V_0^*, V_0^{}). \end{aligned}$$

Proof

There is a canonical decomposition

$$\begin{aligned} \mathrm {End}(W_s) = \mathrm {End}(V_s^{}) \oplus \mathrm {Hom}(V_s^*, V_s^{}) \oplus \mathrm {Hom} (V_s^{}\,,V_s^*)\oplus \mathrm {End}(V_s^*) \end{aligned}$$

for \(s = 0, 1\). Let \(s = 1\) and write the corresponding decomposition of \(X \in \mathrm {End}(W_1)\) as

$$\begin{aligned} X = \mathsf {A} \oplus \mathsf {B} \oplus \mathsf {C} \oplus \mathsf {D} . \end{aligned}$$

Substituting \(w = v + \varphi \) and \(w^\prime = v^\prime + \varphi ^\prime \), the defining condition \(S(Xw,w^\prime ) = - S(w,X w^\prime )\) for \(X \in \mathfrak {o}(W_1)\) then transcribes to

$$\begin{aligned} \varphi ^\prime (\mathsf {A}v) = - (\mathsf {D}\varphi ^\prime )(v) \;, \quad (\mathsf {C}v)(v^\prime ) = - (\mathsf {C}v^\prime )(v) \;, \quad \varphi ^\prime (\mathsf {B}\varphi ) = - \varphi (\mathsf {B}\varphi ^\prime ), \end{aligned}$$

for all \(v, v^\prime \in V_1\) and \(\varphi , \varphi ^\prime \in V_1^*\). Thus \(\mathsf {D} = - \mathsf {A}^\mathrm {t}\), and the maps \(\mathsf {B}, \mathsf {C}\) are alternating. This already proves the statement for the case of \(\mathfrak {o}(W_1)\).

The situation for \(\mathfrak {sp}(W_0)\) is identical but for a sign change: the symmetric form S is replaced by the alternating form \(A\,\), and this causes the parity of \(\mathsf {B}, \mathsf {C}\) to be reversed. \(\square \)

By adding up dimensions, Lemmas 2.3 and 2.4 entail the following consequence.

Corollary 2.1

As a \(\mathbb {Z}_2\) -graded vector space, \(\mathfrak {osp}(V \oplus V^*)\) is isomorphic to \(\mathbb {K}^{p|q}\) where \(p = d_0 (2d_0 + 1) + d_1 (2d_1 - 1)\), \(q = 4 d_0 d_1\,\) , and \(d_s = \dim V_s \,\).

There exists another way of thinking about \(\mathfrak {osp}(W)\), which will play a key role in the sequel. To define it and keep the sign factors consistent and transparent, we need to be meticulous about our ordering conventions. Hence, if \(v \in V\) is a vector and \(\varphi \in V^*\) is a linear function, we write the value of \(\varphi \) on v as

$$\begin{aligned} \varphi (v) \equiv \langle v , \varphi \rangle. \end{aligned}$$

Based on this notational convention, if V is a \(\mathbb {Z}_2\)-graded vector space and \(X \in \mathrm {End}(V)\) is a homogeneous operator, we define the supertranspose \(X^\mathrm {st} \in \mathrm {End}(V^*)\) of X by

$$\begin{aligned} \langle v , X^\mathrm {st} \varphi \rangle := (-1)^{|X| |v|} \langle Xv , \varphi \rangle \quad (v \in V_0 \cup V_1 , \,\, \varphi \in V^*). \end{aligned}$$

This definition differs from the usual transpose by a change of sign in the case when X has a component in \(\mathrm {Hom}(V_1 , V_0)\). From it, it follows directly that the negative supertranspose \(\mathfrak {gl}(V) \rightarrow \mathfrak {gl}(V^*)\), \(X \mapsto -X^\mathrm {st}\) is an isomorphism of Lie superalgebras:

$$\begin{aligned} - [X,Y]^\mathrm {st} = [-X^\mathrm {st} , -Y^\mathrm {st}]. \end{aligned}$$

The modified notion of transpose goes hand in hand with a modified notion of what it means for an operator in \(\mathrm {Hom}(V,V^*)\) or \(\mathrm {Hom}(V^*, V)\) to be symmetric. Thus, define the subspace \(\mathrm {Sym}(V^*,V) \subset \mathrm {Hom}(V^*,V)\) to consist of the elements, say \(\mathsf {B}\), which are symmetric in the \(\mathbb {Z}_2\)-graded sense:

$$\begin{aligned} \forall \, \varphi , \varphi ^\prime \in V_0^*\cup V_1^*: \quad \langle \mathsf {B} \varphi , \varphi ^\prime \rangle = \langle \mathsf {B} \varphi ^\prime , \varphi \rangle \, (-1)^{|\varphi | |\varphi ^\prime |}. \end{aligned}$$
(2.3)

By the same principle, define \(\mathrm {Sym}(V,V^*) \subset \mathrm {Hom}(V,V^*)\) as the set of solutions \(\mathsf {C}\) of

$$\begin{aligned} \forall \, v, v^\prime \in V_0 \cup V_1 : \quad \langle v , \mathsf {C}v^\prime \rangle = \langle v^\prime , \mathsf {C} v\rangle \, (-1)^{|v| |v^\prime | + |v| + |v^\prime |}. \end{aligned}$$
(2.4)

To make the connection with the decomposition of Lemma 2.3 and 2.4, notice that

$$\begin{aligned} \mathrm {Sym}(V,V^*)\cap \mathrm {Hom}(V_s^{} \,, V_s^*) = \left\{ \begin{array}{ll}\mathrm {Sym} (V_0^{}\,, V_0^*) &{}s = 0,\\ \mathrm {Alt}(V_1^{} , V_1^*) &{}s = 1, \end{array} \right. \end{aligned}$$

and similar for the corresponding intersections involving \(\mathrm {Sym}(V^*,V)\).

Next, expressing the orthosymplectic form Q of \(W = V \oplus V^*\) as

$$\begin{aligned} Q(v + \varphi , v^\prime + \varphi ^\prime ) = \langle v , \varphi ^\prime \rangle - (-1)^{|v^\prime | |\varphi |} \langle v^\prime , \varphi \rangle, \end{aligned}$$

and writing out the conditions resulting from \(Q(Xw,w^\prime ) + (-1)^ {|X| |w|} Q(w,Xw^\prime ) = 0\) for the case of \(X \equiv \mathsf {B} \in \mathrm {Hom}(V^*,V)\) and \(X \equiv \mathsf {C} \in \mathrm {Hom}(V,V^*)\), one sees that

$$\begin{aligned} \mathfrak {osp}(W) \cap \mathrm {Hom}(V,V^*) = \mathrm {Sym}(V,V^*)\;, \quad \mathfrak {osp}(W) \cap \mathrm {Hom}(V^*,V) = \mathrm {Sym}(V^*,V). \end{aligned}$$

This situation is summarized in the next statement.

Lemma 2.5

The orthosymplectic Lie superalgebra of \(\, W = V \oplus V^*\) decomposes as

$$\begin{aligned} \mathfrak {osp}(W) = \mathfrak {g}^{(-2)}\oplus \mathfrak {g}^{(0)}\oplus \mathfrak {g}^{(+2)}\;, \end{aligned}$$

where \(\mathfrak {g}^{(+2)} := \mathrm {Sym}(V,V^*)\) , and \(\mathfrak {g}^{(-2)} := \mathrm {Sym}(V^*,V)\) , and

$$\begin{aligned} \mathfrak {g}^{(0)} := (\mathrm {End}(V)\oplus \mathrm {End}(V^*))\cap \mathfrak {osp}(W)\;. \end{aligned}$$

The decomposition of Lemma 2.5 can be regarded as a \(\mathbb {Z}\)-grading of \(\mathfrak {osp}(W)\). By the ‘block’ structure inherited from \(W = V \oplus V^*\), this decomposition is compatible with the bracket \([\, , \,]:\)

$$\begin{aligned}{}[ \mathfrak {g}^{(m)} , \, \mathfrak {g}^{(m^\prime )} ] \subset \mathfrak {g}^{(m + m^\prime )}, \end{aligned}$$

where \(\mathfrak {g}^{(m + m^\prime )} \equiv 0\) if \(m + m^\prime \notin \{ \pm 2, 0\}\). It follows that each of the three subspaces \(\mathfrak {g}^{( +2)}\), \(\mathfrak {g}^{(-2)}\), and \(\mathfrak {g}^{ (0)}\) is a Lie superalgebra, the first two with vanishing bracket.

Lemma 2.6

The embedding \(\mathrm {End}(V)\rightarrow \mathrm {End}(V) \oplus \mathrm {End} (V^*)\) by \(\mathsf {A} \mapsto \mathsf {A} \oplus (-\mathsf {A}^ \mathrm {st})\) projected to \(\mathfrak {osp}(W)\) is an isomorphism of Lie superalgebras \(\mathfrak {gl}(V) \rightarrow \mathfrak {g}^{(0)}\).

Proof

Since the negative supertranspose \(\mathsf {A} \mapsto - \mathsf {A} ^\mathrm {st}\) is a homomorphism of Lie superalgebras, so is our embedding \(\mathsf {A} \mapsto \mathsf {A} \oplus (- \mathsf {A} ^\mathrm {st})\). This map is clearly injective. To see that it is surjective, consider any homogeneous \(X = \mathsf {A} \oplus \mathsf {D} \in \mathrm {End}(V) \oplus \mathrm {End}(V^*)\) viewed as an operator in \(\mathrm {End}(W)\). The condition for X to be in \(\mathfrak {osp}(W)\) is (2.1). To get a non-trivial condition, choose \((w,w^\prime ) = (v,\varphi )\) or \((w,w^\prime ) = (\varphi ,v)\). The first choice gives

$$\begin{aligned} Q(Xv,\varphi ) + (-1)^{|X||v|} Q(v,X\varphi ) = \langle \mathsf {A} v , \varphi \rangle + (-1)^{|\mathsf {A}||v|} \langle v , \mathsf {D}\varphi \rangle = 0. \end{aligned}$$

Valid for all \(v \in V_0 \cup V_1\) and \(\varphi \in V^*\), this implies that \(\mathsf {D} = - \mathsf {A}^\mathrm {st}\). The second choice leads to the same conclusion. Thus \(X = \mathsf {A} \oplus \mathsf {D}\) is in \(\mathfrak {osp}(W)\) if and only if \(\mathsf {D} = - \mathsf {A}^\mathrm {st}\). \(\square \)

In the following subsections we will often write \(\mathfrak {osp}(W) \equiv \mathfrak {osp}\) for short.

2.2.1 Roots and root spaces

A Cartan subalgebra of a Lie algebra \(\mathfrak {g}_0\) is a maximal commutative subalgebra \(\mathfrak {h} \subset \mathfrak {g}_0\) such that \(\mathfrak {g}_0\) (or its complexification if \(\mathfrak {g}_0\) is a real Lie algebra) has a basis consisting of eigenvectors of \(\mathrm {ad}(H)\) for all \(H \in \mathfrak {h} \,\). Recall that \(|[X,Y]|=|X|+|Y|\) for homogeneous elements XY of a Lie superalgebra \(\mathfrak {g}\,\). From the vantage point of decomposing \(\mathfrak {g}\) by eigenvectors or root spaces, it is therefore reasonable to call a Cartan subalgebra of \(\mathfrak {g}_0\) a Cartan subalgebra of \(\mathfrak {g}\,\). We will see that \(X \in \mathfrak {osp}_1\) and \([X,H] = 0\) for all \(H \in \mathfrak {h} \subset \mathfrak {osp}_0\) imply \(X = 0\,\), i.e., there exists no commutative subalgebra of \(\mathfrak {osp}\) that properly contains a Cartan subalgebra. Lie superalgebras with this property are called of type I in [1].

Let us determine a Cartan subalgebra and the corresponding root space decomposition of \(\mathfrak {osp}\,\). For \(s, t = 0, 1\) choose bases \(\{ e_{s, 1}\, , \ldots , \, e_{s,\,d_s} \}\) of \(V_s\) and associated dual bases \(\{ f_{t,1} , \ldots , f_{t,\, d_t} \}\) of \(V_t^*\,\). Then for \(j = 1, \ldots , d_s\) and \(k = 1, \ldots , d_t\) define rank-one operators \(E_{s, j \, ; \, t ,\, k}\) by the equation \(E_{s,j\, ; \, t,\, k} (e_{u,l}) = e_{s,j}\, \delta _{t,u} \, \delta _{k,l}\) for all \(u = 0, 1\) and \(l = 1, \ldots , d_u \,\). These form a basis of \(\mathrm {End}(V)\), and by Lemma 2.6 the operators

$$\begin{aligned} X_{s j, \, t k}^{(0)} := E_{s , j\,; \, t ,\, k} \oplus (- E_{s,j\,;\,t,\,k})^\mathrm {st} \end{aligned}$$

form a basis of \(\mathfrak {g}^{(0)}\). Similarly, let bases of \(\mathrm {Hom}(V^*,V)\) and \(\mathrm {Hom}(V,V^*)\) be defined by

$$\begin{aligned} F_{s,j\, ; \,t,\,k} (f_{u,l}) = e_{s,j} \, \delta _{t,u} \, \delta _{k,l}, \qquad \tilde{F}_{s,j\,;\,t,\,k} (e_{u,l}) = f_{s,j} \,\delta _{t,u}\, \delta _{k,l}, \end{aligned}$$

for index pairs in the appropriate range. Then by Lemma 2.5 and Eqs. (2.3, 2.4) the subalgebras \(\mathfrak {g}^{(-2)}\) and \(\mathfrak {g}^{(2)}\) are generated by the sets of operators

$$\begin{aligned}&X_{s j, \, t k}^{(-2)} := F_{s,j\,;\,t,\,k} + F_{t,\,k\,;\,s,j} \, (-1)^{|s| |t|}, \\&X_{s j, \, t k}^{(2)} := \tilde{F}_{s,j\,;\,t,\,k} + \tilde{F}_{t,\,k\,;\,s,j} \, (-1)^{|s| |t| + |s| + |t|}\;. \end{aligned}$$

Since \(\mathfrak {osp}_0 \simeq \mathfrak {o}(W_1) \oplus \mathfrak {sp}(W_0)\), a Cartan subalgebra of \(\mathfrak {osp}\) is the direct sum of a Cartan subalgebra of \(\mathfrak {o}(W_1)\) and a Cartan subalgebra of \(\mathfrak {sp}(W_0)\). Letting \(\mathfrak {h}\) be the span of the diagonal operators

$$\begin{aligned} H_{s j} := X_{s j ,\, s j}^{(0)}\quad (s=0,1;\, j= 1,\ldots ,d_s)\;, \end{aligned}$$

one has that \(\mathfrak {h}\) is a Cartan subalgebra of \(\mathfrak {osp} \,\). Indeed, if \(\{ \vartheta _{s j} \}\) denotes the basis of \(\mathfrak {h}^*\) dual to \(\{ H_{s j} \}\,\), inspection of the adjoint action of \(\mathfrak {h}\) on \(\mathfrak {osp}\) gives the following result.

Lemma 2.7

The operators \(X_{s j\, , \, t k}^{(m)}\) are eigenvectors of \(\mathrm {ad}(H)\) for all \(H \in \mathfrak {h}:\)

$$\begin{aligned}{}[H\, ,X^{(m)}_{s j\, ,\, t k}] = {\left\{ \begin{array}{ll} (\vartheta _{s j} - \vartheta _{t k})(H) \, X^{(m)}_{s j\, ,\, t k} &{} m = 0, \\ (\vartheta _{s j} + \vartheta _{t k})(H) \, X^{(m)}_{s j\, , \, t k} &{} m = -2, \\ (- \vartheta _{s j} - \vartheta _{t k})(H)\, X^{(m)}_{s j\,, \,t k} &{} m = 2\;. \end{array}\right. } \end{aligned}$$

A root of a Lie superalgebra \(\mathfrak {g}\) is called even if its root space is in \(\mathfrak {g}_0\,\), it is called odd if its root space is in \(\mathfrak {g}_1\,\). We denote by \(\Delta _0\) and \(\Delta _1\) the set of even roots and the set of odd roots, respectively. For \(\mathfrak {g} = \mathfrak {osp}\) we have

$$\begin{aligned} \Delta _0 =\{ \pm \vartheta _{1j} \pm \vartheta _{1k}\, , \, \pm \vartheta _{0 j} \pm \vartheta _{0 l} \mid j < k , \, j\le l\} , \quad \Delta _1 = \{ \pm \vartheta _{1 j} \pm \vartheta _{0 k} \}. \end{aligned}$$

2.2.2 Casimir elements

As before, let \(\mathfrak {g} = \mathfrak {g}_0 \oplus \mathfrak {g}_1\) be a Lie superalgebra, and let \(\mathsf {U}(\mathfrak {g}) = \cup _{n = 0}^\infty \mathsf {U}_n (\mathfrak {g})\) be its universal enveloping algebra. Denote the symmetric algebra of \(\mathfrak {g}_0\) by \(\mathrm {S}(\mathfrak {g}_0)\) and the exterior algebra of \(\mathfrak {g}_1\) by \(\wedge (\mathfrak {g}_1)\). The Poincaré–Birkhoff–Witt theorem for Lie superalgebras states that for each n there is a bijective correspondence

$$\begin{aligned} \mathsf {U}_n (\mathfrak {g}) / \mathsf {U}_{n-1} (\mathfrak {g}) \mathop {\rightarrow }\limits ^{\sim } \sum \nolimits _{k +l = n} \wedge ^k( \mathfrak {g}_1 ) \otimes \mathrm {S}^l (\mathfrak {g}_0). \end{aligned}$$

The collection of inverse maps lift to a vector-space isomorphism,

$$\begin{aligned} \wedge (\mathfrak {g}_1) \otimes \mathrm {S}(\mathfrak {g}_0) \mathop {\rightarrow }\limits ^{\sim } \mathsf {U}(\mathfrak {g}), \end{aligned}$$

called the super-symmetrization mapping. In other words, given a homogeneous basis \(\{ e_1, \ldots , e_d \}\) of \(\mathfrak {g} \,\), each element \(x \in \mathsf {U}(\mathfrak {g})\) can be uniquely represented in the form \(x = \sum _n \sum _{i_1, \ldots , \, i_n} x_{i_1, \ldots , \, i_n}\, e_{i_1} \cdots \, e_{i_n}\) with super-symmetrized coefficients, i.e.,

$$\begin{aligned} x_{i_1,\ldots ,\, i_l,\, i_{l+1}, \ldots , \, i_n} = (-1)^{|e_{i_l}| |e_{i_{l+1}}|} x_{i_1,\, \ldots ,\, i_{l+1},\, i_l,\, \ldots ,\, i_n} \quad (1 \le l < n). \end{aligned}$$

The isomorphism \(\wedge (\mathfrak {g}_1) \otimes \mathrm {S} (\mathfrak {g}_0) \simeq \mathsf {U}(\mathfrak {g})\) gives \(\mathsf {U} (\mathfrak {g})\) a \(\mathbb {Z}\)-grading (by the degree n).

Now recall that \(\mathsf {U}(\mathfrak {g})\) comes with a canonical bracket operation, the supercommutator \([\, , \,] : \, \mathsf {U} (\mathfrak {g}) \times \mathsf {U}(\mathfrak {g}) \rightarrow \mathsf {U} (\mathfrak {g})\). An element \(X \in \mathsf {U}(\mathfrak {g})\) is said to lie in the center of \(\mathsf {U} (\mathfrak {g})\), and is called a Casimir element, iff \([X,Y]= 0\) for all \(Y \in \mathsf {U} (\mathfrak {g})\). By the formula (2.2), a necessary and sufficient condition for that is \([X,Y] = 0\) for all \(Y \in \mathfrak {g}\,\).

In the case of \(\mathfrak {g} = \mathfrak {osp}\,\), for every \(\ell \in \mathbb {N}\) there is a Casimir element \(I_\ell \) of degree \(2\ell \), which is constructed as follows. Consider the bilinear form \(B : \, \mathfrak {osp} \times \mathfrak {osp} \rightarrow \mathbb {K}\) given by the supertrace (in some representation), \(B(X,Y) : = \mathrm {STr}\, (XY)\). Recall that this form is \(\mathrm {ad}\)-invariant, which is to say that \(B([X,Y],Z) = B(X,[Y,Z])\) for all \(X, Y, Z \in \mathfrak {g}\,\).

Taking the supertrace in the fundamental representation of \(\mathfrak {osp} \,\), the form B is non-degenerate, and therefore, if \(e_1,\, \ldots , \, e_d\) is a homogeneous basis of \(\mathfrak {osp}\,\), there is another homogeneous basis \(\widetilde{e}_1 , \, \ldots , \, \widetilde{e}_d\) of \(\mathfrak {osp}\) so that \(B(\widetilde{e}_i \, , \, e_j) = \delta _{ij}\,\). Note \(|\widetilde{e}_i | = |e_i|\) and put

$$\begin{aligned} I_\ell := \sum _{i_1,\,\ldots ,\,i_{2\ell }=1}^d \widetilde{e}_ {i_1}\cdots \, \widetilde{e}_{i_{2\ell }}\,\mathrm {STr}\, (e_{i_{2\ell }}\cdots \, e_{i_1})\in \mathsf {U}(\mathfrak {osp}). \end{aligned}$$
(2.5)

(Notice that, in view of the remark following Lemma 2.1, there is no point in making the same construction with an odd number of factors.)

Lemma 2.8

For all \(\ell \in \mathbb N\) the element \(I_\ell \) is Casimir, and \(|I_\ell | = 0\).

Proof

By specializing the formula (2.2) to the present case,

$$\begin{aligned}{}[I_\ell \, , \,X] = \sum \sum _{k=1}^{2\ell } (-1)^{|X| (|\widetilde{e}_{i_{k+1}}| + \ldots +|\widetilde{e}_{i_{2\ell }}|)} \widetilde{e}_{i_1} \cdots \, \widetilde{e}_{i_{k-1}} \, [ \widetilde{e}_{i_k} , X] \, \widetilde{e}_{i_{k+1}} \cdots \, \widetilde{e}_{i_{2\ell }}\,\mathrm {STr}\,(e_{i_{2\ell }}\cdots \, e_{i_1})\;. \end{aligned}$$

Now if \([\widetilde{e}_i \, , X] = \sum _j X_{ij}\, \widetilde{e}_j\) then from \(\mathrm {ad}\)-invariance, \(B([\widetilde{e}_i \, , X], e_j) = X_{ij} = B(\widetilde{e}_i \, , [X,e_j])\), one has \([X,e_j] = \sum _i e_i\, X_{ij}\,\). Using this relation to transfer the \(\mathrm {ad}(X)\)-action from \(\widetilde{e}_{i_k}\) to \(e_{i_k}\,\), and reading the formula (2.2) backwards, one obtains

$$\begin{aligned}{}[I_\ell \,,\,X] = \sum \widetilde{e}_{i_1}\cdots \,\widetilde{e}_{ i_{2\ell }}\, \mathrm {STr}\,([X,e_{i_{2\ell }}\cdots \, e_{i_1}]). \end{aligned}$$

Since the supertrace of any bracket vanishes, one concludes that \([I_\ell \, , X ] = 0\).

The other statement, \(|I_\ell | = 0\,\), follows from \(|\widetilde{e}_i | = |e_i|\), the additivity of the \(\mathbb {Z}_2\)-degree and the fact that \(\mathrm {STr}\,(a) = 0\) for any odd element \(a \in \mathsf {U}(\mathfrak {g})\). \(\square \)

We now describe a useful property enjoyed by the Casimir elements \(I_\ell \) of \(\mathfrak {osp}(V \oplus V^*)\) in the special case of isomorphic components \(V_0 \simeq V_1\,\). Recalling the notation of Sect. 2.2.1, let \(\partial := \sum _j X^{(0)}_{0j,1j}\) and \(\widetilde{\partial } := - \sum _j X_{1j ,0j}^{(0)}\,\). These are odd elements of \(\mathfrak {osp}\,\). Notice that the bracket \({C} := [\partial , \widetilde{\partial }] = - \sum _{s,j} H_{s j}\) is in the Cartan algebra of \(\mathfrak {osp}\,\). From \([\partial , \partial ] = 2 \, \partial ^2 = 0\) and the Jacobi identity one infers that

$$\begin{aligned}{}[\partial ,{C}] = [[\partial ,\partial ],\widetilde{\partial }] - [\partial , [\partial , \widetilde{\partial }]] = - [\partial , {C}] = 0. \end{aligned}$$

By the same argument, \([\widetilde{\partial },{C}] = 0\,\). One also sees that \({C}^2 = \mathrm {Id}\,\).

Now define \(F_\ell \) to be the following odd element of \(\mathsf {U}(\mathfrak {osp})\):

$$\begin{aligned} F_\ell = - \sum _{i_1,\,\ldots ,\,i_{2\ell }=1}^d \widetilde{e}_{i_1} \cdots \, \widetilde{e}_{i_{2\ell }}\, \mathrm {STr}\,\left( e_{i_{2\ell }} \cdots \, e_{i_1} \widetilde{\partial } {C}\right). \end{aligned}$$

Lemma 2.9

Let \(\mathfrak {osp}(V \oplus V^*)\) be the orthosymplectic Lie superalgebra for a \(\mathbb {Z}_2\) -graded vector space V with isomorphic components \(V_0 \simeq V_1\,\) . Then for all \(\ell \in \mathbb {N}\) the Casimir element \(I_\ell \) is expressible as a bracket: \(I_\ell = [\partial , F_\ell ]\,\).

Proof

By the same argument as in the proof of Lemma 2.8,

$$\begin{aligned}{}[\partial , F_\ell ] = - \sum \widetilde{e}_{i_1}\cdots \, \widetilde{e}_{i_{2\ell }}\,\mathrm {STr}\,([\partial ,e_{i_{2\ell }} \cdots e_{i_1}] \, \widetilde{\partial } {C}). \end{aligned}$$

Using the relations \([\partial , {C}] = 0\) and \({C}^2 = \mathrm {Id}\,\), one has for any \(a\in \mathsf {U} (\mathfrak {osp})\) that

$$\begin{aligned} - \mathrm {STr}\, ([\partial , \, a]\, \widetilde{\partial } {C}) = \mathrm {STr}\, (\widetilde{\partial } {C} \, [\partial , \, a]) = \mathrm {STr}\, ([\widetilde{\partial } , \partial ] {C} \, a) = \mathrm {STr}\,({C}^2 a) = \mathrm {STr}\, (a), \end{aligned}$$

where the second equality sign is from \(\mathrm {STr}\, (c,[b,a]) = \mathrm {STr}\, ([c,b]\, a)\). The statement of the lemma now follows on setting \(a = e_{i_{2\ell }} \cdots \, e_{i_1}\,\). \(\square \)

As we shall see in Sect. 5.1.3, Lemma 2.9 has the drastic consequence that all \(\mathfrak {osp}\)-Casimir elements \(I_\ell \) are zero in a certain class of representations of \(\mathfrak {osp}(V \oplus V^*)\) for \(V_0 \simeq V_1\,\).

2.3 Howe pairs in \(\mathfrak {osp}(W)\)

In the present context, a pair \((\mathfrak {h},\mathfrak {h}')\) of subalgebras \(\mathfrak {h}, \mathfrak {h}' \subset \mathfrak {g}\) of a Lie superalgebra \(\mathfrak {g}\) is called a dual pair whenever \(\mathfrak {h}'\) is the centralizer of \(\mathfrak {h}\) in \(\mathfrak {g}\) and vice versa. In this subsection, let \(\mathbb {K} = \mathbb {C}\,\).

Given a \(\mathbb {Z}_2\)-graded complex vector space \(U = U_0 \oplus U_1\) we let \(V := U \otimes \mathbb {C}^N\), where \(\mathbb {C}^N\) is equipped with the standard representation of \(\mathrm {GL} (\mathbb {C}^N)\), \(\mathrm {O}(\mathbb {C}^N)\,\), or \(\mathrm {Sp} (\mathbb {C}^N)\,\), as the case may be. As a result, the Lie algebra \(\mathfrak {k}\) of whichever group is represented on \(\mathbb {C}^N\) is embedded in \(\mathfrak {osp}(V \oplus V^*)\). We will now describe the dual pairs \((\mathfrak {h},\mathfrak {k})\) in \(\mathfrak {osp} (W)\) for \(W = V \oplus V^*\). These are known as dual pairs in the sense of R. Howe.

Let us begin by recalling that for any representation \(\rho :\, K \rightarrow \mathrm {GL}(E)\) of a group K on a vector space \(E\,\), the dual representation \(\rho ^*: \, K \rightarrow \mathrm {GL}(E^*)\) on the linear forms on E is given by \((\rho (k)\varphi )(x) = \varphi (\rho (k)^{-1}x)\). By this token, every representation \(\rho :\, K \rightarrow \mathrm {GL}(\mathbb {C}^N)\) induces a representation \(\rho _W = (\mathrm {Id} \otimes \rho ) \times (\mathrm {Id} \otimes \rho ^*)\) of K on \(W = V \oplus V^*\).

Lemma 2.10

Let \(\rho : \, K \rightarrow \mathrm {GL}(\mathbb {C}^N)\) be any representation of a Lie group \(K\,\) . If \(V = U \otimes \mathbb {C}^N\) for a \(\mathbb {Z}_2\) -graded complex vector space \(U = U_0 \oplus U_1\,\) , the induced representation \({\rho _W}_*(\mathfrak {k})\) of the Lie algebra \(\mathfrak {k}\) of K on \(W = V \oplus V^*\) is a subalgebra of \(\mathfrak {osp}(W)_0\,\).

Proof

The K-action on \(\mathbb {C}^N \otimes (\mathbb {C}^N)^*\) by \(z \otimes \zeta \mapsto \rho (k)z \otimes \rho ^*(k)\zeta \) preserves the canonical pairing \(z \otimes \zeta \mapsto \zeta (z)\) between \(\mathbb {C}^N\) and \((\mathbb {C}^N)^*\). Consequently, the K-action on \(V \otimes V^*\) by \((\mathrm {Id} \otimes \rho ) \otimes (\mathrm {Id} \otimes \rho ^*)\) preserves the canonical pairing \(V \otimes V^*\rightarrow \mathbb {C}\,\). Since the orthosymplectic form \(Q : \, W \times W \rightarrow \mathbb {C}\) uses nothing but that pairing, it follows that

$$\begin{aligned} Q( \rho _W(k) w \, , \, \rho _W(k) w^\prime ) = Q(w,w^\prime ) \quad (\text {for all } w,w^\prime \in W)\;. \end{aligned}$$

Passing to the Lie algebra level one obtains \({\rho _W}_*( \mathfrak {k}) \subset \mathfrak {osp}(W)\). The operator \(\rho _W(k)\) preserves the \(\mathbb {Z}_2\)-grading of W; therefore one actually has \({\rho _W}_*( \mathfrak {k}) \subset \mathfrak {osp}(W)_0\,\). \(\square \)

Let us now assume that the complex Lie group K is defined by a non-degenerate bilinear form \(B :\, \mathbb {C}^N \times \mathbb {C}^N \rightarrow \mathbb {C}\) in the sense that

$$\begin{aligned} K = \{ k \in \mathrm {GL}(\mathbb {C}^N) \mid \forall z\, , z^\prime \in \mathbb {C}^N : \, B(k z \, ,\, k z^\prime ) = B(z\,,z^\prime )\}\;. \end{aligned}$$

We then have a canonical isomorphism \(\psi : \, \mathbb {C}^N \rightarrow (\mathbb {C}^N)^*\) by \(z \mapsto B(z\, ,\,)\), and an isomorphism \(\Psi : \, (U \oplus U^*) \otimes \mathbb {C}^N \rightarrow W\) by \((u + \varphi ) \otimes z \mapsto u \otimes z + \varphi \otimes \psi (z)\).

Lemma 2.11

\(\rho _W(k) = \Psi \circ (\mathrm {Id} \otimes k) \circ \Psi ^{-1}\) for all \(k \in K\,\).

Proof

If \(u \in U\), \(\varphi \in U^*\), and \(z \in \mathbb {C}^N\), then by the definition of \(\Psi \) and \(\rho _W(k)\),

$$\begin{aligned} \rho _W(k) \Psi ((u+\varphi ) \otimes z) = u \otimes k z + \varphi \otimes \psi (z) k^{-1}. \end{aligned}$$

Since B is K-invariant, one has \(\psi (z) k^{-1} = \psi (kz)\), and therefore

$$\begin{aligned} u \otimes k z + \varphi \otimes \psi (z) k^{-1} = u \otimes k z + \varphi \otimes \psi (k z) = \Psi ((\mathrm {Id} \otimes k)((u+\varphi )\otimes z)). \end{aligned}$$

Thus \(\rho _W(k) \circ \Psi = \Psi \circ (\mathrm {Id} \otimes k)\). \(\square \)

Let us now examine what happens to the orthosymplectic form Q on W when it is pulled back by the isomorphism \(\Psi \) to a bilinear form \(\Psi ^*Q\) on \((U\oplus U^*) \otimes \mathbb {C}^N:\)

$$\begin{aligned} \Psi ^*Q((u + \varphi ) \otimes z\, , (u' + \varphi ') \otimes z') = \varphi '(u)\, \psi (z')(z) - (-1)^{|u'||\varphi |} \varphi (u') \, \psi (z)(z'). \end{aligned}$$

By definition, \(\psi (z)(z^\prime ) = B(z\, ,z^\prime )\), and writing \(B(z\, , z^\prime ) = (-1)^\delta B(z^\prime ,z)\) where \(\delta = 0\) if B is symmetric and \(\delta = 1\) if B is alternating, we obtain

$$\begin{aligned} \Psi ^*Q((u + \varphi ) \otimes z \, , (u^\prime + \varphi ^\prime ) \otimes z^\prime ) = (\varphi ^\prime (u) - (-1)^{|u'||\varphi | + \delta } \varphi (u')) B(z',z). \end{aligned}$$
(2.6)

In view of this, let \(\widetilde{U}\) denote the vector space \(U = U_0 \oplus U_1\) with the twisted \(\mathbb {Z}_2\)-grading, i.e., \(\widetilde{U}_s := U_{s+1}\) (\(s \in \mathbb {Z}_2\)). Moreover, notice that \(\Psi \) determines an embedding

$$\begin{aligned} \mathrm {End}(U \oplus U^*) \otimes \mathrm {End}(\mathbb {C}^N) \rightarrow \mathrm {End}(W), \quad X \otimes k \mapsto \Psi \circ (X \otimes k) \circ \Psi ^{-1}, \end{aligned}$$

whose restriction to \(\mathrm {End}(U \oplus U^*) \otimes \{ \mathrm {Id} \} \rightarrow \mathrm {End}(W)\) is an injective homomorphism.

In the following we often write \(\mathrm {O}(\mathbb {C}^N) \equiv \mathrm {O}_N\) and \(\mathrm {Sp}(\mathbb {C}^N) \equiv \mathrm {Sp}_N\) for short.

Corollary 2.2

For \(K = \mathrm {O}_N\) and \(K = \mathrm {Sp}_N\,\) , the map \(X \mapsto \Psi \circ (X \otimes \mathrm {Id}) \circ \Psi ^{-1}\) defines a Lie superalgebra embedding into \(\mathfrak {osp}(W)\) of \(\, \mathfrak {osp}(U \oplus U^*)\) resp. \(\mathfrak {osp} (\widetilde{U} \oplus \widetilde{U}^*)\).

Proof

For \(K = \mathrm {O}_N\) the bilinear form B of \(\mathbb {C}^N\) is symmetric and the bilinear form Q of W pulls back—see (2.6)—to the standard orthosymplectic form of \(U \oplus U^*\).

For \(K = \mathrm {Sp}_N\,\), the form B is alternating. Its pullback, the orthosymplectic form of \(U \oplus U^*\) twisted by the sign factor \((-1)^\delta \), is restored to standard form by switching to the \(\mathbb {Z}_2\)-graded vector space \(\widetilde{U} \oplus \widetilde{U}^*\) with the twisted \(\mathbb {Z}_2\)-grading. \(\square \)

To go further, we need a statement concerning \(\mathrm {Hom}_G(V_1 ,V_2)\), the space of G-equivariant homomorphisms between two modules \(V_1\) and \(V_2\) for a group \(G\,\).

Lemma 2.12

Let \(X_1\, , X_2 \, , Y_1\, , Y_2\) be finite-dimensional vector spaces all of which are representation spaces for a group \(G\,\) . If the G -action on \(X_1\) and \(X_2\) is trivial, then

$$\begin{aligned} \mathrm {Hom}_G(X_1 \otimes Y_1\, , X_2 \otimes Y_2) \simeq \mathrm {Hom}(X_1\, ,X_2) \otimes \mathrm {Hom}_G(Y_1\, ,Y_2). \end{aligned}$$

Proof

 \(\mathrm {Hom}(X_1 \otimes Y_1\, , X_2 \otimes Y_2)\) is canonically isomorphic to \(X_1^*\otimes Y_1^*\otimes X_2 \otimes Y_2\) as a G-representation space, with G-equivariant maps corresponding to G-invariant tensors. Since the G-action on \(X_1^*\otimes X_2^{}\) is trivial, one sees that \(\mathrm {Hom}_G(X_1 \otimes Y_1\, , X_2 \otimes Y_2)\) is isomorphic to the tensor product of \(X_1^*\otimes X_2^{} \simeq \mathrm {Hom}(X_1, X_2)\) with the space of G-invariants in \(Y_1^*\otimes Y_2^{} \,\). The latter in turn is isomorphic to \(\mathrm {Hom}_G(Y_1, Y_2)\). \(\square \)

Proposition 2.1

Writing \(\mathfrak {g}_N \equiv \mathfrak {g}(\mathbb {C}^N)\) for \(\mathfrak {g} = \mathfrak {gl}\,\), \(\mathfrak {o}\,\), \(\mathfrak {sp}\,\), the following pairs are dual pairs in \(\mathfrak {osp}(W):\) \((\mathfrak {gl}(U),\mathfrak {gl}_N)\), \((\mathfrak {osp}(U\oplus U^*), \mathfrak {o}_N)\), \((\mathfrak {osp}(\widetilde{U}\oplus \widetilde{U}^*), \mathfrak {sp}_N)\).

Proof

Here we calculate the centralizer of \(\mathfrak {k}\) in \(\mathfrak {osp}(W)\) for each of the three cases \(\mathfrak {k} = \mathfrak {gl}_N\,\), \(\mathfrak {o}_N\,\), \(\mathfrak {sp}_N\) and refer the reader to [10] for the remaining details.

Since both \(V \subset W\) and \(V^* \subset W\) are K-invariant subspaces, \(\mathrm {End}_K(W)\) decomposes as

$$\begin{aligned} \mathrm {End}_K(W) = \mathrm {End}_K(V) \oplus \mathrm {Hom}_K(V^*,V) \oplus \mathrm {Hom}_K(V,V^*) \oplus \mathrm {End}_K(V^*). \end{aligned}$$

By Schur’s lemma, \(\mathrm {End}_K(\mathbb {C}^N) \simeq \mathbb {C}\,\), and therefore Lemma 2.12 implies

$$\begin{aligned} \mathrm {End}_K(V) = \mathrm {End}_K(U \otimes \mathbb {C}^N) \simeq \mathrm {End}(U) \otimes \mathrm {End}_K(\mathbb {C}^N) = \mathrm {End}(U)\;. \end{aligned}$$

By the same reasoning, \(\mathrm {End}_K(V^*) = \mathrm {End}(U^*)\). Applying Lemma 2.12 to the two remaining summands, we obtain

$$\begin{aligned} \mathrm {Hom}_K(V,V^*) \simeq \mathrm {Hom}(U,U^*) \otimes \mathrm {Hom}_K(\mathbb {C}^N,{\mathbb {C}^N}^*), \end{aligned}$$

plus the same statement where each vector space is replaced by its dual.

If \(K = \mathrm {GL}(\mathbb {C}^N) \equiv \mathrm {GL}_N\,\), then \(\mathrm {Hom}_K(\mathbb {C}^N ,{\mathbb {C}^N}^*) = \mathrm {Hom}_K ({\mathbb {C}^N}^* , \mathbb {C}^N) = \{ 0\}\). Hence,

$$\begin{aligned} \Phi : \,\, \mathrm {End}(U) \oplus \mathrm {End}(U^*) \rightarrow \mathrm {End}_{\mathrm {GL}_N}(W)\;, \quad X \oplus Y \mapsto (X \otimes \mathrm {Id}) \times (Y \otimes \mathrm {Id})\;, \end{aligned}$$

for \(W = U \otimes \mathbb {C}^N \oplus U^*\otimes {\mathbb {C}^N}^*\) is an isomorphism. This means that the centralizer of \(\mathfrak {gl}_N\) in \(\mathfrak {osp}(W)\) is the intersection \(\Phi (\mathrm {End}(U)\oplus \mathrm {End}(U^*)) \cap \mathfrak {osp}(W)\), which can be identified with \(\mathrm {End}(U) = \mathfrak {gl}(U)\). Thus we have the first dual pair, \((\mathfrak {gl} (U), \mathfrak {gl}_N)\).

In the case of \(K = \mathrm {O}_N\,\), the discussion is shortened by recalling Lemma 2.11 and the K-equivariant isomorphism \(\Psi : \, (U \oplus U^*) \otimes \mathbb {C}^N \rightarrow W\). By Schur’s lemma, these imply \(\mathrm {End}_K(W) \simeq \mathrm {End}(U \oplus U^*)\). From Corollary 2.2 it then follows that the intersection \(\mathfrak {osp}(W) \cap \mathrm {End}_K(W)\) is isomorphic as a Lie superalgebra to \(\mathfrak {osp}(U \oplus U^*)\). Passing to the Lie algebra level for K, we get the second dual pair, \((\mathfrak {osp}(U \oplus U^*), \mathfrak {o}_N)\).

Finally, if \(K = \mathrm {Sp}_N\,\), the situation is identical except that Corollary 2.2 compels us to switch to the \(\mathbb {Z}_2\)-twisted structure of orthosymplectic Lie superalgebra in \(\mathrm {End}_K(W) \simeq \mathrm {End}(U \oplus U^*)\). This gives us the third dual pair, \((\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*), \mathfrak {sp}_N)\). \(\square \)

2.4 Clifford–Weyl algebra \(\mathfrak {q}(W)\)

Let \(\mathbb {K} = \mathbb {C}\) or \(\mathbb {K} = \mathbb {R}\) (in this subsection the choice of number field again is immaterial) and recall from Example 2.3 the definition of the Jordan–Heisenberg Lie superalgebra \(\widetilde{W} = W \oplus \mathbb {K}\,\), where \(W = W_0 \oplus W_1\) is a \(\mathbb {Z}_2\)-graded vector space with components \(W_1^{} = V_1^{} \oplus V_1^*\) and \(W_0^{} = V_0^{} \oplus V_0^*\). The universal enveloping algebra of the Jordan–Heisenberg Lie superalgebra is called the Clifford–Weyl algebra (or quantum algebra). We denote it by \(\mathfrak {q}(W) \equiv \mathsf {U}(\widetilde{W})\).

Equivalently, one defines the Clifford–Weyl algebra \(\mathfrak {q}(W)\) as the associative algebra generated by \(\widetilde{W} = W \oplus \mathbb {K}\) subject to the following relations for all \(w, w^\prime \in W_0 \cup W_1:\)

$$\begin{aligned} w w' - (-1)^{|w||w'|} w' w = Q(w , w'). \end{aligned}$$

In particular, \(w_0\, w_1 = w_1\, w_0\) for all \(w_0 \in W_0\) and \(w_1 \in W_1\,\). Reordering by this commutation relation defines an isomorphism of associative algebras \(\mathfrak {q}(W) \simeq \mathfrak {c}(W_1) \otimes \mathfrak {w}(W_0)\), where the Clifford algebra \(\mathfrak {c}(W_1)\) is generated by \(W_1 \oplus \mathbb {K}\) with the relations \(w w^\prime + w^\prime w = S(w,w^\prime )\) for \(w,w^\prime \in W_1\,\), and the Weyl algebra \(\mathfrak {w}(W_0)\) is generated by \(W_0 \oplus \mathbb {K}\) with the relations \(w w^\prime - w^\prime w = A(w,w^\prime )\) for \(w, w^\prime \in W_0\,\).

As a universal enveloping algebra the Clifford–Weyl algebra \(\mathfrak {q} (W)\) is filtered,

$$\begin{aligned} \mathfrak {q}_0(W) := \mathbb {K} \subset \mathfrak {q}_1(W) := W \oplus \mathbb {K} \subset \ldots \subset \mathfrak {q}_n(W) \ldots, \end{aligned}$$

and it inherits from the Jordan–Heisenberg algebra \(\widetilde{W}\) a canonical \(\mathbb {Z}_2\)-grading and a canonical structure of Lie superalgebra by the supercommutator—see Sect. 2.1.2 for the definitions. The next statement is a sharpened version of Lemma 2.2.

Lemma 2.13

\([\mathfrak {q}_n(W) , \mathfrak {q}_{n^\prime }(W)] \subset \mathfrak {q}_{n + n^\prime - 2}(W)\).

Proof

Lemma 2.2 asserts the commutation relation \([\mathsf {U}_n (\mathfrak {g}), \mathsf {U}_{n^\prime } (\mathfrak {g}) ] \subset \mathsf {U}_{n + n^\prime -1} (\mathfrak {g})\) for the general case of a Lie superalgebra \(\mathfrak {g}\) with bracket \([ \mathfrak {g}, \mathfrak {g}] \subset \mathfrak {g}\,\). For the specific case at hand, where the fundamental bracket \([W , W] \subset \mathbb {K}\) has zero component in W, the degree \(n + n^\prime - 1\) is lowered to \(n + n^\prime -2\) by the very argument proving that lemma. \(\square \)

It now follows that each of the subspaces \(\mathfrak {q}_n(W)\) for \(n \le 2\) is a Lie superalgebra. Since \([\mathfrak {q}_2(W), \mathfrak {q}_1(W)] \subset \mathfrak {q}_1(W)\), the quotient space \(\mathfrak {q}_2(W)/\mathfrak {q}_1(W)\) is also a Lie superalgebra. By the Poincaré–Birkhoff–Witt theorem, there exists a vector-space isomorphism

$$\begin{aligned} \mathfrak {q}_2(W) / \mathfrak {q}_1(W) \mathop {\rightarrow }\limits ^{\sim } \mathfrak {s}, \end{aligned}$$

sending \(\mathfrak {q}_2(W) / \mathfrak {q}_1(W)\) to \(\mathfrak {s}\,\), the space of super-symmetrized degree-two elements in \(\mathfrak {q}_2 (W)\). Hence \(\mathfrak {q}_2(W)\) has a direct-sum decomposition \(\mathfrak {q}_2(W) = \mathfrak {q}_1(W) \oplus \mathfrak {s}\,\).

If \(\{ e_i \}\) is a homogeneous basis of W, every \(a \in \mathfrak {s}\) is uniquely expressed as

$$\begin{aligned} a = \sum \nolimits _{i,j} a_{ij}\, e_i \, e_j, \quad a_{ij} = (-1)^{|e_i| |e_j|} a_{ji}. \end{aligned}$$
(2.7)

By adding and subtracting terms,

$$\begin{aligned} 2 w w^\prime = (w w^\prime + (-1)^{|w||w^\prime |} w^\prime w) + (w w^\prime - (-1)^{|w||w^\prime |} w^\prime w), \end{aligned}$$

one sees that the product \(w w^\prime \) for \(w,w^\prime \in W\) has scalar part \((w w^\prime )_\mathbb {K} = \frac{1}{2}[w,w^\prime ] = \frac{1}{2}Q(w,w^\prime )\) with respect to the decomposition \(\mathfrak {q}_2(W) = \mathbb {K} \oplus W \oplus \mathfrak {s}\,\).

Lemma 2.14

\([\mathfrak {s} , \mathfrak {s}] \subset \mathfrak {s}\,\).

Proof

From the definition of \(\mathfrak {s}\) and \([W,W] \subset \mathbb {K}\) it is clear that \([\mathfrak {s}, \mathfrak {s}] \subset \mathbb {K} \oplus \mathfrak {s}\,\). The statement to be proved, then, is that [ab] for \(a,\, b \in \mathfrak {s}\) has zero scalar part.

By the linearity of the supercommutator, it suffices to consider a single term of the sum (2.7). Thus we put \(a = w w^\prime + (-1)^{|w| |w^\prime |} w^\prime w\), and have

$$\begin{aligned} {\textstyle {\frac{1}{2}}} [a,b] = [w w^\prime ,b] = w\, [w^\prime ,b] + [w,b] \, w^\prime (-1)^{|w^\prime | |b|}. \end{aligned}$$

Now we compute the scalar part of the right-hand side. Using the Jacobi identity for the Lie superalgebra \(\mathfrak {q}(W)\) we obtain

$$\begin{aligned}{}[a,b]_\mathbb {K} = [w,[w^\prime ,b]] + [[w,b],w^\prime ] (-1)^{|w^\prime | |b|} = [[w,w^\prime ],b]. \end{aligned}$$

The last expression vanishes because \([w,w^\prime ] \subset \mathbb {K}\) lies in the center of \(\mathfrak {q}(W)\). \(\square \)

2.5 \(\mathfrak {osp}(W)\) inside \(\mathfrak {q}(W)\)

As a subspace of \(\mathfrak {q}(W)\) which closes w.r.t. the supercommutator \([\, , \, ]\), \(\mathfrak {s}\) is a Lie superalgebra. Now from Lemma 2.13 and the Jacobi identity for \(\mathfrak {q}(W)\), one sees that \(\mathfrak {s} \subset \mathfrak {q}_2(W)\) acts on each of the quotient spaces \(\mathfrak {q}_n(W) / \mathfrak {q}_{n - 1}(W)\) for \(n \ge 1\) by \(a \mapsto [a,\,\,]\,\). In particular, \(\mathfrak {s}\) acts on \(\mathfrak {q}_1(W) / \mathfrak {q}_0(W) = W\) by \(a \mapsto [a,\,\,]\), which defines a homomorphism of Lie superalgebras

$$\begin{aligned} \tau : \, \mathfrak {s} \rightarrow \mathfrak {gl}(W), \quad a \mapsto \tau (a) = [a , \,\, ]. \end{aligned}$$

The mapping \(\tau \) is actually into \(\mathfrak {osp}(W) \subset \mathfrak {gl}(W)\). Indeed, for \(w,w^\prime \in W\) one has

$$\begin{aligned} Q(\tau (a)w,w')+(-1)^{|\tau (a)||w|}Q(w,\tau (a)w') = [[a,w],w']+(-1)^{|a||w|}[w,[a,w']], \end{aligned}$$

and since \([a,[w,w^\prime ]] = 0\,\), this vanishes by the Jacobi identity.

Lemma 2.15

The map \(\tau :\, \mathfrak {s} \rightarrow \mathfrak {osp}(W)\) is an isomorphism of Lie superalgebras.

Proof

Being a homomorphism of Lie superalgebras, the linear mapping \(\tau \) is an isomorphism of such algebras if it is bijective. We first show that \(\tau \) is injective. So, let \(a \in \mathfrak {s}\) be any element of the kernel of \(\tau \,\). The equation \(\tau (a) = 0\) means that \([[a,w],w^\prime ] = [\tau (a)w,w^\prime ]\) vanishes for all \(w, w^\prime \in W\). To fathom the consequences of this, let \(\{e_i\}\) and \(\{ \widetilde{e}_i \}\) be two homogeneous bases of W so that \(Q(e_i\,,\widetilde{e}_j) = \delta _{ij} \,\). Using that \(a \in \mathfrak {s}\) has a uniquely determined expansion \(a = \sum a_{ij}\, e_i \, e_j\) with super-symmetric coefficients \(a_{ij} = (-1)^{|e_i| |e_j|} a_{j\,i}\,\), one computes

$$\begin{aligned}{}[ [ a , \widetilde{e}_j ] , \widetilde{e}_i ] = a_{ij} + (-1)^{|e_i| |e_j|} a_{j\,i} = 2 a_{ij}. \end{aligned}$$

Thus the condition \([[a,w],w^\prime ] = 0\) for all \(w, w^\prime \in W\) implies \(a = 0\,\). Hence \(\tau \) is injective.

By the Poincaré–Birkhoff–Witt isomorphism

$$\begin{aligned} \mathfrak {s} \simeq \mathfrak {q}_2(W) / \mathfrak {q}_1(W) \simeq \sum \nolimits _{k+l = 2}\wedge ^k (W_1) \otimes \mathrm {S}^l(W_0), \end{aligned}$$

the dimensions of the \(\mathbb {Z}_2\)-graded vector space \(\mathfrak {s} = \mathfrak {s}_0 \oplus \mathfrak {s}_1\) are

$$\begin{aligned} \dim \, \mathfrak {s}_0 = \dim \, \wedge ^2 (W_1) + \dim \mathrm {S}^2 (W_0), \quad \dim \, \mathfrak {s}_1 = \dim W_1 \, \dim W_0. \end{aligned}$$

These agree with those of \(\mathfrak {osp}(W)\) as recorded in Corollary 2.1. Hence our injective linear map \(\tau : \, \mathfrak {s} \rightarrow \mathfrak {osp}(W)\) is in fact a bijection. \(\square \)

Remark 2.1

By the isomorphism \(\tau \) every representation \(\rho \) of \(\mathfrak {s} \subset \mathfrak {q}(W)\) induces a representation \(\rho \circ \tau ^{-1}\) of \(\mathfrak {osp}(W)\).

Let us conclude this subsection by writing down an explicit formula for \(\tau ^{-1}\). To do so, let \(\{ e_i \}\) and \(\{ \widetilde{e}_j \}\) be homogeneous bases of W with \(Q(e_i \, , \, \widetilde{e}_j) = \delta _{ij}\) as before. For \(X \in \mathfrak {osp}(W)\) notice that the coefficients \(a_{ij} := Q(e_i \, , X e_j) (-1)^{|e_j|}\) are super-symmetric:

$$\begin{aligned} a_{ij} = (-1)^{|X||e_i| + 1 + |e_j|} Q(X e_i \, ,\, e_j) = (-1)^{|X||e_i|} Q(e_j \, , X e_i) = (-1)^{|e_i| |e_j|} a_{j\,i}, \end{aligned}$$

where the last equality sign uses \((-1)^{|e_i| |e_j|}\, a_{j\,i} = (-1)^{|e_i| |X e_i|}\, a_{j\,i} = (-1)^{|e_i| |X| + |e_i|}\, a_{j\,i}\,\).

The inverse map \(\tau ^{-1} : \, \mathfrak {osp}(W) \rightarrow \mathfrak {s}\) is now expressed as

$$\begin{aligned} \tau ^{-1}(X) = {\textstyle {\frac{1}{2}}} \sum \nolimits _{i,j} Q(e_i \, , X e_j) (-1)^{|e_j| + 1} \, \widetilde{e}_i \, \widetilde{e}_j. \end{aligned}$$
(2.8)

To verify this formula, one calculates the double supercommutator \([ e_i\, , [\tau ^{-1}(X) , e_j] ]\) and shows that the result is equal to \([e_i \, , X e_j] = Q(e_i\, , X e_j)\), which is precisely what is required from the definition of \(\tau \) by \([\tau ^{-1}(X),\, e_j] = X e_j\,\).

2.6 Spinor–oscillator representation

As before, starting from a \(\mathbb {Z}_2\)-graded \(\mathbb {K}\)-vector space \(V = V_0 \oplus V_1\,\), let the direct sum \(W = V\oplus V^*\) be equipped with the orthosymplectic form Q and denote by \(\mathfrak {q}(W)\) the Clifford–Weyl algebra of W.

Consider now the following tensor product of exterior and symmetric algebras:

$$\begin{aligned} \mathfrak {a}(V) := \wedge (V_1^*) \otimes \mathrm {S}(V_0^*). \end{aligned}$$

Following R. Howe we call it the spinor–oscillator module of \(\mathfrak {q}(W)\). Notice that \(\mathfrak {a}(V)\) can be identified with the graded-commutative subalgebra in \(\mathfrak {q}(W)\) which is generated by \(V \oplus \mathbb {K}\,\). As such, \(\mathfrak {a}(V)\) comes with a canonical \(\mathbb {Z}_2\)-grading and its space of endomorphisms carries a structure of Lie superalgebra, \(\mathfrak {gl} (\mathfrak {a}(V)) \equiv \mathrm {End}(\mathfrak {a}(V))\).

The algebra \(\mathfrak {a}(V)\) now is to become a representation space for \(\mathfrak {q}(W)\). Four operations are needed for this: the operator \(\varepsilon (\varphi _1): \, \wedge ^k(V_1^*) \rightarrow \wedge ^{k+1} (V_1^*)\) of exterior multiplication by a linear form \(\varphi _1 \in V_1^*\,\); the operator \(\iota (v_1): \, \wedge ^k( V_1^*) \rightarrow \wedge ^{k-1}(V_1^*)\) of alternating contraction with a vector \(v_1\in V_1\,\); the operator \(\mu (\varphi _0):\, \mathrm {S}^l (V_0^*) \rightarrow \mathrm {S}^{l+1}(V_0^*)\) of multiplication with a linear function \(\varphi _0 \in V_0^*\,\); and the operator \(\delta (v_0): \, \mathrm {S}^l (V_0^*) \rightarrow \mathrm {S}^{l-1} (V_0^*)\) of taking the directional derivative by a vector \(v_0 \in V_0\,\).

The operators \(\varepsilon \) and \(\iota \) obey the canonical anti-commutation relations (CAR), which is to say that \(\varepsilon (\varphi )\) and \(\varepsilon (\varphi ^\prime )\) anti-commute, \(\iota (v)\) and \(\iota ( v^\prime )\) do as well, and one has

$$\begin{aligned} \iota (v) \varepsilon (\varphi ) + \varepsilon (\varphi ) \iota (v) = \varphi (v) \, \mathrm {Id}_{\wedge (V_1^*)}. \end{aligned}$$

The operators \(\mu \) and \(\delta \) obey the canonical commutation relations (CCR), i.e., \(\mu (\varphi )\) and \(\mu (\varphi ^\prime )\) commute, so do \(\delta (v)\) and \(\delta (v^\prime )\), and one has

$$\begin{aligned} \delta (v) \mu (\varphi ) - \mu (\varphi ) \delta (v) = \varphi (v) \, \mathrm {Id}_{\mathrm {S}(V_0^*)}. \end{aligned}$$

Given all these operations, one defines a linear mapping \(q : \, W \rightarrow \mathrm {End}(\mathfrak {a}(V))\) by

$$\begin{aligned} q(v_1 + \varphi _1 + v_0 + \varphi _0) = \iota (v_1) + \varepsilon (\varphi _1) + \delta (v_0) + \mu (\varphi _0) \quad (v_s \in V_s\, , \, \varphi _s \in V^*_s)\;, \end{aligned}$$

with \(\iota (v_1)\), \(\varepsilon (\varphi _1)\) operating on the first factor of the tensor product \(\wedge (V_1^*) \otimes \mathrm {S} (V_0^*)\), and \(\delta (v_0)\), \(\mu (\varphi _0)\) on the second factor. Of course the two sets \(\varepsilon , \iota \) and \(\mu , \delta \) commute with each other. In terms of \(q\,\), the relations CAR and CCR are succinctly summarized as

$$\begin{aligned}{}[q(w)\,,\,q(w^\prime )] = Q(w,w^\prime )\, \mathrm {Id}_{\mathfrak {a}(V)}\;, \end{aligned}$$
(2.9)

where \([\, , \,]\) denotes the usual supercommutator of the Lie superalgebra \(\mathfrak {gl}(\mathfrak {a}(V))\). By the relation (2.9) the linear map q extends to a representation of the Jordan–Heisenberg Lie superalgebra \(\widetilde{W} = W \oplus \mathbb {K}\,\), with the constants of \(\widetilde{W}\) acting as multiples of \(\mathrm {Id}_{\mathfrak {a}(V)}\,\).

Moreover, being a representation of \(\widetilde{W}\), the map q yields a representation of the universal enveloping algebra \(\mathsf {U}( \widetilde{W}) \equiv \mathfrak {q}(W)\). This representation is referred to as the spinor–oscillator representation of \(\mathfrak {q}(W)\). In the sequel we will be interested in the \(\mathfrak {osp} (W)\)-representation induced from it by the isomorphism \(\tau ^{-1}\).

There is a natural \(\mathbb {Z}\)-grading \(\mathfrak {a}(V) = \bigoplus _{m \ge 0} \, \mathfrak {a}^m(V)\,\),

$$\begin{aligned} \mathfrak {a}^m(V) = {\textstyle \bigoplus \nolimits }_{k + l = m} \wedge ^k(V_1^*)\otimes \mathrm {S}^l(V_0^*). \end{aligned}$$

Note that the operators \(\varepsilon (\varphi _1)\) and \(\mu (\varphi _0)\) increase the \(\mathbb {Z}\)-degree of \(\mathfrak {a}(V)\) by one, while the operators \(\iota (v_1)\) and \(\delta (v_0)\) decrease it by one. Note also if \({C} = (- \mathrm {Id}_V) \oplus \mathrm {Id}_{V^*}\) is the \(\mathfrak {osp}\)-element introduced in Sect. 2.2.2, then a direct computation using the formula (2.8) shows that \(\mathfrak {a}^m(V)\) is an eigenspace of the operator \((q \circ \tau ^{-1}) ({C})\) with eigenvalue \(m\,\). Thus \({C} \in \mathfrak {osp}\) is represented on the spinor–oscillator module \(\mathfrak {a}(V)\) by the degree.

2.6.1 Weight constraints

We now specialize to the situation of \(V = U \otimes \mathbb {C}^N\) with \(U = U_0 \oplus U_1\) a \(\mathbb {Z}_2\)-graded vector space as in Sect. 2.3, and we require \(U_0\) and \(U_1\) to be isomorphic with dimension \(\dim U_0 = \dim U_1 = n\,\). Recall that

$$\begin{aligned} (\mathfrak {osp}(U\oplus U^*),\mathfrak {o}_N)\;, \quad (\mathfrak {osp} (\widetilde{U} \oplus \widetilde{U}^*),\mathfrak {sp}_N)\;, \end{aligned}$$

are Howe dual pairs in \(\mathfrak {osp}(W)\). As of now we denote these by \((\mathfrak {g}, \mathfrak {k})\). There is a decomposition

$$\begin{aligned}&\mathfrak {g} = \mathfrak {g}^{(-2)} \oplus \mathfrak {g}^{(0)} \oplus \mathfrak {g}^{(2)}\;, \quad \mathfrak {g}^{(0)} = \mathfrak {g} \cap (\mathrm {End}(U)\oplus \mathrm {End}(U^*)), \\&\mathfrak {g}^{(-2)} = \mathfrak {g} \cap \mathrm {Hom}(U^*,U)\;, \quad \mathfrak {g}^{(2)} = \mathfrak {g} \cap \mathrm {Hom}(U,U^*), \end{aligned}$$

in both cases. The notation highlights the fact that the operators in \(\mathfrak {g}^{(m)} \hookrightarrow \mathfrak {osp}(V \oplus V^*)\) change the degree of elements in \(\mathfrak {a}(V)\) by \(m\,\). Note that the Cartan subalgebra \(\mathfrak {h}\) of diagonal operators in \(\mathfrak {g}\) is contained in \(\mathfrak {g}^{(0)}\) but \(\mathfrak {h}\ne \mathfrak {g}^{(0)} \).

Since the Lie algebra \(\mathfrak {k}\) is defined on \(\mathbb {C}^N\), the \(\mathfrak {k}\)-action on \(\mathfrak {a}(V)\) preserves the degree. This action exponentiates to an action of the complex Lie group K on \(\mathfrak {a}(V)\).

Proposition 2.2

The subalgebra \(\mathfrak {a}(V)^K\) of K -invariants in \(\mathfrak {a}(V)\) is an irreducible module for \(\mathfrak {g}\,\) . The vacuum \(1 \in \mathfrak {a}(V)^K\) is contained in it as a cyclic vector such that

$$\begin{aligned} \mathfrak {g}^{(-2)}.1 = 0, \quad \mathfrak {g}^{(0)}.1 = \langle 1 \rangle _{\mathbb {C}}, \quad \langle \mathfrak {g}^{(2)}.1 \rangle _{\mathbb {C}} = \mathfrak {a}(V)^K. \end{aligned}$$

Proof

This is a restatement of Theorems 8 and 9 of [10]. \(\square \)

Remark 2.2

In the case of \((\mathfrak {g}, \mathfrak {k}) = (\mathfrak {osp}(U \oplus U^*), \mathfrak {o}_N )\) it matters that \(K = \mathrm {O}_N\,\), as the connected Lie group \(K = \mathrm {SO}_N\) has invariants in \(\mathfrak {a}(V)\) not contained in \(\langle \mathfrak {g}^{(2)}.1 \rangle _\mathbb {C} \,\).

Proposition 2.2 has immediate consequences for the weights of the \(\mathfrak {g}\)-representation on \(\mathfrak {a}(V)^K\). Using the notation of Sect. 2.2.1, let \(\{ H_{sj} \}\) be a standard basis of \(\mathfrak {h}\) and \(\{ \vartheta _{sj} \}\) the corresponding dual basis. We now write \(\vartheta _{0j} =: \phi _j\,\) and \(\vartheta _{1j} =: \mathrm {i}\psi _j\) (\(j = 1, \ldots , n\)).

Corollary 2.3

The representations of \(\mathfrak {osp}(U\oplus U^*)\) on \(\mathfrak {a}(V)^{\mathrm {O}_N}\) and \(\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)\) on \(\mathfrak {a}(V)^{ \mathrm {Sp}_N}\) each have highest weight \(\lambda _N = \frac{N}{2}\sum _{j=1}^n (\mathrm {i}\psi _j - \phi _j)\) . Every weight of these representations is of the form \(\sum _{j=1}^n (\mathrm {i} m_j \psi _j - n_j \phi _j)\) with \(-\frac{N}{2}\le m_j \le \frac{N}{2} \le n_j\,\).

Proof

Recall from Sect. 2.3 the embedding of \(\mathfrak {osp} (U\oplus U^*)\) and \(\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)\) in \(\mathfrak {osp}(W)\), and from Lemma 2.15 the isomorphism \(\tau ^{-1} : \, \mathfrak {osp}(W) \rightarrow \mathfrak {s}\) where \(\mathfrak {s}\) is the Lie superalgebra of super-symmetrized degree-two elements in \(\mathfrak {q}(W)\). Specializing formula (2.8) to the case of a Cartan algebra generator \(H_{sj} \in \mathfrak {h} \subset \mathfrak {g}\) one gets

$$\begin{aligned} \tau ^{-1}(H_{s j}) = - \tfrac{1}{2} \sum \nolimits _{a = 1}^N ((f_{s,j}\otimes f_a)(e_{s,j} \otimes e_a) + (-1)^s (e_{s,j}\otimes e_a )(f_{s,j}\otimes f_a) ), \end{aligned}$$

where \(\{e_a \}\) is a basis of \(\mathbb {C}^N\) and \(\{ f_a \}\) the dual basis of \((\mathbb {C}^N)^*\).

Now let \(\tau ^{-1}(H_{sj}) \in \mathfrak {s}\) act by the corresponding operator, say \(\hat{H}_{sj} := (q \circ \tau ^{-1})(H_{sj})\), in the spinor–oscillator representation q of \(\mathfrak {s} \subset \mathfrak {q} (W)\). Application of that operator to the highest-weight vector \(1 \in \mathbb {C} \equiv \wedge ^0(V_1^*) \otimes \mathrm {S}^0 (V_0^*) \subset \mathfrak {a}(V)^K\) yields

$$\begin{aligned} \hat{H}_{1 j}\, 1&= \tfrac{1}{2}\sum \nolimits _a \iota (e_{1,j} \otimes e_a)\varepsilon (f_{1,j}\otimes f_a) 1 = \tfrac{N}{2}\;,\\ \hat{H}_{0 j}\, 1&= - \tfrac{1}{2}\sum \nolimits _a \delta (e_{0,j} \otimes e_a) \mu (f_{0,j}\otimes f_a) 1 = - \tfrac{N}{2}. \end{aligned}$$

Altogether this means that \(\hat{H}\, 1 = \lambda _N(H) 1\) where \(\lambda _N(H)= \frac{N}{2} \sum _j (\mathrm {i}\psi _j(H) - \phi _j(H))\).

From Lemma 2.7 the roots \(\alpha \) corresponding to root spaces \(\mathfrak {g}_\alpha \subset \mathfrak {g}^{(2)}\) are of the form

$$\begin{aligned} -\phi _j -\phi _{j^{\,\prime }}, \quad -\mathrm {i}\psi _j -\mathrm {i} \psi _{j^{\,\prime }}\;,\quad -\phi _j-\mathrm {i}\psi _{j^{\,\prime }}\;, \end{aligned}$$

where the indices \(j, j^{\,\prime }\) are subject to restrictions that depend on \(\mathfrak {g}\) being \(\mathfrak {osp} (U\oplus U^*)\) or \(\mathfrak {osp}( \widetilde{U}\oplus \widetilde{U}^*)\). From \(\mathfrak {a}(V)^K = \mathfrak {g}^{ (2)}.1\) one then has \(m_j \le \frac{N}{2} \le n_j\) for every weight \(\gamma = \sum (\mathrm {i} m_j \psi _j - n_j \phi _j)\) of the \(\mathfrak {g}\)-representation on \(\mathfrak {a}(V)^K\).

The restriction \(m_j \ge \frac{N}{2} - N\) results from \(\wedge ( V_1^*) = \wedge (U_1^*\otimes (\mathbb {C}^N)^*)\) being isomorphic to \(\otimes _{j=1}^n \wedge (\mathbb {C}^N)^*\) and the vanishing of \(\wedge ^k (\mathbb {C}^N)^*= 0\) for \(k > N\,\). \(\square \)

Corollary 2.4

For each of our two cases \(\mathfrak {g} = \mathfrak {osp}(U \oplus U^*)\) and \(\mathfrak {g} = \mathfrak {osp}(\widetilde{U}\oplus \widetilde{U}^*)\) the element \({C} = - \sum _{s,j} H_{s j} \subset \mathfrak {g}\) is represented on \(\mathfrak {a}(V)^K\) by the degree operator.

Proof

Since the K-action on \(\mathfrak {a}(V)\) preserves the degree, the subalgebra \(\mathfrak {a}(V)^K\) is still \(\mathbb {Z}\)-graded by the same degree. Summing the above expressions for \((q \circ \tau ^{-1}) (H_{s j})\) over sj and using CAR and CCR to combine terms, we obtain

$$\begin{aligned} (q \circ \tau ^{-1})({C}) = \sum _{j=1}^n \sum _{a=1}^N \big ( \mu (f_{0,j}\otimes f_a) \delta (e_{0,j} \otimes e_a) + \varepsilon (f_{1,j}\otimes f_a) \iota (e_{1,j}\otimes e_a) \big ), \end{aligned}$$

which is in fact the operator for the degree of the \(\mathbb {Z}\)-graded module \(\mathfrak {a}(V)^K\). \(\square \)

2.6.2 Positive and simple roots

We here record the systems of simple positive roots that we will use later (in Sect. 5.4). In the case of \(\mathfrak {osp}(U \oplus U^*)\) this will be

$$\begin{aligned} \phi _1 - \phi _2, \ldots , \phi _{n-1} - \phi _n, \phi _n - \mathrm {i}\psi _1, \mathrm {i}\psi _1 - \mathrm {i} \psi _2, \ldots , \mathrm {i}\psi _{n-1} - \mathrm {i}\psi _n, \mathrm {i} \psi _{n-1} + \mathrm {i} \psi _n. \end{aligned}$$

The corresponding system of positive roots for \(\mathfrak {osp}(U \oplus U^*)\) is

$$\begin{aligned} \phi _j \pm \phi _k, \, \mathrm {i}\psi _j \pm \mathrm {i}\psi _k \,\, (j < k), \,\, 2\phi _j \,, \,\, \phi _j \pm \mathrm {i}\psi _k \,\, (j\, ,k =1, \ldots , n). \end{aligned}$$

In the case of \(\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)\) we choose the system of simple positive roots

$$\begin{aligned} \phi _1 - \phi _2, \ldots , \phi _{n-1} - \phi _n, \phi _n - \mathrm {i}\psi _1, \mathrm {i}\psi _1 - \mathrm {i}\psi _2, \ldots , \mathrm {i}\psi _{n-1} - \mathrm {i}\psi _n, 2\mathrm {i}\psi _n. \end{aligned}$$

The corresponding positive root system then is

$$\begin{aligned} \phi _j \pm \phi _k, \, \mathrm {i}\psi _j \pm \mathrm {i}\psi _k \,\, (j < k), \,\, 2\mathrm {i}\psi _j \,, \,\, \phi _j \pm \mathrm {i}\psi _k \,\, (j\, ,k =1, \ldots , n). \end{aligned}$$

In both cases the roots

$$\begin{aligned} \phi _j - \phi _k, \, \mathrm {i}\psi _j - \mathrm {i}\psi _k \,\, (j < k), \quad \phi _j - \mathrm {i}\psi _k \,\, (j\, , k = 1, \ldots , n), \end{aligned}$$

form a system of positive roots for \(\mathfrak {gl}(U) \simeq \mathfrak {g}^{(0)} \subset \mathfrak {osp}\).

2.6.3 Unitary structure

We now equip the spinor–oscillator module \(\mathfrak {a}(V)\) for \(V = V_0 \oplus V_1\) with a unitary structure. The idea is to think of the algebra \(\mathfrak {a}(V)\) as a subset of \(\mathcal {O}(V_0\, , \, \wedge V_1^*)\), the holomorphic functions \(V_0 \rightarrow \wedge (V_1^*)\). For such functions a Hermitian scalar product is defined via Berezin’s notion of superintegration as follows.

For present purposes, it is imperative that V be defined over \(\mathbb {R}\), i.e., \(V = V_\mathbb {R} \otimes \mathbb {C}\), and that V be re-interpreted as a real vector space \(V^\prime := V_\mathbb {R} \oplus J\, V_\mathbb {R}\) with complex structure \(J \simeq \mathrm {i}\,\). Needless to say, this is done in a manner consistent with the \(\mathbb {Z}_2\)-grading, so that \(V^\prime = V_0^\prime \oplus V_1^\prime \) and \(V_s^\prime = V_{s,\mathbb {R}} \oplus J\, V_{s,\mathbb {R}} \simeq V_{s,\mathbb {R}} \otimes \mathbb {C} = V_s\,\).

From \(V_s = U_s \otimes \mathbb {C}^N\) and \(U_1 \simeq U_0\) we are given an isomorphism \(V_1 \simeq V_0\,\). This induces a canonical isomorphism \(\wedge ({V_1^\prime }^*) \simeq \wedge ( {V_0^\prime }^*)\), which gives rise to a bundle isomorphism \(\Omega \) sending \(\Gamma (V_0^\prime \, , \, \wedge {V_1^\prime }^*)\), the algebra of real-analytic functions on \(V_0^\prime \) with values in \(\wedge ({V_1^\prime }^*)\), to \(\Gamma ({V_0^\prime } \,,\, \wedge T^*V_0^\prime )\), the complex of real-analytic differential forms on \(V_0^\prime \,\). Fixing some orientation of \(V_0^\prime \,\), the Berezin (super-)integral for the \(\mathbb {Z}_2\)-graded vector space \(V^\prime = V_0^\prime \oplus V_1^\prime \) is then defined as the composite map

$$\begin{aligned} \Gamma (V_0^\prime \,,\, \wedge {V_1^\prime }^*) \mathop {\longrightarrow }\limits ^{\Omega } \Gamma (V_0^\prime \,,\,\wedge T^*V_0^\prime ) \ \mathop {\longrightarrow }\limits ^{\int } \mathbb {C}\;,\quad \Phi \mapsto \Omega [\Phi ]\mapsto \int _{V_0^\prime }\Omega [\Phi ]\;, \end{aligned}$$

whenever the integral over \(V_0^\prime \) exists. Thus the Berezin integral is a two-step process: first the section \(\Phi \) is converted into a differential form, then the form \(\Omega [\Phi ]\) is integrated in the usual sense to produce a complex number. Of course, by the rules of integration of differential forms only the top-degree component of \(\Omega [\Phi ]\) contributes to the integral.

The subspace \(V_\mathbb {R} \subset V^\prime \) has played no role so far, but now we use it to decompose the complexification \(V^\prime \otimes \mathbb {C}\) into holomorphic and anti-holomorphic parts: \(V^\prime \otimes \mathbb {C} = V \oplus \overline{V}\) and determine an operation of complex conjugation \(V^*\rightarrow \overline{V^*}\). We also fix on \(V = V_0 \oplus V_1\) a Hermitian scalar product (a.k.a. unitary structure) \(\langle \, , \, \rangle \) so that \(V_0 \perp V_1\,\). This scalar product determines a parity-preserving complex anti-linear bijection \(c : \, V \rightarrow V^*\) by \(v \mapsto cv = \langle v , \, \rangle \). Composing c with complex conjugation \(V^*\rightarrow \overline{V^*}\) we get a \(\mathbb {C}\)-linear isomorphism \(V \rightarrow \overline{V^*}\), \(v \mapsto \overline{cv}\,\).

In this setting there is a distinguished Gaussian section \(\gamma \in \Gamma (V_0^\prime \, , \wedge {V_1^\prime }^*\otimes \mathbb {C})\) singled out by the conditions

$$\begin{aligned} \forall v_0 \in V_0\, , v_1 \in V_1 : \quad \delta (v_0) \gamma = - \mu (\overline{c v_0}) \gamma, \quad \iota (v_1) \gamma = - \varepsilon (\overline{c v_1}) \gamma. \end{aligned}$$
(2.10)

To get a close-up view of \(\gamma \,\), let \(\{ e_{0,j }\}\) and \(\{ e_{1,j} \}\) be orthonormal bases of \(V_0\) resp. \(V_1\,\), and let \(z_j = c e_{0,j}\) and \(\zeta _j = c e_{1,j}\) be the corresponding coordinate functions, with complex conjugates \(\overline{z}_j\) and \(\overline{\zeta }_j\,\). Viewing \(\zeta _j\,\), \(\overline{\zeta }_j\) as generators of \(\wedge ({V_1^\prime }^*\otimes \mathbb {C})\), our section \(\gamma \in \Gamma (V_0^\prime \,,\, \wedge {V_1^\prime }^*\otimes \mathbb {C})\) is the standard Gaussian

$$\begin{aligned} \gamma = \mathrm {const} \times \mathrm {e}^{-\sum _j (z_j \overline{z}_j + \zeta _j \overline{\zeta }_j )}. \end{aligned}$$

We fix the normalization of \(\gamma \) by the condition \(\int _{V_0^\prime } \Omega [\gamma ] = 1\).

A unitary structure on the spinor–oscillator module \(\mathfrak {a}(V)\) is now defined as follows. Let complex conjugation \(V^*\rightarrow \overline{V^*}\) be extended to an algebra anti-homomorphism \(\mathfrak {a}(V) \rightarrow \mathfrak {a}(\overline{V})\) by the convention \(\overline{\Phi _1 \Phi _2} = \overline{\Phi }_2 \, \overline{\Phi }_1\) (without any sign factors). Then, if \(\Phi _1\, , \Phi _2\) are any two elements of \(\mathfrak {a}(V)\), we view them as holomorphic maps \(V_0 \rightarrow \wedge (V_1^*)\), multiply \(\overline{\Phi }_1\) with \(\Phi _2\) to form \(\overline{\Phi }_1 \Phi _2 \in \Gamma (V_0^\prime \,, \wedge {V_1^\prime }^*\otimes \mathbb {C})\), and define their Hermitian scalar product by

$$\begin{aligned} \left\langle \Phi _1 \, , \Phi _2 \right\rangle _{\mathfrak {a}(V)} := \int _{V_0^\prime } \Omega [ \gamma \, \overline{\Phi }_1 \Phi _2 ]\;. \end{aligned}$$
(2.11)

Let us mention in passing that (2.11) coincides with the unitary structure of \(\mathfrak {a}(V)\) used in the Hamiltonian formulation of quantum field theories and in the Fock space description of many particle systems composed of fermions and bosons. The elements

$$\begin{aligned} \bigwedge \nolimits _j \zeta _j^{m_j} \otimes \prod \nolimits _j z_j^{n_j} / \sqrt{n_j \, !} \end{aligned}$$
(2.12)

for \(m_j \in \{ 0, 1 \}\) and \(n_j \in \{ 0, 1, \ldots \}\) form an orthonormal set in \(\mathfrak {a}(V)\), which in physics is called the occupation number basis of \(\mathfrak {a}(V)\).

Lemma 2.16

For all \(v_0 \in V_0\) and \(v_1 \in V_1\) the pairs of operators \(\delta (v_0)\), \(\mu (cv_0)\) and \(\iota (v_1)\), \(\varepsilon (cv_1)\) in \(\mathrm {End} (\mathfrak {a}(V))\) obey the relations

$$\begin{aligned} \delta (v_0)^\dagger = \mu (c v_0), \quad \iota (v_1)^\dagger = \varepsilon (c v_1), \end{aligned}$$

i.e., they are mutual adjoints with respect to the unitary structure of \(\mathfrak {a} (V)\).

Proof

Let \(v \in V_0\,\). Since \(\overline{\Phi }_1 \in \mathfrak {a} (\overline{V})\) is anti-holomorphic, we have \(\delta (v) \overline{\Phi }_1 = 0\,\). By the first defining property of \(\gamma \) in (2.10) and the fact that \(\delta (v)\) is a derivation,

$$\begin{aligned} \gamma \, \overline{\Phi }_1 \delta (v) \Phi _2 = \delta (v) \left( \gamma \, \overline{\Phi }_1 \Phi _2 \right) + \mu (\overline{cv}) \gamma \, \overline{\Phi }_1 \Phi _2, \end{aligned}$$

and passing to the Hermitian scalar product by the Berezin integral we obtain

$$\begin{aligned} \left\langle \Phi _1 , \delta (v) \Phi _2 \right\rangle _{\mathfrak {a}(V)} = \int _{V_0} \Omega [\gamma \,\overline{\Phi }_1 \overline{\mu (cv)}\Phi _2] = \left\langle \mu (cv)\Phi _1,\Phi _2\right\rangle _{\mathfrak {a}(V)}. \end{aligned}$$

By the definition of the \(\dagger \)-operation this means that \(\delta (v)^\dagger = \mu (cv)\).

In the case of \(v \in V_1\) the argument is similar but for a few sign changes. Our starting relation changes to

$$\begin{aligned} \gamma \, \overline{\Phi }_1 \iota (v) \Phi _2 = (-1)^{|\Phi _1|} \iota (v) \left( \gamma \, \overline{\Phi }_1 \Phi _2 \right) + (-1)^{|\Phi _1|} \varepsilon (\overline{cv})\gamma \, \overline{\Phi }_1 \Phi _2, \end{aligned}$$

since the operator \(\iota (v)\) is an anti-derivation. If \(v \mapsto \tilde{v}\) denotes the isomorphism \(V_1 \rightarrow V_0\,\), then \(\Omega \circ \iota (v) = \iota (\tilde{v}) \circ \Omega \) and the first term on the right-hand side Berezin-integrates to zero because \(\iota (\tilde{v})\) lowers the degree in \(\wedge T^*V_0^\prime,\). Therefore,

$$\begin{aligned} \left\langle \Phi _1 , \iota (v) \Phi _2 \right\rangle _{\mathfrak {a}(V)} = \int _{V_0^\prime } \Omega [\gamma \,\overline{\Phi }_1 \overline{ \varepsilon (cv)} \Phi _2] = \left\langle \varepsilon (cv) \Phi _1 , \Phi _2 \right\rangle _{\mathfrak {a}(V)}\;, \end{aligned}$$

which is the statement \(\iota (v)^\dagger = \varepsilon (cv)\). \(\square \)

By the Hermitian scalar product (2.11) and the corresponding \(L^2\)-norm, the spinor–oscillator module \(\mathfrak {a}(V)\) is completed to a Hilbert space, \(\fancyscript {A}_V\,\). A nice feature here is that, as an immediate consequence of the factors \(1 / \sqrt{n_j \, !}\) in the orthonormal basis (2.12), the \(L^2\)-condition \(\langle \Phi , \Phi \rangle _{\mathfrak {a}(V)} < \infty \) implies absolute convergence of the power series for \(\Phi \in \fancyscript {A}_V\,\). Hence \(\fancyscript {A}_V\) can be viewed as a subspace of \(\mathcal {O}(V_0\, , \wedge V_1^*)\):

$$\begin{aligned} \fancyscript {A}_V = \{ \Phi \in \mathcal {O}(V_0 \, , \wedge V_1^*) \mid \langle \Phi , \Phi \rangle _{\mathfrak {a}(V)} < \infty \}. \end{aligned}$$

In the important case of isomorphic components \(V_0 \simeq V_1\,\), we may regard \(\fancyscript {A}_V\) as the Hilbert space of square-integrable holomorphic differential forms on \(V_0\,\).

Note that although \(\delta (v)\) and \(\mu (\varphi )\) do not exist as operators on the Hilbert space \(\fancyscript {A}_V\,\), they do extend to linear operators on \(\mathcal {O}(V_0 \, , \wedge V_1^*)\) for all \(v \in V_0\) and \(\varphi \in V_0^*\,\).

2.7 Real structures

In this subsection we define a real structure for the complex vector space \(W = V\oplus V^*\) and describe, in particular, the resulting real forms of the (\(\mathbb {Z}_2\)-even components of the) Howe dual partners introduced above.

Recalling the map \(c:\, V \rightarrow V^*\), \(v \mapsto \langle v, \, \rangle \,\), let \(W_\mathbb {R} \simeq V\) be the vector space

$$\begin{aligned} W_{\mathbb {R}} = \{ v+cv \mid v \in V \} \subset V \oplus V^* = W. \end{aligned}$$

Note that \(W_\mathbb {R}\) can be viewed as the fixed point set \(W_\mathbb {R} = \mathrm {Fix}(C)\) of the involution

$$\begin{aligned} C : \,\, W \rightarrow W, \quad v + \varphi \mapsto c^{-1}\varphi + cv. \end{aligned}$$

By the orthogonality assumption, \(W_\mathbb {R} = W_{0,\mathbb {R}} \oplus W_{1,\mathbb {R}}\) where \(W_{s,\mathbb {R}} = W_s \cap W_\mathbb {R}\,\).

The symmetric bilinear form S on \(W_1^{} = V_1^{} \oplus V_1^*\) restricts to a Euclidean structure

$$\begin{aligned} S\, :\,\, W_{1,\mathbb {R}}\times W_{1,\mathbb {R}}\rightarrow \mathbb {R}, \quad (v + cv \, , v'+cv' ) \mapsto 2\, \mathfrak {Re}\langle v,v'\rangle \end{aligned}$$

whereas the alternating form A on \(W_0^{} = V_0^{} \oplus V_0^*\) induces a real-valued symplectic form

$$\begin{aligned} \omega = \mathrm {i}A \, : \,\, W_{0,\mathbb {R}} \times W_{0,\mathbb {R}} \rightarrow \mathbb {R}, \quad (v+cv\,, v'+cv') \mapsto 2\, \mathfrak {Im}\langle v,v'\rangle. \end{aligned}$$

Please be warned that, since \(Q = S + A\) fails to be real-valued on \(W_\mathbb {R}\,\), the intersection \(\mathfrak {osp}(W) \cap \mathrm {End}(W_\mathbb {R})\) is not a real form of the complex Lie superalgebra \(\mathfrak {osp}(W)\).

The connected classical real Lie groups associated to the bilinear forms S and \(\omega \) are

$$\begin{aligned} \mathrm {SO}(W_{1,\mathbb {R}})&:=\{ g \in \mathrm {SL}(W_{1,\mathbb {R}}) \mid \forall w, w^\prime \in W_{1,\mathbb {R}} : \, S(gw,gw') = S(w,w')\}, \\ \mathrm {Sp}(W_{0,\mathbb {R}})&:= \{ g\in \mathrm {GL}(W_{0,\mathbb {R}}) \mid \forall w,w^\prime \in W_{0,\mathbb {R}} : \, \omega (gw,gw') = \omega (w,w') \}. \end{aligned}$$

They have Lie algebras denoted by \(\mathfrak {o}(W_{1,\mathbb {R}})\) and \(\mathfrak {sp} (W_{0,\mathbb {R}})\). By construction we have \(\mathfrak {osp}(W)_0 \cap \mathrm {End}(W_\mathbb {R}) \simeq \mathfrak {o}(W_{1,\mathbb {R}}) \oplus \mathfrak {sp} (W_{0,\mathbb {R}})\), and this in fact is a real form of the complex Lie algebra \(\mathfrak {osp}(W)_0 \simeq \mathfrak {o} (W_1) \oplus \mathfrak {sp}(W_0)\).

Proposition 2.3

The elements of \(\mathfrak {o}(W_{1,\mathbb {R}}) \oplus \mathfrak {sp}(W_{0,\mathbb {R}}) \subset \mathfrak {osp}(W)\) are mapped via \(\tau ^{-1}\) and the spinor–oscillator representation to anti-Hermitian operators in \(\mathrm {End}(\mathfrak {a}(V))\).

Proof

Let \(X \in \mathfrak {o}(W_{1,\mathbb {R}}) \oplus \mathfrak {sp} (W_{0,\mathbb {R}})\). We know from Lemma 2.15 that \(\tau ^{-1}(X)\) is a super-symmetrized element of degree two in the Clifford–Weyl algebra \(\mathfrak {q}(W)\). To see the explicit form of such an element, recall the definition \(\tau (a) w = [a , w]\,\). Since \(Q = S + A\,\), and A restricts to \(\mathrm {i}\omega \,\), the fundamental bracket \([\, , \,] : \, W_\mathbb {R} \times W_\mathbb {R} \rightarrow \mathbb {C}\) given by \([w , w^\prime ] = Q(w,w^\prime )\) is real-valued on \(W_{1,\mathbb {R}}\) but imaginary-valued on \(W_{0,\mathbb {R}}\). Therefore,

$$\begin{aligned} \tau ^{-1}(\mathfrak {o}(W_{1,\mathbb {R}}))&= \mathrm {span}_\mathbb {R} \{ w w^\prime - w^\prime w \} \quad (w, w^\prime \in W_{1,\mathbb {R}})\;, \\ \tau ^{-1} (\mathfrak {sp}(W_{0,\mathbb {R}}))&= \mathrm {span}_\mathbb {R} \{\mathrm {i} w w^\prime + \mathrm {i} w^\prime w \} \quad (w, w^\prime \in W_{0,\mathbb {R}}). \end{aligned}$$

The proposed statement \(X^\dagger = -X\) now follows under the assumption that the spinor–oscillator representation maps every \(w \in W_\mathbb {R}\) to a self-adjoint operator in \(\mathrm {End}( \mathfrak {a}(V))\). But every element \(w \in W_\mathbb {R}\) is of the form \(v_1 + c v_1 + v_0 + c v_0\) and this maps to the operator \(\iota (v_1) + \varepsilon (c v_1) + \delta (v_0) + \mu (c v_0)\), which is self-adjoint by Lemma 2.16. \(\square \)

Given the real structure \(W_\mathbb {R}\) of W, we now ask how \(\mathrm {End}( W_\mathbb {R})\) intersects with the Howe pairs \((\mathfrak {osp}(U \oplus U^*), \mathfrak {o}_N)\) and \((\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*), \mathfrak {sp}_N)\) embedded in \(\mathfrak {osp}(W)\). By the observation that Q restricted to \(W_\mathbb {R}\) is not real-valued, \(\mathfrak {osp}(U \oplus U^*) \cap \mathrm {End}( W_\mathbb {R})\) fails to be a real form of the complex Lie superalgebra \(\mathfrak {osp}(U \oplus U^*)\), and the same goes for \(\mathfrak { osp} (\widetilde{U} \oplus \widetilde{U}^*)\). Nevertheless, it is still true that the even components of these intersections are real forms of the complex Lie algebras \(\mathfrak {osp}(U\oplus U^*)_0\) and \(\mathfrak {osp} (\widetilde{U}\oplus \widetilde{U}^*)_0\,\).

The real forms of interest are best understood by expressing them via blocks with respect to the decomposition \(W = V \oplus V^*\). Since \(W_\mathbb {R} = \mathrm {Fix}(C)\), the complex linear endomorphisms of W stabilizing \(W_\mathbb {R}\) are given by

$$\begin{aligned} \mathrm {End}(W_\mathbb {R}) \simeq \{ X \in \mathrm {End}(W) \mid X = C X C^{-1} \}. \end{aligned}$$

Writing X in block-decomposed form

$$\begin{aligned} X = \mathsf {A} \oplus \mathsf {B} \oplus \mathsf {C} \oplus \mathsf {D} \equiv \begin{pmatrix} \mathsf {A} &{}\mathsf {B}\\ \mathsf {C} &{}\mathsf {D} \end{pmatrix}, \end{aligned}$$

where \(\mathsf {A} \in \mathrm {End}(V)\), \(\mathsf {B} \in \mathrm {Hom}(V^*,V)\), \(\mathsf {C} \in \mathrm {Hom}(V,V^*)\), and \(\mathsf {D} \in \mathrm {End}(V^*)\), the condition \(X = C X C^{-1}\) becomes

$$\begin{aligned} \mathsf {C} = \overline{\mathsf {B}}, \quad \mathsf {D} = \overline{\mathsf {A}}. \end{aligned}$$

The bar here is a short-hand notation for the complex anti-linear maps

$$\begin{aligned} \mathrm {Hom}(V^*,V) \rightarrow \mathrm {Hom}(V,V^*)\;, \quad&\mathsf {B} \mapsto \overline{\mathsf {B}} :=c \mathsf {B} c,\\ \mathrm {End}(V) \rightarrow \mathrm {End}(V^*)\;, \quad&\mathsf {A} \mapsto \overline{\mathsf {A}} := c \mathsf {A} c^{-1}. \end{aligned}$$

When expressed with respect to compatible bases of V and \(V^*\), these maps are just the standard operation of taking the complex conjugate of the matrices of \(\mathsf {A}\) and \(\mathsf {B}\).

Now, to get an understanding of the intersections \(\mathfrak {o}_N \cap \mathrm {End}(W_\mathbb {R})\) and \(\mathfrak {sp}_N \cap \mathrm {End}(W_\mathbb {R})\), recall the relation \(\mathsf {D} = - \mathsf {A}^\mathrm {t}\) for \(X \in \mathfrak {osp}(W)_0\) and the fact that the action of the complex Lie algebras \(\mathfrak {o}_N = \mathfrak {o}(\mathbb {C}^N)\) and \(\mathfrak {sp}_N = \mathfrak {sp} (\mathbb {C}^N)\) on W stabilizes the decomposition \(W = V \oplus V^*\), with the implication that \(\mathsf {B} = \mathsf {C} = 0\) in both cases. Combining \(\mathsf {D} = - \mathsf {A}^ \mathrm {t}\) with \(\mathsf {D} = \overline{\mathsf {A}}\) one gets the anti-Hermitian property \(\mathsf {A} = - \overline{\mathsf {A}}^ \mathrm {t}\), which means that \(\mathfrak {o}_N \cap \mathrm {End}(W_\mathbb {R})\) and \(\mathfrak {sp}_N \cap \mathrm {End}(W_\mathbb {R})\) are compact real forms of \(\mathfrak {o}_N\) and \(\mathfrak {sp}_N\,\).

Turning to the Howe dual partners of \(\mathfrak {o}_N\) and \(\mathfrak {sp}_N\,\), recall from Sect. 2.3 the isomorphism \(\psi : \, \mathbb {C}^N \rightarrow (\mathbb {C}^N)^*\) and arrange for it to be an isometry, \(\overline{\psi ^{-1}} = \psi ^\mathrm {t}\), of the unitary structures of \(\mathbb {C}^N\) and \((\mathbb {C}^N)^*\). Recall also the embedding of the two Lie superalgebras \(\mathfrak {osp}(U \oplus U^*)\) and \(\mathfrak {osp} (\widetilde{U} \oplus \widetilde{U}^*)\) into \(\mathfrak {osp}(W) = \mathfrak {osp}(U \otimes \mathbb {C}^N \oplus U^*\otimes (\mathbb {C}^N)^*)\) by

$$\begin{aligned} \begin{pmatrix} \mathsf {a} &{}\mathsf {b} \\ \mathsf {c} &{}\mathsf {d} \end{pmatrix} \mapsto \begin{pmatrix} \mathsf {a} \otimes \mathrm {Id} &{}\mathsf {b} \otimes \psi ^{-1} \\ \mathsf {c} \otimes \psi &{}\mathsf {d} \otimes \mathrm {Id} \end{pmatrix} = \begin{pmatrix} \mathsf {A} &{}\mathsf {B} \\ \mathsf {C} &{}\mathsf {D} \end{pmatrix}. \end{aligned}$$

Here the notation still means the same, i.e., \(\mathsf {a} \in \mathrm {End}(U)\), \(\mathsf {b} \in \mathrm {Hom}(U^*,U)\), and so on.

Let a real structure \((U \oplus U^*)_\mathbb {R}\) of \(U \oplus U^*\) be defined in the same way as the real structure \(W_\mathbb {R} = (V \oplus V^*)_\mathbb {R}\) of \(W = V \oplus V^*\).

Proposition 2.4

\(\mathfrak {osp}(U \oplus U^*)_0 \cap \mathrm {End}(W_\mathbb {R}) \simeq \mathfrak {o}( (U_1^{} \oplus U_1^*)_\mathbb {R} ) \oplus \mathfrak {sp} ((U_0^{} \oplus U_0^*)_\mathbb {R})\).

Proof

The intersection is computed by transferring the conditions \(\mathsf {D} = \overline{\mathsf {A}}\) and \(\mathsf {C} = \overline{ \mathsf {B}}\) to the level of \(\mathfrak {osp}(U \oplus U^*)_0\,\). Of course \(\mathsf {D} = \overline{\mathsf {A}}\) just reduces to the corresponding condition \(\mathsf {d} = \overline{\mathsf {a}}\,\). Because the isometry \(\psi : \, \mathbb {C}^N \rightarrow (\mathbb {C}^N)^*\) in the present case is symmetric one has \(\overline{\psi ^{-1}} = \psi ^\mathrm {t} = + \psi \), so the condition \(\mathsf {C} = \overline{\mathsf {B}}\) transfers to \(\mathsf {c} = \overline{\mathsf {b}}\,\). For the same reason, the parity of the maps \(\mathsf {b}, \mathsf {c}\) is identical to that of \(\mathsf {B}, \mathsf {C}\), i.e., \(\mathsf {b} \vert _{U_0^*\rightarrow U_0^{}}\) is symmetric, \(\mathsf {b} \vert _{U_1^*\rightarrow U_1^{}}\) is skew, and similar for \(\mathsf {c}\,\). Hence, computing the intersection \(\mathfrak {osp}(U \oplus U^*)_0 \cap \mathrm {End}(W_\mathbb {R})\) amounts to the same as computing \(\mathfrak {osp}(V \oplus V^*)_0 \cap \mathrm {End}(W_\mathbb {R})\), and the statement follows from our previous discussion of the latter case. \(\square \)

In the case of the Howe pair \((\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*), \mathfrak {sp}_N)\) the isometry \(\psi : \, \mathbb {C}^N \rightarrow (\mathbb {C}^N)^*\) is skew, so that \(\overline{ \psi ^{-1}} = \psi ^\mathrm {t} = - \psi \,\). At the same time, the parity of \(\mathsf {b}, \mathsf {c}\) is reversed as compared to \(\mathsf {B}, \mathsf {C}\): now the map \(\mathsf {b} \vert _{U_0^*\rightarrow U_0^{}}\) is skew and \(\mathsf {b} \vert _{U_1^*\rightarrow U_1^{}}\) is symmetric (and similar for \(\mathsf {c})\). Therefore,

$$\begin{aligned} \mathfrak {osp}(\widetilde{U}\oplus \widetilde{U}^*)_0\cap \mathrm {End}(W_{1,\mathbb {R}})\simeq \{ \begin{pmatrix} \mathsf {a} &{}\mathsf {b}\\ - \overline{\mathsf {b}} &{}-\mathsf {a}^\mathrm {t} \end{pmatrix} \in \mathrm {End} (U_1^{} \oplus U_1^*) \mid \mathsf {a} = - \overline{\mathsf {a}}^\mathrm {t}, \, \mathsf {b} = + \mathsf {b}^\mathrm {t} \}, \end{aligned}$$

which is a compact real form \(\mathfrak {usp}(U_1^{} \oplus U_1^*)\) of \(\mathfrak {sp}(U_1 \oplus U_1^*)\); and

$$\begin{aligned} \mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)_0 \cap \mathrm {End}(W_{0,\mathbb {R}}) \simeq \{ \begin{pmatrix} \mathsf {a} &{}\mathsf {b}\\ - \overline{\mathsf {b}} &{}-\mathsf {a}^\mathrm {t} \end{pmatrix} \in \mathrm {End}(U_0^{} \oplus U_0^*) \mid \mathsf {a} = -\overline{\mathsf {a}}^\mathrm {t}, \, \mathsf {b} = -\mathsf {b}^\mathrm {t} \}, \end{aligned}$$

which is a non-compact real form of \(\mathfrak {o} (U_0^{} \oplus U_0^*)\) known as \(\mathfrak {so}^*(U_0^{} \oplus U_0^*)\).

Let us summarize this result.

Proposition 2.5

\(\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)_0 \cap \mathrm {End}(W_\mathbb {R}) \simeq \mathfrak {usp} (U_1^{} \oplus U_1^*) \oplus \mathfrak {so}^*(U_0^{} \oplus U_0^*)\).

3 Semigroup representation

As before, we identify the complex Lie superalgebra \(\mathfrak {g} := \mathfrak {osp}(W)\) with the space of super-symmetrized degree-two elements in \(\mathfrak {q}_2(W)\), so that

$$\begin{aligned} \mathfrak {q}_2(W) = \mathfrak {g} \oplus \mathfrak {q}_1(W), \quad \mathfrak {q}_1(W) = W \oplus \mathbb {C}\;. \end{aligned}$$

The adjoint representation of \(\mathfrak {g}\) on \(\mathfrak {q}(W)\) restricts to the Lie algebra representation of \(\mathfrak {g}_0 = \mathfrak {o}(W_1) \oplus \mathfrak {sp}(W_0)\) on \(W = W_1\oplus W_0\) which is just the direct sum of the fundamental representations of \(\mathfrak {o}(W_1)\) and \(\mathfrak {sp}(W_0)\). These are integrated by the fundamental representations of the complex Lie groups \(\mathrm {SO}(W_1)\) and \(\mathrm {Sp}(W_0)\) , respectively.

Since the Clifford–Weyl algebra \(\mathfrak {q}(W)\) is an associative algebra, one can ask if, given \(x \in \mathfrak {g}_0 \subset \mathfrak {q}(W)\), the exponential series \(\mathrm {e}^x\) makes sense. The existence of a one-parameter group \(\mathrm {e}^{tx}\) for \(x \in \mathfrak {g}_0\) would of course imply that

$$\begin{aligned} \frac{d}{dt} \, \mathrm {e}^{\, tx} \, w \, \mathrm {e}^{-tx} \, \Big \vert _{t=0} = \mathrm {ad}(x)\, w \qquad (w \in W)\,. \end{aligned}$$
(3.1)

Now \(\mathfrak {q}(W) = \mathfrak {c}(W_1) \otimes \mathfrak {w}(W_0)\). Since the Clifford algebra \(\mathfrak {c}(W_1)\) is finite-dimensional, the series \(\mathrm {e}^x\) for \(x \in \mathfrak {o}(W_1) \hookrightarrow \mathfrak {g}_0\) does make immediate sense. In this way one is able to exponentiate the Lie algebra \(\mathfrak {o}(W_1)\) in \(\mathfrak {c}(W_1)\). The associated complex Lie group, which is then embedded in \(\mathfrak {c}(W_1)\), is \(\mathrm {Spin}(W_1)\). This is a 2:1 cover of the complex orthogonal group \(\mathrm {SO}(W_1)\). Its conjugation representation on \(W_1\) as in (3.1) realizes the covering map as a homomorphism \(\mathrm {Spin}(W_1) \rightarrow \mathrm {SO}(W_1)\).

Viewing the other summand \(\mathfrak {sp}(W_0)\) of \(\mathfrak {g}_0\) as being in the infinite-dimensional Weyl algebra \(\mathfrak {w}(W_0)\), it is definitely not possibly to exponentiate it in such a naive way. This is in particular due to the fact that for most \(x \in \mathfrak {sp}(W_0)\) the formal series \(\mathrm {e}^x\) is not contained in any space \(\mathfrak {w}_n (W_0)\) of the filtration of \(\mathfrak {w} (W_0)\).

As a first step toward remedying this situation, we consider \(\mathfrak {q}(W)\) as a space of densely defined operators on the completion \(\fancyscript {A}_V\) (cf. Sect. 2.6.3) of the spinor–oscillator module \(\mathfrak {a}(V)\). Since all difficulties are on the \(W_0\) side, for the remainder of this chapter we simplify the notation by letting \(W := W_0\) and discussing only the oscillator representation of \(\mathfrak {w}(W)\). Recall that this representation on \(\mathfrak {a}(V)\) is defined by multiplication \(\mu ( \varphi )\) for \(\varphi \in V^*\) and the directional derivative \(\delta (v)\) for \(v \in V\).

For \(x \in \mathfrak {w}(W)\) there is at least no formal obstruction to the exponential series of x existing in \(\mathrm {End}(\mathcal A_V)\). However, direct inspection shows that convergence cannot be expected unless some restrictions are imposed on \(x\,\). This is done by introducing a notion of unitarity and an associated semigroup of contraction operators.

3.1 The oscillator semigroup

Here we introduce the basic semigroup in the complex symplectic group. Various structures are lifted to its canonical 2:1 covering. Actions of the real symplectic and metaplectic groups are discussed along with the role played by the cone of elliptic elements.

3.1.1 Contraction semigroup: definitions, basic properties

Letting \(\langle \, , \, \rangle \) be the unitary structure on V which was fixed in the previous chapter, we recall the complex anti-linear bijection \(c : \, V \rightarrow V^*\), \(v \mapsto \langle v \, , \, \rangle \,\). There is an induced map \(C : \, W \rightarrow W\) on \(W = V \oplus V^*\) by \(C(v + \varphi ) = c^{-1}\varphi + cv\,\). As before, we put \(W_\mathbb {R} := \mathrm {Fix}(C) \subset W\).

Since we have restricted our attention to the symplectic side, the vector spaces W and \(W_\mathbb {R}\) are now equipped with the standard complex symplectic structure A and real symplectic form \(\omega = \mathrm {i}A\) , respectively. From here on in this chapter we abbreviate the notation by writing \(\mathrm {Sp} := \mathrm {Sp}(W)\) and \(\mathfrak {sp} := \mathfrak {sp} (W)\). Let an anti-unitary involution \(\sigma : \, \mathrm {Sp} \rightarrow \mathrm {Sp}\) be defined by \(g \mapsto C g \, C^{-1}\). Its fixed point group \(\mathrm {Fix} (\sigma )\) is the real form \(\mathrm {Sp} (W_\mathbb {R}\)) of main interest. We here denote it by \(\mathrm {Sp}_\mathbb {R}\) and let \(\mathfrak {sp}_\mathbb {R}\) stand for its Lie algebra.

Given A and C, consider the mixed-signature Hermitian structure

$$\begin{aligned} W \times W \rightarrow \mathbb {C}, \quad (w\, ,w^\prime ) \mapsto A(Cw\, ,w^\prime ), \end{aligned}$$

which we denote by \(A(Cw\, ,w^\prime ) =: \langle w \, , w^\prime \rangle _s\,\), with subscript s to distinguish it from the canonical Hermitian structure of W given by \(\langle v + \varphi , v^\prime + \varphi ^\prime \rangle := \langle v \, , v^\prime \rangle + \langle c^{-1} \varphi ^\prime , c^{-1} \varphi \rangle \). The relation between the two is

$$\begin{aligned} \langle w \, , w^\prime \rangle _s = \langle w \, , s w^\prime \rangle, \qquad s = (- \mathrm {Id}_V) \oplus \mathrm {Id}_{V^*}. \end{aligned}$$

Note also the relation

$$\begin{aligned} \sigma (g) = C g\, C^{-1} = s\, (g^{-1})^\dagger s \qquad (g \in \mathrm {Sp}). \end{aligned}$$

Now observe that the real form \(\mathrm {Sp}_\mathbb {R}\) is the subgroup of \(\langle \, , \, \rangle _s\)-isometries in \(\mathrm {Sp}:\)

$$\begin{aligned} \mathrm {Sp}_{\mathbb {R}} = \{g \in \mathrm {Sp} \mid \forall w \in W : \; \langle g w , g w \rangle _s = \langle w , w \rangle _s \}. \end{aligned}$$

Then define a semigroup \(\mathrm {H}(W^s)\) in \(\mathrm {Sp}\) by

$$\begin{aligned} \mathrm {H}(W^s) := \{ h \in \mathrm {Sp} \mid \forall w \in W ,\; w \not = 0 : \; \langle h w , h w \rangle _s < \langle w , w \rangle _s \} \,. \end{aligned}$$

Note that the operation \(g \mapsto g^\dagger \) of Hermitian conjugation with respect to \(\langle \, ,\, \rangle \) stabilizes \(\mathrm {Sp}\) and that \(\mathrm {Sp}_\mathbb {R}\) is defined by the condition \(g^\dagger s g = s\,\). The semigroup \(\mathrm {H}(W^s)\) is defined by \(h^\dagger s h < s\,\), or equivalently, \(s - h^\dagger s h\) is positive definite. We will see later that \(\mathrm {H}(W^s)\) (or, rather, a 2:1 cover thereof) acts by contraction operators on the Hilbert space \(\fancyscript {A}_V\).

It is immediate that \(\mathrm {H}(W^s)\) is an open semigroup in \(\mathrm {Sp}\) with \(\mathrm {Sp}_\mathbb {R}\) on its boundary. Furthermore, \(\mathrm {H}(W^s)\) is stabilized by the action of \(\mathrm {Sp}_\mathbb {R} \times \mathrm {Sp}_\mathbb {R}\) by \(h\mapsto g_1 h \, g_2^{-1}\).

The map \(\pi :\, \mathrm {Sp} \rightarrow \mathrm {Sp}\,\), \(h \mapsto h \sigma (h^{-1})\), will play an important role in our considerations. It is invariant under the \(\mathrm {Sp}_\mathbb {R}\)-action by right multiplication, \(\pi (hg^{-1}) = \pi (h)\), and is equivariant with respect to the action defined by left multiplication on its domain of definition and conjugation on its image space, \(\pi (gh) = g \pi (h) g^{-1}\). Direct calculation shows that in fact the \(\pi \)-fibers are exactly the orbits of the \(\mathrm {Sp}_\mathbb {R}\)-action by right multiplication. Observe that if \(h = \exp (\mathrm {i}X)\) for \(X \in \mathfrak {sp}_\mathbb {R}\,\), then \(\sigma (h) = h^{-1}\) and \(\pi (h) = h^2\). In particular, if \(\mathfrak {t}\) is a Cartan subalgebra of \(\mathfrak {sp}\) which is defined over \(\mathbb {R}\,\), then \(\pi \vert _{\exp (\mathrm {i} \mathfrak {t}_\mathbb {R})}\) is just the squaring map \(t \mapsto t^2\).

3.1.2 Actions of \(\mathrm {Sp}_{\mathbb {R}}\,\)

We now fix a Cartan subalgebra \(\mathfrak {t}\) having the property that \(T_\mathbb {R} = \exp (\mathfrak {t}_\mathbb {R})\) is contained in the unitary maximal compact subgroup defined by \(\langle \, ,\, \rangle \) of \(\mathrm {Sp} (W_\mathbb {R})\). This means that T acts diagonally on the decomposition \(W = V \oplus V^*\) and there is a (unique up to order) orthogonal decomposition \(V = E_1 \oplus \ldots \oplus E_d\) into one-dimensional subspaces so that if \(F_j := c(E_j)\), then T acts via characters \(\chi _j\) on the vector spaces \(P_j := E_j \oplus F_j\) by \(t(e_j\, ,f_j) = (\chi _j(t) e_j \, , \chi _j(t)^{-1} f_j)\).

In other words, we may choose \(\{ e_j \}_{j = 1, \ldots , d}\) to be an orthonormal basis of V and equip \(V^*\) with the dual basis so that the elements \(t \in T\) are in diagonal form:

$$\begin{aligned} t = \mathrm {diag}(\lambda _1^{},\ldots ,\lambda _d^{ }\, ,\lambda _1^{-1}, \ldots , \lambda _d^{-1})\;, \ \qquad \lambda _j = \chi _j(t). \end{aligned}$$

Observe that, conversely, the elements of \(\mathrm {Sp}\) that stabilize the decomposition \(W = P_1 \oplus \ldots \oplus P_d\) and act diagonally in the above sense, are exactly the elements of T. Moreover, \(\exp (\mathrm {i} \mathfrak {t}_\mathbb {R})\) is the subgroup of elements \(t \in T\) with \(\chi _j(t) \in \mathbb {R}_+\) for all \(j\,\). Note that the complex symplectic planes \(P_j\) are A-orthogonal and defined over \(\mathbb {R}\).

We now wish to analyze \(\mathrm {H}(W^s)\) via the map \(\pi :\, h \mapsto h \sigma (h^{-1})\). However, for a technical reason related to the proof of Proposition 3.1 below, we must begin with the opposite map, \(\pi ^\prime : \, h \mapsto \sigma (h^{-1}) h\,\). Thus let \(M := \pi ^\prime (\mathrm {H}(W^s))\) and write \(\pi ^\prime :\, \mathrm {H} (W^s)\rightarrow M\,\). The toral semigroup \(T_+ := \exp (\mathrm {i} \mathfrak {t}_\mathbb {R})\,\cap \, \mathrm {H}(W^s)\) consists of those elements \(t \in \exp ( \mathrm {i} \mathfrak {t}_\mathbb {R})\) that act as contractions on \(V^*\), i.e., \(0 < \chi _j(t)^{-1} < 1\) for all \(j\,\). The restriction \(\pi ^\prime \vert _{T_+} = \pi \vert _{T_+}\) is, as indicated above, the squaring map \(t \mapsto t^2\); in particular we have \(T_+ \subset M\) and the set \(\{ g\, t g^{-1} \mid t \in T_+ \, , \, g \in \mathrm {Sp}_\mathbb {R} \}\) is likewise contained in \(M\,\).

In the sequel, we will often encounter the action of \(\mathrm {Sp}_ \mathbb {R}\) on \(T_+\) and M by conjugation. We therefore denote this action by a special name, \(\mathrm {Int}(g)\, t := g\, t g^{-1}\).

Proposition 3.1

\(M = \mathrm {Int}(\mathrm {Sp}_\mathbb {R}) T_+ \,\).

Proof

For \(g \in \mathrm {Sp}\) one has \(\sigma (g^{-1}) = C g^{-1} C^{-1} = s g^\dagger s\,\). Hence if \(M \ni m = \sigma (h^{-1})h\) with \(h \in \mathrm {H}(W^s)\), then \(m = s h^\dagger s h\,\). Consequently, \(\langle w, m w \rangle _s = \langle h w , h w \rangle _s < \langle w , w \rangle _s\) for all \(w \in W {\setminus} \{ 0 \}\). In particular, \(\langle w , m w \rangle _s\in \mathbb {R}\) and if \(w \not = 0\) is an m-eigenvector with eigenvalue \(\lambda \), it follows that \(\lambda \in \mathbb {R}\) and \(\lambda \langle w , w \rangle _s < \langle w , w \rangle _s \not = 0\,\).

Now we have \(C h C^{-1} = \sigma (h)\) and hence \(C m C^{-1} = m^{-1}\). As a result, if \(w \not = 0\) is an m-eigenvector with eigenvalue \(\lambda \), then so is C w with eigenvalue \(\lambda ^{-1}\). Since \(C s C^{-1} = - s\), the product of \(\langle w , w \rangle _s\) with \(\langle Cw , Cw \rangle _s\) is always negative. If \(\langle w , w \rangle _s > 0\) it follows that \(\lambda < 1\) and \(\lambda ^{-1} > 1\); if \(\langle Cw , Cw \rangle _s > 0\) then \(\lambda ^{-1} < 1\) and \(\lambda > 1\). In both cases \(0 < \lambda \not = 1\).

Since m does indeed have at least one eigenvector, we have constructed a complex 2-plane \(Q_1\) as the span of the linearly independent vectors w and Cw. The plane \(Q_1\) is defined over \(\mathbb {R}\) and, because \(0 \not = \langle w , w \rangle _s = A(Cw,w)\), it is A-non-degenerate. Its A-orthogonal complement \(Q_1^\perp \) is therefore also non-degenerate and defined over \(\mathbb {R}\).

The transformation \(m \in \mathrm {Sp}\) stabilizes the decomposition \(W = Q_1 \oplus Q_1^\perp \,\). Hence, proceeding by induction we obtain an A-orthogonal decomposition \(W = Q_1 \oplus \ldots \oplus Q_d\,\). Since the \(Q_j\) are m-invariant symplectic planes defined over \(\mathbb {R}\), there exists \(g \in \mathrm {Sp}_\mathbb {R}\) so that \(t := g m g^{-1}\) stabilizes the above T-invariant decomposition \(W = P_1 \oplus \ldots \oplus P_d\,\). Exchanging w with C w if necessary, we may assume that t acts diagonally on \(P_j = E_j \oplus F_j\) by \((e_j\, ,f_j) \mapsto (\lambda _j \, e_j \, , \lambda _j^{-1} f_j)\) with \(\lambda _j > 1\). In other words, \(t \in T_+\,\). \(\square \)

If we let

$$\begin{aligned} \mathrm {Sp}_\mathbb {R} T_+ \mathrm {Sp}_\mathbb {R} := \{g_1 t g_2^{-1} \mid g_1 , g_2 \in \mathrm {Sp}_\mathbb {R} \,, \, t \in T_+ \}\;, \end{aligned}$$

then we now have the following analog of the KAK-decomposition.

Corollary 3.1

The semigroup \(\mathrm {H}(W^s)\) decomposes as \(\mathrm {H}(W^s) = \mathrm {Sp}_\mathbb {R} T_+ \mathrm {Sp}_\mathbb {R}\,\) . In particular, \(\mathrm {H}(W^s)\) is connected.

Proof

By definition, \(\mathrm {H}(W^s) = {\pi ^\prime }^{-1}(M)\), and from Proposition 3.1 one has \(\mathrm {H}(W^s)= {\pi ^\prime }^{-1} (\mathrm {Int}(\mathrm {Sp}_\mathbb {R}) T_+)\). Now the map \(\pi ^\prime \vert _{T_+} : \, T_+ \rightarrow T_+ \,\), \(t\mapsto t^2\) is surjective. Therefore \({\pi ^\prime }^{-1}(T_+) = \mathrm {Sp}_\mathbb {R} T_+\,\), which is to say that each point in the fiber of \(\pi ^\prime \) over \(t \in T_+\) lies in the orbit of \(\sqrt{t} \in T_+\) generated by left multiplication with \(\mathrm {Sp}_\mathbb {R}\,\). On the other hand, by the property \(g \pi ^\prime (h) g^{-1}= \pi ^\prime (hg^{-1})\) of \(\mathrm {Sp}_\mathbb {R}\)-equivariance we have

$$\begin{aligned} \mathrm {Int}(\mathrm {Sp}_\mathbb {R}) T_+ = \mathrm {Int} (\mathrm {Sp}_\mathbb {R}) \pi ^\prime ( {\pi ^\prime }^{-1}(T_+) ) = \pi ^\prime ( {\pi ^\prime }^{-1}(T_+)\mathrm {Sp}_\mathbb {R} ) \end{aligned}$$

and hence \(\mathrm {H}(W^s) = {\pi ^\prime }^{-1}( \mathrm {Int} (\mathrm {Sp}_\mathbb {R}) T_+) = {\pi ^\prime }^{-1} (T_+) \mathrm {Sp}_\mathbb {R} = \mathrm {Sp}_\mathbb {R} T_+ \mathrm {Sp}_\mathbb {R} \,\).

Because \(\mathrm {Sp}_\mathbb {R}\) and \(T_+\) are connected, so is \(\mathrm {H}(W^s) = \mathrm {Sp}_\mathbb {R} T_+ \mathrm {Sp}_\mathbb {R} \,\). \(\square \)

It is clear that \(M = \mathrm {Int}( \mathrm {Sp}_\mathbb {R} ) T_+ \subset \mathrm {H}(W^s)\). Furthermore, since both \(T_+\) and \(\mathrm {Sp}_\mathbb {R}\) are invariant under the operation of Hermitian conjugation \(h\mapsto h^\dagger \) and under the involution \(h \mapsto s h s\), we have the following consequences.

Corollary 3.2

\(\mathrm {H}(W^s)\) is invariant under \(h \mapsto h^\dagger \) and also under \(h \mapsto s h s\) . In particular, \(\mathrm {H}(W^s)\) is stabilized by the map \(h \mapsto \sigma (h^{-1}) = s h^\dagger s\).

Remark 3.1

Letting \(h^\prime := \sigma (h)^{-1}\) one has \(\pi ^\prime (h) = \sigma (h)^{-1} h = h^\prime \sigma (h^\prime )^{-1} = \pi (h^\prime )\) and hence \(M = \pi ^\prime (\mathrm {H}(W^s)) = \pi (\mathrm {H}(W^s))\). The stability of \(\mathrm {H}(W^s)\) under \(h \mapsto \sigma (h)^{-1}\) was not immediate from our definition of \(\mathrm {H}(W^s)\), which is why we have been working from the viewpoint of \(\mathrm {H}(W^s) = {\pi ^\prime }^{-1}(M)\) so far. Now that we have it, we may regard \(\mathrm {H}(W^s)\) as the total space of an \(\mathrm {Sp}_\mathbb {R}\)-principal bundle \(\pi :\, \mathrm {H}(W^s) \rightarrow M\). We are going to see in Corollary 3.3 that this principal bundle is trivial.

Next observe that, since \(\sigma (m) = m^{-1}\) for \(m = \sigma (h)^{-1} h = h^\prime \sigma (h^\prime )^{-1} \in M\,\), the maps \(\pi : \, M \rightarrow M\) and \(\pi ^\prime : \, M \rightarrow M\) coincide and are just the square \(m \mapsto m^2\). Thus the claim that the elements of M have a unique square root in M can be formulated as follows.

Proposition 3.2

The squaring map \(\pi = \pi ^\prime : \, M \rightarrow M\) is bijective.

Proof

Recall from Proposition 3.1 that every \(m \in M\) is diagonalizable in the sense that \(M = \mathrm {Int}( \mathrm {Sp}_\mathbb {R}) T_+\,\). Since \(\pi : \, T_+\rightarrow T_+\) is surjective, the surjectivity of \(\pi : \, M \rightarrow M\) is immediate. For the injectivity of \(\pi \) we note that the m-eigenspace with eigenvalue \(\lambda \) is contained in the \(m^2\)-eigenspace with eigenvalue \(\lambda ^2\). The result then follows from the fact that all eigenvalues of m are positive real numbers. \(\square \)

Corollary 3.3

Each of the two \(\mathrm {Sp}_\mathbb {R}\) -equivariant maps \(\mathrm {Sp}_\mathbb {R} \times M \rightarrow \mathrm {H} (W^s)\) defined by \((g,m) \mapsto gm\) and by \((g,m) \mapsto m g^{-1}\) , is a bijection.

Proof

Consider the map \((g,m) \mapsto gm\,\). Surjectivity is evident from Corollary 3.1 and \(M = \mathrm {Int}(\mathrm {Sp}_\mathbb {R}) T_+\,\). For the injectivity it suffices to prove that if \(m_1 , m_2 \in M\) and \(g \in \mathrm {Sp}_\mathbb {R}\) with \(g m_1 = m_2\,\), then \(m_1 = m_2\,\). But this follows directly from \(\pi ^\prime (m_1) = \pi ^\prime (g m_1) = \pi ^\prime (m_2)\) and the fact that \(\pi ^\prime \vert _M\) is the bijective squaring map.

The proof for the map \((g,m) \mapsto m g^{-1}\) is similar, with \(\pi \) replacing \(\pi ^\prime \). \(\square \)

3.1.3 Cone realization of M

Let us look more carefully at M as a geometric object. First, as we have seen, the elements m of M satisfy the condition \(m = \sigma (m^{-1})\). We regard \(\psi :\, \mathrm {H}(W^s) \rightarrow \mathrm {H}(W^s) \,\), \(h \mapsto \sigma (h^{-1})\), as an anti-holomorphic involution and reformulate this condition as \(M \subset \mathrm {Fix} (\psi )\). In the present section we are going to show that M is a closed, connected, real-analytic submanifold of \(\mathrm {H}(W^s)\) which locally agrees with \(\mathrm {Fix} (\psi )\). This implies in particular that M is totally real in \(\mathrm {H}(W^s)\) with \(\dim _\mathbb {R} M = \dim _\mathbb {C} \mathrm {H}(W^s)\). We will also show that the exponential map identifies M with a precisely defined open cone in \(\mathrm {i} \mathfrak {sp}_\mathbb {R}\,\). We begin with the following statement.

Lemma 3.1

The image M of \(\pi \) is closed as a subset of \(\mathrm {H}(W^s)\).

Proof

Let \(h \in \mathrm {cl}(M) \subset \mathrm {H} (W^s)\). By the definition of M, we still have \(h \sigma (h)^{-1} =: m \in M\,\). If \(h_n\) is any sequence from M with \(h_n \rightarrow h\,\), then \(h_n \sigma (h_n)^{-1} \rightarrow m\,\). But m has a unique square root \(\sqrt{m} \in M\) and \(h_n = \sigma (h_n)^{-1} \rightarrow \sqrt{m} \,\). Hence \(h = \sqrt{m} \in M\,\). \(\square \)

Remark 3.2

M of course fails to be closed as a subset of \(\mathrm {Sp}\,\). For example, \(g = \mathrm {Id}\) is in the closure of \(M \subset \mathrm {Sp}\) but is not in \(M\,\).

Lemma 3.2

The exponential map \(\exp : \, \mathfrak {sp} \rightarrow \mathrm {Sp}\) has maximal rank along \(\mathfrak {t}_+\,\).

Proof

We are going to use the fact that the squaring map \(S : \, \mathrm {Sp} \rightarrow \mathrm {Sp}\,\), \(g \mapsto g^2\), is a local diffeomorphism of \(\mathrm {Sp}\) at any point \(t \in T_+\,\). To show this, we compute the differential of S at t and obtain

$$\begin{aligned} D_t S = dL_{\,t^2} \circ (\mathrm {Id}_\mathfrak {sp} + \mathrm {Ad}(t^{-1})) \circ dL_{\,t^{-1}}, \end{aligned}$$

where \(dL_g\) denotes the differential of the left translation \(L_g : \,\mathrm {Sp}\rightarrow \mathrm {Sp}\,\), \(g_1 \mapsto g g_1\,\). The middle map \(\mathrm {Id}_\mathfrak {sp} + \mathrm {Ad}(t^{-1}) : \, \mathfrak {sp} \rightarrow \mathfrak {sp}\) is regular because all of the eigenvalues of \(t \in T_+\) are positive real numbers. Since \(dL_{\, t^{-1}} : \, T_t \mathrm {Sp} \rightarrow \mathfrak {sp}\) and \(dL_{\,t^2} : \, \mathfrak {sp} \rightarrow T_{t^2} \mathrm {Sp}\) are isomorphisms, it follows that \(D_t S :\, T_t \mathrm {Sp} \rightarrow T_{t^2} \mathrm {Sp}\) is an isomorphism.

Turning to the proof of the lemma, given \(\xi \in \mathfrak {t}_+\) we now choose \(n \in \mathbb {N}\) so that \(2^{-n} \xi \) is in a neighborhood of \(0 \in \mathfrak {sp}\) where \(\exp \) is a diffeomorphism. It follows that for U a sufficiently small neighborhood of \(\xi \,\), the exponential map expressed as

$$\begin{aligned} U \ni \eta \mapsto 2^{-n} \eta \mapsto \exp (2^{-n}\eta ) \mathop {\mapsto }\limits ^{S^n}(\exp (2^{-n}\eta ))^{2^n} = \exp (\eta ) \end{aligned}$$

is a diffeomorphism of U onto its image. \(\square \)

Now recall \(M = \mathrm {Int}(\mathrm {Sp}_\mathbb {R}) T_+\) and consider the cone

$$\begin{aligned} \mathcal {C} := \mathrm {Ad}(\mathrm {Sp}_\mathbb {R}) \mathfrak {t}_+ \subset \mathrm {i}\mathfrak {sp}_\mathbb {R}. \end{aligned}$$

It follows from the equivariance of \(\exp \), i.e., \(\exp (\mathrm {Ad} (g) \xi ) = \mathrm {Int}(g) \exp (\xi )\), that \(\exp : \, \mathcal {C} \rightarrow \mathrm {Int}(\mathrm {Sp}_\mathbb {R}) T_+ = M\) is surjective. Furthermore, \(\exp \vert _{\mathfrak {t}_+} : \, \mathfrak {t}_+ \rightarrow T_+\) is injective and for every \(\xi \in \mathfrak {t}_+\) the isotropy groups of the \(\mathrm {Sp}_\mathbb {R}\)-actions at \(\xi \) and \(\exp (\xi )\) are the same. Therefore \(\exp : \, \mathcal {C} \rightarrow M\) is also injective.

In fact, much stronger regularity holds. For the statement of this result we recall the anti-holomorphic involution \(\psi :\, \mathrm {H} (W^s) \rightarrow \mathrm {H}(W^s)\) defined by \(h \mapsto \sigma (h^{-1})\) and let \(\mathrm {Fix}(\psi )^0\) denote the connected component of \(\mathrm {Fix} (\psi )\) that contains M.

Proposition 3.3

The image \(M \subset \mathrm {H}(W^s)\) of \(\pi : \, h \mapsto h \sigma (h^{-1})\) is the closed, connected, totally real submanifold \(\mathrm {Fix}(\psi )^0\), which is half-dimensional in the sense that \(\dim _\mathbb {R} M = \dim _\mathbb {C} \mathrm {H} (W^s)\). The set \(\mathcal {C} = \mathrm {Ad}(\mathrm {Sp}_\mathbb {R}) \mathfrak {t}_+\,\), which is an open positive cone in \(\mathrm {i} \mathfrak {sp}_\mathbb {R}\,\), is in bijection with M by the real-analytic diffeomorphism \(\exp : \, \mathcal {C} \rightarrow M\).

Proof

Lemma 3.2 implies that \(\mathfrak {t}_+\) possesses an open neighborhood U in \(\mathrm {i} \mathfrak {sp}_\mathbb {R}\) so that \(\exp \vert _U\) is everywhere of maximal rank. Because \(T_+\) lies in \(\mathrm {H}( W^s)\) and \(\mathrm {H}(W^s)\) is open in \(\mathrm {Sp} \,\), by choosing U small enough we may assume that \(\exp (\frac{1}{2} U) \subset \mathrm {H} (W^s)\) and therefore that \(\exp (U) \subset M = \exp (\mathcal {C})\). Since \(\exp \) is a local diffeomorphism on U, we may also assume that \(U \subset \mathcal {C} = \mathrm {Ad} (\mathrm {Sp}_\mathbb {R}) \mathfrak {t}_+\,\), and it then follows that \(\mathcal {C} = \mathrm {Ad}(\mathrm {Sp}_\mathbb {R}) U\). In particular, this shows that \(\mathcal {C}\) is open in \(\mathrm {i}\mathfrak {sp}_\mathbb {R}\,\). In summary,

$$\begin{aligned} \mathcal {C} = \mathrm {Ad}(\mathrm {Sp}_\mathbb {R}) U \mathop {\longrightarrow }\limits ^{\exp } \mathrm {Int}(\mathrm {Sp}_\mathbb {R}) \exp (U) = M \subset \mathrm {Fix}(\psi )^0. \end{aligned}$$

By the equivariance of \(\exp \,\), we also know that it is everywhere of maximal rank on \(\mathcal {C}\,\).

Now \(\psi \) is an anti-holomorphic involution. Therefore, \(\mathrm {Fix}(\psi )^0\) is a totally real, half-dimensional closed submanifold of \(\mathrm {H}(W^s)\), and since \(\mathcal {C}\) is open in \(\mathrm {i} \mathfrak {sp}_\mathbb {R}\,\), we also know that \(\dim _\mathbb {C} \mathcal {C} = \dim _\mathbb {R} \mathrm {Fix}(\psi )^0\). The maximal rank property of \(\exp \) then implies that \(M = \mathrm {im}(\exp : \mathcal {C} \rightarrow \mathrm {Fix} (\psi )^0)\) is open in \(\mathrm {Fix} (\psi )^0\). In Lemma 3.1 it was shown that M is closed in \(\mathrm {H} (W^s)\). Thus it is open and closed in the connected manifold \(\mathrm {Fix}(\psi )^0\), and consequently \(\exp : \, \mathcal {C} \rightarrow M = \mathrm {Fix}(\psi )^0\) is a local diffeomorphism of manifolds. Since we already know that \(\exp : \, \mathcal {C} \rightarrow M\) is bijective, the desired result follows. \(\square \)

Corollary 3.4

The two identifications \(\mathrm {Sp}_\mathbb {R} \times M = \mathrm {H}(W^s)\) defined by \((g,m) \mapsto gm\) and \((g,m) \mapsto m g^{-1}\) are real-analytic diffeomorphisms. The fundamental group of \(\mathrm {H}(W^s)\) is isomorphic to \(\pi _1(\mathrm {Sp}_\mathbb {R}) \simeq \mathbb {Z}\,\).

Proof

The first statement is proved by explicitly constructing a smooth inverse to each of the two maps. For this let \(g^\prime m^\prime = m g^{-1} = h \in \mathrm {H}(W^s)\) and note that \(m = \sqrt{\pi (h)}\) and \(m^\prime = \sqrt{\pi ^\prime (h)}\). Since the square root is a smooth map on \(M\,\), a smooth inverse in the two cases is defined by

$$\begin{aligned} h \mapsto ( h \sqrt{\pi ^\prime (h)}^{\ -1} , \sqrt{\pi ^\prime (h)}), \quad \text {resp.} \quad h \mapsto ( h^{-1} \sqrt{\pi (h)} , \sqrt{\pi (h)} ). \end{aligned}$$

The second statement follows from \(\mathcal {C} \simeq M\) by \(\exp \) and the fact that \(\mathrm {Sp}_\mathbb {R}\) is a product of a cell and a maximal compact subgroup \(K\,\). We choose K to be the unitary group \(\mathrm {U} = \mathrm {U}(V, \langle \, ,\,\rangle )\) acting diagonally on \(W = V \oplus V^*\) and recall that \(\pi _1(\mathrm {U}) \simeq \mathbb {Z}\,\). \(\square \)

3.2 Oscillator semigroup and metaplectic group

Recall that we are concerned with the Lie algebra representation of \(\mathfrak {sp}_\mathbb {R} \subset \mathfrak {sp}\) which is defined by the identification of \(\mathfrak {sp}\) with the set of symmetrized elements of degree two in the Weyl algebra \(\mathfrak {w}(W)\) and the representation of \(\mathfrak {w}(W)\) on \(\mathfrak {a}(V)\). In Sect. 3.4 we construct the oscillator representation of the metaplectic group \(\mathrm {Mp}\,\), a 2:1 cover of \(\mathrm {Sp}_\mathbb {R}\,\), which integrates this Lie algebra representation. Observe that since \(\pi _1(\mathrm {Sp}_\mathbb {R}) \simeq \mathbb {Z}\) and \(\mathbb {Z}\) has only one subgroup of index two, there is a unique such covering \(\tau : \, \mathrm {Mp} \rightarrow \mathrm {Sp}_\mathbb {R}\,\). The method of construction [11] we use first yields a representation of the 2:1 covering space \(\widetilde{\mathrm {H}} (W^s)\) of \(\mathrm {H}(W^s)\) and then realizes the oscillator representation of \(\mathrm {Mp}\) by taking limits that correspond to going to \(\mathrm {Sp}_\mathbb {R}\) in the boundary of \(\mathrm {H}(W^s)\). This representation of the oscillator semigroup \(\widetilde{\mathrm {H}}(W^s)\) is for our purposes at least as important as the representation of \(\mathrm {Mp}\,\).

The goal of the present section is to lift all essential structures on \(\mathrm {H}(W^s)\) to \(\widetilde{\mathrm {H}}(W^s)\).

3.2.1 Lifting the semigroup

We begin by recalling a few basic facts about covering spaces. If G is a connected Lie group, its universal covering space U carries a canonical group structure: an element \(u \in U\) in the fiber over \(g \in G\) is a homotopy class \(u \equiv [\alpha _g]\) of paths \(\alpha _g : \, [0,1] \rightarrow G\) connecting g with the neutral element \(e \in G\,\); and an associative product \(U \times U \rightarrow U\), \((u_1,u_2) \mapsto u_1 u_2\,\), is defined by taking \(u_1 u_2\) to be the unique homotopy class which is given by pointwise multiplication of any two paths representing the homotopy classes \(u_1, u_2\,\). The fundamental group \(\pi _1(G) \equiv \pi _1(G,e)\) acts on U by monodromy, i.e., if \([\alpha _g] = u \in U\) and \([c] = \gamma \in \pi _1(G)\), then one sets \(\gamma (u) := [ \alpha _g *c] \in U\) where \(\alpha _g *c\) is the path from g to e which is obtained by composing the path \(\alpha _g\) with the loop c based at \(e\,\). This \(\pi _1(G)\)-action satisfies the compatibility condition \(\gamma _1 (u_1) \gamma _2(u_2) = (\gamma _1 \gamma _2)(u_1 u_2)\) and in that sense is central.

The situation for our semigroup \(\mathrm {H} (W^s)\) is analogous except for the minor complication that the distinguished point \(e = \mathrm {Id}\) does not lie in \(\mathrm {H}(W^s)\) but lies in the closure of \(\mathrm {H}(W^s)\). Hence, by the same principles, the universal cover U of \(\mathrm {H}(W^s)\) comes with a product operation and there is a central action of \(\pi _1(\mathrm {H} (W^s))\) on U. Moreover, the product \(U \times U \rightarrow U\) still is associative. To see this, first notice that the subsemigroup \(T_+ \subset \mathrm {H}(W^s)\) is simply connected and as such is canonically embedded in U. Then for \(u_1, u_2, u_3 \in U\) observe that \(u_1 (u_2 u_3) = \gamma ((u_1 u_2) u_3)\) where \(\gamma \in \pi _1 (\mathrm {H}(W^s))\) could theoretically depend on the \(u_j\,\). However, any such dependence has to be continuous and the fundamental group is discrete, so in fact \(\gamma \) is independent of the \(u_j\) and, since \(\gamma \) is the identity when the \(u_j\) are in \(T_+\) (lifted to U), the associativity follows.

Let now \(\Gamma \simeq 2\mathbb {Z}\) denote the subgroup of index two in \(\pi _1 (\mathrm {H}(W^s)) \simeq \mathbb {Z}\) and consider \(\widetilde{ \mathrm {H}} (W^s) := U/\Gamma \), which is our object of interest. Since the \(\Gamma \)-action on U is central, i.e., \(\gamma _1 (u_1) \gamma _2 (u_2) = (\gamma _1 \gamma _2) (u_1 u_2)\) for all \(\gamma _1 , \gamma _2 \in \Gamma \) and \(u_1 , u_2 \in U\), the product \(U \times U \rightarrow U\) descends to a product \(U/\Gamma \times U/\Gamma \rightarrow U/\Gamma \,\). Thus \(U/ \Gamma = \widetilde{\mathrm {H}} (W^s)\) is a semigroup, and the situation at hand is summarized by the following statement.

Proposition 3.4

The 2:1 covering \(\tau _H : \, U / \Gamma = \widetilde{\mathrm {H}} (W^s) \rightarrow \mathrm {H}(W^s)\), \([\alpha _h]\, \Gamma \mapsto h\,\), is a homomorphism of semigroups.

3.2.2 Actions of the metaplectic group

Recall that we have two 2:1 coverings: a homomorphism of groups \(\tau :\, \mathrm {Mp} \rightarrow \mathrm {Sp}_\mathbb {R}\,\), along with a homomorphism of semigroups \(\tau _H : \, \widetilde{\mathrm {H}}(W^s) \rightarrow \mathrm {H}(W^s)\). Now, a pair of elements \((g^\prime ,g) \in \mathrm {Sp}_\mathbb {R} \times \mathrm {Sp}_\mathbb {R}\) determines a transformation \(h \mapsto g^\prime h g^{-1}\) of \(\mathrm {H} (W^s)\), and by the homotopy lifting property of covering maps a corresponding action of \(\mathrm {Mp} \times \mathrm {Mp}\) on \(\widetilde{ \mathrm {H}}(W^s)\) is obtained as follows.

Consider the canonical mapping \(\mathrm {Mp} \times M \rightarrow \mathrm {Sp}_\mathbb {R} \times M\) given by \(\tau \,\). By the real-analytic diffeomorphism \(\mathrm {Sp}_\mathbb {R} \times M \rightarrow \mathrm {H} (W^s)\), \((g,m) \mapsto g m\,\), this map can be regarded as a 2:1 covering of \(\mathrm {H}(W^s)\), and since any two 2:1 coverings are isomorphic we get an identification of the covering space \(\widetilde{\mathrm {H}}(W^s)\) with \(\mathrm {Mp} \times M\,\). Moreover, the action of the group \(\mathrm {Mp}\) on itself by left translations induces on \(\mathrm {Mp} \times M \simeq \widetilde{\mathrm { H}} (W^s)\) an \(\mathrm {Mp}\)-action which, by construction, satisfies the relation

$$\begin{aligned} \tau _H(g \cdot h) = \tau (g) \tau _H(h) \quad (g \in \mathrm {Mp} \, , \,\, h \in \widetilde{\mathrm {H}}(W^s) ). \end{aligned}$$

This can be viewed as a statement of \(\mathrm {Mp}\)-equivariance of the covering map \(\tau _H\,\).

Now, we have another real-analytic diffeomorphism \(\mathrm {Sp}_\mathbb {R} \times M \rightarrow \mathrm {H}(W^s)\) by \((g,m) \mapsto m g^{-1}\), which transfers left translation in \(\mathrm {Sp}_\mathbb {R}\) to right multiplication on \(\mathrm {H}(W^s)\), and by using it we can repeat the above construction. The result is another identification \(\widetilde{ \mathrm {H}}(W^s) \simeq \mathrm {Mp} \times M\) and another \(\mathrm {Mp}\)-action on \(\widetilde{\mathrm {H}}(W^s)\). Altogether we then have two actions of \(\mathrm {Mp}\) on \(\widetilde{\mathrm {H}} (W^s)\). The essence of the next statement is that they commute.

Proposition 3.5

There is a real-analytic action \((g_1,g_2) \mapsto (g_1,g_2) \cdot h\) of \(\, \mathrm {Mp} \times \mathrm {Mp}\) on \(\widetilde{\mathrm {H}} (W^s)\) such that the covering \(\tau _H : \, \widetilde{\mathrm {H}} (W^s) \rightarrow \mathrm {H} (W^s)\) is \((\mathrm {Mp} \times \mathrm {Mp})\) -equivariant:

$$\begin{aligned} \tau _H( (g_1,g_2) \cdot h) = \tau (g_1) \tau _H(h) \tau (g_2)^{-1}\;. \end{aligned}$$

Proof

By construction, the stated equivariance property of \(\tau _H\) holds for each of the two actions of \(\mathrm {Mp}\) separately. It then follows that it holds for all \((g_1,g_2) \in \mathrm {Mp} \times \mathrm {Mp}\) if the two actions commute. But by \(\tau _H( (g_1,e) \cdot h) = \tau (g_1) \tau _H(h)\) and \(\tau _H( (e,g_2) \cdot h) = \tau _H(h) \tau (g_2)^{-1}\) the commutator

$$\begin{aligned} g := (g_1,e) (e,g_2) (g_1,e)^{-1} (e,g_2)^{-1} \end{aligned}$$

acts trivially on \(\mathrm {H}(W^s)\) by \(\tau _H\,\), i.e., \(\tau _H(g \cdot h) = \tau _H(h)\). Therefore g can be regarded as being in the covering group \(\tau ^{-1}(\mathrm {Id}) = \mathbb {Z}_2\) of the covering \(\tau : \, \mathrm {Mp} \rightarrow \mathrm {Sp}_\mathbb {R}\,\). Since we can connect both \(g_1\) and \(g_2\) to the identity \(e \in \mathrm {Mp}\) by a continuous curve, it follows from the discreteness of \(\mathbb {Z}_2\) that \(g \in \mathrm {Mp} \times \mathrm {Mp}\) acts trivially on \(\widetilde{\mathrm {H}}(W^s)\). \(\square \)

Notice that since the submanifold \(M \subset \mathrm {H} (W^s)\) is simply connected, there exists a canonical lifting of M (which we still denote by M) to the cover \(\widetilde{\mathrm {H}}(W^s)\); this is the unique lifting by which \(T_+ \subset M\) is embedded as a subsemigroup in \(\widetilde{\mathrm {H}}(W^s)\). Proposition 3.5 then allows us to write \(\widetilde{ \mathrm {H}}(W^s) = \mathrm {Mp}.M.\mathrm {Mp}\).

3.2.3 Lifting involutions

Let us now turn to the issue of lifting the various involutions at hand. As a first remark, we observe that any Lie group automorphism \(\varphi : \, \mathrm {Sp}_\mathbb {R} \rightarrow \mathrm {Sp}_\mathbb {R}\) uniquely lifts to a Lie group automorphism \(\widetilde{\varphi }\) of the universal covering group \(\widetilde{\mathrm {Sp}}_\mathbb {R}\,\), and the latter induces an automorphism of the fundamental group \(\pi _1(\mathrm {Sp}_\mathbb {R}) \simeq \mathbb {Z}\) viewed as a subgroup of the center of \(\widetilde{\mathrm {Sp}}_\mathbb {R}\,\). Now \(\mathrm {Aut}( \pi _1( \mathrm {Sp}_\mathbb {R} )) \simeq \mathrm {Aut} (\mathbb {Z}) \simeq \mathbb {Z}_2\) and both elements of this automorphism group stabilize the subgroup \(\Gamma \simeq 2\mathbb {Z}\) in \(\pi _1(\mathrm {Sp}_\mathbb {R})\). Therefore \(\widetilde{ \varphi }\) induces an automorphism of \(\mathrm {Mp} = \widetilde{\mathrm {Sp} }_\mathbb {R} / \Gamma \,\).

Since the operation \(h\mapsto h^{-1}\) canonically lifts from \(\mathrm {Sp}_\mathbb {R}\) to \(\mathrm {Mp}\) and \(h \mapsto (h^{-1} )^\dagger \) is a Lie group automorphism of \(\mathrm {Sp}_\mathbb {R}\,\), it follows that Hermitian conjugation \(h\mapsto h^\dagger \) has a natural lift to \(\mathrm {Mp}\,\). The same goes for the Lie group automorphism \(h \mapsto s h s\) of \(\mathrm {Sp}_\mathbb {R}\,\).

Proposition 3.6

Hermitian conjugation \(h \mapsto h^\dagger \) and the involution \(h \mapsto s h s\) lift to unique maps with the property that they stabilize the lifted manifold M. In particular, the basic anti-holomorphic map \(\psi :\, \mathrm {H}(W^s) \rightarrow \mathrm {H}(W^s)\), \(h \mapsto \sigma (h^{-1})= s h^\dagger s\) lifts to an anti-holomorphic map \(\widetilde{\psi }: \, \widetilde{\mathrm {H}} (W^s)\rightarrow \widetilde{\mathrm {H}}(W^s)\) which is the identity on M and \(\mathrm {Mp} \times \mathrm {Mp}\) -equivariant in that \(\widetilde{\psi } (g_1 x\, g_2^{-1}) = g_2 \widetilde{\psi }(x) g_1^{-1}\) for all \(g_1, g_2\in \mathrm {Mp}\) and \(x \in \widetilde{\mathrm {H}}(W^s)\).

Proof

Recall that the simply connected space \(M \subset \mathrm {H} (W^s)\) has a canonical lifting (still denoted by M) to \(\widetilde{ \mathrm {H}}(W^s)\). Since all of our involutions stabilize M as a submanifold of \(\mathrm {H}(W^s)\), they are canonically defined on the lifted manifold \(M\,\). In particular, the involution \(\psi \) on M is the identity map, and therefore so is the lifted involution \(\widetilde{\psi }\).

Note furthermore that the involution defined by \(h \mapsto s h s\) is holomorphic on \(\mathrm {H}(W^s)\) and that the other two are anti-holomorphic. Now \(\widetilde{\mathrm {H}}(W^s)\) is connected and the lifted version of M is a totally real submanifold of \(\widetilde{\mathrm {H}}(W^s)\) with \(\dim _\mathbb {R} M = \dim _\mathbb {C} \widetilde{ \mathrm {H}}(W^s)\). In such a situation the identity principle of complex analysis implies that there exists at most one extension (holomorphic or anti-holomorphic) of an involution from M to \(\widetilde{ \mathrm {H}}(W^s)\). Therefore, it is enough to prove the existence of extensions.

Since \(h \in \widetilde{\mathrm {H}}(W^s)\) is uniquely representable as \(h = g m\) with \(g \in \mathrm {Mp}\) and \(m \in M\), the involution \(h \mapsto h^\dagger \) is extended by \(gm \mapsto (gm)^\dagger = m^\dagger g^\dagger \). Similarly, \(h \mapsto s h s\) extends by \(gm \mapsto (sgs)(sms)\), and \(h \mapsto s h^\dagger s\) does so by the composition of the other two.

The equivariance property of \(\widetilde{\psi }\) follows from the fact that \(g \mapsto s g^\dagger s\) on \(\mathrm {Mp}\) coincides with the operation of taking the inverse, \(g \mapsto g^{-1}\). \(\square \)

3.3 Oscillator semigroup representation

Here we construct the fundamental representation of the semigroup \(\widetilde{\mathrm {H}}(W^s)\) on the Hilbert space \(\fancyscript {A}_V\), which in the present context we call Fock space. Our approach is parallel to that of Howe [11]: the Fock space we use is related to the \(L^2\)-space of Howe’s work by the Bargmann transform [9]. (Using the language of physics one would say that Howe works with the position space wave function while our treatment relies on the phase space wave function.) In particular, following Howe we take advantage of a realization of \(\mathrm {H} (W^s)\) as the complement of a certain determinantal variety in the Siegel upper half plane.

3.3.1 Cayley transformation

Let us begin with some background information on the Cayley transformation, which is defined to be the meromorphic mapping

$$\begin{aligned} C : \,\, \mathrm {End}(W) \rightarrow \mathrm {End}(W), \quad g \mapsto \frac{\mathrm {Id}_W + g}{\mathrm {Id}_W - g} \,. \end{aligned}$$

If \(g \in \mathrm {Sp}\,\), then from \(A(gw , gw^\prime ) = A(w , w^\prime )\) we have

$$\begin{aligned} A((\mathrm {Id}_W + g)w \, , (\mathrm {Id}_W - g)w') + A((\mathrm {Id}_W - g) w \, , (\mathrm {Id}_W + g)w') = 0 \end{aligned}$$

for all \(w , w' \in W\). By assuming that \((\mathrm {Id}_W - g)\) is invertible and then replacing w and \(w'\) by \((\mathrm {Id}_W - g)^{-1}w\) resp. \((\mathrm {Id}_W - g)^{-1} w'\), we see that C maps the complement of the determinantal variety \(\{ g \in \mathrm {Sp} \mid \mathrm {Det}(\mathrm {Id}_W - g) = 0\}\) into \(\mathfrak {sp}\,\).

The inverse of the Cayley transformation is given by

$$\begin{aligned} g = C^{-1}(X) = \frac{X - \mathrm {Id}_W}{X + \mathrm {Id}_W} \,. \end{aligned}$$

Reversing the above argument, one shows that if \((X + \mathrm {Id}_W)\) is invertible and \(X \in \mathfrak {sp}\,\), then \(C^{-1}(X) \in \mathrm {Sp}\,\). Moreover, by the relation \(X + \mathrm {Id}_W = 2 (\mathrm {Id}_W - g)^{-1}\) for \(C(g) = X\,\), if \(\mathrm {Id}_W - g\) is regular, then so is \(X + \mathrm {Id}_W\,\), and vice versa. Thus if we introduce the sets

$$\begin{aligned} D_{\mathrm {Sp}} := \{g \in \mathrm {Sp} \mid \mathrm {Det}( \mathrm {Id}_W - g) = 0 \}\;, \quad D_{\mathfrak {sp}} := \{X \in \mathfrak {sp} \mid \mathrm {Det}(X + \mathrm {Id}_W) = 0 \}\;, \end{aligned}$$

the following is immediate.

Proposition 3.7

The Cayley transformation defines a bi-holomorphic map

$$\begin{aligned} C:\,\, \mathrm {Sp} {\setminus} D_{\mathrm {Sp}}\rightarrow \mathfrak {sp} {\setminus} D_{\mathfrak {sp}}. \end{aligned}$$

Now we consider the restriction of C to the semigroup \(\mathrm {H} (W^s)\). Letting \(\dagger \) be the Hermitian conjugation of the previous section, denote by \(\mathfrak {Re}(X) = \frac{1}{2}(X+ X^\dagger )\) the real part of an operator \(X \in \mathrm {End}(W)\) and define the associated Siegel upper half space \(\mathfrak {S}\) to be the subset of elements \(X \in \mathrm {End}(W)\) which are symmetric with respect to the canonical symmetric bilinear form S on \(W = V \oplus V^*\) with \(\mathfrak {Re} (X) > 0\,\). Notice the relations \(S(w,w^\prime ) = A(w,sw^\prime )\) and \(A(sw,sw^\prime ) = - A(w,w^\prime )\), from which it is seen that X is symmetric if and only if \(sX \in \mathfrak {sp}\,\). Define \(D_\mathfrak {S} := \{X\in \mathfrak {S}\mid \mathrm {Det}(sX + \mathrm {Id}_W) = 0 \}\), let

$$\begin{aligned} \zeta ^+_s := \mathfrak {S} {\setminus} D_\mathfrak {S} \,, \end{aligned}$$

and define a slightly modified Cayley transformation by

$$\begin{aligned} a(g) := s \, \frac{\mathrm {Id}_W + g}{\mathrm {Id}_W - g}\,. \end{aligned}$$

Translating Proposition 3.7, it follows that a defines a bi-holomorphic map from \(\mathrm {Sp} {\setminus} D_{\mathrm {Sp}}\) onto the set of S-symmetric operators with \(D_\mathfrak {S}\) removed.

Proposition 3.8

The modified Cayley transformation \(a : \, \mathrm {Sp} {\setminus} D_{\mathrm {Sp}} \rightarrow \mathrm {End}(W)\) given by \(g \mapsto s \, (\mathrm {Id}_W + g) (\mathrm {Id}_W - g)^{-1}\) restricts to a bi-holomorphic map

$$\begin{aligned} a : \,\, \mathrm {H}(W^s) \rightarrow \zeta ^+_s\,. \end{aligned}$$

This result is an immediate consequence of the following identity.

Lemma 3.3

For \(g \in \mathrm {Sp} {\setminus} D_{\mathrm {Sp}}\) let \(a(g) = X\) and define \(Y := (sX + \mathrm {Id}_W)^{-1}\). Then

$$\begin{aligned} {\frac{1}{2}} ( s - g^\dagger s g ) = Y^\dagger (X + X^\dagger ) Y\,. \end{aligned}$$

In particular, one has the following equivalence:

$$\begin{aligned} \big ( \mathfrak {Re}(X) > 0 \;\; \text {and} \;\; \mathrm {Det}(sX + \mathrm {Id}_W) \not = 0 \big ) ~ \Leftrightarrow ~ s - g^\dagger s g > 0. \end{aligned}$$

Proof

It is convenient to rewrite \(s - g^\dagger s g\) as

$$\begin{aligned} s - g^\dagger s g = {\textstyle {\frac{1}{2}}} (\mathrm {Id}_W - g^\dagger ) s (\mathrm {Id}_W + g)+ {\textstyle {\frac{1}{2}}}(\mathrm {Id}_W + g^\dagger ) s (\mathrm {Id}_W - g)\,. \end{aligned}$$

Using \(a(g) = X\) one directly computes that

$$\begin{aligned} {\textstyle {\frac{1}{2}}}(\mathrm {Id}_W - g) = (sX + \mathrm {Id}_W)^{-1} \quad \text {and} \quad {\textstyle {\frac{1}{2}}}(\mathrm {Id}_W + g) = (sX + \mathrm {Id}_W)^{-1} sX \,. \end{aligned}$$

The desired identity follows by inserting these relations in the previous equation. \(\square \)

Remark 3.3

The modified Cayley transformation intertwines the anti-holomorphic involution \(\psi : \, h \mapsto s h^\dagger s\) with the operation of taking the Hermitian conjugate \(X \mapsto X^\dagger \):

$$\begin{aligned} a(\psi (h)) = a(h)^\dagger. \end{aligned}$$

Since \(\zeta _{\,s}^+\) is obviously stable under Hermitian conjugation, this is another proof of the stability of \(\mathrm {H}(W^s)\) under the involution \(\psi \,\); cf. Corollary 3.2.

3.3.2 Construction of the semigroup representation

Let us now turn to the main goal of this section. Recall that we have a Lie algebra representation of \(\mathfrak {sp}\) on \(\mathfrak {a}(V) = \mathrm {S}(V^*)\) which is defined by its canonical embedding in \(\mathfrak {w}_2(W)\). We shall now construct the corresponding representation of the semigroup \(\widetilde{\mathrm {H}}(W^s)\) on the Fock space \(\fancyscript {A}_V\).

It will be seen later that the character of this representation on the lifted toral semigroup \(T_+\) is \(\mathrm {Det}^{-\frac{1}{2}}(s - sh)\). This extends to \(M = \mathrm {Int}(\mathrm {Mp}) T_+\) by the invariance of the character with respect to the conjugation action of \(\mathrm {Mp}\,\). Since \(\widetilde{\mathrm {H}}(W^s)\) is connected and M is totally real of maximal dimension in \(\widetilde{\mathrm {H}} (W^s)\), the identity principle then implies that if a semigroup representation of \(\widetilde{\mathrm {H}}(W^s)\) can be constructed with a holomorphic character, this character must be given by the square root function \(h \mapsto \mathrm {Det}^{-\frac{1}{2}}(s - sh)\).

There is no difficulty discussing the square root on the simply connected submanifold M. However, in order to make sense of the square root function on the full semigroup, we must lift all considerations to \(\widetilde{\mathrm {H}}(W^s)\). For convenience of notation, given \(h \in \mathrm {H}(W^s)\) we let \(a_h := a(h)\), and for \(x \in \widetilde{\mathrm {H}}(W^s)\) we simply write \(a_x \equiv a(\tau _H(x))\) where \(\tau _H: \, \widetilde{\mathrm {H}}(W^s) \rightarrow \mathrm {H}(W^s)\) is the canonical covering map. Then we put

$$\begin{aligned} f(h) := \mathrm {Det}(a_h + s) = \mathrm {Det}(2s (\mathrm {Id}_W - h)^{-1}), \end{aligned}$$

and wish to define \(\phi :\, \widetilde{\mathrm {H}}(W^s)\rightarrow \mathbb {C}\) to be the square root of f which agrees with the positive square root on \(T_+\,\). (Here we regard \(T_+\) as being in \(\widetilde{\mathrm {H}} (W^s)\) by its canonical lifting as a subsemigroup of \(\mathrm {H} (W^s)\) as in the previous section). This is possible because \(\phi \) is naturally defined on \(\{(\xi , \eta ) \in \mathrm {H}(W^s)\times \mathbb {C} \mid f(\xi ) = \eta ^2 \}\) which is itself a 2:1 cover of \(\mathrm {H} (W^s)\). Since up to equivariant equivalence there is only one such covering, namely \(\widetilde{\mathrm {H}}(W^s) \rightarrow \mathrm {H}(W^s)\), it follows that we may define \(\phi \) on \(\widetilde{\mathrm {H}} (W^s)\) as desired. For the construction of the oscillator representation it is useful to observe that \(\phi \) can be extended to a slightly larger space. This extension is constructed as follows.

Regard the complex symplectic group \(\mathrm {Sp}\) as the total space of an \(\mathrm {Sp}_\mathbb {R}\)-principal bundle \(\pi : \, \mathrm {Sp} \rightarrow \pi (\mathrm {Sp})\), \(g \mapsto g \sigma (g^{-1})\). Recall that the restricted map \(\pi : \, M \rightarrow M\) is a diffeomorphism, and that M contains the neutral element \(\mathrm {Id} \in \mathrm {Sp}\) in its boundary. We choose a small ball B centered at \(\mathrm {Id}\) in the base \(\pi (\mathrm {Sp})\), and using the fact that M can be identified with a cone in \(\mathrm {i}\mathfrak {sp}_\mathbb {R}\) we observe that \(A := B \, \cup \, M \subset \pi (\mathrm {Sp})\) is contractible. Now \(U := \pi ^{-1}(A)\) is diffeomorphic to a product \(\mathrm {Sp}_\mathbb {R} \times A\) and thus comes with a 2:1 covering \(\widetilde{U} \rightarrow U\) defined by \(\tau : \, \mathrm {Mp} \rightarrow \mathrm {Sp}_\mathbb {R}\,\). The covering space \(\widetilde{U}\) contains \(\widetilde{ \mathrm {H}}(W^s)\), and is invariant under the \(\mathrm {Mp}\)-action by right multiplication. By construction it also contains the metaplectic group \(\mathrm {Mp}\,\), which covers the group \(\mathrm {Sp}_\mathbb {R}\) in \(\mathrm {Sp}\,\).

Recall the definition of the determinant variety \(D_{\mathrm {Sp}} = \{g\in \mathrm {Sp} \mid \mathrm {Det}(\mathrm {Id}_W - g) = 0 \}\). Let \(\widetilde{D}_{\mathrm {Sp}}\) denote the set of points in \(\widetilde{U}\) which lie over \(D_{\mathrm {Sp}} \cap U\) by the covering \(\widetilde{U} \rightarrow U\).

Proposition 3.9

There is a unique continuous extension of \(\phi \) from \(\widetilde{\mathrm {H}}(W^s)\) to its closure in \(\widetilde{U}\) so that \(\phi ^2\) agrees with the lift of f from U. The intersection of \(\widetilde{D}_{\mathrm {Sp}}\) with any \(\mathrm {Mp}\) -orbit in \(\widetilde{U}\) is nowhere dense in that orbit and the restriction of the extended function \(\phi \) to the complement of that intersection is real-analytic.

Remark 3.4

Before beginning the proof, it should be clarified that at the points of the lifted determinant variety, i.e., where the lifted square root \(\phi \) of the function \(f(g) = \mathrm {Det}(2s\,(\mathrm {Id}_W - g)^{-1})\) has a pole, continuity of the extension means that the reciprocal of \(\phi \) extends to a continuous function which vanishes on that set.

Proof

The intersection of \(D_{\mathrm {Sp}}\) with any \(\mathrm {Sp}_\mathbb {R}\)-orbit in U is nowhere dense in that orbit; therefore the same holds for the intersection of \(\widetilde{D}_{\mathrm {Sp}}\) with any \(\mathrm {Mp}\)-orbit in \(\widetilde{U}\).

Let \(x \in \widetilde{U} {\setminus} \widetilde{D}_{\mathrm {Sp}}\) be a point of the boundary of \(\widetilde{\mathrm {H}}(W^s)\). Choose a local contractible section \(\Sigma \subset \widetilde{U}\) of \(\widetilde{U} \rightarrow A\) with \(x \in \Sigma \) and a neighborhood \(\Delta \) of the identity in \(\mathrm {Mp}\) so that the map \(\Delta \times \Sigma \rightarrow \widetilde{U}\,\), \((g,s) \mapsto s g^{-1}\), realizes \(\Delta \times \Sigma \) as a neighborhood \(\widetilde{V}\) of x which has empty intersection with \(\widetilde{D}_{\mathrm {Sp}}\,\). By construction \(\widetilde{V} \cap \widetilde{\mathrm {H}}(W^s)\) is connected and is itself simply connected. Thus the desired unique extension of \(\phi \) exists on \(\widetilde{V}\). At x this extension is simply defined by taking limits of \(\phi \) along arbitrary sequences \(\{ x_n \}\) from \(\widetilde{\mathrm {H}}(W^s)\). Thus the extended function (still called \(\phi \)) is well-defined on the closure of \(\widetilde{\mathrm {H}}(W^s)\) and is real-analytic on the complement of \(\widetilde{D} _{\mathrm {Sp}}\) in every \(\mathrm {Mp}\)-orbit in that closure. It extends as a continuous function on the full closure of \(\widetilde{ \mathrm {H}} (W^s)\) by defining it to be identically \(\infty \) on \(\widetilde{D}_{ \mathrm {Sp}}\,\), i.e., its reciprocal vanishes identically at these points. \(\square \)

Now let us proceed with our main objective of defining the semigroup representation on \(\widetilde{\mathrm {H}}(W^s)\). Recall the involution \(\psi : \, \mathrm {H}(W^s) \rightarrow \mathrm {H}(W^s)\), \(h \mapsto \sigma (h)^{-1}\), and its lift \(\widetilde{\psi }\) to \(\widetilde{\mathrm {H}}(W^s)\). The following will be of use at several points in the sequel.

Proposition 3.10

\(\phi \circ \widetilde{\psi } = \overline{\phi }\,\).

Proof

By direct calculation, \(f \circ \psi = \overline{f}\,\). Thus, since \(f = \phi ^2\), we have either \(\phi \circ \widetilde{\psi } = \overline{\phi }\) or \(\phi \circ \widetilde{\psi } = - \overline{ \phi }\,\). The latter is not the case, as \(\phi \) is not purely imaginary on the non-empty set \(\mathrm {Fix} (\widetilde{\psi })\). \(\square \)

The semigroup representation \(R :\, \widetilde{\mathrm {H}}(W^s) \rightarrow \mathrm {End}(\fancyscript {A}_V)\) will be given by a certain averaging process involving the standard representation of the Heisenberg group. The latter representation is defined as follows. For elements \(w = v + cv\) of the real vector space \(W_\mathbb {R}\,\), the operator \(\delta (v) + \mu (cv)\) is self-adjoint and its exponential

$$\begin{aligned} T_{v+cv} := \mathrm {e}^{\mathrm {i}\delta (v) + \mathrm {i}\mu (cv)} \end{aligned}$$

converges and is unitary (see, e.g., [15]). These operators satisfy the relation

$$\begin{aligned} T_{w^{}} T_{w'} = T_{w + w^\prime } \, \mathrm {e}^{\frac{\mathrm {i}}{2} \omega (w,w')} \quad (w, w^\prime \in W_\mathbb {R}), \end{aligned}$$
(3.2)

where \(\omega := \mathrm {i} A \vert _{W_\mathbb {R}}\) is the induced real symplectic structure. If \(T \mapsto T^\dagger \) denotes the adjoint operation in \(\mathrm {End}(\fancyscript {A}_V)\), it follows from \(\delta (v)^\dagger = \mu (cv)\) that

$$\begin{aligned} T_w^\dagger = T_{-w}^{} = T_w^{-1} \quad (w \in W_\mathbb {R}). \end{aligned}$$

Thus if \(H := W_\mathbb {R} \times \mathrm {U}_1\) is equipped with the Heisenberg group multiplication law,

$$\begin{aligned} (w,z)\,(w',z') := (w + w', z \, z' \mathrm {e}^{ \frac{\mathrm {i}}{2} \omega (w,w')}), \end{aligned}$$

then \((w,z)\mapsto z\, T_w\) is an irreducible unitary representation of H on \(\mathcal A_V\). It is well known that up to equivariant isomorphisms there is only one such representation.

The oscillator representation \(x \mapsto R(x)\) of \(\widetilde{ \mathrm {H}}(W^s)\) is now defined by

$$\begin{aligned} R(x) = \int _{W_\mathbb {R}} \gamma _x(w) T_w \, \mathrm {dvol}(w), \quad \gamma _x(w) := \phi (x) \, \mathrm {e}^{-\frac{1}{4} \langle w ,\, a_x w \rangle } \,. \end{aligned}$$

Here \(\mathrm {dvol}\) is the Euclidean volume element on \(W_\mathbb {R}\) which we normalize so that

$$\begin{aligned} \int _{W_\mathbb {R}} \mathrm {e}^{-\frac{1}{4} \langle w , w \rangle } \, \mathrm {dvol}(w) = 1. \end{aligned}$$

It should be stressed that we often parameterize \(W_\mathbb {R} \simeq V\) by the map \(v \mapsto v + cv = w\,\).

Notice that by the positivity of \(\mathfrak {Re}(a_x)\) the Gaussian function \(w \mapsto \gamma _x(w)\) decreases rapidly, so that all integrals involved in the discussion above and below are easily seen to converge. In particular, since the unitary operator \(T_w\) (for \(w \in W_\mathbb {R}\)) has \(L^2\)-norm \(\Vert T_w \Vert = 1\), it follows for any \(x \in \widetilde{\mathrm {H}} (W^s)\) that

$$\begin{aligned} \Vert R(x) \Vert \le \vert \phi (x) \vert \int _{W_\mathbb {R}} \mathrm {e}^{-\frac{1}{4}\langle w , \, \mathfrak {Re}(a_{x}) w\rangle }\, \mathrm {dvol}(w) =: C(x), \end{aligned}$$

where the bound C(x) by direct computation of the integral is a finite number:

$$\begin{aligned} C(x) = \vert \phi (x) \vert \, \mathrm {Det}^{-1/2} \left( \mathfrak {Re}(a_x) \right) = 2^{\, \dim _\mathbb {C} V} \left| \frac{\mathrm {Det} (\mathrm {Id}_W - h)}{\mathrm {Det} (s - h^\dagger s h)} \right| ^{1/2}\;, \quad h = \tau _H(x). \end{aligned}$$
(3.3)

Thus R(x) is a bounded linear operator on \(\fancyscript {A}_V\). In Proposition 3.17 we will establish the uniform bound \(\Vert R(x) \Vert \le C(x) < 1\) for all \(x \in M\). It is also clear that the operator R(x) depends continuously on \(x \in \widetilde{ \mathrm {H}}(W^s)\).

The main point now is to prove the semigroup multiplication rule \(R(xy) = R(x)R(y)\). For this we apply the Heisenberg multiplication formula (3.2) to the inner integral of

$$\begin{aligned} R(x)R(y) = \int _{W_\mathbb {R}} \left( \int _{W_\mathbb {R}} \gamma _x(w-w') \gamma _y(w') T_{w-w'} T_{w'}\, \mathrm {dvol}(w') \right) \, \mathrm {dvol}(w) \end{aligned}$$

to see that \(R(xy) = R(x)R(y)\) is equivalent to the multiplication rule \(\gamma _{xy} = \gamma _x \sharp \gamma _y\) where the right-hand side means the twisted convolution

$$\begin{aligned} \gamma _x \sharp \gamma _y (w) := \int _{W_\mathbb {R}} \gamma _x(w-w') \gamma _y(w') \, \mathrm {e}^{\frac{\mathrm {i}}{2}\omega (w,w')} \, \mathrm {dvol}(w'). \end{aligned}$$
(3.4)

For the proof of the formula \(\gamma _x \sharp \gamma _y = \gamma _{xy} \,\), we will need to know that \(\phi \) transforms as

$$\begin{aligned} \phi (xy) = \phi (x)\phi (y)\mathrm {Det}^{-\frac{1}{2}}(a_x + a_y)\;. \end{aligned}$$
(3.5)

This transformation behavior, in turn, follows directly from the expression for the semigroup multiplication rule transferred to the upper half space \(\zeta ^+_{\,s}\); we record this expression in the following statement and refer to [11], p. 78, for the calculation.

Proposition 3.11

Identifying \(\mathrm {H}(W^s)\) with \(\zeta _{\,s}^+\) by the modified Cayley transformation and denoting by \((X,Y) \mapsto X \circ Y\) the semigroup multiplication on \(\zeta ^+_{\,s}\), one has

$$\begin{aligned} X \circ Y + s = (Y+s)(X+Y)^{-1}(X+s) = X + s - (X-s)(X+Y)^{-1}(X+s) \,. \end{aligned}$$
(3.6)

Given Proposition 3.11, to prove the transformation rule (3.5) just set \(X = a_h\) and \(Y = a_{h'}\) and note that, since the semigroup multiplication law for \(\widetilde{\mathrm {H}}(W^s)\) by definition yields \(a_h \circ a_{h'} = a_{hh'}\,\), the first expression in (3.6) implies

$$\begin{aligned} f(hh') = f(h) f(h') \mathrm {Det}^{-1} (a_h + a_{h'}), \end{aligned}$$
(3.7)

where \(f(h) = \mathrm {Det}(a_h + s)\) as above. The transformation rule for \(\phi \) follows by taking the square root of (3.7). As usual, the sign of the square root is determined by taking the positive sign at points of the lift of \(T_+\) in \(\widetilde{\mathrm {H}}(W^s)\).

Now we come to the main point.

Proposition 3.12

The twisted convolution for \(x , y \in \widetilde{\mathrm {H}}(W^s)\) satisfies \(\gamma _x \sharp \gamma _y = \gamma _{xy}\).

Proof

Observe that the phase factor for \(w \, , w^\prime \in W_\mathbb {R}\) can be reorganized as

$$\begin{aligned} \mathrm {e}^{\frac{\mathrm {i}}{2} \omega (w,w^\prime )} = \mathrm {e}^{-\frac{1}{2} A(w,w^\prime )} = \mathrm {e}^{\frac{1}{4} \langle w^\prime , \, s w \rangle - \frac{1}{4} \langle w , \, s w^\prime \rangle }\;. \end{aligned}$$

Inserting the definitions of \(\gamma _x\) and \(\gamma _y\) in \(\gamma _x \sharp \gamma _y\) we then have

$$\begin{aligned} \gamma _x \sharp \gamma _y (w) = \phi (x)\phi (y) \, \mathrm {e}^{ - \frac{1}{4}\langle w, \, a_x w\rangle }\int \mathrm {e}^{- \frac{1}{4}\langle w^\prime , (a_x + a_y) w^\prime \rangle + \frac{1}{4}\langle w^\prime , (a_x + s) w \rangle + \frac{1}{4} \langle w,(a_x - s)w^\prime \rangle }\,\mathrm {dvol}(w^\prime )\;. \end{aligned}$$

Since \(\mathfrak {Re} (a_x + a_y) > 0\,\), the integrand is a rapidly decreasing function of \(w^\prime \in W_\mathbb {R}\) and convergence of the integral over the domain \(W_\mathbb {R}\) is guaranteed.

We now wish to explicitly compute the integral by completing the square and shifting variables. For this it is a useful preparation to write

$$\begin{aligned} \langle w^\prime , a_x w \rangle = A(w^\prime , s a_x w) \quad (w^\prime \in W_\mathbb {R}), \end{aligned}$$

and similarly for the other terms. We then holomorphically extend the right-hand side to \(w^\prime \) in W, and by making a shift of integration variables

$$\begin{aligned} w^\prime \rightarrow w^\prime + (a_x + a_y)^{-1} (a_x + s) w, \end{aligned}$$

we bring the convolution integral into the form

$$\begin{aligned} \gamma _x \sharp \gamma _y (w) = \mathrm {e}^{-\frac{1}{4} A(w,\, s (a_x \circ a_y) w)} \phi (x) \phi (y) \int _{W_\mathbb {R}} \mathrm {e}^{-\frac{1}{4} A(w^\prime , \, (s a_x + s a_y) w^\prime )} \, \mathrm {dvol}(w^\prime ), \end{aligned}$$

where \(a_x \circ a_y = -s + (a_y + s)(a_x + a_y)^{-1} (a_x + s) = a_x - (a_x - s)(a_x + a_y)^{-1}(a_x+s)\) is the semigroup multiplication on \(\zeta _{\,s}^+\). Using \(A(w,s w^\prime ) = \langle w , w^\prime \rangle \) for \(w \in W_\mathbb {R}\) and the defining relation \(a_x \circ a_y = a_{xy}\,\), we see that the first factor on the right-hand side is \(\mathrm {e}^{- \frac{1}{4} \langle w , \, a_{xy} w \rangle }\). By the transformation rule (3.5) the integral is evaluated as

$$\begin{aligned} \int _{W_\mathbb {R}} \mathrm {e}^{-\frac{1}{4} \langle w^\prime ,\, (a_x + a_y) w^\prime \rangle }\, \mathrm {dvol}(w^\prime ) = \mathrm {Det}^{-1/2}(a_x+a_y) = \frac{\phi (xy)}{\phi (x)\phi (y)}\;, \end{aligned}$$

and multiplying factors it follows that

$$\begin{aligned} \gamma _x \sharp \gamma _y(w) = \phi (xy) \, \mathrm {e}^{- \frac{1}{4} \langle w ,\, a_{xy} w \rangle } = \gamma _{xy}(w)\;, \end{aligned}$$

which is the desired semigroup property. \(\square \)

Corollary 3.5

The mapping \(R : \, \widetilde{\mathrm {H}}(W^s) \rightarrow \mathrm {End} ({\fancyscript {A}}_V)\) defined by

$$\begin{aligned} R(x) = \int _{W_\mathbb {R}} \gamma _x(w) T_w \,\mathrm {dvol}(w) \end{aligned}$$

is a representation of the semigroup \(\widetilde{\mathrm {H}}(W^s)\).

We conclude this section by deriving a formula for the adjoint.

Proposition 3.13

The adjoint of R(x) is computed as \(R(x)^\dagger = R(\widetilde{ \psi }(x))\). In particular, \(R(x)R(x)^\dagger = R(x\, \widetilde{ \psi }(x))\).

Proof

We recall the relations \(\overline{\phi } = \phi \circ \widetilde{\psi }\) from Proposition 3.10 and \((a_h)^\dagger = a_{\psi (h)}\) from Remark 3.3. Since \(\overline{\langle w , a_h w \rangle } = \langle w , (a_h)^\dagger w \rangle \,\), it follows that

$$\begin{aligned} \overline{\gamma _x} = \gamma _{\widetilde{\psi }(x)} \,. \end{aligned}$$

The desired formula, \(R(x)^\dagger = R(\widetilde{\psi }(x))\), now results from this equation and the identities \(T_w^\dagger = T_{-w}\) and \(\gamma _x(-w) = \gamma _x(w)\). With this in hand, the second statement \(R(x) R(x)^\dagger = R(x\, \widetilde{\psi } (x))\) is a consequence of the semigroup property. \(\square \)

3.3.3 Basic conjugation formula

Here we compute the effect of conjugating (in the semigroup sense) operators of the form q(w), \(w \in W\), with operators R(x) coming from the semigroup. This is an immediate consequence of an analogous result for the operators \(T_w\,\). For this we first allow \(T_w\) to be defined for \(w = v + \varphi \in W\) by

$$\begin{aligned} T_w := \mathrm {e}^{\mathrm {i} q(w)} = \mathrm {e}^{\mathrm {i} \delta (v) + \mathrm {i} \mu (\varphi )}. \end{aligned}$$

These operators are no longer defined on Fock space, but are defined on \(\mathcal O(V)\). They satisfy

$$\begin{aligned} T_w T_{\tilde{w}} = T_{w + \tilde{w}} \, \mathrm {e}^{- \frac{1}{2} {A(w,\tilde{w})}}\,. \end{aligned}$$
(3.8)

Note that for \(x \in \widetilde{\mathrm {H}}(W^s)\) and \(w \in W\) the operators \(R(x) T_w\) and \(T_{\tau _H(x) w} R(x)\) are bounded on \(\fancyscript {A}_V\). Thus we interpret the following as a statement about operators on that space.

Proposition 3.14

For \(w \in W\) and \(x \in \widetilde{\mathrm {H}}(W^s)\) one has the relation

$$\begin{aligned} R(x) T_w = T_{\tau _H(x) w} R(x)\,. \end{aligned}$$

Proof

For convenience of notation we write

$$\begin{aligned} R(x) = \phi (x) \int _{W_\mathbb {R}} \mathrm {e}^{-\frac{1}{4} A(\tilde{w} , \, s a_x \tilde{w})} T_{\tilde{w}}\, \mathrm {dvol} (\tilde{w})\,. \end{aligned}$$

Thus

$$\begin{aligned} R(x) T_w = \phi (x) \int _{W_\mathbb {R}}\mathrm {e}^{-\frac{1}{4} A(\tilde{w} ,\,s a_x \tilde{w}) - \frac{1}{2} A(\tilde{w},w)} T_{\tilde{w} + w}\, \mathrm {dvol}(\tilde{w})\;. \end{aligned}$$

Now let \(h := \tau _H(x)\) and change variables by the translation \(\tilde{w} \mapsto \tilde{w} - w + h w\,\). Using the definition \(s a_x = (\mathrm {Id}_W + h)(\mathrm {Id}_W - h)^{-1}\) and the relation

$$\begin{aligned} A((\mathrm {Id}_W - h) w_1 ,(\mathrm {Id}_W + h)w_2) = - A((\mathrm {Id}_W + h)w_1, (\mathrm {Id}_W - h)w_2) \end{aligned}$$

for all \(w_1 , w_2 \in W\), one simplifies the exponent to obtain

$$\begin{aligned} R(x) T_{w} = \phi (x) \int _{W_\mathbb {R}}\mathrm {e}^{-\frac{1}{4} A(\tilde{w} ,\,s a_x \tilde{w}) -\frac{1}{2} A(h w , \tilde{w})} T_{h w + \tilde{w}}\, \mathrm {dvol}(\tilde{w})\;. \end{aligned}$$

Reading (3.8) backwards one sees that this expression equals \(T_{hw} R(x)\). \(\square \)

The basic conjugation rule now follows immediately.

Proposition 3.15

For every \(x \in \widetilde{\mathrm {H}}(W^s)\) and \(w \in W\) it follows that

$$\begin{aligned} R(x) q(w) = q(\tau _H(x)w) R(x)\,. \end{aligned}$$

Proof

Apply Proposition 3.14 for w replaced by tw and differentiate both sides of the resulting formula at \(t=0\). \(\square \)

3.3.4 Spectral decomposition and operator bounds

Numerous properties of R are derived from a precise description of the spectral decomposition of R(x) for \(x \in M\,\). Since every orbit of \(\mathrm {Sp}_\mathbb {R}\) acting by conjugation on M has non-empty intersection with \(T_+\,\), it is important to understand this decomposition when \(x \in T_+\,\). For this we begin with the case where V is one-dimensional.

Proposition 3.16

Suppose that V is one-dimensional and that the \(T_+\) -action on \(W = V \oplus V^*\) is given by \(x \cdot (v+\varphi ) = \lambda v + \lambda ^{-1}\varphi \) where \(\lambda > 1\,\) . If f is a basis vector of \(V^*\) then the monomials \(\{f^m \}_{m \in \mathbb {N} \cup \{0\}}\) form a basis of \(\mathcal A_V\) and one has

$$\begin{aligned} R(x)f^m = \lambda ^{-m-1/2} f^m. \end{aligned}$$

Proof

First note that if \(w = v + cv\,\), then

$$\begin{aligned} a_x w = s \frac{1 + x}{1 - x} \cdot (v+cv) = - \frac{1+\lambda }{1-\lambda }\,v +\frac{1+\lambda ^{-1}}{1 - \lambda ^{-1}}\,cv = \frac{\lambda +1}{\lambda -1}(v+cv). \end{aligned}$$
(3.9)

Thus the Gaussian function \(\gamma _x(w)\) in the present case is

$$\begin{aligned} \gamma _x (v + cv) = \phi (x)\, \mathrm {e}^{-\frac{1}{2} \frac{\lambda +1}{\lambda -1} |v|^2}. \end{aligned}$$

To apply the operator \(T_{v+cv}\) to \(f^m\) we use the description

$$\begin{aligned} T_{v + cv} = \mathrm {e}^{\mathrm {i}\delta (v) + \mathrm {i}\mu (cv)} = \mathrm {e}^{\mathrm {i}\mu (cv)} \mathrm {e}^{-\frac{1}{2} |v|^2} \mathrm {e}^{\mathrm {i}\delta (v)}\,. \end{aligned}$$

Decomposing \(T_{v+cv}\) in this way is not allowed on the Fock space, but is allowed if we regard \(T_{v+cv}\) as an operator on the full space \(\mathcal {O}(V)\) of holomorphic functions. The calculations are now carried out on this larger space.

Recall that \(\delta (v) f^m = m f(v) f^{m-1}\). From this we obtain the explicit expression

$$\begin{aligned} T_{v+cv} f^{m} = \mathrm {e}^{-\frac{1}{2}\vert v\vert ^2} \mathrm {e}^{\mathrm {i} \mu (cv)} \sum _{l=0}^m \frac{\mathrm {i}^l}{l!}m(m-1)\cdots (m-l+1) f(v)^l f^{m - l}\,. \end{aligned}$$

Our goal is to compute

$$\begin{aligned} I := \int _V \mathrm {e}^{-\frac{1}{2}\frac{\lambda +1}{\lambda -1} |v|^2} T_{v+cv} f^m \, \mathrm {dvol}(v), \end{aligned}$$

where \(\mathrm {dvol}(v)\) corresponds to \(\mathrm {dvol}(w)\) by the isomorphism \(V \simeq W_\mathbb {R}\,\). Expanding the exponential \(\mathrm {e}^{ \mathrm {i} \mu (cv)}\) and using \(\mu (cv) f^m = |v|^2 f(v)^{-1} f^{m+1}\), the integral I is a sum of Gaussian expected values of terms of the form \(|v|^{2k} f(v)^{-k} f(v)^{l}\). The only terms which survive are those with \(k = l\,\). Thus

$$\begin{aligned} I&= f^m \sum _{k =0}^m (-1)^k \left( {\begin{array}{c}m\\ k\end{array}}\right) \int _V \frac{|v|^{2k}}{k!} \, \mathrm {e}^{-\frac{\lambda }{\lambda -1} |v|^2}\, \mathrm {dvol}(v) \\&= 2^{-1} \sum _{k=0}^m (-1)^k \left( {\begin{array}{c}m\\ k\end{array}}\right) \bigg (\frac{\lambda -1}{\lambda }\bigg )^{k+1} f^m = 2^{-1} (1 - \lambda ^{-1})\lambda ^{-m} f^m. \end{aligned}$$

Now \(\phi (x)^2 = \mathrm {Det}(a_x+s) = \mathrm {Det}(2s (\mathrm {Id}_W-\tau _H(x))^{-1})= (-2/ (1-\lambda )) (2/(1 - \lambda ^{-1}))\), and \(\phi (x) = 2 \lambda ^{-1/2} (1 - \lambda ^{-1})^{-1}\), since we are to take the positive square root at points \(x \in T_+\,\). Hence, \(R(x) f^m = \phi (x) 2^{-1} (1- \lambda ^{-1}) \lambda ^{-m} f^m = \lambda ^{-m-1/2} f^m\) as claimed.\(\square \)

Remark 3.5

Note that as \(x \in T_+\) goes to the unit element (or, equivalently, \(\lambda \rightarrow 1\)), the expression \(R(x) f^m\) converges to \(f^m\) in the strong sense for all \(m \in \mathbb {N} \cup \{0\}\,\).

Now let V be of arbitrary dimension and assume that \(x \in T_+\) is diagonalized on \(W = V \oplus V^*\) in a basis \(\{ e_1 , \ldots , e_d , c e_1,\ldots ,c e_d \}\) with eigenvalues \(\lambda _1, \ldots \lambda _d , \lambda _1^{-1}, \ldots ,\lambda _d^{-1}\) respectively. Since \(x \in T_+\), we have \(\lambda _i > 1\) for all i. For \(f_i := c e_i\) and \(m := (m_1,\ldots , m_d)\) we employ the standard multi-index notation \(f^m := f_1^{m_1} \cdots f_d^{m_d}\) and \(\lambda ^m := \lambda _1^{m_1} \cdots \lambda _d^{m_d}\). In this case the multi-dimensional integrals split up into products of one-dimensional integrals. Thus, the following is an immediate consequence of the above.

Corollary 3.6

Let \(x \in T_+\) be diagonal in a basis \(\{ e_i \}\) of V with eigenvalues \(\lambda _i\) ( \(i = 1, \ldots , d\) ). If \(f^m\) is a monomial \(f^m \equiv \prod _i (c e_i)^{m_i}\) , then \(R(x)f^m = \lambda ^{-m-1/2} f^m\).

One would expect the same result for the spectrum to hold for every conjugate \(g T_+ g^{-1}\), and this expectation is indeed borne out. However, in the approach we are going to take here, we first need the existence and basic properties of the oscillator representation of the metaplectic group. The following is a first step in this direction.

Proposition 3.17

The operator norm function \(\mathrm {Mp} \times T_+ \rightarrow \mathbb {R}^{>0}\), \((g,t) \mapsto \Vert R(g\,t g^{-1}) \Vert \) is bounded by a continuous \(\mathrm {Mp}\) -independent function \(C(t) < 1\,\).

Proof

For any \(x \in \widetilde{\mathrm {H}}(W^s)\) we already have the bound \(\Vert R(x) \Vert \le C(x)\) where C(x) was computed in (3.3). That function C(x) clearly is invariant under conjugation \(x \mapsto g x g^{-1}\) by \(g \in \mathrm {Mp}\,\). Evaluating it for the case of an element \(x \equiv t \in T_+\) with eigenvalues \(\lambda _i\) one obtains

$$\begin{aligned} C(t) = 2^{\, \dim _\mathbb {C} V} \prod \nolimits _i \left( \lambda _i^{1/2}+\lambda _i^{-1/2} \right) ^{-1}. \end{aligned}$$

The inequality \(C(t) < 1\) now follows from the fact that \(\lambda _i > 1\) for all \(i\,\). \(\square \)

Since \(R(x)^\dagger = R(\widetilde{\psi }(x))\) and \(\Vert R(t g) \Vert ^2 = \Vert R(t g)^\dagger R(t g) \Vert = \Vert R(g^{-1} t^2 g) \Vert \), we infer the following estimates.

Corollary 3.7

For all \(t \in T_+\) and \(g \in \mathrm {Mp}\) one has \(\Vert R(t g) \Vert < 1\) and \(\Vert R(gt) \Vert < 1\,\).

3.4 Representation of the metaplectic group

Recall that we have realized the metaplectic group \(\mathrm {Mp}\) in the boundary of the oscillator semigroup \(\widetilde{\mathrm {H}} (W^s)\) and that \(\widetilde{\mathrm {H}}(W^s)\) contains the lifted manifold \(T_+\) in such a way that the neutral element \(\mathrm {Id} \in \mathrm {Mp}\) is in its boundary. Here we show that for \(x \in T_+\) and \(g \in \mathrm {Mp}\) the limit \(\lim _{x \rightarrow \mathrm {Id}} R(gx)\) is a well-defined unitary operator \(R'(g)\) on Fock space and \(R' :\, \mathrm {Mp} \rightarrow \mathrm {U} (\fancyscript {A}_V)\) is a unitary representation. The basic properties of this oscillator representation are then used to derive important facts about the semigroup representation \(R\,\).

Convergence will eventually be discussed in the so-called bounded strong* topology (see [11], p. 71). For the moment, however, we shall work with the slightly weaker notion of bounded strong topology where one only requires uniform boundedness and pointwise convergence of the operators themselves (with no mention made of their adjoints). Note that since \(\Vert R(gx) \Vert < 1\) by Corollary 3.7, we need only prove the convergence of R(gx) f on a dense set of functions \(f \in \fancyscript {A}_V\). Let us begin with \(g = \mathrm {Id}\,\).

Lemma 3.4

If a sequence \(x_n \in T_+\) converges to \(\mathrm {Id} \in \mathrm {Mp}\,\), then the sequence \(R(x_n)\) converges in the bounded strong topology to the identity operator on Fock space.

Proof

If f is any \(T_+\)-eigenfunction, the sequence \(R(x_n) f\) converges to f by the explicit description of the spectrum given in Corollary 3.6. The statement then follows because the subspace generated by these functions is dense. \(\square \)

Using this lemma along with the semigroup property, we now show that the limiting operators exist and are well-defined.

Proposition 3.18

If \(x_n \in T_+\) converges to \(\mathrm {Id} \in \mathrm {Mp}\,\), then for every \(g \in \mathrm {Mp}\) the sequence of operators \(R(g x_n)\) converges pointwise, i.e., \(R(g x_n) f \rightarrow R^\prime (g) f\) for all f in \({\fancyscript{A}}_V\). The limiting operator \(R'(g)\) is independent of the sequence \(\{ x_n \}\,\).

Proof

For any \(m , n \in \mathbb {N}\) there exists some \(t = t(m,n) \in T_+\) sufficiently near the identity so that \(\tilde{x}_m = t^{-1} x_m\) and \(\tilde{x}_n = t^{-1} x_n\) are still in \(T_+\,\). By the semigroup property \(R(g x_n) = R(g t) R(\tilde{x}_n)\) we then have

$$\begin{aligned} \Vert R(g x_m)f - R(g x_n)f \Vert \le \Vert R(g t) \Vert \, \Vert R(\tilde{x}_m)f - R(\tilde{x}_n)f \Vert \,. \end{aligned}$$

Letting \(t = t(m,n)\rightarrow \mathrm {Id}\) it follows from Corollary 3.7 that

$$\begin{aligned} \Vert R(g x_m)f - R(g x_n)f \Vert \le \Vert R(x_m)f - R(x_n)f \Vert \,. \end{aligned}$$

Thus the Cauchy property of \(R(x_n)f\) is passed on to \(R(g x_n)f\) and therefore the sequence \(R(g x_n)f\) converges in the Hilbert space \(\fancyscript {A}_V\). Let \(\lim _{n \rightarrow \infty } R(g x_n)f =: R^\prime (g) f\).

To show that the limit is well-defined, pick from \(T_+\) another sequence \(y_n \rightarrow \mathrm {Id}\,\), let \(\lim _{n \rightarrow \infty } R(g y_n)f =: R^{\prime \prime }(g) f\,\), and notice that \(\Vert R^\prime (g) f - R^{\prime \prime }(g)f \Vert \) is no bigger than

$$\begin{aligned} \Vert R^\prime (g) f - R(g x_n)f \Vert + \Vert R(g x_n) f - R(g y_n)f \Vert + \Vert R(g y_n)f - R^{\prime \prime }(g) f \Vert. \end{aligned}$$

Using the same reasoning as above, the middle term is estimated as

$$\begin{aligned} \Vert R(g x_n) f - R(g y_n) f \Vert \le \Vert R(x_n) f - R(y_n) f \Vert \le \Vert R(x_n) f - f \Vert + \Vert R(y_n) f - f \Vert. \end{aligned}$$

In the limit \(n \rightarrow \infty \) this yields the desired result \(R^\prime (g) = R^{\prime \prime }(g)\). \(\square \)

Remark 3.6

Since \(\Vert R(g x_n) \Vert < 1\) the sequence \(R(g x_n)\) converges to \(R^\prime (g)\) in the bounded strong topology. Such convergence preserves the product of operators, which is to say that if \(A_n \rightarrow A\) and \(B_n \rightarrow B\), then \(A_n B_n \rightarrow AB\,\). Indeed,

$$\begin{aligned} \Vert (A_n B_n - AB) f \Vert \le \Vert A_n (B_n - B)f \Vert + \Vert (A_n - A) B f \Vert, \end{aligned}$$

and convergence follows from \(\Vert A_n \Vert < 1\) and \(A_n \rightarrow A\), \(B_n \rightarrow B\). Note in particular that if \(R(g x_n) \rightarrow R^\prime (g)\) and \(R(g^{-1} x_n) \rightarrow R^\prime (g^{-1})\) then \(R(g x_n) R(g^{-1} x_n) \rightarrow R^\prime (g) R^\prime (g^{-1})\).

The bounded strong\(^*\) topology also requires pointwise convergence of the sequence of adjoint operators. Therefore we must also consider sequences of the form \(R(g x_n)^\dagger \). For this (see the proof of Theorem 3.1 below) we will use the following fact.

Lemma 3.5

Let \(\{ A_n \}\) and \(\{ B_n \}\) be sequences of bounded operators and let \(C_n := A_n B_n\,\). If \(C_n\) and \(B_n\) converge pointwise with \(B_n \rightarrow B\) and the sequence \(\{ A_n \}\) is uniformly bounded, then \(A_n\) converges pointwise on the image of \(B\,\).

Proof

If \(f \in \mathrm {im}\, B\,\) then \(f = \lim f_n\) where \(f_n = B_n h\) for some Hilbert vector h. Write

$$\begin{aligned} (A_m - A_n)f = A_m (f - f_m) + (A_m B_m - A_n B_n)h + A_n (f_n - f) \end{aligned}$$

and use the uniform boundedness of \(A_n\) to show that \(A_n f\) converges. \(\square \)

Applying this with \(A_n = R(g x_n g^{-1})\), \(B_n = R(x_n)\) , and \(C_n = A_n B_n = R(g x_n)R(g^{-1} x_n)\), we have the following statement about convergence along the conjugate \(g T_+ g^{-1}\).

Proposition 3.19

For \(g \in \mathrm {Mp}\) and \(\{ x_n \}\) any sequence in \(T_+\) with \(x_n \rightarrow \mathrm {Id} \in \mathrm {Mp}\,\), it follows that \(R(g x_n g^{-1})\) converges pointwise to \(R'(g) R'(g^{-1})\).

Next, if we take three sequences in \(T_+\) and write

$$\begin{aligned} R(x_n y_n z_n) = R(x_n g^{-1}) R(g y_n g^{-1}) R(g z_n) \rightarrow \mathrm {Id}_{\fancyscript {A}_V}, \end{aligned}$$
(3.10)

then it follows that the sequence \(R(x_n g^{-1})\) converges to an operator \(B(g^{-1})\) on the image of \(R'(g) R'(g^{-1}) R'(g)\) with \(B(g^{-1}) R'(g) R'(g^{-1}) R'(g) = \mathrm {Id}_{\fancyscript {A}_V}\,\). In particular, the operator \(R'(g)\) is injective for all \(g \in \mathrm {Mp}\,\). Finally, we define \(y_n\) by \(y_n^2 = x_n\) and write \(R(g x_n) = R(g y_n g^{-1}) R(g y_n)\). Taking the limit of both sides of this equation entails that

$$\begin{aligned} R'(g) = R'(g) R'(g^{-1}) R'(g), \end{aligned}$$
(3.11)

and since \(R'(g)\) is injective, this now allows us to reach the main goal of this section.

Theorem 3.1

For every \(g \in \mathrm {Mp}\) and every sequence \(\{ x_n \} \subset T_+\) with \(x_n \rightarrow \mathrm {Id}\) the sequence \(\{R( g x_n) \}\) converges in the bounded strong \(^*\) topology. The limit \(R'(g)\) is independent of the sequence and defines a unitary representation \(R':\,\mathrm {Mp} \rightarrow \mathrm {U} ({\fancyscript{A}}_V)\).

Proof

From (3.11) we have \(R'(g)(\mathrm {Id}_{ \fancyscript {A}_V} - R'(g^{-1}) R'(g)) = 0\) and, since \(R'(g)\) is injective, \(R'(g^{-1}) R'(g) = \mathrm {Id}_{\fancyscript {A}_V}\,\). Hence \(R'(g^{-1})\) is surjective, and thus \(R'(g) \in \mathrm {GL}(\fancyscript {A}_V)\) by exchanging \(g \leftrightarrow g^{-1}\). For the homomorphism property we write \(R(g_1 x_n) R(g_2 y_n) = R(g_1 x_n g_2 y_n) = R(g_1 x_n g_1^{-1}) R(g_1 g_2 y_n)\) and take limits to obtain \(R'(g_1) R'(g_2) = R'(g_1 g_2)\).

Convergence in the bounded strong\(^*\) topology also requires convergence of the adjoint. This property follows from \(R(g x_n)^\dagger = R(\widetilde{\psi }(g x_n)) = R(x_n g^{-1})\) and the discussion after (3.10), since \(R'(g)\) is now known to be an isomorphism. Unitarity of the representation is then immediate from \(R(g x_n)^\dagger \rightarrow R'(g)^\dagger \) and \(R(x_n g^{-1}) \rightarrow B(g^{-1}) = R'(g)^{-1}\).

Finally, we must show that \(R' : \, \mathrm {Mp} \rightarrow \mathrm {U} (\fancyscript {A}_V)\) is continuous. This amounts to showing that if \(\{ g_k \}\) is a sequence in \(\mathrm {Mp}\) which converges to \(g\,\), then \(R'(g_k) f \rightarrow R'(g) f\) for any \(f \in \fancyscript {A}_V\). Hence, we let \(\{ x_n \}\) be a sequence in \(T_+\) with \(x_n \rightarrow \mathrm {Id}\) and choose \(t = t(m,n)\) as in the proof of Proposition 3.18 so that

$$\begin{aligned} \Vert R(g_k x_m) - R(g_k x_n) \Vert \le \Vert R(g_k t) \Vert \, \Vert R(\tilde{x}_m) - R(\tilde{x}_n) \Vert, \end{aligned}$$

and then let \(t \rightarrow \mathrm {Id}\,\). Using the uniform boundedness of \(R(g_k t)\) as \(t \rightarrow \mathrm {Id}\,\), this shows that the convergence \(R(g_k x_n) \rightarrow R'(g_k)\) is uniform in \(g_k\,\). Since we have \(g_k x_n \rightarrow g x_n\) for every fixed \(n\,\), the continuity of \(x \mapsto R(x) f\) then implies that \(R'(g_k)f \rightarrow R'(g)f\). \(\square \)

Let us underline two important consequences.

Proposition 3.20

For \(g_1 , g_2 \in \mathrm {Mp}\) and \(x \in \widetilde{\mathrm {H}} (W^s)\) it follows that

$$\begin{aligned} R(g_1 x\, g_2) = R'(g_1) R(x) R'(g_2). \end{aligned}$$

Proof

If \(y_m\) and \(z_n\) are sequences in \(T_+\) which converge to \(\mathrm {Id}\,\), then, since \(x \mapsto R(x) f\) is continuous for all f in Fock space, \(R(g_1 y_m x\, g_2 z_n)\) converges pointwise to \(R(g_1 x \, g_2)\). On the other hand, we have \(R(g_1 y_m x\, g_2 z_n) = R(g_1 y_m) R(x) R(g_2 z_n)\) by the semigroup property, and the right-hand side converges pointwise to \(R'(g_1) R(x) R'(g_2)\). \(\square \)

We refer to \(R' :\, \mathrm {Mp} \rightarrow \mathrm {U}(\fancyscript {A}_V)\) as the oscillator representation of the metaplectic group. It has the following fundamental conjugation property.

Proposition 3.21

Let \(\mathrm {Mp} \times W \rightarrow W\), \((g,w) \mapsto \tau (g) w\) denote the representation of \(\mathrm {Mp}\) on W defined by first applying the covering map \(\mathrm {Mp} \rightarrow \mathrm {Sp}_\mathbb {R}\) and then the standard representation of \(\mathrm {Sp}\,\) . If we let W act on \(\mathfrak {a}(V)\) by the Weyl representation q then

$$\begin{aligned} R'(g) q(w) R'(g)^{-1} = q(\tau (g) w)\,. \end{aligned}$$

Proof

Since the inverse operator \(R'(g)^{-1}\) is now available, this follows from the conjugation property at the semigroup level (see Proposition 3.15). \(\square \)

Note that analogously we have the classical conjugation formula for the representation of the Heisenberg group on the Fock space \({\fancyscript{A}}_V\), i.e.,

$$\begin{aligned} R'(g) T_w R'(g)^{-1} = T_{\tau (g) w} \end{aligned}$$

for all \(g \in \mathrm {Mp}\) and \(w \in W_\mathbb {R}\) (see Proposition 3.14).

3.4.1 The trace-class property

Recall that a linear operator L on a Hilbert space \(\mathcal {V}\) is of trace class if and only if the non-negative self-adjoint operator \(\sqrt{LL^\dagger }\) has finite trace. If L is of trace class, then for every unitary basis \(\{ u_i \}\) of \(\mathcal {V}\) the trace

$$\begin{aligned} \mathrm {Tr}\,L := \sum \langle u_i\, , L\, u _i\rangle \end{aligned}$$

converges absolutely. It is independent of the choice of unitary basis and defines a linear functional on the space \(C_1\) of operators of trace class.

Proposition 3.22

For every \(x \in \widetilde{\mathrm {H}}(W^s)\) the operator R(x) is of trace class.

Proof

Recall that \(R(x) R(x)^\dagger = R(y)\), where \(y = x \widetilde{\psi }(x)\in M\). Since \(y = g t^2 g^{-1}\) for some \(t\in T_+\) and \(\sqrt{R(g t^2 g^{-1})} = R'(g)R(t)R'(g)^{-1}\), the desired result follows from the explicit formula in Corollary 3.6 for the eigenvalues of t.

Proposition 3.23

An integral representation of the trace functional \(\mathrm {Tr} : \, C_1 \rightarrow \mathbb {C}\) on the space of trace-class operators on Fock space is given by

$$\begin{aligned} \mathrm {Tr}\, L = \sqrt{2}^{\,\mathrm {dim} W_\mathbb {R}} \Phi (L),\quad \Phi (L) = \int _{W_\mathbb {R}} \langle T_w 1, L\, T_w 1 \rangle _{\fancyscript {A}_V}\, \mathrm {dvol}(w). \end{aligned}$$

Proof

By the relation \(w = v+cv\) we have \(T_w 1 = T_{v+cv} 1 = \mathrm {e} ^{-\frac{1}{2}|v|^2 + \mathrm {i} \mu (cv)}\). As before, if \(\{ e_i \}\) is an orthonormal basis of V let \(f_i := c e_i\) denote the dual basis of \(V^*\). Expanding \(\mathrm {e}^{\mathrm {i} \mu (cv)}\) with the help of multi-index notation \(m = (m_1,\ldots ,m_d)\) we get

$$\begin{aligned} T_{v+cv} 1 = \mathrm {e}^{-\frac{1}{2}|v|^2} \sum \frac{(\mathrm {i}\overline{v})^m}{m!} f^m, \end{aligned}$$

where \(m! = m_1! \cdots m_d!\) and \(f^m = f_1^{m_1} \cdots f_d^{m_d}\). By the isomorphism \(V \simeq W_\mathbb {R}\) the integral \(\Phi (L)\) pulls back to \(\Phi (L) = \int _V \langle T_{v+cv} 1, L\, T_{v+cv} 1 \rangle \, \mathrm {dvol}(v)\) and inserting the series expansion of \(T_{v+cv} 1\) we obtain a double sum

$$\begin{aligned} \Phi (L) = \sum \frac{\langle f^m,L f^{m^\prime }\rangle }{m! \, m^\prime !} \int _V (-\mathrm {i}v)^{m} (\mathrm {i} \overline{v})^{m^\prime } \mathrm {e}^{-|v|^2} \mathrm {dvol}(v)\;. \end{aligned}$$

If \(m \not = m^\prime \) the integral vanishes, while direct computation for \(m = m^\prime \) yields

$$\begin{aligned} \int _V v^{\,m} \overline{v}^{\,m} \mathrm {e}^{-|v|^2} \mathrm {dvol}(v) = 2^{- \mathrm {dim}_\mathbb {C} V} m!. \end{aligned}$$

The normalization here is determined by our convention \(\int _V \mathrm {e}^{-\frac{1}{2} |v|^2} \mathrm {dvol}(v) = 1\). Thus \(2^{\mathrm {dim}_\mathbb {C} V} \Phi (L) = \sum m!^{-1} \langle f^m , L f^m \rangle \). The formula for \(\mathrm {Tr}\, L\) now follows from \(\mathrm {dim}\, W_\mathbb {R} = 2\, \mathrm {dim}_\mathbb {C} V\) and the fact that the set of normalized monomials \(\{f^m / \sqrt{m!}\}\) constitute an orthonormal basis of Fock space. \(\square \)

Proposition 3.24

For every \(P_1,P_2\) in the Weyl algebra and every \(x \in \widetilde{\mathrm {H}}(W^s)\) the operator \(q(P_1) R(x) q(P_2)\) is of trace class on the Fock space \(\fancyscript {A}_V\). Furthermore, the function \(\widetilde{\mathrm {H}} (W^s) \rightarrow \mathbb {C}\,\), \(x \mapsto \mathrm {Tr}\; q(P_1) R(x) q(P_2)\), is holomorphic.

Proof

Note that every element in \(\widetilde{\mathrm {H}}(W^s)\) can be written as a product xyz of elements in \(\widetilde{\mathrm {H}} (W^s)\). Accordingly we are going to show that operators of the form

$$\begin{aligned} q(P_1) R(xyz) q(P_2) = (q(P_1)R(x)) R(y) (R(z)q(P_2)) \end{aligned}$$

are of trace class. Recall that the space of trace-class operators is a left and right ideal in the space of compact operators. It therefore suffices to prove that operators of the form \(B = q(P) R(x)\) are compact. By linearity it is enough to handle the special case where \(q(P) = \mu (cv)^k \delta (v')^l\) for \(k,l \in \mathbb {N} \cup \{0\}\). With respect to a basis of Fock space consisting of naturally defined monomials we will show that the matrix of \(BB^\dagger \) has finite trace. Thus the Hilbert–Schmidt norm \(\Vert B \Vert _\mathrm {HS}^2 := \mathrm {Tr}(BB^\dagger )\) is finite. Since Hilbert–Schmidt operators are compact, the desired result follows.

For the computations we begin by observing that

$$\begin{aligned} A := B B^\dagger = q(P) R(x) R(x)^\dagger q(P)^\dagger = q(P) R(y) q(P)^\dagger, \end{aligned}$$

where \(y = x \widetilde{\psi }(x)\in M\). Let \(\{e_i\}\) (resp. \(\{f_i \}\)), \(1 \le i \le d\), be the basis of V (resp. dual basis of \(V^*\)) so that R(y) is diagonalized, i.e., \(R(y) f^m = \lambda ^{-m-1/2} f^m\). As before, we are using multi-index notation and the numbers \(\lambda _i\) are the eigenvalues of \(y \in M\) on the basis vectors \(e_i\) so that \(\lambda _i > 1\) for all \(i\,\). Let \(v = \sum a_i \, e_i\) and \(v^\prime = \sum b_i \, e_i\,\). Thus

$$\begin{aligned} \delta (v')^l = \sum b_\beta \frac{\partial ^\beta }{\partial f^\beta } \quad \text {and} \quad \mu (cv)^k = \sum \overline{a}_\alpha f^\alpha \,, \end{aligned}$$

where the sums run over all multi-indices \(\alpha \) and \(\beta \) with \(|\alpha | = k\) and \(|\beta | = l\,\).

After expanding \(q(P) R(y) q(P)^\dagger \) we have individual terms

$$\begin{aligned} C(\alpha ,\beta ,\tilde{\alpha },\tilde{\beta }) := f^\alpha \frac{\partial ^\beta }{\partial f^\beta } R(y) f^{\tilde{\beta }} \frac{\partial ^{\tilde{\alpha }}}{\partial f^{\tilde{\alpha }}}\;. \end{aligned}$$

Since we want to compute the trace of the matrix of A with respect to the basis of monomials \(\{ f^m \}\), we need only consider those operators \(C \equiv C(\alpha ,\beta ,\tilde{\alpha },\tilde{\beta })\) for which

$$\begin{aligned} \alpha - \beta = \tilde{\alpha } - \tilde{\beta }\;. \end{aligned}$$
(3.12)

Now we write \(A f^m\) in the monomial basis and must estimate the coefficient of \(f^m\,\). We will do this by estimating the eigenvalue of C on \(f^m\) for those C which satisfy (3.12), and will do so with an estimate \(K_m\) independent of \(\alpha , \beta , \tilde{\alpha },\tilde{\beta }\,\). Hence we may eventually estimate this coefficient by \(N \, K_m\) where N is the number of operators \(C(\alpha , \beta , \tilde{\alpha }, \tilde{\beta })\) with multi-indices satisfying (3.12). Since N is bounded by a constant independent of \(m\,\), it comes out as a factor in our estimate of \(\mathrm {Tr}\, A\) and is therefore of no relevance to the argument. Due to the fact that the vectors \(v = \sum a_i \, e_i\) and \(v^\prime = \sum b_i \, e_i\) are also fixed, it is enough to compute \(\sum _m K_m\,\).

For those C which satisfy (3.12) it follows that \(C f^m = r\, \lambda ^{-(m - \alpha + \beta ) - 1/2} f^m\) where

$$\begin{aligned} r = \frac{m!}{(m-\tilde{\alpha })!}\; \frac{(m - \tilde{\alpha }+ \tilde{\beta })!}{(m - \tilde{\alpha }+ \tilde{\beta }- \beta )!}\,. \end{aligned}$$

Now, using the inequality \(\frac{I!}{(I-J)!} \le \vert I \vert ^{\vert J \vert }\) and replacing each \(\lambda _i\) by the smallest eigenvalue \(\lambda _\mathrm{min}\,\), up to a constant independent of \(|m| = m_1 + \ldots + m_d\) we have

$$\begin{aligned} r\, \lambda ^{- (m - \alpha + \beta ) - 1/2} \le \mathrm {const} \times |m|^p \, \lambda _\mathrm{min}^{-|m|} =: K_m, \end{aligned}$$

where p is a positive integer which does not depend on \(m\,\).

To complete the computation, notice that the number of m with \(|m| = n\) is bounded by \(n^{d-1}\) times a constant. Therefore we get a finite sum

$$\begin{aligned} \sum K_m \le \mathrm {const} \times \sum _{n=0}^\infty n^{p+d-1} \lambda _\mathrm {min}^{-n} < \infty. \end{aligned}$$

The holomorphicity follows by inserting \(L(x) = q(P_1) R(x) q(P_2)\) in the formula of Proposition 3.23 and interchanging the \(\bar{\partial }_x\)-operator with the integral. \(\square \)

3.5 Compatibility with Lie algebra representation

We now show that the semigroup representation \(R : \, \widetilde{ \mathrm {H}}(W^s) \rightarrow \mathrm {End}({\fancyscript{A}}_V)\) is compatible with the \(\mathfrak {sp}\)-representation

$$\begin{aligned} \mathfrak {sp} \mathop {\longrightarrow }\limits ^{\tau ^{-1}} \mathfrak {w} (W) \mathop {\longrightarrow }\limits ^{q} \mathfrak {gl} (\mathfrak {a}(V)), \quad \mathfrak {a}(V) = \mathrm {S}(V^*). \end{aligned}$$

Let \(h \in \mathrm {H}(W^s)\) and \(Y \in \mathfrak {sp}\,\). Then, since the semigroup \(\mathrm {H}(W^s)\) is open in \(\mathrm {Sp}\,\), there exists some \(\varepsilon > 0\) so that the curve \([-\varepsilon , \varepsilon ] \ni t \mapsto \mathrm {e}^{tY} h\) lies in \(\mathrm {H} (W^s)\). Fix some point \(x \in \tau _H^{-1}(h)\) and let \(t \mapsto \mathrm {e}^{tY} \cdot x\) denote the lifted curve in \(\widetilde{\mathrm {H}}(W^s)\).

Lemma 3.6

For all \(x \in \widetilde{\mathrm {H}}(W^s)\) and all \(Y \in \mathfrak {sp}\) it follows that

$$\begin{aligned} \frac{d}{dt}\bigg \vert _{t = 0} R(\mathrm {e}^{tY} \cdot x) = q (\tau ^{-1}(Y)) R(x). \end{aligned}$$

Proof

Recall that the operator R(x) is the result of integrating the Heisenberg translations \(T_w\) against the Gaussian density \(\gamma _x(w)\, \mathrm {dvol}(w)\). Thus

$$\begin{aligned} \frac{d}{dt}\bigg \vert _{t = 0} R(\mathrm {e}^{tY} \cdot x) = \int _{W_\mathbb {R}} \frac{d}{dt}\bigg \vert _{t = 0} \gamma _{\, \mathrm {e}^{tY} \cdot x}(w) T_w \, \mathrm {dvol}(w). \end{aligned}$$
(3.13)

For \(w_1 , w_2 \in W\) the linear transformation \(Y: \, w \mapsto w_1 A(w_2 , w) + w_2 A(w_1 , w)\) is in \(\mathfrak {sp}\,\), and \(\mathfrak {sp}\) is spanned by such transformations. It is therefore sufficient to prove the statement of the lemma for Y of this form. Hence let \(Y := w_1 A(w_2 ,\cdot ) + w_2 A(w_1 ,\cdot )\) and observe that the corresponding element in the Weyl algebra is

$$\begin{aligned} \tau ^{-1}(Y) = {\textstyle {\frac{1}{2}}} (w_1 w_2 + w_2 w_1). \end{aligned}$$

Now, defining \(T_w\) for \(w \in W\) by \(T_w = \mathrm {e}^{\mathrm {i} q(w)}\) as before, we have

$$\begin{aligned} q(\tau ^{-1}(Y)) R(x) = - \frac{d^2}{dt_1 dt_2}\bigg \vert _{ t_1 = t_2 = 0} T_{t_1 w_1 + t_2 w_2} R(x). \end{aligned}$$
(3.14)

Therefore, for \(\tilde{w} := t_1 w_1 + t_2 w_2\) consider the expression

$$\begin{aligned} T_{\tilde{w}} R(x) = \int _{W_\mathbb {R}} \gamma _x(w) T_{\tilde{w}} T_w \, \mathrm {dvol}(w). \end{aligned}$$

Using \(T_{\tilde{w}} T_w = \mathrm {e}^{-\frac{1}{2} A(\tilde{w},w)} T_{\tilde{w} + w}\) and shifting integration variables \(w \rightarrow w - \tilde{w}\) we obtain

$$\begin{aligned} T_{\tilde{w}} R(x) = \int _{W_\mathbb {R}} \gamma _x(w - \tilde{w})\, \mathrm {e}^{-\frac{1}{2} A(\tilde{w},w)} T_w \, \mathrm {dvol}(w)\;. \end{aligned}$$
(3.15)

Comparing Eqs. (3.14,3.15) with (3.13) we see that the formula of the lemma is true if

$$\begin{aligned} \frac{d}{dt} \bigg \vert _{t = 0} \gamma _{\,\mathrm {e}^{tY} \cdot x}(w) = - \frac{d^2}{dt_1 dt_2} \bigg \vert _{t_1 = t_2 = 0} \gamma _x(w - t_1 w_1 - t_2 w_2) \, \mathrm {e}^{-\frac{1}{2} A(t_1 w_1 + t_2 w_2 , w)}. \end{aligned}$$

But checking this equation is just a simple matter of computing derivatives. Recall that \(\gamma _x(w) = \phi (x)\, \mathrm {e}^{- \frac{1}{4} A(w,\, sa_x w)}\) and \(\phi (x) = \mathrm {Det}^{1/2}(a_x + s)\). Writing \(h := \tau _H (x)\) and using \(\mathrm {Tr}\, Y=0\) one computes the left-hand side to be

$$\begin{aligned} \frac{d}{dt} \bigg \vert _{t = 0} \gamma _{\,\mathrm {e}^{tY} \cdot x}(w) = \gamma _x(w) \left( {\textstyle {\frac{1}{4}}} \mathrm {Tr}\, (Y s a_x) + {\textstyle {\frac{1}{2}}} A(h(1 - h)^{-1} w , Y h (1 - h)^{-1} w) \right). \end{aligned}$$

On substituting \(Y = w_1 A(w_2 , \cdot ) + w_2 A(w_1 , \cdot )\), this expression immediately agrees with the result of taking the two derivatives on the right-hand side. \(\square \)

4 Spinor–oscillator character

The purpose of this chapter is to introduce the character of the spinor–oscillator representation of a certain super-semigroup \((\widetilde{H},\mathcal {F})\) in the orthosymplectic Lie supergroup of \(W = V \oplus V^*= W_0 \oplus W_1\,\). A summary of this short chapter is as follows.

Referring to [1] and [12] for details, we begin by briefly recalling the basic notions of Lie supergroups (in this case semigroups) and their representations. Next, we recall from Sect. 2.6 the infinite-dimensional representation of the complex Lie superalgebra \(\mathfrak {osp}(W)\) on the spinor–oscillator module \(\fancyscript {A}_V\) (a.k.a. Fock space). The complex Lie group \(G = \mathrm {SO}(W_1) \times \mathrm {Sp}(W_0)\) associated to the even part of \(\mathfrak {osp}(W)\) is the base manifold of the associated Lie supergroup \(\mathrm {OSp}\,\). As a 2 : 1 covering space of the domain \(\mathrm {SO} (W_1) \times \mathrm {H} (W_0^s)\) in G, the semigroup \(\widetilde{H} := \mathrm {Spin}(W_1) \times _{\mathbb {Z}_2} \widetilde{\mathrm {H}}(W_0^s)\) inherits complex supermanifold structure. The \(\mathfrak {osp}\)-representation is integrated to \(\widetilde{H}\) as a super-semigroup representation on \(\fancyscript {A}_V\). Using the character of a supergroup representation as a model, we introduce a superfunction \(\chi \) on \(\widetilde{H}\) which we regard as the character of this semigroup representation. We refer to it as the spinor–oscillator character for short.

In the last subsection of the chapter we let \(V = U \otimes \mathbb {C}^N\) and recall the setting of a Howe dual pair \((\mathfrak {g},\mathfrak {k}) = (\mathfrak {osp}(U \oplus U^*),\mathfrak {o}_N)\) or \((\mathfrak {osp} (\widetilde{U} \oplus \widetilde{U}^*),\mathfrak {sp}_N)\). In this setting, we evaluate the spinor–oscillator character \(\chi \) in two respects: (1) we Haar average it over K, which amounts to projecting from \(\fancyscript {A}_V\) to the submodule \(\fancyscript {A}_V^K\) of K-invariants, and (2) we restrict it to a toral set \(T^+ \subset \widetilde{H}\) in a super-semigroup over \(\mathfrak {g}\,\). We then show that the restricted character \(\chi _{T^+} (t)\) coincides with the integral function I(t) of Eq. (1.1).

4.1 Background on Lie supergroups and their representations

Given a (finite-dimensional) complex Lie superalgebra \(\mathfrak {g}\,\), an associated complex Lie supergroup is a ringed space \((G,\mathcal {F})\) where G is a complex Lie group associated to \(\mathfrak {g}_0\) which in addition integrates the representation of \(\mathfrak {g}_0\) on \({\mathfrak g}_1\). The group operations on G lift to sheaf morphisms that satisfy the natural conditions imposed by associativity, inverse, and fixing the identity. Uniqueness theorems allow us to choose \(\mathcal {F}\) as the sheaf of germs of holomorphic functions with values in the Grassmann algebra \(\Lambda := \wedge \mathfrak {g}_1^*\). Here we follow Berezin’s construction of the group structure, in particular his construction of the derivations associated to \(\mathfrak {g}\) which are defined by left and right ‘multiplication’.

The first step of this construction is to consider the complex Lie algebra

$$\begin{aligned} \tilde{\mathfrak {g}} = \Lambda _0 \otimes \mathfrak {g}_0 + \Lambda _1 \otimes \mathfrak {g}_1 \,, \end{aligned}$$

with Lie bracket

$$\begin{aligned}{}[\alpha \otimes X , \beta \otimes Y] := \beta \alpha \otimes [X,Y] \,, \end{aligned}$$

where \(\Lambda _0\) is the subspace of even elements in \(\Lambda \) including the degree-zero elements \(\wedge ^0 \mathfrak {g}_1^* = \mathbb {C}\), and \(\Lambda _1\) is the subspace of odd elements. Letting \(\Lambda _0'\) denote the subspace of \(\Lambda _0\) without the complex line \(\wedge ^0 \mathfrak {g}_1^*\), we observe that \(\mathfrak {n} := \Lambda _0^\prime \otimes \mathfrak {g}_0 + \Lambda _1 \otimes \mathfrak {g}_1\) is an ideal in \(\tilde{g}\) consisting of nilpotent elements. It can therefore be integrated to a simply connected nilpotent complex Lie group. This leads to the semidirect-product complex Lie group \(\tilde{G} = G N\) which is associated to \(\tilde{g}\).

Grassmann analytic continuation (GAC) is a process that extends functions in the structure sheaf \(\mathcal {F}\) of G to holomorphic functions with values in \(\Lambda \) on the complex Lie group \(\tilde{G}\) ([1], p. 250–257; see also [12], Sect. §\(1\)). The Lie supergroup structure morphisms of \(\mathcal {F}\) are defined by the standard complex Lie group structure of \(\tilde{G}\). Indeed, the left and right representations of \(\mathfrak {g}\) as derivations on \(\mathcal {F}\) are defined via the standard invariant vector fields defined by \(\tilde{\mathfrak {g}}\) on \(\tilde{G}\,\); and representations of the Lie supergroup \((G,\mathcal {F})\) are defined by representations (with coefficients in \(\Lambda \)) of the complex Lie group \(\tilde{G}\). We will sketch some aspects of this below, referring to [1] and [12] for details.

Our goal here is to introduce the spinor–oscillator representation and define its character. While this is a super-semigroup representation on an infinite-dimensional space, we begin by recalling the basics of Lie supergroup representations on finite-dimensional spaces. In abstract terms, a representation of a Lie supergroup \((G, \mathcal {F}_G)\) is a morphism \((\rho , \rho ^*)\) of Lie supergroups to \((\mathrm {GL}(V), \mathcal {F}_{\mathrm {GL}(V)})\), where \(V = V_0 \oplus V_1\) is some graded vector space. Here we are interested in holomorphic representations, so that \(\mathrm {GL}(V)\) is the complex Lie group \(\mathrm {GL}(V_0)\times \mathrm {GL}(V_1)\) and \(\mathcal {F}_{\mathrm {GL} (V)}\) is its standard matrix structure sheaf with values in the Grassmann algebra \(\Lambda \,\). The map \(\rho : \; G \rightarrow \mathrm {GL}(V_0) \times \mathrm {GL}(V_1)\) is a holomorphic homomorphism of complex Lie groups.

Let us assume that we are given a representation \(\rho _*:\;\mathfrak {g}\rightarrow \mathfrak {gl}(V)\), which we extend to \(\rho _*: \; \tilde{\mathfrak {g}} \rightarrow \Lambda _0 \otimes \mathfrak {gl}(V)_0 + \Lambda _1 \otimes \mathfrak {gl} (V)_1\) by \(\rho _*(\alpha \otimes X) = \alpha \otimes \rho _*(X)\). With \(\rho \) as above we then explicitly construct the morphism \(\rho ^*\), and hence the character \(\rho ^*(\mathrm {STr})\), as follows. First, writing elements \(\tilde{g} \in \tilde{G}\) as \(\tilde{g} = g \exp (\Xi )\) with \(g\in G\) and \(\Xi \in \mathfrak {n}\,\), we consider the Lie group representation \({\tilde{\rho}}:{\tilde{G}}\rightarrow \Lambda\otimes \text{End}(V)\) given by \(g{\text{e}}^{\Xi}\mapsto\rho(g){\text{e}}^{\rho_{\ast}(\xi)}.\) By the \(\mathbb{Z}_2\)-grading \(V=V_0\otimes V_1\) the image matrices are of the form \(\left(\begin{array}{cc}A&B\\ C&D \end{array}\right),\) where the coefficients in A and D are elements of \(\Lambda_0\) and those in B and C are elements of \(\Lambda_1.\)Now if f is a superfunction in \(\mathcal {F}_{\mathrm {GL}(V)}\) with Grassmann analytic continuation \(\tilde{f}\), then \(\tilde{f} \circ \tilde{\rho }\) is a \(\Lambda \)-valued holomorphic function on \(\tilde{G}\) which is the GAC of a function \(\rho ^*(f)\) in \(\mathcal {F}_G\,\). To determine the latter, one restricts \(\tilde{f} \circ \tilde{\rho }\) to a characteristic subset \(\Gamma (\xi _1, \ldots ,\xi _m)\) which is defined by a basis \(\{\xi _1, \ldots ,\xi _m\}\) of \(\mathfrak {g}_1^*\). If \(\{F_1, \ldots ,F_m\}\) denotes the dual basis, \(\Gamma (\xi _1,\ldots ,\xi _m)\) is the image of the map \(G\rightarrow \tilde{G}\) given by \(g \mapsto g\,\mathrm {e}^{\,\sum \xi _j \, F_j}\), so that \(\rho ^*(f) := \tilde{f}(\rho (g)\,\mathrm {e}^{\,\sum \xi _j\, \rho _* (F_j)})\). The character of the representation \((\rho , \rho ^*)\) then is the superfunction in \(\mathcal {F}_G\) which is defined in the expected way: \(\chi (g) := \mathrm {STr}\,(\rho (g)\, \mathrm {e}^{\,\sum \xi _j\, \rho _*(F_j)})\).

4.2 Character of the spinor–oscillator representation

In Sect. 3 we constructed a semigroup representation \(R \equiv R_0 :\; \widetilde{\mathrm {H}}(W_0^s)\rightarrow \mathrm {End}(\fancyscript {A}_{V_0})\) and also a group representation \(R^\prime :\; \mathrm {Mp}\rightarrow \mathrm {U} (\fancyscript {A}_{V_0})\), which exponentiate the oscillator representation \(\mathfrak {sp}(W_0) \rightarrow \mathrm {End}(\fancyscript {A}_{V_0})\). These are compatible in that \(\mathrm {Mp}\) acts on \(\widetilde{\mathrm {H}} (W_0^s)\) by translation on the left and right, and \(R_0(g_1 x \, g_2) = R'(g_1) R_0(x) R'(g_2)\) for all \(g_1 , g_2 \in \mathrm {Mp}\) and \(x \in \widetilde{\mathrm {H}} (W_0^s)\).

In the same vein, the complex spinor representation \(\mathfrak {o}(W_1) \rightarrow \mathfrak {gl}(\wedge V_1^*)\) exponentiates to a holomorphic Lie group representation \(R_1:\, \mathrm {Spin}(W_1) \rightarrow \mathrm {GL}(\wedge V_1^*)\). Consider now the tensor product \(\fancyscript {A}_V := \wedge V_1^*\otimes \fancyscript {A}_{V_0}\,\). By trivial extension, our representations \(R_0, R_1\) give rise to representations \(R_0 :\; \widetilde{\mathrm {H}} (W_0^s)\rightarrow \mathrm {End} (\fancyscript {A}_V)\) and \(R_1 :\, \mathrm {Spin} (W_1) \rightarrow \mathrm {GL} (\fancyscript {A}_V)\). Note that \(R_1\) and \(R_0\) commute, as they act on different factors of the tensor product \(\fancyscript {A}_V\). Note also that the group \(\mathbb {Z}_2\) acts on \(\widetilde{\mathrm {H}}(W_0^s)\) and \(\mathrm {Spin} (W_1)\) by deck transformations of the 2 : 1 coverings \(\widetilde{\mathrm {H}} (W_0^s) \rightarrow \mathrm {H}(W_0^s)\) and \(\mathrm {Spin}(W_1) \rightarrow \mathrm {SO} (W_1)\). The non-trivial element of \(\mathbb {Z}_2\) is represented by a sign change, \(R_0 \rightarrow - R_0\) and \(R_1 \rightarrow - R_1\,\).

In the sequel, let \(\widetilde{H}\) denote the semigroup

$$\begin{aligned} \widetilde{H} := \mathrm {Spin}(W_1) \times _{\mathbb {Z}_2} \widetilde{\mathrm {H}} (W_0^s) . \end{aligned}$$

Given the representations \(R_1\) and \(R_0\,\), we form the semigroup representation

$$\begin{aligned} R : \; \widetilde{H} \rightarrow \mathrm {End}({\fancyscript{A}}_V)\;, \quad [g_1 ; g_0] \equiv [-g_1 ; -g_0] \mapsto R_1(g_1) R_0(g_0). \end{aligned}$$

From Sect. 2.6 and our labors in Sect. 3 we know that the semigroup representation R does the job of partially integrating the spinor–oscillator representation \(q :\; \mathfrak {g} \rightarrow \mathfrak {gl} (\fancyscript {A}_V)\) of \(\mathfrak {g} = \mathfrak {osp}(W) \subset \mathfrak {q}(W)\). Thus, in summary, what we have is a representation \((R,R_*\equiv q)\) on \({\fancyscript{A}}_V\) of the complex Lie super-semigroup \((\widetilde{H},\mathcal {F})\) with complex Lie superalgebra \(\mathfrak {g}\,\).

We now proceed according to the blueprint of the finite-dimensional setting in Sect. 4.1. Since the semigroup structure of \(\widetilde{H}\) comes from the semigroup structure of the complex domain \(\mathrm {H}(W_0^s)\) in \(\mathrm {Sp}(W_0)\), the mapping \(R:\; \widetilde{H} \rightarrow \mathrm {End}(\fancyscript {A}_V)\) is a semigroup morphism which is holomorphic in the appropriate infinite-dimensional sense. In order to regard this as a morphism of super-semigroups, we extend it as in the finite-dimensional setting to a semigroup representation \(\tilde{R}\) with values in \(\Lambda \otimes \mathrm {End}(\fancyscript {A}_V)\). Just as in the finite-dimensional case, the morphism \(R^*\) is defined by pulling back superfunctions defined on the image space of endomorphisms. By an immediate extension of Proposition 3.24, our semigroup representation is such that for \(x \in \widetilde{H}\) the operators R(x) multiplied by any polynomial in the \(\mathfrak {osp}\)-generators are trace class. Thus \(R^*(\mathrm {STr})\) is well-defined and, restricting to the characteristic set as above, we define the character of the spinor–oscillator super-semigroup representation by

$$\begin{aligned} \chi (x) := \mathrm {STr}_{{\fancyscript{A}}_V} R(x)\, \mathrm {e}^{\,\sum \xi _j\,q(F_j)} , \end{aligned}$$

which is a holomorphic function on \(\widetilde{H}\) with values in \(\wedge \mathfrak {g}_1^*= \wedge \mathfrak {osp}_1^*\).

4.3 Identification of the restricted character with I(t)

Here we show that restriction of \(\chi \) to a certain toral set in \(\widetilde{H}\) yields the integrand Z(tk) of the autocorrelation function I(t) described in Sect. 1. In other words, we show that Z(tk) can be expressed as the supertrace of an operator on the spinor–oscillator module \({\fancyscript {A}}_V\). Then, by taking the Haar average over the compact group K, we identify I(t) with the supertrace of an operator on the submodule \(\fancyscript {A}_V^K\) of K-invariants.

We begin with a summary of the relevant facts. From Proposition 3.15 we recall the basic conjugation rule for the oscillator representation:

$$\begin{aligned} R_0(x)\, q(w) = q(\tau _H(x)\,w) R_0(x) , \end{aligned}$$

where \(x \in \widetilde{ \mathrm {H}}(W_0^s)\), \(w \in W_0\,\), and \(\tau _H(x) \in \mathrm {H}(W_0^s) \subset \mathrm {Sp}(W_0)\). On the side of the spinor representation, the corresponding conjugation formula is

$$\begin{aligned} R_1(y)\, q(w) = q(\tau _S(y)\, w) R_1(y) , \quad w \in W_1 \,, \quad y \in \mathrm {Spin}(W_1) . \end{aligned}$$

This defines the 2:1 covering homomorphism \(\mathrm {Spin}(W_1) \rightarrow \mathrm {SO}(W_1)\), \(y \mapsto \tau _S(y)\), which exponentiates the isomorphism of Lie algebras \(\tau :\, \mathfrak {s} \cap \mathfrak {c}_2 (W_1) \rightarrow \mathfrak {o}(W_1)\).

In Sect. 3 we have discussed the oscillator character \(\phi \) in great detail. In particular, we know that

$$\begin{aligned} \mathrm {Tr}\, R_0(x) = \phi (x), \quad \phi (x)^2 = \mathrm {Det}^{-1}(\mathrm {Id}_{W_0} - \tau _H(x)) . \end{aligned}$$

An analogous result is known for the case of the spinor representation; see, e.g., the textbook [2]. Defining the spinor character as the supertrace with respect to the canonical \(\mathbb {Z}_2\)-grading of the spinor module, one has

$$\begin{aligned} \mathrm {STr}\, R_1(y) = \psi (y) , \quad \psi (y)^2 = \mathrm {Det}(\mathrm {Id}_{W_1} - \tau _S(y)) . \end{aligned}$$

Thus the spinor character, just like the oscillator character, is a square root. By taking the supertrace over the total Fock representation space, we obtain the formula

$$\begin{aligned} \mathrm {STr}_{\fancyscript {A}_V} R_1(y) R_0(x) = \phi (x) \psi (y) =: \sqrt{ \frac{\mathrm {Det}(\mathrm {Id}_{W_1} - \tau _S(y))}{\mathrm {Det}(\mathrm {Id}_{W_0} - \tau _H(x))}} . \end{aligned}$$
(4.1)

For \(W_0 = W_1\,\), the case of our interest, \(\mathrm {Spin} (W_1)\) intersects with \(\widetilde{\mathrm {H}} (W_0^s)\) and the square root \(\psi (y)\) is defined in such a way that \(\psi (y) = \phi (x)^{-1}\) for \(x = y \in \mathrm {Spin}(W_1) \cap \widetilde{\mathrm {H}} (W_0^s)\).

Now let \(U = U_0 \oplus U_1\) be a \(\mathbb {Z}_2\)-graded vector space, and let \(V = U \otimes \mathbb {C}^N\). The Lie group \(\mathrm {GL}(V_1) \times \mathrm {GL}(V_0)\) acts on \(W = (V_1^{} \oplus V_1^*) \oplus (V_0^{} \oplus V_0^*)\) by

$$\begin{aligned} (g_1,g_0) . (v_1 \oplus \varphi _1 \oplus v_0 \oplus \varphi _0) = (g_1 v_1) \oplus (\varphi _1 \circ g_1^{-1}) \oplus (g_0 v_0) \oplus (\varphi _0 \circ g_0^{-1}) . \end{aligned}$$

This action serves to realize the group \(G := (\mathrm {GL}(U_1) \times \mathrm {GL}(U_0)) \times _{\mathbb {C}^\times } \mathrm {GL}(\mathbb {C}^N)\) as a subgroup of \(\mathrm {GL}(V_1) \times \mathrm {GL}(V_0) \subset \mathrm {SO}(W_1) \times \mathrm {Sp}(W_0)\). For the purpose of letting G act on \(\fancyscript {A}_V\), let this representation be lifted to that of a double covering \(\widetilde{G}\) of G by

$$\begin{aligned} \iota : \; \widetilde{G} \hookrightarrow \widetilde{H} = \mathrm {Spin} (W_1) \times _{\mathbb {Z}_2} \widetilde{\mathrm {H}} (W_0^s) . \end{aligned}$$

Our next statement gives the value of the spinor–oscillator character on \((t_1,t_0;g) \in \widetilde{G}\) where \(g \in \mathrm {GL}(\mathbb {C}^N)\) and \(t_s = \mathrm {diag}(t_{s,1}, \ldots , t_{s,\,n})\) are diagonal matrices in \(\mathrm {GL}(U_s)\) (for \(s = 0, 1\)) or rather, in the pertinent double covering. Note that since the action of \(\widetilde{G}\) on the spinor–oscillator module \(\mathfrak {a}(V)\) is degree-preserving, there is no longer any need to work with the completion \(\fancyscript {A}_V\) to a Hilbert space.

Lemma 4.1

If \(\dim U_0 = \dim U_1 = n\) and \(| t_{0,j} | > 1\) for all \(j = 1, \ldots , n\,\), then

$$\begin{aligned} \mathrm {STr}_{\mathfrak {a}(V)}(R\circ \iota )(t_1,t_0;g) = \prod _{j=1}^n \sqrt{\frac{t_{1,j}}{t_{0,j}}}^{\,N}\frac{\mathrm {Det}(\mathrm {Id}_N - (t_{1,j}\,g)^{-1})}{\mathrm {Det}(\mathrm {Id}_N-(t_{0,j}\,g)^{-1})}\;. \end{aligned}$$

Proof

Since \(t_1\) and \(t_0\) are assumed to be of diagonal form, the statement holds true for a general value of n if it does so for the special case of \(n = 1\). Hence let \(n = 1\).

In that case \(t_1\) and \(t_0\) are single numbers and \(t_s g\) acts on \(W_s = V_s^{} \oplus V_s^*\simeq \mathbb {C}^N \oplus (\mathbb {C}^N)^*\) as \((t_s g).(v \oplus \varphi ) = (t_s g v) \oplus \varphi \circ (t_s g)^{-1}\) for \(s = 0,1\). From equation (4.1) we then have

$$\begin{aligned} \mathrm {STr}_{\mathfrak {a}(V)}(R\circ \iota )(t_1,t_0;g) = \sqrt{\frac{ \mathrm {Det} (\mathrm {Id}_N - t_1 g)\,\mathrm {Det}(\mathrm {Id}_N - (t_1 g)^{-1})}{\mathrm {Det}(\mathrm {Id}_N-t_0 g)\,\mathrm {Det} (\mathrm {Id}_N-(t_0 g)^{-1})}}, \end{aligned}$$

which turns into the stated formula on pulling out a factor of \(\mathrm {Det}(-t_1 g) / \mathrm {Det}(- t_0 g)\) from under the square root. (Of course, the double covering of \(\mathrm {GL}(U_1) \times \mathrm {GL}(U_0)\) is to be used in order to define this square root globally.) \(\square \)

In the formula of Lemma 4.1 we now set \(t_{1,j} = \mathrm {e}^{\mathrm {i}\psi _j}\) and \(t_{0,j} = \mathrm {e}^{\phi _j}\). We then put \(g^{-1} \equiv k \in K\) and integrate against Haar measure dk of unit mass on K. This integral and the summation defining the supertrace can be interchanged, as \(\mathrm {STr}_{\mathfrak {a}(V)} (R\circ \iota )(t_1,t_0;k^{-1})\) is a finite sum of power series and the conditions \(\mathfrak {Re} \, \phi _j > 0\) ensure uniform and absolute convergence. Averaging over K with respect to Haar measure has the effect of projecting from \(\mathfrak {a}(V)\) to the K-trivial isotypic component \(\mathfrak {a}(V)^K\), thus we arrive at

$$\begin{aligned} \mathrm {STr}_{\mathfrak {a}(V)^K} (R\circ \iota )(t_1,t_0;\mathrm {Id}) =\mathrm {e}^{(N/2) \sum _j (\mathrm {i}\psi _j - \phi _j)}\, \int \limits _K \prod _{j=1}^n \frac{\mathrm {Det}(\mathrm {Id}_N - \mathrm {e}^{- \mathrm {i}\psi _j}\, k)}{\mathrm {Det}( \mathrm {Id}_N - \mathrm {e}^{- \phi _j}\, k)} \, dk. \end{aligned}$$
(4.2)

In the case of an even dimension N, the domain of definition of this formula is a complex torus \(T^+ := T_1 \times T_0^+\) where \(T_1 = (\mathbb {C}^\times )^n\) and \(T_0^+ \subset (\mathbb {C}^\times )^n\) is the open subset determined by the conditions \(|t_{0,j} | = \mathrm {e}^{ \mathfrak {Re}\, \phi _j} > 1\) for all \(j\,\). For odd N we must continue to work with a double cover (also denoted by \(T^+\)) to take the square root \(\mathrm {e}^{(N/2) \sum _j (\mathrm {i}\psi _j - \phi _j)}\).

Let now \(\mathfrak {g}\) be the Howe dual partner of \(\mathrm {Lie}(K)\) in \(\mathfrak {osp}(W)\). We know from Proposition 2.1 that \(\mathfrak {g} = \mathfrak {osp}(U \oplus U^*)\) for \(K = \mathrm {O}_N\) and \(\mathfrak {g} = \mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)\) for \(K = \mathrm {USp}_N\,\). Recall also from Sect. 2.6.1 that the \(\mathfrak {g}\)-representation on \(\mathfrak {a}(V)^K\) is irreducible and of highest weight \(\lambda _N = (N/2) \sum _j (\mathrm {i}\psi _j - \phi _j)\). Denote by \(\Gamma _\lambda \) the set of weights of this representation. Let \(B_\gamma = (-1)^{ |\gamma |} \dim {\mathfrak {a}(V)^K}_\gamma \) be the dimension of the weight space \({\mathfrak {a}(V)^K}_\gamma \) multiplied with the correct sign to form the supertrace.

Corollary 4.1

On \(T^+\) we have

$$\begin{aligned} \sum _{\gamma \in \Gamma _\lambda } B_\gamma \, \mathrm {e}^\gamma = \mathrm {e}^{\lambda _N} \int \limits _K \prod _{j=1}^n \frac{ \mathrm {Det}(\mathrm {Id}_N - \mathrm {e}^{-\mathrm {i}\psi _j} \, k)}{\mathrm {Det}(\mathrm {Id}_N - \mathrm {e}^{-\phi _j}\, k)}\, dk. \end{aligned}$$

Remark 4.1

On the right-hand side we recognize the correlation function (see Sect. 1) which is the object of our study and, as we have explained, is related to the character of the irreducible \(\mathfrak {g}\)-representation on \(\mathfrak {a}(V)^K\). The left-hand side gives this character (restricted to the toral set \(T^+\)) in the form of a weight expansion, some information about which has already been provided by Corollary 2.3 of Sect. 2.6.1.

5 Proof of the character formula

Here we complete our task of deriving the formula (1.3),

$$\begin{aligned} I(t)=\sum _{[w] \in W \!\!/ W_\lambda } \mathrm {e}^{ w(\lambda _N)} \frac{\prod _{\beta \in \Delta _{\lambda ,1}^+}(1-\mathrm {e}^{ -w(\beta )})}{\prod _{\alpha \in \Delta _{\lambda ,0}^+} (1 - \mathrm {e}^{-w(\alpha )})} (\ln t). \end{aligned}$$
(5.1)

Let us sketch this derivation, thereby giving an outline of this chapter.

Throughout we will be concerned with the irreducible \(\mathfrak {g} \)-representation on the subspace \({\fancyscript {A}}_V^K\) of K-invariants in Fock space, where \(\mathfrak {g} \subset \mathfrak {osp}(V \oplus V^*)\) denotes the Howe partner defined by the K-action on \(V \oplus V^*\), \(V = U \otimes \mathbb {C}^N\). Here our dealings with the ‘big’ Lie superalgebra \(\mathfrak {osp}(V \oplus V^*)\) in Sect. 4.2 are repeated at the level of the ‘small’ Lie superalgebra \(\mathfrak {g}\,\). In particular, we associate with \(\mathfrak {g}\) a complex super-semigroup \((\widetilde{H}^\prime , \mathcal {F})\) which serves to partially integrate the \(\mathfrak {g}\)-representation on \(\fancyscript {A}_V^K\).

We then focus our attention on the spinor–oscillator character \(\chi \) pulled back to \(\widetilde{H}^\prime \times K\) and Haar averaged over K. Let \(\chi ^\prime \) be the resulting superfunction on \(\widetilde{H}^\prime \). Its restriction \(\chi _{T^+}^\prime \) to a toral set \(T^+ \subset \widetilde{H}^\prime \) is the numerical function that we were led to consider in Sect. 4.3; by Eq. (4.2) it is the function I(t) which is to be computed.

To prove the formula (5.1) for the character I(t), we study the full superfunction \(\chi ^\prime : \; \widetilde{H}^\prime \rightarrow \wedge \mathfrak {g}_1^*\). General methods show that \(\chi ^\prime \) has two distinctive properties: (i) it is radial with respect to the vector fields given by \(\mathfrak {g}\,\), and (ii) it is an eigenfunction for every Laplace-Casimir operator, i.e., every differential operator D(I) associated to a Casimir invariant I, on \((\widetilde{H} ^\prime , \mathcal {F})\). Hence we look closely at the differential equations \(D(I) \chi ^\prime = \lambda \chi ^\prime \). For \(I_\ell = \sum \big ( \phi _j^{2\ell } - (-1)^\ell \psi _j^{2\ell } \big )\), \(\ell \in \mathbb {N}\), regarded as an element of the center of the universal enveloping algebra of \(\mathfrak {g}\), we show that \(D(I_\ell ) \chi ^\prime = 0\). It follows that the radial part of \(D(I_\ell )\), which is the differential operator corresponding to \(D(I_\ell )\) on \(T^+\), annihilates the restricted character \(I(t) = \chi _{T^+}^\prime (t)\).

Hence we come to the final steps of our proof of the formula for I(t), the first of which is to derive explicit formulas for the radial parts of the Laplace-Casimir operators \(D(I_\ell )\). For this we implement a good portion of Berezin’s theory of radial operators, which has been adapted to the context of the present paper in [12]. Using this theory, we show that the character \(\chi _{T^+}^\prime \) is annihilated by an infinite set of differential operators \(D_\ell \circ J\) (\(\ell \in \mathbb {N}\)), where

$$\begin{aligned} D_\ell = \sum _{j=1}^n \frac{\partial ^{2\ell }}{\partial \phi ^{2\ell }_j}- (-1)^\ell \sum _{j=1}^n \frac{\partial ^{2\ell }}{\partial \psi ^{2\ell }_j} \end{aligned}$$
(5.2)

and J is the square root of a certain Jacobian. These differential equations alone are not enough to pin down a Weyl-group invariant holomorphic function on \(T^+\). However, we are able to derive additional information about the region of the non-zero weights of the Fourier development of \(\chi _{T^+}^\prime \), and we then show that the function on the right-hand side of (5.1) is up to normalization the unique Weyl-group invariant holomorphic function on \(T^+\) which satisfies these additional conditions and is annihilated by the \(D_\ell \circ J\).

5.1 Properties of the character \(\chi ^\prime \)

Here we will be working with the following data:

  • a complex Lie superalgebra \(\mathfrak {g} = \mathfrak {osp}(U \oplus U^*)\) or \(\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)\);

  • a complex Lie super-semigroup \((\widetilde{H}^\prime , \mathcal {F})\) over \(\mathfrak {g}\,\);

  • the character \(\chi ^\prime \) of a representation \((\rho ,\rho _*)\) of \((\widetilde{H}^\prime , \mathcal {F}, \mathfrak {g})\) on \(\fancyscript {A}_V^K\).

To begin, we fill in some details omitted from the introductory part of this chapter.

First of all, to construct \(\widetilde{H}^\prime \) we take \(G \subset \mathrm {SO}(W_1) \times \mathrm {Sp}(W_0)\) to be the complex Lie group associated to the even part \(\mathfrak {g}_0 \subset \mathfrak {g}\) and let \(H^\prime \subset G\) be the semigroup which is defined by intersecting G with \(\mathrm {SO}(W_1) \times \mathrm {H} (W_0^s)\). We then define \(\widetilde{H}^\prime \) to be the pre-image of \(H^\prime \) in the 2:1 covering space \(\widetilde{H} = \mathrm {Spin}(W_1) \times _{\mathbb {Z}_2} \widetilde{\mathrm {H}}(W_0^s)\) of \(\mathrm {SO}(W_1) \times \mathrm {H} (W_0^s)\).

Second, the semigroup representation \(\rho \) emerges in a natural way, as follows. As a Lie semigroup with Lie algebra \(\mathfrak {g}_0 \oplus \mathfrak {k}\) contained in \(\mathfrak {osp}(W)\) (by the isomorphism \(\Psi \) of Sect. 2.3), the direct product \(\widetilde{H}^\prime \times K\) is naturally embedded into \(\widetilde{H}\):

$$\begin{aligned} \iota : \; \widetilde{H}^\prime \rightarrow \widetilde{H} \times K . \end{aligned}$$

The semigroup representation \(\rho : \; \widetilde{H}^\prime \rightarrow \mathrm {End}(\fancyscript {A}_V^K)\) is then defined by pulling back R to \(\widetilde{H}^\prime \times \{\mathrm {Id}\}\) and projecting on \(\fancyscript {A}_V^K\):

$$\begin{aligned} \rho (x)= (R\circ \iota )(x\,,\mathrm {Id})\big \vert _{\fancyscript {A}_V^K}\;. \end{aligned}$$

Third, by the general principles explained earlier, the character \(\chi ^\prime \) of the super-semigroup representation \((\rho ,\rho _*)\) of \((\widetilde{H}^\prime , \mathcal {F} , \mathfrak {g})\) on \(\fancyscript {A}_V^K\) is determined by

$$\begin{aligned} \chi ^\prime (x) = \mathrm {STr}_{{\fancyscript{A}}_V^K}\, \rho (x) \, \mathrm {e}^{\sum \xi _j\, \rho _*(F_j)} . \end{aligned}$$

Here \(\{ F_j \}\) is a basis of \(\mathfrak {g}_1\) as usual, \(\{ \xi _j \}\) is the dual basis of \(\mathfrak {g}_1^*\), and the Lie superalgebra representation \(\rho _*: \; \mathfrak {g} \rightarrow \mathfrak {gl}(\mathfrak {a} (V)^K)\) is obtained by pulling back the spinor–oscillator representation by the canonical embedding (Sect. 2.3) of the Howe dual pair \((\mathfrak {g},\mathfrak {k})\) into \(\mathfrak {osp}(W)\) and projection to \(\mathfrak {a}(V)^K\).

In the current subsection we show that the character \(\chi ^\prime \) is, as would be expected, a radial superfunction. We also show that it is an eigenfunction of every Laplace-Casimir operator D(I) and if \(\mathrm {dim}\, U_0 = \mathrm {dim}\, U_1 = \mathbb {C}^n\), i.e., if we are dealing with \(\mathfrak {g} = \mathfrak {osp}_{2n|2n}\,\), then the Laplace-Casimir operators annihilate \(\chi ^\prime \).

To simplify our notation, we now drop the primes and write \(\widetilde{H}, \chi \) instead of \(\widetilde{H}^\prime , \chi ^\prime \).

5.1.1 Radiality of \(\chi \)

A holomorphic superfunction f on \(\widetilde{H}\) is radial if and only if for every \(X \in \mathfrak {g}\) the sum \(L_X + R_X\) of the derivations defined by the left and right representations of X annihilate it. For a homogeneous element X of \(\mathfrak {g}\) the action of these derivations on f is defined as follows (see [1], p. 258, and [12], Sect. \(1\)). First, one considers the Grassmann analytic continuation (GAC) \(\tilde{f}\) of f. If X is even, then one differentiates \(\tilde{f}\) with respect to the local action of the 1-parameter group \(\mathrm {e}^{tX}\). If X is odd, then one chooses an arbitrary element \(\alpha \in \Lambda _1\) and differentiates \(\tilde{f}\) with respect to the local action of \(\mathrm {e}^{tY}\) where \(Y = \alpha X\). One shows in this latter case that the result is of the form \(\alpha L_X(\tilde{f})\) where \(L_X\) is an odd derivation which does not depend on \(\alpha \). Of course \(\alpha L_X\) could be identically zero; so it might be necessary to extend the Grassmann algebra in order to prevent this from happening unless \(L_X\) vanishes identically. Thus in both the odd and even cases we have an operator \(L_X\) on the sheaf of \(\Lambda \)-valued holomorphic functions on the complex Lie group \(\tilde{G}\). One checks that these operators stabilize the subspace of functions on \(\tilde{G}\) which arise through GAC from \((G,\mathcal {F}_G)\) and that the resulting map \(\mathfrak {g} \rightarrow \mathrm {Der}(\mathcal {F}_G)\), \(X\mapsto L_X\) is a Lie superalgebra morphism. Carrying this out in the analogous way by multiplying the 1-parameter groups \(\mathrm {e}^{-tX}\) and \(\mathrm {e}^{ -tY}\) on the right, one obtains the morphism defined by \(X\mapsto R_X\,\).

The key for showing that the operators \(L_X+R_X\) annihilate the character \(\chi \) is the fact that the GAC \(\tilde{\chi }\) of \(\chi \) is \(\mathrm {STr}\; \tilde{\rho }\), where \(\tilde{\rho }\) is the associated complex Lie semigroup representation of \(\widetilde{H} N\). One defines this by \(\tilde{\rho }(x\, \mathrm {e}^{\Xi }) := \rho (x)\, \mathrm {e}^{\rho _*(\Xi )}\), as before.

Proposition 5.1

The character \(\chi \) is a radial holomorphic superfunction on \(\widetilde{H}\).

Proof

Let \(X \in \mathfrak {g}_1\) and \(Y = \alpha X\) be as above. By using the multiplicative semigroup property, the fact that \(\mathrm {STr}\; [ \tilde{\rho }(g) , \rho _*(Y) ] = 0\), and using the \(\Lambda \)-linearity of \(\mathrm {STr}\) to factor out \(\alpha \), we observe that \(\alpha (L_X + R_X)\) annihilates \(\mathrm {STr}\, \tilde{\rho }\). As we mentioned above, in order to conclude that \(L_X + R_X\) annihilates this, it may be necessary to extend the Grassmann coefficients. For \(X \in \mathfrak {g}_0\) the argument is even simpler, as it isn’t necessary to multiply by \(\alpha \). \(\square \)

5.1.2 The character \(\chi \) is a Laplace-Casimir eigenfunction

Let us emphasize that with or without \(\alpha \), after returning from GAC functions on \(\widetilde{H} N\) to functions on \(\widetilde{H}\), the left derivation of \(\chi \) by \(X \in \mathfrak {g}\) is given with respect to a basis by \((L_X \chi )(x) = \mathrm {STr}\,(\rho _*(X)\,\rho (x) \, \mathrm {e}^{ \sum \xi _j \, \rho _*(F_j)})\). If I is any element of the universal enveloping algebra \(\mathsf {U}(\mathfrak {g})\) and D(I) is the differential operator associated to I by \(X \mapsto L_X\), then

$$\begin{aligned} D(I)\chi (x) = \mathrm {STr}\, (\rho _*(I)\, \rho (x)\, \mathrm {e}^{\,\sum \xi _j \, \rho _*(F_j)})\,. \end{aligned}$$

If I is in the center of \(\mathsf {U}(\mathfrak {g})\), we refer to D(I) as a Laplace-Casimir operator.

Proposition 5.2

\(\chi \) is an eigenfunction of every Laplace-Casimir operator D(I).

Proof

Since I lies in the center of \(\mathsf {U}(\mathfrak {g})\), the operator \(\rho _*(I)\) commutes with all operators defined by \(\mathsf {U} (\mathfrak {g})\) on \(\mathfrak {a}(V)^K\). Now according to Proposition 2.2 the subalgebra \(\mathfrak {g}^{(-2)} \oplus \mathfrak {g}^{(0)} \subset \mathfrak {g}\) of degree-non-increasing operators stabilizes the vacuum space \(\langle 1 \rangle _\mathbb {C} \subset \mathfrak {a}(V)^K\). By the irreducibility of the \(\mathfrak {g}\)-representation on \(\mathfrak {a}(V)^K\) this subalgebra stabilizes no other proper subspace of \(\mathfrak {a}(V)^K\). Therefore, the linear operator \(\rho _*(I)\) stabilizes \(\langle 1 \rangle _\mathbb {C}\) with some eigenvalue \(\lambda (I)\). Furthermore, \(1 \in \mathbb {C} \subset \mathfrak {a}(V)^K\) is a cyclic vector for the action of \(\mathsf {U}(\mathfrak {g})\) on \(\mathfrak {a}(V)^K\). Thus \(\rho _*(I) \equiv \lambda (I)\, \mathrm {Id}_{\mathfrak {a}(V)^K}\) and the desired result follows. \(\square \)

5.1.3 Vanishing of the \(D(I_\ell )\)-eigenvalues

Recall now from Sect. 2.2.2 that for every \(\ell \in \mathbb {N}\) we have a Casimir element \(I_\ell \in \mathsf {U} (\mathfrak {osp})\) of degree \(2\ell \). Recall also that under the assumption of equal dimensions \(V_0 \simeq V_1\) we introduced \(\partial , \widetilde{\partial } \in \mathfrak {osp}_1\, \), \(C = [\partial , \widetilde{\partial } ] \in \mathfrak {osp}_0 \,\), and \(F_\ell \in \mathsf {U} (\mathfrak {osp})\) such that \(I_\ell = [\partial ,F_\ell ]\) and \([\partial ,C] = 0\,\). For the proof of Proposition 5.3 below, we will make use of these objects at the level of \(U_0 \simeq U_1\,\).

Consider now any irreducible \(\mathfrak {osp}\)-representation on a \(\mathbb {Z}_2\)-graded vector space \(\mathcal {V}\) with the property that the \(\mathcal {V}\)-supertrace of \(\mathrm {e}^{-tC}\) (\(t>0\)) exists. Let \(\lambda (I_\ell )\) be the scalar value of the Casimir invariant \(I_\ell \) in the representation \(\mathcal {V}\). Then a short computation using \(I_\ell = [\partial , F_\ell ]\) and \([\partial ,C] = 0\) shows that \(\lambda (I_\ell )\) multiplied by \(\mathrm {STr}_\mathcal {V}\, \mathrm {e}^{-t C}\) vanishes:

$$\begin{aligned} \lambda (I_\ell )\, \mathrm {STr}_\mathcal {V}\, \mathrm {e}^{-t C} = \mathrm {STr}_\mathcal {V}\, \mathrm {e}^{-t C} I_\ell = \mathrm {STr}_\mathcal {V}\, \mathrm {e}^{-t C} [\partial , F_\ell ] = \mathrm {STr}_\mathcal {V}\, [\partial , \mathrm {e}^{-t C} F_\ell ] = 0, \end{aligned}$$

since the supertrace of any bracket is zero. Thus we are facing a dichotomy: either we have \(\mathrm {STr}_\mathcal {V}\, \mathrm {e}^{ -t C} = 0\,\), or else \(\lambda (I_\ell ) = 0\) for all \(\ell \in \mathbb {N}\,\). Now it turns out that our representation \(\mathfrak {a} (V)^K\) realizes the latter alternative, which leads to the following consequence.

Proposition 5.3

Let \(U = U_0 \oplus U_1\) be a \(\mathbb {Z}_2\) -graded vector space with \(U_0 \simeq U_1\,\) and \(\chi \) be the character of the super-semigroup representation of \(\widetilde{H}\) which is the integrated form of the irreducible \(\mathfrak {g}\)-representation on \(\mathfrak {a}(V)^K\) for \(V = U \otimes \mathbb {C}^N\). Then \(D(I_\ell )\chi = 0\) for all \(\ell \in \mathbb {N}\,\).

Remark 5.1

The condition \(U_0 \simeq U_1\) is needed in order for the formula \(I_\ell = [ \partial , F_\ell ]\) of Lemma 2.9 to be available.

Proof

For any real parameter \(t>0\) the supertrace of the operator \(\rho ( \mathrm {e}^{-t C})\) on \(\fancyscript {A}_V^K\) certainly exists and is non-zero. In fact, using formula (4.2) one computes the value as

$$\begin{aligned} \mathrm {STr}_{\fancyscript {A}_V^K}\,\rho (\mathrm {e}^{-t C}) = \int _K \frac{\mathrm {Det}^n (\mathrm {Id}_N-\mathrm {e}^{-t}k)}{\mathrm {Det}^n (\mathrm {Id}_N - \mathrm {e}^{-t}k)}\, dk = 1 \not = 0. \end{aligned}$$

The dichotomy of \(\lambda (I_\ell )\, \mathrm {STr}_{\fancyscript {A}_V^K} \, \rho (\mathrm {e}^{-t C}) = 0\) therefore gives \(D(I_\ell ) \chi = \lambda (I_\ell ) \chi = 0\,\). \(\square \)

5.2 Derivation of the differential equations

Here we outline a foundational result which leads to a proof that the differential operators \(D_\ell \circ J\), where J is the square root of a certain (super-)Jacobian and \(\ell \in \mathbb N\), annihilate I(t). Due primarily to Berezin [1], this result has been adapted to our context in [12].

5.2.1 Radial operators

At this stage, another object enters: a space \(T^+\) which plays the role of maximal complex torus in \(\widetilde{H}\). To introduce it, we recall from the beginning of Sect. 5.1 that we are given a complex semigroup \(H^\prime \) inside the complex Lie group G with Lie algebra \(\mathfrak {g}_0\,\). In terms of this structure, the space \(T^+\) is defined as the pre-image in \(\widetilde{H}\) of the intersection of the standard Cartan torus \(T \subset G\) with \(H^\prime \).

Let now B be a neighborhood (open in \(\widetilde{H}\)) of a regular point \(x \in T^+\). Then if \(f :\; B \rightarrow \wedge g_1^*\) is any radial holomorphic (super-)function, we denote by

$$\begin{aligned} \mathcal {R} f : \; T^+ \cap B \rightarrow \wedge ^0 \mathfrak {g}_1^*= \mathbb {C} \end{aligned}$$

its restriction to a function (with numerical values) on the toral subset. The restriction map \(\mathcal {R}\) so defined is known to be injective, irrespective of the choice of B [12]. As before, for I an element of the universal enveloping algebra \(\mathsf {U}(\mathfrak {g})\), we let D(I) denote the associated differential operator. If I lies in the center of \(\mathsf {U}( \mathfrak {g})\) then D(I) takes radial holomorphic functions to radial holomorphic functions. In this case, we may use the injectivity of the restriction map to define the radial part \(\dot{D}(I)\) by

$$\begin{aligned} \dot{D}(I) \circ \mathcal {R} = \mathcal {R} \circ D(I) . \end{aligned}$$

We now require an understanding of the correspondence \(I \mapsto \dot{D}(I)\) between Casimir invariants and the radial parts of invariant differential operators. While this correspondence can be described in fairly explicit terms and has been the subject of recent research by one of the authors [12], here we only summarize the final outcome needed for the present paper. For this one defines the meromorphic function

$$\begin{aligned} J(t) = \frac{\prod _{\alpha \in \Delta _0^+} 2\sinh \frac{\alpha (\ln t)}{2}}{\prod _{\beta \in \Delta _1^+} 2 \sinh \frac{\beta (\ln t)}{2}}\;, \end{aligned}$$

where \(\Delta ^+ = \Delta _0^+ \cup \Delta _1^+\) is a system of even and odd positive roots (see Sect. 2.6.2).

From Sect. 2.2.2 we again recall that for every \(\ell \in \mathbb {N}\) we have a Casimir element \(I_\ell \in \mathsf {U} (\mathfrak {osp})\) of degree \(2\ell \). We also recall the expression (5.2) for the differential operators \(D_\ell \) in terms of the local coordinates \(\phi _1, \ldots , \phi _n, \psi _1, \ldots , \psi _n\) we have been using all along.

Theorem 5.1

Let the neighborhood \(B \subset \widetilde{H}\) of a regular point \(x \in T^+\) be such that all points of the intersection \(B \cap T^+\) are regular. Then, defining the radial part \(\dot{D} (I_\ell )\) of the Laplace-Casimir operator \(D(I_\ell )\) as above, one has

$$\begin{aligned} \dot{D} (I_\ell ) = J^{-1} (D_\ell + Q_{\ell -1}) \circ J\,, \end{aligned}$$

where \(Q_{\ell -1}\) is a polynomial combination with constant coefficients of the operators \(D_1, \ldots , D_{\ell -1}\) which is of total degree at most \(2\ell - 2\).

Remark 5.2

While some choice of domain B is necessary to ensure that both J(t) and \(J(t)^{-1}\) exist for all \(t \in B \cap T^+\), the expression for \(\dot{D}(I_\ell )\) does not depend on B.

Remark 5.3

The statement of Theorem 5.1 is the local version of a result due to Berezin [1]. The proof is in [12].

5.2.2 The differential equations

In view of the formula for \(\dot{D}(I_\ell )\), and knowing that \(D(I_\ell ) \chi = 0\) for all \(\ell \), the following is a key technical step.

Lemma 5.1

For all \(\ell \in \mathbb {N}\) we have \(D_\ell \, J = 0\).

Proof

One has the relation \(\sinh \frac{x+y}{2} \sinh \frac{x-y}{2} = \frac{1}{2}(\cosh x -\cosh y)\). Hence,

$$\begin{aligned} J = \frac{\prod _{j<k} 4\,(\cosh \phi _j-\cosh \phi _k)(\cos \psi _j -\cos \psi _k)}{\prod _{j,\,k} 2\,(\cosh \phi _j -\cos \psi _k)} \; R \end{aligned}$$

with \(R = \prod _{j=1}^n 2\sinh \phi _j\) or \(R = \prod _{j=1}^n 2\mathrm {i} \sin \psi _j\) depending on whether \(\mathfrak {g} = \mathfrak {osp} (U\oplus U^*)\) or \(\mathfrak {g} = \mathfrak {osp} (\widetilde{U} \oplus \widetilde{U}^*)\). By the Cauchy determinant formula

$$\begin{aligned} \mathrm {Det} \left( \frac{1}{x_j - y_k}\right) _{j,\,k=1,\ldots ,\,n} = \frac{\prod _{j<k} (x_j-x_k)(y_k-y_j)}{\prod _{j,\,k=1}^n (x_j - y_k)}, \end{aligned}$$

we obtain, up to a constant factor,

$$\begin{aligned} J \propto \mathrm {Det} \left( \frac{\sinh \phi _j}{\cosh \phi _j - \cos \psi _k} \right), \quad \text {or} \quad J \propto \mathrm {Det} \left( \frac{\mathrm {i}\sin \psi _j}{\cosh \phi _j - \cos \psi _k} \right) . \end{aligned}$$

Consider the case of \(\mathfrak {g} = \mathfrak {osp}(U \oplus U^*)\). Expanding the determinant J as a sum over permutations and applying the differential operator \(D_\ell \) to the summands, we then obtain \(D_\ell \, J \propto \)

$$\begin{aligned}&D_\ell \sum _{\sigma \in \mathrm {S}_n} (-1)^{|\sigma |} \prod _{j=1}^n \frac{\sinh \phi _j}{\cosh \phi _j -\cos \psi _{\sigma (j)}} = \sum _\sigma (-1)^{|\sigma |} D_\ell \prod _{j=1}^n \frac{\sinh \phi _j}{\cosh \phi _j - \cos \psi _{\sigma (j)}} = \\&\sum _\sigma (-1)^{|\sigma |} \sum _{k=1}^n \prod _{j\ne k} \frac{\sinh \phi _j}{\cosh \phi _j -\cos \psi _{\sigma (j)}} \left( \frac{\partial ^{2\ell }}{\partial \phi _k^{2\ell }} -(-1)^\ell \frac{\partial ^{2\ell }}{\partial \psi _{\sigma (k)}^{2\ell }} \right) \frac{\sinh \phi _k}{\cosh \phi _k - \cos \psi _{\sigma (k)}}. \end{aligned}$$

Now

$$\begin{aligned} d_\ell \equiv \frac{\partial ^{2\ell }}{\partial \phi ^{2\ell }} - (-1)^\ell \frac{\partial ^{2\ell }}{\partial \psi ^{2\ell }} = d_1 \sum _{j=0}^{\ell -1} (-1)^{j}\frac{\partial ^{2\ell -2}}{\partial \phi ^{2\ell -2-2j}\partial \psi ^{2j}}, \end{aligned}$$

and the statement for \(\mathfrak {g} = \mathfrak {osp}(U \oplus U^*)\) is a consequence of the equation

$$\begin{aligned} \left( \frac{\partial ^2}{\partial \phi ^2}+\frac{\partial ^2}{\partial \psi ^2}\right) \frac{\sinh \phi }{\cosh \phi - \cos \psi } = 0 , \end{aligned}$$

which results from \((\cosh \phi - \cos \psi )^{-1} 2 \sinh \phi = \coth ( \frac{\phi + \mathrm {i}\psi }{2}) + \coth (\frac{\phi - \mathrm {i}\psi }{2})\) and the fact that every (anti-)holomorphic function on a domain in \(\mathbb {C}\) is harmonic.

The reasoning for the case of \(\mathfrak {g} = \mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)\) is no different. \(\square \)

Corollary 5.1

The restriction \(\chi _{T^+}\) of the character \(\chi \) from \(\widetilde{H}\) to \(T^+\) satisfies the system of differential equations \(D_\ell \, J \, \chi _{T^+} = 0\) for all \(\ell \in \mathbb {N}\,\).

Proof

Since \(D(I_\ell ) \chi = 0\) and hence by restriction \(\dot{D}(I_\ell ) \chi _{T^+} = 0\), it follows from Theorem 5.1 that \(J^{-1} (D_\ell + Q_{\ell -1}) J \chi _{T^+} = 0\) for every \(\ell \in \mathbb {N}\,\). Now by applying the operator \(\dot{D}(I_\ell )\) to the constant function 1 and using Lemma 5.1, we obtain \(0 = J \dot{D}(I_\ell ) 1 = D_\ell \, J + Q_{\ell -1}\, J = c_{\ell -1}\,J\), where \(c_{\ell -1}\) is the constant term of the differential operator \(Q_{\ell -1}\,\), and from this we conclude that \(c_{\ell - 1} = 0\) for all \(\ell \in \mathbb {N}\,\). It then follows by induction on \(\ell \) that \(D_\ell \, J \, \chi = 0\) for all \(\ell \in \mathbb {N}\,\). \(\square \)

The methods of this section can also be used to derive differential equations for the characters of a certain class of irreducible representations of \(\mathfrak {gl}(U) \simeq \mathfrak {g}^{(0)}\). Define

$$\begin{aligned} J_0 = \frac{\prod _{j < k} 4 \sinh \frac{\mathrm {i}(\psi _j - \psi _k)}{2} \, \sinh \frac{\phi _j - \phi _k}{2}}{\prod _{j,\,k} 2 \sinh \frac{\phi _j - \mathrm {i} \psi _k}{2}}. \end{aligned}$$

Here, \(\{ \mathrm {i}(\psi _j - \psi _k), \phi _j - \phi _k \mid j < k \}\) and \(\{ \phi _j - \mathrm {i} \psi _k \}\) are the sets of even and odd positive roots of \(\mathfrak {g}^{(0)}\). The following statement is Corollary 4.12 of [7] adapted to the present context and notation. The idea of the proof is the same as that of Proposition 5.3 in conjunction with Corollary 5.1.

Corollary 5.2

Let \(\gamma \) be the (restricted) character of an irreducible representation of the Lie supergroup \((\mathfrak {gl}(U),\mathrm {GL}(U_0) \times \mathrm {GL}(U_1))\) on a finite-dimensional \(\mathbb {Z}_2\) -graded vector space \(V = V_0 \oplus V_1\,\). If \(U_0 \simeq U_1\) but \(\mathrm {dim}(V_0) \not = \mathrm {dim}(V_1)\), then \(D_\ell J_0\,\gamma = 0\) for all \(\ell \in \mathbb {N}\,\).

5.3 Global \(G_\mathbb R\)-invariance and the Weyl group

Recall that \(\chi \) is only invariant by the local action of the supergroup \((G,\mathfrak {g})\) on \(\widetilde{H}\). However, there exists a real form \(G_\mathbb {R}\) which acts globally on \(\widetilde{H}\) by conjugation and therefore \(\chi \) is invariant by this action.

In order to identify these real symmetry groups \(G_\mathbb {R}\) in the two cases at hand, we first observe that the good real group acting in the spinor–oscillator representation \(\fancyscript {A}_V\) is

$$\begin{aligned} \mathrm {Spin}(W_{1,\mathbb {R}}) \times _{\mathbb {Z}_2} \mathrm {Mp}(W_{0,\mathbb {R}}) =: G^\prime, \end{aligned}$$

which contains \(K = \mathrm {O}_N\) and \(K = \mathrm {USp}_N\) as subgroups. Since we are studying the character \(\chi \) of the \(\mathfrak {g}\)-representation on the subspace \(\fancyscript {A}_V^K\) of K-invariants, we now seek the subgroup \(G_\mathbb {R} \subset G^\prime \) which centralizes K; this means that we are asking the exponentiated version of a question which was answered at the Lie algebra level in Sect. 2.7. Here, restricting the group \(G^\prime \) to the centralizer of K we find

$$\begin{aligned} G_\mathbb {R} = \left\{ \begin{array}{ll} \mathrm {Spin}( (U_1^{ }\oplus U_1^*)_\mathbb {R})\times _{\mathbb {Z}_2} \mathrm {Mp}( (U_0^{} \oplus U_0^*)_\mathbb {R}) &{}\quad K = \mathrm {O}_N, \\ \mathrm {USp}(U_1^{} \oplus U_1^*) \times \mathrm {SO}^*(U_0^{} \oplus U_0^*) &{}\quad K = \mathrm {USp}_N\;.\end{array}\right. \end{aligned}$$

We observe that \(G_\mathbb {R}\) for the case of \(K = \mathrm {O}_N\) is just the lower-dimensional copy of \(G^\prime \) which corresponds to \(U_s\) taking the role of \(V_s\,\). We also see immediately that the Lie algebras \(\mathrm {Lie}(G_\mathbb {R})\) coincide with the real forms described in Propositions 2.4 and 2.5.

Since \(\chi \) is invariant under the \(G_\mathbb {R}\)-action by conjugation, its restriction to a real toral semigroup in \(T^+\) is invariant under the action of the Weyl group W defined by \(G_\mathbb {R}\,\). Since \(\chi \) is holomorphic, its restriction to the complexification \(T^+\) is likewise W-invariant. Now \(G_\mathbb {R}\) decomposes as a direct product of two factors and so W also decomposes in this way. For both cases (\(K = \mathrm {O}_N\), \(\mathrm {USp}_N\)) the second factor of the Weyl group W is just the permutation group \(\mathrm {S}_n\,\). As a matter of fact, conjugation of a diagonal element \(t_0 \in M_\mathrm {Sp}\) or \(t_0 \in M_\mathrm {SO}\) by \(g \in \mathrm {Mp}((U_0^{} \oplus U_0^*)_\mathbb {R})\) or \(g \in \mathrm {SO}^*( U_0^{ } \oplus U_0^*)\) can return another diagonal element only by permutation of the eigenvalues \(\mathrm {e}^{\phi _1} , \ldots , \mathrm {e}^{\phi _n}\) of \(t_0\,\). (No inversion \(\mathrm {e}^{\phi _j} \rightarrow \mathrm {e}^{- \phi _j}\) is possible, as this would mean transgressing the oscillator semigroup.) This factor \(\mathrm {S}_n\) of W will play no important role in the following, as the expressions we will encounter are automatically invariant under such permutations.

The first factors of W are of greater significance. For the two cases of \(K = \mathrm {O}_N\) and \(K = \mathrm {USp}_N\) these are the Weyl groups \(W_{\mathrm {SO}_{2n}}\) and \(W_{\mathrm {Sp}_{2n}}\) respectively. An explicit description of these groups is as follows. Let \(\{ e_1 , \ldots , e_n \}\) be an orthonormal basis of U and decompose \(U \oplus U^*\) into a direct sum of 2-planes,

$$\begin{aligned} U \oplus U^* = P_1 \oplus \ldots \oplus P_n, \end{aligned}$$

where \(P_j\) is spanned by the vector \(e_j\) and the linear form \(c e_j = \langle e_j \, , \cdot \rangle \) (\(j = 1, \ldots ,n\)). In both cases at hand, i.e., for the symmetric form S as well the alternating form A, this is an orthogonal decomposition. The real torus under consideration is parameterized by \((\mathrm {e}^{\mathrm {i}\psi _1} , \ldots , \mathrm {e}^{\mathrm {i}\psi _n}) \in (\mathrm {U}_1)^n\) acting by \(\mathrm {e}^{\mathrm {i}\psi _j} . (e_j) = \mathrm {e}^{\mathrm {i} \psi _j} \, e_j\) and \(\mathrm {e}^{\mathrm {i} \psi _j}. (c e_j) = \mathrm {e}^{-\mathrm {i}\psi _j} ce_j\,\).

The Weyl group \(W_{\mathrm {Sp}}\) is generated by the permutations of these planes and the involutions which are defined by conjugation by the mapping that sends \(e_j \mapsto c e_j\) and \(c e_j \mapsto - e_j\,\). The Weyl group \(W_\mathrm {SO}\) is generated by the permutations together with the involutions which are induced by the mappings that simply exchange \(e_j\) with \(c e_j\,\). Since we are in the special orthogonal group and the determinant for a single exchange \(e_j \leftrightarrow c e_j\) is \(-1\), the number of involutions in any word in \(W_\mathrm {SO}\) has to be even.

In summary, the W-action on our standard bases of linear functions, \(\{ \mathrm {i}\psi _j\}\) and \(\{\phi _j \}\), is given by the respective permutations together with the action of the involutions defined by sign change, \(\mathrm {i} \psi _j \mapsto - \mathrm {i} \psi _j\,\). In the sequel, the Weyl group action will be understood to be either this standard action or alternatively, depending on the context, the corresponding action on the exponentiated functions \(\{ \mathrm {e}^{ \mathrm {i} \psi _j} \}\) and \(\{ \mathrm {e}^{\phi _j} \}\). As a final remark, let us note that the Weyl-group symmetries of the function \(\chi _{T^+}\) can also be read off directly from the explicit expression (4.2). In particular, the absence of reflections \(\phi _j \rightarrow -\phi _j\) is clear from the conditions \(\mathfrak {Re}\, \phi _j > 0\,\).

5.4 Formula for \(\chi _{T^+}\)

Recall that the main goal of this paper is to compute the restriction \(\chi _{T^+}\) to \(T^+\) of the character \(\chi \) which is defined on \(\widetilde{H}\) and plays the role of a character of the \(\widetilde{H}\)-representation on the space of invariants \({\fancyscript{A}}_V^K\) in the spinor–oscillator module. Here \(\widetilde{H}\) is the 2 : 1 cover of an open semigroup in the complex Lie group G of the Howe partner supergroup of K. From now on we will only deal with the restriction of this numerical part and therefore we simplify notation by denoting it by \(\chi _{T^+} \equiv \chi \).

We have restricted ourselves to the cases where K is either \(\mathrm {O}_N\) or \(\mathrm {USp}_N\,\). The representation on \(\mathfrak {a}(V)^K\) is defined at the infinitesimal level on the full complex Lie superalgebra \(\mathfrak {g}\) which is the Howe partner of K in the canonical realization of \(\mathfrak {osp}\) in the Clifford–Weyl algebra of \(V \oplus V^*\). It has been shown that \(\chi : \, T^+ \rightarrow \mathbb {C}\) satisfies the differential equations \(D_\ell \, J\, \chi = 0\,\). We now recall that the non-zero weights of the Fourier expansion of \(\chi \) are constrained to a certain region and show that \(\chi \) is the unique holomorphic function satisfying both the weight constraints and the differential equations.

5.4.1 Uniqueness

Recall that \(\Gamma _\lambda \) denotes the set of weights of the \(\mathfrak {g}\)-representation on \(\mathfrak {a}(V)^K\). From Corollary 2.3 we know that the weights \(\gamma = \sum _{j = 1}^n (\mathrm {i} m_j \psi _j - n_j \phi _j) \in \Gamma _\lambda \) satisfy the weight constraints \(- \frac{N}{2} \le m_j \le \frac{N}{2} \le n_j\,\). The highest weight is \(\lambda = \frac{N}{2} \sum (\mathrm {i} \psi _j - \phi _j)\). By the definition of the torus \(T^+\) the weights \(\gamma \in \Gamma _\lambda \) are analytically integrable and we now view \(\mathrm {e}^\gamma \) as a function on \(T^+\).

Theorem 5.2

The character \(\chi : \, T^+ \rightarrow \mathbb {C}\) is annihilated by all differential operators \(D_\ell \circ J\) for \(\ell \in \mathbb {N}\,\), and it has a convergent expansion \(\chi = \sum B_\gamma \, \mathrm {e}^{\gamma }\) where the sum runs over weights \(\gamma = \sum _{j = 1}^n (\mathrm {i} m_j \psi _j - n_j \phi _j)\) satisfying the constraints \(- \frac{N}{2} \le m_j \le \frac{N}{2} \le n_j\,\). For the case of \(K = \mathrm {USp}_N\) it is the unique W-invariant function on \(T^+\) with these two properties and \(B_\lambda = 1\). For \(K = \mathrm {O}_N\) it is the unique W-invariant function on \(T^+\) with these two properties and \(B_\lambda = 1\,\), \(B_{\lambda - \mathrm {i}N\psi _n} = 0\,\).

Remark 5.4

To verify the property \(B_{\lambda - \mathrm {i}N\psi _n} = 0\) which holds for the case of \(K = \mathrm {O}_N\,\), look at the right-hand side of the formula of Corollary 4.1: in order to generate a term \(\mathrm {e}^\gamma = \mathrm {e}^{\lambda - \mathrm {i}N\psi _n}\) in the weight expansion, you must pick the term \(\mathrm {e}^{- \mathrm {i} N\psi _n}\) in the expansion of the determinant for \(j = n\) in the numerator; but the latter term depends on k as \(\mathrm {Det}(-k)\) which vanishes upon taking the Haar average for \(K = \mathrm {O}_N \,\). By W-invariance the property \(B_{\lambda - \mathrm {i}N\psi _n} = 0\) is equivalent to \(B_{\lambda - \mathrm {i} N\psi _j} = 0\) for all \(j\,\).

In view of this Remark and Corollaries 2.3 and 5.1, it is only the uniqueness statement of Theorem 5.2 that remains to be proved here. This requires a bit of preparation, in particular to appropriately formulate the condition \(D_\ell \, J \chi = 0\,\). For that we develop \(J \chi \) in a series \(J \chi = \sum _\tau a_\tau f_\tau \) where the \(f_\tau \) are \(D_\ell \)-eigenfunctions for every \(\ell \in \mathbb {N}\,\).

The first step is to determine an appropriate expansion for J. Recall that

$$\begin{aligned} J = \frac{\prod _{\alpha \in \Delta ^+_0} (\mathrm {e}^{ \frac{\alpha }{2}}- \mathrm {e}^{- \frac{\alpha }{2}})}{\prod _{\beta \in \Delta ^+_1} (\mathrm {e}^{\frac{\beta }{2}} - \mathrm {e}^{-\frac{\beta }{2}})}. \end{aligned}$$

Given a factor in the denominator of this representation, we wish to factor out, e.g., \(\mathrm {e}^{-\frac{ \beta }{2}}\) to obtain a term \((1 - \mathrm {e}^{-\beta })^{-1}\) which we will attempt to develop in a geometric series. In order for this to converge uniformly on compact subsets of \(T^+\) it is necessary and sufficient for \(\mathfrak {Re}\, \beta \) to be positive on \(\mathfrak {t} = \mathrm {Lie}(T^+)\). This of course depends on the root \(\beta \,\). Fortunately, the sets of odd positive roots for our two cases of \(K= \mathrm {O}_N\) and \(K= \mathrm {USp}_N\) are the same (see Sect. 2.6.2):

$$\begin{aligned} \Delta _1^+ =\{\phi _j\pm \mathrm {i}\psi _k \mid j\,,k=1,\ldots ,n\}\;. \end{aligned}$$

So indeed, if we factor out \(\mathrm {e}^{-\frac{\beta }{2}}\) from each term in the denominator and do the same in the numerator, we obtain the expression

$$\begin{aligned} J = \mathrm {e}^{\delta }\, \frac{\prod _{\alpha \in \Delta ^+_0} (1 - \mathrm {e}^{-\alpha })}{\prod _{\beta \in \Delta ^+_1} {(1 - \mathrm {e}^{-\beta })}}, \end{aligned}$$

and it is possible to expand each term of the denominator in a geometric series. Here

$$\begin{aligned} \delta = \frac{1}{2}\sum _{\alpha \in \Delta ^+_0}\alpha - \frac{1}{2} \sum _{\beta \in \Delta ^+_1} \beta \end{aligned}$$

is half the graded sum of the positive roots.

Now let \(\{\sigma _1, \ldots , \sigma _r\}\) be a basis of simple positive roots (cf. Sect. 2.6.2) and expand the terms \((1- \mathrm {e}^{-\beta })^{-1}\) in geometric series to obtain

$$\begin{aligned} J = \mathrm {e}^{\delta } \, \sum _{b\ge 0} A_b \, \mathrm {e}^{b\sigma }, \end{aligned}$$

which converges uniformly on compact subsets of \(T^+\). In this expression b and \(\sigma \) denote the vectors \(b = (b_1, \ldots , b_r)\) and \(\sigma = (\sigma _1 , \ldots , \sigma _r)\), respectively, and \(b \sigma := \sum b_i \sigma _i\,\). Following the usual multi-index notation, \(b\ge 0\) means \(b_i \ge 0\) for all \(i\,\). Note \(A_0 = 1\).

Now we know that the character has a convergent series representation

$$\begin{aligned} \chi = \sum _{\gamma \in \Gamma _\lambda } B_\gamma \, \mathrm {e}^{\gamma }. \end{aligned}$$

Thus we may write

$$\begin{aligned} J\chi = \sum _{\gamma \in \Gamma _\lambda } B_\gamma \sum _{b\ge 0} A_b \,\mathrm {e}^{\delta + \gamma + b\sigma }\;. \end{aligned}$$

For convenience of the discussion we let \(\tilde{\gamma } := \gamma + b \sigma \) and reorganize the sums as

$$\begin{aligned} J\chi =\sum _{\tilde{\gamma }} \Big ( \sum A_b\, B_{\tilde{\gamma } -b \sigma } \Big )\, \mathrm {e}^{\delta + \tilde{\gamma }}, \end{aligned}$$
(5.3)

where the inner sum is a finite sum which runs over all \(b\ge 0\) such that \(\tilde{\gamma }- b\sigma \in \Gamma _\lambda \,\).

We are now in a position to explain the recursion procedure which shows that \(\chi \) is unique. Start by applying \(D_\ell \) to \(J\chi \) as represented in the expression (5.3). Since \(\delta +\tilde{\gamma }\) is of the form \(\sum (\mathrm {i} m_k \psi _k - n_k \phi _k)\), we immediately see that it is an eigenfunction with eigenvalue \(E(\ell ,\tilde{\gamma }) := (-1)^\ell \sum (m_k^{2\ell } - n_k^{2\ell })\). The functions \(\mathrm {e}^{\delta +\tilde{\gamma }}\) in the sum are independent eigenfunctions. Hence it follows that

$$\begin{aligned} 0 = E(\ell ,\tilde{\gamma }) \sum A_b \, B_{\tilde{\gamma }- b\sigma } \end{aligned}$$
(5.4)

for all \(\tilde{\gamma }\) fixed and then for all \(\ell \in \mathbb {N}\,\).

From now on we consider the Eq. (5.4) only in those cases where \(\tilde{\gamma }\) is itself a weight of our representation. (We have license to do so as only the uniqueness part of Theorem 5.2 is at stake here). In this case we have the following key fact.

Lemma 5.2

If \(\gamma \in \Gamma _\lambda \) and the eigenvalue \(E(\ell , \gamma )\) vanishes for all \(\ell \in \mathbb {N}\,\), then \(\gamma \) is the highest weight \(\lambda \).

Proof

Our first job is to compute \(\delta \). For the list of even and odd positive roots we refer the reader to Sect. 2.6.2. Direct computation shows that if \(K = \mathrm {O}_N\,\), then

$$\begin{aligned} \delta = \sum _{k=1}^n (k-1) (\mathrm {i} \psi _{n-k+1} - \phi _k)\;. \end{aligned}$$

The same computation for the case of \(K = \mathrm {USp}_N\) shows that

$$\begin{aligned} \delta = \sum _{k=1}^n k\, (\mathrm {i} \psi _k - \phi _{n-k+1}). \end{aligned}$$

Now we write \(\gamma = \sum _k (\mathrm {i} m_k \psi _k - n_k \phi _k)\) with the weight constraints \(-\frac{N}{2} \le m_k \le \frac{N}{2} \le n_k\,\). The assumption that \(E(\ell , \gamma )\) vanishes for all \(\ell \) means that

$$\begin{aligned} \sum _k (m_{n-k+1} + k - 1)^{2\ell } = \sum _k (n_k + k - 1)^{2\ell } \quad \text {for all}\ \ell \end{aligned}$$

in the case of \(K = \mathrm {O}_N\,\). In the case of \(K = \mathrm {USp}_N\) it means that

$$\begin{aligned} \sum _k (m_{n-k+1} + k)^{2\ell } = \sum _k (n_k + k)^{2\ell } \quad \text {for all}\ \ell. \end{aligned}$$

In the second case the only solution for \(m_k\) and \(n_k\) satisfying the weight constraints is the highest weight \(\lambda \) itself. In the first case there is one other solution, namely that which is obtained from the highest weight by replacing \(m_n = \frac{N}{2}\) by \(m_n = - \frac{N}{2}\,\). However, one directly checks that in the \(\mathrm {O}_N\) case, where \(2\mathrm {i}\psi _n\) is not a root, it is not possible to obtain such a \(\gamma \) by adding some combination of roots from \(\mathfrak {g}^{(2)}\) to \(\lambda \,\). \(\square \)

We are now able to give the proof of the uniqueness statement of Theorem 5.2.

Proof

We will determine \(B_\gamma \) recursively, starting from \(B_\lambda = 1\). Let \(\gamma \not = \lambda \) be a weight that satisfies the weight constraints. Then if \(K = \mathrm {USp}_N\) we know that \(E(\ell , \gamma )\) is non-zero for some \(\ell \,\). It therefore follows from Eq. (5.4) and \(A_0 = 1\) that

$$\begin{aligned} 0 = B_\gamma + \sum A_b \, B_{\gamma - b\sigma }, \end{aligned}$$
(5.5)

where the sum runs over all \(b \ne 0\) (recall that \(b \ge 0\) is always the case) such that \(\gamma - b\sigma \in \Gamma _\lambda \,\). Since the weights \(\gamma - b\sigma \) involved in the sum are smaller than \(\gamma \) in the natural partial order defined by the basis of simple roots, Eq. (5.5) defines a recursion procedure for determining all coefficients \(B_\gamma \,\).

In the case of \(K = \mathrm {O}_N\) we are confronted with the fact that the weight \(\gamma = \lambda - \mathrm {i} N \psi _n\) satisfies the weight constraints and yet gives \(E(\ell ,\gamma ) = 0\) for all \(\ell \). However, in this exceptional situation the conditions of Theorem 5.2 provide that \(B_{\lambda - \mathrm {i}N\psi _n} = 0\). Thus the expansion coefficients \(B_\gamma \) are still uniquely determined by our recursion procedure. \(\square \)

5.4.2 Explicit solution of the differential equations

As before let \(\Delta ^+\) be a set of positive roots of \(\mathfrak {g} = \mathfrak {osp} (U\oplus U^*)\) or \(\mathfrak {osp}(\widetilde{U} \oplus \widetilde{U}^*)\). We now decompose these sets as

$$\begin{aligned} \Delta ^+ = \Delta _\lambda ^+ \cup (\Delta ^+ {\setminus} \Delta _\lambda ^+)\;,\quad \Delta _\lambda ^+ := \{\alpha \in \Delta ^+ \mid \mathfrak {g}_\alpha \subset \mathfrak {g}^{(-2)}\}, \end{aligned}$$

which means that \(\Delta ^+ {\setminus} \Delta _\lambda ^+\) is a set of positive roots of \(\mathfrak {g}^{(0)}\). Let \(\Delta _\lambda ^+\) be further decomposed as \(\Delta _\lambda ^+ = \Delta _{\lambda ,0}^+ \cup \Delta _{\lambda ,1}^+\) where \(\Delta _{\lambda ,0}^+\) and \(\Delta _{\lambda ,1}^+\) are the subsets of even and odd \(\lambda \)-positive roots. Then the function J has a factorization as \(J = J_0\, Z^{-1} \mathrm {e}^{\delta ^\prime }\) with

$$\begin{aligned} J_0 = \frac{\prod _{\alpha \in \Delta _0^+ {\setminus} \Delta _{\lambda ,0}^+} 2 \sinh \frac{\alpha }{2}}{\prod _{\beta \in \Delta _1^+ {\setminus} \Delta _{\lambda ,1}^+} 2 \sinh \frac{\beta }{2}}\ ,\quad Z = \frac{\prod _{\beta \in \Delta _{\lambda ,1}^+} (1 - \mathrm {e}^{-\beta })}{\prod _{ \alpha \in \Delta _{\lambda ,0}^+} (1-\mathrm {e}^{-\alpha })}\;, \end{aligned}$$

and \(\delta ^\prime = \frac{1}{2} ( \sum \alpha - \sum \beta )\) is half the graded sum of \(\lambda \)-positive roots. For the case of \(K = \mathrm {O}_N\) one finds \(\delta ^\prime = - \frac{1}{2} \sum (\mathrm {i}\psi _j - \phi _j) = - \lambda _{N=1}\,\), while for \(K = \mathrm {USp}_N\) one has \(\delta ^\prime = \lambda _1\,\).

The Weyl group W acts on \(T^+\) and therefore on functions on \(T^+\). Let \(W_\lambda \subset W\) be the subgroup which stabilizes the highest weight \(\lambda = \lambda _N\) and thus the corresponding function \(\mathrm {e}^\lambda \) on \(T^+\). Note that \(W_\lambda \) is the direct product of the permutations of the set \(\{ \mathrm {e}^{\phi _j} \}\) and the permutations of the set \(\{ \mathrm {e}^{\mathrm {i}\psi _j} \}\). The symmetrizing operator \(S_W\) from \(W_\lambda \)-invariant analytic functions to W-invariant analytic functions on \(T^+\) is given by

$$\begin{aligned} S_W (f) := \sum _{[w] \in W/W_\lambda } w(f). \end{aligned}$$

Notice that the function \(\mathrm {e}^\lambda Z\) is \(W_\lambda \)-invariant; the symmetrized function \(S_W( \mathrm {e}^\lambda Z)\) then is W-invariant. We now wish to show that this function coincides with our character \(\chi \). In this endeavor, an obstacle appears to be that \(\mathrm {e}^\lambda \chi \) by Corollary 4.1 is a polynomial in the variables \(\mathrm {e}^{\mathrm {i}\psi _1}, \ldots , \mathrm {e}^{ \mathrm {i} \psi _n }\), whereas the function Z has poles at \(\mathrm {e}^{ \mathrm {i}(\psi _j + \psi _k)} = 1\). Hence our next step is to show that these poles are actually canceled by the process of W-symmetrization.

Lemma 5.3

The function \(S_W (\mathrm {e}^{\lambda } Z)\) is holomorphic on \(\cap _{j=1}^n \{ \mathfrak {Re}\, \phi _j > 0 \}\).

Proof

An even root \(\alpha \in \Delta _0^+\) is some linear combination of either the functions \(\phi _j\) or the functions \(\psi _j\,\). Denoting the latter subset of even roots by \(\Delta _0^+(\psi ) \subset \Delta _0^+ \,\), let \(\Sigma _\alpha \subset T\) for \(\alpha \in \Delta _0^+(\psi )\) be the complex submanifold

$$\begin{aligned} \Sigma _\alpha := \{ t \in T^+ \mid \mathrm {e}^{\alpha }(t) = 1 \}. \end{aligned}$$

By definition, the function \(S_W (\mathrm {e}^\lambda Z)\) is holomorphic on

$$\begin{aligned} \Big ( \cap _{j=1}^n\{\mathfrak {Re}\, \phi _j > 0 \} \Big ) {\setminus} \left( \cup _{\alpha \in \Delta _0^+(\psi )} \Sigma _\alpha \right). \end{aligned}$$

Now it is a theorem of complex analysis that if a function is holomorphic outside an analytic set of complex codimension at least two, then this function is everywhere holomorphic. Therefore, since the intersection of two or more of the submanifolds \(\Sigma _\alpha \) is of codimension at least two in \(T^+\), it suffices to show that for any \(\alpha \in \Delta _0^+(\psi )\) our function \(S_W (\mathrm {e} ^\lambda Z)\) extends holomorphically to

$$\begin{aligned} D_\alpha := \Sigma _\alpha {\setminus} \left( \cup _{\Delta _0^+(\psi ) \ni \alpha ^\prime \not =\alpha }\,\Sigma _{\alpha ^\prime }\right). \end{aligned}$$

Hence let \(\alpha \) be some fixed root in \(\Delta _0^+(\psi )\). There exists a Weyl-group element \(w \in W\) and a w-invariant neighborhood U of \(D_\alpha \) such that \(w : \, U \rightarrow U\) is a reflection fixing the points of \(D_\alpha \,\). Let \(z_\alpha : \, U \rightarrow \mathbb {C}\) be a complex coordinate which is transverse to \(D_\alpha \) in the sense that \(w(z_\alpha ) = - z_\alpha \,\). Because the root \(\alpha \) occurs at most once in the product Z, the function \(S_W(\mathrm {e}^\lambda Z)\) has at most a simple pole in \(z_\alpha \,\). We may choose U in such a way that \(S_W(\mathrm {e} ^\lambda Z)\) is holomorphic on \(U {\setminus} D_\alpha \,\). Doing so we have a unique decomposition

$$\begin{aligned} S_W (\mathrm {e}^\lambda Z) = \frac{A}{z_\alpha } + B \end{aligned}$$

where A and B are holomorphic on U. Since \(S_W(\mathrm {e} ^\lambda Z)\) is W-invariant, we conclude that \(w(A) = - A\) and hence \(A = 0\) along \(D_\alpha \,\). \(\square \)

Lemma 5.4

For all \(\ell \, , N \in \mathbb {N}\) the function \(\varphi :\, T^+ \rightarrow \mathbb {C}\) defined by \(\varphi = S_W(\mathrm {e}^{\lambda _N} Z)\) is a solution of the differential equation \(D_\ell \, J \varphi = 0\,\).

Proof

Using \(Z = \mathrm {e}^{\delta ^\prime } J_0 / J\) we write \(\varphi = S_W(\mathrm {e}^{\lambda _N + \delta ^\prime } J_0/J)\). Then, lifting the sum over cosets \([w] \in W/W_\lambda \) to a sum over Weyl-group elements \(w \in W\) we obtain

$$\begin{aligned} \mathrm {ord}(W_\lambda ) \, J \varphi = J \sum _{w \in W} w(\mathrm {e}^{\lambda _N + \delta ^\prime } J_0 / J) =\sum _{w \in W} \mathrm {sgn}(w)\, w(J_0\,\mathrm {e}^{\lambda _N+\delta ^\prime })\;, \end{aligned}$$

where \(w \mapsto \mathrm {sgn}(w) \in \mathbb {Z}_2 = \{ \pm 1 \}\) is the determinant of \(w \in W \subset \mathrm {O}(\mathfrak {t}) = \mathrm {O}_{2n}\,\).

The factor \(\mathrm {e}^{\lambda _N + \delta ^\prime }\) is the character of the representation \((\frac{N \mp 1}{2} \mathrm {STr}\, , \mathrm {SDet}^{\frac{N \mp 1}{2}})\) of (a double cover of) the Lie supergroup \((\mathfrak {g}^{(0)} , \mathrm {GL}(U_0) \times \mathrm {GL}(U_1))\). This representation is one-dimensional, and from Corollary 5.2 we have \(D_\ell (J_0\, \mathrm {e}^{ \lambda _N + \delta ^\prime }) = 0\) for all \(\ell , N \in \mathbb {N}\,\).

The statement of the lemma now follows by applying the W-invariant differential operator \(D_\ell \) to the formula for \(\mathrm {ord}(W_\lambda ) J \varphi \) above. \(\square \)

5.4.3 Weight constraints

Here we carry out the final step in proving the explicit formula for the character \(\chi \) of our representation. Since the formula in the case of \(K = \mathrm {SO}_N\) follows directly from that for \(K = \mathrm {O}_N\) (see Sect. 1) and the case of \(K = \mathrm {U}_N\) has been handled in [7], we need only discuss the cases of \(K = \mathrm {O}_N\) and \(K = \mathrm {USp}_N\,\).

In order to show that the character is indeed given by \(\chi = \varphi \) with \(\varphi = S_W(\mathrm {e}^\lambda Z)\), it remains to prove that in the series development \(\varphi = \sum B_{\gamma }\, \mathrm {e}^\gamma \) of the function defined by

$$\begin{aligned} \varphi = \sum _{[w] \in W/W_\lambda } \varphi _{[w]}, \quad \varphi _{[w]} := \mathrm {e}^{w(\lambda _N)} \frac{\prod _{\beta \in \Delta _{\lambda ,1}^+} (1 - \mathrm {e}^{ -w(\beta )})}{\prod _{\alpha \in \Delta _{\lambda ,0}^+} (1 - \mathrm {e}^{-w(\alpha )})}, \end{aligned}$$
(5.6)

the only non-zero coefficients \(B_\gamma \) are those where the linear functions \(\gamma \) are of the form \(\gamma = \sum (\mathrm {i} m_k \psi _k - n_k \phi _k)\) with \(- \frac{N}{2} \le m_k \le \frac{N}{2} \le n_k\, \). We also have to show that \(B_\gamma = 0\) in the case of the exceptional weight \(\gamma = \lambda - \mathrm {i}N\psi _n\) occurring for \(K = \mathrm {O}_N\,\).

We have shown above that \(\varphi = S_W(\mathrm {e}^\lambda Z)\) is holomorphic on the product of the full complex torus of the variables \(\mathrm {e}^{\mathrm {i} \psi _k}\) with the domain defined by \(\mathfrak {Re}\, \phi _k > \,0\,\). Although the individual terms \(\varphi _{ [w]}\) in the representation of \(\varphi \) have poles (which cancel in the Weyl-group averaging process) we may still develop each term of \(\varphi \) in a series expansion; this will in fact yield the desired weight constraints.

We begin with the situation where \(K = \mathrm {O}_N\,\). In this case \(\Delta ^+_{\lambda ,0}\) consists of the roots \(\mathrm {i} \psi _j + \mathrm {i} \psi _k\,\) (\(j<k\)) and \(\phi _j + \phi _k\) (\(j \le k)\), and \(\Delta ^+_{ \lambda ,1}\) is the set of roots of the form \(\mathrm {i} \psi _j + \phi _k\) (\(j,\,k = 1, \ldots , n\)). Let us first consider the term of \(\varphi \) where \([w] = W_\lambda \,\). Its denominator can be developed in a geometric series on the region corresponding to \(\mathfrak {Re}\, \phi _k > 0\) for all \(k\,\). There we may write this term as

$$\begin{aligned} \varphi _{[\mathrm {Id}]}=\mathrm {e}^{\lambda _N}\prod _{\beta } (1-\mathrm {e}^{-\beta }) \prod _\alpha \sum _{n \ge 0} \mathrm {e}^{-n\alpha }. \end{aligned}$$

Here and for the remainder of this paragraph \(\alpha \) runs through the \(\lambda \)-positive even roots and \(\beta \) through the \(\lambda \)-positive odd roots.

Recall that \(\lambda _N = \frac{N}{2}\sum (\mathrm {i} \psi _k - \phi _k )\), and note that all powers of \(\mathrm {e}^{\mathrm {i} \psi _k}\) and \(\mathrm {e}^{\phi _k}\) occurring in the series expansion of

$$\begin{aligned} \prod _\beta (1 - \mathrm {e}^{-\beta }) \prod _\alpha \sum _{n \ge 0} \mathrm {e}^{- n\alpha } \end{aligned}$$

are non-positive. Thus, if \(\gamma = \sum (\mathrm {i} m_k \psi _k - n_k \phi _k)\) is a weight which arises in \(\varphi _{[\mathrm {Id}]}\,\), then \(n_k \ge \frac{N}{2}\) and \(m_k \le \frac{N}{2}\). In the case of the \(m_k\) this is a statement only about the term \(\varphi _{ [\mathrm {Id}]}\,\), but, since the action of the Weyl group on the variables \(\phi _k\) is just by permutation of the indices, it follows that \(n_k \ge \frac{N}{2}\) holds always, independent of the term \(\varphi _{[w]}\) under consideration. Hence, we ignore the \(\phi _k\) in our further discussion and only analyze the powers of the exponentials \(\mathrm {e}^{\mathrm {i} \psi _k}\) which arise in the other terms \(\varphi _{[w]}\,\).

Given a fixed index \(k \in \{1, \ldots , n\}\) we will develop every term \(\varphi _{[w]}\) on a region \(R = R(k)\) defined by certain inequalities which in the case of \(k = 1\) are

$$\begin{aligned} \mathfrak {Re}(\mathfrak {i} \psi _1) > \cdots > \mathfrak {Re} (\mathrm {i} \psi _n) > 0. \end{aligned}$$

We now discuss this case in detail.

Recall that \(\mathrm {i} \psi _1\) occurs in the denominator in factors of the form

$$\begin{aligned} (1-\mathrm {e}^{-w(\alpha )})^{-1} = (1 - \mathrm {e}^{- w(\mathrm {i}\psi _1) - w(\mathrm {i} \psi _j)})^{-1} \end{aligned}$$

for \(j > 1\,\). If \(w(\mathrm {i}\psi _1) = \mathrm {i}\psi _1\,\), then we expand these factors just as in the case of \(\varphi _{ [\mathrm {Id}]} \,\). Convergence of the resulting series is guaranteed no matter what w does to \(\psi _j\,\).

In the situation where \(w(\mathrm {i} \psi _1) = - \mathrm {i} \psi _1\) we rewrite the factors in the denominator as \((1-\mathrm {e}^{-w (\alpha )})^{-1} = -\mathrm {e}^{w(\alpha )} (1 - \mathrm {e}^{w(\alpha ) })^{-1}\) and expand, and convergence in R is again guaranteed. Adding these series we obtain a series representation

$$\begin{aligned} \varphi = \sum _{[w]} \varphi _{[w]} = \sum _\gamma B_\gamma \, \mathrm {e}^\gamma, \end{aligned}$$

which is convergent on \(R\,\).

Lemma 5.5

If \(\gamma = \sum (\mathrm {i} m_k \psi _k - n_k \phi _k)\) and \(B_\gamma \not = 0\,\), then \(m_1 \le \frac{N}{2}\, \).

Proof

If \(w(\mathrm {i} \psi _1) = \mathrm {i} \psi _1\,\), then by the same argument as in the case of \([w] = [\mathrm {Id}]\) we see that \(\mathrm {e}^{ \mathrm {i} \psi _1}\) occurs in the series development of \(\varphi _{[w]}\) with a power \(m_1\) of at most \(\frac{N}{2}\,\).

Now suppose that \(w(\mathrm {i}\psi _1) = - \mathrm {i} \psi _1\,\). Then, following the prescription above we rewrite the \(\psi _1\)-dependent factors in \(\varphi _{[w]}\) as

$$\begin{aligned} \mathrm {e}^{\frac{N}{2} w(\mathrm {i}\psi _1)} \frac{\prod _{j \ge 1} (1 - \mathrm {e}^{-w(\mathrm {i}\psi _1 + \phi _j)})}{\prod _{j \ge 2} (1 - \mathrm {e}^{-w(\mathrm {i}\psi _1 + \mathrm {i}\psi _j)})} = \mathrm {e}^{(-\frac{N}{2}+1) \mathrm {i}\psi _1}\frac{\prod _{j\ge 1} (\mathrm {e}^{-\mathrm {i}\psi _1} - \mathrm {e}^{-\phi _j})}{\prod _{j \ge 2}(\mathrm {e}^{- \mathrm {i}\psi _1}- \mathrm {e}^{-w(\mathrm {i} \psi _j)})}, \end{aligned}$$
(5.7)

and expand the r.h.s. in powers of \(\mathrm {e}^{- \mathrm {i} \psi _1}\). It follows that in this case \(m_1 \le - \frac{N}{2} + 1\,\), which for any positive integer N implies that \(m_1 \le \frac{N}{2}\,\). \(\square \)

Using Weyl-group invariance, this estimate for \(m_1\) will now yield the desired result.

Lemma 5.6

Suppose that \(K = \mathrm {O}_N\) and let \(\varphi = \sum B_\gamma \, \mathrm {e}^\gamma \) be the globally convergent series expansion of the proposed character \(\varphi = S_W(\mathrm {e}^\lambda Z)\). Then for every weight \(\gamma = \sum (\mathrm {i} m_k \psi _k - n_k \phi _k)\) with \(B_\gamma \not = 0\) it follows that \(- \frac{N}{2} \le m_k \le \frac{N}{2} \le n_k\,\).

Proof

The inequality \(n_k \ge \frac{N}{2}\) was proved above as an immediate consequence of the fact that the Weyl group W effectively acts only on the \(\psi _j\,\).

Above we showed that on the region R the proposed character \(\varphi \) has a series development where in every \(\gamma \) the coefficient \(m_1\) of \(\mathrm {i} \psi _1\) is at most \(\frac{N}{2}\,\). Recalling the fact that the function \(\varphi \) is holomorphic on \(T^+\), we infer that \(m_1 \le \frac{N}{2}\) also holds true for the globally convergent series development \(\sum B_\gamma \, \mathrm {e}^\gamma \).

To get the same statement for \(\mathrm {i} \psi _k\) with \(k \not = 1\) we just change the definition of R to R(k) defined by the inequalities \(\mathfrak {Re} (\mathrm {i} \psi _k) > \mathfrak {Re} (\mathrm {i} \psi _1) > \ldots > 0 \, \). Arguing for general k as we did for \(k = 1\) in the above lemma, we show that the coefficient \(m_k\) of \(\mathrm {i} \psi _k\) in every \(\gamma \) in the series expansion of every \(\varphi _{[w]}\) on R(k) is at most \(\frac{N}{2}\,\). By the holomorphic property, the same is true for the global series expansion of the proposed character \(\varphi \).

Hence, to complete the proof we need only show the inequality \(m_k \ge -\frac{N}{2}\,\). But for this it suffices to note that for every k there is an element w of the Weyl group with \(w(\mathrm {i} \psi _k) = - \mathrm {i}\psi _k\,\). Indeed, using the Weyl invariance of \(\varphi \), if there was some \(\gamma \) where \(m_k < - \frac{N}{2}\,\), then the coefficient of \(\mathrm {i} \psi _k\) in \(w(\gamma )\) would be larger than \(\frac{N}{2}\,\). \(\square \)

To complete our work, we must prove Lemma 5.6 for the case \(K = \mathrm {USp}_N\,\). For this we use the same notation as above for the basic linear functions, namely \(\mathrm {i} \psi _k\) and \(\phi _k\,\). Here, compared to the \(\mathrm {O}_N\) case, there are only slight differences in the \(\lambda \)-positive roots and the Weyl group. The only difference in the roots is in \(\Delta ^+_{\lambda ,0}\) where \(\mathrm {i} \psi _j + \mathrm {i} \psi _k\) occurs in the larger range \(j \le k\) and \(\phi _j + \phi _k\) in the smaller range \(j < k\,\). The Weyl group acts by permutation of indices on both the \(\mathrm {i} \psi _j\) and \(\phi _j\) and by sign reversal on the \(\mathrm {i} \psi _j\, \). In this case, as opposed to the case above where only an even number of sign reversals were allowed, every sign reversal transformation is in the Weyl group.

In order to prove Lemma 5.6 in this case, we need only go through the argument in the \(\mathrm {O}_N\) case and make minor adjustments. In fact, the main step is to prove Lemma 5.5 and, there, the only change is that the range of j for the factor \(1 - \mathrm {e}^{-\mathrm {i}(\psi _1 + \psi _j)}\) is larger. This is only relevant in the case \(w(\mathrm {i} \psi _1) = - \mathrm {i} \psi _1\,\), where we rewrite the additional denominator term \((1 - \mathrm {e}^{-w(2\mathrm {i} \psi _1)})^{-1}\) as \(- \mathrm {e}^{-2\mathrm {i} \psi _1}(1 - \mathrm {e}^{- 2\mathrm {i} \psi _1})^{-1}\). Hence the factor in front of the ratio of products on the r.h.s. of equation (5.7) gets an additional factor of \(\mathrm {e}^{-2\mathrm {i}\psi _1}\) and now is \(\mathrm {e}^{-\mathrm {i} (\frac{N}{2} + 1)\psi _1}\). Thus \(m_1 \le -\frac{N}{2} -1\) which certainly implies \(m_1 \le \frac{N}{2}\,\).

Let us summarize this discussion.

Theorem 5.3

For both \(K = \mathrm {O}_N\) and \(K = \mathrm {USp}_N\) every weight \(\gamma = \sum (\mathrm {i} m_k \psi _k - n_k \phi _k)\) occurring in the series expansion \(S_W(\mathrm {e}^\lambda Z) = \sum B_\gamma \, \mathrm {e}^\gamma \) obeys the weight constraints

$$\begin{aligned} - \frac{N}{2} \le m_k \le \frac{N}{2} \le n_k \quad (k = 1, \ldots , n)\;. \end{aligned}$$

Moreover, using the fact that the Weyl-group transformations for \(K = \mathrm {O}_N\) always involve an even number of sign changes, one sees that \(B_{\lambda - \mathrm {i}N\psi _n} = 0\) in that case. As a consequence of the uniqueness theorem (Theorem 5.2) we therefore have

$$\begin{aligned} \chi = S_W(\mathrm {e}^\lambda Z) = \sum _{[w] \in W/W_\lambda } \mathrm {e}^{w(\lambda _N)}\frac{\prod _{\beta \in \Delta _{\lambda ,1}^+} (1 - \mathrm {e}^{-w(\beta )})}{\prod _{\alpha \in \Delta _{\lambda ,0}^+} (1 - \mathrm {e}^{-w(\alpha )})} \end{aligned}$$

in both the \(\mathrm {O}_N\) and \(\mathrm {USp}_N\) cases. Since the \(\mathrm {SO}_N\) case has been handled as a consequence of the result for \(\mathrm {O}_N\,\), our work is now complete.