Super-Macdonald Polynomials: Orthogonality and Hilbert Space Interpretation

Abstract

The super-Macdonald polynomials, introduced by Sergeev and Veselov (Commun Math Phys 288: 653–675, 2009), generalise the Macdonald polynomials to (arbitrary numbers of) two kinds of variables, and they are eigenfunctions of the deformed Macdonald–Ruijsenaars operators introduced by the same authors in Sergeev and Veselov (Commun Math Phys 245: 249–278, 2004). We introduce a Hermitian form on the algebra spanned by the super-Macdonald polynomials, prove their orthogonality, compute their (quadratic) norms explicitly, and establish a corresponding Hilbert space interpretation of the super-Macdonald polynomials and deformed Macdonald–Ruijsenaars operators. This allows for a quantum mechanical interpretation of the models defined by the deformed Macdonald–Ruijsenaars operators. Motivated by recent results in the nonrelativistic ($q\rightarrow 1$) case, we propose that these models describe the particles and anti-particles of an underlying relativistic quantum field theory, thus providing a natural generalisation of the trigonometric Ruijsenaars model.

1 Introduction

As is well-known, the Macdonald polynomials [Mac95] can be viewed as eigenfunctions of a commuting family of difference operators associated with a relativistic generalisation of the integrable quantum Calogero-Moser-Sutherland systems of trigonometric A-type [Rui87]. Such relativistic quantum systems were originally conceived by Ruijsenaars as an integrable quantum mechanical description of a relativistic quantum field theory in 1+1 spacetime dimensions known as the quantum sine-Gordon theory, restricted to sectors where the particle number is fixed [RS86, Rui01].

While the standard Ruijsenaars systems account for one particle type, a relativistic quantum field theory typically has two kinds of particle: particles and anti-particles. This strongly suggests to us that Ruijsenaars’ systems should have generalisations allowing for two particle types, and we propose that, in the trigonometric regime, such a generalisation is given by the so-called deformed Macdonald–Ruijsenaars operators ${\mathcal M}_{n,m;q,t}$ and ${\mathcal M}_{n,m;q^{-1},t^{-1}}$ (specified in (1) below), and their joint eigenfunctions, the super-Macdonald polynomials, introduced and studied by Sergeev and Veselov [SV04, SV09a]. The super-Macdonald polynomials $SP_\lambda ((x_1,\ldots ,x_n),(y_1,\ldots ,y_m);q,t)$ depend on arbitrary numbers, n and m, of two types of variables, $x_i$ and $y_j$, and we expect that these two variable types correspond to particles and anti-particles in an underlying quantum field theory.

In any quantum mechanical model, there is a scalar product providing the space of wave functions with a Hilbert space structure, and this structure is essential for the physical interpretation of the model. For the trigonometric Ruijsenaars systems and the Macdonald polynomials such a Hilbert space structure is provided by the scalar product denoted as $\langle \cdot ,\cdot \rangle ^\prime _n$ in Macdonald’s book [Mac95]. In particular, with respect to this scalar product, the commuting family of difference operators alluded to above, which include operators that define the Hamiltonian and momentum operator in the model, are self-adjoint and the Macdonald polynomials form an orthogonal system with explicitly known Hilbert space norms [Mac95]. By contrast, for the deformed Macdonald–Ruijsenaars operators and the super-Macdonald polynomials, such a Hilbert space structure has been missing. Our main purpose with this paper is to provide this missing Hilbert space structure and thereby substantiate our proposal, as formulated above. Moreover, recent quantum field theory results in the nonrelativistic case [AL17, BLL20], discussed in Sect. 5, provide further support in favour of our proposal.

To describe our results in more detail, we recall from [SV09a] that the super-Macdonald polynomials are joint eigenfunctions of a large commutative algebra of difference-operators, containing the so-called deformed Macdonald–Ruijsenaars operator introduced in [SV04]:¹

$$\begin{aligned} {\mathcal M}_{n,m;q,t} = \frac{t^{1-n}}{1-q} \sum _{i=1}^n A_i\big (T_{q,x_i}-1\big ) + \frac{q^{m-1}}{1-t^{-1}} \sum _{j=1}^m B_j\big (T_{t^{-1},y_j}-1\big ) \end{aligned}$$

(1)

with coefficients

$$\begin{aligned} \begin{aligned} A_i&= \prod _{i^\prime \ne i}^n \frac{tx_i - x_{i^\prime }}{x_i- x_{i^\prime }}\cdot \prod _{j=1}^m \frac{t^{1/2} x_i - q^{1/2} y_j}{t^{1/2} x_i - q^{-1/2}y_j},\\ B_j&= \prod _{j^\prime \ne j}^m \frac{q^{-1}y_j - y_{j^\prime }}{y_j - y_{j^\prime }}\cdot \prod _{i=1}^n \frac{q^{-1/2} y_j - t^{-1/2} x_i}{q^{-1/2} y_j - t^{1/2} x_i}, \end{aligned} \end{aligned}$$

(2)

and where $T_{q,x_i}$ and $T_{t^{-1},y_j}$ act on functions f(x, y) of $x=(x_1,\ldots ,x_n)\in \mathbb {C}^n$ and $y=(y_1,\ldots ,y_m)\in \mathbb {C}^m$ by shifting $x_i\rightarrow qx_i$ and $y_j\rightarrow t^{-1}y_j$, respectively, while leaving the remaining variables unaffected.

For our Hilbert space results, it will be important to restrict attention to parameter values

$$\begin{aligned} 0<q<1,\ \ \ 0<t<1. \end{aligned}$$

However, as discussed briefly in the final paragraph of Sect. 5, some of our results extend analytically to complex q and t (with modulus in (0, 1)).

Deformed Macdonald–Ruijsenaars operators first appeared in the $m=1$ case in work by Chalykh [Cha97, Cha00]. Further examples, including deformed Koornwinder operators, were later obtained and studied by Feigin [Fei05], Sergeev and Veselov [SV09b] and Feigin and Silantyev [FS14].

Taking $m=0$, the operator given by (1)–(2) reduces to

$$\begin{aligned} {\mathcal M}_{n;q,t} = \frac{t}{1-q}\cdot t^{-n}\sum _{i=1}^n \prod _{i^\prime \ne i}^n \frac{tx_i - x_{i^\prime }}{x_i- x_{i^\prime }} \big (T_{q,x_i}-1\big ), \end{aligned}$$

(3)

and the super-Macdonald polynomials reduce to the ordinary (monic symmetric) Macdonald polynomials $P_\lambda ((x_1,\ldots ,x_n);q,t)$. Note that, up to the overall factor $t/(1-q)$, ${\mathcal M}_{n;q,t}$ (3) coincides with the operator $E_n$ in Madonald’s book [Mac95, Section VI.4]. Moreover, ${\mathcal M}_{n;q,t}$ is closely related to the trigonometric limit of the elliptic operator $\widehat{S}_1$ introduced by Ruijsenaars in [Rui87]. The precise relationship, which was first observed by Koornwinder (in unpublished notes), is, e.g., detailed in [vDie95, Section 5.2] and [Has97, Section 5.1]. We recall that $\widehat{S}_1\pm \widehat{S}_{-1}$, where $\widehat{S}_{-1}$ is similarly related to ${\mathcal M}_{n;q^{-1},t^{-1}}$, essentially amount to the Hamiltonian and momentum operator, respectively, in Ruijsenaars’ model. This state of affairs suggests to us that the super-Macdonald operators define a quantum mechanical model generalising the trigonometric Ruijsenaars model by allowing two kinds of particles; see Appendix B for a proof of the relativistic invariance of this generalized model. However, such an interpretation requires a compatible Hilbert space structure.

As is well known, the Macdonald polynomials form an orthogonal system on the n-dimensional torus ${\mathbb T}^n\equiv {\mathbb T}_1^n$, where

$$\begin{aligned} {\mathbb T}_\xi ^n = \big \{x=(x_1,\ldots ,x_n)\in {\mathbb C}^n\mid |x_i|=\xi \ \ (i=1,\ldots ,n)\big \}\ \ \ (\xi > 0), \end{aligned}$$

(4)

with respect to the weight function

$$\begin{aligned} \Delta _n(x;q,t) = \prod _{1\le i\ne j\le n}\frac{(x_i/x_j;q)_\infty }{(tx_i/x_j;q)_\infty }, \end{aligned}$$

(5)

where $(a;q)_\infty =\prod _{k=0}^\infty (1-aq^k)$ is the usual q-Pochhammer symbol. Moreover, the corresponding (quadratic) norms are given by remarkably simple and explicit formulas [Mac95, Section VI.9]; see (25)–(27).

These orthogonality results, together with the corresponding Hilbert space structure, entail a natural quantum mechanical interpretation of the Macdonald polynomials $P_\lambda ((x_1,\ldots ,x_n);q,t)$ and the commuting Macdonald–Ruijsenaars operators ${\mathcal M}_{n;q,t}$ and ${\mathcal M}_{n;q^{-1},t^{-1}}$; this is the trigonometric Ruijsenaars model.

In this paper, we obtain analogous results for the the super-Macdonald polynomials $SP_\lambda (x,y;q,t)$. More specifically, we establish orthogonality relations with respect to a sesquilinear form given by

$$\begin{aligned} \begin{aligned} \langle P,Q\rangle ^\prime _{n,m;q,t} = \frac{1}{n!m!}\int _{{\mathbb T}_\xi ^n} \frac{dx_1}{2\pi \mathrm {i}x_1}\cdots \frac{dx_n}{2\pi \mathrm {i}x_n} \int _{{\mathbb T}_{\xi ^\prime }^m} \frac{dy_1}{2\pi \mathrm {i}y_1}\cdots \frac{dy_m}{2\pi \mathrm {i}y_m} \\ \times \Delta _{n,m}(x,y;q,t) P(x,y)\overline{Q(\bar{x}^{-1},\bar{y}^{-1})}, \end{aligned} \end{aligned}$$

(6)

with weight function

$$\begin{aligned} \Delta _{n,m}(x,y;q,t) = \frac{\Delta _n(x;q,t)\Delta _m(y;t,q)}{\prod _{i=1}^n\prod _{j=1}^m(1-q^{-1/2}t^{1/2}x_i/y_j)(1-q^{-1/2}t^{1/2}y_j/x_i)}, \end{aligned}$$

(7)

and where P, Q are polynomials in the space spanned by the super-Macdonald polynomials, the bar denotes complex conjugation and

$$\begin{aligned} \bar{x}^{-1}:=(1/\bar{x}_1,\ldots ,1/\bar{x}_n),\ \ \ \bar{y}^{-1}:=(1/\bar{y}_1,\ldots ,1/\bar{y}_m). \end{aligned}$$

(8)

Furthermore, in order to ensure that we avoid the poles of the weight function, we integrate x and y over tori ${\mathbb T}_\xi ^n$ and ${\mathbb T}_{\xi ^\prime }^m$ with radii $\xi ,\xi ^\prime >0$ that are sufficiently separated. Our main results are:

• (I) The expression (6) defines a Hermitian product that is independent of $\xi ,\xi ^\prime >0$ provided $|\log (\xi /\xi ^\prime )|>\frac{1}{2}|\log (q/t)|$,

• (II) the orthogonality relations $\langle SP_\lambda ,SP_\mu \rangle ^\prime _{n,m;q,t}=0$ hold true for all $\lambda \ne \mu$,

• (III) the (squared) norms $\langle SP_\lambda ,SP_\lambda \rangle ^\prime _{n,m;q,t}$ are given by the simple and explicit formulas (47)–(48).

Remark 1

The attentive reader might wonder why we do not simply integrate over ${\mathbb T}^n\times {\mathbb T}^m$ in (6) since, clearly, poles in the denominator of (7) would also be avoided by choosing $\xi =\xi ^\prime =1$. This can be readily understood in the simplest non-trivial case $n=m=1$ since, in this case, the integral in (6) can be easily computed; see Appendix D for details. One finds that the integral is the same for $\xi \gg \xi ^\prime$ and $\xi ^\prime \gg \xi$, but the integral for $\xi =\xi ^\prime$ differs by a non-trivial residue term which spoils our orthogonality results, as described above.

Remarkably, even though we are working with a complex-valued weight function (since the denominator in (7) is only real if $\xi =\xi ^\prime$), we find that all norms are given by non-negative real numbers. In addition, the super-Macdonald polynomials with non-zero norms are characterised by the simple condition $\lambda _{n}\ge m\ge \lambda _{n+1}$. As discussed in Sect. 3.2, the product $\langle \cdot ,\cdot \rangle ^\prime _{n,m;q,t}$ therefore provides the space spanned by the super-Macdonald polynomials with non-zero norm with a Hilbert space structure allowing for a quantum mechanical interpretation of the model defined by the commuting deformed Macdonald–Ruijsenaars operators ${\mathcal M}_{n,m,q,t}$ and ${\mathcal M}_{n,m,q^{-1},t^{-1}}$.

The results in this paper can be considered as natural q-deformations of the orthogonality relations and norm formula we obtained in [AHL19] for the super-Jack polynomials. As compared to loc. cit., significant simplifications occur: Since the eigenvalues of ${\mathcal M}_{n,m;q,t}$ separate the super-Macdonald polynomials $SP_\lambda$, there is no need to involve higher order eigenoperators; and the fact that $\Delta _{n,m}$ is a meromorphic function simplifies arguments involving contour deformations.

Our plan is as follows. In Sect. 2, we briefly review known facts about the Macdonald functions (Sect. 2.1) and super-Macdonald polynomials (Sect. 2.2) that we need. Our results can be found in Sect. 3: a precise formulation of our orthogonality result is given in Theorem 1 (Sect. 3.1), followed by a discussion of the Hilbert space interpretation of the super-Macdonald polynomials suggested by this (Sect. 3.2). The proof of Theorem 1 is given in Sect. 4. We conclude with a short discussion of research questions motivated by our results in Sect. 5. Three appendices explain how the conventions on super-Macdonald polynomials we use are related to the ones of Sergeev and Veselov [SV09a] (Appendix A), prove the relativistic invariance of the generalized Ruijsenaars model (Appendix B), give proof details to make this paper self-contained (Appendix C), and shortly discuss the special case $n=m=1$ (Appendix D).

1.1 Notation

We denote as $\mathcal {P}$ the space of all partitions, i.e., $\lambda \in \mathcal {P}$ means that $\lambda =(\lambda _1,\lambda _2,\ldots )$ with integers $\lambda _i\ge 0$ satisfying $\lambda _i\ge \lambda _{i+1}$, $i=1,2,\ldots$, and only finitely many $\lambda _i$’s non-zero; the non-zero $\lambda _i$’s are called parts of $\lambda$, and partitions differing only by a string of zeros at the end are not distinguished. For any partition $\lambda$, $\ell (\lambda )$ is the number of parts of $\lambda$, and $|\lambda |$ is the sum of its parts; $\ell (\lambda )$ and $|\lambda |$ are called length and weight of $\lambda$, respectively. Moreover, for $\lambda \in \mathcal {P}$, $\lambda '$ denotes the conjugate of $\lambda$ (so that the Young diagrams of $\lambda$ and $\lambda '$ are transformed into each other by reflection in the main diagonal). We also recall the definition of the dominance partial ordering on the set of partitions of a fixed weight: for $\lambda ,\mu \in \mathcal {P}$ such that $|\lambda |=|\mu |$,

$$\begin{aligned} \lambda \le \mu \Leftrightarrow \sum _{i=1}^j \lambda _i \le \sum _{i=1}^j \mu _i \quad (j=1,2,\ldots ). \end{aligned}$$

In addition, for $\lambda ,\mu \in \mathcal {P}$, $\mu \subseteq \lambda$ is short for $\mu _i\le \lambda _i$ for all i, and $\lambda \cup \mu$ denotes the partition obtained by merging and re-ordering the parts of $\lambda$ and $\mu$.

For $N\in {\mathbb Z}_{\ge 2}$, $1\le i\ne j\le N$ means $i,j=1,\ldots ,N$, $i\ne j$, and we write

$$\begin{aligned} \prod _{i\ne j}^N\ \ \ \text { short for } \ \ \ \prod _{\begin{array}{c} 1\le i\le N\\ i\ne j \end{array}}. \end{aligned}$$

For $z\in {\mathbb C}$, $\bar{z}$ is the complex conjugate of z. For $z=(z_1,\ldots ,z_N)\in {\mathbb C}^N$ with $N\in {\mathbb Z}_{\ge 2}$, $z^{-1}$, $\bar{z}$ and $\bar{z}^{-1}$ are short for $(1/z_1,\ldots ,1/z_N)$, $(\bar{z}_1,\ldots ,\bar{z}_N)$ and $(1/\bar{z}_1,\ldots ,1/\bar{z}_N)$, respectively. We write $\mathrm {i}:=\sqrt{-1}$ and ${\mathbb C}^*:={\mathbb C}\setminus \{0\}$.

2 Prerequisites

We collect definitions and results we need, following Macdonald [Mac95] in Sect. 2.1 and Sergeev and Veselov [SV09a] in Sect. 2.2.

2.1 Macdonald functions

Unless mentioned otherwise, $\lambda ,\mu$ are arbitrary partitions.

2.1.1 Symmetric functions

We consider the complex vector space $\Lambda =\Lambda _{{\mathbb C}}$ of symmetric functions in infinitely many variables $x=(x_1,x_2,\ldots )$ (we work over ${\mathbb C}$ to make the connection to quantum mechanics). It can be defined as the space of all finite linear combinations, with complex coefficients, of the symmetric monomial functions $m_\lambda$, labeled by partitions $\lambda$, and defined as follows:

$$\begin{aligned} m_\lambda (x) := \sum _a x_1^{a_1}x_2^{a_2}\cdots , \end{aligned}$$

(9)

where the sum is over all distinct permutations $a=(a_1,a_2,\ldots )$ of $\lambda =(\lambda _1,\lambda _2,\ldots )$. Thus, the symmetric monomial functions constitute a (vector space) basis in $\Lambda$ labeled by partitions. Another such basis is given by the products

$$\begin{aligned} p_\lambda := \prod _{i=1}^{\ell (\lambda )} p_{\lambda _i}\quad (\lambda \in \mathcal {P}) \end{aligned}$$

(10)

of the Newton sums

(11)

The space $\Lambda$ has a natural algebra structure and, as such, is freely generated by the Newton sums $p_r$, $r\in {\mathbb Z}_{\ge 1}$.

2.1.2 Macdonald functions

The space $\Lambda$ becomes a (pre-)Hilbert space when equipped with the scalar product $\langle \cdot ,\cdot \rangle _{q,t}$ characterised by linearity in its first (and antilinearity in its second) argument and²

$$\begin{aligned} \langle p_\lambda ,p_\mu \rangle _{q,t} = \delta _{\lambda \mu } z_\lambda \prod _{i=1}^{\ell (\lambda )}\frac{1-q^{\lambda _i}}{1-t^{\lambda _i}}, \end{aligned}$$

(12)

where $z_\lambda := \prod _{i=1}^{\lambda _1} i^{m_i}m_i!$ with $m_i=m_i(\lambda )$ the number of parts of $\lambda$ equal to i (setting $i^0 0!=1$), and $\delta _{\lambda \mu }$ the Kronecker delta.

As proved in [Mac95], the Macdonald functions $P_\lambda =P_\lambda (x;q,t)\in \Lambda$, $\lambda \in \mathcal {P}$, can be defined by the following two conditions: triangular structure,

$$\begin{aligned} P_\lambda = m_\lambda + \sum _{\mu <\lambda } u_{\lambda \mu } m_\mu \end{aligned}$$

for certain coefficients $u_{\lambda \mu }=u_{\lambda \mu }(q,t)$, and orthogonality,

$$\begin{aligned} \langle P_\lambda ,P_\mu \rangle _{q,t} = 0\quad (\lambda \ne \nu ). \end{aligned}$$

It is known that the Macdonald functions $P_\lambda$ are eigenfunctions of the inverse limit ${\mathcal M}_{q,t}$ of the operators ${\mathcal M}_{n;q,t}$ (3):

$$\begin{aligned} {\mathcal M}_{q,t}P_\lambda (x;q,t) = d_\lambda (q,t)P_\lambda (x;q,t), \end{aligned}$$

(14)

and the corresponding eigenvalues are given by

$$\begin{aligned} d_\lambda (q,t) = \sum _{i\ge 1}t^{1-i}\frac{q^{\lambda _i}-1}{1-q}. \end{aligned}$$

(15)

Moreover, the Macdonald functions are known to be invariant under $(q,t)\rightarrow (q^{-1},t^{-1})$, i.e.,

$$\begin{aligned} P_\lambda (x;q^{-1},t^{-1})=P_\lambda (x;q,t). \end{aligned}$$

(16)

We recall the definition of the dual Macdonald functions $Q_\lambda =Q_\lambda (x;q,t)$:

$$\begin{aligned} Q_\lambda = b_\lambda P_\lambda \end{aligned}$$

(17)

with $b_\lambda =b_\lambda (q,t):=\langle P_\lambda ,P_\lambda \rangle ^{-1}$ given by

$$\begin{aligned} b_\lambda (q,t) = \prod _{(j,k)\in \lambda }\frac{1-q^{\lambda _j-k}t^{\lambda _k'-j+1}}{1-q^{\lambda _j-k+1}t^{\lambda _k'-j}} = \frac{1}{b_{\lambda ^\prime }(t,q)}, \end{aligned}$$

(18)

where the product is over all (j, k) such that $j=1,\ldots ,\ell (\lambda )$ and $k=1,\ldots ,\lambda _j$. The two kinds of Macdonald functions obey $\langle P_\lambda ,Q_\lambda \rangle = \delta _{\lambda \mu }$, and they are related by the Macdonald involution

$$\begin{aligned} \omega _{q,t}: \Lambda \rightarrow \Lambda ,\ p_r\mapsto (-1)^{r-1}\frac{1-q^r}{1-t^r}p_r\quad (r\in {\mathbb Z}_{\ge 1}) \end{aligned}$$

(19)

as follows,

$$\begin{aligned} \omega _{q,t}(P_\lambda (x;q,t)) = Q_{\lambda '}(x;t,q). \end{aligned}$$

(20)

We also make use of the fact that the dual Macdonald functions $Q_\lambda$, like the Macdonald functions $P_\lambda$, are homogenous of degree $|\lambda |$:

$$\begin{aligned} Q_\lambda (sx;q,t)= s^{|\lambda |}Q_\lambda (x;q,t) \quad (s\in {\mathbb C}^*). \end{aligned}$$

(21)

2.1.3 Skew functions

Let $f^{\lambda }_{\mu \nu }=f^{\lambda }_{\mu \nu }(q,t):=\langle Q_\lambda ,P_\mu P_\nu \rangle _{q,t}$. Then the skew functions $P_{\lambda /\mu }\in \Lambda$ can be defined by

$$\begin{aligned} P_{\lambda /\mu }(x;q,t) : = \sum _{\nu } f^{\lambda '}_{\mu '\nu '}(t,q) P_\nu (x;q,t) \end{aligned}$$

(22)

with $P_\nu (x;q,t)$ the Macdonald functions. It is well-known that $f^{\lambda '}_{\mu '\nu '}$ is non-zero only if $\mu \subseteq \lambda$, $\nu \subseteq \lambda$, and $|\mu |+|\nu |=|\lambda |$.

These skew functions are homogenous of degree $|\lambda |-|\mu |$:

$$\begin{aligned} P_{\lambda /\mu }(sx;q,t)= s^{|\lambda |-|\mu |}P_{\lambda /\mu }(x;q,t) \quad (s\in {\mathbb C}^*), \end{aligned}$$

(23)

and they appear in the following expansion of Macdonald functions $P_\lambda (z;q,t)$ for variables $z=(x,y)$ obtained by merging two infinite sets of variables $x=(x_1,x_2,\ldots )$ and $y=(y_1,y_2,\ldots )$:

$$\begin{aligned} P_\lambda (x,y;q,t) = \sum _{\mu \subseteq \lambda } P_{\lambda /\mu }(x;q,t)Q_\mu (y;q,t). \end{aligned}$$

(24)

In the following Lemma, we state a well-known technical result that we need.

Lemma 1

The coefficients $f^{\lambda ^\prime }_{\mu ^\prime \nu ^\prime }(t,q)$ in (22) are non-zero only if

$$\begin{aligned} \mu \cup \nu \le \lambda \le \mu +\nu , \end{aligned}$$

and in the extremal cases they are given by

$$\begin{aligned} f^{(\mu \cup \nu )^\prime }_{\mu ^\prime \nu ^\prime }(t,q)=1,\ \ \ f^{(\mu +\nu )^\prime }_{\mu ^\prime \nu ^\prime }(t,q)=\frac{b_\mu (q,t)b_\nu (q,t)}{b_{\mu +\nu }(q,t)}. \end{aligned}$$

(For the convenience of the reader, we give a proof in Appendix C.1.)

2.1.4 Macdonald polynomials

The Macdonald polynomials $P_\lambda ((x_1,\ldots ,x_n);q,t)$ are obtained from the Macdonald functions $P_\lambda ((x_1,x_2,\ldots );q,t)$ by setting $x_{i}=0$ for all $i>n$, and similarly for $Q_\lambda$ and $P_{\lambda /\mu }$.

It is know that $P_\lambda ((x_1,\ldots ,x_n);q,t)$ is non-zero only for partitions $\lambda =(\lambda _1,\ldots ,\lambda _n)$ of length less or equal to n. Moreover, as already discussed in the introduction, the Macdonald polynomials are orthogonal with respect to the following scalar product,

$$\begin{aligned} \langle P,Q\rangle ^\prime _{n;q,t} := \int _{{\mathbb T}^n} \frac{dx_1}{2\pi \mathrm {i}x_1} \cdots \frac{dx_n}{2\pi \mathrm {i}x_n} \Delta _n(x;q,t) P(x)\overline{Q(x)} \end{aligned}$$

(25)

for P, Q symmetric polynomials in the variables $x=(x_1,\ldots ,x_n)\in {\mathbb C}^n$, ${\mathbb T}^n={\mathbb T}_1^n$ as in (4), and $\Delta _n(x;q,t)$ in (5): for all $P_\lambda =P_\lambda (x;q,t)$ with $x=(x_1,\ldots ,x_n)\in {\mathbb C}^n$ and $\lambda =(\lambda _1,\ldots ,\lambda _n)$,

$$\begin{aligned} \langle P_\lambda ,P_\mu \rangle ^\prime _{n;q,t} = \delta _{\lambda \mu }N_n(\lambda ;q,t), \end{aligned}$$

(26)

where

$$\begin{aligned} N_n(\lambda ;q,t)= \prod _{1\le i<j\le n} \frac{(q^{\lambda _i-\lambda _j}t^{j-i};q)_\infty (q^{\lambda _i-\lambda _j+1}t^{j-i};q)_\infty }{(q^{\lambda _i-\lambda _j}t^{j-i+1};q)_\infty (q^{\lambda _i-\lambda _j+1}t^{j-i+1};q)_\infty }. \end{aligned}$$

(27)

We also need

$$\begin{aligned} (x_1\ldots x_n)^k P_\lambda ((x_1,\ldots ,x_n);q,t) = P_{\lambda +(k^n)}((x_1,\ldots ,x_n);q,t) \quad (k\in {\mathbb Z}_{\ge 0}) \end{aligned}$$

(28)

where $\lambda +(k^n)=(\lambda _1+k,\ldots ,\lambda _n+k)$.

2.2 Super-Macdonald polynomials

Following Sergeev and Veselov [SV09a], we define $\Lambda _{n,m;q,t}$ as the algebra of complex polynomials P(x, y) in $n+m$ variables $(x,y)=(x_1,\ldots ,x_n,y_1,\ldots ,y_m)\in {\mathbb C}^n\times {\mathbb C}^m$ that are symmetric in each set of variables separately, i.e.,

$$\begin{aligned} P(\sigma x;\tau y) = P(x,y)\ \ \ ((\sigma ,\tau )\in S_n\times S_m) \end{aligned}$$

(29)

where $S_n$ is the group of permutations of n objects, and, furthermore, that satisfy the symmetry conditions³

$$\begin{aligned} \big (T_{q,x_i}-T_{t^{-1},y_j}\big )P(x,y) = 0\ \text { at } \ q^{1/2}x_i = t^{-1/2}y_j \ \ \ (\forall i,j). \end{aligned}$$

(30)

This algebra, $\Lambda _{n,m;q,t}$, is generated by the following deformed Newton sums,

$$\begin{aligned} p_{r}(x,y;q,t)= \sum _{i=1}^n x_i^r -\frac{q^{r/2}-q^{-r/2}}{t^{r/2}-t^{-r/2}} \sum _{k=1}^my_j^r \ \ \ (r\in {\mathbb Z}_{\ge 1}) \end{aligned}$$

(31)

for $(x,y)\in {\mathbb C}^n\times {\mathbb C}^m$ [SV09a, Theorem 5.8].

Remark 2

Many results in [SV09a] require a restriction to so-called non-special parameters q, t, i.e., $q^{i}t^{j}\ne 1$ for all $i,j\in {\mathbb Z}_{\ge 0}$ such that $i+j\ge 1$; see e.g. [SV09a, Theorem 5.8].⁴ However, since we assume $0<q,t<1$, we can ignore this restriction.

The super-Macdonald polynomials were defined in [SV09a] as the image of the Macdonald functions $P_\lambda$ under the homomorphism

$$\begin{aligned} \varphi _{n,m;q,t}: \Lambda \rightarrow \Lambda _{n,m;q,t},\ p_r((x_1,x_2,\ldots ))\mapsto p_r((x_1,\ldots ,x_n),(y_1,\ldots ,y_m);q,t). \end{aligned}$$

(32)

Thus, if $c_{\lambda \mu }(q,t)$ are the coefficients of the Macdonald polynomials defined by the expansion

$$\begin{aligned} P_\lambda (x;q,t)=\sum _{\mu } c_{\lambda \mu }(q,t) p_\mu (x)\quad (x=(x_1,x_2,\ldots )), \end{aligned}$$

(33)

then

$$\begin{aligned} SP_\lambda (x,y;q,t)=\sum _{\mu } c_{\lambda \mu }(q,t) p_\mu (x,y;q,t)\quad ((x,y)\in {\mathbb C}^n\times {\mathbb C}^m) \end{aligned}$$

(34)

where $p_\mu (x,y;q,t) =\prod _{i=1}^{\ell (\mu )}p_{\mu _i}(x,y;q,t)$.

From [SV09a, Theorem 5.4], we recall that $\varphi _{n,m;q,t}$ intertwines the operator ${\mathcal M}_{q,t}$ and the deformed Macdonald–Ruijsenaars operator ${\mathcal M}_{n,m;q,t}$ defined by (1)–(2):

$$\begin{aligned} \varphi _{n,m;q,t}\circ {\mathcal M}_{q,t} = {\mathcal M}_{n,m;q,t}\circ \varphi _{n,m;q,t}. \end{aligned}$$

(35)

Combining (14) with (35), we immediately see that

$$\begin{aligned} {\mathcal M}_{n,m;q,t}SP_\lambda (x,y;q,t) = d_\lambda (q,t)SP_\lambda (x,y;q,t); \end{aligned}$$

(36)

cf. [SV09a, Corollary 5.7]. We note that (16) implies that the coefficients $c_{\mu \nu }(q,t)$ in (33) are invariant under the transformation $(q,t)\rightarrow (q^{-1},t^{-1})$, and since the deformed Newton sums in (31) also have this invariance, (34) implies

$$\begin{aligned} SP_\lambda (x,y;q^{-1},t^{-1}) =SP_\lambda (x,y;q,t). \end{aligned}$$

(37)

Thus, the super-Macdonald polynomials $SP_\lambda ((x_1,\ldots ,x_n),(y_1,\ldots ,y_m);q,t)$ are also eigenfunctions of the deformed Macdonald–Ruijsenaars operator ${\mathcal M}_{n,m;q^{-1},t^{-1}}$ with eigenvalue $d_\lambda (q^{-1},t^{-1})$.

We also recall that $SP_\lambda (x,y;q,t)$ for $(x,y)\in {\mathbb C}^n\times {\mathbb C}^m$ is non-zero if and only if $\lambda$ belongs to the following set of partitions,

$$\begin{aligned} H_{n,m} := \{ \lambda = (\lambda _1,\lambda _2,\ldots )\in \mathcal {P}\ | \ \lambda _{n+1} \le m \} ; \end{aligned}$$

(38)

cf. [SV09a, Theorem 5.6].

Below we give an explicit representation of the super-Macdonald polynomials needed in the proof of our main result (this is a slight refinement of a result in [SV09a]).

Lemma 2

For $(x,y)\in {\mathbb C}^n\times {\mathbb C}^m$ and $\lambda \in H_{n,m}$, we have

$$\begin{aligned} SP_\lambda (x,y;q,t) = \sum _\mu (-q^{-1/2}t^{1/2})^{|\mu |} P_{\lambda /\mu ^\prime }(x;q,t)Q_\mu (y;t,q) \end{aligned}$$

(39)

where the sum runs over all partitions $\mu$ such that

$$\begin{aligned} (\langle \lambda ^\prime _1-n\rangle ,\ldots ,\langle \lambda ^\prime _m-n\rangle )\subseteq \mu \subseteq (\lambda ^\prime _1,\ldots ,\lambda ^\prime _m) \end{aligned}$$

(40)

where $\langle k\rangle :=\max (k,0)$.

Proof

Working with infinite sets of variables $x=(x_1,x_2,\ldots )$ and $y=(y_1,y_2,\ldots )$, we infer from (24) that

$$\begin{aligned} P_\lambda (x,y;q,t) = \sum _{\mu \subseteq \lambda ^\prime }P_{\lambda /\mu ^\prime }(x;q,t)P_{\mu ^\prime }(y;q,t) \end{aligned}$$

(where we have taken $\mu \rightarrow \mu ^\prime$ and used that $\mu ^\prime \subseteq \lambda$ if and only if $\mu \subseteq \lambda ^{\prime }$). Applying, with respect to y, the automorphism $\sigma _{q,t}: \Lambda \rightarrow \Lambda$ characterised by

$$\begin{aligned} (\sigma _{q,t}(p_r))(y) = (\omega _{q,t}(p_r))(-q^{-1/2}t^{1/2}y) = -\frac{q^{r/2}-q^{-r/2}}{t^{r/2}-t^{-r/2}}p_r(y)\ \ \ (r\ge 1), \end{aligned}$$

and setting $x_i=0$ for $i>n$ and $y_j=0$ for $j>m$, it is clear from (31)–(34) that $P_\lambda (x,y;q,t)$ is mapped to $SP_\lambda ((x_1,\ldots ,x_n),(y_1,\ldots ,y_m);q,t)$, so that (20) and (21) imply

$$\begin{aligned}&SP_\lambda ((x_1,\ldots ,x_n),(y_1,\ldots ,y_m);q,t)\\&\quad = \sum _{\mu \subseteq \lambda ^\prime }(-q^{-1/2}t^{1/2})^{|\mu |} P_{\lambda /\mu ^\prime }((x_1,\ldots ,x_n);q,t)Q_\mu ((y_1,\ldots ,y_m);t,q). \end{aligned}$$

To justify the conditions in (40): (i) Note that $\mu \not \subseteq (\lambda ^\prime _1,\ldots ,\lambda ^\prime _m)$ and $\mu \subseteq \lambda ^\prime$ can be simultaneously satisfied only if $\mu _{m+1}\ne 0$, in which case $Q_\mu (y_1,\ldots ,y_m;t,q)\equiv 0$, (ii) $(\langle \lambda ^\prime _1-n\rangle ,\ldots ,\langle \lambda ^\prime _m-n\rangle )\not \subseteq \mu$ is only possible if there exists $j=1,\ldots ,m$ such that $\lambda ^\prime _j-\mu _j>n$, but then $P_{\lambda /\mu ^\prime }((x_1,\ldots ,x_n);q,t)\equiv 0$ by Lemma 1 (a detailed justification of the latter can be found in Appendix C.2). $\quad \square$

3 Results

We now turn to our results. In Sect. 3.1, we introduce the relevant scalar product on the space $\Lambda _{n,m;q,t}$, spanned by the super-Macdonald polynomials, and state our main results in Theorem 1. The proof of this theorem is deferred to Sect. 4. The Hilbert space interpretation of deformed Macdonald–Ruijsenaars operators and super-Macdonald polynomials, as provided by this scalar product, is discussed in Sect. 3.2.

In what follows, we use the short-hand notation

$$\begin{aligned} d\omega _n(x):=\frac{1}{(2\pi \mathrm {i})^n}\frac{dx_1}{x_1}\cdots \frac{dx_n}{x_n} \end{aligned}$$

for variables $x=(x_1,\ldots ,x_n)\in {\mathbb C}^n$ and $n\in {\mathbb Z}_{\ge 1}$; we also recall the definition of the n-torus ${\mathbb T}_\xi ^n$ of radius $\xi >0$ in (4).

3.1 Orthogonality

We let $L_{n,m}=\mathbb {C}[x_1^{\pm 1},\ldots ,x_n^{\pm 1},y_1^{\pm 1},\ldots ,y_m^{\pm 1}]$ be the algebra of complex Laurent polynomials in the variables $x_1,\ldots ,x_n$ and $y_1,\ldots ,y_m$. For $f\in L_{n,m}$, we define its conjugate $f^*$ by

$$\begin{aligned} f^*(x,y)=\overline{f(\bar{x}^{-1},\overline{y}^{-1})}, \end{aligned}$$

(41)

where $\bar{x}^{-1}$ and $\bar{y}^{-1}$ are as in (8). We recall that $\Lambda _{n,m;q,t}$ is the space of polynomials P(x, y) in the variables $(x,y)=((x_1,\ldots ,x_n),(y_1,\ldots ,y_m))\in {\mathbb C}^n\times {\mathbb C}^m$ with complex coefficients satisfying the conditions in (29)–(30).

As already described in the introduction, the Hermitian product of $P,Q\in \Lambda _{n,m;q,t}$ is obtained by integrating the product of $P(x,y)Q^*(x,y)$ with the weight function $\Delta _{n,m}(x,y;q,t)$ in (5)–(7) over the $n+m$-dimensional torus ${\mathbb T}_\xi ^n\times {\mathbb T}_{\xi ^\prime }^m$ with suitable radii $\xi ,\xi ^\prime >0$; see (6). To see that we need to restrict the radii, we note that, while $P(x,y)Q^*(x,y)\in L_{n,m}$, and thus is holomorphic for $(x,y)\in (\mathbb {C}^*)^n\times (\mathbb {C}^*)^m$, the weight function $\Delta _{n,m}(x,y;q,t)$ is meromorphic with simple poles located along the hyperplanes

$$\begin{aligned} x_i= & {} q^{\frac{\delta }{2}}t^{-\frac{\delta }{2}}y_j\ \ \ (i = 1,\ldots ,n,\ j = 1,\ldots ,m,\ \delta = \pm 1), \end{aligned}$$

(42a)

$$\begin{aligned} tq^kx_i= & {} x_{i^\prime }\ \ \ (1\le i\ne i^\prime \le n,\ \ k\in \mathbb {Z}_{\ge 0}), \end{aligned}$$

(42b)

$$\begin{aligned} qt^ky_j= & {} y_{j^\prime }\ \ \ (1\le j\ne j^\prime \le m,\ \ k\in \mathbb {Z}_{\ge 0}). \end{aligned}$$

(42c)

Clearly, ${\mathbb T}_\xi ^n\times {\mathbb T}_{\xi ^\prime }^m$ is contained in the complement of these hyperplanes provided the radii $\xi ,\xi ^\prime >0$ are constrained as follows:

$$\begin{aligned} \xi /\xi ^\prime < \min _{\delta =\pm 1}\big (q^{\frac{\delta }{2}}t^{-\frac{\delta }{2}}\big ) \ \ \text {or}\ \ \xi /\xi ^\prime > \max _{\delta =\pm 1}\big (q^{\frac{\delta }{2}}t^{-\frac{\delta }{2}}\big ); \end{aligned}$$

(43)

if we restrict ourselves to such radii, we avoid all singularities of the integrand and thus obtain well-defined integrals; see Remark 1. Note that the condition in (43) can be written in a more compact way as follows, $|\log (\xi /\xi ^\prime )|>\frac{1}{2}|\log (q/t)|$.

Definition 1

For $\xi ,\xi ^\prime >0$ satisfying either of the two conditions in (43), we define a sesquilinear form $\langle \cdot ,\cdot \rangle _{n,m;q,t}^\prime$ on $\Lambda _{n,m;q,t}$ by

$$\begin{aligned} \langle P,Q\rangle _{n,m;q,t}^\prime = \frac{1}{n!m!}\int _{{\mathbb T}_\xi ^n}d\omega _n(x)\int _{{\mathbb T}_{\xi ^\prime }^m}d\omega _m(y)\Delta _{n,m}(x,y;q,t)P(x,y)Q^*(x,y) \end{aligned}$$

(44)

for arbitrary $P,Q\in \Lambda _{n,m;q,t}$.

Using that the integrand in (44) is analytic everywhere except along the hyperplanes (42), it is not difficult to prove that this sequilinar form does not depend on $\xi ,\xi ^\prime$ as long as they vary over only one of the two regions in (43); see Lemma 5. This argument applies to any Laurent polynomials $P,Q\in L_{n,m}$, but it does not rule out the possibility that the value of $\langle P,Q\rangle _{n,m;q,t}^\prime$ in the former region $\xi /\xi '< \min _{\delta =\pm 1}\big (q^{\frac{\delta }{2}}t^{-\frac{\delta }{2}}\big )$ is different from that in the latter region $\xi /\xi '> \max _{\delta =\pm 1}\big (q^{\frac{\delta }{2}}t^{-\frac{\delta }{2}}\big )$. However, as we will show, if P and Q belong to $\Lambda _{n,m;q,t}$, then the value of $\langle P,Q\rangle _{n,m;q,t}^\prime$ is the same in both regions.

In order to appreciate the significance of the conditions (29)–(30), it is instructive to consider the simplest non-trivial case $n=m=1$, in which the above claim can be verified by direct computations; the interested reader can find the details in Appendix D.

To state our main result in Theorem 1 below, we need two mappings e and s on partitions. For that, we observe that a partition $\lambda \in H_{n,m}$ such that $(m^n)\subseteq \lambda$ satisfies the conditions

$$\begin{aligned} \lambda _n\ge m\ge \lambda _{n+1} \end{aligned}$$

and, for this reason, it can be written as

$$\begin{aligned} \lambda =(e(n,m;{\lambda })+(m^n),s(n,m;{\lambda })') \end{aligned}$$

with two partitions $e(n,m;{\lambda })$ and $s(n,m;{\lambda })$ of lengths less or equal to n and m, respectively, and determined by $\lambda =(\lambda _1,\lambda _2,\ldots )$ as follows,

$$\begin{aligned} \begin{aligned} e(n,m;{\lambda })&:=(\lambda _1-m,\lambda _2-m,\ldots ,\lambda _n-m) = (\lambda _{m+1}^\prime ,\lambda _{m+2}^\prime ,\ldots )^\prime \\ s(n,m;{\lambda })&:=(\lambda _{n+1},\lambda _{n+2},\ldots )^\prime = (\lambda _1^\prime -n,\ldots ,\lambda _m^\prime -n); \end{aligned} \end{aligned}$$

(45)

see [AHL19, Section 2.2] for more details on these mappings e (short for east) and s (short for south), including the motivation for these names. To simplify notation, we write $e(\lambda )$ short for $e(n,m;{\lambda })$ and $s(\lambda )$ short for $s(n,m;{\lambda })$ if no confusion can arise.

Theorem 1

(a) The sesquilinear form $\langle \cdot ,\cdot \rangle _{n,m;q,t}^\prime$ from Definition 1 is Hermitian, i.e.

$$\begin{aligned} \langle P,Q\rangle _{n,m;q,t}^\prime = \overline{\langle Q,P\rangle _{n,m;q,t}^\prime },\ \ \ (P,Q\in \Lambda _{n,m;q,t}), \end{aligned}$$

and independent of the integration radii $\xi ,\xi ^\prime >0$ provided (43) holds true.

(b) The super-Macdonald polynomials $SP_\lambda =SP_\lambda ((x_1,\ldots ,x_n),(y_1,\ldots ,y_m);q,t)$, $\lambda \in H_{n,m}$, satisfy the orthogonality relations

$$\begin{aligned} \langle SP_\lambda ,SP_\mu \rangle ^\prime _{n,m;q,t} = \delta _{\lambda \mu }N_{n,m}(\lambda ;q,t), \end{aligned}$$

(46)

with (quadratic) norms

$$\begin{aligned} N_{n,m}(\lambda ;q,t) = 0\quad \text {if}\ \ (m^n)\not \subseteq \lambda \end{aligned}$$

(47)

and

$$\begin{aligned} \begin{aligned} N_{n,m}(\lambda ;q,t)&= (t/q)^{|s(\lambda )|}\frac{b_{e(\lambda )}(q,t)b_{s(\lambda )}(t,q)}{b_\lambda (q,t)}\\&\quad \cdot N_n(e(\lambda );q,t)N_{m}(s(\lambda );t,q)\ \ \text {if}\ \ (m^n)\subseteq \lambda , \end{aligned} \end{aligned}$$

(48)

cf. (18), (27) and (45).

3.2 Hilbert space interpretation

From Theorem 1, we see that the kernel of the Hermitian product (44) is spanned by the super-Macdonald polynomials with zero norm:

$$\begin{aligned} \begin{aligned} K_{n,m;q,t}&:= \ker \, \langle \cdot ,\cdot \rangle _{n,m;q,t}^\prime \\&= {{\,\mathrm{span}\,}}\big \{SP_\lambda ((x_1,\ldots ,x_n),(y_1,\ldots ,y_m);q,t)\mid (m^n)\not \subseteq \lambda \in H_{n,m}\big \}. \end{aligned} \end{aligned}$$

Since the remaining norms $N_{n,m}(\lambda ;q,t)$, where $(m^n)\subseteq \lambda$, are positive, we have the following result.

Proposition 1

The Hermitian form $\langle \cdot ,\cdot \rangle _{n,m;q,t}^\prime$ descends to a (positive definite) scalar product on the factor space

$$\begin{aligned} V_{n,m;q,t} := \Lambda _{n,m;q,t}/K_{n,m;q,t}, \end{aligned}$$

and the renormalised super-Macdonald polynomials

$$\begin{aligned} \widetilde{SP}_\lambda (x,y;q,t) := N_{n,m}(\lambda ;q,t)^{-1/2}SP_\lambda (x,y;q,t),\ \ \ (m^n)\subseteq \lambda , \end{aligned}$$

with $N_{n,m}(\lambda ;q,t)$ given by (48), yield an orthonormal basis in the resulting (pre-)Hilbert space.

Moreover, since the deformed Macdonald–Ruijsenaars operators ${\mathcal M}_{n,m;q^\pm ,t^\pm }$ leave $K_{n,m;q,t}$ invariant and their eigenvalues are all real (cf. (36)–(37)), they define (essentially) self-adjoint operators in $V_{n,m;q,t}$. Hence, we have assembled everything needed for a quantum mechanical interpretation of the model defined by ${\mathcal M}_{n,m;q^\pm ,t^\pm }$. However, as discussed in Sect. 5, these results are only the starting point for a fully satisfactory physical interpretation of the deformed trigonometric Ruijsenaars model.

It is interesting to note that the subset $\{\lambda \in H_{n,m}\mid (m^n)\subset \lambda \}\subset H_{n,m}$, which labels super-Macdonald polynomials with nonzero norm, is in a simple one-to-one correspondence with the subset $\{ (\mu ,\nu )\in \mathcal {P}\times \mathcal {P}\mid \ell (\mu )\le n,\ \ell (\nu )\le m\}\subset \mathcal {P}\times \mathcal {P}$, given explicitly by

$$\begin{aligned} \lambda = ((m^n)+\mu ,\nu ') ,\ \ \ \mu =e(n,m;\lambda ),\ \ \nu =s(n,m;\lambda ), \end{aligned}$$

cf. (45). Moreover, while the physical interpretation of a partition $\lambda$ of arbitrary length is not clear, the partitions $\mu =(\mu _1,\ldots ,\mu _n)$ and $\nu =(\nu _1,\ldots ,\nu _m)$ have a natural physical interpretation as momentum quantum numbers: The corresponding super-Macdonald polynomials

$$\begin{aligned} SP_{(\mu +(m^n),\nu ')}(x,y;q,t) \ \ \ ((x,y)\in {\mathbb C}^n\times {\mathbb C}^m) \end{aligned}$$

yield an orthogonal basis in $V_{n,m;q,t}$, and it is natural to interpret them as wave functions describing n and m particles of two different kinds labeled by a pair $(\mu ,\nu )$ of momentum quantum numbers.

From a physics point of view, it would be natural to express wave functions and operators in terms of the “additive” variables $(u,v)=(u_1,\ldots ,u_n,v_1,\ldots ,v_m)$ defined as follows,

$$\begin{aligned} x_i = \mathrm {e}^{2\pi \mathrm {i}u_i/L},\quad y_j=\mathrm {e}^{2\pi \mathrm {i}v_j/L}\ \ \ (L>0) \end{aligned}$$

(49)

and parameters

$$\begin{aligned} q = \mathrm {e}^{-2\pi \beta /L},\quad t = \mathrm {e}^{-2\pi \gamma /L}\ \ \ (\beta ,\gamma >0), \end{aligned}$$

(50)

cf. [RS86, Rui87]. Here $u_i\in [-L/2,L/2]$ and $v_i\in [-L/2,L/2]$ have the physical interpretation of position coordinates of particles. Indeed, in the original quantum field theoretic context of the Ruijsenaars model one is, eventually, interested in the limit $L\rightarrow \infty$, where space is the real line, but, to have a well-defined model, it is convenient to work with a circle of finite circumference L.

Taking, for simplicity, parameters $r,a,b>0$, we recall from [Rui97, Section III.C & Section V.C] the trigonometric Gamma function

$$\begin{aligned} G(r,a;z) = \prod _{k=0}^\infty \big (1-\exp (2\mathrm {i}r(z+\mathrm {i}a k+\mathrm {i}a/2))\big )^{-1} \end{aligned}$$

and weight function

$$\begin{aligned} w(r,a,b;z) = \frac{G(r,a;z+\mathrm {i}b-\mathrm {i}a/2)G(r,a;-z+\mathrm {i}b-\mathrm {i}a/2)}{G(r,a;z-\mathrm {i}a/2)G(r,a;-z-\mathrm {i}a/2)}. \end{aligned}$$

(51)

We note that $w(z)=w(r,a,b;z)$ is a (globally) meromorphic function with simple poles located at

$$\begin{aligned} z = j\pi /r-\mathrm {i}b-\mathrm {i}a(2k-1)\ \ \ (j\in \mathbb {Z},\ k\in \mathbb {Z}_{\ge 1}), \end{aligned}$$

and zeros at

$$\begin{aligned} z = j\pi /r-\mathrm {i}a(2k-1)\ \ \ (j\in \mathbb {Z},\ k\in \mathbb {Z}_{\ge 1}). \end{aligned}$$

Due to the manifest complex conjugation property

$$\begin{aligned} \overline{G(r,a;z)}=G(r,a;-\bar{z}), \end{aligned}$$

it follows, in particular, that w(z) is a regular and (strictly) positive function in $\mathbb {R}$. Assuming that $u\in (\mathbb {R}+\mathrm {i}\epsilon )^n$ and $v\in (\mathbb {R}+\mathrm {i}\epsilon ^\prime )^m$ for some $\epsilon ,\epsilon ^\prime \in {\mathbb R}$, we can thus introduce the (formal) groundstate wave function

$$\begin{aligned}&\Psi _0(u,v;\beta ,\gamma )\nonumber \\&\quad = \frac{\left( \prod _{1\le i<j\le n}w(\pi /L,\beta ,\gamma ;u_i-u_j)\right) ^{1/2} \left( \prod _{1\le i<j\le m}w(\pi /L,\gamma ,\beta ;v_i-v_j)\right) ^{1/2}}{\prod _{i=1}^n\prod _{j=1}^m 2\sin (\pi (u_i-v_j+\mathrm {i}\gamma /2-\mathrm {i}\beta /2)/L)},\nonumber \\ \end{aligned}$$

(52)

(where we take the positive square roots), and obtain a natural factorisation of the weight function $\Delta _{n,m}$ (7), as detailed in the following Lemma.

Lemma 3

For $(x,y)\in {\mathbb T}_\xi ^n\times {\mathbb T}_{\xi ^\prime }^m$, we have

$$\begin{aligned} \Delta _{n,m}(x,y;q,t) = \exp (nm\pi (\gamma -\beta )/L)\Psi _0(u,v;\beta ,\gamma )\overline{\Psi _0}(u,v;\beta ,\gamma ), \end{aligned}$$

where $\overline{\Psi _0}(u,v;\beta ,\gamma ):=\overline{\Psi _0(\bar{u},\bar{v};\beta ,\gamma )}$.

Proof

For x and z complex variables related as

$$\begin{aligned} x=\mathrm {e}^{2\pi \mathrm {i}z/L}, \end{aligned}$$

we use (50) to deduce

$$\begin{aligned} \begin{aligned} \frac{(x;q)_\infty }{(tx;q)_\infty }&= \prod _{k=0}^\infty \frac{1-\exp \big (\frac{2\pi \mathrm {i}}{L}(z+\mathrm {i}\beta k)\big )}{1-\exp \big (\frac{2\pi \mathrm {i}}{L}(z+\mathrm {i}\gamma +\mathrm {i}\beta k)\big )}\\&= \frac{G(\pi /L,\beta ;z+\mathrm {i}\gamma -\mathrm {i}\beta /2)}{G(\pi /L,\beta ;z-\mathrm {i}\beta /2)} \end{aligned} \end{aligned}$$

and

$$\begin{aligned}&(1-q^{-1/2}t^{1/2}x)(1-q^{-1/2}t^{1/2}x^{-1})\\&\quad = 4\mathrm {e}^{\pi (\beta -\gamma )/L}\sin (\pi (z+\mathrm {i}\gamma /2-\mathrm {i}\beta /2)/L)\sin (\pi (z-\mathrm {i}\gamma /2+\mathrm {i}\beta /2)/L). \end{aligned}$$

From (7) and (51)–(52), the statement can now be inferred by a straightforward computation. $\quad \square$

If we consider wave functions of the form

$$\begin{aligned} \Psi ^{(P)}(u,v;\beta ,\gamma ) : = P(x,y)\Psi _0(u,v;\beta ,\gamma ) \quad (u=(u_1,\ldots ,u_n),\, v=(v_1,\ldots ,v_m)) \end{aligned}$$

with $P\in \Lambda _{n,m;q,t}$, then we can use Lemma 3 to rewrite our Hermitian form as a suitably regularised version of a conventional Hilbert space product for a quantum mechanical model describing particles moving on the circle $[-L/2,L/2]$.

Proposition 2

Let $\epsilon ,\epsilon ^\prime \in \mathbb {R}$ be such that $|\epsilon -\epsilon ^\prime |>|\gamma -\beta |/2$. Then we have

$$\begin{aligned}&\int _{([-L/2,L/2]+\mathrm {i}\epsilon )^n}du\int _{([-L/2,L/2]+\mathrm {i}\epsilon ^\prime )^m}dv\, \Psi ^{(P)}(u,v;\beta ,\gamma )\overline{\Psi ^{(Q)}}(u,v;\beta ,\gamma )\nonumber \\&\quad = n!m! L^{n+m}\exp (nm\pi (\beta -\gamma )/L)\langle P,Q\rangle _{n,m;q,t}^\prime \end{aligned}$$

(53)

for all $P,Q\in \Lambda _{n,m;q,t}$, where $\overline{\Psi ^{(Q)}}(u,v;\beta ,\gamma )=\overline{\Psi ^{(Q)}(\bar{u},\bar{v};\beta ,\gamma )}$.

Proof

Taking

$$\begin{aligned} \xi = \mathrm {e}^{-2\pi \epsilon /L},\ \ \ \xi ^\prime = \mathrm {e}^{-2\pi \epsilon ^\prime /L}, \end{aligned}$$

it is readily seen that the constraint (43) is equivalent to $|\epsilon -\epsilon ^\prime |>|\gamma -\beta |/2$. Changing variables according to (49)–(50) in (44) and invoking Lemma 3, the equality (53) results. $\quad \square$

Remark 3

Note that, from a physics point of view, the positions $u_i$ and $v_j$ are real, but one has to continue the arguments of the super-Macdonald polynomials to the complex plane in order to compute their scalar product. This bears some resemblance to the fact that an eigenfunction of Ruijsenaars’ (analytic) difference operators needs to have sufficient analyticity in order for the corresponding eigenvalue equations to make sense, see e.g. [Rui01].

As we demonstrate in Appendix C.3, the terms in the deformed Macdonald–Ruijsenaars operator ${\mathcal M}_{n,m;q,t}$ in (1)–(2) not involving a shift operator add upp to a constant. Dropping this overall constant, we get the operator

$$\begin{aligned} M_{n,m;q,t} := \frac{t^{1-n}}{1-q} \sum _{i=1}^n A_iT_{q,x_i} + \frac{q^{m-1}}{1-t^{-1}} \sum _{j=1}^m B_jT_{t^{-1},y_j}. \end{aligned}$$

(54)

Changing variables and parameters according to (49) and performing a similarity transformation with $\Psi _0$, a direct computation, using the difference equation

$$\begin{aligned} \frac{G(z+\mathrm {i}a/2)}{G(z-\mathrm {i}a/2)}=1-\mathrm {e}^{2\mathrm {i}rz}, \end{aligned}$$

satisfied by the trigonometric Gamma function (cf. [Rui97, Section III.C]), yields

$$\begin{aligned}&R_{n,m;\beta ,\gamma }^- := \mathrm {e}^{(m-1)\pi \beta /L}\mathrm {e}^{(1-n)\pi \gamma /L}\Psi _{0} M_{n,m;q,t} \Psi _{0}^{-1}\nonumber \\&\quad = \frac{1}{2\sinh (\pi \beta /L)}\sum _{i=1}^n \prod _{i^\prime \ne i}^n f_+(u_i-u_{i^\prime };\gamma )\cdot \mathrm {e}^{\mathrm {i}\beta \partial _{u_i}}\cdot \prod _{i^\prime \ne i}^n f_-(u_i-u_{i^\prime };\gamma )\nonumber \\&\qquad -\frac{1}{2\sinh (\pi \gamma /L)}\sum _{j=1}^m \prod _{i=1}^n \frac{\sin (\pi (u_i-v_j-\mathrm {i}\gamma /2+\mathrm {i}\beta /2)/L)}{\sin (\pi (u_i-v_j+\mathrm {i}\gamma /2+\mathrm {i}\beta /2)/L)}\nonumber \\&\qquad \cdot \prod _{j^\prime \ne j}^m f_-(v_j-v_{j^\prime };\beta )\cdot \mathrm {e}^{-\mathrm {i}\gamma \partial _{v_j}} \cdot \prod _{j^\prime \ne j}^m f_+(v_j-v_{j^\prime };\beta )\nonumber \\&\qquad \cdot \prod _{i=1}^n \frac{\sin (\pi (u_i-v_j+\mathrm {i}\gamma /2-\mathrm {i}\beta /2)/L)}{\sin (\pi (u_i-v_j-\mathrm {i}\gamma /2-\mathrm {i}\beta /2)/L)} \end{aligned}$$

(55)

and

$$\begin{aligned}&R_{n,m;\beta ,\gamma }^+ := \mathrm {e}^{(1-m)\pi \beta /L}\mathrm {e}^{(n-1)\pi \gamma /L}\Psi _{0} M_{n,m;q^{-1},t^{-1}} \Psi _{0}^{-1}\nonumber \\&\quad = -\frac{1}{2\sinh (\pi \beta /L)}\sum _{i=1}^n \prod _{j=1}^m \frac{\sin (\pi (u_i-v_j-\mathrm {i}\gamma /2+\mathrm {i}\beta /2)/L)}{\sin (\pi (u_i-v_j-\mathrm {i}\gamma /2-\mathrm {i}\beta /2)/L)}\nonumber \\&\quad \quad \cdot \prod _{i^\prime \ne i}^n f_-(u_i-u_{i^\prime };\gamma )\cdot \mathrm {e}^{-\mathrm {i}\beta \partial _{u_i}}\cdot \prod _{i^\prime \ne i}^n f_+(u_i-u_{i^\prime };\gamma )\nonumber \\&\quad \quad \cdot \prod _{j=1}^m \frac{\sin (\pi (u_i-v_j+\mathrm {i}\gamma /2-\mathrm {i}\beta /2)/L)}{\sin (\pi (u_i-v_j+\mathrm {i}\gamma /2+\mathrm {i}\beta /2)/L)}\nonumber \\&\quad \quad + \, \frac{1}{2\sinh (\pi \gamma /L)}\sum _{j=1}^m\prod _{j^\prime \ne j}^m f_+(v_j-v_{j^\prime };\beta )\cdot \mathrm {e}^{\mathrm {i}\gamma \partial _{v_j}} \cdot \prod _{j^\prime \ne j}^m f_-(v_j-v_{j^\prime };\beta ), \end{aligned}$$

(56)

with

$$\begin{aligned} f_\pm (z;\gamma ) = \left( \frac{\sin (\pi (z\pm \mathrm {i}\gamma )/L)}{\sin (\pi z/L)}\right) ^{1/2}, \end{aligned}$$

and where $\partial _{u_i}=\partial /\partial u_i$ and $\partial _{v_j}=\partial /\partial v_j$.

The structure of these operators occupies a sort of middle-ground between the trigonometric degeneration of Ruijsenaars’ original (undeformed elliptic) operator ${\hat{S}}_{\pm 1}$ and a similarity transform $\mathcal {A}_{\pm 1}=U^{-1/2}{\hat{S}}_{\pm 1}U^{1/2}$ with a trigonometric ‘scattering function’ U. (Explicit expressions for the latter operators and the pertinent scattering function (in the hyperbolic case) can, e.g., be found in [HR14].) In particular, setting $m=0$ in (5556)–(5556) and comparing the resulting expressions with the trigonometric instances of Eqs. (2.3) and (A4) (taking $\nu =\pi /L$ and $\mu =i\gamma$), we find that

$$\begin{aligned} R_{n,0;\beta ,\gamma }^\pm = \frac{\pm 1}{2\sinh (\pi \beta /L)}{\hat{S}}_{\pm 1}. \end{aligned}$$

(57)

Moreover, as shown in Appendix B, the operators $R_{n,m;\beta ,\gamma }^\pm$ define a model that is relativistically invariant, for arbitrary particle numbers n and m.

4 Proofs

This section is devoted to the proof of Theorem 1. In place of (44), we write

$$\begin{aligned}&\langle P,Q\rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime )\nonumber \\&\quad = \frac{1}{n!m!}\int _{{\mathbb T}_\xi ^n}d\omega _n(x)\int _{{\mathbb T}_{\xi ^\prime }^m}d\omega _m(y)\Delta _{n,m}(x,y;q,t)P(x,y)Q^*(x,y) \end{aligned}$$

(58)

for $P,Q\in \Lambda _{n,m;q,t}$, so that we easily can keep track of the choice of integration radii $\xi ,\xi ^\prime >0$. Introducing the maximum function

$$\begin{aligned} M(q,t) := \max _{\delta =\pm 1}\big (q^{\frac{\delta }{2}}t^{-\frac{\delta }{2}}\big ), \end{aligned}$$

we note that the conditions (43) can be expressed as

$$\begin{aligned} \xi /\xi ^\prime > M(q,t) \ \ \text {or}\ \ \xi /\xi ^\prime < 1/M(q,t). \end{aligned}$$

(59)

In Sect. 4.1, we prove preliminary results on the sesquilinear form given by (58). Based on this, we prove Theorem 1 in Sect. 4.2.

4.1 Preliminary results

First, we establish a simple transformation property of the form (58) under the inversion $(\xi ,\xi ^\prime )\rightarrow (1/\xi ,1/\xi ^\prime )$ of integration radii.

Lemma 4

For all $P,Q\in \Lambda _{n,m;q,t}$, we have

$$\begin{aligned} \langle P,Q\rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime ) = \overline{ \langle Q,P\rangle _{n,m;q,t}^\prime (1/\xi ,1/\xi ^\prime ) }. \end{aligned}$$

(60)

Proof

We find it convenient to work with the “additive” variables given by (49) with $L=2\pi$. In order to avoid cumbersome and lengthy formulas, we suppress the parameters q, t and use the short-hand notation

$$\begin{aligned} \exp (\mathrm {i}u) := (\mathrm {e}^{\mathrm {i}u_1},\ldots ,\mathrm {e}^{\mathrm {i}u_n}),\ \ \ \exp (\mathrm {i}v) := (\mathrm {e}^{\mathrm {i}v_1},\ldots ,\mathrm {e}^{\mathrm {i}v_m}). \end{aligned}$$

Observing that

$$\begin{aligned} \Delta _{n,m}(x,y) = \Delta _{n,m}(x^{-1},y^{-1}), \end{aligned}$$

we thus rewrite the left-hand side of (60) as

$$\begin{aligned}&\frac{1}{(2\pi )^{n+m}}\int _{[-\pi ,\pi ]^n}du \int _{[-\pi ,\pi ]^m}dv\, \overline{\Delta _{n,m}(\xi ^{-1}\exp (\mathrm {i}u),(\xi ^\prime )^{-1}\exp (\mathrm {i}v))}\\&\quad \cdot P(\xi \exp (\mathrm {i}u),\xi ^\prime \exp (\mathrm {i}v))\overline{Q(\xi ^{-1}\exp (\mathrm {i}u),(\xi ^\prime )^{-1}\exp (\mathrm {i}v))}. \end{aligned}$$

Finally, using the observation

$$\begin{aligned} P(\xi \exp (\mathrm {i}u),\xi ^\prime \exp (\mathrm {i}v)) = \overline{P^*(\xi ^{-1}\exp (\mathrm {i}u),(\xi ^\prime )^{-1}\exp (\mathrm {i}v))}, \end{aligned}$$

we see that this integral is equal to the right-hand side of (60). $\quad \square$

We proceed to show that (58) is invariant under continuous deformations of the integration radii as long as they satisfy (59).

Lemma 5

For any $P,Q\in \Lambda _{n,m;q,t}$, the value of $\langle P,Q\rangle ^\prime _{n,m;q,t}(\xi ,\xi ^\prime )$ does not change as $\xi ,\xi ^\prime >0$ vary within one of the two regions $\xi /\xi ^\prime >M(q,t)$ and $\xi /\xi ^\prime <1/M(q,t)$.

Proof

We note that $\xi /\xi ^\prime <1/M(q,t)$ if and only if $(1/\xi )/(1/\xi ^\prime )>M(q,t)$. Hence, thanks to Lemma 4, we may and shall restrict attention to the region $\xi /\xi ^\prime >M(q,t)$.

By Cauchy’s theorem, we can deform the integration contours in (58) one at a time, without changing the value of the integral, as long as we do not encounter any of the poles (42a)–(42c). In particular, taking

$$\begin{aligned} \varvec{\xi } = (\xi _1,\ldots ,\xi _n)\in (0,\infty )^n,\ \ \ \varvec{\xi }^\prime = (\xi ^\prime _1,\ldots ,\xi ^\prime _m)\in (0,\infty )^m, \end{aligned}$$

such that

$$\begin{aligned}&t^{1/2}<\xi _i/\xi _{i^\prime }<t^{-1/2}\ \ (1\le i\ne i^\prime \le n), \end{aligned}$$

(61a)

$$\begin{aligned}&q^{1/2}<\xi _j^\prime /\xi _{j^\prime }^\prime <q^{-1/2}\ \ (1\le j\ne j^\prime \le n), \end{aligned}$$

(61b)

$$\begin{aligned}&\xi _i/\xi _j^\prime >M(q,t)\ \ (i = 1,\ldots ,n,\ j = 1,\ldots ,m), \end{aligned}$$

(61c)

we can replace ${\mathbb T}_\xi ^n$ and ${\mathbb T}_{\xi ^\prime }^m$ by

$$\begin{aligned} {\mathbb T}_{\varvec{\xi }}^n := {\mathbb T}_{\xi _1}\times \cdots \times {\mathbb T}_{\xi _n},\ \ \ {\mathbb T}_{\varvec{\xi }^\prime }^m := {\mathbb T}_{\xi ^\prime _1}\times \cdots \times {\mathbb T}_{\xi ^\prime _m}, \end{aligned}$$

respectively, and the resulting integral

$$\begin{aligned} \int _{{\mathbb T}_{\varvec{\xi }}^n}d\omega _n(x)\int _{{\mathbb T}_{\varvec{\xi }^\prime }^m}d\omega _m(y)\, \Delta _{n,m}(x,y;q,t)P(x,y)\overline{Q}(x^{-1},y^{-1}) \end{aligned}$$

is independent of $\varvec{\xi }\in (0,\infty )^n$ and $\varvec{\xi }^\prime \in (0,\infty )^m$ provided the inequalities (61a)–(61c) are satisfied. Indeed, these inequalities clearly define an open and (pathwise) connected subset of $(0,\infty )^n\times (0,\infty )^m$, so that, regardless of the initial integration radii, any admissible radii $\varvec{\xi }$ and $\varvec{\xi }^\prime$ can be reached in a finite number of steps, where each step consist of a deformation of a single radius. Since (61a)–(61c) are satisfied whenever $\xi _1=\cdots =\xi _n=\xi$ and $\xi ^\prime _1=\cdots =\xi _m^\prime =\xi ^\prime$ with $\xi /\xi ^\prime >M(q,t)$, the lemma follows. $\quad \square$

We note that this result leaves open the possibility that the sesquilinear form (58) takes different values in the two regions $\xi /\xi ^\prime >M(q,t)$ and $\xi /\xi ^\prime <1/M(q,t)$—this possibility is only ruled out by arguments given in Sect. 4.2.

To conclude, we show that the deformed Macdonald operator ${\mathcal M}_{n,m;q,t}$ (as defined by (1)–(2)) is self-adjoint.

Lemma 6

For all $P,Q\in \Lambda _{n,m;q,t}$, we have

$$\begin{aligned} \langle {\mathcal M}_{n,m;q,t}P,Q\rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime ) = \langle P,{\mathcal M}_{n,m;q,t}Q\rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime ) \end{aligned}$$

(62)

provided $\xi ,\xi '>0$ satisfy (59).

Proof

We observe that the terms in ${\mathcal M}_{n,m;q,t}$ (1)–(2) not involving shift operators add up to a real constant, which clearly is self-adjoint:

$$\begin{aligned} \frac{t^{1-n}}{1-q}\sum _{i=1}^nA_i(x,y;q,t) + \frac{q^{m-1}}{1-t^{-1}}B_j(x,y;q,t) = \frac{1-t^{-n}q^m}{(1-t^{-1})(1-q)} ; \end{aligned}$$

(63)

for the convenience of the reader, we include an elementary proof of this identity in Appendix C.3.

Since ${\mathcal M}_{n,m;q,t}$ leaves $\Lambda _{n,m;q,t}$ invariant [SV09a, Proposition 5.3], Lemma 5 ensures that no generality is lost when replacing (59) with

$$\begin{aligned} \xi /\xi ^\prime <(qt)^{1/2}\ \ \text { or } \ \ \xi /\xi ^\prime >(qt)^{-1/2}. \end{aligned}$$

(64)

Under this stronger condition on the integration radii $\xi$ and $\xi ^\prime$, we proceed to show that all terms $A_iT_{q,x_i}$ and $B_jT_{t^{-1},y_j}$, for $i=1,\ldots ,n$ and $j=1,\ldots ,m$, in ${\mathcal M}_{n,m;q,t}$ are separately self-adjoint. In what follows, we do not indicate the dependence on q and t, to simplify notation.

Fixing $i=1,\ldots ,n$, we introduce the function

$$\begin{aligned} W_i(x,y):= \prod _{i'\ne i}^n \frac{(x_i/x_{i'};q)_\infty }{(tx_i/x_{i'};q)_\infty }\cdot \prod _{j=1}^m \frac{1}{1-q^{-1/2}t^{1/2}x_i/y_j}, \end{aligned}$$

(65)

so that $W_i(x,y)W_i^*(x,y)$ amounts to all $x_i$-dependent factors in $\Delta _{n,m}(x,y)$, and note that self-adjointness of $A_i(x,y)T_{q,x_i}$ follows once we show that

$$\begin{aligned}&\oint _{|x_i|=\xi } \frac{dx_i}{2\pi \mathrm {i}x_i} W_i(x,y)W_i^*(x,y) Q^*(x,y)A_i(x,y)T_{q,x_i}P(x,y) \nonumber \\&\quad = \oint _{|x_i|=\xi } \frac{dx_i}{2\pi \mathrm {i}x_i} W_i(x,y)W_i^*(x,y) P(x,y) (A_i(x,y)T_{q,x_i}Q)^*(x,y). \end{aligned}$$

(66)

To this end, we observe that

$$\begin{aligned} \frac{T_{q,x_i} W_i(x,y)}{W_i(x,y)} = \prod _{i'\ne i}^n \frac{1-tx_i/x_{i'}}{1-x_i/x_{i'}}\cdot \prod _{j=1}^m \frac{(1-q^{-1/2}t^{1/2}x_i/y_j)}{(1-q^{1/2}t^{1/2}x_i/y_j)} = q^{-m}A_i(x,y), \end{aligned}$$

and using this, we can write (66) as follows,

$$\begin{aligned}&\oint _{|x_i|=\xi } \frac{dx_i}{2\pi \mathrm {i}x_i} W_i^*(x,y) Q^*(x,y) (T_{q,x_i}W_iP)(x,y) \\&\quad = \oint _{|x_i|=\xi } \frac{dx_i}{2\pi \mathrm {i}x_i} W_i(x,y) P(x,y) (T_{q,x_i}W_iQ)^*(x,y). \end{aligned}$$

We now change variables $x_i\rightarrow q x_i$ in the latter integral to obtain

$$\begin{aligned}&\oint _{|x_i|=\xi /q} \frac{dx_i}{2\pi \mathrm {i}x_i} (T_{q,x_i}W_iP)(x,y) W_i^*(x,y)Q^*(x,y)\nonumber \\&\quad = \oint _{|x_i|=\xi } \frac{dx_i}{2\pi \mathrm {i}x_i} (T_{q,x_i}W_iP)(x,y) W_i^*(x,y)Q^*(x,y), \end{aligned}$$

(67)

where the equality holds true due to Cauchy’s theorem, since the integrand, which is the same in both integrals, is an analytic function of $x_i$ in the region $\xi \le |x_i|\le \xi /q$ when

$$\begin{aligned} |x_{i'}|=\xi \quad (i'=1,\ldots ,i-1,i+1,\ldots ,n),\quad |y_j|=\xi ^\prime \quad (j=1,\ldots ,m) \end{aligned}$$

(68)

and (64) is satisfied. A proof of this analyticity property of the integrand can be found in Appendix C.4. We have thus established (66) and, as previously noted, self-adjointness of $A_i(x,y)T_{q,x_i}$ immediately follows.

A proof of self-adjointness of the terms $B_j(x,y)T_{t^{-1},y_j}$, under the condition (64), can be obtained in a similar manner, and the details are therefore omitted.⁵$\quad \square$

4.2 Proof of Theorem 1

Making use of results from Sect. 4.1, we prove the two parts of the theorem in reverse order.

4.2.1 Part (b)

Using (36) and Lemma 6, we deduce

$$\begin{aligned} \big (d_\lambda (q,t)-d_\mu (q,t)\big )\langle SP_\lambda ,SP_\mu \rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime ) = 0. \end{aligned}$$

Assuming that $\lambda \ne \mu$, we see from (15) that

$$\begin{aligned} p_{\lambda \mu }(q,t) := t^{\max (\ell (\lambda ),\ell (\mu ))-1}\big (d_\lambda (q,t)-d_\mu (q,t)\big ) \end{aligned}$$

is a non-zero polynomial function in q and t for $0<q,t<1$. Introducing its zero set

$$\begin{aligned} Z_{\lambda \mu } := \{(q,t)\in (0,1)^2\mid p_{\lambda \mu }(q,t)=0\}, \end{aligned}$$

we can thus conclude that $\langle SP_\lambda ,SP_\mu \rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime )$ must vanish for all $(q,t)\in (0,1)^2\setminus Z_{\lambda \mu }$. Since the Hermitian form $\langle \cdot ,\cdot \rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime )$ depends continuously on (q, t) and $(0,1)^2\setminus Z_{\lambda \mu }$ is a dense (open) subset of $(0,1)^2$, it clearly follows that the orthogonality relations (46) hold true for all $(q,t)\in (0,1)^2$.

We proceed to compute the norms $N_{n,m}(\lambda ;q,t):=\langle SP_\lambda ,SP_\lambda \rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime )$. Due to Lemma 5, we can do this by fixing $\xi ^\prime =1$ (say) and taking the limit $\xi \rightarrow \infty$. To this end, we note that the asymptotic behaviour of the weight function in (7) for $x\in {\mathbb T}_\xi ^n$ and $y\in {\mathbb T}^m$ as $\xi \rightarrow \infty$ is given by

$$\begin{aligned} \begin{aligned} \Delta _{n,m}(x,y;q,t) = \frac{\Delta _n(x;q,t)\Delta _m(y;t,q)}{\prod _{i=1}^n\prod _{j=1}^m(-q^{-1/2}t^{1/2}x_i/y_j)}(1+O(1/\xi )) \\ = (-q^{-1/2}t^{1/2})^{-nm} e_n^m(x^{-1})e_m^n(y)\Delta _n(x;q,t)\Delta _m(y;t,q)(1+O(1/\xi )) \end{aligned} \end{aligned}$$

(69)

where

$$\begin{aligned} e^m_n(x^{-1}):= (x_1\cdots x_n)^{-m},\ \ \ e_m^n(y):=(y_1\cdots y_m)^{n}. \end{aligned}$$

Introducing the notation

$$\begin{aligned} \mu _\text {min} := (\langle \lambda ^\prime _1-n\rangle ,\ldots ,\langle \lambda ^\prime _m-n\rangle ), \ \ \ \mu _\text {max} := (\lambda ^\prime _1,\ldots ,\lambda ^\prime _m), \end{aligned}$$

(70)

where $\langle k\rangle :=\max (0,k)$, we use Lemma 2 and (23) to deduce that, for $x\in {\mathbb T}_\xi ^n$ and $y\in {\mathbb T}^m$,

$$\begin{aligned}&SP_\lambda (x,y;q,t)SP_\lambda (x^{-1},y^{-1};q,t)\nonumber \\&\quad = (-q^{-1/2}t^{1/2})^{|\mu _\text {min}|+|\mu _\text {max}|}P_{\lambda /\mu _\text {min}^\prime }(x;q,t)P_{\lambda /\mu _\text {max}^\prime }(x^{-1};t,q)\nonumber \\&\quad \cdot Q_{\mu _\text {min}}(y;t,q)Q_{\mu _\text {max}}(y^{-1};t,q)+O(\xi ^{|\mu _\text {max}|-|\mu _\text {min}|-1}) \end{aligned}$$

(71)

as $\xi \rightarrow \infty$. From (69)–(71) we readily obtain,

$$\begin{aligned}&N_{n,m}(\lambda ;q,t) = (-q^{-1/2}t^{1/2})^{|\mu _\text {min}|+|\mu _\text {max}|-mn}\cdot \xi ^{|\mu _\text {max}|-|\mu _\text {min}|-nm}\\&\quad \cdot \langle P_{\lambda /\mu _\text {min}^\prime },e_n^mP_{\lambda /\mu _\text {max}^\prime }\rangle _{n;q,t}^\prime \langle e_m^nQ_{\mu _\text {min}},Q_{\mu _\text {max}}\rangle _{m;t,q}^\prime \\&\quad +O(\xi ^{|\mu _\text {max}|-|\mu _\text {min}|-nm-1})\ \ \ (\xi \rightarrow \infty ) \end{aligned}$$

using (23), with $\langle \cdot ,\cdot \rangle _{n;q,t}$ in (25) (the factor $\xi ^{|\mu _\text {max}|-|\mu _\text {min}|-nm}$ is due to the change of variables $x\rightarrow x/\xi$ transforming ${\mathbb T}_\xi ^n\rightarrow {\mathbb T}_1^n={\mathbb T}^n$). Since $|\mu _\text {max}|-|\mu _\text {min}|\le nm$ with equality if and only if $(m^n)\subseteq \lambda$, the validity of (47) immediately follows.

In the remaining cases $(m^n)\subseteq \lambda$,

$$\begin{aligned} \mu _\text {max}=\mu _\text {min}+(n^m),\ \ \ \mu _\text {min}=s(\lambda ) \end{aligned}$$

(72)

(cf. (45) and (70)), and therefore

$$\begin{aligned} (-q^{-1/2}t^{1/2})^{|\mu _\text {min}|+|\mu _\text {max}|-mn}\cdot \xi ^{|\mu _\text {max}|-|\mu _\text {min}|-nm} = (t/q)^{|s(\lambda )|} \end{aligned}$$

independent of $\xi$. Thus, by taking the limit $\xi \rightarrow \infty$, we obtain

$$\begin{aligned} N_{n,m}(\lambda ;q,t) = (t/q)^{|s(\lambda )|} \langle P_{\lambda /\mu _\text {min}^\prime },e_n^mP_{\lambda /\mu _\text {max}^\prime }\rangle _{n;q,t}^\prime \langle e_m^nQ_{\mu _\text {min}},Q_{\mu _\text {max}}\rangle _{m;t,q}^\prime \end{aligned}$$

(73)

with $\mu _\text {max}$ and $\mu _\text {min}$ in (72).

We are left to compute the scalar products in (73). We start with the second one:

$$\begin{aligned} \begin{aligned} \langle e_m^nQ_{\mu _\text {min}},Q_{\mu _\text {max}}\rangle _{m;t,q}^\prime&= \, \langle e_m^nQ_{s(\lambda )},Q_{s(\lambda )+(n^m)}\rangle _{m;t,q}^\prime \\&= \, b_{s(\lambda )}(t,q) b_{s(\lambda )+(n^m)}(t,q) \langle P_{s(\lambda )+(n^m)},P_{s(\lambda )+(n^m)}\rangle _{m;t,q}^\prime \\&= \, b_{s(\lambda )}(t,q) b_{s(\lambda )+(n^m)}(t,q) N_m(s(\lambda );t,q) \end{aligned} \end{aligned}$$

by (17), (26) and (28), using

$$\begin{aligned} N_n(\lambda +(k^n);q,t)= N_n(\lambda ;q,t)\quad (k\in {\mathbb Z}_{\ge 1}); \end{aligned}$$

cf. (27). The first product can be computed in a similar manner, using (22),

$$\begin{aligned} \begin{aligned} \langle P_{\lambda /\mu _\text {min}^\prime },e_n^mP_{\lambda /\mu _\text {max}^\prime }\rangle _{n;q,t}^\prime =&\sum _{\nu _1,\nu _2} f^{\lambda ^\prime }_{\mu _\text {min},\nu _1^\prime }(t,q) f^{\lambda ^\prime }_{\mu _\text {max},\nu _2^\prime }(t,q)\langle P_{\nu _1},P_{\nu _2+(m^n)}\rangle ^\prime _{n;q,t} \\ =&\sum _\nu f^{\lambda ^\prime }_{\mu _\text {min},(\nu +(n^m))^\prime }(t,q) f^{\lambda ^\prime }_{\mu _\text {max},\nu ^\prime }(t,q) N_n(\nu +(m^n);q,t) \\ =&\sum _\nu f^{\lambda ^\prime }_{s(\lambda ),(\nu +(m^n))^\prime }(t,q) f^{\lambda ^\prime }_{s(\lambda )+(n^m),\nu ^\prime }(t,q) N_n(\nu ;q,t) . \end{aligned} \end{aligned}$$

By Lemma 1, $f^{\lambda ^\prime }_{s(\lambda ),(\nu +(m^n)^\prime }$ and $f^{\lambda ^\prime }_{s(\lambda )+(n^m),\nu ^\prime }$ are non-zero only if

$$\begin{aligned} s(\lambda )^\prime \cup (\nu +(m^n))\le \lambda \end{aligned}$$

(74)

and

$$\begin{aligned} \lambda \le (s(\lambda )+(n^m))^\prime + \nu = (s(\lambda )^\prime \cup (m^n)) + \nu , \end{aligned}$$

(75)

respectively. The largest $\nu$ (in the sense of dominance ordering) satisfying (74) is $\nu =e(\lambda )$ and, for this partition and only this,

$$\begin{aligned} s(\lambda )^\prime \cup (\nu +(m^n))=\lambda \ \ \text { for } \nu =e(\lambda ). \end{aligned}$$

Similarly, the smallest $\nu$ satisfying (75) is $\nu =e(\lambda )$ and, in this case and only then,

$$\begin{aligned} (s(\lambda )^\prime \cup (m^n)) + \nu =\lambda \ \ \text { for } \nu =e(\lambda ). \end{aligned}$$

Thus, the $\nu$-sum in the last expression for $\langle P_{\lambda /\mu _\text {min}^\prime },e_n^mP_{\lambda /\mu _\text {max}^\prime }\rangle _{n;q,t}^\prime$ above has only a single non-zero term, namely $\nu =e(\lambda )$, and therefore

$$\begin{aligned} \langle P_{\lambda /\mu _\text {min}^\prime },e_n^mP_{\lambda /\mu _\text {max}\prime }\rangle _{n;q,t}^\prime = f^{\lambda ^\prime }_{s(\lambda ),(e(\lambda )+(m^n))^\prime }(t,q) f^{\lambda ^\prime }_{s(\lambda )+(n^m),e(\lambda )^\prime }(t,q) N_n(e(\lambda );q,t) . \end{aligned}$$

From Lemma 1 we get

$$\begin{aligned} f^{\lambda ^\prime }_{s(\lambda ),(e(\lambda )+(m^n))^\prime }(q,t)=1,\ \ \ f^{\lambda ^\prime }_{s(\lambda )+(n^m),e(\lambda )^\prime }(t,q) =\frac{b_{s(\lambda )'\cup (m^n)}(q,t)b_{e(\lambda )}(q,t)}{b_\lambda (q,t)}. \end{aligned}$$

By inserting these results in (73) we obtain

$$\begin{aligned} \begin{aligned} N_{n,m}(\lambda ;q,t) = (t/q)^{|s(\lambda )|} \frac{b_{s(\lambda )'\cup (m^n)}(q,t)b_{e(\lambda )}(q,t)}{b_\lambda (q,t)}N_n(e(\lambda );q,t) \\ \cdot b_{s(\lambda )}(t,q) b_{s(\lambda )+(n^m)}(t,q) N_m(s(\lambda );t,q), \end{aligned} \end{aligned}$$

and since

$$\begin{aligned} b_{s(\lambda )'\cup (m^n)}(q,t) = b_{(s(\lambda )+(m^n))^\prime }(q,t) = \frac{1}{b_{s(\lambda )+(m^n)}(t,q)} \end{aligned}$$

by (18), we arrive at the result in (48). $\quad \square$

4.2.2 Part (a)

Since the the super-Macdonald polynomials $SP_\lambda ((x_1,\ldots ,x_n), (y_1,\ldots ,y_m);q,t)$, $\lambda \in H_{n,m}$, form a basis in the algebra $\Lambda _{n,m;q,t}$ and (47)–(48) clearly imply that the (quadratic) norms $N_{n,m}(\lambda ;q,t)$ are real, it follows from Part (b) that our sesquilinear form $\langle \cdot ,\cdot \rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime )$ (58) is Hermitian, i.e.

$$\begin{aligned} \langle P,Q\rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime ) = \overline{\langle Q,P\rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime )},\ \ \ (P,Q\in \Lambda _{n,m;q,t}). \end{aligned}$$

By invoking Lemma 4, we can thus infer

$$\begin{aligned} \langle P,Q\rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime ) = \overline{\langle Q,P\rangle _{n,m;q,t}^\prime (1/\xi ,1/\xi ^\prime )} = \langle P,Q\rangle _{n,m;q,t}^\prime (1/\xi ,1/\xi ^\prime ). \end{aligned}$$

Due to the fact that $\xi /\xi ^\prime <1/M(q,t)$ if and only if $(1/\xi )/(1/\xi ^\prime )>M(q,t)$, Lemma 5 implies that $\langle \cdot ,\cdot \rangle _{n,m;q,t}^\prime (\xi ,\xi ^\prime )$ is independent of $\xi ,\xi ^\prime$ as long as (59) (or equivalently (43)) is satisfied. This concludes the proof of Part (a) and hence the theorem.

5 Conclusions and Outlook

We introduced a Hermitian product $\langle \cdot ,\cdot \rangle ^\prime _{n,m;q,t}$ on the algebra $\Lambda _{n,m;q,t}$, in which the super-Macdonald polynomials constitute an orthogonal basis (cf. [SV09a, Theorem 5.6] and Theorem 1), and we proved that this product endows the factor space $V_{n,m;q,t}=\Lambda _{n,m;q,t}/K_{n,m;q,t}$, where $K_{n,m;q,t}$ denotes the kernel of $\langle \cdot ,\cdot \rangle ^\prime _{n,m;q,t}$, with a Hilbert space structure.

Furthermore, we argued that these results provides the means for a quantum mechanical interpretation of the model defined by the deformed Macdonald operators ${\mathcal M}_{n,m;q^\pm ,t^\pm }$ or, equivalently, $R^\mp _{n,m;\beta ,\gamma }$, cf. (1)–(2) or (5556)–(5556), respectively. This model describes two kinds of particles, and we proposed that they represent particles and anti-particles in an underlying relativistic quantum field theory, which is the same theory that inspired the Ruijsenaars models [RS86, Rui01].

5.1 Physical interpretation

We stress that the results obtained in the present paper only constitute a first step towards a fully satisfactory physical interpretation of the deformed trigonometric Ruijsenaars model, and much work remains to be done.

In the non-relativistic limit $q\rightarrow 1$, the trigonometric Ruijsenaars model reduces to the trigonometric Calogero–Moser–Sutherland model and, in this case, a quantum field theory formulation is known, which naturally includes the deformed models [AL17]. Moreover, parts of this construction were extended recently to the elliptic case [BLL20, Section III.A]. Our results in this paper suggest that these quantum field theory results can be generalised to the Ruijsenaars case. A natural starting point would be a well-established quantum field theory description of the trigonometric Ruijsenaars model [SKAO92], which allows for an elliptic generalisation [FHHSY09].

Another important question concerns the completeness of the wave functions given by the super Macdonald polynomials. In the undeformed case, the physically natural Hilbert space is spanned by Laurent polynomials of the form

$$\begin{aligned} (x_1\cdots x_n)^k P_\lambda ((x_1,\ldots ,x_n)),\ \ \ k\in {\mathbb Z}. \end{aligned}$$

(In this way, one includes both positive and negative momentum quantum numbers.) Similarly, it is readily seen that

$$\begin{aligned} (x_1\cdots x_n)^k(y_1\cdots y_m)^l SP_\lambda ((x_1,\ldots ,x_n),(y_1,\ldots ,y_m)) \end{aligned}$$

are eigenfunctions of the deformed Macdonald–Ruijsenaars operators ${\mathcal M}_{n,m;q^{\pm 1},t^{\pm 1}}$ whenever

$$\begin{aligned} \frac{k}{l} = -\frac{\log t}{\log q}. \end{aligned}$$

At present, it is unclear what physical significance these further eigenfunctions have, but we suspect that the solution of this conundrum can be achieved through the quantum field theoretic interpretation indicated above.

5.2 Elliptic and hyperbolic generalisations

We proceed to highlight two possible directions for future research, which, in particular, are interesting from the quantum field theory point of view.

An elliptic generalisation of our results is one of these directions. We recall that the elliptic generalisation of the deformed Macdonald–Ruijsenaars operator $M_{n,m;q,t}$ (54) is known from [AHL14]. Specifically, rewriting the additive difference operator in Eq. (70) in multiplicative form, we obtain

$$\begin{aligned} M_{n,m;p,q,t} = \frac{t^{1-n}q^m}{\theta (q;p)} \sum _{i=1}^n A_iT_{q,x_i} + \frac{t^{-n}q^{m-1}}{\theta (t^{-1};p)} \sum _{j=1}^m B_jT_{t^{-1},y_j}, \end{aligned}$$

(76)

with the elliptic deformation parameter p in the range $0\le p<1$ and coefficients

$$\begin{aligned} \begin{aligned} A_i&= \prod _{i^\prime \ne i}^n \frac{\theta (tx_i/x_{i^\prime };p)}{\theta (x_i/x_{i^\prime };p)}\cdot \prod _{j=1}^m \frac{\theta (t^{1/2} x_i/q^{1/2} y_j;p)}{\theta (t^{1/2}q^{1/2} x_i/y_j;p)},\\ B_j&= \prod _{j^\prime \ne j}^m \frac{\theta (q^{-1}y_j/y_{j^\prime };p)}{\theta (y_j/y_{j^\prime };p)}\cdot \prod _{i=1}^n \frac{\theta (q^{-1/2}t^{1/2} y_j/ x_i;p)}{\theta (q^{-1/2} y_j/t^{1/2} x_i;p)}, \end{aligned} \end{aligned}$$

(77)

where $\theta (z;p) : = (z;p)_\infty (p/z;p)_\infty$. We note that $M_{n,m;p,q,t}$ reduces, up to an additive constant, to the deformed Macdonald–Ruijsenaars operator $M_{n,m;q,t}$ in the trigonometric limit $p\rightarrow 0$.

Moreover, the results in [AHL14] suggest that a natural elliptic generalisation of our Hermitian product is as in (6), but with the weight function

$$\begin{aligned} \Delta _{n,m}(x,y;p,q,t) = \frac{\Delta _n(x;p,q,t)\Delta _m(y;p,t,q)}{\prod _{i=1}^n\prod _{j=1}^m\theta (q^{-1/2}t^{1/2}x_i/y_j;p)\theta (q^{-1/2}t^{1/2}y_j/x_i;p)}, \end{aligned}$$

$$\begin{aligned} \Delta _n(x;p,q,t) = \prod _{1\le i\ne j\le n}\frac{\Gamma (tx_i/x_j;p,q)}{\Gamma (x_i/x_j;p,q)}, \end{aligned}$$

where $\Gamma (z;p,q):= \prod _{k=0}^\infty (p^{k+1}q/z;q)_\infty /(p^k z;q)_\infty$ is the elliptic Gamma function. We note that, in the special case $(n,m)=(n,0)$, this reduces to the operator and weight function of the elliptic Ruijsenaars model (see e.g. [Has97, Section 5]).

However, at this point, very little is known about the eigenfunctions of the deformed elliptic Macdonald–Ruijsenaars operator $M_{n,m;p,q,t}$. In fact, even in the ordinary $m=0$ case, the understanding of these eigenfunctions is still far from complete; see, however, [Shi19, LNS20] for recent progress in this direction. Our results provide further motivation for any attempt at developing a theory of eigenfunctions at the deformed elliptic level, and generalising the results in [Shi19, LNS20] to the deformed case could be an interesting starting point.

It is interesting to note that, while the deformed elliptic Calogero-Moser-Sutherland system was introduced already in 2004 by Sergeev and Veselov [SV04], this very model was re-discovered in a systematic search for kernel functions of such system in 2010 [Lan10] and, more recently, in the context of super-symmetric gauge theories [Nek17, CKL20]. It would be interesting to explore possible relativistic generalisations of the latter results.

The hyperbolic case is another compelling direction for future research. In the recent paper [HR21], Ruijsenaars and one of the authors proved that the particles in the hyperbolic version of the ordinary Ruijsenaars system exhibit soliton scattering (conservation of momenta and factorisation of the S-matrix) and reproduce the sine-Gordon soliton scattering for suitable values of the coupling parameter. This confirmed a long-standing conjecture due to Ruijsenaars. To extend these results to the deformed case, and explore their potential physical implications, strikes us as an intriguing and challenging problem.

5.3 An integrable elliptic model of four types of particles

We should mention the well-known fact that the hyperbolic undeformed ($m = 0$) Ruijsenaars model can accommodate a second kind of particles, obtained from the first kind by shifting the positions by an imaginary half-period, see e.g. [Rui94]. However, the nature of this model of two types of particles is rather different from the deformed hyperbolic model; in fact, the latter model has a deformation describing four different types of particles.

To be specific, we present the operator defining this model of four particle types in the general elliptic case (the corresponding hyperbolic model can be obtained from this in a suitable limit): we start from the operator given by (76)–(77), choose arbitrary particle numbers $n_1$, $n_2$, $m_1$, $m_2$ so that $n=n_1+n_2$, $m=m_1+m_2$, and re-write the operator in terms of variables $x_i$ ($i=1,\ldots ,n_1$), $\tilde{x}_i$ ($i=1,\ldots ,n_2$), $y_j$ ($j=1,\ldots ,m_1$), and $\tilde{y}_j$ ($j=1,\ldots ,m_2$), where the variable sets marked by tildes are obtained by half-period shifts from the ones without tildes as follows, so that

$$\begin{aligned} \tilde{x}_{i} := p^{1/2}x_{i+n_1} \quad (i= 1,\ldots ,n_2),\quad \tilde{y}_{j} := p^{1/2}y_{j+m_1} \quad (j= 1,\ldots ,m_2). \end{aligned}$$

By straightforward computations, this yields the following operator describing a system of four distinct types of non-distinguishable particles:

$$\begin{aligned} \begin{aligned} M_{n_1,n_2,m_1,m_2;p,q,t} =&\frac{t^{1-n_1-n_2}q^{m_1+m_2}}{\theta (q;p)} \sum _{i=1}^{n_1} A_iT_{q,x_i} + \frac{t^{1-n_2}q^{m_2}}{\theta (q;p)} \sum _{i=1}^{n_2} \tilde{A}_iT_{q,\tilde{x}_i} \\&+ \frac{t^{-n_1-n_2}q^{m_1+m_2-1}}{\theta (t^{-1};p)} \sum _{j=1}^{m_1} B_jT_{t^{-1},y_j} + \frac{t^{-n_2}q^{m_2-1}}{\theta (t^{-1};p)} \sum _{j=1}^{m_2} \tilde{B}_jT_{t^{-1},\tilde{y}_j} \end{aligned} \end{aligned}$$

with coefficients

$$\begin{aligned} \begin{aligned} A_i =&\prod _{i^\prime \ne i}^{n_1} \frac{\theta (tx_i/x_{i^\prime };p)}{\theta (x_i/x_{i^\prime };p)} \cdot \prod _{i^\prime =1}^{n_2} \frac{\tilde{\theta }(tx_i/\tilde{x}_{i^\prime };p)}{\tilde{\theta }(x_i/\tilde{x}_{i^\prime };p)}\\&\cdot \prod _{j=1}^{m_1} \frac{\theta (t^{1/2} x_i/q^{1/2} y_j;p)}{\theta (t^{1/2} x_i/q^{-1/2}y_j;p)} \cdot \prod _{j=1}^{m_2} \frac{\tilde{\theta }(t^{1/2} x_i/q^{1/2} \tilde{y}_j;p)}{\tilde{\theta }(t^{1/2} x_i/q^{-1/2}\tilde{y}_j;p)} ,\\ \tilde{A}_i =&\prod _{i^\prime =1}^{n_1} \frac{\tilde{\theta }(t\tilde{x}_i/x_{i^\prime };p)}{\tilde{\theta }(\tilde{x}_i/x_{i^\prime };p)}\cdot \prod _{i^\prime \ne i}^{n_2} \frac{\theta (t\tilde{x}_i/\tilde{x}_{i^\prime };p)}{\theta (\tilde{x}_i/\tilde{x}_{i^\prime };p)}\\&\cdot \prod _{j=1}^{m_1} \frac{\tilde{\theta }(t^{1/2} \tilde{x}_i/q^{1/2} y_j;p)}{\tilde{\theta }(t^{1/2} \tilde{x}_i/q^{-1/2}y_j;p)} \cdot \prod _{j=1}^{m_2} \frac{\theta (t^{1/2} \tilde{x}_i/q^{1/2} \tilde{y}_j;p)}{\theta (t^{1/2} \tilde{x}_i/q^{-1/2}\tilde{y}_j;p)},\\ B_j =&\prod _{i=1}^{n_1} \frac{\theta (ty_j/x_{i};p)}{\theta (y_j/x_{i};p)} \cdot \prod _{i=1}^{n_2} \frac{\tilde{\theta }(ty_j/\tilde{x}_{i};p)}{\tilde{\theta }(y_j/\tilde{x}_{i};p)}\\&\cdot \prod _{j^\prime \ne j}^{m_1} \frac{\theta (t^{1/2} y_j/q^{1/2} y_{j^\prime };p)}{\theta (t^{1/2} y_j/q^{-1/2}y_{j^\prime };p)} \cdot \prod _{j^\prime =1}^{m_2} \frac{\tilde{\theta }(t^{1/2} y_j/q^{1/2} \tilde{y}_{j^\prime };p)}{\tilde{\theta }(t^{1/2} y_j/q^{-1/2}\tilde{y}_{j^\prime };p)} ,\\ \tilde{B}_j =&\prod _{i =1}^{n_1} \frac{\tilde{\theta }(t\tilde{y}_j/x_{i};p)}{\tilde{\theta }(\tilde{y}_j/x_{i};p)}\cdot \prod _{i =1}^{n_2} \frac{\theta (t\tilde{y}_j/\tilde{x}_{i};p)}{\theta (\tilde{y}_j/\tilde{x}_{i};p)}\\&\cdot \prod _{j^\prime =1}^{m_1} \frac{\tilde{\theta }(t^{1/2} \tilde{y}_j/q^{1/2} y_{j^\prime };p)}{\tilde{\theta }(t^{1/2} \tilde{y}_j/q^{-1/2}y_{j^\prime };p)} \cdot \prod _{j^\prime \ne j}^{m_2} \frac{\theta (t^{1/2} \tilde{y}_j/q^{1/2} \tilde{y}_{j^\prime };p)}{\theta (t^{1/2} \tilde{y}_j/q^{-1/2}\tilde{y}_{j^\prime };p)},\\ \end{aligned} \end{aligned}$$

where $\tilde{\theta }(x;p):= \theta (p^{1/2}x;p)$ (and we used $\theta (p^{-1/2}x;p)=-p^{-1/2}x\tilde{\theta }(x;p)$).

Recent results in [BLL20] on the non-relativistic limit of this model suggest the following interpretation: variables without and with tildes describe particles of chirality $+$ and −, respectively; particles of the same chirality interact via the function $\theta (x;p)$ which remains non-trivial in the trigonometric limit, $\lim _{p\rightarrow 0}\theta (x;p)=1-x$, whereas particles of different chirality interact with the function $\tilde{\theta }(x;p)= (p^{1/2}x;p)(p^{1/2}/x;p)$ becoming trivial, $\lim _{p\rightarrow 0}\tilde{\theta }(x;p)=1$. Thus, while all four particle types have non-trivial interactions in the general case $p\ne 0$, one obtains two decoupled system of the kind studied in the present paper in the trigonometric limit $p\rightarrow 0$.

We mention in passing that recent results in [HLNR21b] imply that this four-particle-type model defines a quantum integrable system.

5.4 Complex parameter values

Finally, we note that the results in Eqs. (46)–(48) remain true even for complex q and t such that $0<|q|<1$ and $0<|t|<1$, provided that the definition of $Q^*(x,y)$ in (44) is changed from (41) to

$$\begin{aligned} Q^*(x,y) = Q(x^{-1},y^{-1}). \end{aligned}$$

(78)

However, then $\langle \cdot ,\cdot \rangle ^\prime _{n,m;q,t}$ is no longer sesquilinear and (in general) not positive (semi)definite, and consequently does not provide $V_{n,m;q,t}$ with a Hilbert space structure.

Appendices

Conventions used by Sergeev and Veselov

Here we explain the relation between the conventions for the super-Macdonald polynomials used in this paper, and the ones used by Sergeev and Veselov (SV) [SV09a]. As will be made clear, it is easy to translate from one convention to the other. Moreover, both conventions have their advantages and disadvantages. More specificially, the advantages of our conventions are that the super-Macdonald polynomials are manifestly invariant under $(q,t)\rightarrow (q^{-1},t^{-1})$, and that Hilbert space adjungation agrees with what one would naively expect; cf. (37) vs. (81) and (41) vs. (83). The advantage of the SV-conventions is that factors $t^{\pm 1/2}$ and $q^{\pm 1/2}$ are avoided, and that some formulas look somewhat more symmetric; cf. (2) vs. (85), (30) vs. (86), and (39) vs. (87).

The deformed power sums used in [SV09a] are

$$\begin{aligned} p^{(\mathrm {SV})}_{r}(x,y;q,t)= \sum _{i=1}^n x_i^r +\frac{1-q^r}{1-t^{-r}} \sum _{j=1}^m y_j^r \quad (r\in {\mathbb Z}_{\ge 1}); \end{aligned}$$

(79)

note that our $t^{-1}$ corresponds to t in [SV09a]. The algebra endomorphism $\varphi ^{(\mathrm {SV})}_{n,m}$ defining the super-Macdonald polynomials is defined exactly as in (32)–(34) but with the deformed power sums in (79). Clearly, the deformed power sums in (79) are obtained from ours in (31) by the transformations $(x,y)\rightarrow (x,q^{1/2}t^{1/2}y)$, and this implies

$$\begin{aligned} SP^{(\mathrm {SV})}_\lambda (x,y;q,t) = SP_\lambda (x;q^{1/2}t^{1/2}y;q,t). \end{aligned}$$

(80)

      We recall that the coefficents $c_{\lambda \mu }(q,t)$ in (33) are invariant under the transformation $(q,t)\rightarrow (q^{-1},t^{-1})$; see (16). However, the arguments (x, y) of $p^{(\mathrm {SV})}_{r}(x,y;q,t)$ transform under this transformation to $(x,q^{-1}t^{-1}y)$. Thus,

$$\begin{aligned} SP^{(\mathrm {SV})}_\lambda (x,y;q^{-1},t^{-1}) = SP^{(\mathrm {SV})}_\lambda (x;q^{-1}t^{-1}y;q,t). \end{aligned}$$

(81)

      Our scalar product in the SV-conventions can be written as in (44) but with a slightly altered weight function, another definition of conjugation, and different constraints on the radii $\xi ,\xi ^\prime >0$:

$$\begin{aligned} \Delta ^{(\mathrm {SV})}_{n,m}(x,y;q,t) = \frac{\Delta _{n}(x;q,t) \Delta _{m}(y;t,q)}{\prod _{i=1}^n\prod _{j=1}^m (1-q^{-1}x_i/y_j)(1-ty_j/x_i)} \end{aligned}$$

(82)

with $\Delta _{n}(x;q,t)$ in (5),

$$\begin{aligned} Q^*(x,y) \equiv \overline{Q(\bar{x}^{-1},q^{-1}t^{-1}\bar{y}^{-1})} , \end{aligned}$$

(83)

and

$$\begin{aligned} \xi /\xi ^\prime < \min (q,t)\ \ \ \text { or } \ \ \ \xi /\xi ^\prime >\max (q,t) . \end{aligned}$$

(84)

      For the convenience of the reader, we also give other important formulas in the SV-conventions. First, formulas for the coefficients in (2) defining the deformed Macdonald operators in (1):

$$\begin{aligned} \begin{aligned} A^{(\mathrm {SV})}_i&= t^{n-1}\prod _{i^\prime \ne i}^n \frac{x_i - t^{-1}x_{i^\prime }}{x_i- x_{i^\prime }}\cdot \prod _{j=1}^m \frac{x_i - qy_j}{x_i - y_j}, \\ B^{(\mathrm {SV})}_j&= q^{1-m}\prod _{j^\prime \ne j}^m \frac{y_j - qy_{j^\prime }}{y_j - y_{j^\prime }}\cdot \prod _{i=1}^n \frac{y_j - t^{-1} x_i}{y_j - x_i}. \end{aligned} \end{aligned}$$

(85)

Second, the symmetry conditions in (30) that characterise the algebra $\Lambda ^{(\mathrm {SV})}_{n,m;q,t}$ spanned by the super-Macdonald polynomials:

$$\begin{aligned} \left( T_{q,x_i}-T_{t^{-1},y_j} \right) P(x,y) =0\ \ \text { at } \ \ x_i=y_j \ \ \ (\forall i,j). \end{aligned}$$

(86)

Third, the representation of the super-Jack polynomials in (39):

$$\begin{aligned} SP^{(\mathrm {SV})}_\lambda (x,y;q,t) = \sum _\mu (-t)^{|\mu |} P_{\lambda /\mu ^\prime }(x;q,t)Q_\mu (y;t,q). \end{aligned}$$

(87)

Relativistic invariance of deformed Ruijsenaars model

In this appendix, we show that the deformed Ruijsenaars model, as defined by the operators $R^\pm _{n,m;\beta ,\gamma }$ from (5556)–(5556), or equivalently $M_{n,m,q^{\pm 1},t^{\pm 1}}$, given by (54), is relativistically invariant.

      As explained by Ruijsenaars in his pioneering paper [Rui87, Eqs. (2.17)–(2.18)], the Ruijsenaars models are relativistically invariant in the sense that his difference operators $\hat{S}_{\pm 1}$ (cf. (57)) provide a representation of the Lie algebra of the Poincaré group in 1+1 spacetime dimensions. Specifically, the operators $\hat{H}=\frac{1}{2}(\hat{S}_{1}+\hat{S}_{-1})$ (Hamiltonian), $\hat{P}=\frac{1}{2}(\hat{S}_{1}-\hat{S}_{-1})$ (momentum operator) and $\hat{B}=-\sum _{i=1}^n u_i/\beta$ (boost operator) satisfy the commutation relations

$$\begin{aligned}{}[\hat{H},\hat{P}]= 0,\quad [\hat{H},\hat{B}]=\mathrm {i}\hat{P},\quad [\hat{P},\hat{B}] = \mathrm {i}\hat{H}, \end{aligned}$$

(88)

(where, for simplicity, we use units such that $m=c=1$).

      This result straightforwardly generalises to the deformed case: The operators

$$\begin{aligned} \begin{aligned} \hat{H} =\,&\frac{1}{2}\big (R^+_{n,m;\beta ,\gamma }-R^-_{n,m;\beta ,\gamma }\big ), \\ \hat{P} =\,&\frac{1}{2}\big (R^+_{n,m;\beta ,\gamma }+R^-_{n,m;\beta ,\gamma }\big ), \\ \hat{B} =\,&-\sum _{i=1}^n\frac{u_i}{\beta } + \sum _{j=1}^m\frac{v_j}{\gamma }, \end{aligned} \end{aligned}$$

satisfy the commutation relations in (88). Note the different signs in $\hat{H}$ and $\hat{P}$, as compared to Ruijsenaars’s expressions in the undeformed case, which can be understood from the presence of the factor $\pm 1/2\sinh (\pi \beta /L)$ in (57).

   To verify this claim, we observe that the operators $M_{n,m;q,t}$ and $M_{n,m;q^{-1},t^{-1}}$ commute on the space $\Lambda _{n,m;q,t}$, which is spanned by the super-Macdonald polynomials.⁶ By (5556)–(5556), this observation implies the first relation in (88). (From [HLNR21a, Lemma 3.1] follows that commutativity as operators on $\Lambda _{n,m;q,t}$ implies commutativity as difference operators.)

   The second and the third relations in (88) are equivalent to

$$\begin{aligned}{}[R^\pm _{n,m;\beta ,\gamma },\hat{B}] = \pm \mathrm {i}R^\pm _{n,m;\beta ,\gamma }, \end{aligned}$$

which is easy to check using the following elementary commutation relations:

$$\begin{aligned} \Big [\mathrm {e}^{\pm \mathrm {i}\beta \partial _{u_i}},u_{i^\prime }\Big ]= \pm \mathrm {i}\beta \delta _{ii^\prime } \mathrm {e}^{\pm \mathrm {i}\beta \partial _{u_i}}, \quad \Big [\mathrm {e}^{\mp \mathrm {i}\beta \partial _{v_j}},v_{j^\prime }\Big ]= \mp \mathrm {i}\gamma \delta _{jj^\prime } \mathrm {e}^{\pm \mathrm {i}\gamma \partial _{v_j}}, \end{aligned}$$

where $\delta _{ii^\prime }$ denotes the Kronecker delta.

Proof details

1.1 Proof of Lemma 1

Remark 4

We adapt a proof in the Jack polynomial case [Sta89, 4.1 Proposition].

The monomial functions in (9) satisfy

$$\begin{aligned} m_\mu m_\nu = m_{\mu +\nu } + \text {lower order terms} \end{aligned}$$

for all partitions $\mu ,\nu$, with “lower order terms” standing for a linear combination of $m_\lambda$ with $\lambda <\mu +\nu$. This and the definition of Macdonald functions $P_\lambda$ (triangular structure) imply

$$\begin{aligned} P_\mu P_\nu = P_{\mu +\nu } + \text {lower order terms} . \end{aligned}$$

On the other hand, by definition [Mac95, Eq. (7.1’)],

$$\begin{aligned} P_\mu (x;q,t) P_\nu (x;q,t) = \sum _{\lambda } f^{\lambda }_{\mu \nu }(q,t) P_\lambda (x;q,t). \end{aligned}$$

(89)

Thus, by comparison,

$$\begin{aligned} f^\lambda _{\mu \nu }(q,t)\equiv 0\ \ \text { unless } \ \ \lambda \le \mu +\nu , \ \ \ f^{\mu +\nu }_{\mu \nu }(q,t)=1. \end{aligned}$$

(90)

Substituting $(\lambda ,\mu ,\nu ,q,t)\rightarrow (\lambda ^\prime ,\mu ^\prime ,\nu ^\prime ,t,q)$, and using that $f^\lambda _{\mu \nu }$ is non-zero only if $|\mu |+|\nu |=|\lambda |$, the latter is equivalent to

$$\begin{aligned} f^{\lambda ^\prime }_{\mu ^\prime \nu ^\prime }(t,q)\equiv 0\ \ \text { unless } \ \ \mu \cup \nu \le \lambda , \ \ \ f^{(\mu \cup \nu )^\prime }_{\mu ^\prime \nu ^\prime }(t,q)=1 \end{aligned}$$

(since $\lambda ^\prime \le \mu ^\prime$ is equivalent to $\mu \le \lambda$ provided $|\mu |=|\lambda |$, and $\mu ^\prime +\nu ^\prime =(\mu \cup \nu )^\prime$ [Mac95]). This proves the first half of the result. The second half is obtained from (90) using the formula

$$\begin{aligned} f^{\lambda ^\prime }_{\mu ^\prime \nu ^\prime }(t,q)=\frac{b_\mu (q,t)b_\nu (q,t)}{b_{\lambda }(q,t)} f^\lambda _{\mu \nu }(q,t) \end{aligned}$$

following from (89) by applying $\omega _{q,t}$ in (19), renaming $(\lambda ,\mu ,\nu ,q,t)\rightarrow (\lambda ^\prime ,\mu ^\prime ,\nu ^\prime ,t,q)$, and using (17) and (20). $\quad \square$

1.2 Detail in the proof of Lemma 2

For the convenience of the reader, we provide a self-contained proof of the fact that $P_{\lambda /\mu ^\prime }((x_1,\ldots ,x_n);q,t)\equiv 0$ if $\lambda ^\prime _j-\mu _j>n$ for some $j\ge 1$; cf. (7.15) in [Mac95, Section VI].

By definition,

$$\begin{aligned} P_{\lambda /\mu ^\prime }((x_1,\ldots ,x_n);q,t) = \sum _{\nu } f^{\lambda ^\prime }_{\mu \nu ^\prime }(t,q) P_{\nu }((x_1,\ldots ,x_n);q,t) \end{aligned}$$

where the sum on the right-hand side is only over partitions $\nu$ of length less or equal to n (since $P_{\nu }((x_1,\ldots ,x_n);q,t)\equiv 0$ otherwise), i.e., all partitions $\nu$ contributing to this sum satisfy

$$\begin{aligned} \nu _j^\prime \le \nu _1^\prime =\ell (\nu )\le n \end{aligned}$$

(91)

for all $j=1,2,\ldots$.

   By Lemma 1, the coefficients $f^{\lambda ^\prime }_{\mu \nu ^\prime }(t,q)$ are non-zero only if $\mu ^\prime \cup \nu \le \lambda$, equivalent to $\lambda ^\prime \le \mu +\nu ^\prime$, i.e.,

$$\begin{aligned} \lambda ^\prime _j-\mu _j\le \nu ^\prime _j \end{aligned}$$

for all $j=1,2,\ldots$. This implies the result. $\quad \square$

1.3 Proof of (63)

We consider the complex function

$$\begin{aligned} f(z) := \prod _{i=1}^n \frac{z - t^{-1/2} x_i}{z-t^{1/2} x_i} \cdot \prod _{j=1}^m \frac{z - q^{1/2} y_j}{z-q^{-1/2} y_j}, \end{aligned}$$

assuming fixed generic values for $x_i$ and $y_j$ (so that all poles of f(z) are of order 1), and compute

$$\begin{aligned} \lim _{\xi \rightarrow \infty }\oint _{|z| = \xi } \frac{dz}{2\pi \mathrm {i}z}f(z) \end{aligned}$$

in two ways: first, using that $f(z)=1+O(1/z)$ as $|z|\rightarrow \infty$, which gives 1; second, invoking the residue theorem (for sufficiently large $\xi$), which gives the sum of all residues. This yields the identity

$$\begin{aligned}&1 = \sum _{i=1}^n \lim _{z\rightarrow t^{1/2}x_i} (z-t^{1/2}x_i) \frac{f(z)}{z} + \sum _{j=1}^m \lim _{z\rightarrow q^{-1/2}y_j} (z-q^{-1/2}y_j)\frac{f(z)}{z} + \lim _{z\rightarrow 0}f(z)\\&\quad = \sum _{i=1}^n (1-t^{-1})t^{1-n} \prod _{i'\ne i}^n \frac{tx_i - x_{i'}}{x_i-x_{i'}} \prod _{j=1}^m \frac{t^{1/2} x_i - q^{1/2} y_j}{t^{1/2} x_i-q^{-1/2} y_j} \\&\qquad + \sum _{j=1}^m (1-q)q^{m-1}\prod _{i=1}^n \frac{q^{-1/2} y_j - t^{-1/2} x_i}{q^{-1/2} y_j-t^{1/2} x_i} \prod _{j'\ne j}^m \frac{q^{-1} y_j - y_{j'}}{y_j -y_{j'}} + t^{-n}q^m, \end{aligned}$$

which clearly is equivalent to the identity (63).

   While we assumed generic $x_i$ and $y_j$ in this argument, it is clear by continuity that the result holds true for arbitrary complex $x_i$ and $y_j$. $\quad \square$

1.4 Proof of (67)

We give a detailed proof of the identity in (67) assuming (64) and (68). We recall that $i\in \{1,\ldots ,n\}$ is fixed.

   As explained in the main text, we only need to show that the common integrand of the two integrals in (67) is an analytic function of the complex variable $x_i$ in the region $\xi \le |x_i|\le \xi /q$, provided the other variables are fixed as in (68) and (64) holds true. Since $P(x^{-1},y^{-1}) T_{q,x_i} \overline{Q}(x,y)$ is an analytic function of (x, y) in $(\mathbb {C}^*)^n\times (\mathbb {C}^*)^m$, we only need to investigate the function $W_i^*(x,y) T_{q,x_i}W_i(x,y)$, which is equal to

$$\begin{aligned} \prod _{i'\ne i}^n \frac{(x_{i'}/x_i;q)_\infty }{(tx_{i'}/x_i;q)_\infty } \frac{(qx_i/x_{i'};q)_\infty }{(tqx_i/x_{i'};q)_\infty } \cdot \prod _{j=1}^m \frac{1}{(1-q^{-1/2}t^{1/2}y_j/x_i)}\frac{1}{(1-q^{1/2}t^{1/2}x_i/y_j)}, \end{aligned}$$

cf. (65). We verify that no poles of the four types of factors in the latter expression are located in the pertinent $x_i$-region that we parametrise as follows: $|x_i|=\xi a/q$ with $q\le a\le 1$.

   The poles of the first type of factors are only encountered when $|q^k tx_{i'}/x_i| = q^k tq/a=1$ for $k\in {\mathbb Z}_{\ge 0}$, i.e., $a=q^{k+1}t<q$; there are no such poles for $q\le a\le 1$. The poles of the second type of factors are all located in the subsets $|q^k tqx_i/x_{i'}|=q^k t a=1$ for $k\in {\mathbb Z}_{\ge 0}$, i.e., $a=1/q^k t>1$; and again there are no such poles for $q\le a\le 1$. The poles of the third type of factors only occur for $|q^{-1/2}t^{1/2}y_j/x_i| = q^{1/2}t^{1/2}\xi ^\prime /\xi a=1$, i.e., for $\xi /\xi ^\prime =q^{1/2}t^{1/2}/a$; if $q\le a\le 1$, these poles occur for $q^{1/2}t^{1/2}\le \xi /\xi ^\prime \le q^{-1/2}t^{1/2}$, and there are no such poles if (64) holds true. Finally, the fourth type only have poles in the subsets $|q^{1/2}t^{1/2}x_i/y_j|=q^{-1/2}t^{1/2}a \xi /\xi ^\prime =1$, i.e., for $\xi /\xi ^\prime = q^{1/2}t^{-1/2}/a$; if $q\le a\le 1$, these poles occur for $q^{1/2}t^{-1/2}\le \xi /\xi ^\prime \le q^{-1/2}t^{-1/2}$, and again there are no such poles if (64) holds true. $\quad \square$

The case n = m =1

In this appendix, we consider the special case $n=m=1$, where we can verify by simple direct computations, that the sesquilinear form of Definition 1 is independent of the integration radii $\xi ,\xi ^\prime >0$ as long as $\xi /\xi ^\prime >M(q,t):=\max _{\delta =\pm 1}\big (q^{\frac{\delta }{2}}t^{-\frac{\delta }{2}}\big )$ or $\xi /\xi ^\prime <\min _{\delta =\pm 1}\big (q^{\frac{\delta }{2}}t^{-\frac{\delta }{2}}\big )=1/M(q,t)$.

   In this case, the conditions (29)–(30) on elements in $\Lambda _{1,1;q,t}$, $P=P(x,y)$ with $(x,y)\in {\mathbb C}\times {\mathbb C}$, reduce to the symmetry condition (30) for $i=j=1$, i.e.,

$$\begin{aligned} \big (T_{q,x}-T_{t^{-1},y}\big )P(x,y) = P(qx,y)-P(x,t^{-1}y)=0\ \text { at } \ q^{1/2}x = t^{-1/2}y; \end{aligned}$$

this can be written as

$$\begin{aligned} P(q^{1/2}x,t^{1/2}x)- P(q^{-1/2}x,t^{-1/2}x)= 0 \end{aligned}$$

(92)

(we inserted $y=q^{1/2}t^{1/2}x$ and renamed $q^{1/2}x\rightarrow x$). Moreover, the right-hand side of (44) is of the form

$$\begin{aligned} I(\xi ,\xi ^\prime ) := \oint _{|x|=\xi }\frac{dx}{2\pi \mathrm {i}x}\oint _{|y|=\xi ^\prime }\frac{dy}{2\pi \mathrm {i}y}\frac{f(x,y)}{(1-q^{-1/2}t^{1/2}x/y)(1-q^{-1/2}t^{1/2}y/x)}, \end{aligned}$$

where $\xi ,\xi ^\prime >0$ satisfy (43) and $f=PQ^*$ has the symmetry property (92). Clearly, by Cauchy’s integral theorem, $I(\xi ,\xi ^\prime )$ is unchanged by continuous deformations of the integration contours as long as the singularities of the integrand are avoided; this is the case if (43) holds. Clearly, it suffices to show that $I(\xi ,\xi ^\prime )=I(\xi ^\prime ,\xi )$. For simplicity, we restrict attention to $t>q$, so that $M(q,t)=q^{-1/2}t^{1/2}$ and $1/M(q,t)=q^{1/2}t^{-1/2}$.

Deforming the y-contour to $|y|=\xi$, we pick up a residue at the simple pole $y=q^{\frac{1}{2}}t^{-\frac{1}{2}}x$, and thus obtain

$$\begin{aligned} I(\xi ,\xi ^\prime ) = I(\xi ,\xi ) + \frac{q}{q-t}\oint _{|x|=\xi }\frac{dx}{2\pi \mathrm {i}x}f(x,q^{1/2}t^{-1/2}x). \end{aligned}$$

(93)

Now deforming the x-contour in $I(\xi ,\xi )$ to $|x|=\xi ^\prime$, picking up a residue at $x=q^{\frac{1}{2}}t^{-\frac{1}{2}}y$, we find that

$$\begin{aligned} I(\xi ,\xi ^\prime ) - I(\xi ^\prime ,\xi ) = \frac{q}{q-t}\oint _{|x|=\xi }\frac{dx}{2\pi \mathrm {i}x}\big (f(q^{-1/2}x,t^{-1/2}x) - f(q^{1/2}x,t^{1/2}x)\big ) \end{aligned}$$

(where we have applied the scaling $x\rightarrow q^{-1/2}x$ to the former residue integral and $x\rightarrow t^{1/2}x$ to the latter, which, by Cauchy’s theorem, does not alter their values). Since f satisfies (92), the integral in the right-hand side is zero, which proves the claim. This highlights the importance of the symmetry condition (30).

      It is interesting to note that if we were to deform the integration radii $\xi ,\xi ^\prime >0$ from the region $\xi /\xi ^\prime >M(q,t)$ or $\xi /\xi ^\prime <1/M(q,t)$ into the excluded region $1/M(q,t)< \xi /\xi ^\prime <M(q,t)$, then our sesquilinear form $\langle P,Q\rangle _{1,1;q,t}^\prime$ would be changed by the addition of a residue term such as the integral in (93). By considering specific examples, is readily seen that this spoils orthogonality of the super-Macdonald polynomials as well as non-negativity.

As we now sketch, the former conclusion can also be reached by demonstrating, through direct and straightforward computations, that the operator ${\mathcal M}_{1,1;q,t}$, or equivalently,

$$\begin{aligned} M_{1,1;q,t} = \frac{1}{1-q} A T_{q,x} + \frac{1}{1-t^{-1}} B T_{t^{-1},y}, \end{aligned}$$

with coefficients

$$\begin{aligned} A = \frac{ t^{1/2} x - q^{1/2} y }{ t^{1/2} x - q^{-1/2} y } , \quad B = \frac{q^{-1/2} y - t^{-1/2} x}{q^{-1/2} y - t^{1/2} x}, \end{aligned}$$

is not self-adjoint when $\xi ,\xi ^\prime >0$ are chosen in the excluded region, cf. Lemma 6.

      For simplicity, we restrict attention to $\xi =\xi ^\prime =1$ and $q<t$, but similar arguments apply for $q>t$ and other values of $\xi ,\xi ^\prime >0$ such that $1/M(q,t)< \xi /\xi ^\prime <M(q,t)$. To start with, we consider

$$\begin{aligned}&\oint _{|x|=1}\frac{d x}{2\pi \mathrm {i}x} \oint _{|y|=1}\frac{d y}{2\pi \mathrm {i}y} \Delta _{1,1}(x,y;q,t) Q^{*}(x,y) A(x,y;q,t) T_{q,x}P(x,y) \\&\quad =\oint _{|x|=1}\frac{d x}{2\pi \mathrm {i}x} \oint _{|y|=1}\frac{d y}{2\pi \mathrm {i}y} \frac{Q^{*}(x,y) A(x,y;q,t) T_{q,x} P(x,y)}{(1-q^{-1/2} t^{1/2} x / y)(1- q^{-1/2} t^{1/2} y / x)}. \end{aligned}$$

By shifting $x \rightarrow q^{-1} x$, the action of $T_{q,x}$ is transferred onto the polynomial Q and we obtain

$$\begin{aligned} \oint _{|x|=q}\frac{d x}{2\pi \mathrm {i}x} \oint _{|y|= 1 }\frac{d y}{2\pi \mathrm {i}y} \frac{ P(x,y) ( T_{q,x}Q)^*(x,y) A(q^{-1} x, y;q,t)}{(1-q^{-3/2} t^{1/2} x / y)(1- q^{1/2} t^{1/2} y / x)}, \end{aligned}$$

where

$$\begin{aligned} \frac{A(q^{-1} x, y;q,t)}{(1-q^{-3/2} t^{1/2} x / y)(1- q^{1/2} t^{1/2} y / x)} = \Delta _{1,1}(x,y;q,t)A^*(x,y;q,t). \end{aligned}$$

Deforming the x-contour back to $|x|=1$, we pick up residues at and only at the simple poles $x=q^{1/2}t^{-1/2}y$ and $x=q^{1/2}t^{1/2}y$, which yield the contribution

$$\begin{aligned}&\frac{q}{1-t}\oint _{|y|=1} \frac{dy}{2\pi \mathrm {i}y}\Big (P(q^{1/2} t^{1/2} y , y) Q^{*}(q^{-1/2} t^{1/2} y , y)\\&\quad - P(q^{1/2} t^{-1/2} y, y) Q^*(q^{-1/2} t^{-1/2}y,y)\Big ). \end{aligned}$$

Rewriting the integral involving $B(x,y;q,t)T_{t^{-1},y}$ in a similar manner produces no further residue terms (under our parameter constraints). The upshot is that

$$\begin{aligned}&\oint _{|x|=1}\frac{d x}{2\pi \mathrm {i}x} \oint _{|y|=1}\frac{d y}{2\pi \mathrm {i}y} \Delta _{1,1}(x,y;q,t) Q^{*}(x,y) M_{1,1;q,t}P(x,y)\\&\qquad - \oint _{|x|=1}\frac{d x}{2\pi \mathrm {i}x} \oint _{|y|=1}\frac{d y}{2\pi \mathrm {i}y} \Delta _{1,1}(x,y;q,t) P(x,y) (M_{1,1;q,t}Q)^{*}(x,y)\\&\quad = \frac{q}{(1-q)(1-t)}\oint _{|y|=1} \frac{dy}{2\pi \mathrm {i}y}\Big (P(q^{1/2} t^{1/2} y , y) Q^{*}(q^{-1/2} t^{1/2} y , y)\\&\qquad - P(q^{1/2} t^{-1/2} y, y) Q^*(q^{-1/2} t^{-1/2}y,y)\Big ) \end{aligned}$$

where, in general, the right-hand side is non-zero.