1 Introduction

In the present paper, we discuss a new insight into the theory of generalized hypergeometric series \({\mathfrak {P}}[\vec {x}|\vec {y}]\) generalizing those introduced by Shiraishi [1]. Here \(\vec y\) parameterize the moduli in the mother-function formalism (see [2] for a recent review), and the partitions, or Young diagrams R (on which the standard symmetric polynomials like Macdonald depend) appear on the particular locus

$$\begin{aligned} y_i = q^{R_i}(st)^{N-i} \end{aligned}$$
(1)

Our main goal is to demonstrate that the construction can and deserves to be further extended, and provides a series \({\mathfrak {P}}_\xi [\vec {x}|\vec {y}]\) for any input function \(\xi (z)\), restricted by two simple conditions

$$\begin{aligned} \xi (1)=0 \quad \mathrm{and} \quad \xi (z^{-1}) \sim z^{-1} \xi (z) \end{aligned}$$
(2)

Then, the Shiraishi series per se is associated with \(\xi (z) =1-z\), while the elliptic deformation, supposedly relevant [3] to the double-elliptic systems [4, 5], is associated with elliptic \(\xi (z) \sim \vartheta (z)\) (see also [6] for the elliptic case). However, at each power of p we get polynomials in x on the locus (1) for arbitrary \(\xi (z)\). Moreover, at least in the leading order in p, these polynomials are as nice as the ordinary Macdonald polynomials: their symmetric part is obtained by a triangular transformation from the symmetric polynomials \(\mathrm{Mon}_R[x]\) and should satisfy a Calogero/Ruijsenaars-like equations (i.e. it is expected to be possible to define an appropriate \(\xi \)-dependent version of the Calogero–Ruijsenaars Hamiltonians and integrable systems). We call this entire construction Shiraishi functor, which is hopefully acting from the space of functions \(\xi (z)\) into that of integrable systems.

In what follows, for each function \(\xi (z)\) we introduce the series that depends on variables \(x_i\), \(y_i\), \(i=1,\ldots ,N\), which is a formal power series in the ratios \(x_i/x_j\) in Sect. 2. After a certain rescaling Shiraishi series becomes also a series in positive powers of \(p^N\). Moreover, being multiplied by a proper monomial \(\prod _i x_i^{R_i}\), every item becomes a polynomial in \(x_i\) after the substitution (1) is made for \(\vec {y}\). This completes the definition of Shiraishi functor.

In the rest of the paper we mostly concentrate on generalized Noumi–Shiraishi (GNS) polynomials, which appear in the order \(p^0\) after symmetrization (see [7] for \(N=2\) GNS in the particular elliptic case). The set of such symmetric polynomials is actually as big as another well known family of deformations of Macdonald polynomials: that of Kerov functions also labeled by an arbitrary functions \(g(z) = \sum _{i=1}^\infty g_iz^i\). Still the Shiraishi set is completely different from the Kerov one, and they intersect only over the Macdonald polynomials. Moreover, as we explain, the GNS polynomials have a good chance to preserve close links to representation theory, preserved by the Macdonald deformation of the Schur polynomials, but violated by the Kerov functions.

Our claim is that, depending on entire arbitrary function \(\xi (z)\), i.e. on infinitely many additional parameters, they remain very similar to Macdonald polynomials, though very different from Kerov functions. They are obtained by a triangular transform from monomial symmetric polynomials, or from Schur polynomials, or from Macdonald polynomials, and thus can be easily continued from the Miwa locus to the entire space of time variables \(p_k\). The coefficients actually depend on \(\xi \) through a peculiar function \(\eta (z) = \frac{\xi (qz)\xi (t/qz)}{\xi (tz)\xi (z)}\), moreover, in the basis of monomial symmetric polynomials there is a simple grading, connecting the power of \(\eta \) and the number of monomial symmetric polynomials in the lexicographical ordered list of partitions at a given level. For the complementary system, conjugate to the GNS polynomials w.r.t. the Schur measure, the grading also shows up in a proper basis. With the help of this conjugate system, one can build up the Cauchy formula and define the skew GNS polynomials through the corresponding generalized Littlewood–Richardson coefficients. We also manage to construct a Hamiltonian that has the GNS polynomials associated with the one-row partitions as its eigenfunctions. In variance with the Kerov case, in the definition of the GNS polynomials, it is sufficient to choose the normal (partial) ordering, and, hence, the intersection of the sets of GNS and Kerov polynomials is exhausted by the Macdonald polynomials only.

Notation. In the paper, we denote the polynomials:

$$\begin{aligned} \begin{array}{ll} \mathrm{Schur}&{}\hbox {Schur polynomials } [7]\\ \mathrm{Mac}&{}\hbox {Macdonald polynomials } [7]\\ \mathrm{Mon}&{}\hbox {monomial symmetric polynomials } [7]\\ \mathrm{Kerov}&{}\hbox {Kerov functions } [8,9]\\ \mathrm{Gns},\ \mathrm{Gns}^\perp &{}\hbox {generalized Noumi}{-}\hbox {Shiraishi polynomials } (17), (23)\\ p_k:=\sum _ix_i^k&{}\hbox {power symmetric polynomials} \end{array} \end{aligned}$$
(3)

For the Young diagram \(R=\{R_1\ge R_2 \ge \cdots \}\), we use the notations \(p_R:=\prod _i p_{R_i}\) and \(z_R:=\prod _k k^{m_k}m_k!\) where \(m_k\) is the number of lines of length k in the diagram R. We use the functions

$$\begin{aligned} \eta (z)= & {} \frac{\xi (qz)\xi (tz/q)}{\xi (tz)\xi (z)}\nonumber \\ \zeta _k(z)= & {} \frac{\xi (q^kz)\xi (tz)}{\xi (q^{k-1}tz)\xi (qz)} = \prod _{i=1}^{k-1} \eta (q^iz) \end{aligned}$$
(4)

Throughout the paper, we call Macdonald locus the choice of function \(\xi (z)=1-z\).

2 Shiraishi functor: promoting Macdonald \(1-z\) to arbitrary function

Now we give a preliminary definition of Shiraishi functor that allows one to convert an arbitrary function \(\xi (z)\) into a map from a power series of 2N variables parameterized by the function \(\xi (z)\) and by four parameters to an infinite set of graded functions parameterized by the Young diagrams R.

Suppose we are given a function \(\xi (z)\) with the symmetry properties

$$\begin{aligned} \xi (1)=0,\quad \xi (z^{-1})=\alpha z^{-1}\xi (z),\quad \alpha \in {\mathbb {C}} \end{aligned}$$
(5)

Define

$$\begin{aligned} \Xi (z;q)_n = {\left\{ \begin{array}{ll} \displaystyle {\prod _{k=0}^{n-1}}\xi (q^k z), \qquad n \ge 0, \\ \displaystyle {\prod _{k=0}^{n-1}} \xi (q^{-k-1} z)^{-1}, \qquad n < 0. \end{array}\right. } \end{aligned}$$
(6)

where \(|q|<1\) is a parameter. Now one can define a \(\xi \)-Shiraishi power series

$$\begin{aligned}&{\mathfrak {P}}_N^{\xi } { (x_i ; p \vert y_i ; s \vert q,t)} \nonumber \\&\quad {:=}\sum _{\lambda ^{(i)}} \prod _{i,j=1}^N \frac{\mathcal {N^\xi }_{\lambda ^{(i)}, \lambda ^{(j)}}^{(j-i)} (t y_j/y_i \vert q,s)}{\mathcal {N^\xi }_{\lambda ^{(i)}, \lambda ^{(j)}}^{(j-i)} (y_j/y_i \vert q,s)} \prod _{\beta =1}^N \prod _{\alpha \ge 1} \left( \frac{p x_{\alpha + \beta }}{t x_{\alpha + \beta -1}} \right) ^{\lambda _\alpha ^{(\beta )}} \nonumber \\ \end{aligned}$$
(7)

where

$$\begin{aligned} \mathcal {N^\xi }_{\lambda , \mu }^{(k)} (u \vert q, s):= & {} \!\!\!\!\!\!\!\! \prod _{\begin{array}{c} b \ge a \ge 1 \\ b-a \equiv k~(\mathrm {mod}~n) \end{array}} \!\!\!\!\!\!\!\! \Xi \Big (uq^{-\mu _a + \lambda _{b+1}} s^{-a +b} ; q\Big )_{_{\lambda _b - \lambda _{ b+1}}}\nonumber \\&\times \!\!\!\!\! \prod _{\begin{array}{c} b \ge a \ge 1 \\ b - a \equiv -k-1~(\mathrm {mod}~n) \end{array}} \!\!\!\!\!\!\!\! \Xi \Big (uq^{\lambda _a - \mu _b} s^{a - b -1} ; q\Big )_{_{\mu _b - \mu _{b+1}}}\nonumber \\ \end{aligned}$$
(8)

where \(\{\lambda ^{(i)}\}\), \(i=1,\ldots ,N\) is a set of N partitions, and we assume \(x_{N+i}=x_i\). Then,

$$\begin{aligned}&{} \mathbf{P}^\xi _R(x_i;p,s,q,t)\nonumber \\&\quad :=\prod _{i=1}^Nx_i^{R_i}\cdot {\mathfrak {P}}_N^{\xi } { \left( p^{N-i}x_i ; p\, \Big |\,y_i=q^{R_i}(st)^{N-i} ; s \,\Big | \, q,\frac{q}{t}\,\right) }\nonumber \\ \end{aligned}$$
(9)

is a graded function of variables \(x_i\) of the weight \(|R|=\sum _iR_i\), which is a series in \(p^{Nk}\):

$$\begin{aligned} \mathbf{P}^\xi _R(x_i;p,s,q,t)= & {} \sum _{k\ge 0}p^{Nk}\cdot {{\mathbf{P}^{{\varvec{\xi }}^{(k)}}}_R (x_i;s,q,t)\over \prod _{i=1}^N x_i^k}\nonumber \\= & {} \sum _{k\ge 0}{} \mathbf{P}^{{\varvec{\xi }}^{(k)}}_R(x_i;s,q,t)\cdot \prod _{i=1}^N\left( {p\over x_i}\right) ^k \end{aligned}$$
(10)

Here \(\mathbf{P}^{{\varvec{\xi }}^{(k)}}_R(x_i;s,q,t)\) is a polynomial of variables \(x_i\) with grade \(|R|+Nk\).

Note that, in (8), instead of using the standard Nekrasov function, we impose the mod n selection rule following [1]. It is essential for assuring \(\mathbf{P}^{{\varvec{\xi }}^{(k)}}_R(x_i;s,q,t)\) to be a polynomial, moreover, to be a symmetric polynomial at \(N=2\) and almost a symmetric polynomial at larger N (see the next section).

If one chooses \(\xi (z)=1-z\), then \({\mathfrak {P}}_N^{\xi } { (x_i ; p \vert y_i ; s \vert q,t)}\) is the standard Shiraishi function. Choosing \(\xi (z)=\sqrt{z}\vartheta _p(z)/(p;p)_\infty \), one gets an elliptic lift of the Shiraishi function (ELS-function). In the both these cases, \(\alpha =-1\).

3 Symmetric polynomials

The main problem with this definition is that defined in such a way \(\mathbf{P}^{{\varvec{\xi }}^{(k)}}_R(x_i;s,q,t)\) are not obligatory symmetric in \(x_i\) (beyond the Macdonald case). This may imply that the definition should be somehow modified. In what follows, we use a very concrete prescription to make symmetric polynomials of \(\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(x_i;s,q,t)\), and obtain their properties which can afterwards be used for their axiomatic definition, independent of (7) and, if necessary, deviating from it.

The question is intimately related to \(x\leftrightarrow y\) symmetry (spectral duality [11,12,13,14,15]), which is lost for \(\xi (z)\) beyond the Macdonald locus and can be somehow restored by introducing another arbitrary function \({{\tilde{\xi }}}(z)\) for the y-dependence, satisfying a \(\xi \leftrightarrow {{\tilde{\xi }}}\) duality, which is, however, beyond the analysis in this text.

Here we construct a basis of symmetric functions in the following way: we define a symmetric function \(\Big [\mathbf{P}^{{\varvec{\xi }}^{(k)}}_R(x_i;s,q,t)\Big ]_{symm}\) as a sum

$$\begin{aligned}&\Big [\mathbf{P}^{{\varvec{\xi }}^{(k)}}_R(x_i;s,q,t)\Big ]_{symm}\nonumber \\&\quad :=\sum _{\mu \vdash |R|} \left[ \mathbf{P}^{{\varvec{\xi }}^{(k)}}_R(x_i;s,q,t)\right] _{\mu }\mathrm{Mon}_{_\mu }[\vec {x}] \end{aligned}$$
(11)

of monomial symmetric polynomials \(\mathrm{Mon}_{_\mu }\),

$$\begin{aligned} \mathrm{Mon}_{_\mu }[\vec {x}] = \sum _{\sigma \in S_N} x_{\sigma (i)}^{\mu _i}, \end{aligned}$$
(12)

where \(\Big [\ldots \Big ]_{\mu }\) denotes the coefficient in front of \(\prod _ix_i^{\mu _i}\).

The difference \(\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(x_i;s,q,t) -\Big [\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(x_i;s,q,t)\Big ]_{symm}\) is always vanishing for \(N=2\) and is non-vanishing for \(N>2\) first time at level 4 for the single diagram [3, 1]. For \(N=4\) this example looks as

$$\begin{aligned}&{} \mathbf{P}^{{\varvec{\xi }}^{(0)}}_{[3,1]}(x_i;s,q,t) -\Big [\mathbf{P}^{{\varvec{\xi }}^{(0)}}_{[3,1]}(x_i;s,q,t)\Big ]_{symm}\nonumber \\&\quad =F_0\left( x_1x_2x_3^2+(x_1x_2+x_1x_3+x_2x_3)x_4^2\right) \nonumber \\&F_0:=\zeta _2(1)^2-\zeta _2(1)\zeta _2(qt)-\zeta _2(1)\zeta _2(q)+\zeta _2(t)\zeta _2(qt)\nonumber \\ \end{aligned}$$
(13)

where \(\mathrm{Mon}_{R}^{(k)}\) denotes the monomial symmetric polynomial of variables \(x_i\), \(i=1,\ldots ,k\).

Similarly, at level 5 one has:

$$\begin{aligned}&{} \mathbf{P}^{{\varvec{\xi }}^{(0)}}_{[4,1]}(x_i;s,q,t)-\Big [\mathbf{P}^{{\varvec{\xi }}^{(0)}}_{[4,1]}(x_i;s,q,t)\Big ]_{symm}\nonumber \\&\quad =F_1\Big (x_1x_2x_3^3+(x_1x_2+x_1x_3+x_2x_3)x_4^3\Big ) \nonumber \\&\qquad +F_2\Big ((x_1x_3^2+x_2x_3^2+x_2^2x_3+x_1^2x_2+x_2^2x_1 +x_2^2x_3)x_4^2\nonumber \\&\qquad +x_1^2x_2x_3^2+x_1x_2^2x_3^2\Big ) +F_3x_1x_2x_3^2x_4+F_4x_1x_2x_3x_4^2 \end{aligned}$$
(14)
$$\begin{aligned}&F_1:=\zeta _2(1)\zeta _3(1)-\zeta _2(q^2t)\zeta _3(1)-\zeta _4(1)+\zeta _4(t)\nonumber \\&F_2:=\zeta _3(1)\Big (\zeta _3(1)-\zeta _2(1)\zeta _2(q^2t)-\zeta _3(q)+\zeta _3(qt)\Big )\nonumber \\&F_3:=\zeta _3(1)\zeta _2(q^2t)F_0\nonumber \\&F_4:=\zeta _3(1)F_1 \end{aligned}$$
(15)

All these combinations \(F_i\) are vanishing at the Macdonald locus and at the ELS-function locus (i.e. the choice \(\xi (z)=\sqrt{z}\vartheta _p(z)/(p;p)_\infty \)) as well.

4 Role of the functor parameters

The role of N.

N appears in Sect. 2 in several roles:

  • As the number of variables \(x_i\)

  • In reduction condition for \(y_i\)

  • As the length of Young diagrams \(R_i\)

In the leading order in p, the polynomial \(\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(x_i;s,q,t)\) can be considered as a graded polynomial of time variables \(p_k:=\sum _i^Nx_i^k\) so that N can be considered arbitrary large, and N does not enter formulas. In the usual terms of symmetric functions associated with representation theory, fixing concrete N corresponds to reduction from \(GL_\infty \) to \(GL_N\). A typical example is restricting the infinite Toda chain to a Toda chain of length N. This analogy can be pushed even further, see the next paragraph.

The role of p: series vs polynomials. The Shiraishi functions belong to the class of mother functions, i.e. they depend not on the Young diagrams, but on continuous variables \(\vec {y}\). Relation/reduction to Young diagram R emerges through the specialization

$$\begin{aligned} y_i = q^{R_i}(st)^{N-i} \end{aligned}$$
(16)

The Shiraishi functions are expressed only through \(\vec {x} = \{x_1,\ldots ,x_N\}\), no immediate lifting to generic time-variables \(p_k\) from the Miwa locus \(p_k^* = \sum _{i=1}^N x_i^k\) is available at generic \(\vec {y}\): they are not symmetric for generic \(\vec y\). Outside (16), these functions are asymmetric series in \(\vec {x}\), interpolating between polynomials of various R-dependent degrees on the loci (16).

The Shiraishi functions depend on a variety of parameters. From the point of view of the series-polynomial relation, distinguished is the role of p. The Shiraishi function is an infinite series, but it becomes a finite Laurent polynomial at each degree of p after specializing \(\vec {y}\) to a Young diagram by (16). Degree of p is \(p^{N\sum _{i,j=1}^N \lambda _{Nj-i+1}^{(i)}}\). At the leading order \(p^0\), one gets after multiplication by \(\prod _{i=1}^N x_i^{R_i}\) and symmetrization, an ordinary symmetric polynomial, which we call generalized Noumi–Shiraishi polynomial, since the mother function of this form for the Macdonald polynomials was studied in detail in [16].

The powers of p are related to imposing the periodicity condition \(x_{N+i}=x_i\). At a given N, p dependence is expressed entirely through \(p^N\), which can serve as a better parameter than p itself. A counterpart of this parameter could be found in the periodic Toda chain: one can consider an infinite Toda chain and impose the periodicity condition with period N. Then, the Baker–Akhiezer function (the eigenfunction of the Lax operator) is quasi-periodic, and the counterpart of the quasiperiodicity parameter is just p. Bringing this parameter to zero, one obtains the open Toda chain, and this is much similar to the way of obtaining the generalized Noumi–Shiraishi polynomial. In algebraic terms, this corresponds to transition from the affine to finite-dimensional algebras.

One could also look for a description of the system at \(N=\infty \) (\(SL_\infty \) level), where \(\vec {x}\) becomes a sort of a function x(p), and p plays the role of the associated loop parameter. Then one can pick up a particular locus (particular solutions of the would-be universal hypergeometric equation), specified by a peculiar N-periodicity in order to get the Shiraishi series for particular N with the condition \(x_{N+i} = p^{-N}x_i\).

The role of s is still obscure to us, thus to avoid confusion we do not comment on it in the present text.

Note also that the role of parameters q and t becomes completely different in variance with the standard Macdonald case: q governs the Pochhammer symbols and is in charge of the deformation from simple polynomials like the Schur polynomials, while t is rather related to concrete details of this deformation. Moreover, t in some parts of formulas is mixed with the parameter s (like the choice of \(y_i\)), but in some other, not.

5 GNS polynomials

The polynomial \(\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(x_i;s,q,t)\) for arbitrary function \(\xi (z)\) does not depend on s, and so does the symmetric polynomial \(\Big [\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(x_i;s,q,t)\Big ]_{symm}\) thus we can denote it through \(\Big [\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(q,t)[x]\Big ]_{symm}\). It also does not depend on N in the following sense: it can be rewritten as a polynomial of the time variables \(p_k:=\sum _{i=1}^N x_i^k\), and after that the coefficients of \(\Big [\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(q,t)\{p_k\}\Big ]_{symm}\) do not depend on N (!). Note that time variables \(p_k\) are denoted by the same letter as the parameter p, both notations are standard and can not be changed. We hope that this will not cause confusion, especially because, in this paper, we mostly discuss the GNS polynomials, which depend on \(p_k\) but no longer on p (they are the \(p^0\) contribution to the Shiraishi series).

The GNS polynomial can be rewritten in the form, similar to the Noumi–Shiraishi representation of the Macdonald polynomials [16], hence, the name:

$$\begin{aligned} \mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(q,t)[x]=\prod _{i=1}^N x_i^{R_i} \cdot \sum _{m_{ij}}{{{\mathcal {C}}}}^R_n(m_{ij}|q,t)\prod _{1\le i<j\le N}\left( {x_j\over x_i}\right) ^{m_{ij}}\nonumber \\ \end{aligned}$$
(17)

where \(m_{ij}=0\) for \(i\ge j\), \(m_{ij}\in {\mathbb {Z}}_{\ge 0}\),

$$\begin{aligned}&{{{\mathcal {C}}}}^R_n(m_{ij},|q,t):=\nonumber \\&\quad = \prod _{k=2}^n\prod _{1\le i<j\le k}{\Xi \Big (q^{R_j-R_i+\sum _{a>k}(m_{ia}-m_{ja})}t^{i-j+1};q\Big )_{m_{ik}} \over \Xi \Big (q^{R_j-R_i+\sum _{a>k}(m_{ia}-m_{ja})}qt^{i-j};q\Big )_{m_{ik}}}\nonumber \\&\qquad \cdot \prod _{k=2}^n\prod _{1\le i\le j<k}{\Xi \Big (q^{R_j-R_i-m_{jk}+\sum _{a>k}(m_{ia}-m_{ja})}qt^{i-j-1};q\Big )_{m_{ik}} \over \Xi \Big (q^{R_j-R_i-m_{jk}+\sum _{a>k}(m_{ia}-m_{ja})}t^{i-j};q\Big )_{m_{ik}}} \nonumber \\ \end{aligned}$$
(18)

Formulas (17) and (11) allow one to give a manifest representation of the GNS symmetric polynomials in terms of monomial symmetric polynomials \(\mathrm{Mon}_{_R}\),

$$\begin{aligned} \mathrm{Gns}_R[x]:=\Big [\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(q,t)[x]\Big ]_{symm}=\sum _{P}C_{RP}^{(q,t)} \cdot \mathrm{Mon}_{_P}[x]\nonumber \\ \end{aligned}$$
(19)

Lifted to the time-variables space \(p_k:=\sum _ix_i^k\), the monomial symmetric polynomials coincide with restriction of Macdonald polynomials to \(t=1\) (which do not depend on q):

$$\begin{aligned} \begin{array}{rcl} \mathrm{Mon}_{_R}[\vec {x}] &{}=&{} \mathrm{Mac}_{_R}^{t=1}[\vec {x}]\\ &{}\downarrow &{}\\ \mathrm{Mon}_{_R}\{p_k\}&{}=&{}\mathrm{Mac}_{_R}^{t=1}\{p_k\} \end{array} \end{aligned}$$
(20)

The Kostka matrix \(C_{RP}^{(q,t)}\) is diagonal in the size of Young diagrams, \(C_{RP}\sim \delta _{|R|,|P|}\), and it is triangular in a matrix form, for the lexicographical ordering of Young diagrams:

(21)

In the lexicographical ordering, \(R>P\) if, for i running from 1, the first non-vanishing difference \(R_i-P_i\) is positive. In fact, similarly to the ordinary Macdonald polynomials, it is sufficient to choose the natural (or dominance) partial ordering [8], since the elements of the Kostka matrix between the unordered Young diagrams vanish. In this ordering, \(R>P\) if \(\sum _{i=1}^k\Big (R_i-P_i\Big )\) for all k. It is a partial ordering: for instance, the natural ordering does not fix the order of the diagrams [2, 2, 2] and [3, 1, 1, 1], and \(C_{[2,2,2],[3,1,1,1]}^{(q,t)}=C_{[3,1,1,1],[2,2,2]}^{(q,t)}=0\).

Thus, the non-deformed GNS polynomial \(\mathrm{Gns}_{R}\) corresponds to the antisymmetric Young diagram, while the most deformed, to the symmetric one.

From representation (17) it follows that the coefficients of GNS polynomials are graded combinations of products of the function \(\eta (z):= \frac{\xi (qz)\xi (tz/q)}{\xi (tz)\xi (z)}\) at various points \(q^it^j\). Hence, there is a hidden symmetry of the polynomials: they are invariant w.r.t. the replace \(\xi (z)\rightarrow z^a\xi (z)\) with an arbitrary a.

If one chooses \(\xi (z)=1-z\), then \(\mathrm{Gns}_R[x]\) is the standard Macdonald polynomial, and (17) gives its Noumi–Shiraishi representation [16].

6 Properties of the GNS polynomials

Orthogonality. Let us define a conjugate system of polynomials in the following way. Denote \(\Psi _{RQ}\) the coefficients of the p-expansion of the GNS

$$\begin{aligned} \mathrm{Gns}_{R}=\sum _Q \Psi _{RQ}\cdot p_Q \end{aligned}$$
(22)

(this expansion is in no way triangular). Then, the set of polynomials

$$\begin{aligned} \mathrm{Gns}_{R}^\perp =\sum _Q \Psi ^{-1}_{QR}\cdot {p_Q\over z_Q} \end{aligned}$$
(23)

with \(\Psi ^{-1}\) being the inverse matrix, is orthogonal,

$$\begin{aligned} \Big <\mathrm{Gns}_{R}\Big |\mathrm{Gns}_{Q}^\perp \Big >=\delta _{RQ} \end{aligned}$$
(24)

w.r.t. to the measure

$$\begin{aligned} \Big \langle p_\Delta \Big |p_{\Delta '}\Big \rangle = z_\Delta \delta _{\Delta ,\Delta '} \end{aligned}$$
(25)

Triangularity. The polynomials \(\mathrm{Gns}_{R}^\perp \) possess a triangular expansion

$$\begin{aligned} \mathrm{Gns}_{R}^\perp =\prod _i \mathrm{Schur}_{[R_i]}+\sum _{P>R}C_{RP}^\perp \prod _i \mathrm{Schur}_{[P_i]} \end{aligned}$$
(26)

again with graded coefficients in \(\eta \). This follows from the orthogonality relations

$$\begin{aligned} \Big <\mathrm{Mon}_{R}\Big |\prod _i \mathrm{Schur}_{[Q_i]}\Big >=\delta _{RQ} \end{aligned}$$
(27)

It also follows that \(C_{RP}^{\perp }\) is just the transposition of the matrix inverse to \(C_{RP}^{(q,t)}\), (21):

$$\begin{aligned} C_{RP}^{\perp }=C^{-1}_{PR} \end{aligned}$$
(28)

and the (partial) natural ordering is again sufficient in (26): the corresponding coefficients are zero like the previously discussed example: \(C^\perp _{[2,2,2],[3,1,1,1]}=C^\perp _{[3,1,1,1],[2,2,2]}=0\).

Fig. 1
figure 1

Symbolic description of zeroes of the product decomposition \(P_{R'}P_{R''}=\sum _Q P_Q\) in the lexicographically ordered set of partitions (Young diagrams) Q. A product of Kerov functions (\(\square \)) gets non-vanishing contributions from everywhere in between \(R'\cup R''\) and \(R'+R''\). Contributions to the product of Macdonald (and Schur) polynomials (\(\bullet \)) are non-vanishing only at some points in this domain, which belong to the product of representations \(R'\otimes R''\). A product of the generalized Noumi–Shiraishi polynomials (\(\blacktriangle \)) can get non-vanishing contributions from the same points as Macdonald polynomials, but also from the region in between \(R'+R''\) and \([1^{|R'|+|R''|}]\), while their conjugate polynomials (\(\blacktriangledown \)) gets contributions from the same points as Macdonald polynomials, but also from the region in between \(R'\cup R''\) and \([|R'|+|R''|-1,1]\)

Thus, the non-deformed \(\mathrm{Gns}_{R}^\perp \) corresponds to the symmetric Young diagram, while the most deformed, to the antisymmetric one.

Note that, in the Macdonald case,

$$\begin{aligned} \mathrm{Gns}_{R}^\perp \Big |_{\xi (z)=1-z}\{p_k\}=\mathrm{{\overline{Mac}}}_{R^\vee }\{(-1)^{k+1}p_k\} \end{aligned}$$
(29)

where the bar over the Macdonald polynomial denotes the permutation of q and t.

The Cauchy formula. Having the orthogonality relation (24), one can also immediately get the Cauchy formula for the GNS polynomials:

$$\begin{aligned} \sum _R \mathrm{Gns}_{R}\{p_k\}\cdot \mathrm{Gns}_{R}^\perp \{p_k'\}=\exp \left( \sum _k{p_kp_k'\over k}\right) \end{aligned}$$
(30)

It follows from (22) and (23):

$$\begin{aligned} \sum _R \mathrm{Gns}_{R}\{p_k\}\cdot \mathrm{Gns}_{R}^\perp \{p_k'\}= & {} \sum _{R,Q,Q'} \Psi _{RQ}\cdot \Psi ^{-1}_{Q'R}\cdot {p_Qp'_{Q'}\over z_{Q'}}\nonumber \\= & {} \sum _{Q} {p_Qp'_{Q}\over z_{Q}}= \exp \left( \sum _k{p_kp_k'\over k}\right) \nonumber \\ \end{aligned}$$
(31)

Ring structure. (Fig. 1) One can construct the structure constants of the ring formed by the polynomials \(\mathbf{P}^{{\varvec{\xi }}^{(0)}}_R(q,t)\), i.e. the generalized Littlewood–Richardson coefficients:

$$\begin{aligned} \mathrm{Gns}_{R'}\{p_k\}\cdot \mathrm{Gns}_{R''}\{p_k\}= \sum _R\mathbf{N}^R_{R'R''}\mathrm{Gns}_R\{p_k\} \end{aligned}$$
(32)

They possess an interesting property: in the case of Kerov functions, non-vanishing are only the coefficients between the partitions \(R'\cup R'' = [r'_1+r''_1,r'_2+r_2'',\ldots ]\) and \(R'+R'' = [\mathrm{ordered\ collection\ of\ all}\ r'_i \ \mathrm{and} \ r''_j]\); in the case of Macdonald polynomials, there are additional zeroes, since only the partitions associated with the irreducible representations (via the Schur–Weyl duality) emerging in the decomposition of \(R'\otimes R''\) contribute, and not all of the representations between \(R'\cup R''\) and \(R'+R''\) (in the partial ordering) emerge in this decomposition. In the GNS case, similarly to the Macdonald case, the coefficients \(\mathbf{N}^R_{R'R''}\) are vanishing for the partitions with the ordering number larger than \(R'\cup R''\), and vanishing for the representations between \(R'\cup R''\) and \(R'+R''\) for R not belonging to the decomposition of \(R'\otimes R''\). However, they are non-vanishing for partitions with the ordering number less than \(R'+R''\). For instance, the group theory decomposition dictates that, for the Macdonald polynomials,

$$\begin{aligned} \mathrm{Mac}_{[2]}\cdot \mathrm{Mac}_{[1,1]}=\mathrm{Mac}_{[3,1]}\oplus \mathrm{Mac}_{[2,1,1]} \end{aligned}$$
(33)

and, in the more general case of Kerov functions,

$$\begin{aligned} \mathrm{Kerov}_{[2]}\cdot \mathrm{Kerov}_{[1,1]} =\mathrm{Kerov}_{[3,1]}\oplus \mathrm{Kerov}_{[2,2]}\oplus \mathrm{Kerov}_{[2,1,1]}\nonumber \\ \end{aligned}$$
(34)

At the same time, for the GNS,

$$\begin{aligned} \mathrm{Gns}_{[2]}\cdot \mathrm{Gns}_{[1,1]}= \mathrm{Gns}_{[3,1]}\oplus \mathrm{Gns}_{[2,1,1]} \oplus \mathrm{Gns}_{[1,1,1,1]}\nonumber \\ \end{aligned}$$
(35)

One can similarly define the generalized Littlewood–Richardson coefficients for the conjugate GNS:

$$\begin{aligned} \mathrm{Gns}^\perp _{R'}\{p_k\}\cdot \mathrm{Gns}^\perp _{R''}\{p_k\}= \sum _R\mathbf{N^\perp }^R_{R'R''}\mathrm{Gns}^\perp _R\{p_k\} \end{aligned}$$
(36)

One may think these polynomials behave exactly like the Macdonald polynomials: the non-vanishing ring structure coefficients are determined by the group theory decomposition. For instance,

$$\begin{aligned} \mathrm{Gns}^\perp _{[2]}\cdot \mathrm{Gns}^\perp _{[1,1]}=\mathrm{Gns}^\perp _{[3,1]}\oplus \mathrm{Gns}^\perp _{[2,1,1]} \end{aligned}$$
(37)

However, this is not always the case: for instance,

$$\begin{aligned} \mathrm{Gns}^\perp _{[1,1]}\cdot \mathrm{Gns}^\perp _{[1,1]}=\mathrm{Gns}^\perp _{[3,1]}\oplus \mathrm{Gns}^\perp _{[2,2]}\oplus \mathrm{Gns}^\perp _{[2,1,1]}\oplus \mathrm{Gns}^\perp _{[1,1,1,1]}\nonumber \\ \end{aligned}$$
(38)

while

$$\begin{aligned} \mathrm{Mac}_{[1,1]}\cdot \mathrm{Mac}_{[1,1]}=\mathrm{Mac}_{[2,2]}\oplus \mathrm{Mac}_{[2,1,1]}\oplus \mathrm{Mac}_{[1,1,1,1]}\nonumber \\ \end{aligned}$$
(39)

Generally, the coefficients \(\mathbf{N^\perp }^R_{R'R''}\) are vanishing for the partitions with the ordering number less than \(R'\cup R''\), and vanishing for the partitions between \(R'\cup R''\) and \(R'+R''\) for R not belonging to the decomposition of representations \(R'\otimes R''\). However, they are non-vanishing for partitions with the ordering number larger than \(R'+R''\) for exception of the partition associated with the symmetric representation \([|R'|+|R''|]\).

Skew polynomials. The skew GNS polynomials can be defined in the standard way:

$$\begin{aligned} \mathrm{Gns}_R\{p_k+p_k'\}=\sum _Q\mathrm{Gns}_{R/Q}\{p_k\}\cdot \mathrm{Gns}_Q\{p_k'\} \end{aligned}$$
(40)

and one can similarly define the dual skew polynomials \(\mathrm{Gns}^\perp _{R/Q}\{p_k\}\). Properties of the skew polynomials follow from the orthogonality relations. In particular, similarly to the standard Macdonald (and further, the Kerov case), the skew GNS polynomials can be constructed from the ring structure coefficients:

$$\begin{aligned} \mathrm{Gns}_{R/Q}=\sum _P{\mathbf{N}^\perp }^R_{QP}\cdot \mathrm{Gns}_{P} \end{aligned}$$
(41)

This follows from the Cauchy formula (30) and the general theorem [17]:

$$\begin{aligned} \begin{array}{ccc} \sum _R \mathrm{Gns}_R^\perp \{p\}\cdot \mathrm{Gns}_R \{p'+p''\} &{}= \exp \left( \sum _k \frac{p_k(p'_k+p''_k)}{k}\right) &{}= \sum _{P,Q} \mathrm{Gns}_P^\perp \{p\}\cdot \mathrm{Gns}_P\{p' \} \mathrm{Gns}_Q^\perp \{p\}\cdot \mathrm{Gns}_Q\{p'' \}\\ &{}&{}\\ || &{}&{} || \\ &{}&{}\\ \sum _{R,Q} \mathrm{Gns}_R^\perp \{p\}\cdot \mathrm{Gns}_{R/Q}\{p'\}\mathrm{Gns}_Q\{p''\} &{}=&{} \sum _{P,Q,R} {\mathbf{N}^\perp }_{PQ}^R \mathrm{Gns}_R^\perp \{p\}\mathrm{Gns}_P\{p' \}\mathrm{Gns}_Q\{p'' \} \end{array} \end{aligned}$$

Assuming that the map \(F:\ f(R,Q,p')\mapsto \tilde{f}(p,p',p{''}):=\sum _{R,Q}\mathrm{Gns}_R^\perp \{p\}\mathrm{Gns}_Q\{p'' \}f(R,Q,p')\) does not have a kernel, one can omit the identical factors \(\mathrm{Gns}_R^\perp \{p\}\mathrm{Gns}_Q\{p'' \}\) at the two sides of the bottom line, and get (41).

One can also define the conjugate skew symmetric polynomials

$$\begin{aligned} \mathrm{Gns}^\perp _R\{p_k+p_k'\}=\sum _Q\mathrm{Gns}^\perp _{R/Q}\{p_k\}\cdot \mathrm{Gns}^\perp _Q\{p_k'\} \end{aligned}$$
(42)

and they are also given by the formula involving the ring structure coefficients:

$$\begin{aligned} \mathrm{Gns}^\perp _{R/Q}=\sum _P{\mathbf{N}}^R_{QP}\cdot \mathrm{Gns}^\perp _{P} \end{aligned}$$
(43)

The generalized Cauchy formula. As usual, one can easily extend the Cauchy formula (30) to the case of skew polynomials:

$$\begin{aligned}&\sum _R \mathrm{Gns}_{R/\eta _1}\{p_k\}\cdot \mathrm{Gns}_{R/\eta _2}^\perp \{p_k'\}\nonumber \\&\quad =\exp \left( \sum _k{p_kp_k'\over k}\right) \sum _\rho \mathrm{Gns}_{\eta _2/\rho }\{p_k\}\cdot \mathrm{Gns}_{\eta _1/\rho }^\perp \{p_k'\}\nonumber \\ \end{aligned}$$
(44)

It is proved much similar to the previous formulas: using the map F. Indeed, apply this map to the l.h.s. of (44) and use the definition of the skew GNS polynomials, formulas (41)-(43) and the Cauchy formula (30):

$$\begin{aligned}&\sum _{\eta _1,\eta _2}\mathrm{Gns}_{\eta _1}\{\bar{p}_k\}\mathrm{Gns}^\perp _{\eta _2}\{{\bar{p}}_k'\} \sum _R \mathrm{Gns}_{R/\eta _1}\{p_k\}\cdot \mathrm{Gns}_{R/\eta _2}^\perp \{p_k'\}\\&\quad =\sum _R \mathrm{Gns}_{R}\{p_k+{\bar{p}}_k\}\cdot \mathrm{Gns}_{R}^\perp \{p_k'+{\bar{p}}'_k\} \\&\quad =\exp \left( \sum _k{(p_k+{\bar{p}}_k)(p_k'+{\bar{p}}'_k)\over k}\right) \\&\quad = \exp \left( \sum _k{p_kp_k'\over k}\right) \cdot \exp \left( \sum _k{p_k{\bar{p}}_k'\over k}\right) \cdot \exp \left( \sum _k{{\bar{p}}_kp_k'\over k}\right) \\&\qquad \cdot \exp \left( \sum _k{{\bar{p}}_k{\bar{p}}_k'\over k}\right) =\exp \left( \sum _k{p_kp_k'\over k}\right) \cdot \sum _{\lambda ,\mu ,\rho } \mathrm{Gns}_{\lambda }\{p_k\}\\&\qquad \cdot \mathrm{Gns}_{\lambda }^\perp \{{\bar{p}}'_k\}\cdot \mathrm{Gns}_{\mu }\{{\bar{p}}_k\}\cdot \mathrm{Gns}_{\mu }^\perp \{p'_k\}\cdot \mathrm{Gns}_{\rho }\{{\bar{p}}_k\}\cdot \mathrm{Gns}_{\rho }^\perp \{{\bar{p}}'_k\}\\&\quad =\exp \left( \sum _k{p_kp_k'\over k}\right) \cdot \sum _{\lambda ,\mu ,\rho ,\eta _1,\eta _2} \mathrm{Gns}_{\lambda }\{p_k\} \mathrm{Gns}_{\mu }^\perp \{p'_k\}\\&\qquad \cdot {\mathbf{N}^\perp }^{\eta _2}_{\lambda \rho } \mathrm{Gns}_{\eta _2}\{{\bar{p}}'_k\}\cdot {\mathbf{N}}^{\eta _1}_{\mu \rho }\mathrm{Gns}_{\eta _1}\{{\bar{p}}_k\}\\&\quad =\exp \left( \sum _k{p_kp_k'\over k}\right) \\&\qquad \cdot \sum _{\eta _1,\eta _2}\mathrm{Gns}_{\eta _1}\{\bar{p}_k\}\mathrm{Gns}^\perp _{\eta _2}\{{\bar{p}}_k'\}\cdot \sum _\rho \mathrm{Gns}_{\eta _2/\rho }\{p_k\}\cdot \mathrm{Gns}_{\eta _1/\rho }^\perp \{p_k'\} \end{aligned}$$

Then, again, formula (44) follows from the assumption of absence of a kernel of the map F.

The Hamiltonian. One of the main features of the standard symmetric functions is that they satisfy the eigenvalue equation with a Hamiltonian [18, 19]. In fact, there are typically infinitely many Hamiltonians, which is a consequence of integrability behind a set of symmetric functions, but the first Hamiltonian is sufficient to unambiguously restore the whole set. Hence, one has to look for a Hamiltonian for the GNS polynomials. At \(N=2\) case, they satisfy the equation

$$\begin{aligned} {\hat{H}}\mathrm{Gns}_{[r]}= & {} 0,\nonumber \\ {\hat{H}}:= & {} {x_1\xi ({\hat{P}}_2)\xi (t{\hat{P}}_1)-x_2\xi ({\hat{P}}_1)\xi (t{\hat{P}}_2)\over x_1-x_2} \end{aligned}$$
(45)

where \(P_i:=q^{x_i\partial _{x_i}}\). Moreover,

$$\begin{aligned} {\hat{H}}\mathrm{Gns}_{[r,s]}= & {} \xi (q^s)\xi (tq^r)\cdot \mathrm{Gns}_{[r,s]}\nonumber \\&+\sum _{1\le i\le (r-s)/2}\alpha _i\cdot \mathrm{Gns}_{[r-i,s+i]} \end{aligned}$$
(46)

and the Young diagrams with more than 2 rows do not contribute in the \(N=2\) case. This means that \(\mathrm{Gns}_{[r,s]}\) with \(s=r,r-1\) are also eigenfunctions along with \(\mathrm{Gns}_{[r]}\), i.e. with \(s=0\), the eigenvalues being \(\xi (q^s)\xi (tq^r)\).

Let us remind that the standard Macdonald Hamiltonian has the form

$$\begin{aligned} {\hat{H}}={x_2-tx_1\over x_2-x_1}{\hat{P}}_1+{x_1-tx_2\over x_1-x_2}{\hat{P}}_2 \end{aligned}$$
(47)

To illustrate how it works, consider the action of this Hamiltonian on the \(\mathrm{Gns}_{[2]}\):

$$\begin{aligned} \xi ({\hat{P}}_2)\xi (t{\hat{P}}_1)x_1^2= & {} x_1^2\xi (1)\xi (tq^2)=0,\nonumber \\ \xi ({\hat{P}}_1)\xi (t{\hat{P}}_2)x_1^2= & {} x_1^2\xi (q^2)\xi (t)\nonumber \\ \xi ({\hat{P}}_2)\xi (t{\hat{P}}_1)x_1x_2= & {} x_1x_2\xi (q)\xi (tq),\nonumber \\ \xi ({\hat{P}}_1)\xi (t{\hat{P}}_2)x_1x_2= & {} x_1x_2\xi (q)\xi (qt)\nonumber \\ \xi ({\hat{P}}_2)\xi (t{\hat{P}}_1)x_2^2= & {} \xi (q^2)\xi (t),\nonumber \\ \xi ({\hat{P}}_1)\xi (t{\hat{P}}_2)x_2^2= & {} x_2^2\xi (tq^2)\xi (1)=0 \end{aligned}$$
(48)

Since

$$\begin{aligned} \mathrm{Gns}_{[2]}=x_1^2+x_2^2+{\xi (q^2)\xi (t)\over \xi (q)\xi (qt)}x_1x_2 \end{aligned}$$
(49)

one immediately obtains

$$\begin{aligned} \Big (x_1\xi ({\hat{P}}_2)\xi (t{\hat{P}}_1)-x_2\xi ({\hat{P}}_1)\xi (t{\hat{P}}_2)\Big )\mathrm{Gns}_{[2]}=0 \end{aligned}$$
(50)

It turns out to be a general rule at any N: the GNS polynomial associated with one-row partition satisfies the equation (we have checked this conjecture up to level 6)

$$\begin{aligned} {\hat{H}}\mathrm{Gns}_{[r]}= & {} 0,\nonumber \\ {\hat{H}}:= & {} \prod _{i<j} \Big (x_i\xi ({\hat{P}}_j)\xi (t{\hat{P}}_i)-x_j\xi ({\hat{P}}_i)\xi (t{\hat{P}}_j)\Big ) \end{aligned}$$
(51)

The standard Macdonald Hamiltonian for arbitrary N has the form

$$\begin{aligned} {\hat{H}}=\sum _i\prod _{j\ne i}{x_j-tx_i\over x_j-x_i}{\hat{P}}_i \end{aligned}$$
(52)

Note that this Hamiltonian depends explicitly on the original function \(\xi (z)\), while their eigenfunctions depend on \(\eta (z)\). In certain sense, the Hamiltonian has an “anomaly” as compared to the eigenfunctions (symmetric polynomials \(\mathrm{Gns}_R\)).

Since \(\mathrm{Gns}^\perp \) associated with one-row partition is just the non-deformed Schur polynomial, it is trivially the eigenfunction of the usual Macdonald Hamiltonian (52).

Thus, we demonstrated that the GNS are associated, at least, with quasi-exactly solvable system [20,21,22].

7 Incompatibility with Kerov functions

In this section, we explain that the GNS polynomials are an extension of the Macdonald polynomials very different from another extension known under the name Kerov functions. We deal with the very explicit polynomials of the first levels, the formulas for the GNS case being collected in the Appendix.

1. Kerov functions were introduced in [9] and recently reviewed in some detail in [10]. They are obtained by a triangular orthogonalization of polynomials (12) w.r.t. the scalar product

$$\begin{aligned} \langle p_{R}|p_{R'}\rangle = \delta _{R,R'}\cdot z_R\cdot g_R \end{aligned}$$
(53)

\(g_R:=\prod _ig_{R_i}\) with arbitrary parameters \(g_k\). The triangular Kostka matrix \(\mathrm{Kerov}_R = \sum _Q C_{RQ} \cdot \mathrm{Mon}_{_Q}\) for the Kerov functions looks as follows:

$$\begin{aligned} C^{(1)}= & {} \left( 1\right) \\ C^{(2)}= & {} \left( \begin{array}{cc} 1 &{} 0 \\ \\ \frac{2g_2}{g_1^2+g_2} &{} 1 \end{array}\right) \\ C^{(3)}= & {} \left( \begin{array}{ccc} 1 &{} 0 &{} 0 \\ \\ \frac{3g_3(g_1^2+g_2) }{\Delta _3^\vee } &{} 1 &{} 0 \\ \\ \frac{6g_2g_3}{\Delta _3^\vee } &{} \frac{6(g_1g_2+g_3)}{\Delta _3} &{} 1 \end{array}\right) \\ C^{(4)}= & {} \left( \begin{array}{ccccc} 1 &{}0 &{}0 &{} 0 &{}0 \\ \\ \frac{4g_2g_4(2g_1^3g_2+3g_1^2g_3+g_2g_3)}{\Delta _4^\vee } &{} 1 &{} 0 &{} 0 &{} 0 \\ \\ \frac{6g_3g_4(g_2^2+g_2)^2}{\Delta _4^\vee } &{} \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! -\frac{2g_2(g_1^2+g_2)(g_1^3g_4-3g_1^2g_2g_3-3g_1g_2g_4-g_3g_4)}{\Delta _4'^\vee } &{} 1 &{} 0 &{} 0 \\ \\ \frac{12g_2g_3g_4(g_1^2+g_2)}{\Delta _4^\vee } &{} \frac{\begin{array}{cc}6g_1^4g_2^2g_3+3g_1^4g_3g_4+4g_1^3g_2^2g_4+\\ +12g_1^2g_2^3g_3 +6g_1^2g_2g_3g_4+12g_1g_2^3g_4+5g_2^2g_3g_4\end{array}}{\Delta _4'^\vee } &{} \frac{2(g_2^2+g_4)(g_1^3+3g_1g_2+2g_3)}{\Delta _4'} &{} 1 &{} 0 \\ \\ \frac{24g_2^2g_3g_4}{\Delta _4^\vee } &{} \frac{12g_2(2g_1^2g_2^2g_3+g_1^2g_3g_4+2g_1g_2^2g_4+g_2g_3g_4)}{\Delta _4'^\vee } &{} \frac{12(g_2^2+g_4)(g_1g_2+g_3)}{\Delta _4'} &{} \frac{12(g_1^2g_2+2g_1g_3+g_2^2+2g_4)}{\Delta _4} &{} 1 \end{array}\right) \end{aligned}$$
(54)

with denominator functions \(\Delta \) described in [10, Eq. (71)]:

$$\begin{aligned} \Delta _3= & {} g_1^3+3g_1g_2+2g_3 = 3!\mathrm{Schur}_{[3]}\{g\} \nonumber \\ \Delta _3^\vee= & {} 2g_1^3g_2+3g_1^2g_3+g_2g_3 = 3!g_1^2g_2g_3\mathrm{Schur}_{[3]}\{g^{-1}\} \nonumber \\ \Delta _4^\vee= & {} 6g_1^4g_2^2g_3+3g_1^4g_3g_4+8g_1^3g_2^2g_4\nonumber \\&+6g_1^2g_2g_3g_4+g_2^2g_3g_4\nonumber \\ \Delta _4'^\vee= & {} 2g_1^5g_2^3+g_1^5g_2g_4+6g_1^4g_2^2g_3+3g_1^4g_3g_4+2g_1^3g_2^2g_4\nonumber \\&+4g_1^2g_2^3g_3 +2g_1^2g_2g_3g_4+3g_1g_2^3g_4+g_2^2g_3g_4\nonumber \\ \end{aligned}$$
(55)

2. There is a minor degree of consistency between (54) and (62), for example, in (62) \(C_{[3][111]}=C_{[3][21]}C_{[2][11]}\), and the same is true for (54):

$$\begin{aligned} \frac{6g_2g_3}{\Delta _3^\vee } = \frac{3g_3(g_1^2+g_2)}{\Delta _3^\vee } \cdot \frac{2g_2}{g_1^2+g_2} \end{aligned}$$
(56)

or, say, \(C_{[4][22]}=C_{[4][31]}C_{[3][21]}\)

$$\begin{aligned} \frac{6g_3g_4(g_2^2+g_2)^2}{\Delta _4^\vee }= & {} \frac{4g_2g_4(2g_1^3g_2+3g_1^2g_3+g_2g_3)}{\Delta _4^\vee } \nonumber \\&\cdot \frac{3g_3(g_1^2+g_2) }{\Delta _3^\vee } \end{aligned}$$
(57)

These relations demonstrate that sometime the numerators in (54) are made from the same denominator functions, but generically these relations are far more involved. The same is true about the consistency domain between (54) and (62): it is too small. Despite the “number of degrees of freedom” in the both cases is the same (it is given by an arbitrary function, or by an infinite set of coefficients), they enter the Kostka matrices in a very different way, and the elements of (54) are severely stronger constrained than the function \(\xi \).

3. There is no room for the second set of GNS polynomials, as it happens in the Kerov case. Normally the second set of polynomials is associated with a transformation like (54), which is triangular w.r.t. the alternative, anti-lexicographic ordering of Young diagrams. However, in the GNS case there is no difference: the second set of polynomials is just the same, the change of ordering does not change the polynomials. This follows from the vanishing of the Kostka matrix elements between partitions differing in the two different orderings. The first such a matrix element emerges at level 6 between the partitions [2, 2, 2] and [3, 1, 1, 1]: \(C_{[2,2,2],[3,1,1,1]}=C_{[3,1,1,1],[2,2,2]}=0\) (and similarly for their transposed [3, 3] and [4, 1, 1]).

Moreover, it is shown in [10, 23] that, within Kerov family, these matrix elements vanish only for the Macdonald polynomials, therefore, one can conclude that the intersection

$$\begin{aligned} \mathrm{GNS\ pols} \ \cap \ \mathrm{Kerov\ funcs} \ =\ \mathrm{Macdonald\ pols} \end{aligned}$$
(58)

4. Orthogonality of the GNS polynomials implies non-diagonality of the scalar product \(\langle p_{R}\Big | p_{R'}\rangle \). For example, at level 3, the diagonalizability of \(C^{(3)}=\left( \begin{array}{ccc} 1 &{} 0&{} 0 \\ a&{} 1&{} 0\\ b&{} c&{} 1\end{array}\right) \) requires that \(c = \frac{(a-3)b}{a^2-a-b}\). It is indeed true for the Kerov case, when

$$\begin{aligned} a= & {} \frac{3g_3(g_1^2+g_2) }{\Delta _3^\vee },\nonumber \\ b= & {} \frac{6g_2g_3}{\Delta _3^\vee },\nonumber \\ c= & {} \frac{6(g_1g_2+g_3)}{\Delta _3} \end{aligned}$$
(59)

but is not true for (62).

In fact, even a little bit more strong statement is correct: the requirement of diagonal scalar product of the GNS polynomials is inconsistent with a diagonal scalar product of the Kerov functions.

8 Conclusion

In this paper, we introduced a new infinite-parametric family of symmetric polynomials, which seems to be an interesting generalization of the two-parametric Macdonald polynomials. They are also obtained by a triangular transformation of monomial symmetric polynomials and Schur polynomials; they form a closed ring under multiplication graded by the size of the Young diagrams; they are independent on the choice of ordering of Young diagrams within the natural partial ordering; and their ring structure coefficients lying between \(R'\cup R''\) and \(R'+R''\) vanish together with those in the Macdonald case, i.e. for representations which do not appear in the product of other two:

$$\begin{aligned}&R_1+R_2 \le Q\le R_1\cup R_2:\ \ \ \ \nonumber \\&{\mathbf{N}^\perp }^Q_{R_1R_2} \ne 0 \Longleftrightarrow Q\in R_1\otimes R_2, \end{aligned}$$
(60)

(note that there is still a deviation from representation theory: some \(\mathbf{N}^\perp \) have “tails”, they do not vanish also for some “more symmetric” \(Q>R_1+ R_2\), while \(\mathbf{N}\), though also satisfying (60), have tails from the other, “antisymmetric” side: for \(Q<R_1\cup R_2\)).

In addition, the Kostka matrix relating GNS polynomials to the Schur ones has a non-trivial grading. This makes this family an interesting alternative, or, better, complement, to the family of Kerov functions [9, 10], which is equally large, but deviates much more from representation theory.

This remarkable family is currently built in a four-step process.

  • First, we consider a far-going generalization of Shiraishi series [1], by introducing an arbitrary function \(\xi (z)\), instead of just \(\xi (z) = 1-z\) in [1] or \(\xi (z)\sim \vartheta (z)\) in [3, 6] related to the Dell deformation of the integrable Calogero–Ruijsenaars systems and Nekrasov theories. We call this generalization Shiraishi functor from the space of functions \(\xi (z)\) restricted only by behaviour w.r.t. \(z\rightarrow 1/z\) to that of hypergeometric-like Shiraishi series.

  • Second, we restrict \(\vec {y}\) variables in Shiraishi mother function to Young diagrams by the usual rule (16), expand in peculiar parameter p and, after an appropriate rescaling, pick up a \(p^0\) term. This procedure mimics that in [16], and we call the result the Generalized Noumi–Shiraishi symmetric polynomials in \(\vec {x}\).

  • Third, we expand these polynomials in, say, Schur functions (expansion appears to be triangular), what allows to raise them from the space of \(\vec {x}\) to that of time-variables \(p_k\). This provides what we call GNS polynomials, \(\mathrm{Gns}_R^\xi \{p\}\).

  • Forth, we construct a system of polynomials , conjugate to \(\mathrm{Gns}_R\{p\}\) w.r.t. the standard scalar product of time-variables \(\Big \langle p_\Delta \Big | p_{\Delta '}\Big \rangle \,=\,z_\Delta \delta _{\Delta ,\Delta '}\). These polynomials appear to be remarkably distinguished by their properties.

This system possesses further generalization by the change of the scalar product and by a look on the higher powers in p. Calculations in the latter case become far more tedious and will be considered elsewhere. The primary task for the future research is to further explore these, \(\mathrm{Gns}_R\{p\}\), \(\mathrm{Gns}^\perp _R\{p\}\), with the hope to find their straightforward definition, for example, by finding a \(\xi \)-dependent scalar product, in which they become self-orthogonal (rather than form a bi-orthogonal system with the GNS polynomials \(\mathrm{Gns}_R\{p\}\)), or by finding a general formula for the \(\xi \)-deformation of the Ruijsenaars Hamiltonians, for which \(\mathrm{Gns}^\perp _R\{p\}\) are the common eigenfunctions.

Clearly, the remarkable polynomials \(\mathrm{Gns}^\perp _R\{p\}\), or their symmetric-polynomial version \(\mathrm{Gns}^\perp _R[x]\) on the Miwa locus \(p_k = \sum _{i=1}^N x_i^k\), deserve a separate name and notation, free of the reference to scaffolding used in our current construction . This is, however, a little premature to decide, first the adequate definitions and language should be found.