Special cycles on unitary Shimura varieties with minuscule parahoric level structure

Abstract

In this paper, we formulate conjectural formulas for the arithmetic intersection numbers of special cycles on unitary Shimura varieties with minuscule parahoric level structure. Also, we prove that these conjectures are compatible with all known results.

Appendix A: Arithmetic intersection numbers of special cycles \({{\mathcal {Z}}}(x_1), \dots , {{\mathcal {Z}}}(x_{n+1})\) in \({{\mathcal {N}}}^1(1,n)\)

In this appendix, we will prove that our conjectures are compatible with [10, Theorem 10.4.4]. This theorem relates the intersection number of ${{\mathcal {Z}}}(x_1), \dots , {{\mathcal {Z}}}(x_{n+1})$ in ${{\mathcal {N}}}^1(1,n)$ to representation densities. This intersection number can be regarded as the intersection number of

$$\begin{aligned} {{\mathcal {Z}}}(x_1), \dots , {{\mathcal {Z}}}(x_n),{{\mathcal {Z}}}(x_{n+1}),{{\mathcal {Y}}}(y_1),\dots ,{{\mathcal {Y}}}(y_{n-1}) \end{aligned}$$

in ${{\mathcal {N}}}^n(1,2n-1)$, where

$$\begin{aligned} B=\left( \begin{array}{c@{\quad }c} h(x_i,x_j) &{} h(x_i,y_l)\\ h(y_k,x_j)&{} h(y_k,y_l) \end{array} \right) _{1\le i,j \le n+1, 1\le k,l \le n-1}=\left( \begin{array}{c@{\quad }c} B_1 &{} \\ &{} \dfrac{1}{\pi } 1_{n-1} \end{array} \right) , \end{aligned}$$

and $B_1 \in X_{n+1}(E_v)$.

Therefore, the Conjecture 3.25 should be compatible with [10, Theorem 10.4.4]. To prove this compatibility, we will use the notation in Sect. 3.2.

Let L be an $O_{E_v}$-lattice with basis $\lbrace u_1, \dots , u_{2n} \rbrace$ such that the matrix of inner product $(\langle u_i, u_j \rangle )$ is $\pi B$. Let $L_1$ be the $O_{E_v}$-lattice of rank $n+1$ with basis $\lbrace u_1, \dots , u_{n+1} \rbrace$, $L_2$ be the $O_{E_v}$-lattice of rank $n-1$ with basis $\lbrace u_{n+1}, \dots , u_{2n} \rbrace$. Note that $L_2$ is unimodular with respect to $\langle \cdot , \cdot \rangle$.

Let M be an $O_{E_v}$-lattice with basis $\lbrace v_1, \dots , v_m \rbrace$ ($m=2n+2r$) such that the matrix of inner product $(\langle v_i,v_j \rangle )$ is $\pi A^{[r]}_{n}.$

For $0 \le i \le n-1$, let $M_i$ be an $O_{E_v}$-lattice with basis $\lbrace v_{1,i}, \dots , v_{2n,i} \rbrace$ such that the matrix of inner product $(\langle v_{k,i},v_{l,i} \rangle )$ is $\pi A_{i}$.

We define the sets

$$\begin{aligned} \begin{array}{lll} J_d^1(L,M)&{}=\lbrace &{} \varphi \in {{\,\mathrm{Hom}\,}}_{O_{E_v}}(L, M/\pi ^{d} M) \vert \\ &{}&{}\langle \varphi (x), \varphi (y) \rangle \equiv \langle x, y \rangle {{\,\mathrm{mod}\,}}\pi ^d, \quad \forall x,y \in L,\\ &{}&{}\varphi (L_1) \subset \pi M^{\vee } \rbrace . \end{array} \end{aligned}$$

and for $0 \le i \le n-1$,

$$\begin{aligned} \begin{array}{lll} J_d^1(L,M_i)&{}=\lbrace &{} \varphi \in {{\,\mathrm{Hom}\,}}_{O_{E_v}}(L, M_i/\pi ^{d} M_i) \vert \\ &{}&{}\langle \varphi (x), \varphi (y) \rangle \equiv \langle x, y \rangle {{\,\mathrm{mod}\,}}\pi ^d, \quad \forall x,y \in L,\\ &{}&{}\varphi (L_1) \subset \pi M_i^{\vee } \rbrace . \end{array} \end{aligned}$$

Then we have

$$\begin{aligned} W_{n-1,n}(B,r)=q^{-2dm(2n)}q^{(d-1)(2n)^2} \vert J_d^1(L,M) \vert , \end{aligned}$$

and

$$\begin{aligned} W_{n-1,i}(B,0)=q^{-2d(2n)^2}q^{(d-1)(2n)^2} \vert J_d^1(L,M_i) \vert , \end{aligned}$$

for sufficiently large d.

Note that L has the rank $n-1$ unimodular sublattice $L_2$. Also, any unimodular sublattice in $M_i$ has rank $\le i$. Therefore, if $i <n-1$, there is no $\varphi \in {{\,\mathrm{Hom}\,}}_{O_{E_v}}(L,M_i/\pi ^dM_i)$ such that $\langle \varphi (x), \varphi (y) \rangle \equiv \langle x,y \rangle {{\,\mathrm{mod}\,}}\pi ^d$ for sufficiently large d. This implies that $J_d(L,M_i)$ is an empty set for $i <n-1$. Therefore, $W_{n-1,i}(B,0)=0$ for $i <n-1$.

This implies that we only need to compute the constant $\beta _{n-1}^{n-1}$ in Conjecture 3.25. We have the following proposition.

Proposition A.1

$\beta _{n-1}^{n-1}=\dfrac{(1-(-q)^n)}{(-q)^{3n+1}(1-(-q))(1-(-q)^{-(n+1)})}$.

Proof

By the proof of theorem 3.16, we have

$$\begin{aligned} {{\mathfrak {B}}}\left( \begin{array}{l} \beta _0^h\\ \vdots \\ \beta _{n-1}^h\\ -\beta _{0}^{2n-h}\\ \vdots \\ -\beta _{n-1}^{2n-h}\\ \delta _h \end{array} \right) =(-q)^{-2n(2n-h)}\left( \begin{array}{c} -(2n-h)\\ \vdots \\ -1\\ 0\\ 1\\ \vdots \\ h \end{array}\right) , \end{aligned}$$

where $0 \le h \le 2n$ (in this proposition, we only need $h=n-1$, but we will use this general notation for later use). Therefore, we need to find the inverse matrix of $(-q)^{2n(2n-h)}{{\mathfrak {B}}}$. Note that

$$\begin{aligned} \begin{array}{l} {{\mathfrak {B}}}=\\ \left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} m_{10}&{} \dots &{} m_{1(n-1)}&{} n_{10}&{} \dots &{} n_{1(n-1)} &{} m_{1n}\\ \vdots &{}\vdots &{}\vdots &{}\vdots &{}\vdots &{}\vdots &{}\vdots \\ m_{(2n+1)0}&{} \dots &{}m_{(2n+1)(n-1)} &{} n_{(2n+1)0} &{} \dots &{} n_{(2n+1)(n-1)}&{}m_{2n+1,n} \end{array}\right) \end{array} \end{aligned}$$

and

$$\begin{aligned} m_{it}= & {} (-q)^{(n-t)(i-(2n-h+1))-2t(2n-h)},\\ n_{it}= & {} q^{-(2n-h)^2+h^2}(-q)^{-(n-t)(i-(2n-h+1))-2th}. \end{aligned}$$

Therefore, we have

$$\begin{aligned} (-q)^{2n(2n-h)}m_{it}= & {} (-q)^{(n-t)(i+2n-h-1)},\nonumber \\ (-q)^{2n(2n-h)}n_{it}= & {} (-q)^{-(n-t)(i-(2n+h+1))}. \end{aligned}$$

(A.0.1)

We define

$$\begin{aligned} \begin{array}{ll} \alpha _{i,h}=(-q)^{(n+1-i)(2n-h)}, &{} 1 \le i \le n;\\ \alpha _{i,h}=(-q)^{(2n+1-i)(2n+h)}, &{} n+1 \le i \le 2n;\\ \alpha _{2n+1,h}=1, \end{array} \end{aligned}$$

and

$$\begin{aligned} \begin{array}{ll} x_i=(-q)^{n+1-i}, &{} 1 \le i \le n; \\ x_i=(-q)^{i-2n-1}, &{} n+1 \le i \le 2n;\\ x_{2n+1}=1. \end{array} \end{aligned}$$

Also, we define a $2n+1$ $\times$ $2n+1$ matrix ${{\mathfrak {a}}}_h={{\,\mathrm{diag}\,}}(\alpha _{1,h}, \dots , \alpha _{2n+1,h})$ and a Vandermonde matrix

$$\begin{aligned} {{\mathfrak {X}}}=\left( \begin{array}{llll} 1 &{} 1&{} \dots &{} 1\\ x_1 &{}x_2 &{} &{}x_{2n+1}\\ x_1^2&{}x_2^2 &{} &{} x_{2n+1}^2\\ \vdots &{} &{} \ddots &{} \vdots \\ x_1^{2n} &{} x_2^{2n} &{} \dots &{} x_{2n+1}^{2n} \end{array}\right) . \end{aligned}$$

Then, we have $(-q)^{2n(2n-h)}{{\mathfrak {B}}}={{\mathfrak {X}}}{{\mathfrak {a}}}_h$

Therefore, we have

$$\begin{aligned} \left( \begin{array}{l} \beta _0^h\\ \vdots \\ \beta _{n-1}^h\\ -\beta _{0}^{2n-h}\\ \vdots \\ -\beta _{n-1}^{2n-h}\\ \delta _h \end{array} \right) = {{\mathfrak {a}}}_h^{-1}{{\mathfrak {X}}}^{-1} \left( \begin{array}{c} -(2n-h)\\ \vdots \\ -1\\ 0\\ 1\\ \vdots \\ h \end{array}\right) \end{aligned}$$

Let ${{\mathfrak {X}}}^{-1}=(y_{ij})_{1 \le i \le 2n+1}$. It is well known that the (i, j)-th entry of the inverse matrix of a Vandermonde matrix is

We can think $y_{ij}$ as the $z^{2n+1-j}$-coefficient of

Since we have

Note that $(1-x_{2n+1})=0$. Therefore we have,

Since $x_n=(-q)$ we have

$$\begin{aligned} x_m-x_n=\left\{ \begin{array}{ll} ((-q)^{n-m}-1)(-q)&{} 1 \le m \le n-1\\ ((-q)^{m-2n-2}-1)(-q)&{}n+1 \le m \le 2n+1, \end{array}\right. , \end{aligned}$$

and

$$\begin{aligned} 1-x_m=\left\{ \begin{array}{ll} 1-(-q)^{n+1-m}&{} 1 \le m \le n-1\\ 1-(-q)^{m-2n-1}&{}n+1 \le m \le 2n \end{array}\right. . \end{aligned}$$

Combining these two and $\alpha _{n,n-1}=(-q)^{n+1}$, we have

$$\begin{aligned} \beta _{n-1}^{n-1}=\dfrac{(1-(-q)^n)}{(-q)^{3n+1}(1-(-q))(1-(-q)^{-(n+1)})}. \end{aligned}$$

$\square$

Proposition A.2

(cf. [7, Proposition 9.9] and Proposition 3.20)

• (1) Let $\lbrace O_j \rbrace$ be a set of representatives for the $U(M_i)$-orbits in the set of all sublattices O of $M_i$ such that O is isometric to $L_2$. Then we have

  $$\begin{aligned} \begin{array}{ll} \vert J_d^1(L,M_i) \vert =&{}\sum _{j}\vert I_d(L_2,M_i;O_j) \vert \\ &{}\times \vert \lbrace \varphi _1 \in J_d^1(L_1,M_i) \vert \langle \varphi (L_1), O_j \rangle \equiv 0 ({{\,\mathrm{mod}\,}}\pi ^d) \rbrace . \end{array} \end{aligned}$$

• (2) Let $\lbrace N_j \rbrace$ be a set of representatives for the U(M)-orbits in the set of all sublattices N of M such that N is isometric to $L_2$. Then we have

  $$\begin{aligned} \begin{array}{ll} \vert J_d^1(L,M) \vert =&{}\sum _{j}\vert I_d(L_2,M;N_j) \vert \\ &{}\times \vert \lbrace \varphi _1 \in J_d^1(L_1,M) \vert \langle \varphi (L_1), N_j \rangle \equiv 0 ({{\,\mathrm{mod}\,}}\pi ^d) \rbrace . \end{array} \end{aligned}$$

Proof

The proof of this proposition is identical to the proof of [7, Proposition 9.9] (and Proposition 3.20). $\square$

From now on, we fix $i=n-1$. Note that $L_2$ is unimodular. Therefore, by the arguments in [7, p.680], all sublattices $O \subset M_{n-1}$ such that O is isometric to $L_2$ are in the same $U(M_{n-1})$-orbit, and $M_{n-1}=O \perp O^{\perp }$ where $O^{\perp }:=(E_v O)^{\perp } \cap M_{n-1}$. Let us fix O the sublattice of $M_{n-1}$ with the basis $\lbrace v_{n+2,n-1}, \dots , v_{2n,n-1} \rbrace$ as a representative of the unique $U(M_{n-1})$-orbit of $L_2$. Then, $O^{\perp }$ is the lattice with basis $\lbrace v_{1,n-1}, \dots , v_{n+1,n-1} \rbrace$. In particular, $O^{\perp } \subset \pi M_{n-1}^{\vee }$.

Also, let us fix N the sublattice of M with the basis $\lbrace v_{n+2}, \dots , v_{2n} \rbrace$. Then N is a representative of the unique U(M)-orbit of $L_2$ and $M=N \perp N^{\perp }$. We have the following analogue of [7, Lemma 9.10] (and Lemma 3.21).

Lemma A.3

• (1) For sufficiently large d, we have

  $$\begin{aligned} \begin{array}{l} \vert \lbrace \varphi _1 \in J_d^1(L_1,M_{n-1}) \vert \langle \varphi _1(L_1),O \rangle \equiv 0 \text { }({{\,\mathrm{mod}\,}}\pi ^d) \rbrace \vert =\vert I_d(L_1,O^{\perp })\vert \end{array}. \end{aligned}$$

• (2) For sufficiently large d, we have

  $$\begin{aligned}&\vert \lbrace \varphi _1 \in J_d^1(L_1,M) \vert \langle \varphi _1(L_1),N \rangle \equiv 0 \text { }({{\,\mathrm{mod}\,}}\pi ^d) \rbrace \vert \nonumber \\&\quad =q^{-2(n+1)}\vert I_d(L_1,N^{\perp } \cap \pi M^{\vee })\vert . \end{aligned}$$

  (A.0.2)

Proof

The proof of this lemma is almost identical to the proof of [7, Lemma 9.10] and Lemma 3.21. Therefore, we will only prove (2). First note that

$$\begin{aligned} \lbrace x \in M \vert \langle x , N \rangle \equiv 0 \text { } ({{\,\mathrm{mod}\,}}\pi ^d) \rbrace =\pi ^d N^{\vee } \perp N^{\perp }, \end{aligned}$$

for sufficiently large d such that $\pi ^d N^{\vee } \subset N$.

Therefore, we have

$$\begin{aligned} \begin{array}{ll} \lbrace \varphi _1 \in J_d(L_1,M) \vert \quad \quad \langle \varphi _1(L_1) , N \rangle \equiv 0 \text { } &{}({{\,\mathrm{mod}\,}}\pi ^d) \rbrace \\ =\lbrace \varphi _1:L_1 \rightarrow \pi ^d N^{\vee } \perp N^{\perp } \text { } ({{\,\mathrm{mod}\,}}\pi ^d) \vert &{} \langle \varphi _1(x), \varphi _1(y) \rangle \equiv \langle x,y\rangle \text { } ({{\,\mathrm{mod}\,}}\pi ^d)\\ &{}\varphi _1(L_1) \subset \pi M^{\vee } \rbrace . \end{array} \end{aligned}$$

Since $\pi ^{d}N^{\vee } \subset \pi M^{\vee }$ for sufficiently large d, we can write (A.0.2) as

$$\begin{aligned} \lbrace \varphi _1:L_1 \rightarrow \pi ^d N^{\vee } \perp (\pi M^{\vee } \cap N^{\perp }) \text { } ({{\,\mathrm{mod}\,}}\pi ^d) \vert \langle \varphi _1(x), \varphi _1(y) \rangle \equiv \langle x,y\rangle \text { } ({{\,\mathrm{mod}\,}}\pi ^d)\rbrace . \end{aligned}$$

Now, fix a positive integer a such that $\pi ^a N^{\vee } \subset N$.

Then we have

$$\begin{aligned} \begin{array}{lll} \quad \vert \lbrace \varphi _1:L_1 \rightarrow \pi ^d N^{\vee } \perp (\pi M^{\vee } \cap N^{\perp }) \text { } &{} ({{\,\mathrm{mod}\,}}\pi ^d) \vert \langle \varphi _1(x), \varphi _1(y) \rangle \equiv \langle x,y\rangle \text { } ({{\,\mathrm{mod}\,}}\pi ^d)\rbrace \vert &{}\\ =\vert \lbrace \varphi _1:L_1 \rightarrow \pi ^d N^{\vee } \perp (\pi M^{\vee } \cap N^{\perp }) \text { }&{} ({{\,\mathrm{mod}\,}}\pi ^d (\pi ^aN^{\vee } \perp (\pi M^{\vee } \cap N^{\perp }))) \vert &{}\\ &{}\langle \varphi _1(x), \varphi _1(y) \rangle \equiv \langle x,y\rangle \text { } ({{\,\mathrm{mod}\,}}\pi ^d)\rbrace \vert \\ \times \vert M:\pi ^a N^{\vee } \perp (\pi M^{\vee } \cap N^{\perp }) \vert ^{-(n+1)} \end{array} \end{aligned}$$

by replacing M by $\pi ^a N^{\vee } \perp (\pi M^{\vee } \cap N^{\perp })$.

Now, we can write $\varphi _1=\psi _1 + \psi _2$ where

$$\begin{aligned} \begin{array}{l} \psi _1:L_1 \rightarrow \pi ^d N^{\vee } / \pi ^{d+a}N^{\vee },\\ \psi _2:L_1 \rightarrow (\pi M^{\vee } \cap N^{\perp })/\pi ^d (\pi M^{\vee } \cap N^{\perp }). \end{array} \end{aligned}$$

Since we assume that d is sufficiently large such that $\pi ^d N^{\vee } \subset N$, we have

$$\begin{aligned} \langle \psi _1(x),\psi _1(y) \rangle \in \langle N, \pi ^d N^{\vee } \rangle \subset \pi ^dO_{E_v}. \end{aligned}$$

This means that the condition $\langle \varphi _1(x),\varphi _1(y) \rangle \equiv \langle x,y \rangle \text { }({{\,\mathrm{mod}\,}}\pi ^d)$ is the same as the condition $\langle \psi _2(x),\psi _2(y) \rangle \equiv \langle x,y \rangle \text { } ({{\,\mathrm{mod}\,}}\pi ^d)$, and we do not need to impose any condition on $\psi _1$.

Therefore, (A.0.2) is equal to

$$\begin{aligned}&\vert M:\pi ^aN^{\vee } \perp (\pi M^{\vee } \cap N^{\perp }) \vert ^{-(n+1)}\vert \pi ^d N^{\vee } : \pi ^{d+a}N^{\vee }\vert ^{n+1} \vert I_d(L_1,(\pi M^{\vee } \cap N^{\perp }))\vert \\&\quad =\vert M:N^{\vee }\perp (\pi M^{\vee } \cap N^{\perp }) \vert ^{-(n+1)} \vert I_d(L_1,(\pi M^{\vee } \cap N^{\perp })) \vert \\&\quad =\vert M:N\perp (\pi M^{\vee } \cap N^{\perp }) \vert ^{-(n+1)}\vert N^{\vee }:N \vert ^{n+1} \vert I_d(L_1,(\pi M^{\vee } \cap N^{\perp })) \vert \\&\quad =q^{-2(n+1)}\vert I_d(L_1,(\pi M^{\vee } \cap N^{\perp })) \vert . \end{aligned}$$

Here, we used the fact that $\vert M:N \perp (\pi M^{\vee } \cap N^{\perp })\vert =q^2$ and $N=N^{\vee }$ (since N is unimodular).

$\square$

Proposition A.2 and Lemma A.3 imply that

$$\begin{aligned} \vert J_d^1(L,M_{n-1}) \vert= & {} \vert I_d(L_2,M_{n-1})\vert \times \vert I_d(L_1,O^{\perp })\vert , \nonumber \\ \vert J_d^1(L,M) \vert= & {} q^{-2(n+1)}\vert I_d(L_2,M) \vert \times \vert I_d(L_1,N^{\perp } \cap \pi M^{\vee })\vert . \end{aligned}$$

(A.0.3)

Note that

$$\begin{aligned} W_{n-1,n}(B,r)= & {} q^{-2dm(2n)}q^{(d-1)(2n)^2}\vert J_d^1(L,M)\vert ;\nonumber \\ W_{n-1,n-1}(B,0)= & {} q^{-2d(2n)^2}q^{(d-1)(2n)^2}\vert J_d^1(L,M_{n-1})\vert ;\nonumber \\ W_{n,n}(A_n,r)= & {} q^{-2dm(2n)}q^{(d-1)(2n)^2}\vert J_d(\Delta ,M)\vert , \end{aligned}$$

(A.0.4)

for sufficiently large d. Here $\Delta$ is the lattice defined in the proof of the theorem 3.19.

Also, we have

$$\begin{aligned} \begin{array}{c} (q^{-d})^{(n-1)(2m-(n-1))} \vert I_d(L_2,M)\vert =\alpha (\pi A_n^{[r]},1_{n-1});\\ \\ (q^{-d})^{(n+1)(2(m-(n-1))-(n+1))} \vert I_d(L_1,N^{\perp } \cap \pi M^{\vee })\vert \\ =\alpha (\left( \begin{array}{c@{\quad }c@{\quad }c} \pi 1_n &{} &{}\\ &{} \pi ^2 &{}\\ &{} &{} \pi 1_{2r} \end{array}\right) ,\pi B_1);\\ \\ (q^{-d})^{(n-1)(2(2n)-(n-1))} \vert I_d(L_2,M_{n-1})\vert =\alpha (\pi A_{n-1},1_{n-1});\\ \\ (q^{-d})^{(n+1)(2(2n-(n-1))-(n+1))} \vert I_d(L_1,O^{\perp })\vert =\alpha (\pi 1_{n+1},\pi B_1);\\ \\ (q^{-d})^{n(2m-n)} \vert I_d(\Delta _2,M)\vert =\alpha (\pi A_n^{[r]},1_{n});\\ \\ (q^{-d})^{n(2(m-n)-n)} \vert I_d(\Delta _1,N^{\perp })\vert =\alpha (\pi 1_{n+2r},\pi 1_{n}); \end{array} \end{aligned}$$

(A.0.5)

for sufficiently large d.

Now, we need to compute these representation densities. We have the following proposition.

Proposition A.4

Let k, m, n be integers such that $k,n \le m$ and let $r=m-n$. Then we have

$$\begin{aligned} \alpha (\left( \begin{array}{c@{\quad }c} \pi 1_k &{} \\ &{} 1_{m-k} \end{array}\right) , 1_n)=\prod _{l=1}^{n}(1-(-q)^{-l+k-r}).\\ \end{aligned}$$

This formula is just a special case of [2, Theorem II]. In the proof of this proposition, we will use the notation in [2] (and [18, Page 921]).

Let $\Lambda _n^+:=\lbrace \lambda =(\lambda _1, \dots , \lambda _n) \in {{\mathbb {Z}}}^n \vert \lambda _1 \ge \lambda _2 \ge \dots \ge \lambda _n \ge 0 \rbrace$.

For each $\mu \in (\mu _1, \dots , \mu _n) \in \Lambda _n^+$, we define

$$\begin{aligned} {\tilde{\mu }}:= & {} (\mu _{1}+1, \dots , \mu _n+1);\\ \vert \mu \vert:= & {} \mathop {\sum }\nolimits _{i=1}^{n} \mu _n;\\ n(\mu ):= & {} \mathop {\sum }\nolimits _{i=1}^{n}(i-1)\mu _i;\\ \mu _i':= & {} \vert \lbrace j \vert \mu _j \ge i \rbrace \vert , \text { } (i \ge 1);\\ \pi ^{\mu }:= & {} {{\,\mathrm{diag}\,}}(\pi ^{\mu _1}, \dots , \pi ^{\mu _n}). \end{aligned}$$

Furthermore, for $\mu \in \Lambda _n^+$, $\xi \in \Lambda _m^+$, we define

$$\begin{aligned} \langle \xi ', \mu ' \rangle :=\sum _{i \ge 1} \xi _i'\mu _i'. \end{aligned}$$

For $\mu , \lambda \in \Lambda _n^+$, we denote by $\mu \le \lambda$ if $\mu _i \le \lambda _i$ for all $1 \le i \le n$.

For $u \ge v \ge 0$, we define

$$\begin{aligned} \left[ \begin{array}{l} u\\ v \end{array}\right] :=\dfrac{\prod _{i=1}^u (1-(-q)^{-i})}{\prod _{i=1}^v (1-(-q)^{-i}) \prod _{i=1}^{u-v} (1-(-q)^{-i})} \end{aligned}$$

Finally, for $\mu , \lambda \in \Lambda _n^+$, we define

$$\begin{aligned} I_j(\mu , \lambda ):=\sum _{i=\mu _{j+1}'}^{\min (({\tilde{\lambda }})'_{j+1}, \mu _j')}(-q)^{i(2({\tilde{\lambda }})'_{j+1}+1-i)/2}\left[ \begin{array}{l} ({\tilde{\lambda }})_{j+1}'-\mu _{j+1}'\\ ({\tilde{\lambda }})_{j+1}'-i \end{array}\right] \left[ \begin{array}{l} ({\tilde{\lambda }})_{j}'-i\\ ({\tilde{\lambda }})_{j}'-\mu _{j}' \end{array}\right] . \end{aligned}$$

Now, we can state the following proposition.

Proposition A.5

([2, Theorem II]) For $\lambda \in \Lambda _n^+$ and $\xi \in \Lambda _m^+$ with $m \ge n$, we have

$$\begin{aligned} \alpha (\pi ^{\xi },\pi ^{\lambda })=\sum _{\begin{array}{c} \mu \in \Lambda _n^+\\ \mu \le {\tilde{\lambda }} \end{array}}(-1)^{\vert \mu \vert }(-q)^{-n(\mu )+(n-m-1)\vert \mu \vert +\langle \xi ',\mu ' \rangle } \prod _{j \ge 1} I_j(\mu ,\lambda ). \end{aligned}$$

Proof of Proposition A.4

In our case, $\lambda =(0,0, \dots , 0) \in \Lambda _n^+$ and $\xi =(1^k,0^{m-k}) \in \Lambda _m^+$. Therefore, ${\tilde{\lambda }}=(1^n) \in \Lambda _n^+$ and hence $\mu$ runs over $(1^l,0^{n-l}) \in \Lambda _n^+$. We write $^l\mu$ for $(1^l,0^{n-l})$.

First, we need to compute $\prod _{j \ge 1} I_j(^l\mu ,\lambda )$.

For $j=1$, we have

$$\begin{aligned} I_1(^l\mu , \lambda ):=\sum _{i=^l\mu _{2}'}^{\min (({\tilde{\lambda }})'_{2}, ^l\mu _1')}(-q)^{i(2({\tilde{\lambda }})'_{2}+1-i)/2}\left[ \begin{array}{l} ({\tilde{\lambda }})_{2}'- (^l\mu )_{2}'\\ ({\tilde{\lambda }})_{2}'-i \end{array}\right] \left[ \begin{array}{l} ({\tilde{\lambda }})_{1}'-i\\ ({\tilde{\lambda }})_{1}'- (^l\mu )_{1}' \end{array}\right] . \end{aligned}$$

Since $({\tilde{\lambda }})'_2=0$, i should be 0. Therefore, we have

$$\begin{aligned} \begin{array}{l} I_1(^l\mu ,\lambda )=\left[ \begin{array}{c} 0\\ 0 \end{array}\right] \left[ \begin{array}{c} n \\ n-l \end{array} \right] =\dfrac{\prod _{i=1}^n (1-(-q)^{-i})}{\prod _{i=1}^{n-l} (1-(-q)^{-i}) \prod _{i=1}^{l} (1-(-q)^{-i})}. \end{array} \end{aligned}$$

For $j>1$, it is easy to see that $I_j(^l\mu ,\lambda )=1$.

Therefore,

$$\begin{aligned} \begin{array}{ll} \alpha (\pi ^{\xi },\pi ^{\lambda })&{}=\sum _{l=0}^{n}(-1)^{\vert ^l\mu \vert }(-q)^{-n(^l\mu )+(n-m-1)\vert ^l\mu \vert +\langle \xi ', ^l\mu ' \rangle } \prod _{j \ge 1} I_j( ^l\mu ,\lambda )\\ &{}=\sum _{l=0}^{n}(-1)^{\vert ^l\mu \vert }(-q)^{-n(^l\mu )+(n-m-1)\vert ^l\mu \vert +\langle \xi ', ^l\mu ' \rangle } \left[ \begin{array}{c} n \\ n-l \end{array}\right] . \end{array} \end{aligned}$$

Since $\vert ^l\mu \vert =l$, $n( ^l\mu )=\dfrac{l(l-1)}{2}$, and $\langle \xi ', ^l\mu ' \rangle =kl$, we have

$$\begin{aligned} \alpha (\pi ^{\xi },\pi ^{\lambda })=\sum _{l=0}^{n}(-1)^{l}(-q)^{-l(l-1)/2-(r+1)l+kl} \left[ \begin{array}{c} n \\ n-l \end{array}\right] . \end{aligned}$$

(A.0.6)

If $k=0$, it is well-known that

$$\begin{aligned} \begin{array}{l} \alpha (\pi ^{\xi },\pi ^{\lambda })=\prod _{l=1}^{n} (1-(-q)^{-l}X). \end{array} \end{aligned}$$

where $X=(-q)^{-r}$ (see [7, page 677]).

Therefore, by changing X to $(-q)^kX$, we have that (A.0.6) is equal to

$$\begin{aligned} \begin{array}{ll} \alpha (\pi ^{\xi },\pi ^{\mu })&{}=\prod _{l=1}^{n} (1-(-q)^{-l}(-q)^kX)\\ &{}=\prod _{l=1}^{n} (1-(-q)^{-l}(-q)^{k-r}). \end{array} \end{aligned}$$

$\square$

Now, we have the following proposition.

Proposition A.6

$$\begin{aligned} \begin{array}{ll} \dfrac{W_{n-1,n}'(B,0)}{W_{n,n}(A_n,0)}-\beta ^{n-1}_{n-1}\dfrac{W_{n-1,n-1}(B,0)}{W_{n,n}(A_n,0)}=\\ \dfrac{1}{q+1}\left\{ \dfrac{\alpha '\left( \left( \begin{array}{cc} 1_n &{} \\ &{} \pi \end{array}\right) ,B_1\right) }{\alpha (1_n,1_n)}-\dfrac{\alpha (1_{n+1},B_1)}{\alpha (1_{n+1},1_{n+1})}\right\} . \end{array} \end{aligned}$$

Proof

From (3.2.3), (A.0.3), (A.0.4), (A.0.5), we have

$$\begin{aligned} \begin{array}{l} W_{n-1,n}(B,r)=q^{-4n^2}q^{-2(n+1)}\alpha (\pi A_n^{[r]},1_{n-1})\alpha \left( \left( \begin{array}{c@{\quad }c@{\quad }c} \pi 1_n &{} &{}\\ &{} \pi ^2 &{}\\ &{} &{} \pi 1_{2r} \end{array}\right) ,\pi B_1\right) ;\\ W_{n-1,n-1}(B,0)=q^{-4n^2}\alpha (\pi A_{n-1},1_{n-1})\alpha (\pi 1_{n+1},\pi B_1);\\ W_{n,n}(A_n,r)=q^{-4n^2}\alpha (\pi A_n^{[r]},1_n)\alpha (\pi 1_{n+2r},\pi 1_n). \end{array} \end{aligned}$$

Note that

$$\begin{aligned} {{\,\mathrm{val}\,}}(\det (B)) \equiv n+1 ({{\,\mathrm{mod}\,}}2) \end{aligned}$$

and hence ${{\,\mathrm{val}\,}}(\det (B_1))\equiv 0 ({{\,\mathrm{mod}\,}}2)$. Therefore $\alpha (\left( \begin{array}{c@{\quad }c} \pi 1_n &{} \\ &{} \pi ^2 \end{array}\right) , \pi B_1)=0$.

This implies that

$$\begin{aligned} W_{n-1,n}'(B,0)=q^{-4n^2}q^{-2(n+1)}\alpha (\pi A_n,1_{n-1})\alpha '\left( \left( \begin{array}{c@{\quad }c@{\quad }c} \pi 1_n &{} \\ &{} \pi ^2 \end{array}\right) , \pi B_1\right) . \end{aligned}$$

By Proposition A.4, we have

$$\begin{aligned} \alpha (\pi A_n,1_{n-1})= & {} \prod _{l=1}^{n-1}(1-(-q)^{-(l+1)})\\ \alpha (\pi A_n,1_n)= & {} \prod _{l=1}^{n}(1-(-q)^{-l})\\ \alpha (\pi A_{n-1},1_{n-1})= & {} \prod _{l=1}^{n-1}(1-(-q)^{-l}). \end{aligned}$$

Therefore,

$$\begin{aligned} \begin{array}{ll} \dfrac{W_{n-1,n}'(B,0)}{W_{n,n}(A_n,0)}&{}=\dfrac{q^{-2(n+1)}\alpha (\pi A_n,1_{n-1})\alpha '\left( \left( \begin{array}{c@{\quad }c@{\quad }c} \pi 1_n &{} \\ &{} \pi ^2 \end{array}\right) , \pi B_1\right) }{\alpha (\pi A_n,1_n)\alpha (\pi 1_n,\pi 1_n)}\\ &{}=\dfrac{q^{-2(n+1)}\prod _{l=1}^{n-1}(1-(-q)^{-(l+1)})q^{(n+1)^2}\alpha '\left( \left( \begin{array}{c@{\quad }c@{\quad }c} 1_n &{} \\ &{} \pi \end{array}\right) , B_1\right) }{\prod _{l=1}^{n}(1-(-q)^{-l})q^{n^2}\alpha (1_n,1_n)}\\ &{}=\dfrac{1}{q+1} \dfrac{\alpha '\left( \left( \begin{array}{c@{\quad }c@{\quad }c} 1_n &{} \\ &{} \pi \end{array}\right) , B_1\right) }{\alpha (1_n,1_n)}. \end{array} \end{aligned}$$

Here, we used the fact that $\alpha (\pi C, \pi D)=q^{n^2}\alpha (C,D)$ where D is a $n \times n$ hermitian matrix.

Similarly,

$$\begin{aligned} \begin{array}{ll} \beta ^{n-1}_{n-1}\dfrac{W_{n-1,n-1}(B,0)}{W_{n,n}(A_n,0)}&{}=\beta ^{n-1}_{n-1}\dfrac{\alpha (\pi A_{n-1},1_{n-1})\alpha (\pi 1_{n+1}, \pi B_1)}{\alpha (\pi A_n,1_n)\alpha (\pi 1_n,\pi 1_n)}\\ \\ &{}=\beta ^{n-1}_{n-1}\dfrac{\prod _{l=1}^{n-1}(1-(-q)^{-l})q^{(n+1)^2}\alpha (1_{n+1}, B_1)}{\prod _{l=1}^{n}(1-(-q)^{-l})q^{n^2}\alpha (1_n,1_n)}\\ \\ &{}=\dfrac{1}{q+1} \dfrac{\alpha (1_{n+1}, B_1)}{\alpha (1_{n+1},1_{n+1})}. \end{array} \end{aligned}$$

Here, we used Proposition A.1 and

$$\begin{aligned} \dfrac{\alpha (1_{n+1},1_{n+1})}{\alpha (1_n,1_n)}=(1-(-q)^{-(n+1)}). \end{aligned}$$

$\square$

Note that $\dfrac{\alpha \bigg (\left( \begin{array}{c@{\quad }c} 1_n &{} \\ &{} \pi \end{array}\right) ,B_1\bigg )}{\alpha (1_n,1_n)}$ is equal to $\partial \text {Den}_{\Lambda }(L)$ and $\dfrac{\alpha (1_{n+1},B_1)}{\alpha (1_{n+1},1_{n+1})}$ is equal to $\text {Den}(L)$ in [10, Theorem 10.4.4] (but, our n is $n-1$ in loc. cit.). Therefore, Conjecture 3.25 is compatible with their result.