1 Introduction

In this work, we deal with some applications of the modular forms (and spectral functions related to the congruence subgroup of \(SL(2, {\mathbb {Z}})\)) to topological vertex and string generating functions. For mathematicians topological vertices (and their respective string generating functions) may be associated to new mathematical invariants for spaces, while, for physicists, they are related to quantum string partition functions.

Having made these general remarks, let us now explain the connection between the specific contents of the various sections in more detail. In Sect. 2 we will explore a remarkable connection between Poincaré polynomials (generating functions) and formal power series associated with dimensions of chains and homologies of Lie algebras (Euler–Poincaré formula). From a concrete point of view this paper consists of applications of spectral functions to quantum string partition functions connected to suitable Lie algebras.

We show that the Poincaré polynomials (Sect. 3) and the associated topological vertex and string amplitude can be converted into product expressions which inherit cohomological properties (in the sense of characteristic classes of foliations [1]) of appropriate polygraded Lie algebras.

The final result for a single Lagrangian brane, stack of branes (Sect. 4), refined vertex, and the partition functions for the case of Calabi–Yau threefolds \({\mathcal {O}}(-1)\oplus {\mathcal {O}}(-1)\mapsto {\mathbb {P}}^{1}\) and \({\mathcal {O}}(0)\oplus {\mathcal {O}}(-2)\mapsto {\mathbb {P}}^{1}\) (Sect. 5) is written in terms of spectral functions of the hyperbolic three-geometry associated with \(q\)-series.

2 Graded algebras and spectral functions of hyperbolic geometry

Before considering topological vertex and string amplitudes we would like to spend some time on the relation between formal power \(q\)-series and homologies of Lie algebras. We would like to show how combinatorial identities could be derived from the initial complex of (graded) Lie algebras. Our interest is the Euler–Poincaré formula associated with a complex consisting of linear spaces. The relations between Lie algebras and combinatorial identities was first discovered by Macdonald; the Euler–Poincaré formula is useful for combinatorial identities known as Macdonald identities. The Macdonald identities are related to Lie algebras in one way or another and can be associated with generating functions in quantum theory.

Let \({\mathfrak {g}}\) be a Lie algebra, and assume that it has a grading, i.e. \({\mathfrak {g}}\) is a direct sum of homogeneous components \({\mathfrak {g}}_{(\lambda )}\), where the \(\lambda \)’s are elements of an abelian group, \([{\mathfrak {g}}_{(\lambda )}, {\mathfrak {g}}_{(\mu )}] \subset {\mathfrak {g}}_{(\lambda + \mu )}\). Let us consider a module \(\mathbf{k}\) over \({\mathfrak {g}}\), or \({\mathfrak {g}}\)-module, which is a left module over the universal enveloping algebra \(U({\mathfrak {g}})\) of \({\mathfrak {g}}\). Let \(C^n({\mathfrak {g}}; \mathbf{k})\) be the space of an \(n\)-dimensional cochain of the algebra \({\mathfrak {g}}\), with coefficients in \(\mathbf{k}\). \(d= d_n : C^n({\mathfrak {g}}; \mathbf{k})\rightarrow C^{n+1}({\mathfrak {g}}; \mathbf{k})\) as \(d_{n+1}\circ d_n =0\), for all \(n\), the set \(C^{\sharp }({\mathfrak {g}}; \mathbf{k}) \equiv \{C^n({\mathfrak {g}}; \mathbf{k}), d_n\}\) is an algebraic complex, while the corresponding cohomology \(H^n({\mathfrak {g}}; \mathbf{k})\) is referred to as the cohomology of the algebra \({\mathfrak {g}}\) with coefficients in \(\mathbf{k}\).

Let \(C_n({\mathfrak {g}}; \mathbf{k})\) be the space of \(n\)-dimensional chains of the Lie algebra \({\mathfrak {g}}\) and \(\delta = \delta _n: C_n({\mathfrak {g}}; \mathbf{k})\rightarrow C_{n-1}({\mathfrak {g}}; \mathbf{k})\). The homology \(H_n({\mathfrak {g}}; \mathbf{k})\) of the complex \(\{C_n({\mathfrak {g}}; \mathbf{k}), \delta _n\}\) is referred to as the homology of the algebra \({\mathfrak {g}}\). We get \(d(C^n_{(\lambda )}({\mathfrak {g}}; \mathbf{k})) \subset C^{n+1}_{(\lambda )}({\mathfrak {g}}; \mathbf{k})\) and \(\delta (C_n^{(\lambda )}({\mathfrak {g}}; \mathbf{k})) \subset C_{n-1}^{(\lambda )}({\mathfrak {g}}; \mathbf{k})\) and both spaces acquire gradings. The chain complex \(C_{\sharp }({\mathfrak {g}})\),  \({\mathfrak {g}}= \oplus _{\lambda =1}^{\infty }{\mathfrak {g}}_{(\lambda )},\, \mathrm{dim}\,{\mathfrak {g}}_{(\lambda )}< \infty \), can be decomposed as

$$\begin{aligned} 0\longleftarrow C_{0}^{(\lambda )}({\mathfrak {g}}) \longleftarrow C_{1}^{(\lambda )}({\mathfrak {g}}) \cdots \longleftarrow C_N^{(\lambda )}({\mathfrak {g}}) \longleftarrow 0 \end{aligned}$$
(2.1)

The well-known Euler–Poincaré formula reads

$$\begin{aligned} \sum _m(-1)^m \mathrm{dim}\,C_m^{(\lambda )}({\mathfrak {g}}) = \sum _m(-1)^m \mathrm{dim}\,H_m^{(\lambda )}({\mathfrak {g}}). \end{aligned}$$
(2.2)

As a consequence, we can introduce the \(q\) variable and rewrite the identity (2.2) as a formal power series [2, 3]:

$$\begin{aligned} \sum _{m,\lambda }(-1)^m q^\lambda \mathrm{dim}\,C_m^{(\lambda )}({\mathfrak {g}})&= \sum _{m,\lambda }(-1)^m q^\lambda \mathrm{dim}\,H_m^{(\lambda )}({\mathfrak {g}})\nonumber \\&= \prod _n(1-q^n)^{\mathrm{dim}\,{\mathfrak {g}}_{n}}. \end{aligned}$$
(2.3)

2.1 Spectral generating functions for string amplitudes

Having given the general scheme let us now produce, for the benefit of the reader, the specific correspondence of the Poincaré series (which can be associated with a conformal structure in two dimensions [4]) and spectral functions of the three-dimensional hyperbolic geometry. Thus, the infinite product identities are [1, 3, 5]:

$$\begin{aligned}&\prod _{n=\ell }^{\infty }(1- q^{\mu n+\varepsilon }) = \prod _{p=0, 1}Z_{\Gamma }(\underbrace{(\mu \ell +\varepsilon )(1-i\varrho (\tau )) + 1 -\mu }_s\nonumber \\&\qquad + \,\,\mu (1 + i\varrho (\tau )p)^{(-1)^p}\nonumber \\&\quad = {\mathcal {R}}(s = (\mu \ell + \varepsilon )(1-i\varrho (\tau )) + 1-\mu ),\end{aligned}$$
(2.4)
$$\begin{aligned}&\prod _{n=\ell }^{\infty }(1+ q^{\mu n+\varepsilon }) \nonumber \\&\quad = \prod _{p=0, 1}Z_{\Gamma }(\underbrace{(\mu \ell +\varepsilon )(1-i\varrho (\tau )) + 1-\mu + i/(2\,\mathrm{Im}\,\tau )}_s\nonumber \\&\qquad +\,\, \mu (1+ i\varrho (\tau )p))^{(-1)^p} \nonumber \\&\quad = {\mathcal {R}}(s \!=\! (\mu \ell \!+\! \varepsilon )(1-i\varrho (\tau )) + 1-\mu \!+\! \,\,i/(2\,\mathrm{Im}\,\tau )), \nonumber \\ \end{aligned}$$
(2.5)

where \(q=\exp (2\pi i\tau ),\, \varrho (\tau )= \mathrm{Re}\tau /\mathrm{Im}\tau ,\, \mu \in {\mathbb R}\), \(\ell \in {\mathbb Z}_+\) and \(\varepsilon \in {\mathbb C}\). The Ruelle function \({\mathcal R}(s)\) is an alternating product of more complicated factors, each of which is a Selberg-type spectral function \(Z_{\Gamma }(s)\) (the reader can find the analytic and modular properties of the Patterson–Selberg spectral function \(Z_{\Gamma }(s)\) in [1]).

At this point one can use the Ruelle functions \({\mathcal {R}}(s)\) to naturally generalize the result (2.4), (2.5) for more general infinite product identities

$$\begin{aligned}&\prod _{n=m}^{\infty }\,\big (1-q^{\mu \,n+ \varepsilon }\big )^{\nu \,n} \nonumber \\&\quad = {\mathcal {R}}\big (s=(\mu \, m + \varepsilon )\, (1-i\varrho (\tau ))+1-\mu \big )^{\nu \, m} \nonumber \\&\qquad \times \!\!\!\prod _{n=m+1}^{\infty } \, {\mathcal {R}}\big (s=(\mu \, n \!+\! \varepsilon )\, (1-i\varrho (\tau ))+1-\mu \big )^{\nu }, \end{aligned}$$
(2.6)
$$\begin{aligned}&\prod _{n=m}^{\infty }\, \big (1+q^{\mu \,n+ \varepsilon }\big )^{\nu \, n} \nonumber \\&\quad = {\mathcal {R}}\big (s = (\mu \,m \!+\! \varepsilon )\, (1-i\varrho (\tau )) \!+\! 1-\mu \!+\! i/(2\,\mathrm{Im}\,\tau )\big )^{\nu \,m} \nonumber \\&\qquad \times \!\!\prod _{n=m+1}^{\infty } \, {\mathcal {R}}\big (s = (\mu \,n + \varepsilon )\, (1-i\varrho (\tau ))\nonumber \\&\qquad + 1-\mu + i/(2\,\mathrm{Im}\,\tau )\big )^{\nu }. \end{aligned}$$
(2.7)

3 Polygraded algebras and polynomial invariants

Let \({\mathfrak {g}}\) be a polygraded Lie algebra, \({\mathfrak {g}} = \bigoplus _{\begin{array}{c} \scriptstyle \lambda _1\ge 0,\ldots , \lambda _k\ge 0 \\ \scriptstyle \lambda _1+\cdots +\lambda _k>0 \end{array}} {\mathfrak {g}}_{(\lambda _1,\ldots , \lambda _k)},\) satisfying the condition \(\mathrm{dim}\, {\mathfrak {g}}_{(\lambda _1,\ldots , \lambda _k)} < \infty .\) For formal power series in \(q_1,\ldots ,q_k\), we have the following identity:

$$\begin{aligned}&\sum _{m,\lambda _1,\ldots ,\lambda _k}(-1)^m q_1^{\lambda _1} \cdots q_k^{\lambda _k} H_m^{(\lambda _1,\ldots ,\lambda _k)}\nonumber \\&\quad =\prod _{n_1,\ldots ,n_k}\left( 1-q_1^{n_1}\cdots q_k^{n_k}\right) ^{\mathrm{dim}\, {\mathfrak {g}}_{n_1,\ldots , n_k}}. \end{aligned}$$
(3.1)

We would like to stress that partition functions can indeed be converted into product expressions. The expression on the right-hand side of (3.1) looks like counting the states in the Hilbert space of a second quantized theory.

For more examples let us proceed to describing the properties of link homologies suggested by their relation to Hilbert spaces of BPS states [6]. Let \({\mathcal {H}}_{k,j}^{{{\mathfrak {sl}}(N)}; R_1, \ldots , R_\ell }(L)\) be the doubly graded homology theory whose graded Euler characteristic is the polynomial invariant \(\overline{P}_{{\mathfrak {sl}}(N);R_1, \ldots , R_{\ell }} (q)\) (the bar means that this invariant is unnormalized invariant; its normalized version obtained by dividing by the invariant of the unknot)

$$\begin{aligned} {\overline{P}}_{{\mathfrak {sl}}(N);R_1, \ldots , R_{\ell }} (q) = \sum _{k,j \in \mathbb {Z}} (-1)^j q^k \dim {\mathcal {H}}_{k,j}^{{{\mathfrak {sl}}(N)}; R_1, \ldots , R_\ell }(L). \end{aligned}$$
(3.2)

Here \(L\) is an oriented link in \(S^3\); we consider the Lie algebra \({\mathfrak {g}} = {\mathfrak {sl}}(N)\) (there is a natural generalization to the other classical Lie algebras B, C, and D [6]) and a colored link is given by a collection of representations \(R_1, \ldots , R_\ell \) of \({\mathfrak {sl}}(N)\). In order to agree with the standard notations we use the variables \(\{q,t\}\), [7], when we discuss link homologies.

The graded Poincaré polynomial has the form [6]

$$\begin{aligned} \overline{{P}}_{{\mathfrak {sl}}(N);R_1, \ldots , R_{\ell }} (q,t) := \sum _{k,j \in \mathbb {Z}} q^k t^j \dim {\mathcal {H}}_{k,j}^{{\mathfrak {sl}(N)}; R_1, \ldots , R_\ell }(L). \end{aligned}$$
(3.3)

By definition, it is a polynomial in \(q^{\pm 1}\) and \(t^{\pm 1}\) with integer non-negative coefficients. In addition, evaluating (3.3) at \(t=-1\) gives (3.2). In the case \(R_a = \Box \) for all \(a=1, \ldots , \ell \), the homology \({\mathcal {H}}_{k,j}^{{\mathfrak {sl}(N)}; R_1, \ldots , R_\ell }(L)\) is known as the Khovanov–Rozansky homology, \({{}_{(KR)}\overline{H}}_{k,j}^{N} (L)\), and its graded Poincaré polynomial takes the form

$$\begin{aligned} \overline{\mathcal {P}}_{{\mathfrak {sl}}(N);\Box , \ldots , \Box } (q,t) = \sum _{k,j \in \mathbb {Z}} q^k t^j \dim {}_{(KR)}{\overline{H}}_{k,j}^N (L). \end{aligned}$$
(3.4)

In order to get this expression in its final form the homology \({}_{(KR)}{\overline{H}}_{k,j}^N (L)\) has to be computed. The further physical interpretation of homological link invariants via Hilbert spaces of BPS states leads to certain predictions regarding the behavior of link homologies with rank \(N\) (for more discussion see [6, 8]).

4 A single Lagrangian brane

Let us recall the connection between topological vertex and open string amplitudes in the presence of a stack of branes. It is known that in the case of a stack of Lagrangian D-branes end on one of the legs of the \(\mathbb {C}^{3}\) (in this section and later on we will mostly follow [9] in the notations and reproduction of necessary results). The partition function is given by

$$\begin{aligned} {\mathcal F}(q;V)=\sum _{\nu }C_{\emptyset \,\emptyset \,\nu }(q^{-1})\,\text{ Tr }_{\nu }V. \end{aligned}$$
(4.1)

Here \(\text{ Tr }_{\nu }V=s_{\nu }(\mathbf{x})\) are the Schur functions, \(\mathbf{x}=\{x_{1},x_{2},\ldots \}\) are the eigenvalues of the holonomy matrix \(V\), and \(C_{\lambda \mu \nu }(q)\) is the topological vertex [10]. It is well known that the topological vertex has a combinatorial interpretation in terms of counting certain 3D partitions with fixed asymptotes [11]. Recall that Schur functions have the property that \(s_{\nu /\lambda }(Q)= Q^{|\nu |-|\lambda |}\), \(\nu \succ \lambda \), and \(s_{\nu /\lambda }(Q)= 0\), otherwise.

For a single Lagrangian brane \(\mathbf{x}=(-Q,0,0,0,\ldots )\) we get the well-known partition function

$$\begin{aligned} {\mathcal {F}}(q;Q)&= \prod _{n=1}^{\infty }(1-Q\,q^{-n+\frac{1}{2}})\mathop {=\!=\!=\!=\!=\!=}\limits ^{\mathrm{by\ Eq.\ (2.4)}}\nonumber \\&\quad \times {\mathcal R}(s =(\alpha -1/2)(1-i\varrho (\tau ))+2). \end{aligned}$$
(4.2)

In order proceed carefully and to agree with the standard notations, we use the variables \(\{Q, q, t\}\) when we talk about topological string amplitudes computed by the topological vertex, cf. [9]. This set of variables are related with our notations \(\{Q^\alpha , q= \exp (2\pi i\tau ), t=\exp (2\pi i\sigma )\}\) as follows: \(\tau = F_1/2\pi i,\, \sigma = F_2/F_1\), where we turn on a constant not self-dual graviphoton field strength \(F = F_1dx^1\wedge dx^2+F_2dx^3\wedge dx^4\), and \(\alpha = -\int _C\omega /F_1\) [9].

The above partition function of a single Lagrangian brane can be interpreted in terms of the Hilbert series of the symmetric product of \(\mathbb {C}\). Indeed, \(s_{\nu }(Q)\) is non-zero only for those partitions for which \(\ell (\nu )=1\). These are exactly the partitions which label the fixed points of the symmetric product of \(\mathbb {C}\), i.e., \(\text{ Sym }^{\bullet }(\mathbb {C})\) has a single fixed point labeled by the partition \(\nu =(\bullet ,0,0,\ldots )\). A generating function of the Hilbert series of the symmetric product is [12]

$$\begin{aligned} {\mathfrak {G}}(\psi ,q):=\sum _{k=0}^{\infty }\psi ^{k}\,H[\,\text{ Sym }^{k}(\mathbb {C})](q). \end{aligned}$$
(4.3)

In order to determine \(H[\,\text{ Sym }^{k}(\mathbb {C})](q)\) note that the \(R_k\) is the ring of symmetric functions in the variables \(\{z_1, z_2,\ldots , z_k\}\) and therefore the Schur functions provide a basis of \(R_k\), \(R_k = \langle s_\nu (z_1,\ldots , z_k) \arrowvert \ell (\nu )\le k\rangle \). The condition \(\ell (\nu )\le k\) is necessary since \(s_\nu (z_1,\ldots , z_k)=0\) for \(\ell (\nu )> k\). \(R_{k}\) is isomorphic to the Hilbert space \({\mathcal {H}}_{k}\), generated by a bosonic oscillator up to charge \(k\). The Hilbert spaces \(\{{\mathcal {H}}_{k}\}_{k=0}^\infty \) form a nested sequence \({\mathcal {H}}_{0}\subset {\mathcal {H}}_{1}\subset {\mathcal {H}}_{2}\subset {\mathcal {H}}_{3}\subset \cdots \), which corresponds to the nested sequence of Young diagrams of increasing number of rows.

The \(\mathbb {C}^{\times }\) action (on \(\mathbb C\) \(q\) acts as a \(\mathbb {C}^{\times }\) action \(z\mapsto qz\)) becomes the action of \(q^{L_{0}}\) on the states in \({\mathcal {H}}\) \((L_{0}=\sum _{n>0}\alpha _{-n}\alpha _{n}\)). It lifts to an action on the \(\text{ Sym }^{\bullet }(\mathbb {C})\) such that the Schur functions \(s_{\nu }(z_{1},\ldots ,z_{k})\) are eigenfunctions with eigenvalue \(q^{|\nu |}\). We have

$$\begin{aligned} H[R_{k}](q)&= \text{ Tr }_{\mathcal{H}_{k}}q^{L_{0}}=\sum _{\nu |\ell (\nu )\le k}\,q^{|\nu |}= \prod _{n=1}^{k}(1-q^{n})^{-1}\nonumber \\&= s_{(k)}(1,q,q^{2},\ldots ) \nonumber \\&= \left[ \frac{{\mathcal R}(s=1-i\varrho (\tau ))}{{\mathcal R}(s=(k+1)(1-i\varrho (\tau )))}\right] ^{-1}\!\!. \end{aligned}$$
(4.4)

Note that the Hilbert series of \(R_{k}\) in this case turns out to be the generating function of the number of partitions with at most \(k\) parts. Thus the generating function \({\mathfrak {G}}(\psi ,q)\) is then given by

$$\begin{aligned} {\mathfrak {G}}(\psi ,q)&= \sum _{k=0}^{\infty }\psi ^{k}H[R_{k}](q)=\sum _{k=0}^{\infty }\psi ^{k}\text{ Tr }_{{\mathcal {H}}_{k}}q^{L_{0}}\nonumber \\&= \sum _{k=0}^{\infty }\psi ^{k}s_{(k)}(1,q,q^{2},\ldots )\nonumber \\&= \sum _{k=0}^{\infty }s_{(k)}(\psi )s_{(k)}(1,q,q^{2},\ldots )\nonumber \\&= \sum _{\nu }s_{\nu }(\psi )s_{\nu }(1,q,q^{2},\ldots )\nonumber \\&= \sum _{\nu }s_{\nu }(q^{-\rho })s_{\nu }(\psi \,q^{-\frac{1}{2}})\nonumber \\&= \sum _{\nu }s_{\nu ^{t}}(q^{\rho })\,s_{\nu }(-q^{-\frac{1}{2}}\psi )\nonumber \\&= \sum _{\nu }C_{\emptyset \,\emptyset \,\nu }(q^{-1})\,\text{ Tr }_{\nu }V = {\mathcal F}(q; Q). \end{aligned}$$
(4.5)

Here \(\text{ Tr }_{\nu }V=s_{\nu }(Q)\) and \(Q=q^{-\frac{1}{2}}\psi \). \(\text{ Tr }_{\nu }V=s_{\nu }(\mathbf{x})\) where \(\mathbf{x}=\{x_{1},x_{2},\ldots \}\) are the eigenvalues of the holonomy matrix \(V\). Thus the partition function takes the form

$$\begin{aligned}&{\mathcal F}(q;V)=\sum _{\nu }C_{\emptyset \,\emptyset \,\nu }(q^{-1})\,s_{\nu }(\mathbf{x}) = \sum _{\nu }s_{\nu ^{t}}(q^{\rho })s_{\nu }(\mathbf{x})\nonumber \\&\quad = \prod _{k,j=1}^{\infty }(1+q^{-k+\frac{1}{2}}x_{j})\nonumber \\&\quad \mathop {=\!=\!=\!=\!=\!=}\limits ^{\mathrm{by\ Eq.\ (2.5)}} \prod _{j=1}^\infty {\mathcal R}(s= (a_j-1/2)(1-i\varrho (\tau ))+2\nonumber \\&\qquad +i/(2\mathrm{Im}\tau )), \end{aligned}$$
(4.6)

where \(a_j\equiv \mathrm{log}\,x_j/\mathrm{log}\,q\). If we move the brane to infinity (\(Q=e^{-\int _C\omega }\mapsto 0\)) the contribution of the higher modes is suppressed. On the other hand as the brane moves toward the origin (\(Q\mapsto 1\)) higher oscillator modes start contributing with equal weight to the partition function. It also follows that the topological vertex \(C_{\emptyset \,\emptyset \,(k)}(q)\) has an interpretation as counting the number of states of a given energy in the Hilbert space \({\mathcal {H}}_{k}\). It is tempting to conjecture that the topological vertex with all three partitions non-trivial has a similar interpretation in terms of spectral functions.

4.1 Stack of branes

Let us consider the case of multiple Lagrangian branes on one of the legs of \(\mathbb {C}^{3}\). Then \(\mathbf{x} = \{x_1, x_2,\ldots , x_N \}\), and the partition function becomes

$$\begin{aligned} {\mathcal F}(\mathbf{x},q)&= \sum _{\nu }C_{\emptyset \,\emptyset \,\nu }(q^{-1})\,s_{\nu }(\mathbf{x}) = \prod _{j=1}^{N}\prod _{k=1}^{\infty }(1+q^{-k+\frac{1}{2}}x_{j})\nonumber \\&= \prod _{j=1}^{N}{\mathcal R}(s= (a_j-1/2)(1-i\varrho (\tau ))\nonumber \\&+\,\,2+i/(2\mathrm{Im}\tau )). \end{aligned}$$
(4.7)

It is clear that the partition function (4.7) is the generating function of the Hilbert series of a product of symmetric products of \(\mathbb {C}\).

5 Topological vertex and string amplitudes

5.1 Refined vertex and open string partition function

In this section we will explain the (combinatorial) interpretation of the vertex in terms of 3D partitions. The refined vertex has an interpretation in terms of generalized open topological string amplitudes in the presence of stacks of A-brane. This vertex can be viewed as a building block for the computation of Khovanov knot invariants that can be obtained from local toric Calabi–Yau manifolds. The stack of D-branes in the context of refined vertex can also be related to the symmetric product of \(\mathbb {C}\), as it was possible in the context of ordinary vertex. The refined topological vertex also has a combinatorial interpretation in terms of 3D partitions [9].

Note that the open string partition function depends on the leg on which the stack of branes is put. Essentially we have three choices for the open string partition function, corresponding to the three legs of \(\mathbb {C}^{3}\) [9]: \(\mathbf{(1)}\,\, C_{\lambda \,\emptyset \,\emptyset }(t,q) = (q/t)^{|\lambda |/2}\,s_{\lambda ^{t}}(t^{-\rho })\,; \,\, \mathbf{(2)}\,\, C_{\emptyset \,\mu \,\emptyset }(t,q) = (q/t)^{(||\mu ^{t}||^{2}-|\mu |)/2}\,s_{\mu ^{t}}(q^{-\rho })\,; \,\,\mathbf{(3)}\,\, C_{\emptyset \,\emptyset \,\nu }(t,q)=q^{||\nu ||^{2}/2}/\) \(\prod _{s\in \nu }(1-t^{1+a(s)}\,q^{\ell (s)}).\)

In the first two cases the partition function is the same as the partition function obtained from the ordinary vertex except that the partition function depends on either \(t\) or \(q\), depending on the leg on which the brane ends. The third case is more interesting, the brane can end on the preferred leg. In the case \(\mathbf{x}= \{-Q, 0, 0,\ldots \}\) the open string amplitude is given by

$$\begin{aligned}&{\mathcal F}(Q,t,q)=\sum _{\nu }C_{\emptyset \,\emptyset \,\nu }(t^{-1},q^{-1})\,s_{\nu }(-Q)\nonumber \\&\quad =\sum _{k=0}^{\infty }C_{\emptyset \,\emptyset \,(k)}(t^{-1},q^{-1})(-Q)^{k}\nonumber \\&\quad = \sum _{k=0}^{\infty }\Big (Q\frac{t}{\sqrt{k}}\Big )^{k}\,\prod _{n=1}^{k}(1-t\,q^{n-1})^{-1}\nonumber \\&\quad \mathop {=\!=\!=\!=}\limits ^{t:=\, q^\sigma } \sum _{k=0}^{\infty }\Big (Q\frac{q^\sigma }{\sqrt{k}}\Big )^{k}\nonumber \\&\qquad \times \,\,\left[ \frac{{\mathcal R}(s= (\sigma -1)(1-i\varrho (\tau )))}{{\mathcal R}(s= (k+\sigma )(1-i\varrho (\tau )))}\right] ^{-1}. \end{aligned}$$
(5.1)

The above partition function can also be written using a more refined Hilbert series of the symmetric product of \(\mathbb {C}\).

5.2 The case of the Calabi–Yau threefold

We have \({\mathcal X}={\mathcal O}(-1)\oplus {\mathcal O}(-1)\mapsto \mathbb {P}^{1}\). As before one can use the refined topological vertex to determine the generalized partition function for various local toric Calabi–Yau threefolds. The compactification of Type IIA string theory on the \({\mathcal X}\mapsto \mathbb {P}^{1}\) gives rise to \(U(1)\) \({\mathcal N}=2\) gauge theory on the transverse \(\mathbb {C}^{2}\) in a particular limit [13]. The topological string partition function is given by

$$\begin{aligned}&{\mathcal F}(q,Q)=\sum _{\nu }Q^{|\nu |}(-1)^{|\nu |}\,C_{\emptyset \,\emptyset \,\nu }(q)\,C_{\emptyset \,\emptyset \,\nu ^{t}}(q)\nonumber \\&\quad = \sum _{\nu }Q^{|\nu |}(-1)^{|\nu |}s_{\nu ^{t}}(q^{-\rho })s_{\nu }(q^{-\rho })\nonumber \\&\quad =\prod _{k,j=1}^{\infty }\Big (1-Q\,q^{k+j-1}\Big )=\prod _{n=1}^{\infty }\Big (1-q^{n}\,Q\Big )^{n}\nonumber \\&\quad \mathop {=\!=\!=\!=\!=\!=}\limits ^{\mathrm{by\ Eq.\ (2.6)}} \prod _{n=1}^\infty {\mathcal R}(s= (n+ \alpha )(1-i\varrho (\tau ))). \end{aligned}$$
(5.2)

Here \(-\text{ log }\,Q = \int _C\omega \) is the Kähler parameter, the size of the \(\mathbb {P}^{1}\). Thus the refined topological vertex can be used to determine the refined partition function. The other choice of the refined partition function is

$$\begin{aligned} {\mathcal F}(t,q,Q)&= \sum _{\lambda }Q^{|\lambda |}(-1)^{|\lambda |}\,C_{\lambda \,\emptyset \,\emptyset }(t,q)\,C_{\lambda ^{t}\,\emptyset \,\emptyset }(q,t)\nonumber \\&= \sum _{\lambda }(-Q)^{|\lambda |}\Big (\tfrac{q}{t}\Big )^{\frac{|\lambda |}{2}}\,s_{\lambda ^{t}}(t^{-\rho })\,\Big (\tfrac{t}{q}\Big )^{\frac{|\lambda |}{2}}\,s_{\lambda }(q^{-\rho })\,\nonumber \\&= \sum _{\lambda }(-Q)^{|\lambda |}s_{\lambda ^{t}}(t^{-\rho })\,s_{\lambda }(q^{-\rho })\nonumber \\&= \prod _{k,j=1}^{\infty }(1-Q\,q^{k-\frac{1}{2}}\,t^{j-\frac{1}{2}})\,\nonumber \\&= \prod _{j=1}^\infty {\mathcal R}(s= (\alpha + \sigma (j-1/2)-1/2)\nonumber \\&\quad \times (1-i\varrho (\tau ))). \end{aligned}$$
(5.3)

5.3 The case of the Calabi–Yau threefold

We have \({\mathcal X}={\mathcal O}(0)\oplus {\mathcal O}(-2)\mapsto \mathbb {P}^{1}\). One can obtain this geometry from local \(\mathbb {P}^{1}\times \mathbb {P}^{1}\) by taking the size of one of the \(\mathbb {P}^{1}\) very large and get two copies of \({\mathcal O}(0)\oplus {\mathcal O}(-2)\mapsto \mathbb {P}^{1}\).

In the topological vertex formalism the partition functions take the form

$$\begin{aligned}&{\mathcal F}(q,Q)\!=\!\sum _{\nu }Q^{|\nu |}(-1)^{|\nu |}C_{\emptyset \,\emptyset \,\nu }(q)\,(-1)^{|\nu |}\,q^{\frac{\kappa (\nu )}{2}}\,C_{\emptyset \,\emptyset \,\nu ^{t}}(q)\nonumber \\&\quad = \sum _{\nu }Q^{|\nu |}s_{\nu ^{t}}(q^{-\rho })\,q^{\frac{\kappa (\nu )}{2}}\,s_{\nu }(q^{-\rho })\nonumber \\&\quad = \sum _{\nu }Q^{|\nu |}s_{\nu ^{t}}(q^{-\rho })\,s_{\nu ^t}(q^{-\rho })\nonumber \\&\quad = \prod _{k,j=1}^{\infty }\Big (1-Q\,q^{k+j-1}\Big )^{-1}\,=\prod _{n=1}^{\infty }\Big (1-Q\,q^{n}\Big )^{-n}\nonumber \\&\quad = \prod _{n=1}^\infty \left[ {\mathcal R}(s= (n+\alpha )(1-i\varrho (\tau )))\right] ^{-1}.\end{aligned}$$
(5.4)
$$\begin{aligned}&{\mathcal F}(t,q,Q)\!=\!\sum _{\lambda }\!Q^{|\lambda |}(-1)^{|\lambda |}\!C_{\emptyset \,\lambda \,\emptyset }(t,q)\,f_{\lambda }(t,q)\,C_{\lambda ^{t}\,\emptyset \,\emptyset }(t,q)\nonumber \\&\quad = \sum _{\lambda }(-Q)^{|\lambda |}\Big (\frac{q}{t}\Big )^{\frac{||\lambda ||^2}{2}} t^{\frac{\kappa (\lambda )}{2}}\,s_{\lambda ^{t}}(q^{-\rho })\,f_{\lambda }(t,q)\,s_{\lambda }(t^{-\rho }) \nonumber \\&\quad = \sum _{\lambda }(Q\sqrt{\tfrac{q}{t}})^{|\lambda |}\,s_{\lambda ^{t}}(t^{-\rho })\,s_{\lambda ^{t}}(q^{-\rho })\nonumber \\&\quad = \prod _{k,j=1}^{\infty }\Big (1-Q\,q^{k}\,t^{j-1}\Big )^{-1}\nonumber \\&\quad = \prod _{j=1}^\infty \left[ {\mathcal R}(s= (1+\alpha +\sigma (j-1))(1-i\varrho (\tau )))\right] ^{-1}.\nonumber \\ \end{aligned}$$
(5.5)

6 Conclusions

The main result and the central concept in all examples here are that the quantum generating functions considered in this paper can be converted into products of spectral functions associated with \(q\)-series. This common feature encodes the connection with infinite-dimensional Lie algebras and their homologies, together with the remarkable link to hyperbolic geometry. The Poincaré polynomials and the associated topological vertices and string amplitudes can be converted into product expressions which inherit modular and homological properties (in the sense of characteristic classes of foliations, as has been stressed for the first time in [1]) of appropriate polygraded algebras.