3d Theories and Modularity

Abstract

In this section, we connect the brane setup of the above 2d A-model to 3d/3d correspondence and shed light on various modular representations coming from geometry.

In this section, we connect the brane setup of the above 2d A-model to 3d/3d correspondence and shed light on various modular representations coming from geometry. In particular, we explain the origin for the explicit form of the S and T matrices in Conjectures 2.1 and 2.2. The modular action in Conjecture 2.1 turns out to be the one of refined Chern–Simons theory [4]. On the other hand, the modular action in Conjecture 2.2 is a “hidden” (surprising) one; it is realized on the vector space spanned by the set of connected components of fixed points under the Hitchin $\text {U}(1)_{\beta }$ action on the moduli space of wild Higgs bundles associated to a certain Argyres–Douglas theory. Furthermore, we propose how non-standard (e.g. logarithmic) modular data of ${{\,\mathrm{\textsf{MTC}}\,}}[M_3]$ can be described in terms of the A-model on the Hitchin moduli space associated with the Heegaard decomposition of $M_3$ and discuss possible connections to skein modules of closed oriented 3-manifolds.

One advantage of connecting the 2d A-model to the three-dimensional perspective is that all of these modular actions admit a natural categorification. In other words, in all of these instances it makes sense to ask if the space of open strings in the Hitchin moduli space can be realized as the Grothendieck groups of a tensor category (possibly, non-unitary or non-semisimple):

Finally, we will see that, in the opposite direction, the relation to the 2d A-model offers a unifying home for the above-mentioned modular data.

3.1 DAHA and Modularity

The fivebrane system in M-theory that provides geometric origins of the modular representations on DAHA modules is the following familiar setting for the 3d/3d correspondence

$$\begin{aligned} \begin{matrix} {\text{ space-time: }} &{} \quad &{} S^1 &{} \times _{q,t} &{} \big (TN &{} \times &{}T^* M_3\big ) \\ \textit{N}~{\text{ M5-branes: }} &{} \quad &{} S^1 &{}\times _q&{} D^2 &{} \times &{} M_3 \end{matrix} \end{aligned}$$

(3.1)

where $M_3$ is a 3-manifold, $D^2$ is a two-dimensional disk (or a cigar), and $TN \cong \mathbb {R}^4$ is the Taub-NUT space. Writing the local complex coordinates $(z_1,z_2)$ on TN, such that $z_1$ also parametrizes $D^2$, we turn on the Omega-background, i.e. a holonomy along $S^1$ that provides a twisting of TN via an isometry

$$\begin{aligned} (z_1, z_2)\rightarrow (qz_1, t^{-1}z_2)~. \end{aligned}$$

(3.2)

In this setting, the symmetry group of the 6d (2, 0) theory on the M5-branes is reduced to

$$\begin{aligned} {{\,\textrm{SO}\,}}(6)_E\times {{\,\textrm{SO}\,}}(5)_R\rightarrow {{\,\textrm{SO}\,}}(3)_1 \times {{\,\textrm{SO}\,}}(3)_2\times {{\,\textrm{SO}\,}}(3)_R\times {{\,\textrm{SO}\,}}(2)_R~, \end{aligned}$$

(3.3)

where ${{\,\textrm{SO}\,}}(3)_1$ and ${{\,\textrm{SO}\,}}(3)_2$ are the space-time symmetry of $S^1\times _q D^2$ and $M_3$, respectively, and ${{\,\textrm{SO}\,}}(3)_R$ is the symmetry of a cotangent fiber of $T^*M_3$. We perform a topological twist by taking the diagonal subgroup ${{\,\textrm{SO}\,}}(3)_{\text {diag}}$ of ${{\,\textrm{SO}\,}}(3)_2\times {{\,\textrm{SO}\,}}(3)_R$ so that the resulting theory is partially topological (along $M_3$). After the partial topological twist, the effective theory on $S^1\times _q D^2$ only depends on topology (but not the metric) on $M_3$ and is described by 3d $\mathcal {N}=2$ theory often denoted $\mathcal {T}[M_3]$,¹ with the R-symmetry given by ${{\,\textrm{SO}\,}}(2)_R$ in (3.3). When $M_3$ is a Seifert manifold, there is an extra $\text {U}(1)_S$ symmetry associated with the two directions in the cotangent bundle normal to the Seifert fiber. As a result, the partition function, called the half-index, of the 3d $\mathcal {N}=2$ theory $\mathcal {T}[M_3]$ on $S^1\times _q D^2$ with a 2d $\mathcal {N}=(0,2)$ boundary condition $\mathcal {B}$ in this setting is defined as

$$\begin{aligned} Z_{\mathcal {T}[M_3]}(S^1\times _q D^2, \mathcal {B})=\text{ Tr }(-1)^F e^{-\beta (\Delta -R-J_3/2)} q^{J_3+S}t^{R-S}~, \end{aligned}$$

(3.4)

where S and R are charges of $\text {U}(1)_S$ and ${{\,\textrm{SO}\,}}(2)_R$, respectively, $\Delta$ is the Hamiltonian, and $J_3$ is an eigenvalue of the Cartan subalgebra of ${{\,\textrm{SO}\,}}(3)_1$. The difference between $\text {U}(1)_S$ and ${{\,\textrm{SO}\,}}(2)_R$ is customarily denoted $\text {U}(1)_\beta$ in [74–76], and its fugacity is the variable t in (3.4).

Notice that the system (3.1) does not involve a once-punctured torus which was used to define the Hitchin moduli space and the parameter t as in the previous section. However, for gauge groups of type A, the following two physical systems are expected to be closely related:

• 6d (2, 0) theory on $S^1\times C_p$ with $C_p$ being a once-punctured torus.

• 4d $\mathcal {N}=2^*$ theory on $S^1$.

Although the two systems would have different spectra,² their BPS sectors are expected to be equivalent. In particular, at low energy both systems realize a 3d sigma-model onto the Hitchin moduli space. The deformation parameters can also be identified as follows.

On one side, the (classical) deformations are parametrized by the triplet $(\alpha _p,\beta _p,\gamma _p)$ of monodromy parameters around the puncture as introduced before. On the other side, for the 4d $\mathcal {N}=2^*$ theory, the triplet of deformation parameters is given by the complex mass of the adjoint hyper-multiplet in 4d together with the holonomy of the $\text {U}(1)$ flavor symmetry along the circle. In the system (3.1), the 4d $\mathcal {N}=2^*$ theory is obtained by the compactification of the 6d theory on $T^2\subset M_3$ with holonomy for $\text {U}(1)_\beta$ along $S^1$ (3.4). In particular, the parameter t defined above is identified with the t in DAHA. In this section, we will be looking at questions whose answers depend holomorphically on t, as required by supersymmetry on $M_3$, and the other deformation parameter $\beta _p$ won’t play a role. For example, what complex connections on $T^2\subset M_3$ can be extended to the entire $M_3$ is a question that is “holomorphic in J” (and given by intersections of (A, B, A)-branes in the Hitchin moduli space). Notice that this non-trivial relation only holds for a gauge group of type A, while for other types the class $\mathcal {S}$ construction of 4d $\mathcal {N}=2^*$ theory is generally unknown, and the once-punctured torus does not lead to either the 4d $\mathcal {N}=2^*$ theory or DAHA.

One statement of the 3d/3d correspondence is the duality between the non-perturbative complex ${{\,\textrm{SL}\,}}(N,\mathbb {C})$ Chern–Simons theory on $M_3$ and the 3d $\mathcal {N}=2$ theory $\mathcal {T}[M_3]$ on $S^1\times _q D^2$, so that the partition functions of both sides are identified. As explained in [68, 75, 76], for a particular class of boundary conditions $\mathcal {B}_b$ labeled by $b\in ({{\,\textrm{Spin}\,}}^c(M_3))^{N-1}$, the partition function of the 3d $\mathcal {N}=2$ theory $\mathcal {T}[M_3]$ on $S^1\times _q D^2$ counts BPS states and, therefore, has a q-expansions with integer coefficients and integer q-powers³

$$\begin{aligned} \widehat{ Z }_{\mathcal {T}[M_3],b } ( q,t ) := Z_{\mathcal {T}[M_3]}(S^1\times _q D^2, \mathcal {B}_b)~. \end{aligned}$$

(3.5)

The relation to Chern–Simons theory involves the same space of boundary conditions with a “dual” basis, related to $\mathcal {B}_b$ via the $\mathbb {S}$-matrix

$$ \mathbb {S}_{ab}=\frac{\sum _{\sigma \in \mathfrak {S}_N}e^{2\pi i \sum _{i=1}^{N-1}{\ell k}(a_i,b_{\sigma (i)})}}{|\textrm{Stab}_{\mathfrak {S}_N}(a)|\cdot |{\textrm{Tor}\,}H_1(M_3,\mathbb {Z})|^{(N-1)/2}}~. $$

In particular, the partition function of the non-perturbative ${{\,\textrm{SL}\,}}(N,\mathbb {C})$ Chern–Simons theory on $M_3$ is given by

$$\begin{aligned} \Big ( - \frac{\log q}{4\pi i} \Big )^{\frac{N-1}{4}} \sum _{a,b\in ({{\,\textrm{Spin}\,}}^c(M_3))^{N-1}} e^{2\pi i \bar{k} \cdot {\ell k}(a,a)} \mathbb {S}_{a b } \widehat{ Z }_{\mathcal {T}[M_3],b } ( q,t ) \end{aligned}$$

(3.6)

with generic $|q|<1$, and specializes to that of ${{\,\textrm{SU}\,}}(N)$ Chern–Simons theory when $q \rightarrow e^{2\pi i / \bar{k}}$ with integer (renormalized) level $\bar{k} = k + N$. The origin of $\log q$ factors is explained in [137]. Note that the linking pairing ${\ell k}$ on ${{\,\textrm{Spin}\,}}^c(M_3)$ is defined by the Pontryagin duality. We will see shortly that it is the basis of BPS partition functions (3.5) and the corresponding boundary conditions $\mathcal {B}_b$ that are most naturally related to DAHA.

Consider a simple example where $M_3 = L(p,1)$ is a Lens space. The lens space L(p, 1) can be constructed by gluing two solid tori with a homeomorphism between the boundary tori sending the meridian (1, 0) of one torus to a (1, p) cycle of the other. The corresponding 3d $\mathcal {N}=2$ ${{\,\textrm{SU}\,}}(N)$ gauge theory $\mathcal {T}[L(p,1)]$ consists of one adjoint chiral multiplet $\Phi$ with R-charge 2 and $\mathcal {N}=2$ Chern–Simons term with level p. Consequently, the factor $\widehat{Z}_{\mathcal {T}[L(p,1)],b}$ labeled by $b\in ({{\,\textrm{Spin}\,}}^c(M_3))^{N-1}$ is defined by

$$\begin{aligned} \widehat{Z}_{\mathcal {T}[L(p,1)],b} (q,t)={\frac{1}{N!}} \int _{|X|=1} \frac{d X}{2\pi i X} \; \Upsilon (X;q,t) \Theta _{b}^{\mathbb {Z}^{N-1};p}(X, q) ~, \end{aligned}$$

(3.7)

where

$$ \Upsilon (X)= \prod _{\alpha \in \textsf{R}} \frac{(X^\alpha ; q^2)_\infty }{ (t^2X^\alpha ; q^2)_\infty }~, \qquad \Theta _{b}^{\mathbb {Z}^{N-1};p}(X, q)=\sum _{n\in p\mathbb {Z}^{N-1}+b}q^{2\sum _{i=1}^{N-1}{n_i^2}/p}\prod _{i=1}^{N-1}X_i^{n_i}~. $$

Here we impose the Neumann boundary condition at the boundary $\partial (S^1\times _q D^2)$ on the vector multiplet and adjoint chiral multiplet, which give rise to the numerator and denominator of the Macdonald measure $\Upsilon$ by one-loop determinant [158] (see also (B.14)). In addition, the boundary partition function $\Theta _{b}^{\mathbb {Z}^{N-1};p}$ encodes the information about the Chern–Simons term with level p, and 2d $\mathcal {N}=(0,2)$ boundary condition at the boundary $\partial (S^1\times _q D^2)$ is labeled by $b\in ({{\,\textrm{Spin}\,}}^c(M_3))^{N-1}$. In fact, $\widehat{Z}_{\mathcal {T}[L(p,1)],b}$ can be understood as the half-index of the 3d/2d coupled system. For more detail, we refer to [75, 76].

When the lens space L(p, 1) is constructed by gluing two solid tori, we can include a Wilson loop in each solid torus. The reduced partition function with boundary condition specified by ${{\,\textrm{Spin}\,}}^c$ structure b results in

$$\begin{aligned} \widehat{Z}_{\mathcal {T}[L(p,1)],b}(\lambda ,\mu ) =\frac{1}{ \widehat{Z}_{\mathcal {T}[L(p,1)],b} } {\frac{1}{N!}} \int _{|X|=1} \frac{d X}{2\pi i X} \; \Upsilon (X)\; \Theta _{b}^{\mathbb {Z}^{N-1};p}(X) \; P_\lambda (X)\overline{P_\mu (X)}~, \end{aligned}$$

(3.8)

where the conjugation $f\mapsto \overline{f}$ is defined in (B.13). In particular, when $p=0$, i.e. $L(0,1)\cong S^1\times S^2$, the partition function vanishes unless the total charge of two Wilson loops is zero. This defines the Macdonald inner product (B.15)

$$\langle P_\lambda ,P_\mu \rangle = \widehat{Z}_{\mathcal {T}[L(0,1)],0}(\lambda ,\mu ) =\delta _{\lambda ,\mu }~g_\lambda (q,t)~.$$

In the case of $M_3=S^3$, this defines the symmetric bilinear pairing [34, 50, 101]

$$\begin{aligned} {\textbf {[}}P_\lambda ,P_\mu {\textbf {]}}=\widehat{Z}_{\mathcal {T}[L(1,1)],0}(\lambda ,\mu ) =P_\lambda (q^{-2\mu }t^{-2\rho })P_\mu (t^{-2\rho })~, \end{aligned}$$

(3.9)

where $\rho$ is the Weyl vector of $\mathfrak {sl}(N)$. As in Appendix B.1.6, this pairing $\mathbb {C}_{q,t}[X]^{\mathfrak {S}_N}\times \mathbb {C}_{q,t}[X]^{\mathfrak {S}_N}\rightarrow \mathbb {C}_{q,t}$ can be defined by transforming the holonomy $\text{ Tr }\,(X)$ along the (1, 0)-cycle in one solid torus to the holonomy $\text{ Tr }\,(Y)$ along the (0, 1)-cycle, and it acts on loop operators in the other solid torus via the polynomials representation when they link:

$$\begin{aligned} {\textbf {[}}f(X),g(X){\textbf {]}}=\textrm{pol}(f(Y^{-1}))\cdot g(X)\Big |_{X\mapsto t^{-2\rho }} \end{aligned}$$

(3.10)

for $f,g\in \mathbb {C}_{q,t}[X]^{\mathfrak {S}_N}$. In the case of ${{\,\textrm{SU}\,}}(2)$, this is indeed (2.114). This can be viewed as a deformed version of the construction of the skein module of type $A_{N-1}$

$$\begin{aligned} {{\,\textrm{Sk}\,}}(M_3,{{\,\textrm{SU}\,}}(N))={{\,\textrm{Sk}\,}}(M_3^+,{{\,\textrm{SU}\,}}(N))\mathop {\otimes }_{{{\,\textrm{Sk}\,}}(C,{{\,\textrm{SU}\,}}(N))}{{\,\textrm{Sk}\,}}(M_3^-,{{\,\textrm{SU}\,}}(N)) \end{aligned}$$

(3.11)

of a closed oriented 3-manifold $M_3$ by using a Heegaard splitting $M_3= M_3^{+}\cup _C M_3^{-}$. As seen in Sect. 2.5, the polynomial representation $\mathscr {P}$ of $S\!\ddot{H}$ can be understood as a deformed Skein module of a solid torus $S^1\times D^2$. In (3.8), $P_\lambda (X)$ (resp. $\overline{P_\mu (X)}$) can be actually regarded as a basis element of the deformed skein module of one (resp. the other) solid torus, and the boundary partition function $\Theta$ glues the two solid tori by the S-transformation (2.89). Thus, the spherical DAHA acts on the left-module via the polynomial representation whereas it acts on the right-module via its S-transformation. As a result, the S-transformation $\sigma (\mathscr {P})$ of the polynomial representation, called the functional representation, can be defined by the symmetric bilinear pairing, which is presented in Appendix B.2.2.

Fig. 3.1

The relation between 3d $\mathcal {N}=2$ theory $\mathcal {T}[S^1\times _\zeta C]$ and 2d sigma-model. A mapping torus $S^1\times _\zeta C$ where the top and bottom tori are identified by $\zeta \in {{\,\textrm{SL}\,}}(2,\mathbb {Z})$ gives rise to an ${{\,\textrm{SL}\,}}(2,\mathbb {Z})$ duality wall on the worldsheet of $(\mathfrak {B}_\text {cc},\mathfrak {B}')$-string

Moreover, the relation between 3d $\mathcal {N}=2$ theory $\mathcal {T}[M_3]$ to the 2d sigma-model explored in Sect. 2 becomes manifest from the fivebrane system (3.1). For the sake of brevity, let $M_3= S^1_\tau \times C$ where $C\cong T^2$. As described above, the compactification of the 6d theory on C with $\text {U}(1)_\beta$ holonomy along $S^1$ leads to 4d $\mathcal {N}=2^*$ theory, and the t parameter in (3.1) can be identified with the ramification parameters $(\alpha _p,\gamma _p)$ via

$$\begin{aligned} tq^{-\frac{1}{2}}=\exp (-\pi ( \gamma _p+i \alpha _p))~. \end{aligned}$$

(3.12)

As in Fig. 3.1, we further compactify the 4d $\mathcal {N}=2^*$ theory on a two-torus $T^2=S^1\times S^1_q\subset S^1\times _q D^2$ to obtain the 2d sigma-model $S^1_\tau \times \textbf{I}\rightarrow {\mathcal {M}}_H(C_p, {{\,\textrm{SU}\,}}(N))$ where the interval $\textbf{I}=[0,1]$ is obtained by reducing along $S^1_q \subset D^2$. The canonical coisotropic brane $\mathfrak {B}_\text {cc}$ arises at the boundary of the strip $S^1_\tau \times \textbf{I}$ corresponding to the center of $D^2$ [129]. In addition, a boundary condition of 3d $\mathcal {N}=2$ theory at $\partial (S^1\times _q D^2)$ gives rise to a brane $\mathfrak {B}'$ at the other boundary of the strip $S^1_\tau \times \textbf{I}$ in the 2d sigma-model.

The theory $\mathcal {T}[S^1_\tau \times C]$ consists of three $\mathcal {N}=2$ adjoint chiral multiplets $Q,\widetilde{Q}$ and $\Phi$ where the Neumann boundary condition is imposed on the $\mathcal {N}=2$ vector multiplet and chiral multiplets $\widetilde{Q}$ and $\Phi$, and the Dirichlet boundary condition is imposed on the $\mathcal {N}=2$ chiral multiplet Q at $\partial (S^1\times _q D^2)$. Moreover, the form (3.4) of the refined index tells us that fermions are periodic and a field $\Psi$ is identified along the time circle $S^1$

$$\begin{aligned} q^{(J_3+S)}t^{(R-S)}\Psi (x^0+\beta ,z_1)\sim \Psi (x^0,z_1)~.\end{aligned}$$

(3.13)

The time derivative is replaced as $\partial _t \rightarrow \partial _t -R-J_3/2$ due to $e^{-\beta (\Delta -R-J_3/2)}$.

	$\text {U}(1)_R$	$\text {U}(1)_S$	bdry cond.
$\Phi$	2	0	N
Q	0	$-2$	D
$\widetilde{Q}$	0	0	N

One important lesson that we learn in this subsection is that the Hilbert space of a non-perturbative complex Chern–Simons TQFT on a 2-torus is the space of representations of the spherical DAHA at $t=1$. A categorified version of this statement would be a relation between the category of line operators in the $\widehat{Z}$ TQFT and the category of modules of $S\!\ddot{H}_{t=1}$,

$$\begin{aligned} {{\,\mathrm{\textsf{MTC}}\,}}(\widehat{Z}) \cong {{\,\mathrm{\textsf{Rep}}\,}}( S\!\ddot{H}_{t=1} )~. \end{aligned}$$

(3.14)

Again, we remind that here and in other places, ${{\,\mathrm{\textsf{MTC}}\,}}$ refers to a tensor category where some of the traditional conditions may need to be relaxed, e.g. it may have an infinite number of simple objects, be non-unitary or non-semisimple. (The latter generalization typically appears when one tries to “truncate” a category with infinitely many simple objects to a finite-dimensional structure.) The modular representations that arise from such generalizations are, in general, more delicate and interesting than familiar vector-valued modular forms that describe the space of genus-1 conformal blocks in a rational VOA. Of course, in some special cases, these more interesting and exotic generalizations do not arise, and ${{\,\mathrm{\textsf{MTC}}\,}}$ is a genuine modular tensor category in its full mathematical sense (justifying the name for generalizations as well); this happens in some of the examples discussed in the following subsections and also in various examples considered in [44, 55, 76].

3.1.1 ${{\,\textrm{SU}\,}}(2)$: Refined Chern–Simons and TQFT Associated to Argyres–Douglas Theory

This connection of 3d theories to the 2d sigma-model clarifies the geometric origin of the modular action. It was proposed in [4] that the fivebrane system (3.1) with $N=2$ M5-branes gives rise to ${{\,\textrm{SU}\,}}(2)$ refined Chern–Simons theory on $M_3$ when the parameters are subject to⁴

$$\begin{aligned} q=\exp \Bigl (\frac{\pi i}{k+2c} \Bigr )~,\qquad t=\exp \Bigl (\frac{c\, \pi i}{k+2c} \Bigr )~. \end{aligned}$$

(3.15)

This condition is equivalent to the existence (2.99b) of the brane $\mathfrak {B}_\textbf{V}$ in the 2d sigma-model Sect. 2.6.3 so that the field identification (3.13) under (3.15) leads to the boundary condition $\mathfrak {B}'=\mathfrak {B}_\textbf{V}$ upon the reduction as in Fig. 3.1. Therefore, the module ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_\textbf{V})$ of DAHA in the 2d sigma-model can be identified with the Hilbert space of ${{\,\textrm{SU}\,}}(2)$ refined Chern–Simons theory on $T^2$ spanned by $\{| P_j \rangle \}$ $(j=0,\ldots ,k)$. The projective action of ${{\,\textrm{SL}\,}}(2,\mathbb {Z})$ on the Hilbert space is manifest in refined Chern–Simons theory, and the matrix elements can be obtained via the 3d/3d correspondence. In fact, the pairing (3.9) at $N=2$ (which is equal to (2.114)) becomes of rank $(k+1)$ when (3.15) holds; it gives the modular S-matrix in Conjecture 2.1 up to a suitable normalization with the Macdonald norm (2.116). Upon reduction to the sigma-model, it can be interpreted as the S-duality wall in the worldsheet of the $(\mathfrak {B}_\text {cc},\mathfrak {B}_\textbf{V})$-string. Thus, the gluing of the two states $\lambda ,\mu \in {{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_\textbf{V})$ by the S-duality wall in the $(\mathfrak {B}_\text {cc},\mathfrak {B}_\textbf{V})$-string can be understood as the Hopf link configuration in refined Chern–Simons theory on $S^3$, illustrated in Fig. 3.2.

Fig. 3.2

The $(\mathfrak {B}_\text {cc},\mathfrak {B}_\textbf{V})$-string with the S-duality wall gives rise to refined Chern–Simons invariant of the Hopf link in $S^3$

Although the parameters q and t are subject to $t^2q^{k}=-1$, there is one free parameter left. If c is generic, refined Chern–Simons theory cannot arise from a fusion category due to Ocneanu rigidity (for instance, see [49]) and, therefore, it is not a modular tensor category (${{\,\mathrm{\textsf{MTC}}\,}}$).⁵ Nonetheless, it provides torus link invariants as we will briefly review below. In addition, the half index in (3.8) provides the deformation of WRT-invariants of the lens space L(p, 1), and moreover $\widehat{Z}_{\mathcal {T}[L(p,1)],b}(q,t)$ in (3.7) exhibits positivity [75, 76]. Despite the failure to be a fusion category, the half indices shed new light on the topology of three-manifolds and link invariants via the 3d/3d correspondence.

The relation between a 3d theory and Conjecture 2.2 is more interesting. It was argued in [116] that the field identification (3.13) under the condition $t^2q^{2\ell -1}=1$ is equivalent to the class $\mathcal {S}$ construction for the Argyres–Douglas theory of type $(A_1,A_{2(\ell -1)})$ in [39, 155], which we briefly review below. The 4d $\mathcal {N}=2$ Argyres–Douglas theory of type $(A_1,A_{2(\ell -1)})$ can be geometrically engineered by compactifying two M5 branes on a sphere $C\cong \mathbb {C}\textbf{P}^1$ with one wild (irregular) singularity at infinity. The theory is specified by the Hitchin system on $C_{\text {wild}}$ where the Higgs field has the asymptotic behavior at infinity described by

$$\begin{aligned} \varphi (z_1)dz_1 \sim z_1^{\frac{2\ell -1}{2}} \sigma _3dz_1~, \end{aligned}$$

(3.16)

where $z_1$ is the coordinate of $C\backslash \infty$ and $\sigma _3$ is the third Pauli matrix. Thus, we denote this Argyres–Douglas theory by $\mathcal {T}[C_{\text {wild}},{{\,\textrm{SU}\,}}(2)]$. The Hitchin action on the moduli space of Higgs bundles can be identified with the $\text {U}(1)_\beta$ symmetry defined below (3.4)

$$ \text {U}(1)_\beta :(A, \varphi ) \rightarrow (A, e^{i \theta } \varphi )~. $$

In the brane setting (3.1), we cannot consider the Hitchin system with (3.16) on $D^2$ in general. However, when the $\Omega$-deformation parameters are subject to $t^2q^{2\ell -1}=1$, the field identification (3.13) for the Higgs field is consistent along the time circle $S^1$

$$ tq^{\frac{2\ell -1}{2}}\varphi (x_0+\beta ,z_1)=\varphi (x_0+\beta ,z_1) \sim \varphi (x_0,z_1)~. $$

Hence, under (2.99c), it is effectively equivalent to the following brane setting:

$$\begin{aligned} \begin{matrix} {\text{ space-time: }} &{} \quad &{} S^1 &{} \times &{} T^*C_{\text {wild}} &{} \times &{}T^* M_3 \\ 2{\text{ M5-branes: }} &{} \quad &{} S^1 &{}\times &{} C_{\text {wild}} &{} \times &{} M_3 \end{matrix} \end{aligned}$$

(3.17)

This system is investigated in detail (including Argyres–Douglas theories of other types) [44, 58, 59, 116], and remarkably there turns out to be an ${{\,\textrm{SL}\,}}(2,\mathbb {Z})$ representation on the set of connected components of $\text {U}(1)_\beta$ fixed points

(3.18)

Moreover, considering the topologically twisted partition function $Z(S^{1} \times M_{3})$ of the Argyres–Douglas theory $\mathcal {T}[C_{\text {wild}},{{\,\textrm{SU}\,}}(2)]$, this ${{\,\textrm{SL}\,}}(2,\mathbb {Z})$ representation can be categorified. Namely, there is a modular tensor category ${{\,\mathrm{\textsf{MTC}}\,}}[A_1,A_{2(\ell -1)}]$ on $M_3$ whose simple objects are in one-to-one correspondence with $\text {U}(1)_{\beta }$ fixed points. In fact, the Argyres–Douglas theory of type $(A_1,A_{2(\ell -1)})$ possesses the discrete global symmetry $\mathbb {Z}_{2\ell +1}$, and if we impose a holonomy $q=e^{-\frac{2\pi \gamma i}{2\ell +1}}$ ($\gamma \in \mathbb {Z}_{2\ell +1}^\times$) of this discrete global symmetry along $S^1$, then the modular matrices in Conjecture 2.2 are those of the corresponding ${{\,\mathrm{\textsf{MTC}}\,}}[A_1,A_{2(\ell -1)}]$ on $M_3$. Although the S and T matrices in Conjecture 2.2 satisfy the ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ relation even for a generic q, the Ocneanu rigidity again forbids them to be those of an ${{\,\mathrm{\textsf{MTC}}\,}}$. Rather, they connect ${{\,\mathrm{\textsf{MTC}}\,}}$’s for different values of a holonomy $q=e^{-\frac{2\pi \gamma i}{2\ell +1}}$ with $\gamma \in \mathbb {Z}_{2\ell +1}^\times$ by the one-parameter family with q.

When $\gamma =1$, the modular matrices coincide with those of the $(2,2\ell +1)$ Virasoro minimal model [44, 116]. Note that the $(2,2\ell +1)$ Virasoro minimal model is the chiral algebra of the Argyres–Douglas theory of type $(A_1,A_{2(\ell -1)})$ [40]. However, the topologically twisted partition function $Z(S^{1} \times M_{3})$ (therefore ${{\,\mathrm{\textsf{MTC}}\,}}[A_1,A_{2(\ell -1)}]$) receives the contribution from Coulomb branch operators whereas a vacuum character of the chiral algebra is given by Higgs branch operators [23]. It is worth noting that there are generally many chiral algebras with the same representation categories [55] so that this coincidence remains very mysterious. (It is sometimes called “4d symplectic duality”.)

3.1.2 ${{\,\textrm{SU}\,}}(N)$: Higher Rank Generalization

Let us briefly consider a higher rank generalization of the 3d modularity. The moduli space of $G_\mathbb {C}$ flat connections over a two-torus $C\cong T^2$ is the quotient space $(T_\mathbb {C}\times T_\mathbb {C})/W$ of the product of the two complex maximal tori by the Weyl group. In particular, when $G_\mathbb {C}={{\,\textrm{SL}\,}}(N,\mathbb {C})$, the fixed points under the action of the Weyl group $W=\mathfrak {S}_N$ consist of the center $\mathbb {Z}_N\times \mathbb {Z}_N \subset T_\mathbb {C}\times T_\mathbb {C}$ so that there are $N^2$ torsion points on the moduli space $\textbf{V}:=(T\times T)/\mathfrak {S}_N$ of ${{\,\textrm{SU}\,}}(N)$-bundles over a torus. For higher ranks, tame ramifications of Higgs bundles are classified by Levi subgroups of ${{\,\textrm{SU}\,}}(N)$ or equivalently partitions of N [84]. To obtain the spherical DAHA $S\!\ddot{H}(\mathfrak {S}_N)$ of type $A_{N-1}$ as ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_\text {cc})$, a simple puncture corresponding to the $[1,N-1]$ partition needs to be introduced on C. Although we have not understood topology and symplectic geometry of the Hitchin moduli space $\mathcal {M}(C_p,{{\,\textrm{SU}\,}}(N))$ over a torus with a simple puncture (for instance, the number of irreducible components of the global nilpotent cone), we can generalize Conjectures 2.1 and 2.2 to the higher ranks. It is a very interesting problem to generalize the analysis in this paper to arbitrary semi-simple gauge groups.

In refined Chern–Simons theory with ${{\,\textrm{SU}\,}}(N)$ gauge group [4], the parameters q and t are usually expressed in terms of a positive integer $k\in \mathbb {Z}_{>0}$ and the continuous parameter c:

$$\begin{aligned} q=\exp \Bigl (\frac{\pi i}{k+c\,N} \Bigr )~,\qquad t=\exp \Bigl (\frac{c\, \pi i}{k+c\,N} \Bigr )~, \end{aligned}$$

(3.19)

so that they are subject to the relation $t^Nq^k=-1$. Under this condition, the moduli space $\textbf{V}$ of ${{\,\textrm{SU}\,}}(N)$-bundles is a Lagrangian submanifold in the symplectic manifold $(\mathcal {M}(C_p,{{\,\textrm{SU}\,}}(N)),\omega _\mathfrak {X})$. As in the $A_1$ case, finite-dimensional representations in the higher rank spherical DAHA $S\!\ddot{H}(\mathfrak {S}_N)$ can be studied by using the raising and lowering operators [105] in the polynomial representation $\mathscr {P}$. The Hilbert space ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_\textbf{V})$ of ${{\,\textrm{SU}\,}}(N)$ refined Chern–Simons theory is spanned by the basis $P_\lambda$ labeled by Young diagrams $\lambda \subset [k^{N-1}]$ inscribed in the $k\times (N-1)$ rectangle. The modular action on the Hilbert space is described by S and T matrices of rank $\frac{(N+k-1)!}{(N-1)!k!}$,

$$\begin{aligned} S_{\lambda \mu }=P_{\lambda }(q^{-2\mu }t^{-2\rho })P_{\mu }(t^{-2\rho })~,\qquad T_{\lambda \mu }= \delta _{\lambda \mu }\cdot q^{\frac{1}{N}|\lambda |^2-||\lambda ||^2} t^{||\lambda ^{t}||^2-N|\lambda |}~, \end{aligned}$$

(3.20)

where $\left\| \lambda \right\| ^2 =\sum \lambda _{i}^{2}$, and $\lambda ^{t}$ denotes the transposition of the Young diagram $\lambda$. They indeed compute invariants of a Seifert manifold and a torus link [4, 36, 37]. Regarding $P_\lambda (X)$ as an element of $S\!\ddot{H}(\mathfrak {S}_N)$, one can define the invariant of a torus link $T_{m,n}$ by

$$\begin{aligned} \theta (\zeta _{m,n}(P_\lambda (X))) ~,\qquad \zeta _{m,n}=\begin{pmatrix}m&{}n\\ *&{}*\end{pmatrix}\in \textrm{SL}(2,\mathbb {Z})~. \end{aligned}$$

(3.21)

where $\zeta _{m,n}$ acts projectively on $P_\lambda (X)\in S\!\ddot{H}(\mathfrak {S}_N)$, and $\theta :S\!\ddot{H}(\mathfrak {S}_N)\rightarrow \mathbb {C}_{q,t}$ is the evaluation map defined in (B.18). The large N limit is conjectured to be equal to the Poincare polynomial of the HOMFLY-PT homology of a torus link up to a change of variables when colors are labeled by a rectangular Young diagram.

After a simple puncture is added on a two-torus $T^2$, the moduli space becomes smooth and the $N^2$ torsion points turn into the corresponding $N^2$ exceptional divisors. Let us denote them by $\textbf{D}_i^{(N)}$ ($i=1,\ldots ,N^2$). They become Lagrangian submanifolds with respect to $\omega _\mathfrak {X}$ when $t^N=q^{-M}$, or

$$\begin{aligned} q=\exp \left( \frac{2\pi i}{M+cN} \right) ~,\qquad t=\exp \left( \frac{2c\pi i}{M+cN} \right) ~, \end{aligned}$$

(3.22)

with coprime (M, N). In fact, under the shortening condition $t^N=q^{-M}$, there are $N^2$ irreducible $S\!\ddot{H}(\mathfrak {S}_N)$-modules of dimension $\frac{(N+M-1)!}{(N-1)!M!N}$, corresponding to the exceptional divisors. Among them, only one irreducible component $\textbf{D}_1^{(N)}$ is invariant under ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$, which is analogous to $\textbf{D}_1$ in the $A_1$ case. We are interested in the modular matrices acting on the corresponding finite-dimensional representation of $S\!\ddot{H}(\mathfrak {S}_N)$.

With the shortening condition $t^N=q^{-M}$, a finite-dimensional module arises as a quotient of the polynomial representation whose basis is spanned by Macdonald polynomials $P_\lambda$ with $\lambda \subset [M^{N-1}]$ inscribed in the $M\times (N-1)$ rectangle. This decomposes into N irreducible modules, and the other $N(N-1)$ irreducible modules can be obtained by their orbits under the symmetry $\Xi \times {{\,\textrm{PSL}\,}}(2,\mathbb {Z})=H^1(C,\mathbb {Z}_N)\times {{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ of $S\!\ddot{H}(\mathfrak {S}_N)$. They correspond to ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_{\textbf{D}_i^{(N)}})$. From the brane perspective, the support of the brane of the polynomial representation intersects with the corresponding N exceptional divisors. When $t^N=q^{-M}$, N Macdonald polynomials $P_{\lambda ^{(i)}}$ $(i=1, \ldots , N)$ of type $A_{N-1}$, where $\lambda ^{(i)}\subset [M^{N-1}]$, are degenerate at each eigenvalue of the Dunkl operator

$$ D(u)=\sum _{r=0}^{n}(-u)^{r} D^{(r)}~, \qquad D^{(r)}=\sum _{\begin{array}{c} I \subset [1,\ldots ,N]\\ |I|=r \end{array}} \prod _{\begin{array}{c} i \in I\\ j\not \in I \end{array}} \frac{t X_{i}-t^{-1}X_{j}}{X_{i}-X_{j}} \varpi _{i} \quad (r=0,1, \ldots , N)~. $$

Here we write variables of the Macdonald polynomials defined in Appendix B.1.5 as $X_i/X_j:=X^\alpha$ for a root $\alpha =e_i-e_j$ and the q-shift operators act as $\varpi _{i} X_j=q^{\delta _{ij}} X_j$. We also note that $D^{(0)}=1=D^{(N)}$. Out of the N irreducible finite-dimensional modules, only one irreducible representation becomes a ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ representation, and its basis is spanned by

$$\begin{aligned} \Bigl \{\sum _{i=1}^N P_{\lambda ^{(i)}}(X)/P_{\lambda ^{(i)}}(t^{-\rho })\Bigl \}_{\lambda ^{(i)}\subset [M^{N-1}]}~. \end{aligned}$$

(3.23)

In fact, the modular S-matrix $S_{\lambda \mu }$ in (3.20) becomes of rank $\frac{(N+M-1)!}{(N-1)!M!N}$ with the shortening condition $t^N=q^{-M}$. As in the $A_1$ case (2.125), we can make a change of basis to (3.23) to obtain a $\frac{(N+M-1)!}{(N-1)!M!N}$-dimensional ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ representation on the irreducible $S\!\ddot{H}(\mathfrak {S}_N)$-module explicitly.

By a similar argument to the one above, the fivebrane system (3.1) at $t^N=q^{-M}$ is equivalent to the Argyres–Douglas theory of type $(A_{N-1},A_{M-1})$ [39, 155] on $S^1\times M_3$, which admits a class $\mathcal {S}$ construction with an ${{\,\textrm{SU}\,}}(N)$ Hitchin system on $\mathbb {C}\textbf{P}^1$ with a wild singularity at $z=\infty$ where the eigenvalues of the Higgs field grow as $|\varphi | \sim |z^{M / N} dz|$. Therefore, the modular matrices acting on the module ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_{\textbf{D}_1^{(N)}})$ can be understood as those of an ${{\,\mathrm{\textsf{MTC}}\,}}[A_{N-1},A_{M-1}]$ associated to the $(A_{N-1},A_{M-1})$ Argyres–Douglas theory, which categorifies the ${{\,\textrm{SL}\,}}(2,\mathbb {Z})$ action on fixed points of the $\text {U}(1)_\beta$ action on the corresponding wild Hitchin moduli space [58]. As a higher rank generalization of Conjecture 2.2, it is expected that they are related to the modular matrices in the $(N,M+N)$ minimal model of the $W_N$-algebra, which is the chiral algebra of the $(A_{N-1},A_{M-1})$ Argyres–Douglas theory [40]. In fact, by normalizing them appropriately with the Macdonald norm (B.15) of type $A_{N-1}$, the modular matrices at $q=e^{-2\pi i/(M+N)}$ coincide with those (3.26) of the $W_N(N,M+N)$ minimal model [20], which are reviewed below. However, we should keep in mind the same caution as the one given at the end of the previous subsection Sect. 3.1.1.

Remarkably, the space ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_{\textbf{D}_1^{(N)}})$ has another intriguing interpretation. In the limit of the spherical rational Cherednik algebra $S\!\ddot{H}^{\textrm{rat}}_{\hbar ,c}(\mathfrak {S}_{N})$, the target space of the sigma-model becomes the Hilbert scheme of $(N-1)$-points on the affine plane $\mathbb {C}^2$, and the exceptional divisor $\textbf{D}_1^{(N)}$ only remains to be a compact Lagrangian submanifold, called punctual Hilbert scheme. (See also Appendix D.2.) It is known that its geometric quantization provides the unique finite-dimensional representation of $S\!\ddot{H}^{\textrm{rat}}_{\hbar ,c}(\mathfrak {S}_{N})$ [14, 78, 79] and it is furthermore isomorphic to the lowest a-degree $\mathcal {H}_{\text {bottom}}(T_{N,M})$ of HOMFLY-PT homology of the (N, M) torus knot $T_{N,M}$ [73]. Thus, we have an isomorphism of vector spaces

$$\begin{aligned} K^0({{\,\mathrm{\textsf{MTC}}\,}}[A_{N-1},A_{M-1}])\cong {{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_{\textbf{D}_1^{(N)}})\cong \mathcal {H}_{\text {bottom}}(T_{N,M})~. \end{aligned}$$

(3.24)

In what follows, we briefly review the modular matrices of the $W_N(N,M+N)$ minimal model [20]. These minimal models admit a coset description:

$$\begin{aligned} W_N(N,M+N)=\frac{{{\,\textrm{SU}\,}}(N)_k\times {{\,\textrm{SU}\,}}(N)_1}{{{\,\textrm{SU}\,}}(N)_{k+1}}~, \qquad \text {with} \quad k=\frac{N}{M}-N~. \end{aligned}$$

(3.25)

Therefore, their modular matrices are constructed from those of ${{\,\textrm{SU}\,}}(N)_k$ affine Lie algebra [20]. The primary fields in the ${{\,\textrm{SU}\,}}(N)_k$ WZW model are classified by

$$ \Phi (N;n):=\Bigl \{\lambda =\left( \lambda _{1}, \ldots , \lambda _{N-1}\right) \in \mathbb {Z}_{>0}^{N-1} \ \vert \ \ \sum _{i=1}^{N-1}\lambda _{i}<n=k+N\Bigr \} $$

where the vacuum corresponds to $\varrho =(1,\ldots ,1)\in \Phi (N;n)$, and the S matrix is given by

$$ S_{\lambda \mu }^{(N ; n)}= \frac{1}{in\sqrt{N}}\exp [2 \pi i\frac{ t(\lambda ) t(\mu )}{N n}] \det \Bigl (\exp [-2 \pi i\frac{\lambda [\ell ] \mu [m]}{n}]\Bigr )_{1 \le \ell ,m \le N} $$

with

$$ \lambda [i]=\sum _{i \le \ell <N}\left( \lambda _{\ell }+1\right) ~, \qquad t(\lambda ):=\sum _{j=1}^{N-1} j \lambda _{j}~. $$

The primary fields of the $W_N(N,M+N)$ minimal model are in one-to-one correspondence with the following set

$$ \Phi [W_N(N,M+N)]=\Bigl \{(\varrho ,\lambda ) \ \vert \ \lambda \in \Phi (N;N+M)~,\ t(\lambda )\equiv 0 \mod N \Bigr \} $$

The modular S and T matrices of the $W_N(N,M+N)$ minimal model are

$$\begin{aligned} S_{(\varrho ,\lambda )(\varrho ,\mu )}&=(N (N+M))^{\frac{3-N}{2}} \exp \left[ -2 \pi i \frac{t(\varrho ) (t(\mu )+t(\lambda ))}{N}\right] S_{\varrho \varrho }^{(N ; N /(N+M))} S_{\lambda \mu }^{(N ;(N+M)/N)}~, \\T_{(\varrho ,\lambda )(\varrho ,\mu )}&=-i\delta _{\lambda \mu }\exp \left[ \pi i\frac{(N+M) \varrho -N \lambda )\cdot ((N+M) \varrho -N \lambda )}{(N+M)N}\right] ~, \end{aligned}$$

(3.26)

where the inner product is defined by

$$ \lambda \cdot \mu :=\sum _{1 \le i<N} \frac{i(N-i)}{N} \lambda _{i} \mu _{i}+\sum _{1 \le i<j<N} \frac{i(N-j)}{N}\left( \lambda _{i} \mu _{j}+\lambda _{j} \mu _{i}\right) ~. $$

3.2 Relation to Skein Modules and MTC$[M_3]$

In the above discussion, we already encountered the skein modules of 3-manifolds and the algebraic data of line operators ${{\,\mathrm{\textsf{MTC}}\,}}[M_3,G]$ in 3d $\mathcal {N}=2$ theory $\mathcal {T}[M_3,G]$,

$$ {{\,\mathrm{\textsf{MTC}}\,}}[M_3,G]:=\textsf{Line}\bigl [\mathcal {T}[M_3,G]\bigr ] $$

that also enters “gluing” of vertex algebras associated to 4-manifolds [55], twisted indices of $\mathcal {T}[M_3,G]$ on general 3-manifolds [76], and modular properties of q-series invariants $\widehat{Z} (M_3)$ [32].

Since 3d theory $\mathcal {T}[M_3,G]$ has only $\mathcal {N}=2$ supersymmetry, it cannot be topologically twisted on a general 3-manifold and, therefore, does not lead to a full 3d TQFT that could have been associated to a tensor category (of its line operators) in a familiar way. Nevertheless, as was pointed out in [76], the structure of line operators and partially twisted partition functions in $\mathcal {T}[M_3,G]$ in many ways is close to (and, in some cases, is described by) that of a tensor category. Hence, the name ${{\,\mathrm{\textsf{MTC}}\,}}[M_3]$, or ${{\,\mathrm{\textsf{MTC}}\,}}[M_3, G]$. The simple objects of ${{\,\mathrm{\textsf{MTC}}\,}}[M_3, G]$ are complex $G_{\mathbb {C}}$ flat connections on $M_3$. For example, when $M_3$ is the Poincaré sphere and $G = {{\,\textrm{SU}\,}}(2)$, there are three simple objects in ${{\,\mathrm{\textsf{MTC}}\,}}[M_3, G]$ and $K^0 ({{\,\mathrm{\textsf{MTC}}\,}}[M_3, G])$ has rank 3. In this example, and more generally, when all $G_{\mathbb {C}}$ flat connections on $M_3$ are isolated, they can be identified with the intersection points of two Heegaard branes $\mathfrak {B}_{H^{\pm }}$ associated with the Heegaard decomposition of $M_3$, illustrated in Fig. 3.3.

Fig. 3.3

A Heegaard decomposition (left panel) of a closed oriented 3-manifold leads to an interpretation of $K^0 ({{\,\mathrm{\textsf{MTC}}\,}}[M_3])$ as the space of $(\mathfrak {B}_{H^{+}},\mathfrak {B}_{H^{-}})$-strings in $\mathcal {M}_\text {flat}(C,G_\mathbb {C})$

Specifically, let $M_3= M_3^{+}\cup _C M_3^{-}$ be a Heegaard splitting of a closed oriented 3-manifold $M_3$. As in (2.85), 3-manifolds with boundary $\partial M_3^\pm =C$ define the (A, B, A)-branes $\mathfrak {B}_{H^{\pm }}$ supported on Lagrangian submanifolds $\mathcal {M}_\text {flat}(M_3^\pm ,G_\mathbb {C})$ in $\mathcal {M}_\text {flat}(C,G_\mathbb {C})$. Hence, $K^0 ({{\,\mathrm{\textsf{MTC}}\,}}[M_3])$ can be interpreted as the space of open strings between two Heegaard branes $\mathfrak {B}_{H^{\pm }}$ associated to $M_3^{\pm }$ and illustrated in Fig. 3.3. Furthermore, via a complex analogue of the Atiyah–Floer conjecture (see e.g. [81]), this ring is expected to be isomorphic to the complex $G_\mathbb {C}$ Floer homology $HF^{\text {inst}}_{0}(M_3,G_\mathbb {C})$ of $M_3$:

$$\begin{aligned} K^0 ({{\,\mathrm{\textsf{MTC}}\,}}[M_3]) \cong {{\,\textrm{Hom}\,}}^0(\mathfrak {B}_{H^{+}},\mathfrak {B}_{H^{-}})\cong HF_0^{\text {symp}}(\mathcal {M}_\text {flat}(C,G_\mathbb {C});H^+,H^-)\cong HF^{\text {inst}}_{0}(M_3,G_\mathbb {C})~. \end{aligned}$$

(3.27)

Here both symplectic and instanton Floer homology groups are $\mathbb {Z}$-graded, and we take the zeroth degree of the homology groups. Physically, this grading comes from non-anomalous $\text {U}(1)$ R-symmetry.

Indeed, the relevant system here is a stack of M5-branes on $\mathbb {R}\times T^2\times M_3$, and we are interested in the Hilbert space $\mathcal {H}_{\mathcal {T}[M_{3} \times T^{2},G]}$. We can interpret this Hilbert space as that of 3d $\mathcal {N}=2$ theory $\mathcal {T}[M_{3},G]$ on $T^2$. The Hilbert space is $\mathbb {Z}$-graded by the $\text {U}(1)$ R-symmetry of the 3d $\mathcal {N}=2$ theory. On the other hand, we can compactify the 6d $\mathcal {N}=(2,0)$ theory on $T^2$, and perform the topological twist of the 4d $\mathcal {N}=4$ theory considered in [157]. The two types of topological twists of the 4d $\mathcal {N}=4$ theory in [157], Vafa-Witten twist [150] and Marcus/GL-twist [117, 123], are equivalent on $\mathbb {R}\times M_3$, and the BPS equations on $M_3$ are satisfied by complex $G_\mathbb {C}$-flat connections. As a result, the Hilbert space can be understood as complex Floer homology of $M_3$. Consequently, the Hilbert space admits two interpretations [76]:

$$ \mathcal {H}_{\mathcal {T}[T^2,G]}(M_{3})\cong \mathcal {H}_{\mathcal {T}[M_{3} \times T^{2},G]}\cong \mathcal {H}_{\mathcal {T}[M_{3},G]}\left( T^{2}\right) ~. $$

In general, complex Floer homology groups are infinite-dimensional due to the presence of reducible solutions and non-compactness of moduli spaces. Nonetheless, it is graded by the R-charges of the 3d $\mathcal {N}=2$ supersymmetry, and we expect that the zeroth degree piece gives precisely (3.27).

Note, that for some manifolds, like $M_3 = T^3$, all complex flat connections are reducible. (In this example, simply because $\pi _1 (M_3)$ is abelian.) Such examples illustrate especially well how the infinite-dimensional complex Floer homology of $M_3$ is re-packaged into its finite-dimensional version $K^{0}\left( {{\,\mathrm{\textsf{MTC}}\,}}[M_{3},G]\right)$. Moreover, half-BPS line operators in $\mathcal {T}[M_{3},G]$ are in one-to-one correspondence with states of the Hilbert space of $\mathcal {T}[M_{3},G]$ on $T^2$. The mapping class group of $T^2$ acts on this Hilbert space, justifying the name for $K^0 ({{\,\mathrm{\textsf{MTC}}\,}}[M_3])$. In practice, this can be a log-modular action, as in [32].

A somewhat similar “regularization” of the complex Floer theory is provided by the skein module ${{\,\textrm{Sk}\,}}(M_3,G)$, which was recently shown to be finite-dimensional [67] for any closed oriented 3-manifold $M_3$. Physically, the ${{\,\textrm{SU}\,}}(N)$-skein module of $M_3$ is a set of all formal linear combinations of line operators in complex ${{\,\textrm{SL}\,}}(N,\mathbb {C})$ Chern–Simons theory, defined as [135, 143]:

$${{\,\textrm{Sk}\,}}(M_3,{{\,\textrm{SU}\,}}(N)) = \mathbb {C}[q^\pm ] (\text {isotopy classes of framed oriented links in } M_3) / \text {skein relations}~.$$

where the skein relations are given by

The analogue for Cartan types other than A is not well explored, and would be an excellent direction for future work.

Focusing on $G={{\,\textrm{SU}\,}}(N)$ and $G_\mathbb {C}={{\,\textrm{SL}\,}}(N,\mathbb {C})$, the above discussion suggests that there may be a relation between $K^{0}({{\,\mathrm{\textsf{MTC}}\,}}[M_3,G])$ that describes line operators in $\mathcal {T}[M_3]$ and the skein module ${{\,\textrm{Sk}\,}}(M_3,G)$. This relation cannot be a simple isomorphism because, e.g. for $M_3 = T^3$ and $G={{\,\textrm{SU}\,}}(2)$, $K^{0}({{\,\mathrm{\textsf{MTC}}\,}}[M_3,G])$ has 10 simple objects whereas $\text {rank}{{\,\textrm{Sk}\,}}(M_3,G) = 9$ [30, 65]. Relegating a better understanding of this relation to future work,⁶ here we merely conjecture that it commutes with the ${{\,\textrm{SL}\,}}(2,\mathbb {Z})$ action, so that ${{\,\textrm{Sk}\,}}(M_3,G)$ also enjoys a (possibly, log-) modular action

As a next natural step, we now turn our attention to a relation between the skein algebra ${{\,\textrm{Sk}\,}}(C)$ of a Riemann surface C and line operators of the 4d $\mathcal {N}=2$ theory $\mathcal {T}[C]$, in particular in the case when C is a (punctured) torus.