Introduction

Abstract

In string theory, the term “brane” is used for certain extended objects. As is typical of string theory, there are many different ways of seeing or defining these objects, depending on one’s preferred point of view.

1.1 Background

In string theory, the term “brane” is used for certain extended objects. As is typical of string theory, there are many different ways of seeing or defining these objects, depending on one’s preferred point of view. For example, from the target space perspective, where string theory can be thought of as modeling the motion of strings in a target space $\mathfrak {X}$, one can picture branes as particular distinguished submanifolds of $\mathfrak {X}$ (decorated with additional data) on which open strings can end. Relatedly, from the point of view of the string worldsheet, branes are simply boundary conditions of the two-dimensional worldsheet theory. But branes can also be viewed as sources for higher-form gauge symmetries in the effective field theory of the target space. In the supergravity approximation, such extended sources produce interesting solutions, called “black branes” by analogy with familiar “black hole” solutions in standard general relativity. This perspective is especially useful in eleven-dimensional M-theory, where a first-quantization perspective (which would replace the string worldsheet by an appropriate “membrane” theory) is currently unavailable.

Branes, or at least models of certain special versions of branes, have also made numerous appearances in the mathematics literature, where they may go by different names. For example, topological string theory (which, from the physical point of view, comes from a twist of the worldsheet sigma-model discussed above) comes in two flavors, known as the A- and B-models. The category of branes in each of these can be identified with a fairly well-defined mathematical structure associated to a Calabi–Yau target space $\mathfrak {X}$. For the B-model, this is the derived category of coherent sheaves on $\mathfrak {X}$, whereas the A-model is expected to be some appropriately defined version of—or generalization of—the Fukaya category ${{\,\mathrm{\textsf{Fuk}}\,}}(\mathfrak {X},\omega _\mathfrak {X})$, where $\omega _\mathfrak {X}$ is the symplectic form. Since this generalization may be nontrivial, we will write $\text {A-}{{{\,\mathrm{\textsf{Brane}}\,}}}(\mathfrak {X},\omega _\mathfrak {X})$ for the category of A-branes, in which ${{\,\mathrm{\textsf{Fuk}}\,}}(\mathfrak {X},\omega _\mathfrak {X})$ is expected to be a full subcategory. The homological mirror symmetry proposal of Kontsevich [110] identifies the category of A-branes on a Calabi–Yau threefold with the category of B-branes on its mirror, and is the subject of ongoing intense mathematical research.

While the category of B-branes belongs squarely to the realm of algebraic geometry, the category of A-branes is much more subtle, and has appeared in numerous different guises in mathematical physics. To give another example, the proposed framework of brane quantization [85] suggests that the problem of quantizing a symplectic manifold M can be approached by studying the topological A-model on a different target space $\mathfrak {X}$, which is chosen to be a so-called “complexification” of M. (When M is the set of real points of an algebraic symplectic manifold, this complexification can be taken to be the obvious one.) This complexification should, in any case, be a complex manifold whose dimension is twice that of M; M should map to $\mathfrak {X}$, and $\mathfrak {X}$ should be equipped with a holomorphic symplectic form $\Omega$, whose real part ${{\,\textrm{Re}\,}}\Omega$ restricts to the symplectic form on M, and imaginary part ${{\,\textrm{Im}\,}}\Omega$ restricts to zero on M.

One is then instructed to consider the A-model of the complexification with respect to the imaginary part of the holomorphic symplectic form, $\omega _\mathfrak {X}= {{\,\textrm{Im}\,}}\Omega$. This gives rise to a category $\text {A-}{{{\,\mathrm{\textsf{Brane}}\,}}}(\mathfrak {X},\omega _\mathfrak {X})$ of A-branes, which includes not only Lagrangian objects but also much more unfamiliar branes supported on coisotropic submanifolds of $\mathfrak {X}$. Coisotropic branes were introduced in [8, 107] conjectured that spaces of morphisms between A-branes should be identified with deformation quantizations of the functions on their intersections. While coisotropic branes remain mysterious in general, and do not occur at all on simply-connected Calabi–Yau three-folds, they are needed for mirror symmetry to work, even on flat target spaces.

In fact, since the dimension of $\mathfrak {X}$ is always zero modulo four, one can define a particularly useful exotic A-brane on $\mathfrak {X}$, known as the canonical coisotropic brane. This brane was introduced in [117], where it played an important role in connecting A-branes to D-modules. Its support is the entire space $\mathfrak {X}$, and it is furthermore expected to have a very interesting algebra of endomorphisms. In fact, in keeping with the proposal of [8], one expects that

$$\begin{aligned} {{\,\textrm{End}\,}}(\mathfrak {B}_\text {cc}) = \mathscr {O}^q(\mathfrak {X}), \end{aligned}$$

(1.1)

where the object on the right-hand side is the deformation quantization of the ring $\mathscr {O}(\mathfrak {X})$ of holomorphic functions (with appropriate polynomial growth conditions at infinity) on the complexification, taken with respect to its holomorphic symplectic form. In the case of an affine variety, $\mathscr {O}(\mathfrak {X})$ is just the coordinate ring. (Although the A-model depends only on the symplectic form $\omega _\mathfrak {X}= {{\,\textrm{Im}\,}}\Omega$, the real part of $\Omega$ enters the definition of the boundary condition $\mathfrak {B}_\text {cc}$, which is only canonically definable on a holomorphic symplectic manifold.)

As with any category, there is an action of this algebra by precomposition (physically speaking, by joining strings at boundary conditions) on the space of morphisms from $\mathfrak {B}_\text {cc}$ to any other A-brane $\mathfrak {B}$. In other words, brane quantization naturally proposes a functor

$$\begin{aligned} {{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},-): \text {A-}{{{\,\mathrm{\textsf{Brane}}\,}}}(\mathfrak {X},\omega _\mathfrak {X}) \rightarrow {{\,\mathrm{\textsf{Rep}}\,}}(\mathscr {O}^q(\mathfrak {X})), \end{aligned}$$

(1.2)

which allows us to generate a representation of this algebra from an A-brane. A category is said to be generated by an object A if ${{\,\textrm{Hom}\,}}(A,-)$ is an equivalence of categories. In fact, Kapustin [98] proposed that $\mathfrak {B}_\text {cc}$ is a generating object of the category of A-branes, and that ${{\,\mathrm{\textsf{Rep}}\,}}(\mathscr {O}^q(\mathfrak {X}))$ can be taken as a definition of the category $\text {A-}{{{\,\mathrm{\textsf{Brane}}\,}}}(\mathfrak {X})$, when $\mathfrak {X}$ is a hyper-Kähler space. We remark that there are some subtleties here. The Fukaya category as typically studied in homological mirror symmetry [110] requires each object to carry a choice of grading, so that there is at least a family of A-branes supported on the same Lagrangian which are shifts of one another, forming a torsor over $\mathbb {Z}$. There is typically no canonical choice of a preferred grading datum on an A-brane. One should more properly expect

$$\begin{aligned} {{\,\mathrm{\textbf{R}Hom}\,}}(\mathfrak {B}_\text {cc},-): D^b\text {A-}{{{\,\mathrm{\textsf{Brane}}\,}}}(\mathfrak {X},\omega _\mathfrak {X}) \rightarrow D^b {{\,\mathrm{\textsf{Rep}}\,}}(\mathscr {O}^q(\mathfrak {X})) \end{aligned}$$

(1.3)

to provide a derived equivalence between the category of A-branes and the derived category of $\mathscr {O}^q(\mathfrak {X})$-modules. (From the physical perspective, this corresponds to working with the notion of equivalence appropriate to the twist, treating A-branes as boundaries for the A-twisted theory rather than boundaries for the full theory that are compatible with the twist.) The relevance of derived categories to boundary conditions in topological string theory has been understood for a long time; see [46, for example].

Returning briefly to the perspective of brane quantization, the gist now consists in the fact that M is a Lagrangian submanifold in $(\mathfrak {X},\omega _\mathfrak {X})$, so that the original symplectic manifold itself can be used to define an A-brane $\mathfrak {B}_M$ in $(\mathfrak {X},\omega _\mathfrak {X})$. In fact, it is shown in [82, 85] that the morphism space ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_M)$ can be identified in a precise fashion with the geometric quantization of M, at least under the assumption that M is a Kähler manifold. As such, brane quantization provides a bridge between deformation quantization—which is guaranteed to formally produce the algebra of quantum observables $\mathscr {O}^q(\mathfrak {X})$, but gives no candidate for a natural module or Hilbert space on which it acts—and standard geometric quantization. (For a recent study of issues in geometric quantization from this perspective, see [87].) However, as we have already argued, the functor ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},-)$ is much more than this: assuming that it is an equivalence, it provides a natural description of the category of $\mathscr {O}^q(\mathfrak {X})$-modules in geometric terms. Indeed, the role of M in the story is no longer distinguished: it is just one A-brane among (at least potentially) many, each of which corresponds naturally to an $\mathscr {O}^q(\mathfrak {X})$-module. This broader perspective was already appreciated in [85], where a particular space $\mathfrak {X}=T^*\mathbb {C}\textbf{P}^1$ was used to generalize the orbit method and give geometric constructions for all representations of ${{\,\textrm{SL}\,}}(2,\mathbb {R})$. Therefore, the proposed equivalence (1.3) between A-branes and $\mathscr {O}^q(\mathfrak {X})$-modules is the natural way to think about a geometric approach to representation theory for algebras that deformation-quantize hyper-Kähler manifolds $\mathfrak {X}$.

As the definition of the A-brane category is not available yet, much of this discussion is not at a mathematical level of rigor. Nonetheless, with an appropriate choice of $(\mathfrak {X},\omega _\mathfrak {X})$, we can provide concrete evidence for the equivalence (1.3) if we restrict ourselves to Lagrangian objects belonging to the Fukaya category ${{\,\mathrm{\textsf{Fuk}}\,}}(\mathfrak {X},\omega _\mathfrak {X})$ of $\mathfrak {X}$, which forms a subcategory in $\text {A-}{{{\,\mathrm{\textsf{Brane}}\,}}}(\mathfrak {X},\omega _\mathfrak {X})$. We will take the target space $\mathfrak {X}$ of the 2d sigma-model to be the moduli space of complex flat connections (or parabolic Higgs bundles) on a once-punctured torus $C_p$. Then, as proved in [131], the algebra $\mathscr {O}^q(\mathfrak {X})$ will be the spherical subalgebra of double affine Hecke algebra (DAHA in short) [35]. One of our goals in this paper is to explore the idea described above in this setup, presenting solid evidence for the equivalence (1.3).¹

Remark: In the past few years, Kontsevich and Soibelman [113] have been developing a new formalism within the framework of ‘holomorphic Floer theory,’ which among other things, allows for a rigorous formulation of brane quantization. According to the generalized Riemann–Hilbert correspondence of Kontsevich–Soibelman, there is an embedding of the Fukaya category ${{\,\mathrm{\textsf{Fuk}}\,}}(\mathfrak {X})$ into the right-hand side of (1.3) as the category of so-called holonomic $D_q$-modules. Their approach provides a realization of the category of representations of $\mathscr {O}^q(\mathfrak {X})$ in terms of sheaves on its Lagrangian skeleton. Some of our results in this paper about DAHA representations can thus be interpreted as a particular example of the generalized Riemann-Hilbert correspondence.

1.2 Results

We first study the representation theory of spherical double affine Hecke algebra $S\!\ddot{H}$ of type $A_1$ from the viewpoint of brane quantization in great detail. We explicitly identify a compact Lagrangian brane in $\mathfrak {X}=\mathcal {M}_\text {flat}(C_p,{{\,\textrm{SL}\,}}(2,\mathbb {C}))$, the moduli space of flat ${{\,\textrm{SL}\,}}(2,\mathbb {C})$-connections on $C_p$, for each finite-dimensional irreducible representation of $S\!\ddot{H}$. In particular, we match objects including the parameter spaces, dimensions and shortening conditions on both sides. We also study the spaces of derived morphisms of the two categories. As a by-product, we find new finite-dimensional representations of $S\!\ddot{H}$ that do not appear in [35]. We see examples in which two irreducible branes can form bound states in more than one way, corresponding to a higher-dimensional ${{\,\textrm{Ext}\,}}^1$; these bound states are related to subtleties defining A-branes supported on singular submanifolds. Hence, the careful study in Chap. 2 in terms of brane quantization provides solid evidence for the following:

Claim 1.1

For $\mathfrak {X}=\mathcal {M}_\text {flat}(C_p,{{\,\textrm{SL}\,}}(2,\mathbb {C}))$, the functor (1.3) restricts to a derived equivalence of the full subcategory of compact Lagrangian A-branes of $\mathfrak {X}$ and the category of finite-dimensional $S\!\ddot{H}$-modules.

We also consider a particular example of a non-compact brane corresponding to the polynomial representation of $S\!\ddot{H}$ studied by Cherednik. In fact, the brane perspective suggests straightforward generalizations of this representation.

While the brane quantization proposal—and thus the physics of the A-model—is our starting point, many of the various other types of branes in string theory and M-theory, and the guises in which they appear, will have a role to play in this paper. As was already emphasized, just for example, in the constructions of [117], the moduli space $\mathfrak {X}$ plays an important role in higher-dimensional gauge theories, which allows for an embedding of the physics of A-branes into a richer system. We focus on one such construction: M5-branes on a once-punctured torus (or equivalently with $\Omega$-deformation orthogonal to M5-branes on a torus) in an appropriate setup of M-theory. This construction will provide many new angles to view the structure of the category of representations of (spherical) DAHA.

As such, branes will lead us to a geometric interpretation of previously known facts about $S\!\ddot{H}$-modules, as well as to new results, not previously known in the representation theory literature. It is rather straightforward from the geometry of the target space $\mathfrak {X}$ to identify finite-dimensional $S\!\ddot{H}$-modules that carry representations of ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$. More interestingly, by connecting the M5-brane setup of the 3d/3d correspondence to the 2d A-model, we can naturally identify the corresponding ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ representations. Let us recall the fivebrane setup for the 3d/3d correspondence where M5-branes are located on $S^1\times D^2\times M_3$ with the $\Omega$-background. Then, a suitable compactification on $T^2\times T^2$ can relate this setup to the 2d A-model described above, where the center of $D^2$ is associated to $\mathfrak {B}_\text {cc}$ and a boundary condition at the boundary of $D^2$ gives rise to $\mathfrak {B}_M$.

For various choices of boundary conditions $\mathfrak {B}_M$, the partition function of 3d $\mathcal {N}=2$ theory $\mathcal {T}[M_3]$ on $S^1 \times D^2$ computes the corresponding invariant of the 3-manifold $M_3$. In some cases, such topological invariants of 3-manifolds can be lifted to a 3d TQFT, i.e. can be constructed via cutting-and-gluing. In turn, the algebraic structure underlying a 3d TQFT often can be encoded in a modular tensor category (that, in general, may be non-unitary or non-semisimple). In particular, in the present setup of 3d/3d correspondence, this algebraic structure itself can be viewed as a special case ${{\,\mathrm{\textsf{MTC}}\,}}[S^1 \times (S^2 \setminus \text {pt}), \mathfrak {B}_M]$ of a more general algebraic structure dubbed ${{\,\mathrm{\textsf{MTC}}\,}}[M_3]$ in [76] for its close resemblance to the structure of a modular tensor category. We will explain how concrete instances of this algebraic structure can be realized via branes in the 2d A-model and the corresponding $S\!\ddot{H}$-modules:

$$\begin{aligned} {{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_M)\cong K^0({{\,\mathrm{\textsf{MTC}}\,}})~. \end{aligned}$$

(1.4)

In particular, one such boundary condition leads to a TQFT associated to a 4d Argyres-Douglas theory. In general, branes supported on M that are invariant under ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ action (not pointwise) give rise to interesting ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ representations ${{\,\textrm{Hom}\,}}(\mathfrak {B}_\text {cc},\mathfrak {B}_M)$, and 3d/3d correspondence can help us to relate them to the modular data (and the Grothendieck group) of an ${{\,\mathrm{\textsf{MTC}}\,}}$-like structure.

Another relevant brane setting appears in the class $\mathcal {S}$ construction [62, 70] of a 4d $\mathcal {N}=2^*$ theory $\mathcal {T}[C_p]$ where M5-branes are placed on $S^1\times \mathbb {R}^3\times C_p$. An algebra of line operators becomes the coordinate ring of the Coulomb branch of 4d $\mathcal {N}=2^*$ theory on $S^1\times \mathbb {R}^3$ [69], and we can study it again in a rank-one case from the relation to $S\!\ddot{H}$. As in [6], the spectrum of line operators in the 4d $\mathcal {N}=2^*$ theory is sensitive to the global structure of the gauge group, which can be specified by imposing additional discrete data. In fact, the Coulomb branch of 4d $\mathcal {N}=2^*$ theory of rank-one is given as the quotient of $\mathcal {M}_\text {flat}(C_p,{{\,\textrm{SL}\,}}(2,\mathbb {C}))$ by this additional discrete choice $\mathbb {Z}_2\subset \mathbb {Z}_2\oplus \mathbb {Z}_2$, which can be interpreted as an automorphism group of $S\!\ddot{H}$. Therefore, we can study the elliptic fibration of the Coulomb branch, and the algebra of line operators on the $\Omega$-background is a $\mathbb {Z}_2$-invariant subalgebra of $S\!\ddot{H}$. Furthermore, by introducing a surface operator of codimension two in the system, an algebra of line operators on the surface operator is related to the full (rather than spherical) DAHA. By compactifying the 4d theory to the 2d sigma-model, we propose a canonical coisotropic brane $\widehat{\mathfrak {B}}_{cc}$ of higher rank where the algebra of $(\widehat{\mathfrak {B}}_{cc},\widehat{\mathfrak {B}}_{cc})$-open strings realizes the full DAHA. In this way, the interplay among moduli spaces, algebras of line operators, and DAHA can be studied from the viewpoint of the compactification of fivebrane systems.

1.3 Structure

The structure of the paper follows a simple principle. We start in the world of two-dimensional physics, and we gradually proceed to higher-dimensional theories. One advantage of this approach is that lower-dimensional theories can be analyzed much more explicitly and often can be described in mathematically rigorous terms. For example, the two-dimensional sigma-model perspective is phrased in the language of the topological A-model, which is reasonably well understood in the mathematical literature. Likewise, many explicit calculations can be done easily and many questions can be answered more concretely in low-dimensional systems. The advantage of higher-dimensional systems, on the other hand, is that they reveal a much richer (higher categorical) structure, that helps to see a “bigger picture,” dualities and relations between various low-dimensional descriptions, which otherwise might seem worlds apart.

To give a concrete overview of what follows: In Chap. 2, we provide a detailed study of the equivalence (1.3) between DAHA representations and A-branes. To this end, we study the 2d sigma-model on the moduli space of flat ${{\,\textrm{SL}\,}}(2,\mathbb {C})$-connections (or the Hitchin moduli space) on the punctured torus $C_p$ in this section. We begin by constructing the spherical DAHA $S\!\ddot{H}$ in the 2d A-model. We review the relevant geometry of the target space in Sect. 2.1. Then we move on to study the algebraic side, reviewing the double affine Hecke algebra of type $A_1$ and its spherical subalgebra $S\!\ddot{H}$ in Sect. 2.2. We introduce the canonical coisotropic brane in Sect. 2.3, showing how the spherical DAHA $S\!\ddot{H}$ arises as the algebra of $(\mathfrak {B}_\text {cc},\mathfrak {B}_\text {cc})$-strings. In the remainder of Chap. 2, we discuss the match between representations of $S\!\ddot{H}$ and open-string states between A-branes. To this end, Sect. 2.4 reviews some details of the category $\text {A-}{{{\,\mathrm{\textsf{Brane}}\,}}}$, explaining the correspondence between branes supported on Lagrangian submanifolds and modules of $S\!\ddot{H}$. In particular, we will find branes for the polynomial representations in Sect. 2.5. Section 2.6 aims to show the match between branes with irreducible compact supports and finite-dimensional $S\!\ddot{H}$ representations. Section 2.7 studies bound states of branes and the corresponding short exact sequences in representations, matching them between the two categories.

Some finite-dimensional $S\!\ddot{H}$-modules carry ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ representations. Taking the 3d/3d correspondence into account, we explore the geometric origin of these ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ representations (and the conditions under which they are present) in Chap. 3. Moreover, the vantage point of three-dimensional physics reveals additional structure concealed behind these ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ representations. We show in Sect. 3.1 that the fivebrane system of the 3d/3d correspondence connects the two-dimensional A-model to three-dimensional topological field theories on a 3-manifold $M_3$. In particular, we show that the choice of an $S\!\ddot{H}$-module with a ${{\,\textrm{PSL}\,}}(2,\mathbb {Z})$ action gives rise to a modular tensor category that describes such a 3d TQFT on $M_3$, whose Grothendieck group is identified with the chosen $S\!\ddot{H}$-module. In Sect. 3.2, we propose that the categorification of the skein module of a closed oriented 3-manifold $M_3$ results in a modular tensor category so that there is a “hidden” ${{\,\textrm{SL}\,}}(2,\mathbb {Z})$ action on the skein module of $M_3$. We also explain the connection to ${{\,\textrm{SL}\,}}(2,\mathbb {C})$ Floer homology groups of $M_3$.

In Chap. 4, we move one more dimension up, and study our category of interest from the vantage point of four-dimensional physics, namely in the context of four-dimensional $\mathcal {N}=2^*$ theories. $\mathcal {N}=2^*$ theories can be constructed by wrapping a stack of M5 branes on the once-punctured torus $C_p$, labeled with some additional discrete data associated to $C_p$. In Sect. 4.1, we study an elliptic fibration of the Coulomb branch of an $\mathcal {N}=2^*$ theory of rank-one on $S^1\times \mathbb {R}^3$, based on the analysis of the Hitchin fibration performed in Sect. 2.1. We also show that the algebra of line operators in the 4d $\mathcal {N}=2^*$ theory in the $\Omega$-background is a subalgebra of $S\!\ddot{H}$ specified by the discrete data. Here as well, the bird’s-eye view provided by the fivebrane system connects 4d physics and 2d sigma-models. We use this in Sect. 4.2 to sort out the relationships among line operators, Coulomb branches, and DAHA. Finally, we introduce a surface operator in the 4d $\mathcal {N}=2^*$ theory and consider an algebra of line operators on the surface operator in Sect. 4.3. We also discuss a higher-rank bundle for the canonical coisotropic brane to realize the full DAHA and the Morita equivalence ${{\,\mathrm{\textsf{Rep}}\,}}(\ddot{H})\cong {{\,\mathrm{\textsf{Rep}}\,}}(S\!\ddot{H})$.

In Appendix A, we list notations and symbols adopted in this paper. A concise summary of some basics of DAHA is given in Appendix B. In Appendix C, we discuss the representation theory of the quantum torus algebra $QT$ in terms of brane quantization. As a toy model, we show the match between representation theory of $QT$ and A-branes on a flat space $\mathbb {C}^\times \times \mathbb {C}^\times$. Then we consider an orbifold quotient by $\mathbb {Z}_2$, and we match A-branes to representations in this context in Appendix C.3. Appendix D is devoted to studying the relation between trigonometric and rational degenerations of the spherical DAHA and Coulomb branches of 3d $\mathcal {N}=4$ theories.