Exact results on ABJ theory and the refined topological string

We study the partition function of the ABJ theory, which is the N=6 superconformal Chern-Simons matter theory with gauge group U(N)xU(N+M) and Chern-Simons levels (k,-k). We exactly compute the ABJ partition function on a three sphere for various k, M and N via the Fermi gas approach. By using these exact data, we show that the ABJ partition function is completely determined by the refined topological string on local P^1 x P^1, including membrane instanton effects in the M-theory dual.


Discussions 24
A. Algorithm for exact computation of ABJ grand partition function

Introduction
Progress in M-theory would hinge on deeper understanding of its non-perturbative effects. A part of these effects appear as membrane instantons in M-theory [1]. During the last couple of years, there has been remarkable progress in understanding the membrane instanton effects. In [2], the authors have completely determined the non-perturbative structure of the free energy of M-theory on AdS 4 × S 7 /Z k based on many different previous results. First of all, the M-theory on this background is expected to be dual to low-energy effective theory of multiple M2-branes [3]. Meanwhile it turns out that the M2-brane theory is described by an N = 6 supersymmetric Chern-Simons matter theory (CSM) with gauge group U (N ) k × U (N ) −k commonly referred to as ABJM theory [4]. While M-theory region (k ≪ N 1/5 ) cannot be accessed by the ordinary perturbative technique in the ABJM theory, the supersymmetry localization [5] reduces the partition function of the ABJM theory on S 3 to a matrix integral [6] called ABJM matrix model. This matrix model has been extensively studied by many approaches [7,8,9,10,11,12,13,14,15,16,17,18,19,2,20]. After the studies of the 't Hooft expansion [8,10,11,7] and the leading order in the M-theory limit [9], there appeared a seminal work [13], which rewrites the ABJM matrix model as an ideal Fermi gas system (see also [12,21]). This Fermi gas approach enables us to reveal structures of the partition function [15,16,17,18,19,2,20] and BPS Wilson loops [22,23,24] in the ABJM theory, including non-perturbative effects expected from the gravity side [25,10,1]. As a result, it turns out that the ABJM free energy is completely determined by the refined topological string on the "diagonal" local P 1 × P 1 whose Kähler parameters for two P 1 's are equal [2]. In this paper we extend the above analysis to the so-called ABJ theory [26,27], which is the N = 6 CSM with more general gauge group U (N ) k × U (N + M ) −k . The ABJ theory is the low-energy effective theory of N M2-branes on C 4 /Z k with M fractional M2-branes at the singularity. From the AdS/CFT perspective, one expects that this theory is dual to the M-theory on AdS 4 × S 7 /Z k with a discrete torsion, and the type IIA superstring theory on AdS 4 × CP 3 with a nontrivial B-field holonomy. Furthermore the recent studies [28,29] indicate that this theory has also a relation to the N = 6 parity-violating Vasiliev theory on AdS 4 with a U (N ) gauge symmetry, when M, k ≫ 1 with M/k and N kept fixed. Thus the ABJ theory is one of the most important supersymmetric gauge theories having M/string/higher-spin theory dual. The aim of this paper is to determine non-perturbative structure of free energy of the M-theory on AdS 4 × S 7 /Z k with the discrete torsion via the supersymmetry localization, Fermi gas formalism, and refined topological string.
Let us briefly summarize our result 1 . Our starting point is a matrix integral representation for the ABJ partition on S 3 obtained by the localization method [6,31,32]: 1 Recently an apparently similar conclusion was obtained in [30] by a different approach. While the values of the canonical partition function and finite part of grand potential presented in [30] are totally consistent with our values, there is an important difference of physical interpretation between [30] and ours. We will clarify an important difference between conclusions of [30] and our work in section 4.4. which we call the ABJ matrix model. After conjectured in [33], the article [34] has proven that this matrix model has another equivalent representation suitable for a Fermi gas approach as explained in the next section. The Fermi gas expression of the ABJ matrix model is obtained by going to the grand canonical ensemble, i.e. by introducing the chemical potential µ and summing over N , as usual in the statistical mechanics. Moreover one can show that the computation of the grand canonical partition function boils down to construct a series of functions, which can be solved recursively. Thus we can exactly calculate the canonical partition function for various k, M , up to a fairly high N = N max . Indeed we have obtained exact values of the partition function for (k, M, N max ) = (2, 1, 65), (3,1,31), (4,1,62), (4,2,29), (6,1,23), (6,2,21) and (6,3,22), which are partly listed in appendix D. Then we compare these exact data with an expectation from the refined topological string on local P 1 × P 1 with general Kähler parameters, which is a natural generalization of the ABJM case [2]. We will argue that the grand potential J (M ) k (µ) of the ABJ theory is completely determined by the refined topological string free energy, where the parameter g s is related to the Chern-Simons level k as and T eff 1,2 are the effective Kähler parameters given by with the effective chemical potential µ eff defined in (4.29). Here F top and F NS are the free energies of the un-refined topological string and the refined topological string in the Nekrasov-Shatashvili limit, respectively [35]. Although each individual terms in (1.2) have apparent divergences for physical integer k, a careful treatment shows that these divergences are actually canceled as in the ABJM case [19]. Then it remains a finite part, which will be presented in (4.39) for even k and (4.40) for odd k, respectively. Given the grand potential, we find that the ABJ partition function is written as the following sum of the Airy functions, where g ℓ,m is a polynomial explicitly determined by the grand potential J (M ) k (µ), and C and B are the coefficients appearing in the perturbative part of the grand potential, (1.6)

ABJ theory as a Fermi Gas
In this section we will show that the partition function of the ABJ theory on S 3 is described by an ideal Fermi gas system as in the ABJM case [13]. This picture enables us to compute the partition function exactly.

Rewriting canonical partition function
Let us consider the U (N ) k × U (N + M ) −k ABJ partition function on S 3 . We begin with the ABJ matrix model (1.1) obtained by the localization method [6,31,32]. It is not so obvious (except M = 0 corresponding to the ABJM case) whether this expression of the ABJ partition function has an ideal Fermi gas description or not. However, it has been proven [34], after conjectured in [33], that the partition function has another equivalent representation, which is suitable for a Fermi gas approach. Although this representation takes seemingly different forms between M ≤ |k|/2 and M > |k|/2, we will show below that the ABJ partition function can be rewritten for 2 all (k, M ) as .
(2.4) 2 For M > |k|, the ABJ partition function vanishes [33,34] since Z (M ) CS (k) = 0 for this case. This is consistent with an expectation that spontaneously breaking of the supersymmetries occurs in this regime [26,37,38]. Hence we consider the case for M ≤ |k| below.
It turns out thatẐ (N,N +M ) (k) has a simple integral representation Here the product over s runs with the step ∆s = 1. By using the Cauchy identity we can rewrite the partition function as an ideal Fermi gas system, where the density matrix ρ(x, y) is given by In the rest of this section, we will prove that the ABJ partition function (1.1) can be written as (2.1) for all k and M .

For M ≤ |k|/2
For M ≤ |k|/2, the previous study has shown [34] that the ABJ partition function (1.1) is equivalent to If we integrate this integrand along the contour C 1 + C 2 + C 3 + C 4 depicted in fig. 1 (Left), then we can easily see that the integration vanishes by the Cauchy integration theorem 3 . Since the integrals along C 2 and C 4 becomes zero in the limit Λ → ∞, we find (2.10) The right hand side is nothing but (2.1).

Remarks on Seiberg-like duality
From the representation (2.1), one can see that the ABJ partition function transforms properly under the Seiberg-like duality [26] between the ABJ theories with different gauge groups This duality comes [43] from the Giveon-Kutasov duality [44], which has been proven for the S 3 partition function [45]. Let us check this duality in terms of (2.1). First, it is well-known that the pure CS partition function Z (M ) CS (k) enjoys the level-rank duality (See e.g. [43] for a proof) (2.16) Next, using (2.14) one can easily see that the integral representation ofẐ (N,N +M ) (k) in (2.5) is also duality invariant, Finally the phase factor e iθ(N,M,k) given in (2.3) is not duality invariant but appropriately transforms as discussed in [43,45].
In what follows, we will assume k > 0 without loss of generality. Since Z (2.18)

Grand canonical formalism
Let us switch to the grand canonical formalism. We define 4 the grand partition function as the generating function ofẐ (N,N +M ) (k) in (2.4) , One could define the grand potential in terms of the whole partition function Z (N,N+M ) (k) rather than Z (N,N+M ) (k). Then the N -dependent factor of the phase (2.3) can be absorbed by redefining the chemical potential as µ → µ + iM π/2, while the N -independent factor of the phase and pure CS partition function Z (M ) CS (k) can also be absorbed by redefinition of the perturbative coefficient A in (4.4). Particularly, in our definition of the grand partition function (2.19), we will see that corresponding topological string free energy is invariant under the Seiberg-like duality (2.15). If we did not drop the phase in defining the grand partition function, then corresponding topological string free energy do not become duality invariant.
where z is related to the chemical potential µ by (2.20) From the representation (2.7) ofẐ (N,N +M ) (k) as the sum over permutations, one can show that the grand partition function is written as a Fredholm determinant, It is also convenient to introduce the grand potential as Given the grand partition function Ξ (M ) k (z), we can easily come back to the canonical partition functionẐ (N,N +M ) (k) bŷ (2.23) As explained in [17,46], the grand potential consists of a primary non-oscillatory part and an oscillatory part. The non-oscillatory part 5 J (M ) k (µ) satisfies the following relation Then we can deform the integral contour in (2.23) to the whole imaginary axis, by just replacing the total grand potentialJ Now let us describe our method for the exact computation of the partition functions. As discussed in [17] (see appendix A for details), thanks to the Tracy-Widom's lemma [47], the grand canonical partition function is determined by a series of functions φ + l (y), where ρ + and φ + l (y) are given by The oscillatory part is defined by although we will not use this expression explicitly below.

Test of AdS/CFT correspondence
In this section, using our exact data of the partition function, we test the AdS/CFT correspondence between the ABJ theory and M-theory on AdS 4 × S 7 /Z k with a discrete torsion.

Expectation from the gravity side
Here we describe some expectations from the gravity side.

Classical supergravity
The U (N ) k × U (N + M ) −k ABJ theory [26,27] describes the low-energy effective theory of N M2-branes with M fractional M2-branes. In the near-horizon limit, the geometry associated with this brane configuration becomes AdS 4 × S 7 /Z k with M units of discrete torsion realized by a discrete holonomy of the 3-form field [26].
When k ≫ N 1/5 , the curvature becomes very small and we expect that the elevendimensional classical SUGRA on AdS 4 × S 7 /Z k provides a good approximation of the gravity side. The free energy of the classical SUGRA with the boundary S 3 obeys the famous N 3/2 -law [48], given by (see e.g. [49] for a derivation) In the next subsection we compare this with our exact result on the ABJ side in the large N regime with fixed k.

One-loop quantum supergravity
The one-loop correction of the SUGRA can also be analyzed by the technique successful in computing the logarithmic correction to the black hole entropy [50,51,52,53,54]. The authors in [36] have shown that the free energy of the eleven dimensional SUGRA on AdS 4 × X 7 , where X 7 is a seven-dimensional manifold including S 7 /Z k , contains the following universal logarithmic correction, For N = 3 necklace quiver CSM with coincident rank of gauge groups, it has been shown that this logarithmic behavior comes from the large N asymptotics of the Airy function [13] in the perturbative part of partition function (See also [55] for N = 2 case). Strictly speaking, in the presence of the discrete torsion, there might be further massless degrees of freedom and the logarithmic correction could change 6 . However, as we will see in section 4, the exact ABJ partition functions show a nice agreement with the Airy function, and hence the ABJ partition function also exhibit this 1-loop behavior (3.2).

Non-perturbative effects
One expects two kinds of non-perturbative effects on the gravity side as in the ABJM case [25,10]. If we identify a direction of M-theory circle with the orbifolding direction and consider a large k regime, then the eleventh dimension in the geometry AdS 4 × S 7 /Z k shrinks and the bulk theory reduces to the type IIA string theory on AdS 4 × CP 3 . Since CP 3 has a nontrivial 2-cycle CP 1 and a Lagrangian submanifold RP 3 , we expect that the dual type IIA string theory has the worldsheet instanton [25] and the D2-brane instanton [10] corrections characterized by the following weights, respectively, (3.3) As discussed in [8], the worldsheet instanton also receives the following contribution from the coupling to the background NSNS 2-form field We also expect that the D2-instanton has a coupling to the background RR 3-form field. From the viewpoint of M-theory, these instantons correspond to M2-branes wrapping three cycles known as membrane instantons [1]. The worldsheet instanton effects in (3.3) and (3.4) have been successfully reproduced from the ABJ matrix model in the 't Hooft limit [8]. In section. 4, we will show that the ABJ partition function contains all the expected instanton effects and determine the structure from the refined topological string.

Comparison with the gravity side
Let us compare our exact result of the partition function with the classical SUGRA result (3.1). In fig. 2 (Left), we plot the exact values of the free energy logẐ (N,N +M ) (k) listed in appendix D against N 3/2 for some cases. As expected from the gravity side, we observe that the free energy is approximately proportional to N 3/2 in the large N regime.  (Right) compares our result with the classical SUGRA result. In this plot, we fit the ratio between the exact free energy and classical SUGRA free energy by linear functions of 1/N , whose intercepts should be almost 1 if the AdS/CFT correspondence is correct. We easily see that the data points are well fitted by the fitting functions in the large N region, and indeed the values of these intercepts are almost 1. Although we have plotted only for (k, M ) = (2, 1), (3, 1), (4, 2) and (6, 3), we have checked that similar result holds also for (k, M ) = (4, 1), (6, 1) and (6,2). We can reproduce the classical SUGRA result also from the thermodynamic limit of the Fermi gas system as in the ABJM case [13]. In the large N limit, the free energy is approximated by a saddle point of the µ-integration in (2.25) as (3.5) Hence we can estimate the large N free energy if we find the large µ asymptotic behavior of the grand potential, which is captured by a semi-classical expansion of the Fermi gas system. By introducing the quantum mechanical operatorsq andp satisfying [q,p] = 2πik, andq|x = x|x , we can rewrite the density matrix ρ(x, y) as a quantum mechanical operator whereĤ is the Hamiltonian of the Fermi gas system. The asymptotic behavior of the classical Hamiltonian for |q|, |p| ≫ 1 is given by which is exactly the same as in the ABJM case [13]. Thus the same analysis as [13] gives the large µ grand potential and the saddle point µ * as We can easily see that plugging this into (3.5) reproduces the result of the classical supergravity. Using the expression of the saddle point value µ * in (3.8), we can translate the instanton effects (3.3) on the gravity side into the language of the grand canonical formalism. In terms of µ * , we can express the instanton factors as If we assume that the free energy on the gravity side receives a series of multi-instanton corrections, we would expect that the ABJ grand potential have the following expansion, where f l,m (µ) is a polynomial of µ, whose coefficients depend on M and k. Although there is no currently available technique to determine the coefficients f l,m (µ) of instantons from the computation of the gravity side, the ABJ matrix model and its relation to the refined topological string give a very concrete prediction for these coefficients, as we will see in the next section.

ABJ partition function from the refined topological string
In this section we propose that the ABJ free energy including the non-perturbative effects is completely determined by the refined topological string on local P 1 × P 1 . As explained in [7], the partition function of the ABJ theory on S 3 can be analytically continued 7 to the partition function of the L(2, 1) lens space matrix model [39,40] which comes from the pure Chern-Simons theory on S 3 /Z 2 . The lens space matrix model corresponds to the topological string on local P 1 × P 1 via a topological version of the large N duality [59]. Thus we expect that structure of the ABJ free energy is captured by the topological string on local P 1 × P 1 .
For the ABJM case, as the special case of the ABJ theory, this expectation has been confirmed quite successfully. In [2] it is discussed that the grand potential of the ABJM theory corresponds to the free energy of the topological string on the "diagonal" local P 1 × P 1 in large radius frame, where the Kähler parameters of two P 1 's are equal. More precisely, it has been shown, based on the exact and numerical results [15,16,17,19], that the perturbative and worldsheet instanton part of the ABJM grand potential are given by the free energy of the un-refined topological string on the local P 1 × P 1 [13,17], while the D2-brane instanton part and its mixed contribution with the worldsheet instanton are captured by the Nekrasov-Shatashvili limit [35] of the refined topological string on the same local P 1 × P 1 .
In this paper we generalize this argument to the ABJ theory. From the topological string viewpoint, this amounts to consider non-diagonal local P 1 × P 1 , or equivalently local P 1 × P 1 with general Kähler parameters. For this purpose, we decompose J(µ) into the perturbative part J pert (µ) and the non-perturbative part J np (µ) as We propose that structure of the ABJ grand potential, or equivalently the ABJ partition function, is completely determined by the topological string including the non-perturbative effect and test this proposal by using our exact data of the partition functions.

Perturbative part
In order to study the non-perturbative structure of the ABJ grand potential in terms of the exact data, we shall first determine the perturbative part of the ABJ partition function to subtract this from the exact data. In this subsection, we determine the perturbative part of the ABJ grand potential by using the un-refined topological string. As explained in [60], the perturbative part of the free energy of un-refined topological string on a Calabi-Yau manifold (CY) X is given by where J is the Kähler form and c 2 is the second Chern class. From this general expression, we expect that the perturbative part of the grand potential of ABJ theory is given by where T 1,2 are the Kähler parameters of local P 1 × P 1 , whose relation to µ will be explained shortly. Here g s denotes the string coupling g s = 2/k defined in (1.3) and A is given by Apart from the first term in (4.4), A is the so-called constant map contribution analyzed in [14] in detail. The first term − log |Z comes from our definition of the grand partition function (2.19). In order to reproduce the worldsheet instanton factor e −4µ/k (3.9) together with the effect of the B-field (3.4), it is natural to make the following identification of the Kähler parameters With this identification, the Seiberg-like duality (2.18) is naturally realized as the exchange of two P 1 's of local Then the perturbative grand potential in (4.3) becomes where C and B are defined in (1.6). This expression (4.7) of the perturbative part is consistent with the result for the ABJM case [13] and the same as the proposal in [30]. Then, as in the ABJM case, the perturbative canonical partition functionẐ (N,N +M ) pert (k) is given by the Airy function, Below, we will show that this is indeed the correct perturbative partition function by comparing with our exact data. In the large N limit with fixed k, the perturbative part of the free energy becomes The first term reproduces the classical SUGRA result (3.1) and the third term agrees with the logarithmic behavior (3.2) of the one-loop quantum supergravity [36]. Let us compareẐ (N,N +M ) pert (k) with our exact data of partition functions. For this purpose we decompose the canonical partition function aŝ (k) should correspond to the non-perturbative part of the partition function and it is expected to behave aŝ in the large N regime. Fig. 3 shows the plots ofẐ (N,N +M ) np (k) against 2π 2N/k in semi-log scale for (k, M ) = (2, 1), (6, 1), (6, 2) and (6,3). We can easily see that the data points are on the straight lines in the large N regime, which imply the exponentially suppressed behavior k . Thus we conclude that J pert (µ) given in (4.7) is indeed the correct perturbative part of the grand potential. Since this consistency check shows the presence of the "− 1 4 log N " term (3.2) in the canonical partition function, we have also tested the AdS/CFT correspondence at 1-loop level of the dual quantum supergravity [36]. Although we have not explicitly shown, similar result holds also for (k, M ) = (3, 1), (4, 1) and (4, 2).

Non-perturbative effects and the refined topological string
In this section we briefly review [2] and write down an expected form of the ABJ grand potential from the refined topological string on local P 1 × P 1 with non-diagonal Kähler parameters.

The refined topological string
Let us first recall the structure of the un-refined topological string on CY X with Kähler parameters T I . The free energy of the un-refined topological string in the large radius frame is given by [61] (4.14) Hereĝ s is the topological string coupling defined byĝ s = 2πig s . The integer numbers n d g are the so-called Gopakumar-Vafa invariants of X. The free energy of the refined topological string is obtained by a one parameter deformation of the standard topological string free energy originated in the Nekrasov instanton partition function [62]. The refined free energy is determined by the supersymmetric index N d j L ,j R which counts the BPS states in the M-theory compactified on X down to 5 dimensions [63]. These BPS states come from M2-branes wrapping a two-cycle of X with degree d. They behave as massive particles in the 5 dimensions, carrying spins (j L , j R ) of the little group SO(4) ≃ SU (2) × SU (2). The refined free energy takes the form of It is often useful to introduce another invariant n d g L .g R by The un-refined topological string corresponds to the following particular limit: As discussed in [2], the Nekrasov-Shatashvili (NS) limit of the refined topological string plays an important role in the D2-instanton corrections.
Since the refined free energy has a simple pole (∼ 1/ǫ 2 ) in this limit, we introduce the NS free energy as where q = e = e 2πi/gs .

ABJ partition function
As a natural generalization of the ABJM case, we conjecture that the ABJ grand potential can be written in terms of the free energy of the refined topological string on local P 1 × P 1 . For later convenience, we decompose the non-perturbative grand potential J np (µ) into the worldsheet instanton part J WS (µ), the D2-brane instanton part J D2 (µ) and the mixed contributions J WS+D2 (µ), As in the ABJM case [2,8,17], we expect that the perturbative part plus the worldsheet instanton part is described by the un-refined topological string, where g s = 2/k in our definition (1.3). More explicitly, the worldsheet instanton part J WS (µ) is given by 8 8 Note that this expression is ill-defined for physical integer values of k. Hence this formula should be understood as the simple analytic continuation to unphysical values of k. As we will see in section 4.2.3, the divergences at integer values of k are actually apparent and canceled out if we take into account contributions from D2-brane instanton.
where N d 1 ,d 2 j L ,j R denotes the BPS index of local P 1 × P 1 partly listed 9 in table 1. It is also expected that the D2-brane instanton part is captured by the Nekrasov-Shatashvili limit with the effective shift of the chemical potential, In this expression, we treat T eff 1,2 and g s as independent variables, namely As discussed in [2], the effective Kähler parameters are determined by the so-called quantum A-period, whose closed form is not known for the general value of g s . However, for the physical case of integer k, we can write down the effective Kähler parameter T eff 1,2 in terms of a hypergeometric function (see appendix C for an explanation) which gives the effective chemical potential µ eff as A part of the data is extracted from the table in section 5.5.2 of [63]. We are grateful to Can Kozçaz to tell us the other data of the BPS index, which is not listed in [63].
The second term of the right hand side in (4.26) takes the following explicit form This shift of the chemical potential also plays an important role to describe the mixed contribution of the worldsheet instantons and D2-instantons by To summarize, we conjecture that the total ABJ grand potential is given by (4.32) We can rewrite this expression in the form of (3.10) and read off the coefficient f l,m (µ) of the term e −(2ℓ+4m/k)µ , as expected from the gravity side. By exponentiating J (M ) k (µ) in (3.10), we find the following expansion where g l,m (µ) is a polynomial of µ explicitly determined by the coefficient f l,m (µ). Then we come back to the canonical formalism by applying the integral transform (2.25) to (4.33), Since the Airy function satisfies the differential equation the canonical partition function given in (4.34) can be rewritten as a combination of the Airy function and its first derivative alone. The expression (4.32) has apparent divergences at some values of the Chern-Simons level k, in particular at physical integer k. For example, for odd k such divergences occur when m in (4.32) is a multiple of k for the worldsheet instanton and even integer for D2instanton, respectively. In the next subsubsection, we discuss that these divergences are actually canceled out and we compute the finite part of the grand potential.

Cancellation of divergence between instantons
Here we show the cancellation of apparent divergences in the grand potential (4.32) and compute its finite part.

Even Chern-Simons level
Let us first consider the even k case; we set k = 2n 0 with some integer n 0 . By expanding 10 the m-th worldsheet instanton term with m = n 0 ℓ (ℓ ∈ Z) around k = 2n 0 , we find that the worldsheet instanton part (4.25) has poles at k = 2n 0 : where we have introduced the notation The contribution (4.30) from F NS , which gives the part of the D2-instanton correction, has also poles for arbitrary m: It is easy to see that these poles are canceled with each other if It turns out that the non-zero values of the BPS index N d 1 ,d 2 j L ,j R for local P 1 × P 1 appear only for the spins obeying the condition (4.38), and hence the poles are indeed canceled. After the pole cancellation, we find the finite part of the grand potential as 11 Some comments are in order here. First, when taking the limit k → 2n 0 in our general expression (4.32), we should treat k and T eff 1,2 as independent variables since we have imposed the condition ∂T eff I /∂g s = 0. This means that k in the expression of T eff 1,2 in (1.4) is set to 2n 0 before taking the limit k → 2n 0 . Second, the resulting finite part (4.39) is invariant under the Seiberg-like duality M ↔ k − M for even integer k. The same comments can be applied also to the odd k case.

Odd Chern-Simons level
When k is odd, there are also apparent divergences in the worldsheet instanton with m = kℓ and the D2-instanton with m = 2ℓ (ℓ ∈ Z). As in the even k case, similar calculation shows that these divergences are canceled again and the finite part of the grand potential is 12 11 We have used the definition of polylogarithm Lis(z) = ∞ p=1 z p p s . 12 We have used ∞ p=1 x 2p−1 2p−1 = Arctanhx.
where P l denotes the projection to even l,

Test of our proposal
Let us test our conjecture of the finite part (4.39) for even k and (4.40) for odd k, expected from the refined topological string. Since the odd number of D2-instanton does not appear in the finite part of the expected grand potential for odd k in (4.40), we can rewrite the expansion of the grand potential (4.34) as 13 whereĝ m (µ) is a polynomial of µ. Since we know the values of the BPS index N d 1 ,d 2 j L ,j R up to d 1 + d 2 = 7 from table 1, we can explicitly write downĝ m (µ) up to m = 7. Using these coefficientsĝ m (µ) (m = 0, · · · , 7), we define the putative partition function If our conjecture is correct, then this quantity should be the canonical partition function including the instanton effects up to the order of 7-th worldsheet instanton correction. Hence we expect that the difference between the exact partition functionẐ (4.44) in the large N regime. Here α is some non-negative integer depending 14 on k and M . In fig. 4, we plot the quantity  gives strong evidence for our conjecture. We can argue more precisely as follows. IfẐ 7−inst properly contains the instanton effects, then this quantity (4.45) should be exponentially suppressed by e −2π 2N k in the large N regime. On the other hand, ifẐ 7−inst had wrong information at 6-th or lower order of worldsheet instanton effects, we could observe an exponential blow-up of this quantity. The semi-log plots in fig. 4 (Left) show that such exponential blow-up does not occur. We also see from the log-log plots in fig. 4 (Right) that the data points give a power law behavior in not so large N region. This behavior is consistent with our expectation (4.44) since the data should show such a power law behavior before going to exponentially suppressed region. Thus we conclude that the grand potential (4.32) coming from the refined topological string is correct at least up to the order of 6-th worldsheet instanton corrections. In particular the case of (k, M ) = (4, 1), plotted in the second bottom of fig. 4 (Left), clearly exhibits the exponentially suppressed behavior. This strongly supports our expectation (4.44).
To summarize, all cases we studied strongly support our conjecture that the ABJ grand potential is given by the general expression (1.2) in terms of the refined topological string on local P 1 × P 1 , with an appropriate identification of the Kähler parameters (1.4).

Comments on Matsumoto-Moriyama proposal
Recently an apparently similar form of the grand potential was proposed in [30] by a different approach, which uses a relation between the ABJ partition function and the half-BPS Wilson loops of the ABJM theory. The work [30] have calculated the ABJ partition function exactly or numerically with high precision for (k, M, N max ) = (2, 1, 4), (3, 1, 3), (4, 1, 2), (4, 2, 2), (6, 1, 3), (6, 2, 2) and (6,3,2). These values are totally consistent with our exact values listed in appendix D. However, there is an important difference of physical interpretation between [30] and ours. This difference comes from an ambiguity for taking the limit to a physical Chern-Simons level in the apparently divergent grand potential (4.32). Suppose that we take the limit k → k 0 (k 0 ∈ Z) in (4.32). In section 4.2.3, we have expanded (4.32) around k = k 0 after fixing k = k 0 in the effective Kähler parameters T eff I since we have imposed ∂T eff I /∂g s = 0, while the authors in [30] expanded the total expression (4.32) around k = k 0 . This difference of taking the limit affects the finite part 15 of the grand potential after the cancellation of divergence. Since the latter prescription does not agree with the exact data, the authors in [30] added an extra new term, whose origin is unclear from the topological string perspective, in such a way that the finite part agrees with their exact and numerical data. It turns out that the finite part of our grand potential, corresponding to (4.39) and (4.40), agrees with the one given in Figure 4 of [30]. However, our grand potential before cancellation (4.32) is different from the proposal in [30] with an extra term added. We believe that our prescription is more natural since everything can be completely explained by the refined topological string.

Discussions
In this paper we have studied the partition function of the ABJ theory on S 3 . By using the Fermi gas formalism and Tracy-Widom's lemma [47], we have exactly computed the ABJ partition function for (k, M, N max ) = (2, 1, 65), (3,1,31), (4,1,62), (4,2,29), (6,1,23), (6,2,21) and (6,3,22). These exact data enable us to test the relation between the ABJ partition function and the free energy of the refined topological string on local P 1 × P 1 with non-diagonal Kähler parameters. We conclude that the ABJ partition function is completely determined by the refined topological string including full series of the membrane instanton effects in the M-theory dual.
Although here we have focused on the usual AdS/CFT correspondence between the ABJ theory and the M-theory on AdS 4 × S 7 /Z k , it is also interesting to study the higher spin limit [28,29], corresponding to the regime where M, k ≫ 1 with M/k and N kept fixed. It would also be interesting to study the eigenvalue problem of the Hamiltonian (3.6) of the ABJ Fermi gas, as in the case of the ABJM theory [20]. We leave this for future work.
Throughout many previous works about the ABJM theory and this work, we have seen that the (refined) topological string is a very powerful tool to determine the structures of the partition function and the BPS Wilson loops in the ABJ(M) theory. However, if we consider more general M2-brane theory beyond 16 the ABJ(M) theory, the correspondence to the topological string is rather obscure, and it might be the case that there is no such relation in general. Therefore even if we can exactly compute the partition function for the case other than the ABJ(M) theory for some discrete values of parameters, we do not have a good guiding principle to extend the non-perturbative effects to more general parameters. Although this problem can be overcome for the "orbifold ABJ(M) theory" [46] due to a relation to the ABJ(M) theory, we need to develop a new approach to study more general M2-brane theories. It is very interesting to study the instanton effects in other M2-brane theories having AdS 4 dual. formula (2.27). Our algorithm is essentially a simple generalization of the ABJM case [16,17].
As discussed in section 2, the ABJ grand partition function is given by with the density matrix (2.8) . Here we define multiplication and trace for certain matrices ρ 1 , ρ 2 as respectively. Then we decompose the density matrix into the even and odd parts: In terms of ρ ± , we rewrite the grand partition function as Note that ρ ± takes the form of Therefore, both ρ ± have the form of where the Tracy-Widom's lemma [47] is applicable. Then the power of ρ ± is determined by a series of functions φ ± l (y) At this stage, it seems that we should find the two series of the functions φ + l (y) and φ − l (y) to obtain the grand potential. However, similar to the ABJM case [17], one can show the following convenient identity, which implies that we need information of only φ + l (y). Plugging this into (A.4), we arrive at (2.26). In the rest of this section, we present a concrete algorithm to compute φ + l (y) and Trρ 2n + exactly.

A.1 Algorithm for even Chern-Simons level
By changing variables as u = e y k , v = e y ′ k , the recursion equation (2.27) becomes If we assume the form of φ + l (u) as The symbol poles indicates the summation over all the poles of the integrand. From φ + l (u), we can find ρ 2n + (u, u) as (A.11) By expanding ρ 2n + (u, u)/u in terms of the power of log u where R (j) n (u) is some rational function of u, then the trace of ρ 2n + can be obtained as a sum of residues Tr ρ 2n (A.13) 17 Here we have used an integral formula noted in [16], where C(v) is a rational function and Bj+1 is the Bernoulli polynomial. Choosing the branch cut of log v as the positive real axis, the integral contour γ goes from +∞ to 0 infinitesimally below the branch cut and then to +∞ infinitesimally above the branch cut.

A.2 Algorithm for odd Chern-Simons level
When k is odd, by introducing the variables u = e y/2k , v = e y ′ /2k , we can rewrite the recursion relation (2.27) as By expanding φ + l+1 as (A.9), we find Then ρ 2n + (u, u) is rewritten as (A. 16) In a similar way as the even k case, by expanding ρ 2n + (u, u)/u in terms of the power of log u the trace of ρ 2n + can be obtained as a sum of residues Tr ρ 2n In this appendix, we will briefly discuss the structure of the highest transcendental term of the partition functions of the ABJ theory listed in appendix D, and the partition functions of the ABJM theory in [17]. As mentioned in [17] for the k = 1 ABJM theory, the coefficient of the highest transcendental term π −N in the canonical partition function can be expressed in terms of the Hermite polynomial using the relation We will see that the partition functions of ABJ theory also have a similar structure.

B.1 (k, M ) = (2, 1)
The partition function of (k, M ) = (2, 1) case has the structure Let us consider the highest transcendental term π −N in Z (N,N +1) (2). We observe that such highest term of φ l (u = 1) comes from the (log u) l term in the expansion (A.9) Tr ρ 2m + with m ≥ 2 is always less transcendental compared to π −2m , and the only highest transcendental term comes from Tr ρ 2 + = 1 32π 2 . Thus we find that the generating function of the highest term π −N is given by (B.5) We further find ∞ m=1 Tr ρ 2m In a similar manner, the highest transcendental term for the (k, M ) = (4, 1) case is found to be (B.7) The next-to-highest term is Therefore, we find

B.3 Summary of the Higher Transcendental Part of Grand Partition Functions
By inspecting the exact partition functions, we find a similar structure for other cases.
Here we summarize the higher transcendental part of grand partition functions Ξ The classical A-period is a solution of the Picard-Fuchs equation (C.1) For the diagonal case z 1 = z 2 = z, this reduces to where we have used the identity As discussed in [17], the effective Kähler parameter is given by the quantum A-period. For even k, the quantum parameter q is equal to the classical value q = e iπk = 1, and the instanton factors for the D2-brane become "diagonal" although the worldsheet instanton factors are non-diagonal e −T 1 = e −T 2 . Therefore, the quantum A-period is reduced to the classical period (C.5). In [19], it is conjectured for the ABJM case that the quantum period for the odd k case, corresponding to q = −1, is also expressed in terms of the hypergeometric function, obtained by simply replacing z → z 2 in (C.5). We expect that this is also the case for the ABJ theory.