Mass deformed ABJM theory on three sphere in large N limit

In this paper the free energy of the mass deformed ABJM theory on S3 in the large N limit is studied. We find a new solution of the large N saddle point equation which exists for an arbitrary value of the mass parameter, and compute the free energies for these solutions. We also show that the solution corresponding to an asymptotically AdS4 geometry is singular at a certain value of the mass parameter and does not exist over this critical value. It is not clear that what is the gravity dual of the mass deformed ABJM theory on S3 for the mass parameter larger than the critical value.


Introduction
The mass deformed ABJM theory [1][2][3] is the theory obtained by deforming the three dimensional U(N ) k ×U(N ) −k N = 6 superconformal Chern-Simons theory (called the ABJM theory) [4] with a set of relevant operators including mass terms for the bi-fundamental chiral multiplets. While the ABJM theory describes the stack of N M2-branes, the mass deformed ABJM is expected to describe the bound states of the M2-branes and the M5branes through the fuzzy sphere configuration given in [1,5]. This theory has special features which make it worth studying. One of them is that the theory has the N = 6 supersymmetry, which is the (almost) maximum amount of the supersymmetries in three dimension. 1 Nevertheless this theory is not conformal, hence has non-trivial dynamics and JHEP03(2017)121 a renormalization group flow. Furthermore, in the large N limit this theory will have a gravity dual which should be obtained by a deformation to the gravity dual of the ABJM theory corresponding to the mass terms. Therefore, this theory will be one of the basic models to be investigated in the large N limit.
To study a supersymmetric field theory, we can use the localization technique [7][8][9] which enables us to obtain the exact partition function as well as some supersymmetric correlators. Each of these results is, however, given typically by a matrix model, i.e. an integration over the N × N matrix variables. It is highly non-trivial to take the large N limit in these matrix models.
In this paper, as in our previous work [10], we continue to study the partition function Z of the mass deformed ABJM theory on S 3 in the large N limit. 2 We find a new solution of the large N saddle point equation with an arbitrary mass parameter and compute the free energy F ∼ N 2 for the solution. 3 We also generalize the ansatz to obtain the free energy F ∼ N 3/2 [10] in full extent, and find that the saddle point solution can not exist for the mass parameter larger than a certain critical value. Because the classical supergravity on an asymptotically AdS 4 spacetime has F ∼ N 3/2 , there would be no gravity duals for the mass deformed ABJM theory on S 3 with the mass parameter larger than the critical value.
This result seems surprising, as the critical mass is reached by a finite and relevant deformation from the ABJM theory. Nevertheless, we can argue that this phase transition indeed occurs. If the dimensionless mass parameter m, which is the mass parameter normalized by the radius of S 3 , is small enough, the free energy F will behave as F ∼ N 3 2 since the theory reduces to the ABJM theory in the limit m → 0. The factor N 3 2 can be interpreted as 1/G N , hence this free energy is consistent with the classical supergravity. On the other hand, if m is sufficiently large we can integrate out the bi-fundamental hypermultiplets first in the computation of the partition function. As a result we will obtain F ∼ N 2 . Indeed, for the new solution we find the free energy scales like F ∼ N 2 (see (3.5) and (3.18)). Therefore, it is possible to have a phase transition in the interpolating regime. 4 The phase transition may be similar to the confinement/deconfinement transition if we regard the change of the mass parameter as a renormalization group flow. We will discuss this aspect in [14].
Note that this phase transition is absent in the N = 2 * supersymmetric Yang-Mills theories on S 4 which is a four dimensional analogue of the mass deformed ABJM theory. For this theory in the strong 't Hooft coupling limit the saddle point solution and the free energy are smooth under the change of the mass parameter [15,16]. Indeed, the free energy of the N = 4 supersymmetric Yang-Mills theory, which is the massless limit of the theory, 2 There are several large N results for the mass deformed ABJM theory [10][11][12]. In [11,12] the authors analyzed the theory by continuing the Chern-Simons levels k and −k to complex numbers, and obtained the saddle point solution which is different from our solutions discussed in the following sections. The solution in [12] may correspond to those discussed in appendix A. Also, in [10] we found two solutions in the region of small mass parameter ζ/k < 1/4. We argue that one of them does not satisfy the saddle point equation at a boundary. 3 We call F = − log Z as the free energy even though we consider the theory on S 3 . 4 The critical value of the mass parameter we found could be different from this phase transition and represent another phase transition, for which we do not have any physical reason to occur.

JHEP03(2017)121
is F ∼ N 2 also, thus both of the massless and the infinite mass limits are consistent with the gravity duals and can be smoothly connected. Needless to say, further investigations of the phase transition are desirable. In particular, we should study the vacuum solution in the supergravity corresponding to the mass deformed ABJM theory on S 3 with an arbitrary mass parameter. We also expect that this kind of phase transition will occur also in the other theories on S 3 describing the M2-branes in various backgrounds such as [3,[17][18][19]. We hope to report on these in near future. This paper is organized as follows. In the next section we introduce the partition function of the mass deformed ABJM theory which is expressed as a 2N dimensional integration. We also write down the saddle point equations to evaluate the large N limit of the partition function. In section 3 and section 4 we solve the saddle point equations and determine the free energy F = − log Z, in the large N limit for various values of the mass deformation parameter. In section 3 we consider the problem in the limit N → ∞ with k kept finite. In section 4 we take the 't Hooft limit k, N → ∞ with k/N finite. In both sections we also evaluate the vacuum expectation values of the 1/6 BPS Wilson loops for the saddle point configurations and argue the interpretation of our results. Section 5 is devoted for discussion and comments on future directions. In appendix A we comment on another solution to the saddle point equations for finite k. This solution give the free energy which is larger than that obtained in the same parameter regime in section 3. Appendix B contains the computation of the O(1/N ) corrections in the saddle point equations in section 3.1 and 3.2.1, which are though irrelevant to the large N free energy. In appendix C, we rederive the solution which has the gravity dual in a similar way in [10].
2 Saddle point approximation of free energy As in [10], we will consider the mass deformed ABJM theory which is the 3d N = 6 U(N ) k × U(N ) −k SUSY Chern-Simons matter theory with the Chern-Simons level ±k deformed by the mass terms and the interaction terms which preserve the N = 6 supersymmetry. The action of this theory on S 3 can be written as 5 where (σ,D) are the auxiliary component fields in the U(N ) k vector multiplet (A µ , σ, λ α , D), and ( σ, D) those in U(N ) −k vector multiplet (see e.g. eq. (3.23) in [23]). Here ζ is a real parameter which is related to the mass of the matter fields as m = r −1 S 3 · ζ/k. The supersymmetric gauge theories on the three sphere were studied in [20][21][22][23], with the help of the localization technique. For the mass deformed ABJM theory, it was found that the partition function is given by the following 2N dimensional integration Here λ i and λ i (i = 1, . . . , N ) respectively denote the eigenvalues of the scalar component field in the vector multiplet for U(N ) k and those for U(N ) −k , which are real constant numbers characterizing the saddle point configurations of the fields in the localization computation as In the limit of N → ∞, these 2N integrations can be evaluated by using the saddle point approximation with the eigenvalues (λ, λ) being solutions to the following saddle point equations Note that λ i and λ i can be complex numbers for the solutions to the saddle point equations, although the original integration contour in the partition function (2.2) is the real axis.
For ζ ∈ R, as argued in [10], we can consistently impose the following reality conditions to the eigenvalues:

JHEP03(2017)121
Under this assumption, the saddle point equations (2.6) reduce to where x i and y i denote the real parts and the imaginary parts of the eigenvalues respectively, i.e.
In the following sections we will solve the saddle point equations (2.8) and (2.9), and evaluate the free energy for the solutions, which is written under the constraint (2.7) as (2.12) We will also compute the vacuum expectation value of the supersymmetric Wilson loops where A µ and σ are the component fields of the U(N ) k vector multiplet and A µ and σ are those in the U(N ) −k vector multiplet. The closed path C is an S 1 in S 3 which is determined by the supersymmetry used in the localization technique. These Wilson loops preserves the 1/6 of the N = 6 supersymmetry [20,[24][25][26] and hence can be computed by the matrix model 2 with the help of the localization method [20]. For simplicity we will consider only the Wilson loops with the fundamental representations, whose vacuum JHEP03(2017)121 expectation values are given in the saddle point approximation as with the substitution of the solution (λ, λ) to the saddle point equations (2.6). 6 Below we will assume ζ ≥ 0 without loss of generality; the results for ζ < 0 are easily generated with the help of the following Z 2 "symmetry" of the partition function (2.2) We will also denote m ≡ ζ/k which is the mass of the hypermultiplets.

Large N limit with finite k
In this section we study the saddle point equations for the free energy of the ABJM theory in the limit N → ∞ with the Chern-Simons levels k kept finite.

Solutions in large ζ/k limit
First, we consider the case ζ/k 1 (which is equivalent to the large radius limit of S 3 with a finite ζ/k). The saddle point equations further are simplified in this regime. We take the following ansatz: where u j and v j are of O(N 0 ). The shift in the real part ζ/k cancels the term −2πiζ, while the last terms in the saddle point equations (2.6) are approximated as which is canceled by the shift in the imaginary part of the eigenvalues. We are finally left with the following equations without ζ

JHEP03(2017)121
The free energy (2.12) also is simplified in this limit as Note that the equations (3.3) and (3.4) are in the same form as the saddle point equations of the matrix model for the Chern-Simons theory without the matter fields, which were analyzed in [27][28][29][30] (with the pure imaginary Chern-Simons levels k → ik). In that sense the correction δf in the free energy corresponds to the free energy of the pure Chern-Simons theory in the large N limit.

Eigenvalue distribution
With the ansatz (3.1), the solution of the saddle point equations is the following: Here g(s) is some function and ∆ is a constant both of which being of O(N 0 ), while n(j) is some integer which can be different for each j. Indeed, after the substitution of these expressions the real part of the saddle point equation Let us evaluate the deviation of the free energy δf for this solution. The second term is obviously of O(N 0 ). Approximating the cosine hyperbolic factor by 1 we can compute the third term exactly as (3.9) Hence the free energy in the large N limit is with the solution,

JHEP03(2017)121
where we have fixed the values of ∆ and n(j) as ∆ = − 1 2 and n(j) = 0, as discussed in appendix B.1, though they actually do not affect the free energy (3.10).
In the definition of the partition function, we neglected the (1/N !) 2 factor coming from the integration over U(N )×U(N ). Including this factor, the free energy becomes f ≈ 4πN 2 ζ k . There is an intuitive way of understanding our results above. First recall that in the mass deformed ABJM theory the mass of the matter fields (adjoint hypermultiplets) is uniformly m = ζ/k which is induced by the Fayet-Illiopoulos term. Hence in the regime ζ/k 1 the matter fields can be integrated separately as the massive free hypermultiplets, which gives (3.12) This precisely reproduces the leading part of the free energy (3.5). On the other hand, after integrating out the matter multiplets in the mass deformed ABJM theories we are left with the pure Chern-Simons theory (with the induced Yang-Mills terms). The saddle point equations for the shifted eigenvalues u i + iv i (3.3) and (3.4) can be interpreted as the saddle point equations for the partition function of this reduced theory.
Here we also comment on the F-theorem [31,32]. Our computations show that the free energy is an increasing function of mass parameter m = ζ/k. However, at the IR fixed point the theory will be the N = 2 pure Chern-Simons theory which has smaller free energy than the one of the UV theory which is the ABJM theory. Thus, our result is consistent with the F-theorem. Indeed, in [32], for free massive theory, the free energy was shown to be increasing function of the mass. 7

Wilson loops
Here we shall compute the vacuum expectation values of the supersymmetric Wilson loops (2.14). First consider the Wilson loop associated with U(N ) k gauge group in U(N ) k × U(N ) −k . With the substitution of the saddle point configuration (3.1) with (3.7) we obtain Similarly, the Wilson loop for U(N ) −k can be computed as If we neglect the O(N −1 ) deviations in the exponent, the leading part of the right-hand side vanishes in both cases. The vanishing of the leading part of the vacuum expectation values of the Wilson loops may have some physical implication, which will be discussed in [14].

JHEP03(2017)121
3.2 Finite ζ/k Below we will consider the limit N → ∞ with both k and ζ kept finite. In this limit the mass deformed ABJM theory is expected to correspond to the eleven dimensional supergravity with some classical geometry which will be asymptotically AdS 4 × S 7 /Z k . We first show that for any finite ζ/k, there is a solution which is a simple generalization of the solution obtained in the last section and has the same expression for the free energy f ∼ 4πN 2 ζ/k in the large N limit. Next we study the solutions which has the free energies f ∼ N 3/2 . We find that the solution to the saddle point equation is unique for ζ/k < 1/4. 8 For ζ/k > 1/4, on the other hand, we find there are no solutions with f ∼ N 3/2 .

Solution with f ∼ N 2 for any ζ/k
Let us start with the small generalization of the ansatz in the last section (3.11) (λ i = x i +iy i ) with g(s) and h(s) some functions and ∆ some real constant, both being of O(N 0 ). Indeed we can show that the left-hand side of the imaginary part of the saddle point equations (2.8) vanishes with the help of the following trivial generalization of the identities (3.8) Similarly the O(N ) terms in the real part of the saddle point equation (2.9) vanish due to We can also solve the O(N 0 ) part of the saddle point equations to determine (f (s), g(s), ∆), though they are irrelevant to the leading part of the free energy. The computation is parallel to those in the large ζ limit and displayed in appendix B.2.
The free energy f for this solution also takes the same form as in the case of the large ζ limit. In the limit N → ∞ the leading parts of the first two terms in (2.12) precisely cancel with each other, hence only the last two terms are relevant

JHEP03(2017)121
To obtain the second line it is convenient to replace the summations over i, j with the integrations of continuous variables s ∼ i/N and s ∼ j/N over s, s ∈ (0, 1). The O(N log N ) denotes the error due to the difference between the integrations and the original discrete summation.

Solutions with
Now we shall go on to the solutions with the free energy f ∼ N 3/2 . We use the continuous notation λ i → λ(s) with s ∼ i/N + const. and take the following form: where z 1 and z 2 are N independent arbitrary complex valued functions of s. 9 Note that the transformationλ only changes the ordering of the U(N ) index of theλ, thus the gauge symmetry. This means that the configuration {λ(s),λ(s)} is equivalent to {λ(s),λ(−s)}. We can see that the form (3.19) includes the ansatz taken in [10] for pure imaginary ζ and for real ζ with the gauge transformation (3.21). 10 Note that here we do not require the reality condition (2.7). 11 The above gauge symmetry also allows us to assume that Re(z 1 (s)) is a monotonically increasing function with respect to s. For simplicity, in this section we shall further assume that the profile functions z 1 (s) and z 2 (s) are piecewise continuous in 0 ≤ s ≤ 1 for this choice of the ordering.
We believe that the form (3.19) is the most general form which gives f ∼ N  2 ) terms in the free energy in order to obtain f ∼ N 3 2 , which makes finding other possible forms highly difficult.
We will evaluate the free energy for the configuration (3.19) which is indeed O(N 3 2 ). The Chern-Simons term, which is proportional to k, and the FI term, which is proportional to ζ, are easily evaluated to The following generalization also gives the large N scaling of the free energy f ∼ N 3/2 However, this ansatz is reduced to (3.19) by an O(N −1/2 )-shift of z1(s) which is irrelevant to our leading analysis. 10 The large N analysis in this section includes those in [10] and the simplest examples in [31,33].
Furthermore, as we will see below, the one in this section is much simpler than those. 11 In the appendix C, we solve the saddle point equation imposing the reality condition, which will be useful to compare the previous studies including [10].

JHEP03(2017)121
For other logarithmic terms in the free energy, for example, we will use the decomposition ds ln(sinh 2 (z)) = 2 ds sgn(R(s)) z where R(s) is a real function, and the decomposition which is obtained by replacing sinh by cosh in (3.24). We take R(s) = Re(z 1 (s) − z 1 (s )). Then, we can see that the terms linear in z cancel each others: Remaining terms can be evaluated by using a formula (here dot · is the abbreviation for d ds ): u(s 0 ) = 0 and the path C ± is a straight line between t = ±v(s 0 ) and t = √ Nu(s)| s=s 0 with N → ∞. Note that the cosh in the formula can be replaced with sinh. Then, the remaining parts of the free energy is where we have assumedż 1 (s ) > 0 and there is no singularities in t-plane for deforming the contour C ± . However, there are singularities in the action where the cosh factor vanish.
We can see that if

JHEP03(2017)121
there is no obstruction for the deformation of the contour. If this is not the case, we can shift z 2 → z 2 + in/2, where n is an integer, to satisfy the condition (3.32). Because the action is invariant under this, we conclude that the free energy is where h ∈ Z/2 such that the condition is satisfied. 12 In the above derivation of the free energy f (3.33), the assumption that Re(z 1 ) is monotonically increasing (after the eigenvalues are rearranged so that the profile functions are piecewise continuous in s) is crucial. This assumption is violated if the eigenvalue distribution has self-overlapping region after projected onto the real axis. In this case (3.33) is corrected by the cross terms such as log sinh π(λ i − λ j ) with λ i and λ j in two different segment with overlapping shades.
Here we will argue that such an overlapping configuration can not be the saddle point solution. First suppose that the values of Im(z 1 ) are different for these two segments and denote the difference as Im(∆z 1 ). We can evaluate the cross terms again using the formula (3.26) and (3.27), but with the contour C ± extended by a straight line [±v(s 0 ), ±v(s 0 ) + iπ √ N Im(∆z 1 (s 0 ))]. Since the integration of log(cosh(t)e −t ) over πi vanishes, the contribution of Im(∆z 1 ) to the free energy depends on the remainder of √ N Im(∆z 1 ) divided by 1. This implies that the profile functions obtained from the variation of the free energy depend non-trivially on the way to take the limit N → ∞, hence the N → ∞ will be ill defined. To obtain a well defined large N limit, we have to choose Im(∆z 1 ) = 0 at the level of the ansatz. In this case, however, the original saddle point equation ∂f /∂(λ i , λ i ) will not be solved by the variational problem, as the degrees of freedom of the variations will be fewer than those for the smooth eigenvalue distributions for multiple segments. The above argument shows that there are no solutions with overlapping segments, at least, if we assume f ∼ N 3/2 . Below we will consider only the cases without overlapping.

JHEP03(2017)121
and 0 = ikz 1 (s) + 4 1 z 1 (s) (z 2 (s) + ih), (3.38) for the variation of z 2 , which implies that These implies that where s 0 is a complex integration constant. Thus, we have where z 0 is the integration constant and Note that because z 1 (s) should be a continuous function of s we defined √ s − s 0 as a continuous function of s although we allowed the overall sign ambiguity. This overall ambiguity should be fixed by the condition that z 1 should be a monotonically increasing function of s.
To obtain the solutions, we need to specify the locations of the boundary points and the solutions should satisfy the condition (3.34) everywhere. Note that for general ζ, above discussions are valid. Indeed, the solutions for pure imaginary ζ also are included in the above solutions. Now we assume ζ is real and there is only one segment in the eigenvalue distributions. We will choose s 0 = ic where c is real by shifting s. Because there is one segment, we choose the boundary points as s = s b and s = s b + 1. Then, the boundary condition is where (γ 1 ) 2 = 1 representing a choice of the boundary values, 13 which lead (assuming ζ = 0) (3.45) 13 The other possibility is z1(s) = g √ s + z0 (s = [0, 1]) which satisfiesż1(s = 0) = ∞ and z0 is fixed by the boundary condition at s = 1. However, considering s ∼ 0, we see that for the condition (3.34) Re(z0) = 0 is needed. This is not satisfied for generic ζ/k, for example, with h = 0, z0 = 0 means ζ/k = 1/4 (see also (3.35)).

JHEP03(2017)121
We obtain z 0 from these boundary conditions: which also lead Thus, we find (3.49) Below, we will check that the solution is indeed a continuous function of s. First, we define thus we find that s = −γ 1 /2 for s = s b and s = γ 1 /2 for s = s b + 1. With these, we find (3.54) which leads (3.55) Here we introduced γ 2 which satisfies (γ 2 ) 2 = 1 for the sign ambiguity of z 0 . In order to satisfy the boundary condition z 2 + ih = ±i/4, we need at the boundaries. 14 This condition implies γ 2 is fixed by the choice of the overall sign in the l.h.s. of (3.56). Furthermore, we will see that for m = ζ/k > 1/4, these conditions are 14 The condition is only for the sign because for γ 2 = 1.

JHEP03(2017)121
not consistent with the continuity of the factor √ D where for s . As we will see below, Re(D) is negative for m > 1/4. Then, the phase e iθ = √ D/| √ D| satisfies π/4 < θ < 3π/4 or −π/4 > θ > −3π/4 and we can easily see that | √ D| > 0. On the other hand, at the two boundaries, we can see that Im( √ D) should have different signs for m > 1/4. These are inconsistent with the continuity for s . For h = 0, we easily see that Re(D) is indeed negative. For h = 0, we find where we have used |s | ≤ 1/2 and |Im(m c )| = |h| ≥ 1/2. Therefore, there are no solutions for m > 1/4. We can also show that there are no solutions for m ≤ 1/4 and h = 0 because (Re(m c )) 2 −(Im(m c )) 2 = m 2 −h 2 < 0 and −|m c | 4 +|m 2 c hs | < |m c | 2 (−(m 2 +h 2 )+|h/2|) < 0, where we have used |h| ≥ 1/2, which implies Re(D) < 0 using (3.59). Therefore, only the possibility is for m ≤ 1/4 and h = 0. For this case, we see that for γ 1 = −1 Re(ż 1 ) = 0 at s = 0. Thus this solution violates the condition (3.34) and we should set γ 1 = 1. The solution is unique and given by where the sign ambiguity of the 1 16m 2 − 1 + i s m is fixed by requiring the condition Re(ż 1 ) ≥ 0 because we arranged the ordering of the eigenvalues such that z 1 (s) is increasing function of s.
Finally, we will consider the multiple segments solutions. The real part of such a solution should not intersect each other because of the extra interactions as explained before. Then, the solutions are just a sum of the single segment solutions with N a eigenvalues where a N a = N . However, the unique single segment solutions for m < 1/4 with different N always have an eigenvalue such that Re(λ) = 0. Thus, there are no multiple segment solutions. 15 Therefore, we conclude there is a unique solution for m = ζ/k < 1/4, and no solutions for m > 1/4. We can check that the solution for m = ζ/k < 1/4 is indeed solution I in [10] which is derived also in appendix C. The free energy of this solution is

62)
15 So far, we have neglected a possibility that the solutions with different h which have same z1 and z2 at a boundary. However, this is not possible because the cancellation of the boundary term requires that (z2 + ih) 2 also should be same at the boundary.

JHEP03(2017)121
as computed in [10]. We can also evaluate the Wilson loop for the solution. The exponent of Wilson-loop can be evaluated and is given as Note that the real part of the exponent does not depend on ζ and forW (C), the result is same. We also note that W (C −1 ) = W (C) where C −1 is the loop C with the inverse direction. This Wilson loop correspond to the BPS M2-brane wrapping the M-circle, and √ N factor represents the tension of the M2-brane.

't Hooft limit
In this section we consider the 't Hooft limit, N, k, ζ → ∞ with N/k and ζ/k kept finite. Note that the mass of the chiral multiplets is proportional to ζ/k, and hence finite in this limit.

Strong 't Hooft coupling limit
First we consider the strong 't Hooft coupling limit: k N . In this case it is easily seen that the eigenvalue distributions and the free energies reduce to those obtained for finite k in section 3. Indeed, if we use the continuous notation λ i → λ(s) with s = i/N − 1/2 the saddle point equation (2.6) is found to depend on (N, k, ζ) only through their ratio (N/k, ζ/k). Hence, as the parameters in the strong 't Hooft coupling limit 1 (k, ζ) N can always be rescaled so that 1 N while k and ζ are finite, we conclude that our analysis of the saddle point solutions and the free energies in the latter regime are still valid in the strong 't Hooft coupling limit.

Weak 't Hooft coupling limit
Second we consider the weak 't Hooft coupling limit: k N . In this limit, by assuming the balance between the first two terms and the second term in the saddle point equations (2.6), i.e. (kλ i − ζ) ∼ N coth π(λ i − λ j ), we find the following scaling behavior of λ i The explicit solution to the saddle point equations is given in the continuous notation as where s ∈ (−1/2, 1/2), together with the eigenvalue density ρ(s) = ( dx ds ) −1 given with

JHEP03(2017)121
Below we first provide the derivation of this solution. Then we evaluate the free energy and the vacuum expectation value of the Wilson loops on this solution.
To obtain the solution (4.2) and (4.3), first let us shift the real/imaginary part of the eigenvalues as By assuming |u i |, |y i | 1 , (4.5) and expanding the trigonometric and hyperbolic functions up to O(u i , y i ) we can simplify the saddle point equations (2.8) and (2.9) as where we have neglected the deviations of O((N/k) 3/2 ). If we further pose the ansatz y i = −u i and switch to the continuous notation the saddle point equations reduce to the following single integration equation which is solved by The real-positive parameter is determined from the normalization condition in (4.7) as = N πk . We find that the weak coupling limit k N is indeed required for the consistency of the initial assumption (4.5). Changing the variable from u to s = πk N u 2 , we finally obtain the solution (4.2) with (4.3).
The free energy (2.12) evaluated on this solution is (4.10) In the limit ζ → 0 the result coincide with that for the ABJM theory [34]. We can also compute the Wilson loop as which are consistent with the results in [20,34] up to O(N/k) and the overall factor.

Discussion
In this paper we have studied the mass deformed ABJM theory in the large N limit with various values of (k, ζ), using the saddle point approximation for the matrix model. Let us rephrase our results, especially for finite k.
In section 3.1 we have considered the limit ζ k. In this parameter regime, since the mass of the matter multiplets is m = ζ/k we can integrate out these fields separately in the partition function. As a result the saddle point equation gets extremely simplified, which is completely independent of ζ. Though the leading part of the free energy is fixed by the one-loop effects of the matter fields as f ∼ 4πζN 2 /k, the eigenvalue distributions are still constrained by the saddle point equations. We have also computed the vacuum expectation values of the Wilson loops in that saddle configuration, and found that they vanish due to non-trivial cancellation among the contributions from N eigenvalues.
In the regime where both ζ and k are finite, we found two different solutions. One is the natural extension of the above solution with f ∼ 4πζN 2 /k which exists for any ζ and k. The other solution with f ∼ N 3/2 which has the AdS 4 gravity dual exists only for ζ/k < 1/4 and coincides with the solution I in [10].
Thus, the theory will be critical at ζ/k = 1/4 although the absolute value of Wilson loop does not depend on ζ/k. (As a matrix model, the eigenvalue distribution itself is the observable and becomes critical at the value.) If we consider the large N partition function on the solid torus [35] which is obtained by cutting S 3 , we might see how the theory becomes critical ζ/k = 1/4 because the eigenvalues are fixed at the boundary of the solid torus. Of course, the analysis in the gravity dual is needed to understand the critical behavior. 16 We hope to report on these in near future.
It is not clear that what is a correct solution for ζ/k > 1/4. One possibility is that it is the solution with f ∼ N 2 we found, which implies that the free energy jumps between ζ/k < 1/4 and ζ/k > 1/4. For finite N , the partition function (2.2) will be continuous with respect to ζ/k, hence so is the free energy f . However, this does not rule out the discontinuous change of the scaling exponent of the large N free energy N 3/2 → N 2 because the finite N correction can make the free energy smooth. Indeed, our solution which has the free energy of the order N 3/2 becomes singular at ζ = k/4, thus it is not valid very near the point. We expect that the analysis very near ζ = k/4 including finite N effects gives a smooth free energy although we leave this problem for future work.
Other important property of the mass deformed ABJM theory is that it will describe the M2-M5 system. Indeed, in the classical analysis [1], the vacua are found to be given by a configuration which is a generalization of the fuzzy sphere to a fuzzy S 3 which represents the M5-brane [1,5]. Thus, it would be natural to think the phase transition at the critical value is due to the non-negligible effects of the spherical M5-branes and the compactified M5branes would explain f ∼ N 2 for ζ/k > 1/4. We hope to report also on this in near future.
A Evidence for another solution for ζ k Below we argue another possible way to solve the saddle point equations for ζ/k 1 (3.3) and (3.4), though the explicit expression is not found. First we would like to assume u i < u j for i < j, without loss of generality. The key point is the following additional assumption: v i is large and varies more frequently than the real part u i . Under this assumption, we can compute the summation over j in (A.1) by approximate u i − u j to be constant while v i − v j spans a period of the cosine function, as where in the second line we have used (the continuous version of) the formula (3.17), and in the third line we have used the fact u i < u j ⇔ i < j. Hence we can solve (A.1) to obtain Here δv is some function of O(1). If δv(i/N ) is randomly distributed and k N , this v i indeed justifies the approximation for the summation above.
To determine the real part u i we have to solve the other equation (A.2) (with the substitution of v i ) . (A.5)

JHEP03(2017)121
Though this equation is difficult to solve as it contains the random part δv, 17 we have observed in the numerical analysis that the solution actually exist with several differentlooking δv. It is not clear whether the solutions of this type are relevant in the regime ζ/k 1. For the numerical solutions we have obtained, however, we observe the following behavior of the free energy with some positive coefficient which is different for each solution. Hence we conclude that the solution given in the section 3.1 is more preferred in the saddle point approximation compared with these solution.
Note that the solutions found in [12] is similar to this solution in the sense that the eigenvalue distribution is of O(N ).

B Sub-leading part of solutions with f ∼ N 2
In section 3.1 and section 3.2.1 we have found the solutions of type In the continuous notation

JHEP03(2017)121
the above equation is written as with ∆ = ∆ + 1/2. To solve this equation, regard the last term in this equation as a linear transformation P 1 [·] acting on a function .
We find the following series of the eigenfunctions and the eigenvalues of this operation Assuming that g(t) is expanded in these eigenfunctions and recalling the following identity used in [10] ∞ α=1 we obtain the following solution ( ∆, g(t)) to the integration equation (B.4) where g i and h i are the abbreviations of g(i/N ) and h(i/N ) respectively. After introducing the continuous notation replacing the discrete index i and the summations the O(N 0 ) part of the saddle point equations can be written as with ∆ = ∆ + 1/2. To clarify the structure of the equations we introduce the following linear transformations with which the saddle point equations (B.11) are written compactly as

JHEP03(2017)121
Since e iαt (α ∈ Z) are eigenfunctions of these transformations P a [e iαt ] = Λ a,α e iαt , (B.14) with Λ a,α some constants, we shall pose the following ansatz Then, with the help of the identity for an infinite summation of the trigonometric functions (B.7) we find that the saddle point equations are satisfied if the coefficients A α and B α satisfy the following equations C Solution for ζ/k < 1/4 In this appendix, we consider the solutions with the free energy f ∼ N 3/2 in a similar way as in [10] in order to compare the results in this paper and the ones in [10] easier. We use the continuous notation λ i → λ(s) with s ∼ i/N − 1/2 ∈ (−1/2, 1/2) and pose the following ansatz for the eigenvalue distribution . We also assume x(s) is a monotonically increasing function of s without loss of generality due to the freedom of the re-numbering of the eigenvalues λ i → λ σ(i) with σ any permutations. Though the ansatz is a slight generalization of that in our previous work [10], the process to determine the solution will look different. Below we first substitute our ansatz to the free energy f (λ, λ) (2.3). The leading part of the free energy in the large N limit can be regarded a functional of the profile functions (x(s), ∆(s), y e (s), y o (s)). Then we can obtain the new set of the "saddle point equations" from the variational problem of this functional. Though the new procedure will be conceptually identical to the direct substitution of an ansatz to the original saddle point equations ∂f /∂λ i and ∂f /∂ λ i (2.6), we find that the derivation of the final set of the equations is substantially simplified.
As a result, we obtain a new boundary condition to the profile functions which were overlooked in the previous analysis and is essential to single out the solution. We finally find that for ζ/k < 1/4 the solution is unique and coincide with the solution I in [10] and that for ζ/k > 1/4 there are no consistent solutions with our ansatz (C.1).

JHEP03(2017)121
With the substitution of the ansatz (C.1) the free energy is evaluated as = ds ẋ −1 (2∆ 2 e − kxẏ e ∆ e ) + ζy e + kh|x| − ky e ∆ e − kx y o +ẋ where the dot "·" denotes the differential with respect to s. We have also introduced the following abbreviation with h ∈ Z/2 defined by | y o | ≤ 1/4. We would like to note that the following integration identity is useful in the computation to derive the expression (C.2) with (C.3): Here we have chosen x as a fundamental variable rather than s, and introduced the eigenvalue density ρ(x) = ds/dx in x direction. The differentials with respect to x are abbreviated with primes " ". In this notation we gain new degrees of freedom for the choice of the x-support I x , as well as new constraints: ρ(x) > 0 and the normalization condition We also obtain the following constraints which come from the variation at the boundaries Interestingly our analysis (almost) derive the constraint ∆ e | boundary = 0 which was posed just by hand in the previous analysis [10].

JHEP03(2017)121
The differential equations (C.2) can be solved as follows. From the first, third and fourth line of the equations we obtain with B an arbitrary constant. Substituting these into the second line of (C.6), we obtain a differential equation containing only y e , which is solved as y e = B − kh|x| ζ − y e , (C.10) with y e = 1 + k 2 h 2 ζ 2 (x 2 + 2b|x| + a). (C.11) Here a and b are arbitrary real numbers. Now we shall determine the moduli of the solution to the differential equations (C.6), which are a, b, B ∈ R together with the choice of the x-support I x , from the normalization condition (C.7) and the boundary constraints (C.8). First we argue that the solution with any disconnected piece in x ≥ 0 in I x is excluded from the boundary constraints (C.8). We focus on the second boundary condition y o | boundary , which is explicitly written as The behavior of the left-hand side as a function of x is displayed in figure 1. For in the case (iii) and (iv), there exist two solutions x = L 1 , L 2 for (C.12) with 0 ≤ L 1 < L 2 . Among them, the case (iii) (a ≥ 0, b ≤ 0) is excluded as | y o | > 1/4 for L 1 < x < L 2 which contradicts to our initial assumption. Hence the support (L 1 , L 2 ) could exist consistently only when the parameters satisfy ζ < k/4, h = 0, a < 0 and b > 0. On the other hand, in the case of (iv) we can easily find that there are no solutions (a, b, L 1 , L 2 ) (0 ≤ L 1 < L 2 ) which also satisfy the first boundary condition in (C.8) ρ∆ e (L 1 ) = ρ∆ e (L 2 ) = 0. Hence we conclude that the x-support I x cannot have any disconnected segment in the region x > 0; it must always be in the form of I x = (−L, L).
In the case I x = (−L, L), in addition to the boundary constraint at x = L, we also require the smoothness of the profile functions at x = 0. Indeed, any points where the profile functions are discontinuous require additional boundary constraints, with which the whole constraints become unsolvable as we have argued above. Then it is obvious that the case h = 0 is excluded. For the same reason we also find that a and b need to satisfy as a > 0 and b = 0. Under these restrictions the moduli (a, b, B, L 1 ) exist only when (see plot (i) in figure 1) and can be uniquely determined from the boundary constraint at x = L and the normalization condition (C.7) as (C.14) with which the explicit expression for the profile functions are . (C.18) The saddle point solution coincide with the solution I obtained in [10]. The free energy of this solution is as computed in [10]. We can check that the above solution of the saddle point equation corresponds to the solution we have introduced in section 3.2.2. To see this we express x as a function of s by integrating ρ = ds/dx s(x) = Open Access. This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.