Thermodynamic limit of Nekrasov partition function for 5-brane web with O5-plane

In this paper, we study 5d $\mathcal{N}=1$ $Sp(N)$ gauge theory with $N_f ( \leq 2N + 3 )$ flavors based on 5-brane web diagram with $O5$-plane. On the one hand, we discuss Seiberg-Witten curve based on the dual graph of the 5-brane web with $O5$-plane. On the other hand, we compute the Nekrasov partition function based on the topological vertex formalism with $O5$-plane. Rewriting it in terms of profile functions, we obtain the saddle point equation for the profile function after taking thermodynamic limit. By introducing the resolvent, we derive the Seiberg-Witten curve and its boundary conditions as well as its relation to the prepotential in terms of the cycle integrals. They coincide with those directly obtained from the dual graph of the 5-brane web with $O5$-plane. This agreement gives further evidence for mirror symmetry which relates Nekrasov partition function with Seiberg-Witten curve in the case with orientifold plane and shed light on the non-toric Calabi-Yau 3-folds including D-type singularities.


Introduction and Main results
Supersymmetric gauge theory has rich structures and applications in the study of nonperturbative quantum field theories. Especially, Seiberg-Witten solutions of four dimensional (4d) N = 2 gauge theory plays an important role in understanding analytic properties in SUSY gauge theory. The basic idea of Seiberg-Witten's work [1,2] is that the low energy physics of SUSY gauge theory can be described by geometry, i.e. a Riemann surface (which is called Seiberg-Witten curve) and periods which can be obtained by integrating a meromorphic differential one form (i.e. Seiberg-Witten differential) along two conjugate homology cycles. Based on comparing infrared limit and ultraviolet limit of certain gauge theory, Nekrasov predicts a relation between SUSY N = 2 Yang-Mills instanton partition functions and Seiberg-Witten prepotential [3]. This relation is also called Nekrasov conjecture which has been verified by Nekrasov-Okounkov [4] for 4d gauge theories with/without matter content and five dimensional (5d) gauge theory compactified on a circle of circumference β based on random partition technique, and proven by Nakajima-Yoshioka [5] and Braverman-Etingof [6] by using blow-up formula and Whittaker vectors technique respectively. Furthermore, Nekrasov conjecture has been also proven for instantons on toric surface which extends the case of instantons counting on C 2 ∼ = R 4 by equivariant localization [7]. Look at it from another angle, this relation can be also understood as mirror symmetry which relates Nekrasov partition function in A-model side with Seiberg-Witten prepotential in B-model side. It is natural to extend this correspondence to more general cases with different gauge groups and different matter contents. It is interesting to ask the following question: How far shall we go along this line? In this paper, we will explore 5d N = 1 Sp(N ) gauge theories with matter contents in the presence of orientifold 5 branes.
Recall that there is a correspondence between (p, q) 5-brane web diagrams and toric diagram underlying Calabi-Yau 3-folds which give the identical gauge theories [8], so it is natural and convenient to use 5-brane web as a main tool to analyze and understand supersymmetric gauge theories. Moreover, a wide class of such geometric examples are constructed in this way and shown to be related by chain of string dualities [9]. Recent development indicates that this brane/toric geometry correspondence can be also generalized to some non-toric examples of Calabi-Yau 3-folds and the methods to construct non-toric Calabi-Yau 3-folds are not unique. For example, the first method to construct a class of non-toric Calabi-Yau 3-folds like n-th local del-Pezzo surface with n = 7, 8 can be realized in terms of 5-brane web with the inclusion of 7-branes at infinity [10]. Correspondingly, topological string partition functions for these examples can be computed by generalizing topological vertex formalism from 5-brane web diagrams to 5-brane web diagrams with 7-branes at infinity [11,12]. The second way to construct non-toric Calabi-Yau 3-folds is to consider 5-brane web diagram with orientifold 5-plane (O5-plane). Indeed, some of them are expected to correspond to the resolutions of D-type singularity [13], which is not toric. However, unlike the toric cases, the systematic method to construct non-toric Calabi-Yau 3-fold from an arbitrary 5-brane web diagram with O5-plane is not yet known enough.
In this paper, we consider 5d N = 1 Sp(N ) gauge theories with N f flavors and discuss its relation with Seiberg-Witten theory based on 5-brane web diagram with O5-plane.
On the one hand, Seiberg-Witten curve can be obtained from dual graph of 5-brane web with O5-plane. In [14], it is discussed that M-theory uplift of the type IIA analogue of the Hanany-Witten brane setup [15] produces the Seiberg-Witten curve [1,2] of 4d N = 2 gauge theories with SU gauge groups. Correspondingly, the Seiberg-Witten curve of the 4d N = 2 gauge theory with SO or Sp gauge group can be also constructed in [16,17] by use of inclusion of an orientifold 4-plane. This construction can be generalized to 5d N = 1 supersymmetric gauge theory compactified on S 1 [16]. It is noted that the more systematical way is to construct based on 5-brane web diagram in [18]. In a similar fashion, the Seiberg-Witten curve for 5d N = 1 gauge theories with Sp(1) gauge group can be constructed by inclusion of O5-plane to the 5-brane web [19]. On the other hand, since supersymmetric gauge theories can be realized and analysed by considering the string theory on Calabi-Yau 3-fold [20,21], then the Seiberg-Witten solution can be obtained from the Nekrasov's partition function by taking thermodynamic limit [4,5,22]. Nekrasov's partition function is known to agree with Topological string partition function by geometric engineering [3,23,24]. Especially, when a toric Calabi-Yau 3-fold is given, the topological string partition function can be computed by using topological vertex formalism [25][26][27][28]. Recently, new generalized formalism for the topological vertex based on the 5brane web with O5-plane was conjectured [29]. According to this new formalism, the topological string partition function can be systematically computed for a given 5-brane web with O5-plane by using cut-reflect-join technique with the assumption that the given toric-like diagram corresponds to a certain Calabi-Yau 3-fold with involution. Similarly, the Nekrasov partition function for the 5d N = 1 pure Sp(1) gauge theory can be also computed explicitly based on this method. Although the expression obtained in this way looks different from the known expression, it is checked to agree up to 10 instantons, which gives a support for the validity of new topological vertex formalism for 5-brane web with O5-plane.
The main results of this paper consist of two parts: In the first half, we compute 5d Seiberg-Witten curve directly from dual graph of 5-brane web with O5-plane [19]. Especially, we discuss the boundary conditions on the Seiberg-Witten curve (2.32), which is a significant characteristic induced by the O5-plane. In the second half, we obtain 5d Seiberg-Witten curve by taking thermodynamic limit of Nekrasov partition function [4] based on topological vertex formalism for 5-brane web with O5-plane [29] and derive its boundary conditions. The comparison result shows that Nekrasov's conjecture relating Nekrosov partition function with Seiberg-Witten prepotential still holds for 5-brane web with O5-plane. As an effective check, we verify the agreement for the prepotentials for 5d N = 1 pure Sp(1) = SU (2) gauge theory with discrete theta angle 0 based on Seiberg-Witten curves from 5-brane web diagram with and without O5-plane in Appendix B.
The structure of this article is as follows. In section 2, we obtain Seiberg-Witten curve from 5-brane web with O5-plane; In section 3, we review topological vertex formalism for 5brane web with O5-plane and compute partition function for 5d N = 1 Sp(N ) gauge theory with N f (≤ 2N + 3) flavors by cut-reflect-join techniques; In section 4, we rewrite partition function as profile function of random partition and obtain 5d Seiberg-Witten curve by taking thermodynamic limit; In section 5, we conclude the main results of this paper and  Nf { < l a t e x i t s h a 1 _ b a s e 6 4 = " b 5 r y Q L 6 G + O x E i T Y 5 C X G f A U H z N 0 g = " > A A A C Z X i c h V H L S s N A F D 2 N r 1 q t r Q 9 E c G G x K K 7 K t C q K q 4 I b l 3 3 Y B 7 S l J H F a Q 9 M k J G m h F n 9 A 3 K o L V w o i 4 m e 4 8 Q d c 9 A t E X F Z w 4 8 L b N C B a 1 D v M z J k z 9 9 w 5 M y M Z q m L Z j H U 8 w t D w y O i Y d 9 w 3 M e m f C g S n Z 7 K W 3 j B l n p F 1 V T f z k m h x V d F 4 x l Z s l e c N k 4 t 1 S e U 5 q b b b 2 8 8 1 u W k p u r Z v t w x e q o t V T a k o s m g T l S q 2 y 8 E w i z A n Q o M g 6 o I w 3 E j o w V s U c Q A d M h q o g 0 O D T V i F C I t a A V E w G M S V 0 C b O J K Q 4 + x z H 8 J G 2 Q V m c M k R i a z R W a V V w W Y 3 W v Z q W o 5 b p F J W 6 S c o Q V t g T u 2 N d 9 s j u 2 Q v 7 + L V W 2 6 n R 8 9 K i W e p r u V E O n C y k 3 / 9 V 1 W m 2 c f i l + t O z j Q q 2 H a 8 K e T c c p n c L u a 9 v H l 1 0 0 z u p l f Y q u 2 a v 5 P + K d d g D 3 U B r v s k 3 S Z 6 6 h I 8 + I P r z u Q d B N h a J r k c 2 k x v h e M z 9 C i 8 W s Y w 1 e u 8 t x L G H B D J 0 b g W n O M O 5 5 1 n w C 3 P C f D 9 V 8 L i a W X w L Y e k T 1 4 u K W A = = < / l a t e x i t > t g T u 2 N d 9 s j u 2 Q v 7 + L V W 2 6 n R 8 9 K i W e p r u V E O n C y k 3 / 9 V 1 W m 2 c f i l + t O z j Q q 2 H a 8 K e T c c p n c L u a 9 v H l 1 0 0 z u p l f Y q u 2 a v 5 P + K d d g D 3 U B r v s k 3 S Z 6 6 h I 8 + I P r z u Q d B N h a J r k c 2 k x v h e M z 9 C i 8 W s Y w 1 e u 8 t x L G H B D J 0 b g W n O M O 5 5 1 n w C 3 P C f D 9 V 8 L i a W X w L Y e k T 1 4 u K W A = = < / l a t e x i t > Figure 2. Dual graph of the 5-brane web diagram with O5-plane.
give some perspective for future study. In Appendix A, we describe explicit expressions for the Seiberg-Witten curves; In Appendix B, we compare Seiberg-Witten prepotentials for SU (2) = Sp(1) gauge theory with O5-plane and without O5-plane; In Appendix C, we discuss the parametrization of the 5-brane web diagram for 5d Sp(N ) gauge theory with N f flavors. Especially, In Appendix C.1, we discuss the derivation of the parametrization and reproduce the relation between a I and m I ; In Appendix C.2, we place importance on parametrization of the pure Sp(1) gauge theory and discuss the transformation of the parameters induced by S-duality; In Appendix D, we obtain IMS prepotential from the tropical limit; In Appendix E, we give the proof of the key identity expressing R λµ in terms of profile functions; In Appendix F, we show that if λ C µ∅λ X λ = 0 is satisfied for arbitrary µ, then X λ vanishes for all Young diagram λ.

Seiberg-Witten curve
In this section, we construct the Seiberg-Witten curve for 5d N = 1 Sp(N ) gauge theory with N f flavors from the 5-brane web diagram with O5-plane by generalizing the computation in [19]. Although this class of theories are expected to have UV fixed point for N f ≤ 2N + 6, we consider N f ≤ 2N + 3 for simplicity in this paper. The 5-brane web diagram is depicted in Figure 1. This diagram includes N + 1 internal D5-branes as well as N f half infinite D5-branes. Note that the mirror image is also included in this figure. Motivated by the "dot diagram" introduced in [10], we introduced the white dot corresponding to the shrunken face. This diagram is interpreted as the brane construction for the 5d N = 1 Sp(N ) gauge theory with N f flavors up to the phase transition discussed in [19].
The Seiberg-Witten curve should be constructed in such a way that its tropical limit reproduces the original 5-brane web diagram. Practically, it can be read off from its dual graph, which is depicted in Figure 2. In this figure, we omitted the triangulation because it does not affect the Seiberg-Witten curve. Analogous to the method discussed in [18], each dot in the dual graph of the 5-brane web diagram corresponds to the monomial appearing in the Seiberg-Witten curve where (m, n) represents the integer coordinates for the dots and the sum is over all the dots included in the dual graph. In our setup, Seiberg-Witten curve is given of the form Since the expression (2.1) has ambiguity of multiplying non-zero value to both sides of the equation, we can choose C 2,0 = 1 without the loss of generality. The effect of the existence of O5-plane is the following two aspects. First, the Seiberg-Witten curve is invariant under w → w −1 . This indicates that P (w) and Q(w) satisfy the constraints This gives the constraints on the coefficients to satisfy C m,n = C m,−n . Second, the Seiberg-Witten curve has double roots at w = 1 and w = −1. These constraints can be written as 1 They are based on the following interpretation: When we take T-duality and uplift to M-theory, the 5-brane web becomes a single M5-brane, whose configuration is identified as the Seiberg-Witten curve [14], while the O5-plane becomes two OM5-planes at w = 1 and w = −1. The constraints (2.5) are understood as the boundary conditions of the M5-brane at the OM5-planes [17,19]. In the following, we introduce some parameters which appear naturally in the region where |t| and/or |w| are large and/or small. They are known to be simply related to the gauge theory parameters. We first consider the region where |t| is small while |w| is finite. In the limit t → 0, the Seiberg-Witten reduces to where m i is known to be identified as the mass parameter in the gauge theory [14,18,30]. This can be regarded as the condition that Q(w) can be written as with C being a constant. We assume |e −βm i | ≤ 1 (i = 1, 2, · · · , N f ) for later convenience.
We also consider the region where |w| is small. If we solve the Seiberg-Witten curve (2.2) in terms of t, we have two solutions t = t + (w) and t = t − (w) which can be approximated as Without the loss of generality, we assume that Re( P (w) 2 − 4Q(w)) > 0 at the region |w| 1, Re(P (w)) > 0 in order to fix the convention for the branches. Note that |t + (w)| |t − (w)| if |w| is small enough since we are considering the case N f ≤ 2N +3. Here, we denote the coefficient of the leading order term of the ratio of these two solutions as (−1) N f e −2βm 0 . That is, The factor e −2βm 0 is identified as the instanton factor in the Sp(N ) gauge theory and m 0 is identified as the mass of the instanton particle. The sign (−1) N f is introduced to make the convention consistent with the past literatures including [19].
We now see that the conditions above are enough to determine essentially all the coefficients of the Seiberg-Witten curve (2.2) with (2.3) by counting degrees of freedom for its coefficients. Originally, there are totally 2N f + 2N + 6 coefficients in (2.3). The conditions (2.4), (2.5), (2.8) and (2.10) reduce the degrees of freedom by (N + 2) + N f , 2, N f and 1, respectively. Thus, we now have N + 1 independent coefficients. Finally, we see that one degree of freedom of the coefficients can be absorbed into the rescaling of t. The remaining N coefficients correspond to the gauge invariant Coulomb moduli parameters, which appear in the Seiberg-Witten curve as it is. We discuss more concrete expressions for the Seiberg-Witten curve in Appendix A by writing down the coefficients C m,n more explicitly.

Seiberg-Witten 1-form and cycle integrals
In the following, we discuss the Seiberg-Witten 1-form which is defined on the Seiberg-Witten curve. Based on the interpretation that the Seiberg-Witten curve is identified as the M5-brane configuration [14], the Seiberg-Witten 1-form is derived as [31][32][33] Since the Seiberg-Witten curve (2.2) can be understood as a double cover of the complex plane with the coordinate w, we introduce the 1-forms λ + SW (w) and λ − SW (w) defined on the complex plane (except on the branch cuts and on other singularities) so that they give the Seiberg-Witten 1-form λ SW defined on the Seiberg-Witten curve as a whole. They are obtained by substituting the solution t = t + (w) and t = t − (w) to (2.11), respectively: Especially, at small |w| to be consistent with (2.9). We see that the square root in the denominator of the second term in (2.12) creates the branch cuts, where the two sheets are connected. Since we are considering the case N f ≤ 2N +3, we expect 2N +4 branch cuts at first sight. However, note that the conditions (2.4) and (2.5) indicate that the function inside the square root is written in the form where w 2N +2 ∆(w) is a polynomial of degree 4N + 4 satisfying ∆(w) = ∆(w −1 ). Therefore, the number of the branch cuts are reduced by two from the naive expectation and we actually have 2N + 2 branch cuts. We denote these 2N + 2 branch cuts as C I (I = 1, 2, · · · , 2N + 2) and their associated branch points as e −βα + I and e −βα − I , where we use this exponentiated expression to make the comparison easier in section 4. We can choose them in such a way that is satisfied, which is required due to the invariance under w → w −1 (2.4). For later convenience, we assume that |e −βα ± I | < 1 for I = 1, 2, · · · , N + 1 and that all the α ± I are different from each other. With this notation, ∆(w) is explicitly written as  We note that the branch cut structure of P (w) 2 − 4Q(w) discussed above indicates That is, t + (w) and t − (w) exchange with each other under the transformation w → w −1 .
We also comment on the singularities of the Seiberg-Witten 1-form at w = ±1. From (2.14), we can derive that the numerator of the second term in (2.12) can be written as where∆(w −1 ) =∆(w). This means that the singularities at w = ±1 in the second term in (2.12) are removable singularities because the factor (w − 1)(w + 1) cancels with each other between the denominator and the numerator. As a result, the Seiberg-Witten 1-form (2.12) can be rewritten as (2.20) We introduce A I -cycles (I = 1, 2, · · · 2N +2) on the Seiberg-Witten curve as the contour going around the branch cut C I counterclockwise on the complex plane on which λ − SW is defined as depicted in Figure 3. This means that the A I -cycle integral of the Seiberg-Witten 1-form λ SW is given as the contour integral of λ ± SW (w) as 2 We also define B I -cycles on the Seiberg-Witten curve so that the intersections of the cycles are given by 3 Then, B I -cycle integral of the Seiberg-Witten 1-form is Finally, we denote the contour going around the pole at w = e −βm i counterclockwise on the complex plane on which λ + SW is defined as "M i -cycles" (i = 1, 2, · · · , N f ) in this paper. 4 The approximation (2.13) is valid also around w = e −βm i , where Q(w) vanishes due to (2.8). This indicates that λ + SW has simple poles at w = e −βm i while λ − SW does not. The relation (2.17) indicates that λ − SW has simple poles at w = e +βm i while λ + SW does not. We denote the contour going around the pole at w = e +βm i clockwise on the complex plane on which λ − SW is defined as "M N f +i -cycles" (i = 1, 2, · · · , N f ). This means that the M i -cycle integral of the Seiberg-Witten 1-form λ SW is given as 5 (2.24) We also note that λ ± SW do not have a pole at w = e ∓βm i and thus, The cycles defined above can be also understood as depicted in Figure 4 due to the following interpretation. The Seiberg-Witten curve is constructed in such a way to reproduce the 5-brane web diagram in the tropical limit [18]. In other word, the Seiberg-Witten curve is obtained by "thickening" the 5-brane web diagram. Analogous to the case in [14], 2 As an abuse of notation, we use the same symbol AI both for the cycles on the Seiberg-Witten curve and the contour on the complex plane which goes around the branch cut CI counterclockwise. AI for the integral of λ + SW in (2.21) means the latter, which corresponds to the opposite of the AI -cycle on the Seiberg-Witten curve. Due to this convention, the minus sign appears. 3 The choice of B-cycle in this paper is different from the standard non-compact cycle. We use this choice in order to avoid the divergence. 4 Although Mi-cycles are not rigorously homological 1-cycles of the Riemann surface, we call so in this paper because they often appears as degenerating limit of the 1-cycles. 5 Mi in this equation denotes the contour going around w = e −βm i conterclockwise rather than the Mi-cycle on the Seiberg-Witten curve, analogous to the convention for AI . the (−p, 1) 5-brane at the left and the (p, 1) 5-brane at the right correspond to the two w-planes, whose positions are given by (2.9), respectively. The color D5 branes correspond to the cuts connecting them, and thus, the A I -cycle goes around the tube corresponding to the I-th color brane. Analogously, the flavor D5 brane corresponds to the poles on the w-planes, and the M i -cycle goes around the tube corresponding to the i-th flavor brane. The B I -cycle goes to left along the I-th color brane, and goes back to right along the I + 1-th color brane so that it intersects with the A I -cycle and the A I+1 -cycle as in (2.22).
By using the A I -cycles and B I -cycles introduced above, the prepotential is given by the following Seiberg-Witten solution: In the following, we compute the integrals over other cycles by using the parameters introduced above. The integrals over A N +1 cycle can be computed by considering the integral over the contour depicted in Figure 5. Since we have assumed |e −βm i | ≤ 1 (i = 1, 2, · · · , N f ) and |e −βα ± I | ≤ 1 (I = 1, 2, · · · , N + 1), A I -cycles and M i -cycles are inside the unit circle |w| = 1. Thus, we find the following identity where we introduced the following 1-form defined on the complex planẽ (2.28) We have added the last term in (2.28) so that the first integral of the left hand side in (2.27) vanishes. The third integral of the left hand side in (2.27) also vanishes because when we parametrize the path as w = e iθ (−π < θ < π), the integrand is an odd function of θ due to (2.17). Here, note that the logarithmic branch cut of log w, which appears in (2.28) and (2.20), exists on the negative real axis of w in our convention. Thus, log w included in the integrand in the second and the fourth terms at the left hand side of (2.27) are different by −2πi, and thus, Here, we used the expression (2.11) and (2.28) at the first equality while we used (2.10) at the last equality. The right hand side in (2.27) can be obtained directly from (2.24) and (2.26) apart from the A N +1 -cycle integral. Therefore, A N +1 -cycle can be obtained from (2.27) as Note that we could have imposed (2.30) instead of the condition (2.10). From the invariance under w → w −1 (2.17) with the convention (2.15), we have The integrals over B N +1 is less obvious and we do not discuss in this paper since it is not necessary to determine the prepotential. Finally, we comment on the difference between the Seiberg-Witten curves for 4d and 5d Sp(N ) gauge theories. The Seiberg-Witten curve for 4d Sp(N ) gauge theory is obtained Figure 5. Contour of the integral.
from the Seiberg-Witten curve for 4d SU (2N + 2) gauge theory and by tuning the parameters in such a way that a N +1 = 0 and a 2N +3−i = −a i (i = 1, 2, . . . , N + 1) are satisfied as mentioned, for example, in [34]. The Seiberg-Witten curve for 5d Sp(N ) gauge theory is obtained from the Seiberg-Witten curve for 5d SU (2N + 4) gauge theory and by tuning the parameters in such a way that a N +1 given in (2.30), a N +2 = 0 and a 2N +5−i = −a i (i = 1, 2, . . . , N + 2) are satisfied. The appearance of the A N +1 -cycle, over which integral gives non-zero a N +1 given in terms of the other parameters, is a remarkable feature for the 5d case. This phenomena is related to the duality between 5d Sp(N ) gauge theory and 5d SU (N + 1) gauge theory proposed in [35], which duality does not hold for the 4d gauge theories.

Seiberg-Witten solution from 5-brane web with O5-plane
We summarize the necessary information to reproduce the prepotential by collecting the necessary information discussed in the previous subsections. The prepotential F (a, m) for the 5d N = 1 Sp(N ) gauge theory with N f (≤ 2N + 3) flavors is essentially uniquely determined 6 up to an integration constant by the following set of equations: The Seiberg-Witten curve with the cycle integrals of the Seiberg-Witten 1-form (2.33) We will reproduce these equations from the thermodynamic limit of the partition function in later section.
3 Partition function via topological vertex formalism with O5-plane

Topological vertex formalism for 5-brane web with O5-plane
In this subsection, we review the "topological vertex formalism with O5-plane" [29,36]. The (unrefined) topological vertex formalism [25] is a systematic algorithm to obtain the topological string partition function for a given toric-web diagram, which specifies the toric Calabi-Yau 3-fold. According to the conjecture in [8], the toric web diagram to specify the toric Calabi-Yau 3-folds can be identified with the 5-brane web diagram of the same shape. This correspondence indicates that the topological vertex formalism can be reinterpreted as the method to compute partition function for a given 5-brane web diagram. This reinterpretation is useful when we generalize the formalism to the case where the 5-brane web diagram includes O5-plane [29]. Given a 5-brane web with O5-plane, for vertices and edges that are not attached to the O5-plane, the rules are the same as the case of 5-brane web without O5-plane. The topological string partition function can be computed based on (p, q) 5-brane web. There are two basic rules about contributions from edges and vertices: • For each edge, we assign a partition λ = (λ 1 ≥ λ 2 ≥ . . . ≥ 0) which is a set of monotonically decreasing non-negative integers such that λ i = 0 for i 0. The size of λ is defined to be the sum of all nonnegative integers |λ| = λ i . The partition is understood as a Young diagram, which is a collection of boxes with λ i boxes at the i-th column. 7 We introduce its transpose λ T by exchanging the rows and the columns of λ. We also define ||λ|| 2 or ||λ T || 2 as Based on these notations, we define the corresponding "edge factors" as Here, the parameter Q corresponding to an edge is given as where ("Length") denotes the length of the edge written in terms of the masses and the Coulomb branch parameters while p and q are the RR charge and the NSNS charge of the corresponding (p, q) 5-brane. This parameter Q corresponds to the Kähler parameter of the corresponding 2-cycle in the geometry side. The parameter g = e −β is given by the omega deformation parameters in terms of the following relation = 1 = − 2 . Finally, the power n = p 1 q 2 − p 2 q 1 is determined by the charges (p 1 , q 1 ) and (p 2 , q 2 ) of the two 5-branes attached to the considered edge that are chosen diagonally.
• For each trivalent vertex, where the three edges with Young diagrams λ, µ, ν meet, we can introduce the following topological vertex C λµν is a skew-Schur function. This topological vertex is known to satisfy the cyclic symmetry C λµν = C µνλ = C νλµ .
The topological string partition function is given by multiplying all the edge factors, the vertex factors and by summing them over all the Young diagrams If the 5-brane web digram includes edges depicted in Figure 6, then we need to introduce an "O5-plane factor" associated with the edges attached to the O5-plane.
O5 - Figure 6. The two different 5-branes intersecting with each other on the O5-plane. The O5-plane factor is introduced corresponding to such edge.
• For two different 5-branes intersecting with each other on the O5-plane, we can introduce the following additional O5-plane factor by assigning identical Young diagrams to these two 5-branes where the parameters Q 1 , Q 2 are exponentials of the minus of the length of the two 5-branes divided by the factor p 2 i + q 2 i respectively, the index n = p 1 q 2 + p 2 q 1 + 1. This rule is reformulated in terms of "O-vertex" in [36].
The (main part of the) topological string partition function is given by multiplying all the edge factors, the vertex factors and the additional O5-factors and by summing them over all possible Young diagrams Although this formalism is based on the 5-brane web, we expect that this gives the Gromov-Witten invariants for the corresponding Calabi-Yau 3-fold X: where is the string coupling constant, t is the collection of the Kähler parameters, and N g d is the genus g Gromov-Witten invariants of the two cycles d. As discussed, for example, in [27,[37][38][39][40][41], in order to obtain the full topological string partition function, we need to multiply the part which cannot be obtained from the computation based on the topological vertex: where a ijk and b i are related to the topological intersection number in X, and c g is the constant called constant map contribution. Especially, the first term is known to be identified as IMS prepotential [42] of the corresponding 5d N = 1 gauge theory.

Comments on the O5-plane factor
In this subsection, we focus on the O5-plane factor to give justification of the rule (3.8).
We start with a fundamental string in the brane setup with D5 branes and an O5-plane as in Figure 7. This fundamental strings can be understood as a single fundamental string stretched between the first D5-brane (D5 1 ) and the mirror image of the second D5-brane (D5 2 ). Especially, in the context of gauge theory, such fundamental string corresponds to one of the components of the W-boson, which should be treated in an equal footing with the other components of the W-boson corresponding to fundamental strings stretched between D5-branes without attaching to the O5-plane.
As discussed in [43], certain combination of S-dualities and T-dualities leads this brane setup to a 5-brane web with an O5-plane in Figure 8. Here, the fundamental string is mapped to an NS5-brane. Therefore, this NS5-brane can be also understood naturally as a single NS5-brane by reflecting part of the 5-brane web diagram and should be treated in an equal footing with other 5-branes which are not attached to the O5-plane.
In order to generalize this observation to (±1, ±1) 5-branes, we consider the phase transition given in Figure 9. This phase transition was originally found in the process of considering the tropical limit of the Seiberg-Witten curve for rank 1 E 2 theory and was called "generalized flop transition" [19]. It may look non-trivial at first sight if we consider only the left half of Figure 9. However, if we reflect part of the web diagram, this can be understood more naturally since it is just moving the strip diagram until the (1, −1) 5-brane intersect with the O5-plane as given in the right half of Figure 9. Since these two different phases are realized in different parameter regions of an identical gauge theory, the partition function should be invariant under this phase transition, analogous to the flop invariance of the topological string partition function [44,45]. Especially, the contribution from the (1, −1) 5-brane at the upper left diagram of Figure 9 should be identical to the contribution from the (1, −1) 5-brane and the (1, 1) 5-brane intersecting at the O5-plane at the lower left of Figure 9. This claim indicates the equivalence of the lower left diagram and the lower right diagram in Figure 9, which are related by the partial reflection of the 5-brane web diagram, because the shape of the (1, −1) 5-brane is identical in the upper left diagram and in the lower right diagram in Figure 9. This observation motivates us to treat "the two different 5-branes intersecting with each other on the O5-plane" in Figure 6 as a  Figure 9. The phase transition found in [19]. It can be understood naturally by partially reflecting the 5-brane web diagram. The (1, −1) 5-brane given by the thick line in the upper left diagram is mapped to the thick lines in the other diagrams by the reflection or by the phase transition. Figure 10. The web diagram to which the O5-plane factor is assigned and its partial reflection.
single (1, −1) 5-brane or a single (1, 1) 5-brane by reflecting part of the 5-brane web and to treat it in an equal footing with other 5-branes which are not attached to the O5-plane. The O5-plane factor (3.8) is basically determined from this idea. The 5-brane web diagram at the center in Figure 10 includes the configuration to which we should assign the O5-plane factor. The left diagram in Figure 10 is a strip diagram obtained by reflecting the right half of the 5-brane web while the right diagram in Figure 10 is obtained by reflecting the left half. Naively, these three should be basically equivalent. However, we need to be careful about the framing factor when we assign Young diagrams also to the external lines.
In order to see this point, we first compare the partition function for the left strip and the right strip in Figure 10. The partition function for the left strip is given by where n = p 1 q 2 + p 2 q 1 . Here, we apply the identity to the vertex factors in (3.12) to obtain (3.14) Since the partition function for the right strip is we find the relation This indicates that the framing factors are multiplied when the corresponding external lines are reflected. Therefore, we should impose that the partition function for the central web diagram in Figure 10 is related to the left one and the right one as Here, we consider the partition function for the web diagram at the center in Figure 10. Suppose that we do not know the rule (3.8) in advance, we introduce a factor corresponding to the (1, −1) 5-brane and (−1, −1) 5-brane intersecting with each other on the O5-plane as I λ,λ (Q 1 , Q 2 ), which is unknown at this stage. Here, we assign different Young diagrams λ λ to the (1, −1) 5-brane and (−1, −1) 5-brane, respectively, in order to keep the generality. The idea of topological vertex formalism is that the whole partition function is obtained by gluing the contributions from the local geometries. Therefore, we impose that this factor depends only on the local structure and is not affected by the Young diagrams which are away from the O5-plane. That is, I λ,λ (Q 1 , Q 2 ) does not depend on µ, ν, ρ, σ. By using this factor the partition function is given in the form Or, if we use the identity (3.13) to the second vertex factor C ρ T λσ T , where we also used the cyclic symmetry of the topological vertex. Substituting (3.12) and (3.19) into (3.17), we obtain the condition The unknown factor I λ,λ (Q 1 , Q 2 ) is determined by imposing that this is satisfied for arbitrary external Young diagrams µ, ν, ρ, σ.

O5 -
Here, we note that if is satisfied for arbitrary µ and ν, where X λ is an arbitrary factor which depends on λ but does not depend on µ and ν, then is satisfied for all λ. By applying this to (3.20) repeatedly, we conclude that have to be satisfied. Taking into account that the edge factor is given in (3.3), we find that the O5-plane factor is given as where n = n + 1 = p 1 q 2 + p 2 q 1 + 1. This is exactly the rule for the O5-plane factor given in (3.8), where λ = λ was imposed from the beginning. In this subsection, we apply the previous rules to the 5-brane web diagram corresponding to the 5d N = 1 Sp(N ) gauge theory with N f flavors. We assign Young diagrams ρ i , λ I and µ I as depicted in Figure 11. We define for later convenience. We denote the Kähler parameters Q M,i (i = 1, · · · , N f ) , Q F,I (I = 1, · · · , 2N + 2) and Q B,I (I = 1, · · · , N + 1) to the edges where Young diagrams ρ i , λ I and µ I are assigned, respectively. Explicitly in terms of gauge theory parameters, as discussed in Appendix C.1. We note that the relation is also derived in Appendix C.1, which reproduces (2.30). By applying the rules discussed in section 3.1 to the web diagram in Figure 11, we obtain The first line is the vertex factors, the second and the third lines are the edge factors, and the last line is the O5-plane factor. Here, we apply the identity (3.13) to the third factor in the first line by identifying We also rewrite the O5-plane factors by using the edge factor function defined in (3.3) as Then, (3.28) can be rewritten as This can be identified as strip diagram depicted in Figure 12. Indeed, if we apply the rules discussed in section 3.1 to the diagram in Figure 12, we reproduce (3.31). Therefore, we can interpret that using the identity (3.13) with (3.29) corresponds to the reflection the right half of the 5-brane web diagram. The topological string amplitude for any strip diagram can be computed by following [46]. Here, we consider a generic strip diagram depicted in Figure 13. In this diagram, N L D5-branes and N R D5-branes are attached to the central (p, 1) 5-brane from the left and from the right, respectively. We denote the height of the left D5-branes as a i (i = 1, 2, · · · , N L ) while the height of the right D5-branes as b j (j = 1, 2, · · · , N R ). We impose a 1 > a 2 > a 3 > · · · > a N L and b 1 > b 2 > b 3 > · · · > b N R while we do not impose any inequality relation between a i and b j . The Young diagrams µ i are assigned to the left D5-branes while ν j are assigned to the right D5-branes. In order to proceed, we introduce some necessary building blocks for this expression: (3.32) In particular, Q = e −βx for some x is the exponentiated length of some 5-brane corresponding to the Kähler parameter of toric Calabi-Yau geometry. It is easy to observe that R λµ (Q) = R µλ (Q) is satisfied. Then, the amplitude for the strip diagram given in Figure   -21 -13 is given as follows: This can be shown by explicit computation for fixed ordering of the parameters, for example, a 1 > b 1 > a 2 > b 2 > · · · , and by using the flop invariance of the topological vertex [44,45].
Applying this formula to the strip diagram given in Figure 12, we find that (3.31) can be explicitly computed to give It has been discussed in various contexts [11,[47][48][49][50][51][52] that the "extra factor" Z extra should be removed from the naive partition function Z 0 , which is (3.30) in our case, in order to obtain the correct partition function The extra factor is the part which does not depend on the Coulomb moduli a I . Therefore, even though it is not always straightforward to obtain the extra factor exactly, this subtlety does not affect our computation as long as we consider the Coulomb moduli dependent part In the following, we concentrate only on such part and treat the partition function up to the factor independent of the Coulomb moduli. We consider the following partition function obtained from (3.30) with (3.35) as where C is a factor which does not depend on the Coulomb moduli. The factor R ∅∅ (e −β(m i −m j ) ) −1 in the first line in (3.35) is also included in this C.
In order to further simplify this expression, we introduce the following notations: First, we define a I with N + 2 ≤ I ≤ 2N + 2 as Also, we define the Young diagram µ I with N + 2 ≤ I ≤ 2N + 2 as Finally, we introduce In terms of these notations, the partition function (3.38) simplifies as For later convenience, we further symmetrize this expression by using the following identity which plays a key role in discussing the flop invariance [44,45]. We use this identity to "half" of the factor R λµ T (Q), or equivalently, use the following identity We also rewrite the factorZ λ by using the identitỹ After a straightforward computation, we find that the partition function can be rewritten as where C = CR ∅∅ (1) N +1 is the prefactor independent of the Coulomb moduli.

Deriving Seiberg-Witten prepotentials from partition function
The goal of this section is to take thermodynamic limit of 5d N = 1 gauge theory partition function and evaluate the Seiberg-Witten prepotential in the presence of O5-plane. There are four steps to achieve this goal: the first step is to rewrite the gauge theory partition function for 5d N = 1 Sp(N ) gauge theory with N f flavors in terms of profile function f λ (x) of random partition λ; the second step is to derive the saddle point equation which profile function should satisfy in the thermodynamic limit; the third step is to introduce resolvent and to evaluate the integrals of resolvent over non-trivial cycles, which is related to the prepotential by Legendre transformation; finally, we derive the Seiberg-Witten curve and its boundary conditions. We find that the results obtained in this section coincide with with the results in section 2. The technique used in this section is based on [4] and also motivated by related works [34,[53][54][55][56][57][58][59].

Profile function of partition diagram
Suppose a Young diagram λ is given, which we depict in Russian style as Figure 14.
The profile function f λ (x) is a piecewise function with Lipschitz constant 1 corresponding to the upper boundary of the partition diagram λ. The precise expression for profile function is The profile function f λ (x) can be easily generalized to f λ (x| 1 , 2 ) by adding two parameters 1 and 2 ( 2 < 0 < 1 ) where 1 and − 2 are scaling constants for two axes λi i x f λ (x) Figure 14. Young diagram λ and its corresponding profile function f λ (x).
respectively. By setting = 1 = − 2 , the profile function f λ (x| ) := f λ (x| , − ) can be simplified as The second derivative f λ (x| ) of the profile function (i.e. a compactly supported distribution on a real line which behaves like a density function for large size partition) can be obtained as It is also not hard to check that the second derivative f λ (x| ) of profile function satisfies the following identities Here |λ| is the size of partition λ, and ||λ|| 2 (or ||λ T || 2 ) are defined in (3.1). The technique playing an important role in our paper is to rewrite the partition function (3.46) in terms of the profile function f λ (x| ) with the help of the following key identity: (4.7) The function γ (x − y) is given in terms of Barnes double gamma function: . (4.8) Interested readers can refer to Appendix E for details about proof of this key identity. In order to rewrite the partition function in a concise way, we introduce where we use the notation given in (3.39), (3.40) and (3.41). Here, we tune a I to be real value. Then, the full Nekrasov partition function for 5d N = 1 Sp(N ) gauge theory with N f (N f ≤ 2N + 3) flavors can be rewritten in terms of profile functions as whereC := log C .

Thermodynamic limit and saddle point equation
In this section, we take the thermodynamic limit of partition function and derive its Seiberg-Witten geometry (curve, differential and prepotential). In the thermodynamic limit = 1 = − 2 → 0, the Seiberg-Witten prepotential F can be extracted as the leading order contribution of the logarithm of the Nekrasov partition function. That is, or equivalently (4.12) In order to obtain the prepotential, we first expand the exponents of the expression in (4.10) in terms of in the following form: Here, E[f µ (x)] is a functional of the second derivative of profile function f µ (x) (4.14) and the constant C 0 , which does not depend on f µ (x), is where F IMS is the IMS prepotential for 5d N = 1 Sp(N ) gauge theory with N f flavors at the region m 1 > m 2 > · · · > m N f > a 1 > a 2 > · · · > a N > 0. We have used the expansion as well as the identity Here, if we consider the full partition function, the IMS prepotential included in C 0 in (4.15) is cancelled by the cubic term in (3.11). Although the interpretation of the remaining terms in C 0 is not very clear, we omit C 0 in the following discussion. Since C 0 does not depend on the profile function, this omission does not affect the following discussion in any case.
When is small enough, the profile function is approximated by a continuous functioñ where we have removed the index µ of the profile function. Since the summand in (4.13) depends on Young diagrams µ I only through the profile functionsf µ (x| ), the partition function, which is a statistical sum over partitions, can be approximated by path integral over the space of continuous functions f . That is (4.20) In order to take into account the constraints on the profile function induced from (4.3) and (4.4), we introduce the following auxiliary functional Here, S is a functional of f and a function of Lagrangian multipliers ξ I and ζ I (I = 1, 2, · · · , N + 1). Now, it would be reasonable to discuss the issues on the measures on profile function. When is small enough, the dominant contributions in (4.13) come from the Young diagrams with sizes of partitions |µ I | are large in such a way that 2 |µ I | is of order 1. This indicates that the dominant contribution in the path integral (4.20) comes from the functions with finite and non-trivial configuration in the finite region around x = a I . Especially, in the thermodynamic limit → 0, the measures on the profile function f µ (x) weakly converges to the Delta measure on f . In other word, the profile function f µ (x) has a limit shape f * (x), where f * (x) gives the critical point of the exponent in (4.20).
For each fixed constants {a I }, we denote the solution of the following (functional) equations to be f * (x), ξ I * and ζ I * : Moreover, we denotef * (x) := f * (x) − f * (−x). With this solution, the leading order contribution of the partition function is simply given as from which we read off the Seiberg-Witten prepotential defined in (4.12) as We denote the union of all distinct local compact supports off * (x) as C = 2N +2 I=1 C I , where each C I (I = 1, 2, · · · , 2N + 2) is the closure of {x|f * (x) = 0}. For convenience, we can use the endpoints α − I and α + I of C I to represent the region C I as C I : α − I ≤ x ≤ α + I . On the other hand, it is easy to observe that f * (x) = 0 for α + I < x < α − I+1 .

Resolvent and its integrals over cycles
In this section, we will introduce a complex holomorphic function R + (z) 8 over C\C which is called "Resolvent". The resolvent R + (z) is defined as follows: The Resolvent R + (z) has the following properties: • R + (z) is an even function satisfying R + (−z) = R + (z) sincef * (x) is an odd function; • R + (z) is periodic in pure imaginary direction: R + z + 2πi β = R(z); • R + (z) satisfies boundary conditions: lim • R + (z) is regular at C\C while discontinuous at C.
In order to understand the last property more in detail, we consider the points just above or below C, that is, z ± i with z ∈ C and > 0. We find that the resolvent R + (z) satisfies the following relations: Here, we have used (4.33), (4.34) and the formula lim where P.V. denotes the principal value integral, which we omit whenever it is clear from the context. Based on this relation, it would be natural to decompose the resolvent R + (z) as the sum of regular part and singular part: (4.37) Here, the regular part R reg (z) is the first two terms in (4.35), which is continuous at z ∈ C. That is (4.38) The singular part changes its sign at z ∈ C as Figure 15. A I , B I , and M i -cycles depicted on the two copies of complex plane with coordinate z. R − (z) is defined on the upper complex plane while R + (z) is defined on the lower complex plane. The two complex planes are connected at the branch cuts and they give the Riemann surface as a whole. Note that the structure is analogous to Figure 3, apart from the locations of M i+N f -cycles (i = 1, · · · , N f ).
This indicates the square root branch cuts at C I . We consider analytic continuationR(z) of the function R + (z) from the complex plane to Riemann surface which is constructed by gluing the cuts in the upper sheet and lower sheet respectively as in Figure 15. The functionR(z) can be regarded as a multi-valued function, which satisfiesR on the upper sheet, whilê on the lower sheet. As depicted in Figure 15, A I -cycles, B I -cycles, and M i -cycles are defined in the zplane as follows: A I -cycles go around C I on the upper sheet. B I -cycles are closed loops starting from α I+1 ∈ (α − I+1 , α + I+1 ) to α I ∈ (α − I , α + I ) clockwise and they are determined by intersection condition A I · B J = δ IJ . M i -cycles (i = 1, · · · , N f ) go around the point z = m i on the lower sheet and M N f +i -cycles (i = 1, · · · , N f ) go around the point z = −m i . In section 2, the corresponding cycles are described in the w-plane by the same name. In the following, we consider the integrals of z nR (z) over A I -cycles, B I -cycles, and M i -cycles, where the case n = 0 and n = 1 are especially our interest.

A-cycle integrals
SinceR(z) = R + (z) on the upper sheet, we can compute the A I -cycle integral by using the definition of the resolvent (4.34). By exchanging the order of the integrals, we find R + (z) Figure 16. Computation of B cycle integrals the A I -cycle integrals are given by due to the constraints (4.27) and (4.28).

B-cycle integrals
To compute the integrals of the ResolventR(z) along B I (I = 1, 2, . . . , N + 1) cycles, it is useful to consider contour integrals depicted in Figure 16. Here the B I cycle is a closed path which starts and ends at α I+1 + i and consists of 4 intervals: α I+1 + i → α I + i , Here (z) + denotes the point on the upper sheet while (z) − denotes the point on the lower sheet. Assume thatR(z) does not diverge at z = α I , then the second and the fourth terms will vanish when we take the limit → 0. Taking into account the expression ofR(z) at the upper sheet and the lower sheet given in (4.40) and (4.41) respectively, we find the B I -cycle integral reduces to z n R reg (z)dz. (4.46) Here, we use again the formula (4.36) as well as the definitions of R + (z) in (4.34) and R reg (z) in (4.38) to obtain 9 where we changed the order of integral and used (4.32) at the last equality. We first consider the case n = 0. Since the integral is computed as =0, (4.49) where the last equality holds due to the derivative of the saddle point equation (4.30). Next, we consider the case n = 1. By integrating by parts, we find Note that we cannot simply use (4.35) here because z ∈ CI is not satisfied in all the integral region.

M-cycle integrals
Since M i -cycles are on the lower sheet, we find where we note that R + (z) is regular outside of the branch cuts. With the explicit expression for R reg (z) in (4.38), this can be further computed as Since n = −1, the M i -cycle integral of z n vanishes: M i z n = 0. Assuming that M i cycle is small enough to include only the pole at z = m i , then only the term for i = i s = −1 remains: We also carry out the analogous computation for M N f +i -cycles. If we define m N f +i := −m i , The expression is valid for i = 1, 2, · · · 2N f : (4.59)

Results of A I , B I , M i -cycle integrals
In conclusion, the A I , B I , M i -cycle integrals are summarized as follows: and

Deriving the Seiberg-Witten curve
In this section, we will derive the Seiberg-Witten curve in terms of the resolvent. When we consider the integral of the resolvent z 0R (z )dz , (4.62) we find that there are two kinds of ambiguity, which are related to the logarithmic branch points and the square root branch points, respectively. One ambiguity is the dependence on the integral path as can be seen from (4.60), which is to add 4πiβ −1 n (n ∈ Z). This can be resolved by considering the exponentiated value since e 2πin = 1 (n ∈ Z). In other word, (4.63) does not have logarithmic branch points although (4.62) does. The other ambiguity is due to the fact that the resolventR(z) = R ± (z) is multivalued on the complex plane, which is related to the square root branch points as discussed before. This can be also resolved by considering the combination which is invariant under the exchange of R + (z) and R − (z). This indicates that M (z) does not have any branch points and is single valued on the complex plane. Since R − (z) can be rewritten in terms of R + (z) and R reg (z) as in (4.41), where R reg (z) is defined in (4.38), we note that the second term in (4.64) is rewritten as by explicitly computing the integral of R reg (z).
Here, we show the periodicity z → z + 2πi β of M (z). Since the periodicity of the last two factors in (4.65) is obvious, it is enough to consider the integral of R + (z). Note that where we used the periodicity of the resolvent R + (z) in the second equality. The third equality holds since the resolvent is an even function. In order to evaluate the first term in (4.66), we need to consider the following contour integral depicted in Figure 17. By Cauchy integral theorem, we have z=L-πiβ -1 Figure 17. Contour integral related to boundary condition where we choose L to be large enough. If we take the limit L → ∞, the third term at the left hand side vanishes. Also, the second term and the fourth term cancel with each other due to the periodicity of R + (z). The right hand side are given by the A-cycle integrals in (4.60). Thus, we can obtain This periodicity implies that M (z) is rewritten more naturally as a function of e −βz . Now, we discuss the singularities of M (z). Due to the periodicity, it would be enough to consider the region −πiβ −1 ≤ Im(z) ≤ πiβ −1 . As mentioned in section 4.3, R + (z) is regular outside of the branch cuts. Moreover, its B I -cycle integral in (4.60) does not depend on the choice of α I and α I+1 in (4.45), which indicates that the integral of R + (z) does not have singularities also on the branch cuts. Therefore, the first term in (4.64) has no singularities except at z → ±∞. The singularities of M (z) at finite region are originated from the denominator of the last factor in (4.65). That is, M (z) has simple poles at z = m i (i = 1, 2, · · · , N f ).
In order to remove the simple poles at z = m i , we define where we have defined it as a function of e −βz since it is invariant under z → z + 2πi β due to (4.69). Denoting w := e −βz , (4.71) P (w) is a single valued function on the w-plane, which has isolated singularities only at w = 0 and w = ∞. Moreover, note that P (w −1 ) = P (w), (4.72) which can be shown from (4.70) by taking into account that R + (z) is an even function.
Since R + (z) → 0 as z → +∞ as discussed in section 4.3, we find Since we are considering N f ≤ 2N + 3, we find from (4.70) and (4.73) that which indicates that w = 0 is a pole of (at most) order N + 2. Combined with (4.72), we find that P (w) must be a Laurent polynomial of the form If we define P (w) can be rewritten in terms of t as Thus, t + defined above can be understood as one of the solutions of the following equation: where we defined This is identified as the Seiberg-Witten curve in (2.32) in section 2. Also, from the definition of w in (4.71) and t + in (4.76), we find This is identified as the Seiberg-Witten one form λ + SW . Indeed, if we identify the cycle integrals in (4.61) reproduce the cycle integrals in (2.33).

Deriving the Boundary conditions
In this section, we will derive the boundary conditions satisfied by the Laurent polynomials P (w) and Q(w).
Boundary condition at w = 1: Starting from (4.70), we set z = 0, we find From (4.79), we set w = 1 and find Thus, we derive the boundary condition at w = 1: which means that the curve (4.78) has a double root at w = 1.
Boundary condition at w = −1: Starting from (4.70), we set z = πi β , then we find (4.85) Since the resolvent R + (z) is an even function, we find from (4.68) that and thus, From (4.79), we find Thus, we derive the boundary condition at w = −1: which means that the curve (4.78) has a double root also at w = −1.
It is remarkable that the boundary conditions (4.84) and (4.89), which were originally claimed from the intuition that the M5-brane is attaching to the OM5-plane at one point [17,19], are now derived independently from the partition function based on the topological vertex formalism with O5-plane.
In this section, we have derived all the information in (2.32) and (2.33) from the thermodynamic limit of the partition function. As is indeed explicitly found in Appendix A, they are enough to determine the Seiberg-Witten curve. Thus, the prepotential defined as the leading order of the partition function as in (4.12) agrees with the prepotential computed from the Seiberg-Witten curve in section 2 up to a constant term which does not depend on the Coulomb moduli a I . This implies that the topological vertex formalism with O5-plane discussed in [29] is consistent with the technique to compute Seiberg-Witten curve from the 5-brane web with O5-plane in [19] and thus, they justify each other.

Conclusion and Discussion
In this paper, we first consider Seiberg-Witten curve obtained by toric-like dot diagram which is the dual graph of a class of (p, q) 5-brane web diagram with O5-plane depicted in Figure 1. Its special case for (N, N f ) = (1, 0) appears in the classification list of phase diagrams aiming at defining discrete theta angles and two different phase diagrams are connected by generalized flop transitions. We discuss the boundary conditions satisfied by Seiberg-Witten curve (2.32) and the prepotential in terms of the period integrals (2.33). Secondly, we study 5d N = 1 Nekrasov partition functions for Sp(N ) gauge theories with N f (≤ 2N + 3) flavors based on new method of "topological vertex formalism with O5-plane" proposed in [29]. Inspired by work of Nekrasov-Okounkov [4], we rewrite the Nekrasov partition function in terms of profile functions for random partition diagrams. After taking thermodynamic limit, we can obtain the saddle point equation for profile functions. By introducing Resolvent, we can reproduce the Seiberg-Witten curve (4.78) and derive the boundary conditions (4.84), (4.89) and the prepotential in terms of the cycle integrals (4.61). It means that the thermodynamic limit of Nekrasov partition function for 5-brane web with O5-plane agrees with the prepotential in terms of the Seiberg-Witten curve obtained from (p, q) 5-brane with O5-plane. This gives further evidence for mirror symmetry conjecture which relates Nekrasov partition function with Seiberg-Witten curve in the case with the orientifold plane. Especially, based on two different Seiberg-Witten curves from 5-brane web diagram with and without O5-plane, we verify the agreement for the prepotentials for 5d N = 1 pure Sp(1) = SU (2) gauge theory with discrete theta angle 0.
In this paper, we have restricted our study to the case with N f (≤ 2N + 3) flavors for simplicity. It would be straightforward to generalize to the case with 2N + 4 ≤ N f ≤ 2N + 6. It would be also possible to generalize to the OSp linear quiver gauge theory. For future perspective, it is interesting to generalize our work to non-Lagrangian field theories constructed from 5-brane web with O5-plane. On the other hand, it is also fascinating to consider similar stories in other limits like Nekrasov-Natashivili limit [60], other types of orientifold planes appearing in different dimensions. It would be also interesting to study from the viewpoint of holomorphic anomaly equation [37,61] or blow-up formula [5], whose generalizations and applications have been studied in [22,[62][63][64][65][66][67][68][69][70][71][72][73]. All in all, we believe that the hidden mathematical structures corresponding to orientifold 5-brane are also charming!

Acknowledgments
This project started around January 2019 which is the first one in the authors' serial explorations about the theory and applications of orientifold 5-brane in mathematics and physics. We would like to thank Sung-Soo Kim for the early stage of the collaboration and comments for the draft. We thank Hirotaka Hayashi, Kimyeong Lee, Yongchao Lu, Yuji Sugimoto, Xing-Yue Wei and Xinyu Zhang for useful discussion and comments. We would like to thank all the teachers and friends we met before. Especially, the first author would like to thank Bohui Chen, An-min Li, Guosong Zhao for their constant support and also thank the 3rd Pan-Pacific International Conference on Topology and Applications (3rd PPICTA), Sichuan Normal University and Southwest Geometry Conference for the invitations to present preliminary results of this paper. Some ideas and discussions have been benefited from the authors' visits to Sichuan University, Peking University, Beijing

A Explicit expression for the Seiberg-Witten curve
In this section, we write down the Seiberg-Witten curve more explicitly by rewriting some of the coefficients in terms of the parameters introduced in section 2. As summarized in section 2.3, the Seiberg-Witten curve is given in the following form: As mentioned around the end of section 2.1, we have a degree of freedom to rescale t because the Seiberg-Witten 1-form λ SW = log w d(log t) is invariant under the rescaling of t. Suppose we redefine t and C 1,n by the rescaling t → C 1 2 t and C 1,n → C 1 2 C 1,n , respectively. After dividing both hand sides by C in (A.1), we have the same Seiberg-Witten curve but now with Under this convention, the condition (2.10) indicates Here, the ambiguity of the overall ± sign has been fixed again by using the redefinition t → −t, which can be done without changing (A.3). From the constraints (2.5), we can rewrite C 1,1 and C 1,0 included in the expression above in terms of other coefficients as follows. If N is even, while if N is odd, Here, we have defined Especially, they are defined as χ s = 1, χ c = 0 for N f = 0. The expressions in (A.5) and (A.6) have two types of ambiguity in the first term: the choice of the sign ± and the choice of χ s , χ c . The first ambiguity is absorbed by the redefinition of m 0 → m 0 + πi/β together with C 1,n → −C 1,n (2 ≤ n ≤ N + 1) and t → −t. Since this ambiguity does not give any significant difference, we choose + for C 1,1 , which means − for C 1,0 , just to fix the convention. The second ambiguity is also absorbed by the redefinition m 1 → −m 1 if N f ≥ 1. However, if N f = 0, the second ambiguity gives the essential difference, which corresponds to the different discrete theta angle of the gauge theory [19]. The difference of the discrete theta angle can be distinguished from the 5brane web diagram. Figure 1 corresponds to the discrete theta angle 0 for N odd while the discrete theta angle π for N even. This diagram is reproduced by the tropical limit of the Seiberg-Witten curve if we choose χ s for C 1,1 , which means χ c for C 1,0 . Although this choice is not essential for N f ≥ 1, it would be natural to use this choice also for N f ≥ 1 in this paper because Figure 1 can be reproduced in the tropical limit with this choice if m i > 0 (i = 1, 2, · · · , N f ) and −m 0 is chosen to be large enough. For the theory with N f = 0 and with the other discrete theta angle, which is π for N odd and 0 for N even, we should choose the opposite choice, which is χ c for C 1,1 and χ s for C 1,0 .
Note that C 1,n (2 ≤ n ≤ N + 1) are left undetermined. These N parameters are interpreted as Coulomb branch parameters. In order to make the expression simpler, we introduce the notation In summary, the Seiberg-Witten curve for 5d N = 1 Sp(N ) gauge theory with N f (≤ 2N + 3) flavors is given in the form where p(w) is given as follows depending on N , N f and the discrete theta angle: • For N even and 1 ≤ N f ≤ 2N + 3, (A.10) • For N even, N f = 0, and discrete theta angle 0 • For N even, N f = 0, and discrete theta angle π (A.13) • For N odd, N f = 0, and discrete theta angle 0 (A.14) • For N odd, N f = 0, and discrete theta angle π The expression for Seiberg-Witten curve obtained above with N = 1 agrees with the Seiberg-Witten curve computed in [19] up to the convention change m 1 → −m 1 for N f ≥ 1.

B Comparison of the Seiberg-Witten curves with and without O5-plane
In this Appendix, we check the agreement about the prepotentials for 5d N = 1 pure Sp(1) = SU (2) gauge theory with discrete theta angle 0, obtained from the two different Seiberg-Witten curves based on the 5-brane web diagrams with and without O5-plane.  The Seiberg-Witten curve for the 5d N = 1 pure SU (2) gauge theory with discrete theta angle 0 has been studied in various literatures including [18,30,74,75]. In order for this Appendix to be self-contained, we review this Seiberg-Witten curve based on the 5brane web diagrams without O5-plane in Figure 18 and the prepotential obtained from it. Analogous to (2.1), the Seiberg-Witten curve is given by the polynomial where (m, n) represents the integer coordinates for the dots and the sum is over all the dots included in the dual graph in Figure 19. The central dot corresponds to C 1,0 . By using the rescaling of t as well as the multiplication of a constant to (B.1), we can fix C 2,0 = C 0,0 = 1. The Seiberg-Witten curve is given in the form Since we do not have O5-plane, the invariance under w → w −1 is not imposed a priori. However, we can choose the convention to satisfyP (w) =P (w −1 ) by using the rescaling of w in this case. Denoting the two solutions of (B.2) for t as the instanton factor q = e −βm 0 is introduced as their ratio at small w analogous to (2.10). Thus, we obtaiñ where U is the Coulomb branch parameter.
The Seiberg-Witten 1-form λ SW is given by Substituting t =t ± (w) into the first line in (B.6), and denoting them as λ ± SW , we find Here, we have defined e −βα ± I (I = 1, 2) to be the solutions of theP 2 (w; U ) − 4 = 0, which are explicitly written as The convention has been fixed by imposing e −βα − 1 < e −βα + 1 e −βα − 2 < e −βα + 2 in the parameter region qU 2 1, U 1 with q and U being real positive, at which region the length of the cuts are short and the two cuts are far from each other.
We define A-cycle to be the contour going around the branch cut between e −βα − 1 and e −βα + 1 counterclockwise in the complex plane, on which λ − SW (w) is defined. Then, B-cycle is defined in such a way that A · B = 1 is satisfied. With this convention, the prepotential is given as We first evaluate them for special value of U for each. The expression for a(U ) simplifies when |U | 1 since A-cycle shrinks in this limit as Changing the integration variable as w = U −1w , we find that A-cycle is now the contour going around the simple pole atw = 1 counterclockwise and its integral is given by the residue as (B.11) The expression for ∂F ∂a (U ) simplifies when U = 2 + 2q 1 2 since B-cycle shrinks in this limit as The integrand in (B.9) is finite around w = −1 while the integral region is vanishing. Therefore, we find that the B-cycle integral vanishes in this case: As for generic value of U , it is not straightforward to compute (B.9) directly. So, we consider their partial derivatives in terms of U where Ω is given as due to the second line in (B.6). Since the total derivative term does not contribute to either of the cycle integrals, we abbreviate it in the following.
Substituting t =t ± (w) in (B.3) into (B.15) and denoting them as Ω ± , respectively, we find Then, by using α ± I given in (B.8), we can rewrite (B. 16) as From this expression, we can identify Ω as the holomorphic 1-form on the Seiberg-Witten curve. Then, (B.14) is given as . (B.18) They are given in terms of the complete elliptic integral of the first kind by using the formula We find (B.14) to be explicitly given as From the computations above combined with the results in (B.11) and (B.13), the Seiberg-Witten solution is given in the form B.2 The Seiberg-Witten curve for pure Sp(1) gauge theory with O5-plane We consider the 5d pure Sp(1) gauge theory with the discrete theta angle 0 obtained from the 5-brane web diagram with O5-plane. The corresponding 5-brane web diagram and its dual graph is given in Figure 20 and Figure 21, respectively. The Seiberg-Witten curve for this theory is obtained by substituting N = 1 and N f = 0 to (A.9) with (A.14). The Seiberg-Witten curve is written as  in this case, since Q(w) = 1. Here,∆(w) and ∆(w) are introduced as in (2.18) and (2.14), which are 27) in this case. They are explicitly given bỹ As discussed in (2.16), we write where e −βα ± I (I = 1, 2) are explicitly given by Defining A I -cycle and B I -cycle as depicted in Figure 3, the prepotential is given as the following Seiberg-Witten solution where A 1 goes around the branch cut between α − 1 and α + 1 . Parallel to Appendix B.1, we first evaluate them for special value of U for each. The expression for a(U ) simplifies when |U | 1 since A 1 -cycle shrinks in this limit as Changing the integration variable as w = U −1w , we find that A 1 -cycle is now the contour going around the simple pole atw = 1 counterclockwise and its integral is given by the residue as The expression for ∂F ∂a (U ) simplifies when U = 2 + 2q 1 2 since B 1 -cycle shrinks in this limit as The integrand in (B.32) is finite around w = e −βα + 1 = e −βα − 2 while the integral region is vanishing. Therefore, we find that the B-cycle integral vanishes in this case: In the following, we consider the partial derivative of in terms of U parallel to Appendix B.1. By repeating the analogous discussion from (B.14) to (B. 16 with together with (B.27), we find that the holomorphic 1-form is given as In order to further simplify the computation, we introduce the coordinate transformation With this convention, the holomorphic 1-form (B.40) simplifies as Also, the A 1 -cycle, which goes around the branch cut between e −βα − 1 and e −βα + 1 maps to the contour which goes around the branch cut between x − 1 and x + 1 . Thus, we find This agrees with the result (B.23). Therefore, we find that the Seiberg-Witten curves from the 5-brane web diagram with and without O5-plane are equivalent to each other.

C Parametrization
In this Appendix, we discuss the parametrization of the 5-brane web diagram for 5d Sp(N ) gauge theory with N f flavors. In Appendix C.1 we discuss the derivation of the parametrization (3.26) and reproduce (2.30) directly from 5-brane web diagram. In Appendix C.2, we focus on the parametrization of pure Sp(1) gauge theory and discuss the transformation of the parameters induced by S-duality. aN-k+1 (k+1)(aN-k+1 -aN-k+2) Figure 23.
A face that appears in the 5-brane web diagram for 5d Sp(N ) gauge theory with N f flavors.

C.1 Derivation on the Kähler parameters
As mentioned in (3.4), the Kähler parameters of the Calabi-Yau 3-folds can be obtained from the corresponding 5-brane web diagram. It is expected to be the case also for 5-brane web diagrams with O5-plane. As in Figure 22, we denote the height of the internal D5-branes to be a I (I = 1, 2, · · · ) while the height of the external D5-branes to be m i (i = 1, 2, · · · ). From this convention, it is straightforward to find the distances between two adjacent D5-branes, from which we find Q F,I = Q F,2N +2−I = e −β(a I −a I+1 ) (I = 1, 2, · · · , N ), as given in (3.26). Suppose we denote the lengths of the k-th internal D5-brane from the bottom as L k (k = 1, 2, · · · , N + 1), it is related to the remaining Kähler parameters Q B,I as Q B,I = e −βL N +2−I (I = 1, 2, · · · , N + 1).
In order to find L k , we concentrate on one face as depicted in Figure 23 This leads to the following relation This is valid also for k = 0 if we define L 0 = a N +2 = 0. Thus, we use this relation recursively to obtain Together with (C.2), we reproduce a J (I = 1, 2, · · · , N + 1) (C.5) given in (3.26).
In the following, we relate m 0 with other parameters. As in Figure 24, we identify the distance between the two points which are obtained by extrapolating the external (N f − 2 − N, 1) 5-brane and (N + 2, 1) 5-brane to the O5-plane as 2m 0 . In this convention, if we extrapolate them to the height of a 1 instead of to the O5-plane, the distance between the two points 2m 0 + (2N + 4 − N f )a 1 . As can be read off from the right of Figure 24, L N +1 is obtained by subtracting X = N f i=1 (m i − a 1 ) from this distance. Therefore, L N +1 can be rewritten in terms of this m 0 as Compare it with (C.4), we find the relation as mentioned at ( also rewritten as (C.8)

C.2 Transformation induced by S-duality for pure Sp(1) gauge theory
In this paper, we find the partition function for 5d Sp(N ) gauge theory with N f flavors. For the special case where N = 1 and N f = 0, the partition function for the pure Sp(1) gauge theory is already computed with the same method in [29]. However, the parametrization in this paper looks different from the one used in this literature. On the one hand, we use the parametrization (C.8), which reduces to where Q F , Q B,1 and Q B,2 are depicted in Figure 25. On the other hand, in [29], they are parametrized as Here, we discuss that they are actually related by the S-duality. In order to understand this point, we discuss the 5-brane web diagram without O5-plane for the pure SU (2) gauge theory. The parametrization for this diagram is given in Figure 26. As discussed in [18,76], the S-duality, which exchange the D5-brane and the NS5-brane, leads to the parameter exchange or equivalently, In the corresponding local P 1 × P 1 geometry, this corresponds to the exchange of the base P 1 and the fiber P 1 discussed in [21]. Taking into account the topological vertex formalism [25], it is straightforward to see that the topological string partition function is invariant under this transformation [76,77]. The partition functions obtained from the 5-brane web diagrams with and without O5plane should agree with each other. Thus, the partition function for 5d Sp(N ) gauge theory with N f flavors with N = 1 and N f = 0 should be also invariant under the transformation (C.12). This transformation relates the convention (C.9) in this paper with the convention (C.10) in [29] and thus, we find the consistency.

D IMS prepotential from the tropical limit
The Seiberg-Witten curve is constructed in such a way to reproduce the 5-brane web diagram in the tropical limit. In this Appendix, we comment on the Seiberg-Witten solution from this point of view. On the one hand, we can compute the prepotential in the tropical limit from the 5-brane web diagram. In the tropical limit, the A I -cycle integrals reduce to the distance between the O5-plane and the corresponding color D5-brane. Analogously, the M i -cycle integrals reduce to the distance between the O5-plane and the corresponding flavor D5-brane. The B I -cycles reduce to the boundary of the faces in the 5-brane web and the B I -cycle integrals reduce to the area of the corresponding face, which is given by where L k is given in (C.4).
On the other hand, since the tropical limit is interpreted as the decompactification limit of the 5d gauge theory on R 4 × S 1 , the prepotential in this limit is known to be given by IMS prepotential [42] F =m 0 if we assume the parameter region m 1 > m 2 > · · · > m N f > a 1 > a 2 > · · · > a N +1 > 0. From this prepotential, we can compute its partial derivative in terms of a I . We can check explicitly that these two computations agree taking into account the relation (2.26) and (2.30).

E Proof of the key identity
In this appendix, we give the proof of the key identity (4.7) used in this paper, which is given by The profile function f λ (x| ) is denoted as f λ (x) for short in this Appendix. The idea of the proof is to show that both sides of the key identity admit the same expressions. On the one hand, the l.h.s. of the identity (E.1) is where we put Q = e −β(a−b) . Since λ i = 0 for i > λ T 1 and µ j = 0 for j > µ T 1 , R λµ (Q) can be reduced to the following form Furthermore, the item in the second bracket in (E.2) can be written as At the last equality, we have introduced a new label j 1 , j 2 , which is related to the original label j as j 1 = λ i − j + 1 and j 2 = j − λ i . Similarly, by exchanging µ ↔ λ and i ↔ j in the computation above, we find that the item in the third bracket in (E.2) can be written as Combining (E.3) and (E.4), R λµ (Q) has the following expression: Here, note that, when Q = e −β(a−b) (a = b) and g = e −βh , we have We first concentrate on the x-integral part of this expression. By using the expression for the second derivative of profile function which is given in the first line in (4.2), we find which reduces the item in the bracket in (E.6) as Next, we go on to the y-integral part of the r.h.s. of the key identity. Here, we use the expression for f µ T (y − b) given in the second line in (4.2). That is, − δ(y − b + (j − µ j )) − δ(y − b + (j − 1)) + δ(y − b + j) . (E.10) The first term in (E.9), which we denote (I), is computed as By using the same recursive relation (E.8) with the identification x := a−b+ (j −µ j ), k := µ j , we find log(1 − e −β(a−b+ ( +j−µ j )) ) = µ j n=1 log(1 − e −β(a−b+ (j−n)) ) (set = µ j − m).

F Derivation of (3.22) from (3.21)
In this appendix, we derive (3.22) from (3.21). That is, we show that if λ C λµν X λ = 0 is satisfied for arbitrary µ and ν, where X λ is an arbitrary factor which depends on λ but does not depend on µ and ν, then X λ vanishes for all Young diagram λ. It turns out that this statement holds even if we relax the condition by choosing ν = ∅. That is, if is satisfied for arbitrary µ, then X λ vanishes, where we use the cyclic symmetry of the topological vertex. By multiplying the Schur functions s µ T (x) to the left hand side of (F.1) and by summing over all the Young diagrams for µ, we obtain where we use the formula (1 + x i y j ). (F.4) Therefore, (F.1) indicates that the right hand side of (F.2) vanishes for arbitrary x i . Suppose a Young diagram Y is given. For given Y , we choose Then, (F.1) and (F.2) leads to where R Y T λ (Q) is defined in (3.32). Here, we use the identity with the unrefined Nekrasov factor It is known [78] that N Y λ (1) = 0 if and only if Y = λ. Therefore, (F.6) leads toX Y = 0 and thus, X Y = 0. Since this argument holds for any given Young diagram Y , we have derived that for all λ.