M5-brane and D-brane scattering amplitudes

We present tree-level n-particle on-shell scattering amplitudes of various brane theories with 16 conserved supercharges. These include the world-volume theory of a probe D3-brane or D5-brane in 10D Minkowski spacetime as well as a probe M5-brane in 11D Minkowski spacetime, which describes self interactions of an abelian tensor supermultiplet with 6D (2, 0) supersymmetry. Twistor-string-like formulas are proposed for tree-level scattering amplitudes of all multiplicities for each of these theories. The R symmetry of the D3-brane theory is shown to be SU(4) × U(1), and the U(1) factor implies that its amplitudes are helicity conserving. Each of 6D theories (D5-brane and M5-brane) reduces to the D3-brane theory by dimensional reduction. As special cases of the general M5-brane amplitudes, we present compact formulas for examples involving only the self-dual B field with n = 4, 6, 8.

This paper proposes explicit formulas for on-shell n-particle scattering amplitudes in the tree approximation for three massless field theories, each of which is a maximally supersymmetric matter theory with 16 unbroken supersymmetries and 16 additional spontaneously broken supersymmetries. The fermions in these theories are Goldstone particles (or Goldstinos) of the type first considered by Volkov and Akulov [1,2]. These three theories arise naturally in string theory as the world-volume theories of branes. The first theory is the world-volume theory of a probe D3-brane (of type IIB superstring theory) in a 10D Minkowski-space background. The second theory is the world-volume theory of a probe D5-brane (of type IIB superstring theory) in a 10D Minkowski-space background. The third theory is the world-volume theory of a probe M5-brane (of M theory) in an 11D Minkowski-space background. We will refer to these theories as the D3 theory, the D5 theory, and the M5 theory. These three theories are closely related. Specifically, both of the 6D theories (D5 and M5) can be truncated (by a procedure called dimensional reduction) to give rise to the 4D theory (D3). These relationships, which are predicted by string theory, will provide powerful checks of the results, as well as a role in their derivation. Another important feature that all three of these theories have in common is that nonvanishing on-shell scattering amplitudes require an even number of particles, i.e.,n must be even. The D3 theory is a 4D Dirac-Born-Infeld (DBI) theory, with N = 4 supersymmetry, which some authors call sDBI theory. It it a self-interacting theory of a massless abelian N = 4 vector supermultiplet, which consists of a vector, four spinors, and six scalars. Its Rsymmetry group is SU(4) × U (1). Although the helicity-conservation property of scattering amplitudes of the D3 theory has also been understood previously [3] using the electric magnetic duality of D3-brane action [4], the additional U(1) factor in the R-symmetry group has not been noted previously. 1 The action for the D3 theory was derived in [5] by dimensional reduction of the action for the D9-brane, which was constructed using stringtheoretic techniques. (See [6][7][8][9] for related work.) The D3 theory has been examined in some detail recently in [10]. There has been a recent proposal for the tree amplitudes of this theory in [11,12]. Our formulas will take a different form, for reasons that will be explained.
The action for the D5 theory also can be obtained by dimensional reduction of the D9brane action. This theory is a self-interacting theory of a single vector supermultiplet with (1, 1) supersymmetry in 6D. The vector supermultiplet consists of a vector, four spinors, and four scalars. The R-symmetry group of the D5 theory is SU(2) × SU (2). The M5 theory is a self-interacting theory of a single tensor supermultiplet with (2, 0) supersymmetry in 6D. This multiplet contains a two-form field B with a self-dual field strength (H = dB = H) as well as four spinors and five scalars. There is an analog of the Born-Infeld action that describes self interactions of the B field, which was constructed in [13]. This theory has 6D Lorentz invariance, though the action only has manifest 5D Lorentz invariance. The five additional Lorentz transformations that involve a particular JHEP12(2017)003 (arbitrarily chosen) direction are not obvious symmetries. These transformations of the Lagrangian give a total derivative. Dimensional reduction of this theory to five dimensions gives pure Born-Infeld theory. The action for the supersymmetric extension of the 6D theory that incorporates the complete (2, 0) supermultiplet, i.e.,the M5 theory, was constructed in [14]. (See [15][16][17][18] for related work.) The R-symmetry group of this theory is USp (4). Certain lower-point amplitudes for the M5 theory have been discussed previously, for example in [19][20][21][22][23]. The requirement that they give D3 amplitudes after dimensional reduction to 4D will play an important role in our analysis.
Another feature that these three theories have in common is that they inherit their symmetries from those of the parent theories, i.e.,M-theory in flat space and Type IIB superstring theory in flat space. By positioning the probe branes in the ambient space, some of the symmetries of the parent theory are spontaneously broken. Broken symmetries include translations perpendicular to the branes and half of the supersymmetries. These symmetries are realized nonlinearly in the brane theories. Thus, the scalars and spinors in these theories are Goldstone particles. As a result, the amplitudes of these theories satisfy various soft theorems. The vector and tensor gauge symmetries are inherited from the background NS-NS 2-form of Type IIB and the M-theory 3-form, respectively [24].
One of the challenges in formulating on-shell scattering amplitudes for these theories is to make their various required symmetries manifest. As has become conventional for massless particles, we use twistor-like spinor-helicity coordinates to represent momenta and supercharges. These introduce a little-group symmetry for each of the scattered particles. As we will explain, this group is SU(2) × SU(2) for the D5 theory, SU(2) for the M5 theory, and U(1) for the D3 theory. The use of spinor-helicity variables allows us to construct on-shell amplitudes with manifest Lorentz invariance even for chiral theories, such as the M5 theory, which has well-known obstructions to constructing a useful Lorentz-invariant action. In addition to super-Poincaré symmetry, each of these theories has an R-symmetry group: SU(2) × SU(2) for the D5 theory, USp(4) for the M5 theory, and SU(4) × U(1) for the D3 theory.
Our formulas for scattering amplitudes in each of the three theories take forms that are similar to the twistor-string formulation of 4D N = 4 super Yang-Mills amplitudes in Witten's classic twistor-string paper [25]. The twistor-string formulation for 4D N = 4 super Yang-Mills is often called Witten-RSV formula, which was studied in detail in in [26][27][28].In particular, we associate a coordinate σ i on the Riemann sphere to the ith particle in an n-particle scattering amplitude. The formula for the amplitude is required to be invariant under a simultaneous SL(2, C) transformation of these coordinates. Following Cachazo et al. [29], in the twistor-string-like formalism that we use, certain rational functions of σ i are associated to the ith particle. These functions are restricted by delta-function constraints in such a way that the number of bosonic delta functions is equal to the number of bosonic integrations. Thus, the formulas are actually algebraic, as they should be for tree amplitudes. Furthermore, the delta-function constraints imply the scattering equations [30], which are j p i · p j /σ ij = 0, where σ ij = σ i − σ j . This approach allows us to formulate all of the amplitudes for the three theories in a uniform way. It also is convenient for verifying some of their essential properties.

JHEP12(2017)003
Our main results are general formulas for the n-particle on-shell tree amplitudes for each of the three theories. These formulas make most of the required symmetries manifest, or at least easy to understand. The exception is the R symmetry, where only a subgroup is manifest. The supermultiplets are incorporated by associating four Grassmann coordinates, with specified transformation properties, to each external particle. The key to making the full R-symmetry group manifest is to carry out a Fourier transformation for half of the Grassmann coordinates -two per particle. The price that one pays for making R symmetry manifest is that the formulas become somewhat more complicated for the 6D theories.
The paper is organized as follows: we begin in section 2 with a discussion of general properties, such as symmetries, conserved charges, and on-shell states, for each of the three theories considered in this paper. We utilize the 4D spinor-helicity formalism for the D3 theory and the 6D one for the M5 theory and the D5 theory. To illustrate the structures and ideas, section 3 examines the four-particle amplitudes for these theories. Section 4 presents a general formula for the n-particle amplitudes of the D3 theory. As mentioned previously, our formulas for scattering amplitudes in each of the three theories take forms that are similar to the twistor-string formulation of 4D N = 4 super Yang-Mills amplitudes [25]. This formulation of the D3 theory is somewhat different from those in the literature. It is more suitable for the generalization to 6D, which is required for the M5 and D5 theories.
In section 5 we propose a new formula, given in eq. (5.31), which gives all of the tree amplitudes of the M5 theory and generalizes the D3 formula in a way that is consistent with dimensional reduction of N = (2, 0) in 6D to N = 4 in 4D. This is our most novel result, providing a mathematical formula for the complete tree-level S-matrix for a theory whose Lagrangian description has well-known issues mentioned earlier. This section also describes various checks of the formula, including symmetries, soft theorems, and factorization. Using knowledge of the lower-point amplitudes and factorization, we obtain compact analytic expressions for certain amplitudes of the self-dual B fields for n = 6 and n = 8. These agree perfectly with the general integral formula and give explicit consistency checks. Despite the apparent differences between the M5 and D5 theories, in section 6 we present a similar integral formula for the D5-brane amplitudes, which reproduces what one obtains from the D5-brane action. Finally, our conclusions and remarks concerning future directions are presented in section 7. Further technical details and an analysis of the R symmetries are presented in the appendix A.

Symmetries, conserved charges, and supermultiplets
The three theories that we are considering have three types of conserved charges, which form a nice superalgebra in each case. These charges, are the momenta p i , supersymmetry charges q i , and R-symmetry charges R i , where the index i = 1, 2, . . . , n labels the n particles participating in an n-particle on-shell scattering amplitude A n . By treating all of the particles symmetrically as ingoing, conservation of these charges is simply the JHEP12(2017)003 In practice, some of these conservation laws are implemented by including appropriate delta functions in the formula for A n . The other charges are represented by differential operators and their conservation is achieved by requiring that A n is annihilated by the appropriate sums of these differential operators. Lorentz invariance will be manifest in all formulas.

M5 theory
The world-volume theory of a probe M5 in an 11D Minkowski space background has (2, 0) 6D supersymmetry. This theory describes a single massless self-interacting tensor supermultiplet. This supermultiplet contains a two-form field B µν , with a three-form field strength H = dB, which is self-dual in the free-theory limit. Such a field gives rise to three on-shell degrees of freedom. The tensor supermultiplet also contains four fermions and five scalars. Altogether, there are eight bosonic and eight fermionic on-shell degrees of freedom. The three multiplicities (1,4,5) correspond to representations of the USp(4) = Spin(5) R-symmetry group, which is an unbroken global symmetry of the M5 theory. This symmetry can be thought of as arising from rotations in the five spatial dimensions that are orthogonal to a flat M5 in 11D Minkowski spacetime. The little group for massless particles in d dimensions is Spin(d − 2). Thus, in 6D it is SU(2) × SU (2). However, in the special case of the tensor multiplet all of the on-shell particles are singlets of one of the two SU(2) factors. Specifically, the self-dual tensor transforms as (3,1), the spinors, which are also chiral, transform as (2, 1) and the scalars transform as (1, 1). Therefore we shall ignore the trivial SU(2) and refer to the nontrivial SU(2) as the little group of this theory. In the case of the D5 theory, considered in the next subsection, both SU(2) factors will be required. It is convenient to introduce four Grassmann coordinates, such that the entire on-shell supermultiplet can be described by a single scalar expression. There are various ways to do this. One obvious choice is to introduce four Grassmann coordinates η I , which transform as the fundamental four-dimensional representation of the USp(4) R-symmetry group. In this way, one can make the R symmetry manifest, and we first discuss this formulation. However, because amplitudes for massless particles are labeled by incoming momenta and little-group indices, in most formulas we will make use of a second description of the supermultiplet that makes little group symmetry manifest.
For theories involving massless particles, it is also convenient to introduce eight bosonic spinor-helicity coordinates λ A a , where A = 1, 2, 3, 4 labels a spinor representation of the 6D Lorentz group Spin(5, 1) and a = ± labels a doublet of the chiral little group discussed above. These coordinates belong to a real representation of the product group, because the spinor representation of the Lorentz group and the doublet little-group representation are both pseudoreal. In terms of these coordinates the momentum of an on-shell massless particle is written [31],

JHEP12(2017)003
This formula is invariant under the SU(2) little group, and therefore three of the eight λ coordinates are redundant, leaving five nontrivial degrees of freedom, as appropriate for the momentum of a massless particle in 6D. Note also that p AB = −p BA is a six-vector of the Lorentz group. p 2 , which gives the square of the mass, is proportional to the Pfaffian of p AB . This vanishes because the 4 × 4 matrix p AB has rank two. When we describe n-particle scattering amplitudes we attach labels i, j, . . ., which take the values 1, 2, . . . , n, to the coordinates. Thus, the ith particle is associated to λ A i+ , λ A i− , and η I i . The 16 supersymmetry charges of the M5 theory can be represented by 2 where the antisymmetric matrix Ω IJ is the symplectic metric. We will find it convenient later to choose Ω 13 = Ω 24 = 1. This formula can be recast as where η I − = η I and η I + = Ω IJ ∂/∂η J . Then This makes the little-group invariance of the supercharges manifest. Note that the supercharges belong to a chiral representation of the Lorentz group, and the opposite chirality representation does not appear. This is what is meant by saying that the theory has (2, 0) supersymmetry. As usual, the supercharges anticommute to give the momenta The ten R charges, R IJ = R JI , are represented by These charges generate USp(4) and they transform the supercharges appropriately The on-shell supermultiplet consists of three kinds of particles: a helicity triplet B ab = B ba , which is an R-symmetry singlet, a helicity doublet ψ a I , which an R-symmetry quartet, and a helicity singlet φ IJ = −φ JI , Ω IJ φ IJ = 0, which is an R-symmetry quintet. These can be combined into a single R-symmetry invariant expression: where we have defined

JHEP12(2017)003
Note that each + or − superscript correspond to half a unit of H 3 , the third component of the little group SU(2) algebra. Each term in Φ, and hence Φ itself, carries a total of one unit of H 3 if we assign a half unit of H 3 to each factor of η. This was to be expected because η I was introduced as a renaming of η I − = η I+ . This description of the supermultiplet has two deficiencies: first, it is not invariant under the little group; second, little-group multiplets are split up among different terms in the expansion. As noted already, both of these deficiencies can be overcome by using a different formulation of the supermultiplet. The price to be paid will be that only an SU(2) subgroup of the USp(4) R-symmetry group will be manifest.
The SU(2) little group is not a global symmetry of the M5 theory. Rather, it is a redundancy in the formalism, analogous to a local symmetry, which is not manifest in the preceding equations. It can be made manifest by Fourier transforming half of the η coordinates. A Fourier transform replaces a Grassmann coordinate by a Grassmann derivative and vice versa. As before, we choose Ω 13 = −Ω 31 = Ω 24 = −Ω 42 = 1, while all other components of Ω IJ vanish. Then we replace η 3 and η 4 by derivatives with respect toη 1 andη 2 and vice versa. Also renaming (η I ,η I ) as (η I − , η I + ). Altogether the four coordinates η I are replaced by four coordinates η I a , which now transform as a doublet of the little group and as a doublet of an SU(2) subgroup of the R symmetry group. The formulas for the 16 supercharges become As promised, we have traded manifest USp(4) R symmetry for little group SU(2) symmetry. This is also the case for the on-shell supermultiplet formula, which is a Grassmann Fourier transform of the one in eq. (2.9). It now takes the form Recall that in Φ(η) the index I takes four values, whereas inΦ(η) it takes two values. (We prefer not to introduce another symbol.) The five scalars are split 1 + 3 + 1 and the four spinors are split 2 + 2 even though they form irreducible R-symmetry multiplets. To summarize, the Φ representation has manifest R symmetry, whereas theΦ representation has manifest little-group symmetry. The latter representation will turn out to be the easier one to deal with, and our main formulas for scattering amplitudes will use this superfield description.

D5 theory
The world-volume theory of a probe D5 in a 10D Minkowski space background has (1, 1) 6D supersymmetry. On-shell superspace with (1, 1) 6D supersymmetry has been used for studying 6D super Yang-Mills theory, see e.g. [32][33][34]. This theory, which is nonchiral, i.e.,parity invariant, describes a single massless self-interacting vector supermultiplet. This supermultiplet contains a one-form field A µ , with a two-form field strength F = dA. Such a field gives rise to four on-shell degrees of freedom. The vector supermultiplet also contains four fermions and four scalars. Altogether, there are eight bosonic and eight fermionic JHEP12(2017)003 on-shell degrees of freedom. The three multiplicities (1,4,4) correspond to representations of the SU(2) × SU(2) = Spin(4) R-symmetry group, which is an unbroken global symmetry of the D5 theory. The representations are (1, 1) for the vector, (2, 2) for the scalars and (1, 2) + (2, 1) for the fermions. This symmetry can be thought of as arising from rotations in the four spatial dimensions that are orthogonal to a flat D5 in 10D Minkowski spacetime.
As discussed earlier, the little group in 6D is also SU(2) × SU (2). Altogether, in terms of four SU(2) factors, with the first two referring to the little group and the second two to the R-symmetry group, the vector supermultiplet contains the following representations: (2.13) Note that, unlike the M5 theory, the D5 theory involves nontrivial representations of both SU(2) factors of the little group. In terms of on-shell fields, these representations correspond to A aâ , φ IÎ , ψ aÎ , and ψâ I , in a notation that should be self-explanatory. As before, we can introduce eight bosonic expressions λ A a , where A = 1, 2, 3, 4 labels a spinor representation of the 6D Lorentz group Spin(5, 1) and a = ± labels a doublet of the first SU(2) factor in the little group. In terms of these coordinates the momentum of an on-shell massless particle can be written in the form (2.14) Three of the eight λ coordinates are redundant, leaving five nontrivial degrees of freedom, as appropriate for a massless particle in 6D. Unlike, in the M5 theory, this is not sufficient.
The Lorentz group has a second four-dimensional spinor representation, corresponding to the opposite chirality, and the little group has a second SU(2) factor, both of which are utilized (on an equal footing with the first ones) in the D5 theory. Therefore, it is natural to introduce an alternative formula for the momentum utilizing them Since the momentum six-vector p µ i is given by 1 2 μ ABpÂB i , where σ and σ are the appropriate Lorentz-invariant tensors, the information encoded in λ i andλ i , modulo little-group transformations, is the same. In fact, if one of them is given, the other is determined up to a little-group transformation. The two four-dimensional representations of the 6D Lorentz group, labeled by the indices A andÂ, are inequivalent. If the group were SU(4) they would be complex conjugates of another, but for Lorentzian signature the group is Spin(5, 1) and each of these representations is pseudoreal. Nonetheless, for either signature it is a fact that the Kronecker product of these two representation gives the adjoint plus a singlet. Therefore, the Kronecker delta δ AÂ is an invariant tensor, and AÂ δ AÂ λ A aλÂ a (2.16) is Lorentz invariant. In fact, this combination must vanish in order that p 2 ∼ δ AÂ δ BB p ABpÂB = 0. It will be important later in the analysis of the M5 theory that JHEP12(2017)003 λ determinesλ up to a little-group transformation and that the combination eq. (2. 16) vanishes. Henceforth, we distinguish the two spinor representations by the use of upper and lower indices, i.e., Â δ AÂλÂ a =λ Aâ . (2.17) Throughout the paper, we will use the above upper and lower indices, and in this notation we have the constraint on spinors given as, (2.18) In the notation introduced above, the 16 supercharges are given by q AI andqÎ A . Then the (1, 1) supersymmetry algebra is These are conveniently represented by where the Grassmann coordinates satisfy Now, there are again two alternative representations of the on-shell superfield distinguished by whether the R symmetry or the little-group symmetry is manifest. The formula with manifest R symmetry utilizes the four anticommuting Grassmann coordinates η I − and ηÎ−, which we simplify to η I andηÎ . In terms of these, the expansion is Φ(η) = A ++ + η I ψ+ I +ηÎ ψ +Î + η IηÎ φ IÎ + η 2 A −+ +η 2 A +− + · · · + η 2η2 A −− , (2.22) where η 2 = 1 2 ε IJ η I η J and similarly forη 2 The alternative representation with manifest little-group symmetry utilizes the I = 1 components of η I a , now denoted η a , and theÎ = 1 components ofηÎ a , now denotedηâ. The on-shell superfield in this representation is Φ(η) = φ 11 + η a ψ a1 +ηâψâ 1 + η aηâ A aâ + η 2 φ 21 +η 2 φ 12 + · · · + η 2η2 φ 22 . (2.23) As before, the two representations are related by a Grassmann Fourier transform. Since the little group and the R symmetry are both SU(2) × SU(2) for the D5 theory the two superfield formulas have the same structure with the role of the R-symmetry and littlegroup symmetry interchanged.

D3 theory
Since the D3 theory can be obtained by dimensional reduction of the M5 theory or the D5 theory, let us consider what happens when all of the momenta are restricted to a 4D Minkowski subspace. The Lorentz group then becomes SL(2, C) and the 4 of Spin(5, 1) decomposes as 2 +2. In fact, this is correct for both of the four-dimensional spinor representations of the 6D Lorentz group, and it is appropriate and consistent to require that λ A a andλ Aâ become identical when restricted to 4D. In standard notation, the spinor index A → (α,α). In terms of λ A a the restriction to 4D is achieved by setting λ α − = 0 and λα + = 0. This then gives p αβ = pαβ = 0 leaving the familiar 4D formula for an on-shell massless particle in helicity variables: Now p 2 is proportional to the determinant of p αα , which vanishes because this matrix has rank one. Let us now focus on reduction of the M5 theory. The case of the D5 theory is very similar. The restrictions on the momenta imply that the supercharges in eq. (2.3) reduce to q αI = λ α η I and qα I =λα ∂ ∂η I I = 1, 2, 3, 4, (2.25) where we have set λ α + = λ α and λα − =λα, which is the standard notation. Also, an unnecessary constant factor has been removed in the formula for qα I . Then q αI and qα I form complex-conjugate representations.
The R symmetry can now be extended to SU(4), with generators given by the traceless expression The SU(4) symmetry is manifest in the on-shell supermultiplet expression derived from eq. (2.9) The middle term now describes six scalars, one of which descends from B +− . The amplitudes of the D3 theory have an additional U(1) symmetry that can be interpreted as conservation of helicity. Its generator is This is the operator that reads off the helicity of a particle, and therefore its conservation, HA n = ( i H i )A n = 0, implies that the total helicity of the particles participating in a nonvanishing n-particle scattering amplitude must be zero. Conservation of this charge implies that the amplitude is homogeneous of degree 2n in these η coordinates. Moreover, SU(4) R symmetry requires that the total number of η's must be a multiple of four. Together these statements imply that n must be even for the D3 theory. In fact, we claim JHEP12(2017)003 that n must also be even for the M5 and D5 theories even though this reasoning is not applicable in those cases. The U(1) symmetry generated by H does not commute with the supercharges. Therefore, by definition, it is an additional R symmetry, extending the R-symmetry group to SU(4) × U(1). Let us now explain how the appearance of this symmetry could have been anticipated. Since the D3 theory can be obtained by dimensional reduction of the D9-brane theory, the SU(4) subgroup can be regarded as arising from rotations in the six dimensions transverse to the D3. So where does the additional U(1) R symmetry come from? Having posed the question, the answer becomes clear. The D3 theory can also be obtained by dimensional reduction of the M5 theory, so the U(1) can be interpreted as rotations in the two extra dimensions of this construction.
In the case of N = 4 super Yang-Mills (SYM) theory the SU(4) R symmetry can also be understood by dimensional reduction starting from SYM in ten dimensions. In fact, like the D3 theory, that is how this theory was originally obtained. However, N = 4 SYM cannot be obtained by dimensional reduction of a perturbative theory in 6D with (2, 0) supersymmetry. There are nonperturbative (2, 0) theories in 6D that reduce to N = 4 SYM when placed on a torus. In such a reduction, the 4D coupling constant is determined by the ratio of the radii of two cycles of the torus, and different choices are related by dualities. This is not the kind of dimensional reduction that would give rise to an extra U(1) symmetry. Even when Kaluza-Klein excitations are omitted, such a reduction does not retain the transverse rotational symmetry that is needed to give an additional U(1) R symmetry. Therefore, in the case of N = 4 SYM, helicity is not conserved and n does not need to be even.
As in the previous examples, there is an alternative form of the supercharges and the on-shell superfield that exhibits manifest little-group symmetry. As a consequence only an SU(2) × SU(2) subgroup of the SU(4) R symmetry remains manifest. As before, this representation is related to the previous one by Fourier transforming two of the four Grassmann coordinates. In this new basis the 16 supercharges take the form The indices I andÎ label doublets of distinct SU(2) subgroups of the R symmetry group. The indices ± keep track of U(1) little-group representations, which corresponds to helicity. In this formulation the on-shell superfield becomes

M5 theory
Before discussing the general case, let us consider four-particle amplitudes, starting with the M5 theory. The plan is to first propose a formula for the result that corresponds to supermultiplets written in the form given in eq. (2.12). This representation has a manifest SU(2) little-group symmetry for each external particle. Up to normalization, the fourparticle amplitude with four derivatives for an abelian tensor supermultiplet with 6D (2, 0) supersymmetry is uniquely given by where A, B = 1, 2, 3, 4 and I = 1, 2. The fermionic delta functions are defined for instance in [32]. It will be useful later to write the momentum-conservation condition in matrix notation as In other words, the matrix (λ + λ T − ) AB is symmetric. This is valid for any number of particles n. In the special case of n = 4, λ + and λ − are square matrices. If n = 4 and λ − is invertible, which is generically the case, this implies that (λ −1 − λ + ) ij is symmetric. This fact will be useful later.
The formula for A 4 manifestly satisfies several requirements: total symmetry in the four particles, Lorentz invariance, conservation of momentum and half of the supercharges, and little-group symmetry. Also, the second factor scales as λ 8 or p 4 , as expected. Conservation of the other half of the supersymmetries is easy to verify. What one needs to show is that This fact is an immediate consequence of {q A iI , q BJ j } = p AB i δ ij δ I J and conservation of momentum.
In order to appreciate eq. (3.1), let us examine what it implies for the scattering of four B particles. They are R-symmetry singlets whose on-shell degrees of freedom are described by a symmetric tensor B ab = B ba of the SU(2) little group. Eq. (3.1) implies that their four-particle amplitudes are given by where P 4 denotes the symmetrization over little group indices. Here and throughout, we make use of a Lorentz-invariant bracket:

JHEP12(2017)003
It is interesting to note that any four B particles have a nonzero amplitude. For example, Similarly, the amplitude for four B −− particles is given by (det λ − ) 2 . On reduction to the D3 theory, B ++ becomes a positive-helicity photon, and this amplitude vanishes. Indeed, eq. (3.4) gives all the four-photon amplitudes correctly, with the only nonzero ones involving two positive-helicity and two negative-helicity photons. It also describes amplitudes involving additional scalars that arises from reduction of B +− = B −+ .
Let us now turn to the more difficult issue: verifying USp(4) R symmetry of an arbitrary four-particle amplitude. We have learned earlier that this symmetry should be manifest in the representation of the supermultiplet given in eq. (2.9). To get to this representation, we rename η I i− as η I i and η I i+ asη I i . Then we Fourier transform the latter coordinates to conjugate Grassmann coordinates denoted ζ iI . Thus, we consider Substituting an integral representation of the delta functions and carrying out theη integrations gives If we now assume that the 4×4 matrix λ A i− is nonsingular, which is generically the case, then and thus More explicitly, the exponent is As was explained earlier, momentum conservation implies that (λ −1 − λ + ) ij is a symmetric matrix. Therefore only the symmetric part of (ηζ) ji contributes, which can therefore be replaced by half of We now claim that E (and hence A 4 ) can be rewritten in a form that has manifest USp(4) symmetry

JHEP12(2017)003
where the only nonzero elements above the diagonal of the symplectic metric are Ω 13 = Ω 24 = 1. Note that we have renamed ζ Ii = η I+2 i . Then η I i belongs to the fundamental representation of the USp(4) R-symmetry group. The same idea discussed here applies to more general n-particle amplitudes as we shown in appendix B.
Note that the amplitude for four B −− particles is given by the first term in the expansion of the exponential, whereas the amplitude for four B ++ particles is given by the last (eighth) term in the series expansion of the exponential. All other four-particle amplitudes are contained in the intermediate powers. Clearly, this representation (with manifest R symmetry) is more complicated than the previous one with manifest little group SU(2) symmetries for each of the scattered particles, that amplitudes are no longer homogenous polynomials in terms of fermionic variables η's.

D5 theory
The four-particle amplitude for this theory is quite similar to the one for the M5 theory. In the representation with manifest little-group symmetry the four Grassmann coordinates that are used in the superfieldΦ(η) are η a andηâ. They transform as (2, 1) and (1, 2) with respect to the SU(2) × SU(2) little group. In terms of these we can define eight anticommuting supercharges Then the desired amplitude is In particular, we can read off the amplitude for scattering four vector particles As in the case of the M5 theory, the R symmetry of the D5 amplitudes can be verified by carrying out a Grassmann Fourier transform to the representation in which that symmetry becomes manifest.

D3 theory
Since the D3 theory can be obtained by dimensional reduction of the M5 theory, its fourparticle amplitude can be deduced from the preceding results. Specifically, eq. (3.1) reduces to As before, this is easily seen to have all of the required properties aside from R symmetry. Alternatively, the same result can be obtained by dimensional reduction of the D5 theory, whose four-particle amplitude is given in eq. (3.15). In this case, dimensional reduction of q A gives q α1 and qα 1 , while dimensional reduction ofq A gives q α2 and qα 2 .
R symmetry can be investigated, as before, by Fourier transforming the ηÎ i+ coordinates. (Recall that I = 1, 2 andÎ = 1, 2 label doublets of the two SU(2) factors of an SU(2) × SU(2) subgroup of the SU(4) R symmetry group.) However, the analysis requires some modification, since the matrix λ − , which was previously assumed to be nonsingular, is now singular. In fact, two of its four columns are identically zero.
Since η + only occurs in the last delta-function factor, let us consider its Fourier transform (3.20) Momentum conservation can be written as the matrix equation (λ Tλ ) αα = 0. Therefore the eight delta functions imply the four relations i λ α i ζ iÎ = 0. From this it follows that where J is a Jacobian factor.It is straightforward to see that the Jacobian is Here we are using the standard notation of 4D spinor helicity formalism, ij = ε αβ λ α i λ β j and [ij] = εαβλα iλβ j . It is important that J should have total symmetry in the four particles. The proof that [12]/ 34 has total antisymmetry, and hence that J has total symmetry, is straightforward using momentum conservation.
To complete the analysis, we define ζ1 = η 3 and ζ2 = η 4 , as before. Then, assembling the results above, the Fourier-transformed scattering amplitude becomes where the index I on q αI i = λ α i η I i now takes four values. This version of four-particle superamplitude has appeared before, for instance in [22]. It now has manifest SU(4) R symmetry, because the Grassmann delta functions contain two factors of ε IJKL η I i η J j η K k η L l , which is SU(4) invariant. The amplitude has an additional U(1) R symmetry, because it contains 2n = 8 factors of η, as explained earlier.

JHEP12(2017)003
4 n-particle amplitudes of the D3 theory This section briefly reviews the n-particle amplitudes for the tree-level S-matrix of the D3 theory. A nice formula with manifest SU(4) R symmetry appeared recently in [11,12]. However, for the purpose of generalizing to the M5 theory, it is more convenient to break the SU(4) R-symmetry SU(4) → SU(2) L × SU(2) R and make the little-group symmetry manifest. A formula of the required type has appeared previously for 4D N = 4 SYM and N = 8 supergravity [29]. It contains complex coordinates σ i (on the Riemann sphere) associated to the n particles. The formula is required to be invariant under simultaneous SL(2, C) transformations of these coordinates. This implies that only n − 3 of them are integrated, while the other three can be set to arbitrarily chosen distinct values.
The on-shell n-particle amplitude formula takes the form where ∆ B is a product of bosonic delta functions and ∆ F is a product of fermionic (or Grassmann) delta functions Here ρ α (σ) and χÎ + (σ) are degree-d polynomials, whileρα(σ) and χ I − (σ) are degree-d polynomials, with d +d = n − 2. (4.4) Thus, ρ α (σ) (bosonic) and χ I − (σ) (fermionic) take the form Also, JHEP12 (2017)003 where σ ij = σ i − σ j . Note that P i (σ) depends on all n of the σ coordinates, but σ i has a distinguished role. The integral is taken over the space of punctures and polynomials, the measure for which contains the following 2n bosonic and 2n fermionic integrations: The integral has a gauge redundancy from the modular and little-group symmetries, so we must divide by the volume of where the modular group SL(2, C) acts on the σ i 's and GL(1, C), the complexified little group, acts on the ρ's andρ's. Eq. (4.1) describes maximally supersymmetric theories with the on-shell states organized according to eq. (2.31). It gives the usual scattering amplitude supplemented by additional delta functions, namely where A n is the usual scattering amplitude including the four momentum-conservation delta functions and eight supercharge-conservation delta functions. (When the momentumconservation delta function is also omitted, the amplitude is denoted T n ). The bracket notation is the same as described following eq. (3.22). The extra delta functions in eq. (4.10) impose the conditions that allow us to introduce the usual on-shell relations of the schematic form p = λλ and q = λη. So, in practice, to extract the scattering amplitudes A n from eq. (4.1), one should use these relations and remove the extra delta functions. Appendix A contains the proof that the 4n bosonic delta functions ∆ B account for the n mass-shell conditions, four momentum conservation equations, and the n − 3 scattering equations. These are precisely the 2n + 1 delta functions that survive after carrying out the (2n − 1)dimensional ρ integration. The choice of the factor I in the integrand depends on the theory. For example, the color-ordered N = 4 SYM amplitudes, discussed in [29], are given by the Parke-Taylorlike factor In the case of YM and SYM theories in 4D, the solutions of the scattering equations can be separated into n − 3 sectors characterized by the total helicity (or "helicity violation") of the n particles participating in the reaction. The sectors, labeled by d = 1, 2, . . . , n − 3, haved − d = n − 2(d + 1) units of helicity violation. In particular, the d = 1 sector, which has n − 4 units of helicity violation, is usually referred to as having "maximal helicity violation" (MHV). If n is even, the sector with d =d = n 2 − 1 is helicity conserving. As was first conjectured in [35] and later proven in [29], the number of solutions of the scattering equations that contribute to the (d,d) sector, denoted N d,d , is given by an Eulerian number.

JHEP12(2017)003
These numbers satisfy N d,d = Nd ,d and N d,1 = 1. They are fully determined by these relations and the recursion relation [29] which accounts for all the solutions of the scattering equations.
Due to the recent progress in understanding CHY representations of scattering amplitudes [36][37][38], it is known that one can pass from YM theories to DBI theories by simply replacing I YM by where S n is an n × n anti-symmetric matrix with where s ij = (p i + p j ) 2 = 2p i · p j are the familiar Mandelstam invariants. Also, Here S i,j i,j means that the i-th and j-th rows and columns of the matrix S n are removed before computing the Pfaffian or determinant. This is required because S n has rank n − 2 if n is even. Then det S n is independent of the choice of i and j and transforms with weight two under SL(2, C) transformations of the σ coordinates. If n is odd, there is no satisfactory nonzero definition. Therefore all nonzero amplitudes of all DBI-like theories must have n even. This includes all three brane theories (D3, D5, M5) that are the main emphasis of this paper.
However, if one examines the actions in the literature for these theories, it is only obvious that n must be even for the bosonic truncation, in each case, but it is not at all obvious when fermions are involved. These actions, which were derived using various string theory considerations, contain vertices involving an odd number of bosons when fermions are also present. Since we claim that on-shell amplitudes with an odd number of bosons always vanish, it must be possible to eliminate all terms in the action that have an odd number of boson fields by field redefinitions. At the leading nontrivial order, the analysis in section 2.1 of [10] implies that this is the case for the D3 theory. Otherwise, this issue does not seem to have been explored.
In the case of the D3 theory, the extra U(1) R symmetry, discussed earlier, implies that only the helicity-conserving sector, with
As indicated in eq. (4.1), one should mod out the volume of G = SL(2, C) × GL(1, C), where SL(2, C) acts on the σ i 's and GL(1, C) acts on the ρ's and χ's. In practice, we may fix any three σ i 's (for instance σ 1 , σ 2 , σ 3 ) and one ρ (for instance ρ 1 0 ) to arbitrary values, with the compensating Jacobian We note that the integral formula is not a "true integral", in the sense that the number of bosonic delta-functions is equal to the number of integration variables (after taking account of the G symmetry). This is not a surprise, of course, since we know that tree amplitudes are entirely algebraic. As mentioned earlier, the counting of bosonic delta functions is as follows: the 4n bosonic delta-functions in ∆ B give rise to n delta functions for mass-shell conditions in the coefficient of A n in eq. (4.10) and four more for momentum conservation, δ 4 ( n 1 p µ i ), which are included in A n . The remaining 3n − 4 delta functions determine the (n − 3) σ's and (2n − 1) ρ's that survive after modding out by the volume of G. The Jacobian that arises from these evaluations is computed explicitly in the appendix A. Also, there are 8n fermionic delta functions in ∆ F and 2n fermionic integrations in dM, leaving an expression of order 6n in fermionic coordinates. 4n of these appear in the coefficient of A n in eq. (4.10). Therefore the remaining 2n η's must be in A n . In fact, half of them are η + 's and half are η − 's. This is the number that we argued earlier are required (in this representation) by the U(1) factor in the R symmetry group of this theory.
The powers of momenta that appear in A n can also be checked. In theories of Born-Infeld type, such as we are considering, one expects that T n ∼ p n . In four dimensions this implies that A n ∼ p n−4 and A n ∼ p 3n−4 . The latter, given for the D3 theory in eq. (4.1), contains p n from the measure, p −4n from ∆ B , p 4n from ∆ F , and p 2n−4 from det S n . These combine to give p 3n−4 , as desired.
Appendix A describes the Jacobian factor generated by pulling out the "wave functions" and the momentum conservation delta function. Using these results for the Jacobian, we have checked explicitly that eq. (4.1), with I = det S n , reproduces the four-point amplitude of the D3 theory given in eq. (3.19) as well as the six-point super amplitudes, which may be found in [22]. The appendix also contains the proof that the amplitudes have SU(4) R symmetry (in addition to the U(1) already demonstrated). 5 n-particle amplitudes of the M5 theory

The proposed formula
This section generalizes the twistor-string-like formula of the D3 theory in eq. (4.1) to the M5 theory with (2, 0) supersymmetry in 6D. The n-particle tree-level scattering amplitude JHEP12(2017)003 for this theory takes the form These delta functions are the natural (2, 0) generalization of the corresponding D3 formulas. The factor det S n is unchanged from the D3 case, since it is a sensible function of the invariants s ij for any space-time dimension. A crucial requirement for the M5 theory amplitudes is that they reproduce the D3 amplitudes under dimensional reduction. The additional factor U (ρ, σ) will be determined by this requirement and 6D Lorentz invariance later in this section. The M5 analog of the D3 formula in eq. (4.10) is The bosonic delta functions ∆ B imply that the n particles are massless and allow us to introduce spinors λ A a andλ Aâ for all the momenta, each of which is unique up to little-group transformations, as explained in section 3.2. The fermionic delta functions ∆ F imply that λ Aâ q AI should vanish, which accounts for the delta functions given above. The vanishing ofλ Aâ q AI also implies that q AI can be expressed as q AI = λ A a η Ia due to eq. (2.18). On reduction to 4D these fermionic delta functions account for the fermionic delta functions that appear in eq. (4.10).
Also by analogy with the D3 theory, ρ A a (σ) and χ I a (σ) are bosonic and fermionic poly- and the measure dM for the M5 case is given by where d = n 2 − 1, just as in the D3 theory. The symmetry that needs to be gauge fixed is now G = SL(2, C) × SL(2, C). (5.7)

JHEP12(2017)003
The first SL(2, C) factor, which concerns the usual modular symmetry transformations of the σ coordinates, removes the integration over three σ i 's. This symmetry will be verified later. The second SL(2, C) factor, which is the complexification of the SU(2) little group of the M5 theory, removes three ρ integrations. The bosonic delta functions completely fix the integration variables, as in the 4D case, leaving a sum over the solutions of the scattering equations. Specifically, the 6n bosonic delta functions give rise to n on-shell conditions p 2 i = 0 and 6D momentum conservation leaving (5n − 6) bosonic delta functions. Since the σ i 's and ρ A m,a 's are constrained by G = SL(2, C) × SL(2, C), there are (n − 3) σ i 's and (4n − 3) ρ A m,a 's to be integrated, which is the right number to be fixed by the remaining (5n − 6) delta functions. The proof of momentum conservation and the scattering equations is essentially the same as described for the D3 theory in appendix A.1.
The gauge-fixing Jacobian for the first SL(2, C) factor is (σ 1 − σ 2 )(σ 2 − σ 3 )(σ 3 − σ 1 ) as usual. The one for the second SL(2, C) factor will be discussed later. These symmetries, as well as other properties, will be verified after we have made a specific proposal for U (ρ, σ). It will be determined by considering dimensional reduction to 4D, with the final result shown in eq. (5.31) or equivalently eq. (5.34).
In contrast to the 4D case, the polynomials ρ Aa (σ) and χ I a (σ) are required to have degree d = n 2 − 1 due to the SU(2) little-group symmetry. Thus, the solutions of the scattering equations, which are implied by ∆ B (p, ρ) = 0, cannot be subdivided into sectors. There is only one sector, which we find interestingly already contains all (n−3)! solutions of the scattering equations. (This assertion has been checked explicitly for n = 4, 6, 8.) When reduced to 4D massless kinematics and for the D3 theory, only a subset of the (n − 3)! solutions contributes, namely those N d,d helicity-conserving solutions.
We have checked explicitly that eq. (5.31) correctly reproduces the amplitudes with lower multiplicities, such as the four-particle amplitude that was discussed previously. As we discussed, to extract the amplitudes, one should take out the "wave functions" from ∆ B and ∆ F defined in eq. (5.2) and eq. (5.3) and one can further extract the momentum and (half of the) supercharge conservation delta functions, namely, The factors J 4,B and J 4,F are Jacobians, generated in this process, which can be found in appendix A. Finally, I 4 is an integral over the remaining delta functions, which are singled out to be special here. Performing the integral, 3 and using U (ρ, σ) given in eq. (5.29) and eq. (5.32) or equivalently eq. (5.35), we find that I 4 precisely cancels the Jacobian factors J 4,B J 4,F , leaving which is the result that was obtained in the previous section.
Higher-point amplitudes in the M5 theory have not appeared in the literature to our knowledge. However, amplitudes with scalars are constrained by soft theorems (as we will describe in a later subsection), and some of them are completely determined by recursion relations [39]. For instance, pure-scalar amplitudes are fixed in terms of the four-point ones.
We have tested numerically that eq. (5.31) indeed reproduces such amplitudes correctly for n = 6, 8. Those results, combined with supersymmetry and R symmetry, which we have explicitly checked for six and eight particles in appendix B.2, imply that eq. (5.31) should be valid for the entire supermultiplet for n = 6, 8. It seems very likely that they are correct for all n, as we find evidence supporting this in the following sections.

Reduction to four dimensions
This subsection will determine the constraint on U (ρ, σ) in eq. (5.31) that arises from requiring that its reduction to 4D cancels the Jacobian that is generated by the dimensional reduction of the M5 amplitude to 4D. So the key step is to evaluate the relevant Jacobian. What dimensional reduction does is to set two components of the six-component momenta equal to zero. In our conventions this means p 12 i → 0 and p 34 i → 0. This can be implemented by inserting dp 12 n dp 34 n n−1 i=1 dp 12 i dp 34 i δ(p 12 i )δ(p 34 i ) (5.12) into the formula for the n-particle amplitude of the M5 theory in eq. (5.31). Note that δ(p 12 i )δ(p 34 i ) is only inserted for n − 1 particles, even though the integration is over all n particles, because momentum conservation in 6D ensures that p 12 n = p 34 n = 0 as well, if p 12 i = p 34 i = 0 for i = 1, 2, . . . , n−1. Since dimensional reduction requires setting λ 1 i,− = λ 2 i,− = 0 and λ 3 i,+ = λ 4 i,+ = 0, therefore we should integrate out the corresponding ρ 1 m,− , ρ 2 m,− and ρ 3 m,+ , ρ 4 m,+ . Due to the fact that only the middle sector contributes to the scattering amplitudes of the D3 theory, we will focus on that sector only in the following computations. Explicitly, we have that the M5 amplitudes given in eq. (5.31) reduce to 4D amplitudes of JHEP12(2017)003 the form given in eq. (4.1), where the factor I is given by Vol(SL(2, C)) dp 12 n dp 34 n n−1 i=1 dp 12 i dp 34 The goal here is to determine the condition on U (ρ, σ) that will ensure that I R = I DBI = det S n . Here the SL(2, C) group is the one that acts on the little-group indices that will be reduced to U(1) after the dimensional reduction. The trivial n − 1 integrations over p 12 i and p 34 i give Vol(SL(2, C)) dp 12 n dp 34 14) The delta functions force ρ 1 a (σ i )ρ 2a (σ i ) and ρ 3 a (σ i )ρ 4a (σ i ) to vanish for i = 1, 2, · · · , n−1. However, ρ 1 a (σ)ρ 2a (σ) and ρ 3 a (σ)ρ 4a (σ) are polynomials of degree 2d = n−2 Because the degree of these polynomials is less than the n − 1 required roots, we conclude that all of the coefficients c 12 m and c 34 m should vanish. Since this also implies that ρ 1 a (σ n )ρ 2a (σ n ) = 0 and ρ 3 a (σ n )ρ 4a (σ n ) = 0, the integrations over p 12 n and p 34 n in eq. (5.14) are trivial.
The formula for I R now contains 2n − 2 delta functions, but there are 2n integrations, so we should use SL(2, C) to fix two of them. This leaves a U(1) unfixed, as expected. is the Vandermonde determinant. It arose from the following combination of factors: and There are no minus sign issues, since n is even. The factor J SL(2,C) is due to gauge-fixing the SL(2, C) symmetry of the complexified little-group symmetry. We have chosen to gauge fix ρ 2 d,− , ρ 4 d,+ and ρ 2 d,+ , and thus the Jacobian due to the gauge-fixing of the complexified SU(2) symmetry is given by The factor J C is the Jacobian that arises due to the change of variables from ρ coordinates to c coordinates. It contains a product of two resultants, and it is given by The resultant has appeared previously in a twistor-string-like formulation of scattering amplitudes in various theories [29,40], and include the D3 theory [12]. Its crucial property is that it vanishes if and only if the two polynomials ρ A a (σ) and ρ B a (σ) have a root in common.
A resultant of the form R(ρ A a , ρ B a ), where ρ A a and ρ B a are both polynomials of degree d, is given by the determinant of a Sylvester matrix M For instance, for n = 6 or d = 2, the Sylvester matrices are 4 × 4, To exhibit the residual U(1) little-group symmetry in 4D, we may set ρ 2 d,− = ρ 4 d,+ = 0 using partly the complexified 6D little-group symmetry SL(2, C). Now we see that all the factors are exactly canceled, except for ρ 4 d,− , which is precisely the Jacobian for the gaugefixing of the left-over U(1) symmetry of the 4D theory. Furthermore, the fermionic delta functions of (2, 0) supersymmetry also reduce to the 4D ones, without any complications. Thus, the proposed formula for the M5 amplitude in eq. (5.1) reduces to the D3 amplitude in eq. (4.10) under dimensional reduction provided that the factor U (ρ, σ) reduces according to

The extra factor U (ρ, σ)
To complete the construction of the M5 amplitudes, we need to determine the extra factor (relative to the D3 formula) U (ρ, σ). We have just learned what it should give when reduced to 4D. This goes a long way towards determining it. We claim that the σ and ρ dependence factorizes already in 6D, so that Note that V −2 has total symmetry in the n σ i 's. As will be verified later, V −2 transforms under SL(2, C) in the way required to compensate for the additional bosonic coordinates in the M5 theory. The factor R(ρ) should scale like p 2d or ρ 4d and on reduction to 4D it should give the product of resultants R(ρ 1 + , ρ 2 + )R(ρ 3 − , ρ 4 − ). This expression does not have 6D Lorentz invariance or little-group symmetry, so it must be embellished by additional pieces that vanish upon dimensional reduction.

JHEP12(2017)003
The crucial observation is that the product of resultants R(ρ 1 + , ρ 2 + )R(ρ 3 − , ρ 4 − ) can be expressed in terms of Pf S n and the Vandermonde determinant V (σ) [12], The above relation is valid for ρ and σ under the constraints of the helicity-conserving sector, eq. (4.17), which is the case here. As functions of s ij and σ i , now both Pf S n and V (σ) can be lifted to 6D straightforwardly without violating Lorentz invariance. This leads to our proposal for all tree-level scattering amplitudes of the M5 theory, which is the main result of the paper. This formula reduces to the D3 amplitude in eq. (4.10) correctly, and it also has many other correct properties that we will discuss shortly. Importantly, eq. (5.31) produces known amplitudes as we mentioned. Alternatively, one can use the definition of the resultant in terms of Sylvester matrix in eq. (5.26). With that, a different possible uplift to 6D is realized by a natural generalization of the resultant and Sylvester matrix. They are given by, where M (4d) is the following 4d × 4d matrix, a generalization of Sylvester matrix, The subscripts + and − are little-group indices, whereas SU(4) Lorentz indices, A = 1, 2, 3, 4, are shown in parentheses. The four submatrices in M (4d) are 2d × 2d matrices, which take the form of Sylvester matrices. Upon dimension reduction, the off-diagonal matrices of M (4d) vanish, and thus R(ρ) also has the required reduction to 4D. So in terms of R(ρ), the scattering amplitudes of M5 theory then take an alternative form, In fact, like the case of 4D where the resultant is related to Pf S n and the Vandermonde determinant V (σ), we find that, under the support of delta function constraint ∆ B , R(ρ) defined in eq. (5.32) is related to Pf S n and V (σ) in the same way, namely, Plugging this result into eq. (5.34) reproduces eq. (5.31). Therefore these two different approaches actually lead to the same result. Although the quantity R(ρ) can be re-expressed in terms of Pf S n and V (σ) on the support of delta-function constraints, it may still be of interest on its own right. Let us make a few comments on it here before closing this subsection. It is straightforward to show that JHEP12(2017)003 R(ρ) is invariant under little-group and Lorentz-group transformations, which together act on R(ρ) as SL(2, C) × SL(4, C). A natural generalization would be invariant under SL(k, C) × SL(2k, C), and it would relate 2k 2 polynomials of degree d, which transform as bifundamentals. The generalization to k > 2 may be relevant for scattering amplitudes of the D-brane theories in dimension greater than six. We will leave this for the future study. The usual resultant, which corresponds to k = 1, vanishes whenever the two polynomials have a common zero. It would be interesting to know the generalization of this statement when k > 1. In any case, these remarks suggest introducing the alternative notation R has 2kd rows and columns. However, we will not utilize that notation in this manuscript.

SL(2, C) modular symmetry
Let us examine whether eq. (5.31) has the correct SL(2, C) modular symmetry under the transformations of the form With this rescaling It is easy to see that the integration measures are also invariant under this transformation, since the Jacobian is the determinant of a triangular matrix with 1's on the diagonal.

Factorization
The formula for the amplitude A n in eq. (5.31) is an integral over sets of polynomials ρ A a (σ) and χ I a (σ) of degree d = n 2 − 1. To study the multi-particle factorization behavior of the amplitudes, one may take a limit on the moduli space such that the higher-degree polynomials degenerate into products of lower-degree ones [41][42][43]. Specifically, there is a "left" factor containing polynomials of degree d L = n L 2 − 1 and a "right" factor containing polynomials of degree d R = n R 2 − 1, where d L + d R = d or n L + n R = n + 2. To achieve this goal, we introduce a parameter s that approaches zero in the desired limit and perform the following rescaling of the ρ m 's 5 and

JHEP12(2017)003
where L or R denotes the set of particles on the left-or right-hand side of a factorization channel. We will show that the left-hand side of the factorization channel has polynomials of degree d L and the right-hand side has polynomials of degree d R . Accordingly, we rename the ρ's as either ρ L or ρ R . Note ρ d L appears on both sides, but we separate it into two coordinates by setting ρ d L = ρ L,0 and ρ d L = ρ R,0 , and introducing dρ L,0 δ(ρ L,0 − ρ R,0 ). Now for the σ i 's, we make the replacements In the limit s → 0, a degree-d polynomial degenerates into a product of degree d L or d R polynomials, depending on whether the particle is on the left-or the right-hand side, namely It is also straightforward to see that the delta functions reduce to the corresponding lowerpoint delta functions, namely, depending on whether σ i is on the left or the right. If i ∈ L, P L,i (σ) = (0 − σ i ) n L j =i (σ ji ), where "0" is the value of the σ coordinate associated to the internal line in the factorization, and similarly for i ∈ R.
It is important that the integrand and the integration measure factorize correctly, and this is straightforward to see for the measure. On the other hand, the building blocks of the integrand, the Vandermonde determinant V (σ) and Pf S n , have already appeared in literature in the construction of scattering amplitudes in other theories; they are also known to factorize correctly. Alternatively for the proposal eq. (5.34), we find the new mathematical object we constructed, R n (ρ), also factorizes properly in the s → 0 limit, Here the subscript of ρ m denotes the index m of ρ A,a m , and we have suppressed Lorentz and little-group indices A and a Finally, because P 2 L ∼ s 2 in the limit s → 0 at a factorization pole 1/P 2 L , the amplitude should go as ds 2 /s 2 [43]. By collecting all of the s factors arising from the integration measure and the various factors in the integrand, we have verified that this is indeed the case. Thus, the general formula eq. (5.31) has the required factorization properties for a tree-level scattering amplitude.

Soft theorems
As we discussed previously, the five scalars of the M5 theory are Goldstone bosons arising from spontaneous breaking of 11D Lorentz symmetry. More specifically, the relevant broken symmetries are translations in the five spatial directions that are orthogonal to the M5brane. Let us now study how the scattering amplitudes of the M5 theory behave in soft limits, i.e.,in the limit where the momentum p AB of a Goldstone boson vanishes. As shown in [44], amplitudes involving such scalars have enhanced soft behavior [45], specifically A(p 1 , · · · , p n−1 , τ p n ) ∼ O(τ 2 ) , (5.55) where particle n is a scalar, with momentum τ p n , and the soft limit is realized by τ → 0. Of course, some of the other momenta should also depend on τ , so as to maintain momentum conservation and masslessness. We claim that the amplitudes obtained from general formula in eq. (5.31) indeed have this enhanced soft behavior. In particular, if we rescale λ A,a n = τ 1/2 λ A,a n , so that the momentum p n is replaced by τ p n , we find that the various pieces that contribute to the amplitude scale as follows and the rest, including the Vandermonde determinant V (σ), scales as τ 0 in the soft limit. As discussed in appendix A.2, J B and J F are Jacobians that arise from extracting various "wave functions" and momentum-conservation delta functions, and from performing integrations over σ's, ρ's, and χ's. J F also depends on the fact that we are considering a scalar component of the supermultiplet. Altogether, we obtain the expected O(τ 2 ) behavior of the amplitudes in the M5 theory. Just for the comparison, in the case of the D3 theory, in the soft limit each piece in eq. (4.1) behaves as det S ∼ τ 2 , J B ∼ τ 0 , J F ∼ τ 0 . (5.57) In total, the amplitudes again scale correctly, namely as O(τ 2 ). We can also study how the amplitudes behave in the double-soft limit, where we let two momenta approach zero simultaneously, say, p n+1 → τ p n+1 and p n+2 → τ p n+2 with τ → 0. For simplicity, here we only consider the leading soft theorems. The result of the double-soft limit depends on the species of particles involved as shown here
The ellipsis denotes higher-order terms in the soft limit, and the lower-point amplitude A n is the amplitude with the two soft particles removed. In the case of soft theorems for B fields, on the right-hand side one should symmetrize the little-group indices a 1 , b 1 and a 2 , b 2 .
The double-soft theorems for the scalars and fermions agree with the known result [44] derived from the Ward identity for scalars that are Goldstone bosons of spontaneously broken higher-dimensional Lorentz symmetry, while the fermions are Goldstinos of broken supersymmetries. The double-soft theorems for B fields are new; it would be of interest to study the corresponding symmetries. If we choose both of the soft B fields to be B +− and reduce to 4D, we obtain the double-soft result for scalars as in the first line of eq. (5.58). If, instead, we take the two soft B fields to be B −− and B ++ , and reduce to 4D, we obtain the double-soft theorem for photons in Born-Infeld theory, namely which agrees with what was found in [11]. Similarly, the double-soft theorem for fermions reproduces that of Volkov-Akulov theory upon reduction to 4D [46]. To obtain these results we have applied the following identities for the dimensional reduction 6D → 4D, according to our convention,

Six-and eight-particle amplitudes of the M5 theory
As an application of the n-particle amplitude in eq. (5.31), this section presents analytic results for some specific amplitudes of the M5 theory, namely six-and eight-particle amplitudes of self-dual B fields. To our knowledge, these amplitudes have not been presented in the literature before. The use of spinor-helicity variables circumvents the usual difficulties associated to the lack of a manifestly covariant formulation of the M5-brane action. Still, it is not easy to directly compute any higher-point amplitudes analytically, especially due to the fact that the scattering-equation constraints are high-degree polynomial equations whose solutions are rather complicated. The approach that we have used to obtain analytic results is to write down an ansatz with unknown coefficients for the amplitude of interest, and then to fix the coefficients by comparing the ansatz with the result obtained from the general formula in eq. (5.31).
Let us begin with the six-particle amplitude of B ++ . Recall that the B particles form a triplet of the SU(2) helicity group.
where we have used the bracket notation defined in eq. (3.5). Using the results of eq. (5.61) and eq. (5.62), it is straightforward to write an ansatz that has the correct factorization properties, here P 6 means summing over all ten factorization channels (nine in addition to the one that is shown).
The ansatz in eq. (5.63) is the simplest guess that has the correct factorization and little-group properties, and it ends up being correct. It is instructive to see how one arrives at this conclusion using Feynman diagrams without recourse to an action. At the poles we can represent the six-point amplitude in eq. (5.63) as a sum of exchange diagrams that are the product of four-point amplitudes and an internal propagator. These diagrams are shown in figure 1.
In evaluating these diagrams, one must sum over all exchange channels as well as all fields allowed to propagate on the internal lines. As we have explained, only B ++ , B −− , or B +− = B −+ can be exchanged. The pure positive and negative helicity states are conjugates of each other, and as with chiral fermions we use an arrow to distinguish them from the neutral helicity.
The sum of such diagrams must be invariant under the little group of the internal particle, and this ends up being the case due to a subtlety in the spinor-helicity formalism. This "glitch" in the spinor-helicity formalism as discussed for 6d SYM in [33] is that the spinors cannot distinguish particles and antiparticles, which causes issues for diagrams with fermions. A new feature of 6d chiral self-dual tensors is that the tensor field itself has this issue with the B ++ and B −− polarizations. The resolution, as outlined in [33], is to add extra factors of i to the spinor-helicity variables when we flip the sign of the momentum for either of these fields: λ A a (−p) = iλ A a (p) (5.64) so that the momentum is properly Figure 1. Exchange diagrams contributing to the 6 B ++ amplitude. The internal line may be any of the three states, and we sum over all the factorization channels as well. It is important to note that these diagrams do not come directly from Feynman rules as there is no covariant action available for the M5 theory; instead, they represent the factorization of the amplitude at the poles where s ijk → 0. This introduces additional minus signs for a four-particle amplitude of the form Applying this recipe to the exchange diagrams of figure 1, one is led directly to eq. (5.63), which does not depend on the little-group structure of the internal line, as it should be.
Of course, eq. (5.63) might not be the final result, since it could differ from the correct answer by terms that have no poles (thought of as a 6-particle contact interaction, depicted in figure 2). The only local term allowed by power counting and little-group constraints is 1 + 2 + 3 + 4 + 1 + 2 + 5 + 6 + 3 + 4 + 5 + 6 + + P 6 . (5.67) It turns out that this local term vanishes identically after summing over the permutations. Thus, we claim that eq. (5.63) is the complete result for the amplitude of six B ++ 's. Indeed, we find perfect agreement by comparing eq. (5.63) numerically with the general integral formula eq. (5.31). One can perform a similar analysis for more general amplitudes of self-dual B fields. In all cases we find that the result takes a form similar to eq. (5.63), is the only amplitude that is non-vanishing after dimensional reduction to 4D. This is consistent with the claim that the amplitudes of the D3 theory are helicity conserving. The helicity-conserving amplitude obtained in this way is where the ellipsis in the parentheses denote summation over all factorization channels, as well as all other independent s ijk 's. It is straightforward to verify that this is the unique amplitude for identical scalars determined by the soft theorem. One can also consider amplitudes of other particles. For instance, we find that the six-particle amplitude of φ IJ in the spectrum eq. (2.12) agrees with the result in eq. (5.70). Also, the amplitude for six fermions can be expressed as where the factorization term A (6) f and the local term A (6) c are given by
Let us now consider the amplitudes with eight B particles. For simplicity, we only consider the amplitude with eight B ++ 's and the amplitude with seven B ++ 's and one B −− . As we will see, they take a very similar form. The strategy is the same as in the case of six-particle amplitudes. We write down an ansatz that includes factorization parts and local terms, and then compare the ansatz against the general formula to determine the unknown coefficients. As before, one can arrive at the ansatz for exchange diagrams by summing diagrams that are products of amplitudes with fewer particles. Unlike the case of six B particles, we find that in general there are contributions from local terms. Explicitly, we find where the little-group index a can be + or − depending on whether B aa is B ++ or B −− , and A (8) f , A (8) c are the factorization part and the local term, respectively. A (8) f and A (8) c are given by where P 8 denotes the summation over independent permutations. As mentioned previously, the amplitudes involving scalars in the M5 theory should satisfy soft theorems. Some such amplitudes are completely fixed by the soft theorems. Therefore they can also be computed in a completely different way via on-shell recursion relations [39]. We have verified that the results agree perfectly with what is obtained from the proposed formula, eq. (5.31), for such amplitudes containing up to eight particles.

n-particle amplitudes of the D5 theory
This section describes the tree-level S matrix for the theory of a single probe D5-brane with 6D N = (1, 1) supersymmetry. The general formula we propose for the D5 theory takes a form similar to that of the M5 theory, which we discussed in the previous section. In particular, the formula contains the same factors of det S n and U (ρ, σ), or equivalently The bosonic delta functions are the same as those in the M5 theory but now there are two kinds of fermionic delta functions due to (1, 1) supersymmetry, The measure is given by As before, d = n 2 − 1. Note that this integration measure does not include d 8ρ mAâ , even thoughρ mAâ do appear explicitly in the formula. The prescription is that theρ mAâ are fixed by the constraint of the conjugate of ∆ B in eq. (6.3), namely, This constraint does not appear explicitly in the general formula eq. (6.2), but we impose it implicitly. To fully fixρ mAâ , we also use the second SU(2) factor of the little-group symmetry to fix three of theρ mAâ coordinates. Since eq. (6.2) takes a form that is very similar to eq. (5.31) for the M5 theory, with a simple change to half of the fermionic delta functions due to the change of chirality for half of the supersymmetry, it is straightforward to show that eq. (6.2) also has all of the required properties, such as correct factorizations, soft theorems, and reduction to the D3 theory. Thus, we will not repeat the analysis and discussion here. For computing scattering amplitudes from eq. (6.2), as in the case of the D3 theory and M5 theory, we again should pull out the bosonic and fermionic "wave functions" first. For the D5 theory, they are given by We have checked explicitly that A n , as defined here, produces the correct fully supersymmetric four-particle amplitudes, as well as many examples of six-and eight-particle amplitudes in the D5 theory. Here we list the analytical results for some of these amplitudes. The amplitude for six photons with the same helicity, given by A 11 , is

JHEP12(2017)003
There are similar expressions for other choices of helicities of A aâ . We have verified that these results agree with the amplitudes obtained directly from the Born-Infeld action. One can also consider the amplitude of eight A 11 's, which takes the form The factorization term A f and the local term A c are given by These results for photon amplitudes in the D5 theory take a form that is very similar to the amplitudes of B ab particles in the M5 theory. They are related to each other by replacing the anti-chiralλâ by the chiral one λ a . The similarity between D5 and M5 amplitudes in the above explicit examples, and more generally the formulas eq. (6.2) and eq. (5.31), may be surprising, especially given the fact that the classical action for the M5 theory is more subtle to write down than the one for the D5 theory. However, one should note that the entire difference between the four-particle amplitudes, which are completely fixed by the symmetries and power counting in the D5 theory and the M5 theory, is just a simple modification of the fermionic delta functions. Since both theories reduce to the same 4D amplitudes, the similarity is really not so surprising. The complication of writing the classical M5 action caused by the selfduality of B field is avoided by considering only the on-shell degrees of freedom for the S matrix using the spinor-helicity formalism.

Conclusion
This paper has proposed general formulas for n-particle on-shell tree-level scattering amplitudes for three theories: the D3 and D5 theories of type IIB superstring theory and, especially, the M5 theory of 11D M-theory. The scattering amplitudes of the M5 theoryeven its bosonic truncation -have been studied little in the previous literature. In each of these theories n is required to be even, and the amplitudes take similar forms, expressed as integrals over rational constraints, built from degree d = n 2 − 1 polynomials. The integrand contains a new mathematical ingredient, a generalization of resultant (denoted R(ρ) in the text), which is equal to the product of Pf S n and the Vandermonde determinant V (σ) on the support of the rational constraints.
The three theories are related to one another in various ways. For instance, dimensional reduction of each of the 6D n-particle amplitudes, which pertain to the D5 and M5 theories, reduces to the same 4D n-particle amplitude, which pertains to the D3 theory. The function U (ρ, σ) in the 6D integrands cancels the Jacobian factors arising from the JHEP12(2017)003 dimensional reduction. As we explained, one consequence is that the R symmetry of the D3 theory is SU(4) × U(1). The U(1) factor implies that the D3 amplitudes are helicity conserving. Interestingly, the formulas for the M5 and D5 amplitudes only differ by a simple modification of the fermionic delta functions that accounts for the chirality difference between (2, 0) supersymmetry and (1, 1) supersymmetry.
We have also checked various general properties such as SL(2, C) modular symmetry, R symmetries, factorization properties, and soft limits. We have further tested the formulas by explicitly computing amplitudes that are fixed by the soft theorems, up to 8 particles. Using the general formulas, compact analytic expressions for six-and eight-particle amplitudes of self-dual B particles of the M5 theory for certain choices of the little-group indices were obtained.
Our formulas for scattering amplitudes are similar to those for the twistor-string formulation of 4D N = 4 super Yang-Mills amplitudes in Witten's twistor-string paper [25]. Those amplitudes, and their generalizations, see e.g. [49,50], are understood in terms of two-dimensional world-sheet twistor-string theories. It would be interesting to explore whether there exists a similar twistor-string theory for the M5 theory. Such an underlying theory ought to generate the M5 amplitudes in eq. (5.31) directly. The fact that a twistorstring-like formulation of the tree-level S-matrix of the M5 theory does exist already points to some deep structures of the theory.
Finally, we note that the rational constraints in 6D consist of a single sector of solutions to the scattering equations, which utilizes all (n − 3)! solutions of the arbitrary-dimensional scattering equations. We do not have a general proof of these assertions, but they have been checked explicitly for the cases n = 4, 6, 8. It would be nice to prove (or disprove) them and to understand better this general feature of the 6D rational constraints. Upon dimensional reduction to the D3 theory, many of these solutions vanish leaving only contributions from those that correspond to the middle (helicity conserving) sector in 4D. It would also be interesting to study the rational constraints in dimensions greater than six, such as 10D or 11D, and to apply them to the D9-brane theory, as well as the various gauge and gravity theories in those dimensions.

A.1 D3 theory
The goal here is to show that the n-particle amplitude A n in eq. (4.1) contains the delta functions exhibited in the formula as well as additional momentum-conservation and supercharge-conservation delta functions, which are included in A n . We also wish to compute the Jacobian J B that arises from extracting the momentum-conservation and mass-shell delta functions from the bosonic delta functions, appearing in the formula for the D3 n-particle amplitude A n . It is clear that these delta functions imply masslessness, since they constrain p αα i to take a factorized (rank one) form. It is less obvious that they imply momentum conservation. The delta functions imply that This will vanish provided that Recalling the definition P i (σ) = j =i σ ij , we note that Then, momentum conservation is a consequence of the following theorem: This is proved by noting that W m is a symmetric polynomial of the n σ variables whose degree does not exceed n − 2 in any of them. Therefore, it vanishes if there are n − 1 zeros

JHEP12(2017)003
in each of the coordinates. This is achieved if W m vanishes when any pair of variables are equal. For example, when σ 1 = σ 2 only V 1 and V 2 are nonvanishing. But then W m (σ) = σ m 1 (V 1 + V 2 ). This vanishes because V 1 + V 2 = 0 when σ 1 = σ 2 . This completes the proof of momentum conservation.
We have seen that n + 4 of the 4n delta functions in ∆ B account for the mass-shell conditions and momentum conservation. The integrations over the ρ andρ coordinates use up 2n − 1 more of the delta functions, leaving n − 3 to account for. The important fact is that the remaining delta functions lead to the scattering equations E i = j =i p i · p j σ ij = 0, i = 1, 2, . . . , n (A.8) and the n − 3 integrations over the σ coordinates imply that one should sum over the solutions of these equations. Only n−3 of the scattering equations are linearly independent, since the mass-shell and momentum-conservation conditions imply that Thus, there is just the right number of delta functions to account for the scattering equations. As discussed earlier, the scattering equations have (n − 3)! solutions, but only N dd of them give nonzero contributions to the amplitudes. These are the ones that are helicity conserving, as required by the U(1) R symmetry.
Let us now verify that the delta functions in ∆ B actually do imply the scattering equations. Substituting for p i · p j gives However, ρ m ρ n = − ρ n ρ m and ρ m ρ n = − ρ n ρ m . Therefore we can replace σ m i σ n j by 1 2 (σ m i σ n j − σ m j σ n i ) = σ ij Q mn (σ i , σ j ) (A.11) where Q mn is a polynomial. It then follows that where Q(σ i , σ j ) = mnm n ρ m ρ n [ρ m ρ n ]Q mn (σ i , σ j )Q m n (σ i , σ j ). (A.13) Since σ ij Q(σ i , σ j ) is a polynomial function of σ j of degree n − 3, the scattering equations E i = 0 follow as a consequence of eq. (A.4). The structure of the 4n delta functions in ∆ B ensures masslessness, momentum conservation, and the scattering equations, which is a total of 2n + 1 conditions. They can be JHEP12(2017)003 expressed as delta functions and used to rewrite ∆ B as these 2n + 1 delta functions times 2n − 1 additional delta functions and a Jacobian factor, which will be described later. Given this, it is natural to examine next what can be learned from the structure of the 8n fermionic delta functions . (A.14) First of all, the delta functions in ∆ F imply the conservation of eight supercharges: This is proved by exactly the same reasoning that was used to establish momentum conservation earlier in this appendix. Note that these eight supercharges are mutually anticommuting, as are the other eight, but there are nonzero anticommutators between the two sets. The conservation of the second set of eight supercharges needs to be established separately.
Next we wish to account for the factors i δ 2 ( λ i q I i )δ 2 ([λ iqÎ i ]) in eq. (A.1). The first set should derive from the first set of delta functions in ∆ F and the second set from the second factor (by identical reasoning). It is important that the bosonic analysis has already been completed, so that masslessness, i.e.,the presence of the factors i δ(p 2 i ), can be invoked to justify writing p αα i = λ α iλα i . Therefore the fermionic delta functions imply that λ i q I i = [λ iqÎ i ] = 0. These relations are implemented by the 4n fermionic delta functions exhibited in eq. (A.1). They provide the justification for using the relations and similarly for the second set of fermionic delta functions.

A.2 M5 theory
Let us now consider the 6D formula for the M5-theory amplitudes. Beginning with the bosonic delta functions, we can extract the mass-shell and momentum-conservation delta functions as follows Next, we proceed similarly for the fermionic delta functions. Extracting the fermionic "wave functions" and supercharge conservation from the fermionic delta functions gives with the Jacobian the case of the four-particle amplitude, the full SU(4) symmetry can be made manifest by performing an appropriate Grassmann Fourier transform. For this purpose, it is useful to first recast the fermionic delta functions as follows and similarly for theq andλ sector. Now let us consider the Grassmann Fourier transformation (B.2) where we have Fourier transformedηÎ i and defined .

(B.3)
Since the bosonic delta functions (not displayed in this appendix) imply that we have t iti = 1/P i (σ). which is now manifestly SU(4) invariant.

B.2 M5 theory
Next we wish to verify the USp(4) R symmetry of the M5 theory. As in the case of 4D, it is useful to begin by decomposing the supercharge-conservation delta functions as follows d m=0 where d = n 2 − 1 and the Jacobian is given by (B.11) Again the choice of singling out Lorentz indices 1, 3 is arbitrary. Ignore all the Jacobi, which are not relevant to the R symmetry, the integration over χ's in the second line of eq. (B.10) reduces to