The set of separable states has no finite semidefinite representation except in dimension $3\times 2$

Given integers n $\geq$ m, let Sep(n,m) be the set of separable states on the Hilbert space $\mathbb{C}^n \otimes \mathbb{C}^m$. It is well-known that for (n,m)=(3,2) the set of separable states has a simple description using semidefinite programming: it is given by the set of states that have a positive partial transpose. In this paper we show that for larger values of n and m the set Sep(n,m) has no semidefinite programming description of finite size. As Sep(n,m) is a semialgebraic set this provides a new counterexample to the Helton-Nie conjecture, which was recently disproved by Scheiderer in a breakthrough result. Compared to Scheiderer's approach, our proof is elementary and relies only on basic results about semialgebraic sets and functions.


Introduction
Entanglement is a fundamental aspect of quantum mechanics. The set of separable states (i.e., nonentangled states) on the Hilbert space C n ⊗ C m is defined as: Sep(n, m) = conv xx † ⊗ yy † : x ∈ C n , |x| = 1, y ∈ C m , |y| = 1 .
Here x † =x T indicates conjugate transpose, |x| 2 = x † x = n i=1 |x i | 2 and conv denotes the convex hull. The set Sep(n, m) lives in the space H nm of Hermitian matrices of size nm × nm, and it is full-dimensional in the subspace of matrices of trace equal to one.
A fundamental computational task in quantum information is to decide membership in the convex set Sep(n, m). One of the first tests designed to check whether a state ρ ∈ H nm is separable is the Peres-Horodecki criterion [Per96,HHH96] (also known as the Positive Partial Transpose (PPT) criterion). It is based on the observation that for any ρ ∈ Sep(n, m), (I ⊗ T)(ρ) is positive semidefinite where I is the identity map, and T the transpose map. Indeed one can easily verify that if ρ = xx † ⊗ yy † then (I ⊗ T)(ρ) = xx † ⊗ (yy † ) T = xx † ⊗ȳȳ † ≥ 0. In other words we have the inclusion Sep(n, m) ⊆ PPT(n, m) where PPT(n, m) = ρ ∈ H nm : ρ ≥ 0, (I ⊗ T)(ρ) ≥ 0, and Tr[ρ] = 1 . (1) It is known, from earlier work of Woronowicz [Wor76], that we have equality Sep(n, m) = PPT(n, m) if, and only if n+m ≤ 5. Thus, the smallest cases where Sep(n, m) = PPT(n, m) are (n, m) = (4, 2) and (n, m) = (3, 3).
Semidefinite programming The description of the set PPT(n, m) in Equation (1) allows us to decide membership, and optimize linear functions on PPT(n, m), via semidefinite programming. Semidefinite programming is a fundamental tool in optimization that has played a crucial role in recent developments in quantum information theory. We say that a convex set C has a semidefinite representation (also called a semidefinite lift) of size r if it can be expressed as where π : R D → R d is a linear map and S ⊂ R D is a convex set defined using a linear matrix inequality where M 0 , . . . , M D are Hermitian matrices of size r × r. A set S of the form (3) is known as a spectrahedron. In this paper we are most interested in when a semidefinite representation of finite size exists, and call this simply a semidefinite representation throughout. If a convex set C admits a semidefinite representation, then optimizing a linear function on C can be cast as a semidefinite program. Equation (1) gives a semidefinite representation of PPT(n, m).
Horodecki's criterion The set of separable states has the following well-known description due to the Horodeckis [HHH96]: Here M k = C k×k and a C-linear map Φ : M m → M n is positive if it is Hermitian preserving and if Φ(X) ≥ 0 for all X ≥ 0. When n = m, the relaxation PPT(n, m) corresponds to having only the identity and transpose maps in Equation (4), which are both positive. A recent result of Skowronek [Sko16] shows that when n = m = 3, there is no finite family of positive maps Φ 1 , . . . , Φ k : M 3 → M 3 such that Sep(3, 3) = ρ ∈ H 9 : (I ⊗ Φ i )(ρ) ≥ 0 ∀i = 1, . . . , k . Note that the right-hand side of the previous equation is a specific semidefinite representable set. Thus Skowronek's result rules out certain specific semidefinite representations for Sep(3, 3). 1 DPS hierarchy In [DPS04], Doherty, Parrilo and Spedalieri proposed a complete hierarchy of approximations to the set of separable states based on semidefinite programming. The first level of the hierarchy coincides with the PPT test, and subsequent levels form tighter and tighter convex relaxations of the set of separable states. If we denote the convex relaxation at level k by DPS k (n, m) we have (dropping the (n, m)): The key property of the DPS hierarchy is that each set DPS k has a semidefinite representation of size min(n, m) O(k) . The hierarchy is known to be complete, meaning that if ρ / ∈ Sep, then there exists a finite k such that ρ / ∈ DPS k . The integer k however depends on the state ρ and it is known that, unless n + m ≤ 5, there is no finite k such that DPS k (n, m) = Sep(n, m) [DPS04, Section VIII.B].
Contributions The main result of this paper is Theorem 1. If Sep(n, m) = PPT(n, m) then Sep(n, m) has no (finite) semidefinite representation. In other words, Sep(n, m) has no semidefinite representation when n + m > 5.
Remark 1. Some remarks concerning Theorem 1: • Note that Theorem 1 contains as a special case the fact that whenever Sep(n, m) = PPT(n, m), then there is no representation of Sep(n, m) as {ρ ∈ H nm : Tr[ρ] = 1 and (I ⊗Φ i )(ρ) ≥ 0, ∀i = 1, . . . , k} where Φ 1 , . . . , Φ k : M m → M n is a finite family of positive maps. This is because the latter set is semidefinite representable (in fact it is a spectrahedron). Let us mention that if one is only interested in approximating the set Sep(n, m), Aubrun and Szarek [AS17b] gave a lower bound on the number k of positive maps needed.
• Our result also includes as a special case the fact that there is no finite k such that DPS k (n, m) = Sep(n, m), when n + m > 5. We note that our result is a strict generalization of this fact. Indeed, the failure of the DPS hierarchy to converge in a finite number of levels does not preclude by itself the existence of another semidefinite program that represents Sep(n, m) exactly. There are well-known examples of convex sets where the sum-of-squares hierarchy (of which DPS can be seen as a particular instance) is never exact and yet a finite semidefinite representation does exist, see e.g., [NPS10,Example 3.7].
• Observe that if Sep(n, m) has no semidefinite representation, then the same is true for Sep(N, m) for N ≥ n. This is because Sep(n, m) can be realized as a linear section of Sep(N, m) as follows: where Tr 2 ρ is the result of tracing out the second subsystem from ρ. Indeed, setting (Tr 2 ρ) ii = 0 implies that in any representation of ρ as ρ = k p k x k x † k ⊗ y k y † k , the vectors x k must satisfy (x k ) i = 0 for all i = n + 1, . . . , N , i.e., that x k ∈ C n × {0} N −n ≃ C n . To prove our theorem it thus suffices to prove that Sep(3, 3) and Sep(4, 2) have no semidefinite representations.
Helton-Nie conjecture The question of finding semidefinite representations for convex sets has attracted a lot of attention in the optimization community [Nem06,GPT13]. Helton and Nie [HN09] gave sufficient conditions for a set to have a semidefinite representation, and conjectured that any convex semialgebraic set has a semidefinite representation. (A set is semialgebraic set if it can be described using a finite boolean combinations of polynomial equations and inequalities. One can verify that Sep(n, m) is a semialgebraic set, see Section 2.3.) In his breakthrough paper, Scheiderer [Sch18] disproved this conjecture and exhibited convex semialgebraic sets that have no semidefinite representations.
Our proof of Theorem 1 is inspired from the arguments of Scheiderer. Compared to the paper of Scheiderer the present paper has two main contributions. First, the proof we give simplifies the arguments of Scheiderer and does not rely on any specialized results from algebraic geometry. We only use basic results from analysis (Taylor expansions), and some standard facts about semialgebraic sets and functions which are elementary to state. The proof should thus be accessible to readers in quantum information and optimization. The second contribution is the application of the method of proof for Sep(n, m) which is defined in terms of complex numbers. This turns out to cause certain difficulties as certain standard facts about real polynomials are not true about Hermitian polynomials, particularly on the relation between homogeneous polynomials and their dehomogenizations (see Section 5 and Appendix A for more details).

Main technical result
Our main technical result, Theorem 2 below and of which Theorem 1 is a corollary, gives a general way to construct a convex set with no semidefinite representation from a nonnegative Hermitian polynomial that is not a sum of squares. We recall that a Hermitian polynomial p(z) is a polynomial with complex coefficients in the indeterminates (z,z) = (z 1 , . . . , z n ,z 1 , . . . ,z n ) such that p(z) ∈ R for all z ∈ C n . A Hermitian polynomial is a sum of squares if it can be written as a sum of squares of Hermitian polynomials. (More details about Hermitian polynomials are given in Section 2.) For the statement of the theorem, we use the monomial notation z u = n i=1 z u i i for u ∈ N n .
Theorem 2 (General theorem). Let p(z) = (u,v)∈A p uv z uzv be a Hermitian polynomial supported on A ⊂ N n × N n , and assume that p is nonnegative on C n but not a sum of squares. Assume Then the convex set is not semidefinite representable, where cl denotes topological closure.
The set of separable states is of the form (5) for well-chosen set A. Indeed, dropping the normalization condition and letting SEP(n, m) be the convex cone of separable states, we have: The attentive reader will notice that this set A is not downward closed, and so does not satisfy the condition of Theorem 2. As a matter of fact, to prove Theorem 1 we apply Theorem 2 with a dehomogenization of A which satisfies the downward closed condition, and then homogenize back to get the desired convex cone. The details are explained in Section 5.
Overview of proof We briefly sketch the main ideas for the proof of Theorem 2.
• We first show that if the set C A has a semidefinite representation, then there exists a finite number of functions f 1 , . . . , f r : R 2n ≃ C n → R such that any nonnegative Hermitian polynomial supported on A∪{(0, 0)} can be written as a sum of squares from span R (f 1 , . . . , f r ). This characterization of semidefinite representations via sums of squares is not new: it follows from the factorization theorem of Gouveia, Parrilo and Thomas [GPT13] and its sum-of-squares interpretation see e.g., [Faw16]. We note that a similar characterization is also used in Scheiderer's paper, see [Sch18,Theorem 3.4].
• One of the main observations needed to prove Theorem 2 is to note that the functions f 1 , . . . , f r can be chosen to be semialgebraic. (We recall the precise definition of semialgebraic functions in Section 2.) One key property of such functions that turns out to be particularly important is that they are smooth almost everywhere. Combining this property with a simple observation regarding smooth sum of squares decompositions of homogeneous polynomials allows us to prove Theorem 2 already in the special case where p is a homogeneous polynomial. This allows us to prove that Sep(n, m) is not semidefinite representable when (n, m) = (5, 3) or (4, 4). The complete proof of Theorem 2 which allows us to cover the cases (n, m) = (4, 2) and (3, 3) for separable states, requires an additional technical argument using Puiseux expansions for univariate continuous semialgebraic functions.
Real version of Theorem 2 We note that one can state an analogue of Theorem 2 dealing with real polynomials instead of Hermitian polynomials. The proof is similar, and we state it below just for convenience and for future reference.
Theorem 3 (Main theorem for real polynomials).
where A ⊂ N n finite, be a real polynomial that is nonnegative on R n but not a sum of squares. Assume furthermore that is not semidefinite representable.
The theorem above can be used to recover the result of Scheiderer [Sch18, Corollary 4.25], that the cone P n,2d of nonnegative (real) forms in n variables of degree 2d is not semidefinite representable when it is distinct from Σ n,2d , the cone of sums of squares. Indeed, it suffices to take p in Theorem 3 to be a dehomogenization of a nonnegative form that is not a sum of squares, and to use the well-known fact that a convex set has a semidefinite representation if and only if its dual has one.
Organization The paper is organized as follows. In Section 2 we set some of the notations and present some background material on Hermitian polynomials, sums of squares, and semialgebraic sets and functions that are useful for the proof of the main theorem. In Section 3 we review the connection between the existence of semidefinite programming representations, and sums of squares. The proof of Theorem 2 is in Section 4 and the proof of Theorem 1 in Section 5.

Preliminaries
We recall in this section some results on Hermitian polynomials, the duality Sep/nonnegative polynomials and PPT/sums of squares and semialgebraic sets and functions.

Hermitian polynomials
For z ∈ C n , we denote the elementwise complex conjugate of z byz = (z 1 , . . . ,z n ). If u ∈ N n we define the monomial z u = z u 1 1 . . . z un n . A Hermitian polynomial p(z) is a polynomial in z andz of the form such that p(z) ∈ R for all z ∈ C n . This is equivalent to saying that p uv = p vu for all u, v. The support of p is supp(p) = {(u, v) : p uv = 0} ⊂ N n × N n . The Hermitian polynomial p is nonnegative if p(z) ≥ 0 for all z ∈ C n . Further, we say that p is a sum of squares if we can write for Hermitian polynomials q k . If p(z) is a Hermitian polynomial we will often consider the real polynomial P (a, b) = p(a + ib) in R[a 1 , . . . , a n , b 1 , . . . , b n ]. One can check that p is a sum-of-squares if and only if P is a sum-of-squares of real polynomials.
Remark 2 (Sums of squares for Hermitian polynomials). Another common definition of a Hermitian polynomial p(z) being a sum-of-squares is that p can be written as p(z) = k |g k (z)| 2 where g k are (holomorphic) polynomials in z only (and not inz). Clearly if p has such a representation then it is a sum-of-squares in the sense (8) since then p = k Re[g k ] 2 + Im[g k ] 2 and Re[g k ] and Im[g k ] are both Hermitian polynomials. The converse however is not true. It is possible that a polynomial p has a representation (8) and cannot be written as a sum of modulus squares of holomorphic polynomial mappings. See e.g., [DP09] for more on this distinction. In this paper we only work with the definition (8) of sums of squares.

Sep, PPT, nonnegative polynomials, and sums of squares
For convenience, we will work in this paper with the cone of separable states, where we drop the normalization condition: One can verify that Sep(n, m) is the compact slice Sep(n, m) = SEP(n, m) ∩ {ρ : Tr ρ = 1}. 2 Let also PPT be the cone of states that have positive partial transpose, i.e., It is well-known that the dual of SEP(n, m) can be identified, via the Choi isomorphism, with the cone of positive maps M n → M m (see e.g., [AS17a, Table 2.2]). Equivalently, the dual of SEP(n, m) can be identified with nonnegative degree-four Hermitian polynomials of the form where p ijkl = Φ(E ij ) lk . Polynomials of the form (9) have a biquadratic structure: they are quadratic independently in each block of variables x and y. The duality between SEP and nonnegative polynomials of the form (9) is in fact immediate from the definition of SEP.
Dual of PPT Using the identification above, it turns out that the dual of PPT (n, m) corresponds to polynomials p(x, y) that are sums of squares. Indeed, it is well-known (see again [AS17a, Table  2.2]) that the dual cone of PPT (n, m) can be identified, via the Choi isomorphism, with the cone of maps Φ : M n → M m that are decomposable, i.e., that can be written Φ = S 1 + S 2 • T where S 1 and S 2 are two completely positive maps, and T is the transpose map. Recall that a map S : M n → M m 2 Indeed if ρ = i p k x k x † k ⊗ y k y † k with Tr ρ = 1 and p k ≥ 0, then by redefining p k ← p k |x k | 2 |y k | 2 we can assume without loss of generality that |x k | = |y k | = 1. Taking the trace on both sides of ρ = i p k x k x † k ⊗ y k y † k tells us that 1 = k p k since Tr ρ = 1, i.e., ρ ∈ Sep(n, m). is completely positive if there exist matrices V t such that S(X) = t V * t XV t . One can verify that a map Φ is decomposable if and only if, the associated Hermitian polynomial (9) is a sum of squares. We did not find any reference for this equivalence, so we include a proof here. (The proofs we found in the literature are only for the direction ⇒ in Proposition 1. The proof of Proposition 1 is a special case of a more general result in [FF19], joint with Kun Fang, where it is shown that the dual of DPS k can be identified with a sum-of-squares condition of degree k.) Proposition 1. A map Φ : M n → M m is decomposable if, and only if, the Hermitian polynomial p(x, y) = y † Φ xx † y is a sum of squares.
is also a sum-of-squares. It follows that if Φ is decomposable then p(x, y) = y † Φ xx † y is a sum-of-squares.
We now prove the converse. Assume p(x, y) = y † Φ(xx † )y is a sum-of-squares, i.e., p(x, y) = t q t (x, y) 2 for some Hermitian polynomials q t . We need to show that Φ is decomposable. Since the coefficient of the monomial x 2 ix i 2 in p is 0, we see that q t cannot have monomials x 2 i ,x i 2 or x ixi . To be sure, let α t ,ᾱ t , β t be the coefficients in q t of these monomials (note that β t ∈ R since Similarly, by looking at the coefficient of x ixi x jxj in p, we see that the q t cannot have monomials of the form x i x j ,x ixj , x ixj orx i x j . The same of course is true for the y's. Thus this means that q t must have the form where M ∈ C n×n and N ∈ C m×m . Squaring q t we get When summing k q 2 t we see that the only terms that can produce monomials of the form x ixj y kȳl (the monomials that appear in p) are the terms 2|g t | 2 and 2|h t | 2 . The sum of all the other terms must thus be equal to 0. At the end we get (including the constant 2 in M t and N t ): The following diagram summarizes the discussion above.

Semialgebraic sets and functions
Semialgebraic sets A semialgebraic subset of R n is a subset that can be defined by a finite boolean combination of polynomial equations (P = 0) and inequalities (P > 0) where P ∈ R[x 1 , . . . , x n ]. For example a set of the form {w ∈ R D : M 0 + w 1 M 1 + · · · + w D M D ≥ 0} is semialgebraic since the condition that a matrix is positive semidefinite can be expressed by a finite number of polynomial inequalities. The set of separable states can be shown to be semialgebraic. One can prove this using the celebrated and powerful result of Tarski stating that the projection of a semialgebraic set is semialgebraic. A consequence of Tarski's theorem is that the convex hull of a semialgebraic set S ⊂ R n is semialgebraic. Indeed this is because we can write conv(S) as the projection on the x component of the following semialgebraic set (x, λ, s 1 , . . . , s n+1 ) ∈ R n × R n+1 × (R n ) n+1 : (Note that, by Carathéodory theorem any element in conv(S) is a convex combination of at most n + 1 points in S.) To see why the set Sep(n, m) is a semialgebraic subset of H nm ≃ R 2(nm) 2 first note that the following set (ρ, x, y) ∈ H nm × C n × C m s.t. ρ = xx † ⊗ yy † and |x| 2 = |y| 2 = 1 (10) is a semalgebraic subset of H nm × C n × C m ≃ R 2(nm) 2 × R 2n × R 2m since the equations can all be written as real polynomial equations in the real and imaginary components. By Tarski's theorem it follows that the projection of (10) on the H nm component, which is precisely the set of pure product states, is semialgebraic. Thus Sep is semialgebraic as the convex hull of a semialgebraic set.
Semialgebraic functions A function f : R n → R m is called semialgebraic if its graph {(x, f (x)) : x ∈ R n } ⊆ R n × R m is a semialgebraic set. Even though semialgebraic functions form a very broad class of functions, they are tame and possess nice regularity properties. Examples of semialgebraic functions are polynomials, rational functions, or power functions (with rational exponent). Functions that are not semialgebraic are e.g., exp(x), or the indicator function of the rationals in R. We state two basic results about semialgebraic functions that will be crucial for us. Clearly Theorem 4 is not true for general functions, cf. the indicator function of the rationals in R. The second result that we will need concerns semialgebraic functions in one variable.
Theorem 5 (Puiseux expansion for one-dimensional semialgebraic functions). Assume f : (0, η) → R where η > 0, is a semialgebraic continuous function that is bounded. Then f can be extended by continuity to the interval [0, η). Additionally, there exists an integer m such that the map t → f (t m ) is C ∞ on [0, ǫ) for some 0 < ǫ < η. We note that the theorem above is not true for arbitrary functions. For example the function x → sin(1/x) is bounded and continuous on any interval (0, η) but cannot be extended by continuity at 0. We finally record the following result which will also be needed for our proof. It simply says that any linear map, restricted to a semialgebraic set always admits a semialgebraic inverse.
Theorem 6. Let S ⊂ R N be a semialgebraic set and let π : R N → R n be a linear map. Then there exists a semialgebraic function F : π(S) → S that satisfies π(F (x)) = x for all x ∈ π(S).

Semidefinite programming lifts
We are now ready to start the proof of Theorem 2 (and thus of Theorem 1 too). The first thing we need is a necessary condition for the existence of a semidefinite representation for a given convex set C. The condition we state in Theorem 7 below is very similar to [GPT13, Theorem 1] 3 . Recall that, for A ⊂ N n × N n we denote by m A (z) the monomial map: We also denote by cl S the (topological) closure of a set S.
has a semidefinite representation of size k. Then there exists 2k 2 + 1 semialgebraic functions f j : C n → R (j = 1, . . . , 2k 2 + 1) such that any nonnegative Hermitian polynomial p supported on A ∪ {(0, 0)} is a sum-of-squares from V = span R (f 1 , . . . , f 2k 2 +1 ), i.e., p = j h 2 j for some h j ∈ V . Furthermore, the magnitude of the coefficients expressing the h j in terms of the basis (f 1 , . . . , f 2k 2 +1 ) are all bounded by φ( p ) where p is the largest magnitude of the coefficients of p, and φ is some polynomial that only depends on the semidefinite representation of C A .
The main difference between the statement above and the one in [GPT13] (see also [Faw16, Theorem 5, Chapter 2]) is that here the functions f 1 , . . . , f 2k 2 +1 are semialgebraic. (We say that a function f : C n → R is semialgebraic if the function F : R n × R n → R defined by F (a, b) = f (a + ib) is semialgebraic.) This observation will be crucial to us. We note that a statement similar to the theorem above appears as Theorem 3.4 in [Sch18]. Instead of working with semialgebraic functions, Scheiderer works with polynomial functions on an algebraic variety X.
Proof. Assume that C A = π(S) where S is a spectrahedron defined using a linear matrix inequality of size k × k: We can assume without loss generality that S has nonempty interior in R N . This in turn implies, using standard results about spectrahedra, that there exists w ∈ S such that M ( w) is positive definite (possibly after changing M ), see e.g., [RG95,Section 2.4].
For any z ∈ C n there exists w(z) ∈ S such that π(w(z)) = m A (z). Since M (w(z)) ≥ 0 we can find F (z) ∈ C k×k such that M (w(z)) = F (z)F (z) † . Furthermore, by Theorem 6, the function z ∈ C n → F (z) ∈ C k×k can be taken to be semialgebraic.
Let p(z) be a Hermitian polynomial supported on A ∪ {(0, 0)}, i.e., p(z) = p, m A (z) + c for some c ∈ R, where p denotes the coefficients of the polynomial p(z) in the monomial basis. Since p ≥ 0 we get p, m A (z) + c ≥ 0 for all z ∈ C n . This implies that p, σ + c ≥ 0 for all σ ∈ C A . We can lift this linear inequality to an inequality on the spectrahedron S, i.e., we have p, π(w) + c ≥ 0 for all w ∈ S, in other words By Farkas' lemma/duality for SDPs, this means that there exists B ≥ 0 and b ≥ 0 such that Plugging w = w(z) we get Since π(w(z)) = m A (z) and p, m A (z) = p(z) we get finally that p(z) If we define the 2k 2 + 1 semialgebraic functions to be the constant function and the z → Re[F ij (z)] and z → Im[F ij (z)], we get the desired claim. For the last statement of the theorem, we show that the coefficients of B (and thus of D) are bounded by a polynomial in the coefficients of p. To get this, we can simply plug the value w = w that makes M (w) positive definite in (11). If we denote by λ > 0 the smallest eigenvalue of M ( w) we get π * ( p), w + c = B, M ( w) + b ≥ λ Tr[B] + b ≥ λ B + b, thus max( B , b) ≤ ( π * ( p), w + c)/ min(λ, 1) ≤ O(max{ p , |c|}).

Proof of Theorem 2
We are now ready to prove our main theorem, Theorem 2. We first recall a piece of notation that we will use throughout the proof: for any function f : C n → R we associate the function of real variables F : R n × R n → R, denoted by a capital letter, defined by F (a, b) = f (a + ib). We will also sometimes think of a vector z ∈ C n as z ∈ R 2n and write for instance F (z).
Assume that C A has an SDP representation. Then, from Theorem 7 there exist semialgebraic functions f 1 , . . . , f r : C n → R such that the following is true: Any nonnegative Hermitian polynomial supported on A ∪ {(0, 0)} is a sum-of-squares of functions from span R (f 1 , . . . , f r ). ( * ) We will now prove that the functions f i can be taken to be smooth at the origin. (By this we mean that the associated functions F i : R n × R n → R are smooth at (0, 0).) This will follow from our assumption that A is downward closed. Since the F i are semialgebraic we know from Theorem 4 that each F i is smooth almost everywhere. Thus we can find a common point z 0 = (a 0 , b 0 ) ∈ R n ×R n such that all functions F i are smooth at z 0 . Now let f i (z) = f i (z − z 0 ) for all i ∈ {1, . . . , r}. We claim that these semialgebraic functions still satisfy the property ( * ). Indeed if q is a nonnegative Hermitian polynomial supported on A ∪ {(0, 0)} then q(z + z 0 ) is nonnegative and is also supported on A, since A is downward closed. It follows that q(z+z 0 ) is a sum-of-squares from span R (f 1 , . . . , f r ). But this implies that q(z) is a sum-of-squares from span R ( f 1 , . . . , f r ).
In the rest of the proof we will thus assume that the F 1 , . . . , F r are smooth at the origin. If p is homogeneous we are almost done by the following simple observation: if P is a real homogeneous polynomial and P = j H 2 j for some functions H j : R m → R that are smooth at the origin, then P is a sum-of-squares of polynomials. This can be proved by a simple Taylor expansion; for example by applying the following proposition to the identity t 2k P (z) = P (tz) = j H j (tz) 2 and observing is a degree k polynomial in z ∈ R 2n (it is the k'th term in the Taylor expansion of H j at z).
Proposition 2. Assume that g j : [0, η) → R are smooth functions 4 and that there exists a ∈ R such that at 2k = j g j (t) 2 for all t ∈ [0, η). Then a = j g (k) Proof. If we Taylor expand the g j at 0 we get at 2k = j (g j (0) + tg ′ j (0) + · · · + t k g (k) j (0)/k! + o(t k )) 2 . By equating powers of t we get that g j (0) = · · · = g (k−1) j (0) = 0 and that a = j (g (k) The case where p is not necessarily homogeneous requires an additional argument. The following argument is inspired from [Sch18,Proposition 4.18]. Let 2d = deg p, and for any t ∈ R consider the Hermitian polynomial p t (z) = t 2d p(z/t). This Hermitian polynomial is nonnegative and is also supported on A. Thus we know from property ( * ) that there exist real coefficients a j (t) ∈ R r s.t.
where we let F (z) = (F 1 (z), . . . , F r (z)). The functions a j (t) are defined by a semialgebraic relation and so can be taken to be semialgebraic. As such the a j must be continuous on some (0, η) for η > 0. From the last part of the statement of Theorem 7 we also know that the a j must be bounded on (0, η). Thus, by Theorem 5 we know that the a j can be extended by continuity to [0, η) and that for large enough m, a j (t m ) is smooth on [0, η ′ ) for some 0 < η ′ < η. From (12) we get: But note that P t m (t m z) = t 2dm P (z). Thus If we let g j (t) = a j (t m ) T F (t m z) we know that the g j are smooth on [0, η ′ ), since the F are smooth at the origin. We can apply the observation of Proposition 2 to get that Smoothness at 0 is smoothness on the right But, from the definition of g j , g (dm) j (0) is a polynomial (of degree d) in z. This contradicts the assumption that p(z) is not a sum of squares of polynomials.

Proof of Theorem 1
The case Sep(3, 3): We prove that SEP(3, 3) has no semidefinite representation. Define the Choi polynomial [Cho75] by It was shown in [Cho75] (see also [Cho80,Appendix B]) that p(x, y) ≥ 0 for all (x, y) ∈ C 3 ×C 3 , and yet p(x, y) is not a sum of squares. In fact the real polynomial p(x, y) when (x, y) ∈ R 3 × R 3 is not a sum of squares. It follows, by a simple homogenization argument, that the Hermitian polynomial p(x 1 , x 2 , y 1 , y 2 ) = p(x 1 , x 2 , 1, y 1 , y 2 , 1) is not a sum of squares. Note that the support ofp satisfies SinceÂ is downward closed it follows from Theorem 2 that does not have a semidefinite representation. To conclude that SEP(3, 3) has no semidefinite representation, it remains to note that CÂ is a hyperplane section of SEP(3, 3). Indeed, first recall that SEP(3, 3) can be written as It is easy to see that there is a one-to-one correspondence betweenÂ and A. In terms of the monomial map m this simply means that m A is the homogenization of mÂ. For example under an appropriate ordering of the monomials we have m A (x 1 , x 2 , x 3 , y 1 , y 2 , y 3 ) = |x 3 | 2 |y 3 | 2 mÂ x x 3 , y y 3 where we let x = (x 1 , x 2 ) and y = (y 1 , y 2 ). It thus follows that SEP(3, 3) can be written as It can then be readily verified that CÂ is a hyperplane section of SEP(3, 3) where the appropriate coordinate (corresponding to the monomial |x 3 | 2 |y 3 | 2 ) is set to 1.
The case Sep(4, 2): Following the same approach as above, we need to exhibit a Hermitian polynomialp(x 1 , x 2 , x 3 , y) supported on A = (α, β, γ, δ) ∈ (N 3 × N) × (N 3 × N) : |α| ≤ 1, |β| ≤ 1, |γ| ≤ 1, |δ| ≤ 1 that is nonnegative but not a sum-of-squares. Since Sep(4, 2) = PPT(4, 2) we know that there exists a Hermitian homogeneous polynomial p(x, y) on (x, y) ∈ C 4 × C 2 of the form p(x, y) = ijkl p ijkl x ixj y kȳl (x ∈ C 4 , y ∈ C 2 ) that is nonnegative but not a sum of squares. Note that such a p satisfies p(λx, µy) = |λ| 2 |µ| 2 p(x, y) for any (λ, µ) ∈ C 2 . To get the desired polynomialp it would suffice to dehomogenize the polynomial p by setting one of the x variables to 1, and one of the y variables to 1. It turns out, however, that one cannot guarantee in general that this dehomogenized polynomial is not a sum of squares. (We give an explicit example in Appendix A.) The reason we could dehomogenize the Choi polynomial in the (3, 3) case was that the Choi polynomial is not a sum of squares when the variables are real.
One cannot expect this to be true for our polynomial in (4,2) variables as it is known that any biquadratic real polynomial in (n, 2) variables is a sum of squares [Cal73]. Nevertheless we show in Appendix A that by choosing an appropriate polynomial p, and an appropriate dehomogenization we can get a polynomialp(x, y) supported onÂ of Equation (14) that is not a sum-of-squares. This implies that CÂ is not semidefinite representable. Using a similar argument as for the (3, 3) case we get that SEP(4, 2) is not semidefinite representable.

A The case Sep(4, 2)
In this section we exhibit a nonnegative Hermitian polynomialp(x 1 , x 2 , x 3 , y) supported onÂ, defined in Equation (14), that is not a sum of squares. Consider the following map Φ : M 2 → M 4 studied in [HK16]: It is shown in [HK16] that the map Φ is positive but not decomposable. We associate to Φ the Hermitian polynomial Then we know from Proposition 1 that W is positive but not a sum of squares. The purpose of this section is to prove the following: Proposition 3. The (nonhomogeneous) Hermitian polynomial W(1, x 2 , x 3 , x 4 , 1, y 2 ) is not a sum of squares.
The proof of this proposition involves some computations, and we use the specific properties of W (its zeros) which have been studied in [HK16]. We note that there are other dehomogenizations of W that are sums of squares. For example, we show later that W(x 1 , x 2 , −6, x 4 , 1, y 2 ) is a sum of squares.
Assume that p = i g 2 i where g i are Hermitian polynomials in x 2 , x 3 , x 4 , α. Since p has no terms |x 3 | 2 we see that the g i cannot contain monomials x 3 orx 3 . Similarly p does not have a term |α| 2 |x 2 | 2 and so p cannot contain monomials αx 2 ,ᾱx 2 , αx 2 orᾱx 2 . It follows that each g i must be a linear combination of the monomials 1, α, x 2 , x 4 , αx 3 , αx 4 ,ᾱx 3 ,ᾱx 4 and their conjugates. In other words, each g i is of the form: where a i , b i , . . . , h i ∈ C. We will now use the information about the zeros of W (and thus of p) to deduce relations about these coefficients and reach a contradiction.
We can clear denominators in (17) by multiplying the expression by |x 1 (α)| 2 . As a result we get that The left-hand side of (18) is a Hermitian polynomial in α that is identically zero. Hence all its coefficients must be equal to 0. This allows us to derive conditions on the coefficients of g i in (16). More precisely: • The coefficient of α 4 is 4h i . Setting 4h i to zero yields h i = 0.
• The coefficient of α 4ᾱ is 4ḡ i + 2h i . Setting to zero we get g i = 0.
• The coefficient of α 2 is −4d i − 8ḡ i . Setting to zero we get d i = 0.
This gives a contradiction: indeed the coefficient of |x 4 | 2 in p is 4 > 0 and yet i |d i | 2 = 0.