Tensor theta norms and low rank recovery

We study extensions of compressive sensing and low rank matrix recovery to the recovery of tensors of low rank from incomplete linear information. While the reconstruction of low rank matrices via nuclear norm minimization is rather well-understand by now, almost no theory is available so far for the extension to higher order tensors due to various theoretical and computational difficulties arising for tensor decompositions. In fact, nuclear norm minimization for matrix recovery is a tractable convex relaxation approach, but the extension of the nuclear norm to tensors is in general NP-hard to compute. In this article, we introduce convex relaxations of the tensor nuclear norm which are computable in polynomial time via semidefinite programming. Our approach is based on theta bodies, a concept from real computational algebraic geometry which is similar to the one of the better known Lasserre relaxations. We introduce polynomial ideals which are generated by the second-order minors corresponding to different matricizations of the tensor (where the tensor entries are treated as variables) such that the nuclear norm ball is the convex hull of the algebraic variety of the ideal. The theta body of order k for such an ideal generates a new norm which we call the θk-norm. We show that in the matrix case, these norms reduce to the standard nuclear norm. For tensors of order three or higher however, we indeed obtain new norms. The sequence of the corresponding unit-θk-norm balls converges asymptotically to the unit tensor nuclear norm ball. By providing the Gröbner basis for the ideals, we explicitly give semidefinite programs for the computation of the θk-norm and for the minimization of the θk-norm under an affine constraint. Finally, numerical experiments for order-three tensor recovery via θ1-norm minimization suggest that our approach successfully reconstructs tensors of low rank from incomplete linear (random) measurements.


Introduction and Motivation
Compressive sensing predicts that sparse vectors can be recovered from underdetermined linear measurements via efficient methods such as 1 -minimization [10,19,22].This finding has various applications in signal and image processing and beyond.It has recently been observed that the principles of this theory can be transferred to the problem of recovering a low rank matrix from underdetermined linear measurements.One prominent choice of recovery method consists in minimizing the nuclear norm subject to the given linear constraint [21,54].This convex optimization problem can be solved efficiently and recovery results for certain random measurement maps have been provided, which quantify the minimal number of measurements required for successful recovery [6,7,30,31,42,54].
There is significant interest in going one step further and to extend the theory to the recovery of low rank tensors (higher-dimensional arrays) from incomplete linear measurements.Applications include image and video inpainting [45], reflectance data recovery [45] (e.g. for use in photo-realistic raytracers), machine learning [55], and seismic data processing [40].Several approaches have already been introduced [24,38,45,51,52], but unfortunately, so far, for none of them a completely satisfactory theory is available.Either the method is not tractable [62], or no (complete) rigorous recovery results quantifying the minimal number of measurements are available [16,24,39,41,45,51,52], or the available bounds are highly nonoptimal [20,38,46].For instance, the computation (and therefore, also the minimization) of the tensor nuclear norm ( [18,56,60]) for higher order tensors is in general NP-hard [23] -nevertheless, some recovery results for tensor completion via nuclear norm minimization are available in [62].Moreover, versions of iterative hard thresholding for various tensor formats have been introduced [51,52].This approach leads to a computationally tractable algorithm, which empirically works well.However, only a partial analysis based on the tensor restricted isometry property has been provided, which so far only shows convergence under a condition on the iterates that cannot be checked a priori.Nevertheless, the tensor restricted isometry property (TRIP) has been analyzed for certain random measurement maps [51][52][53].These near optimal bounds on the number of measurements ensuring the TRIP, however, provide only a hint on how many measurements are required because the link between the TRIP and recovery is so far only partial [52,53].
This article introduces a new approach for tensor recovery based on convex relaxation.The idea is to further relax the nuclear norm in order to arrive at a norm which can be computed (and minimized under a linear constraint) in polynomial time.The hope is that the new norm is only a slight relaxation and possesses very similar properties as the nuclear norm.Our approach is based on theta bodies, a concept from computational algebraic geometry [2,26,47] which is similar to the better known Lasserre relaxations [44].We arrive at a whole family of convex bodies (indexed by a polynomial degree), which form convex relaxations of the unit nuclear norm ball.The resulting norms are called theta norms.The corresponding unit norm balls are nested and contain the unit nuclear norm ball.Even more, the sequence of the unit-θ k -norm balls converges asymptotically to the unit tensor nuclear norm ball.They can be computed by semidefinite optimization, and also the minimization of the θ k -norm subject to a linear constraints is a semidefinite program (SDP) whose solution can be computed in polynomial time -the complexity growing with k.
The basic idea for the construction of these new norms is to define polynomial ideals, where each variable corresponds to an entry of the tensor, such that its algebraic variety consists of the rank-one tensors of unit Frobenius norm.The convex hull of this set is the tensor nuclear norm ball.The ideals that we propose are generated by the minors of order two of all matricizations of the tensor (or at least of a subset of the possible matricizations) together with the polynomial corresponding to the squared Frobenius norm minus one.Here, a matricization denotes a matrix which is generated from the tensor by combining several indices to a row index, and the remaining indices to a column index.In fact, all such minors being zero simultaneously means that the tensor has rank one.The k-theta body of the ideal corresponds then to a relaxation of the convex hull of its algebraic variety, i.e., to a further relaxation of the tensor nuclear norm.The index k ∈ N corresponds to a polynomial degree involved in the construction of the theta bodies (some polynomial is required to be k-sos modulo the ideal, see below), and k = 1 leads to the largest theta body in a family of convex relaxations.
We will show that for the matrix case (tensors of order 2), our approach does not lead to new norms.All resulting theta norms are rather equal to the matrix nuclear norm.This fact suggests that the theta norms in the higher order tensor case are all natural generalizations of the matrix nuclear norm.
We derive the corresponding semidefinite programs explicitly and present numerical experiments which show that θ 1 -norm minimization successfully recovers tensors of low rank from few random linear measurements.Unfortunately, a rigorous theoretical analysis of the recovery performance of θ k -minimization is not yet available but will be the subject of future studies.
1.1.Low rank matrix recovery.Before passing to tensor recovery, we recall some basics on matrix recovery.Let X ∈ R n1×n2 of rank at most r min{n 1 , n 2 }, and suppose we are given linear measurements where A : R n1×n2 → R m is a linear map with m n 1 n 2 .Reconstructing X from y amounts to solving an underdetermined linear system.Unfortunately, the rank minimization problem of computing the minimizer of min Z∈R n 1 ×n 2 rank(Z) subject to A(Z) = y is NP-hard in general.As a tractable alternative, the convex optimization problem min has been suggested [21,54], where the nuclear norm Z * = j σ j (Z) is the sum of the singular values of Z.This problem can be solved efficiently by various methods [3].For instance, it can be reformulated as a semidefinite program [21], but splitting methods may be more efficient [50,58].
While it is hard to analyze low rank matrix recovery for deterministic measurement maps, optimal bounds are available for several random matrix constructions.If A is a Gaussian measurement map, i.e., A(X) j = k, A jk X k , j ∈ [m] := {1, 2, . . ., m}, where the A jkl , j ∈ [m], k ∈ [n 1 ], ∈ [n 2 ], are independent mean-zero, variance one Gaussian random variables, then a matrix X of rank at most r can be reconstructed exactly from y = A(X) via nuclear norm minimization (1) with probability at least 1 − e −cm provided that where the constants c, C > 0 are universal [6,12].Moreover, the reconstruction is stable under passing to only approximately low rank matrices and under adding noise on the measurements.Another interesting measurement map corresponds to the matrix completion problem [7,9,13,30], where the measurements are randomly chosen entries of the matrix X. Measurements taken as Frobenius inner products with rank-one matrices are studied in [42], and arise in the phase retrieval problem as special case [8].Also here, m ≥ Crn (or m ≥ Crn log(n) for certain structured measurements) is sufficient for exact recovery.
1.2.Tensor recovery.An order-d tensor (or mode-d-tensor) is an element Of course, the case d = 2 corresponds to matrices.For d ≥ 3, several notions and computational tasks become much more involved than for the matrix case.Already the notion of rank requires some clarification, and in fact, several different definitions are available, see for instance [29,35,36,43].We will mainly work with the canonical rank or CP-rank in the following.A dth-order tensor X ∈ R The CP-rank (or canonical rank and in the following just rank) of a tensor X ∈ R n1×n2×•••×n d , similarly as in the matrix case, is the smallest number of rank-one tensors that sum up to X.
Given a linear measurement map A : R n1×•••×n d → R m (which can represented as a (d + 1)th-order tensor), our aim is to recover a tensor The matrix case d = 2 suggests to consider minimization of the tensor nuclear norm for this task, min where the nuclear norm is defined as Unfortunately, in the tensor case, computing the canonical rank of a tensor, as well as computing the nuclear norm of a tensor is NP-hard in general, see [23,34,37].Let us nevertheless mention that some theoretical results for tensor recovery via nuclear norm minimization are contained in [62].
We remark that, unlike in the matrix scenario, the tensor rank and consequently the tensor nuclear norm are dependent on the choice of base field, see for example [4,17,23].In other words, the rank (and the nuclear norm) of a given tensor with real entries depends on whether we regard it as a real tensor or as a complex tensor.In this paper, we focus only on tensors with real-valued entries, i.e., we work over the field R.
The aim of this article is to introduce relaxations of the tensor nuclear norm, based on theta bodies, which is both computationally tractable and whose minimization allows for exact recovery of low rank tensors from incomplete linear measurements.
Let us remark that one may reorganize (flatten) a low rank tensor X ∈ R n×n×n into a low rank matrix X ∈ R n×n 2 and simply apply concepts from matrix recovery.However, the bound (2) on the required number of measurements then reads m ≥ Crn 2 . ( Moreover, it has been suggested in [24,45,59] to minimize the sum of nuclear norms of the unfoldings (different reorganizations of the tensor as a matrix) subject to the linear constraint matching the measurements.Although this seems to be a reasonable approach at first sight, it has been shown in [49], that it cannot work with less measurements than stated by the estimate in (3).This is essentially due to the fact that the tensor structure is not represented.That is, instead of solving a tensor nuclear norm minimization problem under the assumption that the tensor is of low rank, the matrix nuclear norm minimization problem is being solved under the assumption that a particular matricization of a tensor is of low rank.
Bounds for a version of the restricted isometry property for certain tensor formats in [53] suggest that m ≥ Cr 2 n measurements should be sufficient when working directly with the tensor structureprecisely, this bound uses the tensor train format [48].(Possibly, the term r 2 may even be lowered to r when using the "right" tensor format.)However, connecting the restricted isometry property in a completely satisfactory way with the success of an efficient tensor recovery algorithm is still open.(Partial results are contained in [53].)In any case, this suggests that one should exploit the tensor structure of the problem rather than reducing to a matrix recovery problem in order to recover a low rank tensor using the minimal number of measurements.Of course, similar considerations apply to tensors of order higher than three, where the difference between the reduction to the matrix case and working directly with the tensor structure will become even stronger.
Unlike in the previously mentioned contributions, we consider the canonical tensor rank and the corresponding tensor nuclear norm, which respects the tensor structure.Even more, it is expected that the bound on the minimal number of measurements needed for low rank tensor recovery via tensor nuclear norm minimization is optimal, see also [62], where tensor completion via tensor nuclear norm minimization has been considered.Unfortunately, it is in general NP-hard to solve this optimization problem (since it is NP-hard to compute the tensor nuclear norm).To overcome this difficulty, in this paper, we provide the tensor θ k -norms -the new tensor norms which can be computed via semidefinite programming.These norms are tightly related to the tensor nuclear norm.That is, the unit θ k -norm balls (which are defined for k ∈ N) satisfy In particular, we show that in the matrix scenario all θ k -norms coincide with the matrix nuclear norm.In case of order-d tensors (d ≥ 3), we prove that the sequence of the unit-θ k -norm balls converges asymptotically to the unit tensor nuclear norm ball.Next, we provide numerical experiments on low rank tensor recovery via θ 1 -norm minimization.We provide numerical experiments for θ 1 -minimization that indicate that this is a very promising approach for low rank tensor recovery.However, we note that standard solvers for semidefinite programs only allow us to test our method on small to moderate size problems.Nevertheless, it is likely that specialized efficient algorithms can be developed.Indeed, recall that θ k -norms all coincide with the matrix nuclear norm and the state-of-the-art algorithms allow us computing the nuclear norm of matrices of large dimensions.This suggests the possibility that new algorithms could be developed which would allow us to apply our method on larger tensors.Thus, this paper presents the first step in a new convex optimization approach to low rank tensor recovery.1.3.Some notation.We write vectors with small bold letters, matrices and tensors with capital bold letters and sets with capital calligraphic letters.The cardinality of a set S is denoted by |S|.
For a matrix A ∈ R m×n and subsets I ⊂ [m], J ⊂ [n] the submatrix of A with columns indexed by I and rows indexed by J is denoted by A I,J .A set of all order-k minors of A is of the form The Frobenius norm of a matrix X ∈ R m×n is given as where the σ i list the singular values of X.The nuclear norm is given by X * = min{m,n} i=1 σ i .It is well-known that its unit ball is the convex hull of all rank-one matrices of unit Frobenius norm.

The vectorization of a tensor
The ordering of the elements in vec(X) is not important as long as it remains consistent.Fibers are a higher order analogue of matrix rows and columns.For k ∈ [d], the mode-k fiber of a dth-order tensor is obtained by fixing every index except for the k-th one.The Frobenius norm of a dth-order tensor X ∈ R n1×n2×•••×n d is defined as Matricization (also called flattening) is the operation that transforms a tensor into a matrix.More precisely, for a dth-order tensor X ∈ R n1×n2×•••×n d and an ordered subset S ⊆ [d], an S-matricization X S ∈ R k∈S n k × ∈S c n is defined as e., the indexes in the set S define the rows of a matrix and the indexes in the set S c = [d] \S define the columns.For a singelton set S = {i}, for i ∈ [d], we call the S-matricization the i-th unfolding.Notice that every S-matricization of a rank-one tensor is a rank-one matrix.Conversely, if every S-matricization of a tensor is a rank-one matrix, then the tensor is of rank one.This is even true, if all unfoldings of a tensor are of rank one.
1.4.Structure of the paper.In Section 2 we will review the basic definition and properties of theta bodies.Section 3 considers the matrix case.We introduce a suitable polynomial ideal whose algebraic variety is the set of rank-one unit Frobenius norm matrices.We discuss the corresponding θ k -norms and show that they all coincide with the matrix nuclear norm.The case of 2 × 2-matrices is described in detail.In Section 4 we pass to the tensor case and discuss first the case of order-three tensors.We introduce a suitable polynomial ideal, provide its reduced Gröbner basis and define the corresponding θ k -norms.We additionally show that considering matricizations corresponding to the TT-format will lead to the same polynomial ideal and thus to the same θ k -norms.The general dth-order case is discussed at the end of Section 4. Here, we define the polynomial ideal J d which corresponds to the set of all possible matricizations of the tensor.We show that a certain set of order-two minors forms the reduced Gröbner basis for this ideal, which is key for defining the θ k -norms.We additionally show that polynomial ideals corresponding to different tensor formats (such as TT format or Tucker/HOSVD format) coincide with the ideal J d and consequently, they lead to the same θ k -norms.In Section 5 we discuss the convergence of the sequence of the unit-θ k -norm balls to the unit tensor nuclear norm ball.Section 6 briefly discusses the polynomial runtime of the algorithms for computing and minimizing the θ k -norms showing that our approach is tractable.Numerical experiments for low rank recovery of third-order tensors are presented in Section 7, which show that our approach successfully recovers a low rank tensor from incomplete Gaussian random measurements.Appendix A discusses some background from computer algebra (monomial orderings and Gröbner bases) that is required throughout the main body of the article.

Theta bodies
As outlined above, we will introduce new tensor norms as relaxations of the nuclear norm in order to come up with a new convex optimization approach for low rank tensor recovery.Our approach builds on theta bodies, a recent concept from computational algebraic geometry, which is similar to Lasserre relaxations [44].
In order to introduce it, we first discuss the necessary basics from computational commutative algebra.For more information, we refer to [14,15] and to the appendix.
For a non-zero polynomial x n ] and a monomial order >, we denote a) the multidegree of f by multideg (f be a polynomial ideal.Its real algebraic variety is the set of all points in x ∈ R n where all polynomials in the ideal vanish, i.e., ν R (J) = {x ∈ R n : f (x) = 0, for all f ∈ J}.By Hilbert's basis theorem [15] every polynomial ideal in R [x] has a finite generating set.Thus, we may assume that J is generated by a set F = {f 1 , f 2 , . . ., f k } of polynomials in R [x] and write or simply J = F .
Its real algebraic variety is the set Throughout the paper, R [x] k denotes the set of polynomials of degree at most k.A degree one polynomial is also called linear polynomial.A very useful certificate for positivity of polynomials is contained in the following definition [26].
A special case of theta bodies was first introduced by Lovász in [47] and in full generality they appeared in [26].Later, they have been analyzed in [25,27].The definitions and theorems in the remainder of the section are taken from [26].Definition 2.2 (Theta body).Let J ⊆ R [x] be an ideal.For a positive integer k, the k-th theta body of J is defined as We say that an ideal Theta bodies are closed convex sets, while conv (ν R (J)) may not necessarily be closed and by definition, The theta-body sequence of J can converge (finitely or asymptotically), if at all, only to conv (ν R (J)).More on guarantees on convergence can be found in [26,27].However, to our knowledge, none of the existing guarantees apply to the cases discussed below.
Given any polynomial, it is possible to check whether it is k-sos mod J using a Gröbner basis and semidefinite programming.However, using this definition in practice requires knowledge of all linear polynomials (possibly infinitely many) that are k-sos mod J.To overcome this difficulty, we need an alternative description of TH k (J) discussed next.
As in [2], we assume that there are no linear polynomials in the ideal J. Otherwise, some variable x i would be congruent to a linear combination of other variables modulo J and we could work in a smaller polynomial ring R Recall that the degree of an equivalence class f + J, denoted by deg (f + J), is the smallest degree of an element in the class.We assume that each element in the basis B = {f i + J} of R [x] /J is represented by the polynomial whose degree equals the degree of its equivalence class, i.e., deg (f i + J) = deg (f i ).In addition, we assume that B = {f i + J} is ordered so that f i+1 > f i , where > is a fixed monomial ordering.Further, we define the set As in [2,26] we consider only monomial bases For determining a θ-basis, we first need to compute the reduced Gröbner basis G of the ideal J, see Definitions A.2 and A.3.The set B will satisfy the second property in the definition of the theta basis if the reduced Gröbner basis is with respect to an ordering which first compares the total degree.Therefore, throughout the paper we use the graded reverse monomial ordering (Definition A.1) or simply grevlex ordering, although also the graded lexicographic ordering would be appropriate.
A technique to compute a θ-basis B of R [x] /J consists in taking B to be the set of equivalence classes of the standard monomials of the corresponding initial ideal where G = g 1 , g 2 , . . ., g s is the reduced Gröbner basis of the ideal J.In other words, a set The next important tool we need is the combinatorial moment matrix of J. To this end, we fix a θ-basis The theta bodies can be characterized via the combinatorial moment matrix as stated in the next result from [26], which will be the basis for computing and minimization the new tensor norm introduced below via semidefinite programming.Definition 2.4.Let J, B and {λ l i,j } be as above.Let y be a real vector indexed by B 2k with y 0 = 1, where y 0 is the first entry of y, indexed by the basis element 1 + J.The k-th combinatorial moment matrix M B k (y) of J is the real matrix indexed by Theorem 2.5.The k-th theta body of J, TH k (J), is the closure of where π R n denotes the projection onto the variables y 1 = y x1+J , . . ., y n = y xn+J .
Algorithm 1 shows a step-by-step procedure for computing TH k (J).
Algorithm 1 Algorithm for computing TH k (J) Compute the reduced Gröbner basis for the ideal J Compute a θ-basis

The matrix case
As a start, we consider the matrix nuclear unit norm ball and provide hierarchical relaxations via theta bodies.The k-th relaxation defines a matrix unit θ k -norm ball with the property for all X ∈ R m×n and all k ∈ N.
However, we will show that all these θ k -norms coincide with the matrix nuclear norm.The first step in computing hierarchical relaxations of the unit nuclear norm ball consists in finding a polynomial ideal J such that its algebraic variety (the set of points for which the ideal vanishes) coincides with the set of all rank-one, unit Frobenius norm matrices Recall that the convex hull of this set is the nuclear norm ball.The following lemma states the elementary fact that a non-zero matrix is a rank-one matrix if and only if all its minors of order two are zero.
For notational purposes, we define the following polynomials in R Then X is a rank-one, unit Frobenius norm matrix if and only if Proof.If X ∈ R m×n is a rank-one matrix with X F = 1, then by definition there exist two vectors u ∈ R m and v ∈ R n such that For the converse, let X •i represent the i-th column of a matrix X ∈ R.Then, for all j, l ∈ [n] with j < l, it holds Thus, the columns of the matrix X span a space of dimension one, i.e., the matrix X is a rank-one matrix.From It follows from Lemma 3.1 that the set of rank-one, unit Frobenius norm matrices coincides with the algebraic variety ν R (J Mmn ) for the ideal J Mmn generated by the polynomials g and f ijkl , i.e., Recall that the convex hull of the set R in (7) forms the unit nuclear norm ball and by definition of the theta bodies, Therefore, the theta bodies form closed, convex hierarchical relaxations of the matrix nuclear norm ball.In addition, the theta body TH k (J Mmn ) is symmetric, TH k (J Mmn ) = − TH k (J Mmn ).Therefore, it defines a unit ball of a norm that we call the θ k -norm.
The next result shows that the generating set of the ideal J Mmn introduced above is a Gröbner basis.
Lemma 3.2.The set G Mmn forms the reduced Gröbner basis of the ideal J Mmn with respect to the grevlex order.
Proof.The set G Mmn is clearly a basis for the ideal J Mmn .By Proposition A.8 in the appendix, we only need to check whether the S-polynomial, see Definition A.6, satisfies S (p, q) → G Mmn 0 for all p, q ∈ G Mmn whenever the leading monomials LM (p) and LM (q) are not relatively prime.Here, S (p, q) → G Mmn 0 means that S (p, q) reduces to 0 modulo G Mmn , see Definition A.5.
Notice that LM (g) = x 2 11 and LM (f ijkl ) = x il x kj are relatively prime, for all 1 ≤ i < k ≤ m and 1 ≤ j < l ≤ n.Therefore, we only need to show that S(f ijkl , f îĵkl ) → G Mmn 0 whenever the leading monomials LM(f ijkl ) and LM(f îĵkl ) are not relatively prime.First we consider The remaining cases are treated with similar arguments.
In order to show that G Mmn is a reduced Gröbner basis (see Definition A.3), we first notice that LC(f ) = 1 for all f ∈ G Mmn .In addition, the leading monomial of f ∈ G Mmn is always of degree two and there are no two different polynomials f i , f j ∈ G Mmn such that LM(f i ) = LM(f j ).Therefore, G Mmn is the reduced Gröbner basis of the ideal J Mmn with respect to the grevlex order.
The Gröbner basis G Mmn of J Mmn = G Mmn yields the θ-basis of R[x]/J Mmn .For the sake of simplicity, we only provide its elements up to degree two, . Given the θbasis, the theta body TH k (J Mmn ) is well-defined.We formally introduce an associated norm next.
The θ k -norm can be computed with the help of Theorem 2.5, i.e., as Given the moment matrix M B k [y] associated with J Mmn , this minimization program is equivalent to the semidefinite program The last constraint might require some explanation.The vector y B1 denotes the restriction of y to the indices in B 1 , where the latter can be identified with the set [m] × [n] indexing the matrix entries.Therefore, y B1 = X means componentwise y x11+J = X 11 , y x12+J = X 12 , . . ., y xmn+J = X mn .For the purpose of illustration, we focus on the θ 1 -norm in R 2×2 in Section 3.1 below, and provide a step-by-step procedure for building the corresponding semidefinite program in (21).
Notice that the number of elements in B 1 is mn + 1, and in , the number of elements of the θ-basis restricted to the degree 2 scales polynomially in the total number of matrix entries mn.Therefore, the computational complexity of the SDP in ( 21) is polynomial in mn.
We will show next that the theta body TH 1 (J) and hence, all TH k (J) for k ∈ N, coincide with the nuclear norm ball.To this end, the following lemma provides expressions for the boundary of the matrix nuclear unit norm ball.
Proof.If suffices to treat the case m ≤ n because X * = X T * for all matrices X, and M ∈ O r if and only if M T ∈ O c .Let X ∈ R m×n such that X * ≤ 1 and let X = UΣV T be its singular value decomposition.For M ∈ O c , the spectral norm satisfies M ≤ 1 and therefore, using that the nuclear norm is the dual of the spectral norm, see e.g.[1, p. 96], For the converse, let X ∈ R m×n be such that tr (MX) ≤ 1, for all M ∈ O c .Let X = UΣ V T denote its reduced singular value decomposition, i.e., U, Σ ∈ R m×m This completes the proof.
Next, using Lemma 3.4, we show that the theta body TH 1 (J) equals the nuclear norm ball.This result is related to Theorem 4.4 in [27].
In other words, Proof.By definition of TH 1 (J Mmn ), it is enough to show that the boundary of the unit nuclear norm can be written as 1-sos mod J Mmn , which by Lemma 3. where I k ∈ R k×k is the identity matrix.For this choice of M, we need to show that 1 − i=1 x ii is 1-sos mod J Mmn , where = min {m, n}.Note that and Therefore, 1 − i=1 x ii is 1-sos mod J Mmn , since the polynomials 1 − i=1 x ii , x ij − x ji , x ij , and x ji are linear and the polynomials 1 x 2 ij and 2 (x ii x jj − x ij x ji ) are contained in the ideal, for all i < j ≤ .
Next, we define transformed variables and x ji is preserved, for all i < j.It remains to show that the ideal is invariant under this transformation.For the polynomial 1 columns in case when m ≤ n and unitary rows in case m ≥ n.In the case of m ≤ n the polynomial x ii x jj − x ij x ji is contained in the ideal J since and the polynomials x ki x lj − x kj x li are contained in J for all i < j ≤ m.Similarly, in case m ≥ n the polynomial x ii x jj − x ij x ji is in the ideal since and polynomials x ik x jl − x il x jk are in the ideal, for all i < j ≤ n.
The following corollary is a direct consequence of Theorem 3.5 and the nestedness property (4) of theta bodies.
Corollary 3.6.The matrix θ 1 -norm coincides with the matrix nuclear norm, i.e., Moreover, Remark 2. The ideal (8) is not the only choice that satisfies (5).For example, in [12] the following polynomial ideal was suggested . ., x mn , u 1 , . . ., u m , v 1 , . . ., u n ].Some tedious computations reveal the reduced Gröbner basis G of the ideal J with respect to the grevlex (and grlex) ordering, Obviously, this Gröbner basis is much more complicated than the one of the ideal J Mmn introduced above.Therefore, computations (both theoretical and numerical) with this alternative ideal seem to be more demanding.In any case, the variables {u i } m i=1 and {v j } n j=1 are only auxiliary ones, so one would like to eliminate these from the above Gröbner basis.By doing so, one obtains the Gröbner basis G Mmn defined in (8).Notice that n j=2 g j 11 together with {g i,j,k,l 12 } form the basis G Mmn .
3.1.The θ 1 -norm in R 2×2 .For the sake of illustration, we consider the specific example of 2 × 2 matrices and provide the corresponding semidefinite program for the computation of the θ 1 -norm explicitly.Let us denote the corresponding polynomial ideal in R [x] = R [x 11 , x 12 , x 21 , x 22 ] simply by The associated algebraic variety is of the form ν R (J) = x : x 12 x 21 = x 11 x 22 , x 2 11 + x 2 12 + x 2 21 + x 2 22 = 1 and corresponds to the set of rank-one matrices with X F = 1.Its convex hull consists of matrices X ∈ R 2×2 with X * ≤ 1.According to Lemma 3.2, the Gröbner basis G of J with respect to the grevlex order is 22 − 1 with the corresponding θ-basis B of R [x] /J restricted to the degree two given as The set B 2 consists of all monomials of degree at most two which are not divisible by a leading term of any of the polynomials inside the Gröbner basis G.For example, x 11 x 12 + J is an element of the theta basis B, but x 2 11 + J is not since x 2 11 is divisible by LT(g 2 ).
Linearizing the elements of B 2 results in Table 1, where the monomials f in the first row stand for an element f T and the following combinatorial moment matrix M B1 (x, y), see Definition 2.4, is given as 22 + 1 + J, where we exploit the second property in Definition 2.3 and the fact that g 2 ∈ J. Replacing x 2 12 + J by y 4 , etc. as in Table 1, yields the stated expression for M B1 (x, y) 2,2 .
By Theorem 2.5, the first theta body TH 1 (J) is the closure of where π x represents the projection onto the variables x, i.e., the projection onto x 11 , x 12 , x 21 , x 22 .Furthermore, θ 1 -norm of a matrix X ∈ R 2×2 induced by the TH 1 (J) and denoted as • θ1 can be computed as which is equivalent to Notice that trace(M) = 2t.By Theorem 3.5, the above program is equivalent to the standard semidefinite program for computing the nuclear norm of a given matrix Remark 3. In compressive sensing, reconstruction of sparse signals via 1 -norm minimization is well-understood, see for example [10,19,22].It is possible to provide hierarchical relaxations via theta bodies of the unit 1 -norm ball.However, as in the matrix scenario discussed above, all these relaxations coincide with the unit 1 -norm ball, [57].

The tensor θ k -norm
Let us now turn to the tensor case and study the hierarchical closed convex relaxations of the unit tensor nuclear norm ball defined via theta bodies.Since in the matrix case all θ k -norms are equal to the matrix nuclear norm, their generalization to the tensor case may all be viewed as natural generalizations of the nuclear norm.We focus mostly on the θ 1 -norm whose unit norm ball is the largest in a hierarchical sequence of relaxations.Unlike in the matrix case, the θ 1 -norm defines a new tensor norm, that up to the best of our knowledge has not been studied before.
The polynomial ideal will be generated by the minors of order two of the unfoldings -and matricizations in the case d ≥ 4 -of the tensors, where each variable corresponds to one entry in the tensor.As we will see, a tensor is of rank one if and only if all order-two minors of the unfoldings (matricizations) vanish.While the order-three case requires to consider all three unfoldings, there are several possibilities for the order-d case when d ≥ 4. In fact, a dth-order tensor is of rank one if all minors of all unfoldings vanish so that it may be enough to consider only the unfoldings.However, one may as well consider the ideal generated by all minors of all matricizations or one may consider a subset of matricizations including all unfoldings.Indeed, any tensor format -and thereby any notion of tensor rank -corresponds to a set of matricizations and in this way, one may associate a θ k -norm to a certain tensor format.We refer to e.g.[32,52] for some background on various tensor formats.However, as we will show later, the corresponding reduced Gröbner basis with respect to the grevlex order does not depend on the choice of the tensor format.We will mainly concentrate on the case that all matricizations are taken into account for defining the ideal.Only for the case d = 4, we will briefly discuss the case, that the ideal is generated only by the minors corresponding to the four unfoldings.
Below, we consider first the special case of third-order tensors and continue then with fourth-order tensors.In Subsection 4.2 we will treat the general dth-order case.
We remark that in real algebraic geometry and commutative algebra, polynomials f (α,β) are known as Hibi relations, see [33].Lemma 4.1.A tensor X ∈ R n1×n2×n3 is a rank-one, unit Frobenius norm tensor if and only if g 3 (X) = 0 and f (α,β) (X) = 0 for all (α, β) ∈ S. ( Proof.Sufficiency of ( 16) follows directly from the definition of the rank-one unit Frobenius norm tensors.For necessity, the first step is to show that mode-1 fibers (columns) span one-dimensional space in R n1 .To this end, we note that for β 2 ≤ α 2 and β 3 ≤ α 3 , the fibers X •α2α3 and X •β2β3 satisfy where we used that f (α,β) (X) = 0 for all (α, β) ∈ S. From g 3 (X) = 0 it follows that the tensor X is normalized.
Using similar arguments, one argues that mode-2 fibers (rows) and mode-3 fibers span one dimensional spaces in R n2 and R n3 , respectively.This completes the proof.
In order to define relaxations of the unit tensor nuclear norm ball we introduce the polynomial ideal J 3 ⊂ R [x] = R [x 111 , x 112, . . ., x n1n2n3 ] as the one generated by i.e., J 3 = G 3 .Its real algebraic variety equals the set of rank-one third-order tensors with unit Frobenius norm and its convex hull coincides with the unit tensor nuclear norm ball.The next result provides the Gröbner basis of J 3 .
Theorem 4.2.The basis G 3 defined in (17) forms the reduced Gröbner basis of the ideal J 3 = G 3 with respect to the grevlex order.
Proof.Similarly to the proof of Theorem 3.2 we need to show that S (p, q) → G3 0 for all polynomials p, q ∈ G 3 whose leading terms are not relatively prime.The leading monomials with respect to the grevlex ordering are given by LM(g 3 ) = x 2 111 and LM(f (α,β) ) = x α x β , (α, β) ∈ S.
Next we show that S (f, g) ∈ J 3 , for f ∈ f (α,β) : (α, β) ∈ S 2 and g ∈ with β = (β 1 , β 2 , α 3 ), where (α, β) ∈ S 2 .Since x α∧ β = x α∧ β , f ( β,α∨ β) ∈ f (α,β) : (α, β) ∈ S 3 , and f (α∨ For the remaining cases one proceeds similarly.In order to show that G 3 is the reduced Gröbner basis, one uses the same arguments as in the proof of Theorem 3.2. Remark 4. The above Gröbner basis G 3 is obtained by taking a particular subset of all order-two minors of all three unfoldings of the tensor X ∈ R n1×n2×n3 (not considering the same minor twice).One might think that the θ 1 -norm obtained in this way corresponds to a (weighted) sum of the nuclear norms of the unfoldings, which has been used in [24,38] for tensor recovery.The examples of cubic tensors X ∈ R 2×2×2 presented in Table 2 show that this is not the case.Assuming that θ 1 -norm is a linear combination of the nuclear norm of the unfoldings, there exist From the first and the second tensor in Table 2 we obtain γ = 0. Similarly, the first and the third tensor, and the first and the fourth tensor give β = 0 and α = 0, respectively.Thus, the θ 1 -norm does not coincide with a weighted sum of the nuclear norms of the unfoldings.In addition, the last tensor shows that the θ 1 -norm does not equal maximum of the norms of the unfoldings.Theorem 4.2 states that G 3 is the reduced Gröbner basis of the ideal J 3 generated by all order-two minors of all matricizations of an order-three tensor.That is, J 3 is generated by the following polynomials where f {k} (α,β) (x) : (α, β) ∈ T {k} is the set of all order-two minors of the kth unfolding and For (α, β), x α {k} x β {k} denotes a monomial where α , for all (α, β) ∈ T {k} , and all k ∈ [3].Let us now consider a TT-format and a corresponding notion of tensor rank.Recall that a TT-rank of an order three tensor is a vector r = (r 1 , r 2 ) where r 1 = rank(X {1} ) and r 2 = rank(X {1,2} ).Consequently, we consider an ideal J 3,TT generated by all order-two minors of matricizations X {1} and X {1,2} of the order-3 tensor.That is, the ideal J 3,TT is generated by the polynomials Theorem 4.3.The polynomial ideals J 3 and J 3,TT are equal.
Remark 5.As a consequence, G 3 is also the reduced Gröbner basis for the ideal J 3,TT with respect to the grevlex ordering.

4.2.
The theta norm for general dth-order tensors.Let us now consider dthorder tensors in R n1×n2×•••×n d for general d ≥ 4. Our approach relies again on the fact that a tensor X ∈ R n1×n2×•••×n d is of rank-one if and only if all its matricizations are rank-one matrices, or equivalently, if all minors of order two of each matricization vanish.
The description of the polynomial ideal generated by the second order minors of all matricizations of a tensor X ∈ R n1×n2×•••×n d unfortunately requires some technical notation.Again, we do not need all such minors in the generating set that we introduce next.In fact, this generating set will turn out to be the reduced Gröbner basis of the ideal.
Similarly to before, the entry (α 1 , α 2 , . . ., α d ) of a tensor X ∈ R n1×n2×•••×n d corresponds to the variable x α1α2•••α d or simply x α .We aim at introducing a set of polynomials of the form which will generate the desired polynomial ideal.These polynomials correspond to a subset of all order-two minors of all the possible dth-order tensor matricizations.
The set S denotes the indices where α and β differ.Since for an order-two minor of a matricization X M the sets α and β need to differ in at least two indices, S is contained in Given the set S of different indices, we require all non-empty subsets M ⊂ S of possible indices which are "switched" between α and β for forming the minors in (18).This implies that, without loss of generality, That is, the same minor is obtained if we require that α j < β j for all j ∈ M and α k > β k for all k ∈ S\M since the set of all two-minors of X M coincides with the set of all two-minors of X S\M .For S ∈ S [d] , we define e S := min{p : p ∈ S}.The set M corresponds to an associated matricization X M .The set of possible subsets M is given as 2 , e S ∈ M \{∅}, otherwise.Notice that P S ∪ P S c ∪ {∅} ∪ S with P S c := {M : S\M ∈ P S } forms the power set of S. The constraint on the size of M in the definition of P S is motivated by the fact that the role of α and β can be switched and lead to the same polynomial f (α,β) d .Thus, for S ∈ S [d] and M ∈ P S , we define a set For notational purposes, we define Since we are interested in unit Frobenius norm tensors, we also introduce the polynomial Our polynomial ideal is then the one generated by the polynomials in i.e., J d = G d .As in the special case of the third-order tensors, not all second order minors corresponding to all matricizations are contained in the generating set G d due to the condition i k < îk for all k ∈ S in the definition of T S d .Nevertheless all second order minors are contained in the ideal J d as will also be revealed by the proof of Theorem 4.4 below.For instance, h(x) = −x 1234 x 2343 + x 1243 x 2334corresponding to a minor of the matricization X M for M = {1, 2} -does not belong to G 4 , but it does belong to the ideal J 4 .Moreover, it is straightforward to verify that all polynomials in G d differ from each other.
The algebraic variety of J d consists of all rank-one unit Frobenius norm order-d tensors as desired, and its convex hull yields the tensor nuclear norm ball.
, where S ∈ S [d] and M ∈ P S .Therefore, we need to show that S(f To this end, we analyze the division algorithm on G d . Let Then it holds LM(f 1 ) = LM(f 2 ).If these leading monomials are not relatively prime, the S-polynomial is of the form The step-by-step procedure of the division algorithm for our scenario is presented in Algorithm 2. We will show that the algorithm eventually stops and that step 2) is feasible, i.e., that there always exist k and such that line 7 of Algorithm 2 holds -provided that S i = 0. (In fact, the purpose of the algorithm is to achieve the condition that in the ith iteration of the algorithm α1,i Algorithm 2 The division algorithm on the ideal G d . Let LM(S i ) = x α1,i x α2,i x α3,i and NLM(S i ) = S i − LT(S i ) 2) Find indices α 1,i , α 2,i ∈ { α1,i , α2,i , α3,i } such that there exist at least one k and at least one for which where 5) i = i + 1 end while Before passing to the general proof, we illustrate the division algorithm on an example for d = 4.The experienced reader may skip this example.(x) = −x 2111 x 3323 + x 3311 x 2123 ∈ G 4 (with the corresponding sets S = {1, 2, 3, 4}, M = {1, 2}).We will show that S(f 1 , f 2 ) = −x 1112 x 2223 x 3311 + x 1212 x 2111 x 3323 → G4 0 by going through the division algorithm.
In iteration i = 0 we set S 0 = S(f 1 , f 2 ) = −x 1112 x 2223 x 3311 + x 1212 x 2111 x 3323 .The leading monomial is LM(S 0 ) = x 1112 x 2223 x 3311 , the leading coefficient is LC(S 0 ) = −1, and the non-leading monomial is NLM(S 0 ) = x 1212 x 2111 x 3323 .Among the two options for choosing a pair of indexes (α 1,0 , α 2,0 ) in step 2), we decide to take α 1,0 = 1112 and α 2,0 = 3311 which leads to the set M 0 = {4}.The polynomial x α 1,0 x α 2,0 − x α 1,0 ∧α 2,0 x α 1,0 ∨α 2,0 then equals the polynomial f (1112,3311) 4 (x) = −x 1111 x 3312 + x 1112 x 3311 ∈ G 4 and we can write The leading and non-leading monomials of S 1 are LM(S 1 ) = x 1111 x 2223 x 3312 and NLM(S 1 ) = x 1212 x 2111 x 3323 , respectively, while LC(S 1 ) = 1.The only option for a pair of indices as in line 7 of Algorithm 2 is α 1,1 = 3312, α 2,1 = 2223, so that the set The index sets of the monomial and therefore it is the non-leading monomial of S 2 , i.e., NLM(S 2 ) = x 1111 x 2212 x 3323 .Thus, LM(S 2 ) = x 1212 x 2111 x 3323 and LC(S 2 (f 1 , f 2 )) = −1.Now the only option for a pair of indices as in step 2) is α 1,2 = 2111, α 2,2 = 1212 with M 2 = {1}.This yields Thus, the division algorithm stops and we obtained after three steps + LC(S 0 ) LC(S 1 ) LC(S 2 )x 3323 f Let us now return to the general proof.We first show that there always exist indices α 1,i , α 2,i satisfying line 7 of Algorithm 2 unless S i = 0. We start by setting x αi = x α1,i x α2,i x α3,i with x α1,i ≥ x α2,i ≥ x α3,i to be the leading monomial and x β i to be the non-leading monomial of S i .The existence of a polynomial h ∈ G d such that LM(h) divides LM(S i ) = x α1,i x α2,i x α3,i = x αi is equivalent to the existence of α 1,i , α 2,i ∈ α1,i , α2,i , α3,i such that there exists at least one k and at least one for which α 1,i k < α 2,i k and α 1,i > α 2,i .If such pair does not exist in iteration i, we have We claim that this cannot happen if S i = 0.In fact, (19) would imply that the monomial x αi = x α1,i x α2,i x α3,i is the smallest monomial x β x γ x η (with respect to the grevlex order) which satisfies . However, then x αi would not be the leading monomial by definition of the grevlex order, which leads to a contradiction.Hence, we can always find indices α 1,i , α 2,i satisfying line 7 in step 2) of Algorithm 2 unless S i = 0.
Next we show that the division algorithm always stops in a finite number of steps.We start with iteration i = 0 and assume that S 0 = 0. We choose α 1,0 , α 2,0 , α 3,0 as in step 2) of Algorithm 2. Then we divide the polynomial S 0 by a polynomial h ∈ G d such that LM(h) = x α 1,0 x α 2,0 .The polynomial h ∈ G d is defined as in step 3) of the algorithm, i.e., The division of S 0 by h results in .
In order to show that G d is the reduced Gröbner basis of J d , first notice that LC(g) = 1 for all g ∈ G d .Furthermore, the leading term of any polynomial in G d is of degree two.Thus, it is enough to show that for every pair of different polynomials f But this follows from the fact that all elements of G d are different as remarked before the statement of the theorem.
We define the tensor θ k -norm analogously to the matrix scenario.
Definition 4.5.The tensor θ k -norm, denoted by • θ k , is the norm induced by the k-theta body TH k (J d ), i.e., The θ k -norm can be computed with the help of Theorem 2.5, i.e., as We have focused on the polynomial ideal generated by all second order minors of all matricizations of the tensor.One may also consider a subset of all possible matricizations corresponding to various tensor decompositions and notions of tensor rank.For example, the Tucker(HOSVD)-rank (corresponding to the Tucker or HOSVD decomposition) of a dth-order tensor X is a d-dimensional vector r HOSV D = (r 1 , r 2 , . . ., r d ) such that r i = rank X {i} for all i ∈ [d], see [28].Thus, we can define an ideal J d,HOSVD generated by all second order minors of unfoldings X {k} , for k ∈ [d].

Convergence of the unit-θ k -norm balls
In this section we show the following result on the convergence of the unit θ k -balls .
Theorem 5.1.The theta body sequence of J d converges asymptotically to the conv (ν R (J)), i.e., To prove Theorem 5.1 we use the following result presented in [2] which is a consequence of Schmüdgen's Positivstellensatz.
Theorem 5.2.Let J be an ideal such that ν R (J) is compact.Then the theta body sequence of J converges to the convex hull of the variety ν R (J), in the sense that Proof of Theorem 5.1.The set ν R (J d ) is the set of rank-one tensors with unit Frobenius norm which can be written as ν R (J d ) = A 1 A 2 where It is well-known that A 1 is closed [11, discussion before Definition 2.2] and since A 2 is clearly compact, ν R (J d ) is compact.Therefore, the result follows from Theorem 5.2.

Computational Complexity
The computational complexity of the semidefinite programs for computing the θ 1 -norm of a tensor or for minimizing the θ 1 -norm subject to a linear constraint depends polynomially on the number of variables, i.e., on the size of B 2k , and on the dimension of the moment matrix M. We claim that the overall complexity scales polynomially in n, where for simplicity we consider dth-order tensors in R n×n×•••×n .Therefore, in contrast to tensor nuclear norm minimization which is NP-hard for d ≥ 3, tensor recovery via θ 1 -norm minimization is tractable.
Indeed, the moment matrix M is of dimension (1 + n d ) × (1 + n d ) (see also (15) for matrices in R 2×2 ) and if a = n d denotes the total number of entries of a tensor X ∈ R n×•••×n , then the number of the variables is at most a•(a+1) 2 ∼ O(a 2 ) which is polynomial in a. (A more precise counting does not give a substantially better estimate.)

Numerical experiments
Let us now empirically study the performance of low rank tensor recovery via θ 1 -norm minimization via numerical experiments, where we concentrate on thirdorder tensors.Given measurements b = Φ(X) of a low rank tensor X ∈ R n1×n2×n3 , where Φ : R n1×n2×n3 → R m is a linear measurement map, we aim at reconstructing X as the solution of the minimization program min Z θ1 subject to Φ(Z) = b. ( As outlined in Section 2, the θ 1 -norm of a tensor Z can be computed as the minimizer of the semidefinite program min t,y t subject to M(t, y, Z) 0, Table 3.The matrices involved in the definition of the moment matrix M (t, y, X).Due to the symmetry only the upper triangle part of the matrices is specified.The other non-specified entries of the matrices M ∈ R (n1n2n3+1)×(n1n2n3+1) from the first column are equal to zero.The matrix M corresponds to the element g + J 3 of the θ-basis specified in the second column.The index I = (i, î, j, ĵ, k, k) is in the range of the last column.The function θ-basis position (p, q) in the matrix M pq Range of i, î, j, ĵ, k, k where M(t, y, X) = M B1 (t, X, y) is the moment matrix of order 1 associated to the ideal J 3 , see Theorem 4.2.This moment matrix for J 3 is explicitly given by and the matrices M 0 , M ijk and M p I are provided in Table 3.For p ∈ {2, 3, . . ., 9}, the function h p denotes an arbitrary but fixed bijection {1, 2, . . ., |M p |} → {(i, î, j, ĵ, k, k)}, where I = (i, î, j, ĵ, k, k) is in the range of the last column of Table 3.As discussed in Section 2 for the general case, the θ 1 -norm minimization problem ( 22) is then equivalent to the semidefinite program min t,y,Z t subject to M (t, y, Z) 0 and Φ(Z) = b.
For our experiments, the linear mapping is defined as (Φ (X)) k = X, Φ k , k ∈ [m], with independent Gaussian random tensors Φ k ∈ R n1×n2×n3 , i.e., all entries of Φ k are independent N 0, 1 m random variables.We choose tensors X ∈ R n1×n2×n3 of rank one as X = u ⊗ v ⊗ w, where each entry of the vectors u, v, and w is taken independently from the normal distribution N (0, 1).Tensors X ∈ R n1×n2×n3 of rank two are generated as the sum of two random rank-one tensors.With Φ and X given, we compute b = Φ(X), run the semidefinite program (23) and compare its minimizer with the original low rank tensor X.For a given set of parameters, i.e., dimensions n 1 , n 2 , n 3 , number of measurements m and rank r, we repeat this experiment 200 times and record the empirical success rate of recovering the original tensor, where we say that recovery is successful if the elementwise reconstruction error is at most 10 −6 .We use MATLAB (R2008b) for these numerical experiments, including SeDuMi 1.3 for solving the semidefinite programs.
Table 4 summarizes the results of our numerical tests for cubic and non-cubic tensors of rank one and two and several choices of the dimensions.Here, the number m 0 denotes the maximal number of measurements for which not even one out of 200 generated tensors is recovered and m 1 denotes the minimal number of measurements for which all 200 tensors are recovered.The fifth column in Table 4 represents the number of independent measurements which are always sufficient for the recovery of a tensor of an arbitrary rank.For illustration, we present the average cpu time (in seconds) for solving the semidefinite programs via SeDuMi 1.3 in the last column.Alternatively, the SDPNAL+ Matlab toolbox (version 0.5 beta) for semidefinite programming [61,63] allows to perform low rank tensor recovery via θ 1 -norm minimization for even higher-dimensional tensors.For example, with m = 95 measurement we managed to recover all rank-one 9 × 9 × 9 tensors out of 200 (each simulation taking about 5min).Similarly, rank-one 11 × 11 × 11 tensors are recovered from m = 125 measurements with one simulation lasting about 50min.Due to these large computation times, more elaborate numerical experiments have not been conducted in these scenarios.We remark that no attempt of accelerating the optimization algorithm has been made.This task is left for future research.Except for very small tensor dimensions, we can always recover tensors of rankone or two from a number of measurements which is significantly smaller than the dimension of the corresponding tensor space.Therefore, low rank tensor recovery via θ 1 -minimization seems to be a promising approach.Of course, it remains to investigate the recovery performance theoretically.
Figures 1 and 2 present the numerical results for low rank tensor recovery via θ 1norm minimization for Gaussian measurement maps, conducted with the SDPNAL+ toolbox.For fixed tensor dimensions n × n × n, fixed tensor rank r, and fixed number m of measurements 50 simulations are performed.We say that recovery is successful if the element-wise reconstruction error is smaller than 10 −3 .Figures 1a, 2a, 3a and 1b, 2b, 3b present experiments for rank-one and rank-two tensors, respectively.The vertical axis in all three figures represents the empirical success rate.In Figure 1 the horizontal axis represents the relative number of measurements,  to be more precise, for a tensor of size n × n × n, the number n on the horizontal axis represents m = n n 3 100 measurements.In Figure 2 for a rank-r tensor of size n × n × n and the number of measurements m, the horizontal axis represents the number m/(3nr).Notice that 3nr represents the degrees of freedom in the corresponding CP-decomposition.In particular, if the number of measurements necessary for tensor recovery is m ≥ 3Crn, for an universal constant C, Figure 2 suggests that the constant C depends on the size of the tensor.In particular, it seems to grow slightly with n (although it is still possible that there exists C > 0 such that m ≥ 3Crn would always be enough for the recovery).With C = 3.3 we would always be able to recover a low rank tensor of size n × n × n with n ≤ 7. The horizontal axis in Figure 3 represents the number m/ (3nr • log(n)).The figure suggests that with the number of measurements m ≥ 6rn • log(n) we would always be able to recover a low rank tensor and therefore it may be possible that a logarithmic factor is necessary.The computation is implemented in MATLAB R2016a, on an Acer Laptop with CPU@1.90GHz and RAM 4GB.
We remark that we have used standard MATLAB packages for convex optimization to perform the numerical experiments.To obtain better performance, new optimization methods should be developed specifically to solve our optimization problem, or more generally, to solve the sum-of-squares polynomial problems.We  Once a monomial ordering is fixed, the meaning of leading monomial, leading term and leading coefficient of a polynomial (see Section 2) is well-defined.For more information on monomial orderings, we refer the interested reader to [14,15].
A Gröbner basis is a particular kind of generating set of a polynomial ideal.It was first introduced in 1965 in the Phd thesis of Buchberger [5].
Definition A.2 (Gröbner basis).For a fixed monomial order, a basis G = {g 1 , . . ., g s } of a polynomial ideal J ⊂ R [x] is a Gröbner basis (or standard basis) if for all f ∈ R [x] there exist a unique r ∈ R [x] and g ∈ J such that f = g + r and no monomial of r is divisible by any of the leading monomials in G, i.e., by any of the monomials LM (g 1 ) , LM (g 2 ) , . . ., LM (g s ).
A Gröbner basis is not unique, but the reduced version defined next is.
Definition A.3.The reduced Gröbner basis for a polynomial ideal J ∈ R [x] is a Gröbner basis G = {g 1 , g 2 , . . ., g s } for J such that 1) LC(g i ) = 1, for all i ∈ [s].
2) g i does not belong to LT(G\{g i }) for all i ∈ [s].
In other words, a Gröbner basis G is the reduced Gröbner basis if for all i ∈ [s] the polynomial g i ∈ G is monic (i.e., LC(g i ) = 1) and the leading monomial LM(g i ) does not divide any monomial of g j , j = i.
Many important properties of the ideal and the corresponding algebraic variety can be deduced via its (reduced) Gröbner basis.For example, a polynomial belongs to a given ideal if and only if the unique r from the Definition A.2 equals zero.Gröbner bases are also one of the main computational tools in solving systems of polynomial equations [15].
With f F we denote the remainder on division of f by the ordered k-tuple F = (f 1 , f 2 , . . ., f k ).If F is a Gröbner basis for an ideal f 1 , f 2 , . . ., f k , then we can regard F as a set without any particular order by Definition A.2, or in other words, the result of the division algorithm does not depend on the order of the polynomials.Therefore, f G = r in Definition A.2.
The following result follows directly from Definition A.2 and the polynomial division algorithm [15].
Corollary A.4. Fix a monomial ordering and let G = {g 1 , g 2 , . . ., g s } ⊂ R [x] be a Gröbner basis of a polynomial ideal J.A polynomial f ∈ R [x] is in the ideal J if it can be written in the form f = a 1 g 1 + a 2 g 2 + . . .+ a s g s , where a i ∈ R [x], for all i ∈ [s], s.t.whenever a i g i = 0 we have multideg (f ) ≥ multideg (a i g i ) .
Definition A.5. Fix a monomial order and let G = {g 1 , g 2 , . . ., g s } ⊂ R [x].Given f ∈ R [x], we say that f reduces to zero modulo G and write f → G 0 if it can be written in the form f = a 1 g 1 + a 2 g 2 + . . .+ a k g k with a i ∈ R [x] for all i ∈ [k] s.t.whenever a i g i = 0 we have multideg (f ) ≥ multideg (a i g i ).
Assume that G in the above definition is a Gröbner basis of a given ideal J. Then a polynomial f is in the ideal J if and only if f reduces to zero modulo G.In other words, for a Gröbner basis G, The Gröbner basis of a polynomial ideal always exists and can be computed in a finite number of steps via Buchberger's algorithm [5,14,15].
Next we define the S-polynomial of given polynomials f and g which is important for checking whether a given basis of the ideal is a Gröbner basis.
Definition A.6.Let f, g ∈ R [x] be a non-zero polynomials.
(2) The S-polynomial of f and g is the combination The following theorem gives a criterion for checking whether a given basis of a polynomial ideal is a Gröbner basis.
Theorem A.7 (Buchberger's criterion).A basis G = {g 1 , g 2 , . . ., g s } for a polynomial ideal J ⊂ R [x] is a Gröbner basis if and only if S (g i , g j ) → G 0 for all i = j.
Computing whether S (g i , g j ) → G 0 for all possible pairs of polynomials in the basis G can be a tedious task.The following proposition tells us for which pairs of polynomials this is not needed.
Therefore, to prove that the set G ⊂ R [x] is a Gröbner basis, it is enough to show that S (g i , g j ) → G 0 for those i < j where LM (g i ) and LM (g j ) are not relatively prime.
4 means that the polynomial 1 − m i=1 n j=1 x ij M ji is 1-sos mod J Mmn for all M ∈ O c ∪ O r .We start by fixing M = I m 0 in case m ≤ n and M = I n 0 in case m > n,

Theorem 4 . 4 .
The set G d forms the reduced Gröbner basis of the ideal J d with respect to the grevlex order.Proof.Again, we use Buchberger's criterion stated in Theorem A.7. First notice that the polynomials g d and f (α,β) d are always relatively prime, since LM(g d ) = x 2 11...1 and LM(f
Recovery of rank-2 tensors