On the interconnection between the higher-order singular values of real tensors

A higher-order tensor allows several possible matricizations (reshapes into matrices). The simultaneous decay of singular values of such matricizations has crucial implications on the low-rank approximability of the tensor via higher-order singular value decomposition. It is therefore an interesting question which simultaneous properties the singular values of different tensor matricizations actually can have, but it has not received the deserved attention so far. In this paper, preliminary investigations in this direction are conducted. While it is clear that the singular values in different matricizations cannot be prescribed completely independent from each other, numerical experiments suggest that sufficiently small, but otherwise arbitrary perturbations preserve feasibility. An alternating projection heuristic is proposed for constructing tensors with prescribed singular values (assuming their feasibility). Regarding the related problem of characterising sets of tensors having the same singular values in specified matricizations, it is noted that orthogonal equivalence under multilinear matrix multiplication is a sufficient condition for two tensors to have the same singular values in all principal, Tucker-type matricizations, but, in contrast to the matrix case, not necessary. An explicit example of this phenomenon is given.


Introduction and problem statement
A space R n 1 ⊗ · · · ⊗ R n d of higher-order tensors is isomorphic to many different matrix spaces of the form ( j∈t R n j ) ⊗ ( i / ∈t R n i ) where t {1, . . . , d}, |t| ≥ 1. Concretely, when identifying tensors with d-dimensional arrays of coordinates with respect to an orthonormal tensor product basis, such an isomorphism is realized by reshaping the array into a matrix. The directions in t indicate the multi-indices for the rows of the resulting matrix, while the other directions are used for the columns. All these different matricizations (also called unfoldings or reshapes in the literature) of the tensor carry some spectral information in form of their singular value decompositions.
For subsets of t that are part of a dimension partition tree, the column spaces of the corresponding matricizations satisfy certain nestedness properties that form the basis for important subspace based low-rank tensor decompositions like the Tucker format [22], the hierarchical Tucker (HT) format [7,9], or the tensor train (TT) format [18,19]. As a by-product, the ranks r t of the corresponding matricizations, that is, the number of nonzero singular values, are estimated as r t ≤ r t 1 · · · r t s , where t = t 1 ∪ · · · ∪ t s is a disjoint partition. In contrast, the interconnections between the singular values themselves have not been studied so far.
At first sight, the singular values of different matricizations could be considered as unnatural or artificial characteristics for tensors, as they ignore their multilinear nature. However, as it turns out, they provide crucial measures for the approximability of tensors in the aforementioned low-rank subspace formats. In the pioneering work [3] the higher-order singular value decomposition has been defined, and it has been shown how it can be practically used to obtain quasi-optimal low-rank approximations in the Tucker format with full error control. The approximation is obtained by an orthogonal projection on the tensor product of subspaces spanned by the dominating singular vectors of the corresponding matricizations in R n j ⊗ ( i = j R n i ) (i.e. corresponding to the choices t = { j} for j = 1, . . . , d). An upper bound of the squared error is then given by the sum of squares of all deleted singular values in all directions. Later, variants of such truncation procedures have been obtained for the TT format [16,17] and the HT format [7] with similar error bounds, but involving singular values of some other matricizations of the tensor.
Building on these available, quasi-optimal bounds for low-rank approximations via higher-order versions of SVD truncation, it is understandable that quite some theorems have been stated making simultaneous assumptions on the singular values of certain matricizations of a tensor. This concerns stability of low-rank ODE integrators [12,14], local convergence of optimization algorithms [20], or approximability by low-rank tensor formats [1], to name a few. Assumed properties of interest are decay rate of and gaps between the singular values, for instance. A principal task would then be to give alternative descriptions of classes of tensors satisfying such assumptions to prevent tautological results or, in worst case, void statements. But this task has turned out to be notoriously difficult. For tensors arising from function discretization, some qualitative statements about the decay of singular values can be obtained from their regularity using explicit analytic approximation techniques by tensor products of (trigonometric) polynomials or wavelets, exponential sum, or cross approximation; see [8,21] and references therein. But these qualitative implications on the decay of singular values obtained from explicit separable approximations can rarely be made quantitatively precise, for instance, if they contain unknown constants, and also provide little insight on the actual interconnection between different matricizations.
In its purest form the question we are interested in is very simple to state. Given prescribed singular values for some matricizations (having, e.g., some favourable properties), does there exist at all a tensor having these singular values? For a matrix this is of course very easy to answer by simply constructing a diagonal matrix. For tensors it turns out to be quite difficult, and seems to depend on how many matricizations are simultaneously considered.
In the present paper we study this and related questions for the singular values related to the classical higher-order SVD, that is, the singular values of the principal, Tucker-type matricizations that separate single indices t = { j}. We call the collection of the d corresponding singular value vectors the higher-order singular values, see Definition 1.1 below. Compared to other systems of matricizations, this framework is the historically most important. It also appears to be the simplest, partly because it is, at least to some extent, very conveniently possible to manipulate the principal matricizations simultaneously via multilinear matrix multiplications. Yet, even in this case, the obtained results remain fragmentary and far from complete. Nevertheless, we consider them as valuable first steps toward future investigations of this important and fascinating subject. Our contributions are as follows.
-We show that not all configurations of higher-order singular values can be feasible.
The proof is nonconstructive (Sect. 3.1). -However, conducted numerical experiments suggest that the singular values for different matricizations are, except for degenerate situations, locally independent from each other. That is, in the neighbourhood of a tensor it is possible to slightly perturb, say, only the singular values of the first matricization, while maintaining the singular values of the other ones. This is fundamentally different from the matrix case, since the singular values of a matrix are always the same as the ones of its transpose. However, currently this remains an unproved conjecture Sect. 3.2). -We propose the method of alternating projections as a heuristic to construct (approximately) tensors with prescribed singular values in certain matricizations (Sect. 3.3). -The higher-order SVD (HOSVD) is a generalization of the SVD from matrices to tensors. The role of the diagonal matrix of singular values is replaced by the core tensor in the HOSVD, representing the normal form under orthogonal equivalence, and characterized by slice-wise orthogonality properties. We show manifold properties of the set of these core tensors (called HOSVD tensors) in the case of strictly decreasing and positive higher-order singular values (Sect. 2.2). -We provide an example of two 2 × 2 × 2 tensors having the same singular values in all three principal matricizations without being orthogonally equivalent (Sect. 2.4).
In this paper we consider real tensors for convenience. Although most concepts seem to generalize to the complex case, some care would be required, e.g., when switching from smooth manifolds to analytic ones.
The rest of this section is devoted to the precise statements of the considered problems. They require some amount of definitions and notation, which will be introduced first.

Preliminaries, definitions, notation
Let d ≥ 3 and n 1 , . . . , n d ∈ N. We consider the d-fold tensor product space R n 1 ⊗· · ·⊗ R n d as isomorphic to the space R n 1 ×···×n d of real n 1 ×· · ·×n d arrays (sometime called hyper-matrices). The entries X i 1 ,...,i d of X ∈ R n 1 ×···×n d will be indexed by multiindices (i 1 , . . . , i d ), with every i j taking values between 1 and n j . For convenience, it will be assumed is called the vector of mode-j singular values. The tuple is called the set of higher-order singular values of the tensor X. 2. Correspondingly, for j = 1, . . . , d, the vector λ ( j) is called the vector of mode-j Gramian eigenvalues. The tuple is called the set of higher-order Gramian eigenvalues of the tensor X. 3. The multilinear rank of the tensor X is the tuple r X = (r (1) X ) being equal to the number of nonzero entries of σ ( j) X . 4. The tensor X is called non-singular, if r X = (n 1 , . . . , n d ).
We note that for matrices the definition of 'non-singular' coincides with the usual definition (in particular, it enforces n 1 = n 2 ). In general, the following is true. Let · F denote the Frobenius norm of matrices and tensors, and · 2 the standard Euclidean norm for vectors. Since matricization of a tensor is an isometric isomorphism in Frobenius norm, and since it holds M ( j) X 2 , an obvious observation for higher-order singular values is σ (1) X 2 = · · · = σ (d) Therefore, we can focus in the following on tensors X on the unit sphere S = X ∈ R n 1 ×···×n d : X F = 1 , and hence higher-order singular values in the set ≥ denotes the set of all nonnegative, decreasingly ordered vectors on the Euclidean unit sphere in R n j . For most results, however, it will be necessary to further restrict to the set S * of non-singular tensors having strictly decreasing modej singular values in every direction j. Therefore, we also introduce the notation > contains the unit norm vectors in R n j with strictly decreasing and strictly positive entries. Then Note that we do not introduce a notation for the slightly larger set of all non-singular tensors in S . The main technical advantage of tensors in S * is that all principal unfoldings admit essentially unique singular value decompositions.
The following two facts are useful to know, and follow immediately from the matrix case. Proof The continuity of X as a function of X follows by isomorphy to R n j ×n c j from the continuity of each σ

Problem statement
Regarding the higher-order singular values of tensors a principle question of interest is the following one.

Problem 1.4 (Feasible higher-order singular values)
Given ∈ S ≥ , does there exist a tensor X ∈ S such that X = ?
Such will be called a feasible. We define In this generality, Problem 1.4 appears to be quite hard, and will not be satisfactorily solved in this article. At least, as a first result, we are able to show that not all ∈ S ≥ are feasible: F = S ≥ . The argument, however, is non-constructive, see Sect. 3.1.
A relaxed question of a more qualitative nature is the following one. Numerical experiments with random tensors seem to indicate that the answer to the second question could be positive when d ≥ 3, but we are not able to prove it. So it remains a conjecture. In fact, we conjecture that for every X in S * it holds that X is an interior point of F, see Sect. 3.2. A striking implication of this conjecture is that given X ∈ S * , its high-order singular values in different directions can be perturbed independently from each other without loosing feasibility (local independence of highorder singular values). In Sect. 3.3 we will present a heuristic approach to do this using an alternating projection method, which seems to work quite reliably for small perturbations, although we are currently neither able to prove its convergence nor that limit points must have the desired property.
To approach Problems 1.4 and 1.5, it seems useful to also study the following problem, which is of some interest in itself. Problem 1.6 (Tensors with same higher-order singular values) Given X ∈ S , characterize sets of tensors having the same singular values X as X.
The corresponding equivalence classes for tensors in S and S * are denoted by The next Sect. 2 provides some results related to Problem 1.6. It is observed that orbits of orthogonally equivalent tensors provide trivial examples of subsets of tensors having the same higher-order singular values. However, other than in the matrix case, their dimension is too small to provide a complete description. Via the tool of HOSVD tensors, which serve as normal forms in the orbits of orthogonally equivalent tensors, we are able to construct an example of two tensors with the same higher-order singular values that are not orthogonally equivalent.

Tensors with the same higher-order singular values
In this section we focus on equivalence classes of tensors having the same higher-order singular values.

Orthogonally equivalent tensors
We recall a fact from matrices: Two rectangular matrices X, Y ∈ R n×n , n ≤ n , have the same singular values, if and only if they are orthogonally equivalent, that is, if there exists U ∈ O(n) and V ∈ O(n ) such that This definition of orthogonal equivalence can be generalized to tensors using the multilinear matrix multiplication, see, e.g., [5,13]. We consider the product unitary group The left action (U (1) , . . . , U (d) ) · X of this group on R n 1 ×···×n d is defined as the canonical action of the tensor product operator In terms of matricizations, in a slight abuse of notation, we note that for j = 1, . . . , d. For matrices (d = 2), these formulas define orthogonal equivalence, which motivates the following generalization.

Proposition 2.2 If two tensors are orthogonally equivalent, then they have the same higher-order singular values.
In particular, the orbit of each X under the group action contains only tensors with identical higher-order singular values. (1) , . . . , U (d) ) · X, whose image is O · X. For X ∈ S \ S * the dimension of O(n 1 × · · · × n d ) · X can be smaller than dim O(n 1 × · · · × n d ). Note that we did not attempt to prove or disprove that the orbits are globally embedded submanifolds.

HOSVD tensors
The compact Lie group O(n 1 ×· · ·×n d ) acts freely on S * . It also acts properly (since it is compact and acts continuously). By a general theorem (e.g. [6, § 16.10.3]), the quotient manifold S * /O(n 1 ×· · ·×n d ) of equivalence classes exists, and the canonical mapping S * → S * /O(n 1 × · · · × n d ) is a submersion. A concrete realization of this abstract quotient manifold is the set H * of regular HOSVD tensors which is now introduced. Given In particular, the rows of (V ( j)

⊗· · ·⊗U
Y . Hence X has the specific property that is a diagonal matrix of decreasing eigenvalues. The reverse relation between X and Y is called the higher-order singular value decomposition (HOSVD) of Y and has been introduced by De Lathauwer et al. [3]. HOSVD tensors can be regarded as representatives of orbits O(n 1 × · · · × n d ) · X of orthogonally equivalent tensors. For X ∈ S * , the representatives are essentially unique as stated next. Here it is instructive to note that for square matrices, the set H * consists of regular diagonal matrices with strictly decreasing diagonal entries. Proposition 2.5 Let X, Y ∈ H * be two HOSVD tensors. If X and Y are orthogonally equivalent, that is, Y = (U (1) , . . . , U (d) ) · X, then the U ( j) must be diagonal orthogonal matrices (i.e. with values ±1 on the diagonal).
The proof is immediate from (2.2), (2.3), and the uniqueness of orthogonal diagonalization up to sign flipping in the case of mutually distinct eigenvalues. Comparing with the explicit form (2.1), we see that the action of (U (1) , . . . , U (d) ) · X with diagonal U ( j) with ±1 entries results in some sign flipping pattern for the entries of X. This provides the following, sometimes useful necessary condition.

Proposition 2.6
If two HOSVD tensors X, Y ∈ H * are orthogonally equivalent, then |X i 1 ,...,i d | = |Y i 1 ,...,i d |. In particular, X and Y have the same zero pattern.
We now turn to the manifold properties of H * .

Theorem 2.7 The set H * is a smooth embedded submanifold of S
Proof The formal setting is as follows. We denote by R n j ×n j sym the space of symmetric n j ×n j matrices, by R n j ×n j sym,0 the subspace with zeros on the diagonal, by π j (Z ) = Z − diag(Z ) the orthogonal projection from R n j ×n j sym onto R n j ×n j sym,0 , and p = π 1 × · · · × π d . Consider g : S * → R n 1 ×n 1 sym × · · · × R n d ×n d sym , X → (G Then f = p • g is a smooth map from S * to R n 1 ×n 1 sym,0 × · · · × R n d ×n d sym,0 , and, by Definition 2.4, we have Since dim(R n 1 ×n 1 sym,0 × · · · × R n d ×n d sym,0 ) = dim(O(n 1 × · · · × n d )) = d j=1 1 2 n j (n j − 1), the assertion will follow from the regular value theorem, if we show that f (X) is surjective for every X ∈ H * . To prove the latter, we show that the range of f (X) contains the spaces W j = {0} × · · · × {0} × R For brevity, we set I = I n 1 . Since ϕ(I ) = X, it follows from the chain rule that the range of f (X) contains the range of ( f • ϕ) (I ). We show that the latter equals W 1 .
X , . . . , (d) X ). Further noting that the tangent space to O(n 1 ) at I is the space R n 1 ×n 1 skew of skew-symmetric n 1 × n 1 matrices, we see that X H T , 0, . . . , 0).
As dim(W 1 ) = 1 2 n 1 (n 1 − 1) = dim(R n 1 ×n 1 skew ), it is enough to show that ( f • ϕ) (I ) is injective in order to finish the proof. This now follows from the fact that, by definition of H * , the diagonal entries of (1) X are strictly decreasing, as it implies that H X H cannot be diagonal for skew-symmetric H = 0. This, however, is equivalent to injectivity of ( f • ϕ) (I ) as given above.

Remark 2.8 In our definition (2.3) of HOSVD tensors we required the diagonal elements of G
( j) X to be decreasing. This has advantages and drawbacks. One advantage are the narrower uniqueness properties leading to the practical condition in Proposition 2.6. A disadvantage is that it is more difficult to design HOSVD tensors "by hand" as in Sect. 2.4. Alternatively, one may define a setH by just requiring the G ( j) X to be diagonal. Then for every X ∈H we have (P (1) , . . . , P (d) ) · X ∈ H , where P ( j) are permutation matrices that sort the diagonal entries of G ( j) X accordingly. For mutually distinct eigenvalues the choice of P ( j) is unique. The corresponding setH * is therefore the finite disjoint union of sets (P (1) , . . . , P (d) ) · H * over all P ( j) , and as such also an embedded submanifold of S * .

Degrees of freedom
A principal challenge in understanding the interconnection between higher-order singular values of tensors arises from the fact that, in contrast to the matrix case, the converse statement of Proposition 2.2 is in general not true when d ≥ 3. Tensors may have the same higher-order singular values without being orthogonally equivalent. This can be seen from the following heuristic.
The set S * is open and dense in S by Proposition 1.3, and therefore is a smooth manifold of dimension The set S > is an open subset of Cartesian products of spheres and hence of dimension dim(S > ) = (n 1 + · · · + n d ) − d.
Therefore, given X ∈ S * , we expect the set S * X of tensors having the same higherorder singular values as X to be at least of "dimension" dim(S * ) − dim(S > ) = (n 1 · · · n d ) − (n 1 + · · · + n 2 ) + (d − 1). When d ≥ 3, by Proposition 2.3, this set cannot only consist of tensors that are orthogonally equivalent to X. 1 In fact, for large d, the orthogonally equivalent tensors will only be a very "low-dimensional" subset of S * X .

A non-equivalent example
The previous considerations suggest that there must exist tensors having the same higher-order singular values without being orthogonally equivalent. We construct here an example of size 2 × 2 × 2 using Proposition 2.6. Let us shortly count the degrees of freedom in this situation. The Euclidean unit sphere S is of dimension seven, the set S ≥ of potential tuples of higher-order singular values is of dimension three, while orbits O(2 × 2 × 2) · X of orthogonally equivalent tensors are of dimension at most three, too. This indicates for every X ∈ S an at least one-dimensional set of non-equivalent tensor with same higher-order singular values. Using a common slice-wise notation of tensors, we consider (currently not normalized) The three matricizations are M (1) In all three matricizations the rows are orthogonal, and the norm of the first row is larger than the norm of the second one. This shows that X is a HOSVD tensor. Its squared higher-order singular values are In particular, X/ X F ∈ S * . As a second tensor consider One checks again that all three matricizations M i.e., matrices, it is the case: as (1.2) is assumed, we have n 1 = n 2 = n, and two square matrices have the same singular values if and only if they are orthogonally equivalent. The formula gives n 2 − 2n + 1 which, however, only equals n 2 − n − (n − 1) = dim(O(n × n)) − (n − 1). The reason is that in the matrix case we know that the singular values of X and X T are the same. Hence the feasible set is only of dimension n − 1, and not of dimension 2(n − 1) (the argument will be repeated in Sect. 3.1). For tensors, however, we conjecture that the dimension of F is indeed (n 1 + · · · + n d ) − d, see Sect. 3.2. have orthogonal rows with squared row norms This shows that X/ X F and Y/ Y F are two HOSVD tensors in S * with the same set of higher-order singular values. By Proposition 2.6, they are not orthogonally equivalent.

The set of feasible configurations
The set F = F(n 1 , . . . , n d ) ⊆ S ≥ of feasible configurations has been defined in (1.2).
In this section we investigate this set. A simple observation worth to mention is that F is closed. This follows from Proposition 1.3 and the compactness of S .
In fact, F is an n 2 -dimensional subset in the (n 1 + n 2 )-dimensional set S ≥ (n 1 , n 2 ). This known phenomenon in the matrix case can be used to give a qualitative proof that also for higher-order tensors not all configurations are feasible. To start, we recall a fact on the HOSVD from the literature. Let X have the left singular vector matrices U ( j) X (column-wise ordered by decreasing singular values), and multilinear rank r = (r 1 , . . . , r d ). Then we can write the "economic" HOSVD as X contains only the first r j columns of U ( j) X , and the core tensor C is of size r 1 × · · · × r d . The multilinear matrix product here corresponds to the action of the tensor product operatorÛ (1) X ⊗ · · · ⊗Û (d) X on R r 1 ⊗ · · · ⊗ R r d , the explicit formulas are similar to (2.1). Note that if X is non-singular, C is just an HOSVD tensor in the orthogonally equivalent orbit of X as defined above. The key observation in the general case is that C is non-singular in R r 1 ×···×r d , and its higher-order singular values in every direction are given by the nonzero higher-order singular values of X [3].
Based on this fact, we can first give trivial examples of singular tensors for which the nonzero singular values in different directions are not independent of each other. X have multilinear rank r = (r 1 , . . . , r d ). Assume r j = 1 for j ≥ 3. Then r 1 = r 2 and {σ (1) 1 , . . . , σ (1)

Lemma 3.1 Let
Proof Let C ∈ R r 1 ×···×r d be the economic HOSVD core tensor of X. The matricizations M Since tensors with r j = 1 for j ≥ 3 considered in the previous lemma are naturally identified as elements of R n 1 ⊗ R n 2 , that is, as matrices, the previous statement may appear rather odd at first. However, using a perturbation argument, it leads to a nonconstructive proof that non-feasible configurations for higher-order singular values do exist even in the non-singular case. In fact, these configurations are of positive volume within S ≥ . Theorem 3.2 For n 1 ≥ n 2 , consider σ (1) ∈ S (1) ≥ and σ (2) ∈ S (2) ≥ such that σ (1) = (σ (2) , 0, . . . , 0), (3.1) where the number of appended zeros on the right side equals n 1 − n 2 . Let further O ( j) , but of σ (1) and σ (2) , respectively. Then there exists > 0 such that Proof Assume to the contrary that for every n there exists a tensor X n ∈ S such that X n ∈ O (1) 1/n × · · · × O (d) 1/n . The sequence of X n has a convergent subsequence with a limit X ∈ S . By Lemma 1.3, X has higher-order singular values X = (σ (1) , σ (2) , e (3) 1 , . . . , e (d) 1 ). Now Lemma 3.1 applies, but is in contradiction to (3.1). Remark 3.3 The condition (3.1) can hold in two cases: (i) the number of nonzero singular values in direction one and two are the same (r 1 = r 2 ), but the singular values themselves are not, or (ii) r 1 = r 2 . The second case has some interesting implications for rectangular tensors. Assume for instance n 1 = n 2 . Then by Theorem 3.2 there cannot exist normalized non-singular tensors in R n 1 ×···×n d for which the singular value vectors σ ( j) in directions j = 3, . . . , d are arbitrarily close to the corresponding unit vector e ( j) . This surprising connection between mode sizes of the tensor and location of the singular value vectors is not obvious, especially given the fact that almost every tensor is non-singular (assuming (1.2)).

A conjecture on interior points
For d = 2 we have seen that F(n 1 , n 2 ) is a set of measure zero within S ≥ (n 1 , n 2 ), even when n 1 = n 2 . One question is whether this is also true for higher-order tensors. Remarkably, the following experiment suggests that this does not need to be the case.
containing the largest singular values of the three directions for 10,000 random tensors X ∈ S . They seem to form a three-dimensional connected set. Hence, the corresponding set of X = (σ (1) X , σ (1) X , σ (1) X ) should be of positive volume in the three-dimensional set S ≥ We generate random 2 × 2 × 2 tensors X of Frobenius norm one. 2 With probability one, the higher-order singular values X = (σ (1) X , σ (2) X , σ (3) X ) are elements of S > (2, 2, 2), which is a set of three dimensions and therefore can be visualized. We simply make the identification, that is, we project on the first coordinate of each singular value vector. In Fig. 1 we see these projected points for 10,000 random examples, and their convex hull computed with a Matlab integrated Delauney triangulation. As the resulting point cloud appears three-dimensional, we suppose that the set of feasible configurations is also three-dimensional. But one can also verify in the plot that not all configurations are feasible. Above we made use of the fact that σ ( j) = (1, 0) (Tucker rank in the direction j equals one) implies σ (i) = σ (k) for i, k = j. This can be seen in the picture as the convex polytope intersects the hyperplanes x = 1, y = 1 and z = 1 in single one-dimensional facets of 45 degree.
We are led to the following conjecture. In fact, the following seems likely (under the same assumptions).
Remark 3.6 During revision of the paper, a possible strategy to prove this conjecture has been revealed. It is based on the observation that X is a relative interior point of F(n 1 , . . . , n d ) if and only if the map g(X) = (G (1) X , . . . , G (d) X ) (that has already been considered in (2.4)) is locally surjective when regarded as a map from the unit sphere S to the Cartesian product of hyperplanes {A ( j) ∈ R n j ×n j sym : tr(A ( j) ) = 1}. In other words, one has to show that the rank of the derivative g (X), when restricted to the tangent space T X S , equals the maximum possible value α = d j=1 1 2 n j (n j + 1) − d. A sufficient condition for this is that g (X) is of rank α + 1 on R n 1 ×···×n d . However, as g (X) depends polynomially on the entries of X, the function X → rank(g (X)) achieves its maximum value for almost all X. Since it is bounded by α+1, it is therefore enough to find a single tensor X for which rank α + 1 is achieved. In this way, one can validate Conjecture 3.5 for different configurations of n 1 , . . . , n d by constructing random X and evaluating the rank of g (X) numerically. A rigorous proof would have to confirm this numerical rank for "simple" candidates X, which we were able to do for 2 × 2 × 2 tensors so far. This approach shall be subject of a future work.

Alternating projection method
Even in the case that one would be given the information that a configuration = (σ (1) , . . . , σ (d) ) ∈ S ≥ is feasible, the question remains how to construct a corresponding tensor. Note that the suggested strategy to prove Conjecture 3.5 by showing full rank of (2.4) may not provide an explicit way for perturbing singular values in single directions.
A (currently) heuristic approach can be taken via the method of alternating projections. It is based on an alternative viewpoint on Problem 1.4: Given σ ( j) ∈ S ( j) ≥ for j = 1, . . . , d, the configuration = (σ (1) , . . . , σ (d) ) is feasible, if and only if there exists a tensor X such that where M ( j) σ denotes the set of all tensors with modej singular values σ . More concretely, The method of alternating projections tries to find X satisfying (3.2) by successively projecting on the sets M ( j) σ ( j) . It hence takes the form (1) σ (1)

Perturbation of a feasible configuration
To support Conjecture 3.5, we now consider random perturbations = σ (1) , . . . , σ (d) = + O( ) ∈ S ≥ of a known feasible configuration ∈ F ∩ S > (obtained again from a random tensor X ∈ S * ). 3 According to the conjecture, we expect that for small the configuration is also feasible, so a corresponding tensor may be found by the alternating projection method (3.3). This can be verified in numerical experiments. The left plot in Fig. 3 shows the errors σ ( j) F for one experiment in R 10×10×10×10 using = 10 −3 .

Infeasible configuration
When conducting our experiments with the alternating projection method, we made the experience that with high probability even a randomly generated configuration will be feasible. Indeed, Fig. 1 supports this in the 2 × 2 × 2 case, as the feasible configurations seem to make up a rather large fraction in S ≥ (2, 2, 2).
To construct an infeasible configuration we therefore mimic the proof of Theorem 3.2: we generate σ ( j) as (1, 0, . . . , 0) + O( j ) (as described in Footnote 3), where we use very small j for j ≥ 3, e.g., j = 10 −6 . By the arguments presented above this should also enforce σ (1) to be close to σ (2) to ensure feasibility. To impede this, we use larger 1 and 2 instead, e.g., 1 = 2 = 10 −3 (an alternative would be to generate σ (1) and σ (2) completely random). Our results suggest that this indeed results in an infeasible configuration. Accordingly, the alternating projection method fails. The right plot in Fig. 3 shows the outcome of one experiment, again in R 10×10×10×10 .