Appendix: Matrix product states
We recall here two representations of tensors that are inspired from physics [23]. The corresponding notions have found applications in deep learning where they are called tensor train or tensor ring decompositions [22, 28, 29, 32].
The symbol \(\mathbb {K}\) denotes the field \(\mathbb {C}\) or \(\mathbb {R}\). For any \(a \in \mathbb {Z}_{>0}\), the vector space \(\mathbb {K}^{a}\) comes with the standard basis \(e_{1},\dots ,e_{a}\). Therefore, a tensor \(T\in \mathbb {K}^{a_{1}}\times \dots \times \mathbb {K}^{a_{q}}\) may be represented as
$$ T=\sum\limits_{1\leq i_{j}\leq a_{j}} \lambda_{i_{1},\dots,i_{q}} e_{i_{1}}\otimes\dots\otimes e_{i_{q}}, $$
which is also written
$$ T[i_{1},\dots,i_{q}]=\lambda_{i_{1},\dots,i_{q}}. $$
Definition A.1 (Site-independent (cyclic) matrix product state)
Fix integers r > 0, k > 0, q > 1, and matrices \(M_{i}\in \mathbb {K}^{r\times r}\) for \(i=1,\dots ,k\). Let \(T\in (\mathbb {K}^{k})^{\otimes q}\) be a tensor given by
$$ T[i_{1},\dots,i_{q}]:=\text{tr}(M_{i_{1}}M_{i_{2}}{\cdots} M_{i_{q}}). $$
The set of all tensors that allow such a representation will be denoted by \(\text {IMPS}(r,k,q)\subset (\mathbb {K}^{k})^{\otimes q}\).
Example A.1
Let us consider the case of matrices (q = 2). Here elements of IMPS(r, k, 2) can be viewed as matrices M such that \(M[i_{1},i_{2}]=\text {tr}(M_{i_{1}}M_{i_{2}})\). This is equivalent to a factorization of M = A ⋅ At for some matrix \(A\in \text {Hom}(\mathbb {K}^{r},\mathbb {K}^{k^{2}})\). In particular, M ∈IMPS(r, k, 2) if and only if M is symmetric and has ranked at most k2. It follows that IMPS(r, k, 2) is closed.
When q = 2, the tensor T corresponds to a symmetric matrix. However, for q > 2, the tensor T will not be a symmetric tensor in general, though the identity \(T[i_{1},\dots ,i_{q}]=T[i_{q},i_{1},\dots ,i_{q-1}]\) continues to hold. In other words, the tensor has cyclic symmetries with respect to the order of the product of the matrices.
Definition A.1 can be regarded as a symmetrization of the following definition of a cyclic matrix product state, where the underlying graph for the tensor network is a cycle.
Fix an integer q > 1 and tuples of positive integers \(\mathfrak {a} = (a_{1},\dots ,a_{q})\), \(\mathfrak {b}=(b_{1},\dots ,b_{q})\). We set aq+ 1 = a1. Then the locus \(\text {MPS}(\mathfrak {a},\mathfrak {b},q) \subset \mathbb {K}^{b_{1}}\otimes {\dots } \otimes \mathbb {K}^{b_{q}}\) is given by the following definition.
Definition A.3 (Cyclic matrix product state)
A tensor \(T \in \mathbb {K}^{b_{1}}\otimes {\dots } \otimes \mathbb {K}^{b_{q}}\) is in \(\text {MPS}(\mathfrak {a},\mathfrak {b},q)\) if there exist matrices
$$ M_{i,j}\in \text{Hom}_{\mathbb{K}}(\mathbb{K}^{a_{j}},\mathbb{K}^{a_{j+1}}),\quad j=1,\dots,q, i = 1,\dots,b_{j}, $$
such that
$$ T[i_{1},\dots,i_{q}]:=\text{tr}(M_{i_{1},1}M_{i_{2},2}{\cdots} M_{i_{q},q}). $$
Example A.4
The situation for q = 2 is analogous to Example A.2. In this case, we have M ∈MPS((a1, a2), (b1, b2), 2) if and only if M = AB where \(A\in \text {Hom}({\mathbb {K}^{b}_{1}},\mathbb {K}^{a_{1}a_{2}})\) and \(B\in \text {Hom}(\mathbb {K}^{a_{1}a_{2}},\mathbb {K}^{b_{2}})\). This can happen if and only if the rank of the matrix M is at most a1a2. Therefore, \(\text {MPS}(\mathfrak {a},\mathfrak {b},2)\) is always closed.
Proposition A.5
The sets IMPS and MPS may be represented as
- 1.
\(\text {IMPS}(r,k,q)=\{f^{\otimes q}(\mathfrak {M}_{r,\dots ,r}) \mid f\in \text {Hom}(\mathbb {K}^{r\times r},\mathbb {K}^{k})\}\).
- 2.
\(\text {MPS}(\mathfrak {a},\mathfrak {b},q)=\{(f_{1}\otimes {\dots } \otimes f_{q})(\mathfrak {M}_{a_{1},\dots ,a_{q}}) \mid f_{i} \in \text {Hom}(\mathbb {K}^{a_{i} \times a_{i+1}},\mathbb {K}^{b_{i}})\}\).
Proof
The proofs of both statements are similar. We prove the first one, as it is more important for this paper. We will be interpreting elements of \(\text {Hom}(\mathbb {K}^{r\times r},\mathbb {K}^{k})\) as r2 × k matrices. First, we note that there is a natural bijection φ between k-tuples of r × r matrices \(\mathcal {M}:=(A_{1},\dots ,A_{k})\) and matrices \(\varphi (\mathcal {M})\in \text {Hom}(\mathbb {K}^{r\times r},\mathbb {K}^{k})\). For 1 ≤ i ≤ k, the i th column of \(\varphi (\mathcal {M})\) is the representation of Ai as a vector of length r2.
Write \(M_{i}={\sum }_{p,q=1}^{r} a_{i,p,q} e_{p,q}\), where ep, q is the matrix with 1 in its (p, q)th entry and zeros everywhere else. Note that \(\varphi (\mathcal {M})(e_{p,q})=a_{i,p,q}\).
We prove the claim by showing that the tensor T ∈IMPS(r, k, q) associated to \(\mathcal {M}\) equals \(\varphi (\mathcal {M})(\mathfrak {M}_{r,\dots ,r})\). Indeed, we have
$$ \begin{array}{@{}rcl@{}} T &=&\sum\limits_{1\leq i_{j}\leq k} \text{tr}(M_{i_{1}}{\cdots} M_{i_{q}}) e_{i_{1}}\otimes\dots\otimes e_{i_{q}} \\ &=& \sum\limits_{1\leq i_{j}\leq k}\left( \sum\limits_{1\leq p_{j}\leq r} a_{i_{1},p_{1},p_{2}}a_{i_{2},p_{2},p_{3}}\cdots a_{i_{q-1},p_{q-1},p_{q}}a_{i_{q},p_{q},p_{1}}\right)e_{i_{1}}\otimes\dots\otimes e_{i_{q}}, \end{array} $$
where in all sums 1 ≤ j ≤ q. We can simplify further:
$$ \begin{array}{@{}rcl@{}} T&= &\sum\limits_{\begin{array}{llll}1\leq i_{j}\leq k \\ 1\leq p_{j}\leq r \end{array}}(a_{i_{1},p_{1},p_{2}}e_{i_{1}})\otimes\cdots\otimes (a_{i_{q},p_{q},p_{1}}e_{i_{q}}) \\ &=&\sum\limits_{1\leq p_{j}\leq r}\left( \sum\limits_{1\leq i_{1}\leq k}a_{i_{1},p_{1},p_{2}}e_{i_{1}}\right)\otimes\cdots\otimes \left( \sum\limits_{1\leq i_{q}\leq k}a_{i_{q},p_{q},p_{1}}e_{i_{q}}\right) \\ &=&\sum\limits_{1\leq p_{j}\leq r}\varphi(\mathcal{M})(e_{p_{1},p_{2}})\otimes\cdots\otimes\varphi(\mathcal{M})(e_{p_{q},p_{1}}) \\ &=&\varphi(\mathcal{M})^{\otimes q}(\mathfrak{M}_{r,\dots,r}). \end{array} $$
□