An ATLD–ALS method for the trilinear decomposition of large third-order tensors

Abstract

CP decomposition of large third-order tensors can be computationally challenging. Parameters are typically estimated by means of the ALS procedure because it yields least-squares solutions and provides consistent outcomes. Nevertheless, ALS presents two major flaws which are particularly problematic for large-scale problems: slow convergence and sensitiveness to degeneracy conditions such as over-factoring, collinearity, bad initialization and local minima. More efficient algorithms have been proposed in the literature. They are, however, much less dependable than ALS in delivering stable results because the increased speed often comes at the expense of accuracy. In particular, the ATLD procedure is one of the fastest alternatives, but it is hardly employed because of the unreliable nature of its convergence. As a solution, multi-optimization is proposed. ATLD and ALS steps are concatenated in an integrated procedure with the purpose of increasing efficiency without a significant loss in precision. This methodology has been implemented and tested under realistic conditions on simulated data sets.

Appendix

In all simulations carried out in this manuscript, synthetic data are generated in the same fashion. The rank R and the dimensions of the artificial tensor I, J and K are set to preference; then, the loading matrices $\mathbf{A } \in {\mathbb {R}}^{ I \times R}$, $\mathbf{B } \in {\mathbb {R}}^{ J \times R}$ and $\mathbf{C } \in {\mathbb {R}}^{ K\times R}$ are generated from a uniform distribution.

Successively, a set value is assigned to the congruence among factors of each loading matrix (CONG). This is achieved by first orthogonalizing the random matrices by means of the QR method and then replacing the upper triangular matrix with the output of a Cholesky decomposition of the square matrix $( R \times R)$ with 1s on the diagonal and the parameter CONG everywhere else. A pure tensor is thus reconstructed using Eq. 1$\tilde{\mathscr {T}}^{I,J,K}$; then, noise contamination is added.

Specifically, two tensors $\mathscr {E}_{\mathrm{HO}}$ and $\mathscr {E}_{\mathrm{HE}}$, containing homoscedastic and heteroscedastic residuals are summed to the noise-free reconstructed data. These error tensors are generated as normally distributed values; however, in the case of $\mathscr {E}_{HE}$, the random values are multiplied by the elements of the pure tensor to ensure different weights.

The percentage of noise contamination HO and HE is specified in terms of proportion of the total variation of $\tilde{\mathscr {T}}^{I,J,K}$, by first imposing a Frobenius norm of $\sum _{k=1}^{K}\tilde{{\mathbf {T}}}_{::k}^{2}$ and then multiplying the array by an appropriate scalar:

$$\begin{aligned} \mathscr {T}^{I,J,K}{=}\tilde{\mathscr {T}}^{I,J,K}{+} \sqrt{\frac{1-\mathrm{HO}}{\mathrm{HO}}} \mathscr {E}_{\mathrm{HO}}{+} \sqrt{\frac{1-{\mathrm{HE}}}{\mathrm{HE}}} \mathscr {E}_{HE} \end{aligned}$$

(11)

All calculations were carried out using R language (R Core Team 2012), version 3.5.0, processor 2,3 GHz Intel Core i7, and all procedure were written using base functions and the ThreeWay (Giordani et al. 2014) and multiway (Helwig 2017) packages.