A Modified Particle Swarm Optimization Algorithm for the Best Low Multilinear Rank Approximation of Higher-Order Tensors

  • Pierre B. Borckmans
  • Mariya Ishteva
  • Pierre-Antoine Absil
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6234)


The multilinear rank of a tensor is one of the possible generalizations for the concept of matrix rank. In this paper, we are interested in finding the best low multilinear rank approximation of a given tensor. This problem has been formulated as an optimization problem over the Grassmann manifold [14] and it has been shown that the objective function presents multiple minima [15]. In order to investigate the landscape of this cost function, we propose an adaptation of the Particle Swarm Optimization algorithm (PSO). The Guaranteed Convergence PSO, proposed by van den Bergh in [23], is modified, including a gradient component, so as to search for optimal solutions over the Grassmann manifold. The operations involved in the PSO algorithm are redefined using concepts of differential geometry. We present some preliminary numerical experiments and we discuss the ability of the proposed method to address the multimodal aspects of the studied problem.


Multi-Linear Rank Higher-Order Tensors Particle Swarm Optimization Grassmann Manifold Global Optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Absil, P.A., Mahony, R., Sepulchre, R.: Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Appl. Math. 80(2), 199–220 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)zbMATHGoogle Scholar
  3. 3.
    Acar, E., Bingol, C.A., Bingol, H., Bro, R., Yener, B.: Multiway analysis of epilepsy tensors. In: ISMB 2007 Conference Proceedings, Bioinformatics, vol. 23(13), pp. i10–i18 (2007)Google Scholar
  4. 4.
    Audet, C., Dennis Jr., J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. on Optimization 17(1), 188–217 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Brits, R., Engelbrecht, A., van den Bergh, F.: A niching particle swarm optimizer. In: Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution and Learning (SEAL 2002), vol. 2, pp. 692–696 (2002)Google Scholar
  6. 6.
    De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    De Lathauwer, L., De Moor, B., Vandewalle, J.: On the best rank-1 and rank-(R 1,R 2,...,R N) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21(4), 1324–1342 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    De Lathauwer, L., Vandewalle, J.: Dimensionality reduction in higher-order signal processing and rank-(R 1,R 2,...,R N) reduction in multilinear algebra. Linear Algebra Appl. 391, 31–55 (2004); Special Issue on Linear Algebra in Signal and Image ProcessingGoogle Scholar
  9. 9.
    Dreisigmeyer, D.W.: Direct search algorithms over Riemannian manifolds (December 2006) (optimization Online 2007-08-1742)Google Scholar
  10. 10.
    Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Engelbrecht, A.P.: Fundamentals of Computational Swarm Intelligence. John Wiley & Sons, Chichester (2006)Google Scholar
  12. 12.
    Hitchcock, F.L.: The expression of a tensor or a polyadic as a sum of products. Journal of Mathematical Physics 6(1), 164–189 (1927)Google Scholar
  13. 13.
    Hitchcock, F.L.: Multiple invariants and generalized rank of a p-way matrix or tensor. Journal of Mathematical Physics 7(1), 39–79 (1927)Google Scholar
  14. 14.
    Ishteva, M.: Numerical methods for the best low multilinear rank approximation of higher-order tensors. Ph.D. thesis, Department of Electrical Engineering, Katholieke Universiteit Leuven (December 2009)Google Scholar
  15. 15.
    Ishteva, M., Absil, P.-A., Van Huffel, S., De Lathauwer, L.: Tucker compression and local optima. Chemometr. Intell. Lab. Syst. (2010), doi:10.1016/j.chemolab.2010.06.006Google Scholar
  16. 16.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995),
  17. 17.
    Kroonenberg, P.M.: Applied Multiway Data Analysis. Wiley, Chichester (2008)zbMATHCrossRefGoogle Scholar
  18. 18.
    Kroonenberg, P.M., de Leeuw, J.: Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45(1), 69–97 (1980)zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Liu, X., Srivastava, A., Gallivan, K.: Optimal linear representations of images for object recognition. IEEE Pattern Anal. and Mach. Intell. 26(5), 662–666 (2004), CrossRefGoogle Scholar
  20. 20.
    McCullagh, P.: Tensor Methods in Statistics. Chapman and Hall, London (1987)zbMATHGoogle Scholar
  21. 21.
    Tucker, L.R.: The extension of factor analysis to three-dimensional matrices. In: Gulliksen, H., Frederiksen, N. (eds.) Contributions to mathematical psychology, pp. 109–127. Holt, Rinehart & Winston (1964)Google Scholar
  22. 22.
    Tucker, L.R.: Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311 (1966)CrossRefMathSciNetGoogle Scholar
  23. 23.
    van den Bergh, F., Engelbrecht, A.P.: A new locally convergent particle swarm optimiser. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics. pp. 96–101 (2002)Google Scholar
  24. 24.
    Vasilescu, M.A.O., Terzopoulos, D.: Multilinear subspace analysis for image ensembles. In: Proc. Computer Vision and Pattern Recognition Conf. (CVPR 2003), Madison, WI, vol. 2, pp. 93–99 (2003)Google Scholar
  25. 25.
    Zhang, J., Zhang, J.R., Li, K.: A sequential niching technique for particle swarm optimization. In: Huang, D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 390–399. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Pierre B. Borckmans
    • 1
  • Mariya Ishteva
    • 1
  • Pierre-Antoine Absil
    • 1
  1. 1.Department of Mathematical EngineeringUniversité catholique de LouvainLouvain-la-NeuveBelgium

Personalised recommendations