Skip to main content
SpringerLink
Log in

Parallel Approximation of Multidimensional Tensors Using GPUs

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

When solving many applied research problems, it is necessary to work with multidimensional arrays (tensors). In practice, an efficient and compact representation of these objects in the form of so-called tensor trains is used. The paper considers a parallel implementation of the TT-cross algorithm, which allows one to obtain a decomposition of a multidimensional array into a tensor train, using a CUDA GPU. The main aspects and features of the parallel implementation of the algorithm are presented. The resulting parallel version of the algorithm was tested on a representative number of examples. A significant reduction in computational time is demonstrated compared to a similar sequential implementation of the algorithm, which indicates the efficiency of the proposed approaches to parallelization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  1. Morozov, A.Yu. and Reviznikov, D.L., Adaptive interpolation algorithm based on a kd-tree for numerical integration of ordinary differential equations systems with interval initial conditions, Differ. Uravn., 2018, vol. 54, no. 7, pp. 963–974. https://doi.org/10.1134/S0374064118070130

    Article  MATH  Google Scholar 

  2. Morozov, A.Yu., Zhuravlev, A.A., and Reviznikov, D.L., Analysis and optimization of an adaptive interpolation algorithm for the numerical solution of an ordinary differential equations system with interval parameters, Differ. Uravn., 2020, vol. 56, no. 7, pp. 960–974. https://doi.org/10.1134/S0374064120070122

    Article  MATH  Google Scholar 

  3. Gidaspov, V.Yu., Morozov, A.Yu., and Reviznikov, D.L., Adaptive interpolation algorithm using TT-decomposition for modeling dynamical systems with interval parameters, Comput. Math. Math. Phys., 2021, vol. 61, no. 9, pp. 1387–1401. https://doi.org/10.1134/S0965542521090098

    Article  MathSciNet  MATH  Google Scholar 

  4. Morozov, A.Yu. and Reviznikov, D.L., Modelling of dynamic systems with interval parameters, Program. Inzh., 2019, vol. 10, no. 2, pp. 69–76. https://doi.org/10.17587/prin.10.69-76

    Article  Google Scholar 

  5. Morozov, A.Yu., Parallel adaptive interpolation algorithm based on sparse grids for modeling dynamic systems with interval parameters, Program. Inzh., 2021, vol. 12, no. 8. https://doi.org/10.17587/prin.12.395-403

  6. Oseledets, I.V., Tensor-train decomposition, SIAM J. Sci. Comput., 2011, vol. 33, no. 5, pp. 2295–2317.

    Article  MathSciNet  MATH  Google Scholar 

  7. Oseledets, I. and Tyrtyshnikov, E., TT-cross approximation for multidimensional arrays, Lin. Algebra Its Appl., 2010, vol. 432, no. 1, pp. 70–88. https://doi.org/10.1016/j.laa.2009.07.024

    Article  MathSciNet  MATH  Google Scholar 

  8. Hitchcock, F.L., The expression of a tensor or a polyadic as a sum of products, J. Math. Phys., 1927, vol. 6, no. 1, pp. 164–189. https://doi.org/10.1002/sapm192761164

    Article  MATH  Google Scholar 

  9. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P., 2.6 Singular Value Decomposition. Numerical Recipes in C, 2nd ed., Cambridge: Cambridge Univ. Press, 1992.

    MATH  Google Scholar 

  10. Tyrtyshnikov, E.E., Tensor approximations for matrices generated by asymptotically smooth functions, Mat. Sb., 2003, vol. 194, no. 6, pp. 147–160. https://doi.org/10.4213/sm747

    Article  MathSciNet  MATH  Google Scholar 

  11. Tyrtyshnikov, E.E. and Shcherbakova, E.M., Methods for nonnegative matrix factorization based on low-rank cross approximations, Comput. Math. Math. Phys., 2019, vol. 59, no. 8, pp. 1251–1267. https://doi.org/10.1134/S0965542519080165

    Article  MathSciNet  MATH  Google Scholar 

  12. Zheltkov, D.A. and Tyrtyshnikov, E.E., Matrix cross approximation method: parallel implementation, Vychisl. Metody Program., 2015, vol. 16, pp. 369–375. https://doi.org/10.26089/NumMet.v16r336

    Article  Google Scholar 

  13. Goreinov, S.A., Zamarashkin, N.L., and Tyrtyshnikov, E.E., Pseudo-skeleton approximations with the help of submatrices of maximal volume, Mat. Zametki, 1997, vol. 62, no. 4, pp. 619–623. https://doi.org/10.4213/MZM1644

    Article  MATH  Google Scholar 

  14. CUDA Zone. https://developer.nvidia.com/cuda-zone.

  15. Keringhan, B.W. and Ritchie, D.M., The C Programming Language, Prentice Hall, 1978.

    Google Scholar 

  16. Josuttis, N.M., The C++ Standard Library: a Tutorial and Reference, Addison-Wesley Pro., 2012.

    Google Scholar 

  17. Matrix Algebra on GPU and Multicore Architectures (MAGMA). https://icl.cs.utk.edu/magma.

  18. cuBLAS. https://docs.nvidia.com/cuda/cublas.

  19. ScaLAPACK. http://www.netlib.org/scalapack.

  20. MATLAB. http://www.mathworks.com/help/matlab.

  21. Oseledets, I.V., Tensor methods and their applications, Doctoral Sci. (Phys.-Math.) Dissertation, Moscow: Marchuk Institute of Numeriical Mathematics RAS, 2009.

  22. Goreinov, S.A., Oseledets, I.V., Savostyanov, D.V., and Tyrtyshnikov, E.E., How to Find a Good Submatrix, Institute for Computational Mathematics Hong Kong Baptist Univ., 2008. https://doi.org/10.1142/9789812836021_0015

    Book  MATH  Google Scholar 

  23. LAPACK. http://www.netlib.org/lapack.

  24. Chrzeszczyk, A. and Anders, J., Matrix Computations on the GPU, Jan Kochanowski Univ., 2017.

    Google Scholar 

  25. Griewank, A.O., Generalized decent for global optimization, J. Optim. Theory Appl., 1981, vol. 34, pp. 11–39. https://doi.org/10.1007/BF00933356

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Moscow Aviation Institute (National Research University), 121552, Moscow, Russia

    N. S. Kapralov, A. Yu. Morozov & S. P. Nikulin

  2. Deutsche Bank Technology Center, 115184, Moscow, Russia

    N. S. Kapralov

  3. Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences, 119333, Moscow, Russia

    A. Yu. Morozov

Authors
  1. N. S. Kapralov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Yu. Morozov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. P. Nikulin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to N. S. Kapralov, A. Yu. Morozov or S. P. Nikulin.

Ethics declarations

The authors declare that they have no conflicts of interest.

Additional information

Translated by E. Chernokozhin

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kapralov, N.S., Morozov, A.Y. & Nikulin, S.P. Parallel Approximation of Multidimensional Tensors Using GPUs. Program Comput Soft 49, 295–301 (2023). https://doi.org/10.1134/S0361768823040060

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768823040060

Keywords:

Search

Navigation