Skip to main content
SpringerLink
Log in

Alternating iterative methods for solving tensor equations with applications

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

Recently, the alternating direction method of multipliers (ADMM) and its variations have gained great popularity in large-scale optimization problems. This paper is concerned with the solution of the tensor equation \(\mathscr{A}\textbf {x}^{m-1}=\textbf {b}\) in which \(\mathscr{A}\) is an m th-order and n-dimensional real tensor and b is an n-dimensional real vector. By introducing certain auxiliary variables, we transform equivalently this tensor equation into a consensus constrained optimization problem, and then propose an ADMM type method for it. It turns out that each limit point of the sequences generated by this method satisfies the Karush-Kuhn-Tucker conditions. Moreover, from the perspective of computational complexity, the proposed method may suffer from the curse-of-dimensionality if the size of the tensor equation is large, and thus we further present a modified version (as a variant of the former) turning to the tensor-train decomposition of the tensor \(\mathscr{A}\), which is free from the curse. As applications, we establish the associated inverse iteration methods for solving tensor eigenvalue problems. The performed numerical examples illustrate that our methods are feasible and efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bader, B.W., Kolda, T.G., et al.: MATLAB Tensor Toolbox Version 2.6. http://www.sandia.gov/~tgkolda/TensorToolbox/index-2.6.html (2015)

  2. Ballani, J., Grasedyck, L.: A projection method to solve linear systems in tensor format. Numer. Linear Algebra Appl. 20(1), 27–43 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2010)

    Article  MATH  Google Scholar 

  4. Chang, T.-H., Hong, M.-Y., Liao, W., Wang, X.-F.: Asynchronous distributed ADMM for large-scale optimization-part i: algorithm and convergence analysis. IEEE Trans. Sig. Process. 64(12), 3118–3130 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen, C.-H., He, B.-S., Yuan, X.-M., Ye, Y.-Y.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1-2), 57–79 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  6. Cooper, J., Dutle, A.: Spectra of uniform hypergraphs. Linear Algebra Appl. 436, 3268–3292 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  7. Ding, W.-Y., Wei, Y.-M.: Solving multi-linear systems with \(\mathcal {M}\)-tensors. J. Sci. Comput. 68, 689–715 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dolgov, S.V., Khoromskij, B.N., Oseledets, I.V.: Fast solution of parabolic problems in the tensor train/quantized tensor train format with initial application to the Fokker-Planck equation. SIAM J. Sci. Comput. 34(6), A3016–A3038 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  9. Fowler, P.W., Pisanski, T.: HOMO-LUMO Maps for chemical graphs. MATCH Commun. Math. Comput. Chem. 64, 373–390 (2010)

    MathSciNet  MATH  Google Scholar 

  10. Forero, P.A., Cano, A., Giannakis, G.B.: Distributed clustering using wireless sensor networks. IEEE J. Selected Topics Sig. Process. 5, 707–724 (2011)

    Article  Google Scholar 

  11. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  12. Grasedyck, L.: Hierarchical singular value decomposition of tensors. SIAM J. Matrix Anal. Appl. 31(4), 2029–2054 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Grasedyck, L., Kressner, D., Tobler, C.: A literature survey of low-rank tensor approximation techniques. GAMM-Mitt. 36(1), 53–78 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  14. Gutman, I., Rouvray, D.: An approximate topological formula for the HOMO-LUMO separation in alternate hydrocarbons. Chem. Phys. Lett. 62, 384–388 (1979)

    Article  Google Scholar 

  15. Hackbusch, W., Kuhn, S.: A new scheme for the tensor representation. J. Fourier Anal. Appl. 15(5), 706–722 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  16. Han, L.-X.: A homotopy method for solving multilinear systems with \(\mathcal {M}\)-tensors. Appl. Math. Lett. 69, 49–54 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  17. Hastie, T., Tibshirani, R., Friedman, J.: The elements of statistical learning: data mining, inference, and prediction. Springer, New York (2001)

    Book  MATH  Google Scholar 

  18. Hillar, C.J., Lim, L.-H.: Most tensor problems are NP-hard. J. ACM. 60(6), 1–39 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. He, B.-S., Tao, M., Yuan, X.-M.: A splitting method for separable convex programming. IMA J. Numer. Anal. 20, 1–33 (2014)

    Google Scholar 

  20. He, B.-S., Yuan, X.-M.: Linearized alternating direction method of multipliers with Gaussian back substitution for separable convex programming. Numer. Algebra Control Optim. 3, 247–260 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Holtz, S., Rohwedder, T., Schneider, R.: The alternating linear scheme for tensor optimization in the tensor train format. SIAM J. Sci. Comput. 34, A683–A713 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Hong, M.-Y., Luo, Z.-Q., Razaviyayn, M.: Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. SIAM J. Optim. 26(1), 337–364 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  23. Kofidis, E., Regalia, P.A.: On the best rank-1 approximation of higher-order supersymmetric tensors. SIAM J. Matrix Anal. Appl. 23, 863–884 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  24. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51, 455–500 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  25. Kolda, T.G., Mayo, J.R.: Shifted power method for computing tensor eigenpairs. SIAM J. Matrix Anal. Appl. 32(4), 1095–1124 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  26. Liavas, A.P., Sidiropoulos, N.D.: Parallel algorithms for constrained tensor factorization via the alternating direction method of multipliers. IEEE Trans. Sig. Process. 63(20), 5450–5462 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  27. Li, D.-H., Xie, S.-L., Xu, H.-R.: Splitting methods for tensor equations. Numer. Linear Algebra Appl. 24(5), 1–16 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  28. Li, X.-T., Ng, M.K.: Solving sparse non-negative tensor equations: algorithms and applications. Front. Math. China 10(3), 649–680 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lim, L.-H.: Singular values and eigenvalues of tensors: a variational approach. In: proceedings of the 1st IEEE international workshop on computational advances of multi-sensor adaptive processing (CAMSAP), December 13-15, pp. 129–132 (2005)

  30. Liu, J., Chen, J., Ye, J.: Large-scale sparse logistic regression. In: Proceedings of the ACM International Conference on Knowledge Discovery and Data Mining, New York, NY, USA, June 28-July 1, pp. 547–556 (2009)

  31. Liu, D.-D., Li, W., Vong, S.W.: The tensor splitting with application to solve multi-linear systems. J. Comput. Appl. Math. 330(1), 75–94 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  32. Luo, Z.-Y., Qi, L.-Q., Xiu, N.-H.: The sparsest solutions to Z-tensor complementarity problems. Optim Lett. 11, 471–482 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  33. Matsuno, Y.: Exact solutions for the nonlinear Klein-Gordon and Liouville equations in four-dimensional Euclidean space. J. Math. Phys. 28(10), 2317–2322 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  34. Qi, L.-Q.: Eigenvalues of a real supersymmetric tensor. J. Symb. Comput. 40, 1302–1324 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  35. Qi, L.-Q., Luo, Z.-Y.: Tensor analysis: spectral theory and special tensors. SIAM, Philadelphia (2017)

    Book  MATH  Google Scholar 

  36. Ortega, J.M., Rheinboldt, W.C.: Iterative solution of nonlinear equations in several variables. Academic Press, New York (1970)

    MATH  Google Scholar 

  37. Oseledets, I.V.: Tensor-train decomposition. SIAM J. Sci. Comput. 33(5), 2295–2317 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  38. Oseledets, I.V., et al.: TT-Toolbox. https://github.com/oseledets/TT-Toolbox (2016)

  39. Oseledets, I.V., Tyrtyshnikov, E.E.: Breaking the curse of dimensionality, or how to use SVD in many dimensions. SIAM J. Sci. Comput. 31(5), 3744–3759 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  40. Sun, D.-F., Toh, K.-C., Yang, L.-Q.: A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J. Optim. 25(2), 882–915 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  41. Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. Roy. Stat. Soc. B 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  42. Wang, Y., Yin, W.-T., Zeng, J.-S.: Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. https://doi.org/10.1007/s10915-018-0757-z (2018)

  43. Xie, Z.-J., Jin, X.-Q., Wei, Y.-M.: A fast algorithm for solving circulant tensor systems. Linear Multilinear Algebra 65(9), 1894–1904 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  44. Xu, Y.-Y., Yin, W.-T., Wen, Z.-W., Zhang, Y.: An alternating direction algorithm for matrix completion with nonnegative factors. Front. Math. China 7(2), 365–384 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  45. Zhang, J.-Y., Wen, Z.-W., Zhang, Y.: Subspace methods with local refinements for eigenvalue computation using low-rank tensor-train format. J. Sci. Comput. 70, 478–499 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  46. Zwillinger, D.: Handbook of Differential Equations, 3rd edn. Academic Press Inc, Boston (1997)

    MATH  Google Scholar 

Download references

Acknowledgements

The authors are very grateful to the editors and two anonymous referees for their constructive comments and valuable suggestions, which greatly improved the original manuscript of this paper. Especially, the first author would like to thank Dr. Yutao Zheng for his selfless help in the process of programming.

Funding

This work was financially supported by the National Natural Science Foundation of China (Grant nos. 11571004 and 11701456). The research of the first author was also financially supported by the Science Foundation of Education Department of Gansu Province (Grant no. 2017A-078) and Tianshui Normal University (Grant no. TAS1603) as well as the Key Discipline Construction Foundation of Tianshui Normal University. The third author was financially supported by the Fundamental Research Funds for the Central Universities (Grant no. lzujbky-2017-it54) as well.

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Lanzhou University, Lanzhou, 730000, People’s Republic of China

    Maolin Liang, Bing Zheng & Ruijuan Zhao

  2. School of Mathematics and Statistics, Tianshui Normal University, Tianshui, 741001, People’s Republic of China

    Maolin Liang

Authors
  1. Maolin Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruijuan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bing Zheng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, M., Zheng, B. & Zhao, R. Alternating iterative methods for solving tensor equations with applications. Numer Algor 80, 1437–1465 (2019). https://doi.org/10.1007/s11075-018-0601-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-018-0601-4

Keywords

Mathematics Subject Classification (2010)

Search

Navigation