Skip to main content
SpringerLink
Log in

Low-rank tensor completion by Riemannian optimization

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

In tensor completion, the goal is to fill in missing entries of a partially known tensor under a low-rank constraint. We propose a new algorithm that performs Riemannian optimization techniques on the manifold of tensors of fixed multilinear rank. More specifically, a variant of the nonlinear conjugate gradient method is developed. Paying particular attention to efficient implementation, our algorithm scales linearly in the size of the tensor. Examples with synthetic data demonstrate good recovery even if the vast majority of the entries are unknown. We illustrate the use of the developed algorithm for the recovery of multidimensional images and for the approximation of multivariate functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Control Optim. 22(1), 135–158 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  2. Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)

    MATH  Google Scholar 

  3. Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations for incomplete data. Chemom. Intell. Lab. Syst. 106, 41–56 (2011)

    Article  Google Scholar 

  4. Bader, B.W., Kolda, T.G., et al.: Matlab tensor toolbox version 2.5 (2012). Available from http://www.sandia.gov/~tgkolda/TensorToolbox/

  5. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  6. Candès, E.J., Tao, T.: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Inf. Theory 56(5), 2053–2080 (2009)

    Article  Google Scholar 

  7. Chern, J.L., Dieci, L.: Smoothness and periodicity of some matrix decompositions. SIAM J. Matrix Anal. Appl. 22(3), 772–792 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  8. Da Silva, C., Herrmann, F.J.: Hierarchical Tucker tensor optimization—applications to tensor completion. In: Proc. 10th International Conference on Sampling Theory and Applications (2013)

    Google Scholar 

  9. De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  10. Foster, D.H., Nascimento, S.M.C., Amano, K.: Information limits on neural identification of colored surfaces in natural scenes. Vis. Neurosci. 21, 331–336 (2004)

    Article  Google Scholar 

  11. Gandy, S., Recht, B., Yamada, I.: Tensor completion and low-n-rank tensor recovery via convex optimization. Inverse Probl. 27(2), 025010 (2011)

    Article  MathSciNet  Google Scholar 

  12. Grasedyck, L., Kressner, D., Tobler, C.: A literature survey of low-rank tensor approximation techniques. GAMM-Mitt. 36(1), 53–78 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  13. Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from noisy entries. J. Mach. Learn. Res. 11, 2057–2078 (2010)

    MATH  MathSciNet  Google Scholar 

  14. Koch, O., Lubich, C.: Dynamical tensor approximation. SIAM J. Matrix Anal. Appl. 31(5), 2360–2375 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  15. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  16. Liu, Y., Shang, F.: An efficient matrix factorization method for tensor completion. IEEE Signal Process. Lett. 20(4), 307–310 (2013)

    Article  MathSciNet  Google Scholar 

  17. Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. In: Proc. IEEE 12th International Conference on Computer Vision, pp. 2114–2121 (2009)

    Google Scholar 

  18. Ma, Y., Wright, J., Ganesh, A., Zhou, Z., Min, K., Rao, S., Lin, Z., Peng, Y., Chen, M., Wu, L., Candès, E., Li, X.: Low-rank matrix recovery and completion via convex optimization. Survey website. http://perception.csl.illinois.edu/matrix-rank/. Accessed: 22 April 2013

  19. Mishra, B., Meyer, G., Bonnabel, S., Sepulchre, R.: Fixed-rank matrix factorizations and Riemannian low-rank optimization (2012). arXiv:1209.0430

  20. Mu, C., Huang, B., Wright, J., Goldfarb, D.: Square deal: lower bounds and improved relaxations for tensor recovery (2013). arXiv:1307.5870

  21. Ngo, T., Saad, Y.: Scaled gradients on Grassmann manifolds for matrix completion. In: Bartlett, P., Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1421–1429 (2012)

    Google Scholar 

  22. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer Series in Operations Research. Springer, Berlin (2006)

    MATH  Google Scholar 

  23. Rauhut, H., Schneider, R., Stojanac, Z.: Low rank tensor recovery via iterative hard thresholding. In: Proc. 10th International Conference on Sampling Theory and Applications (2013)

    Google Scholar 

  24. Schwab, C., Gittelson, C.J.: Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs. Acta Numer. 20, 291–467 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  25. Signoretto, M., De Lathauwer, L., Suykens, J.A.K.: Nuclear norms for tensors and their use for convex multilinear estimation. Tech. Rep. 10-186, K. U. Leuven (2010)

  26. Signoretto, M., Tran Dinh, Q., De Lathauwer, L., Suykens, J.A.K.: Learning with tensors: a framework based on convex optimization and spectral regularization. Tech. Rep. 11-129, K. U. Leuven (2011)

  27. Signoretto, M., Van de Plas, R., De Moor, B., Suykens, J.A.K.: Tensor versus matrix completion: a comparison with application to spectral data. IEEE Signal Process. Lett. 18(7), 403–406 (2011)

    Article  Google Scholar 

  28. Uschmajew, A.: Zur Theorie der Niedrigrangapproximation in Tensorprodukten von Hilberträumen. Ph.D. thesis, Technische Universität, Berlin (2013)

  29. Uschmajew, A., Vandereycken, B.: The geometry of algorithms using hierarchical tensors. Linear Algebra Appl. 439(1), 133–166 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  30. Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM J. Optim. 23(2), 1214–1236 (2013)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MATHICSE-ANCHP, Section de Mathématiques, École Polytechnique Fédérale de Lausanne, Station 8, 1015, Lausanne, Switzerland

    Daniel Kressner & Michael Steinlechner

  2. Department of Mathematics, Princeton University, Fine Hall, Princeton, NJ, 08544, USA

    Bart Vandereycken

Authors
  1. Daniel Kressner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Steinlechner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bart Vandereycken
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Steinlechner.

Additional information

Communicated by Axel Ruhe.

The work of M. Steinlechner has been supported by the SNSF research module Riemannian optimization for solving high-dimensional problems with low-rank tensor techniques within the SNSF ProDoc Efficient Numerical Methods for Partial Differential Equations.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kressner, D., Steinlechner, M. & Vandereycken, B. Low-rank tensor completion by Riemannian optimization. Bit Numer Math 54, 447–468 (2014). https://doi.org/10.1007/s10543-013-0455-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-013-0455-z

Keywords

Mathematics Subject Classification (2010)

Search

Navigation