Algorithms and Literate Programs for Weighted Low-Rank Approximation with Missing Data

  • Ivan Markovsky
Conference paper
Part of the Springer Proceedings in Mathematics book series (PROM, volume 3)


Linear models identification from data with missing values is posed as a weighted low-rank approximation problem with weights related to the missing values equal to zero. Alternating projections and variable projections methods for solving the resulting problem are outlined and implemented in a literate programming style, using Matlab/Octave’s scripting language. The methods are evaluated on synthetic data and real data from the MovieLens data sets.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J.-F. Cai, E. Cand´es, and Z. Shen: A singular value thresholding algorithm for matrix completion. candes/papers/SVT. pdf, 2009.
  2. 2.
    E. Cand´es and B. Recht: Exact matrix completion via convex optimization. Found. of Comput. Math., 2009.Google Scholar
  3. 3.
    B. De Moor: Structured total least squares and L2 approximation problems. Linear Algebra Appl., 188–189:163–207, 1993.CrossRefGoogle Scholar
  4. 4.
    K. Gabriel and S. Zamir: Lower rank approximation of matrices by least squares with any choice of weights. Technometrics 21, 1979, 489–498.MATHCrossRefGoogle Scholar
  5. 5.
    G. Golub and V. Pereyra: Separable nonlinear least squares: the variable projection method and its applications. Institute of Physics, Inverse Problems 19, 2003, 1–26.CrossRefMathSciNetGoogle Scholar
  6. 6.
    G. Golub and C. Van Loan: An analysis of the total least squares problem. SIAM J. Numer. Anal. 17, 1980, 883–893.MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    GroupLens: Movielens data sets.
  8. 8.
    N. Higham: Computing the nearest correlation matrix – a problem from finance. IMA Journal of Numerical Analysis 22(3), 2002, 329–343.MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    H. Kiers: Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems. Comput. Statist. Data Anal. 41, 2002, 157–170.MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    D. Knuth: Literate Programming. Cambridge University Press, 1992.Google Scholar
  11. 11.
    W. Krijnen: Convergence of the sequence of parameters generated by alternating least squares algorithms. Comput. Statist. Data Anal. 51, 2006, 481–489.MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    J. Manton, R. Mahony, and Y. Hua: The geometry of weighted low-rank approximations. IEEE Trans. Signal Process. 51(2), 2003, 500–514.CrossRefMathSciNetGoogle Scholar
  13. 13.
    I. Markovsky: Algorithms and Literate Programs for Weighted Low-Rank Approximation with Missing Data. Technical Report 18296, ECS, Univ. of Southampton,,2009.
  14. 14.
    I. Markovsky, M.-L. Rastello, A. Premoli, A. Kukush, and S. Van Huffel: The element-wise weighted total least squares problem. Comput. Statist. Data Anal. 50(1), 2005, 181–209.CrossRefGoogle Scholar
  15. 15.
    I. Markovsky and S. Van Huffel: Left vs right representations for solving weighted low rank approximation problems. Linear Algebra Appl. 422, 2007, 540–552.MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    D. Marquardt: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 1963, 431–441.MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    N. Ramsey: noweb. nr/noweb/.
  18. 18.
    N. Ramsey: Literate programming simplified. IEEE Software 11, 1994, 97–105.CrossRefGoogle Scholar
  19. 19.
    T. Segaran: Programming Collective Intelligence: Building Smart Web 2.0 Applications. O’Reilly Media, 2007.Google Scholar
  20. 20.
    N. Srebro: Learning with Matrix Factorizations. PhD thesis, MIT, 2004.Google Scholar
  21. 21.
    P. Wentzell, D. Andrews, D. Hamilton, K. Faber, and B. Kowalski: Maximum likelihood principal component analysis. J. Chemometrics 11, 1997, 339–366.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ivan Markovsky
    • 1
  1. 1.School of Electronics & Computer ScienceUniv. of SouthamptonSouthamptonUK

Personalised recommendations