Bounded Matrix Low Rank Approximation

  • Ramakrishnan KannanEmail author
  • Mariya Ishteva
  • Barry Drake
  • Haesun Park
Part of the Signals and Communication Technology book series (SCT)


Low rank approximation is the problem of finding two matrices \(\mathbf {P} \in \mathbb {R}^{m \times k}\) and \(\mathbf {Q} \in \mathbb {R}^ {k \times n}\) for input matrix \(\mathbf {R} \in \mathbb {R}^{m \times n}\), such that \(\mathbf {R} \approx \mathbf {PQ} \). It is common in recommender systems rating matrix, where the input matrix \(\mathbf {R}\) is bounded in the closed interval \([r_{min},r_{max}]\) such as [1, 5]. In this chapter, we propose a new improved scalable low rank approximation algorithm for such bounded matrices called bounded matrix low rank approximation (BMA) that bounds every element of the approximation \(\mathbf {PQ}\). We also present an alternate formulation to bound existing recommender systems algorithms called BALS and discuss its convergence. Our experiments on real-world datasets illustrate that the proposed method BMA outperforms the state-of-the-art algorithms for recommender system such as stochastic gradient descent, alternating least squares with regularization, SVD++ and bias-SVD on real-world datasets such as Jester, Movielens, Book crossing, Online dating, and Netflix.



This work was supported in part by the NSF Grant CCF-1348152, the Defense Advanced Research Projects Agency (DARPA) XDATA program grant FA8750-12-2-0309, Research Foundation Flanders (FWO-Vlaanderen), the Flemish Government (Methusalem Fund, METH1), the Belgian Federal Government (Interuniversity Attraction Poles—IAP VII), the ERC grant 320378 (SNLSID), and the ERC grant 258581 (SLRA). Mariya Ishteva is an FWO Pegasus Marie Curie Fellow. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF or the DARPA.


  1. 1.
    D.P. Bertsekas, Nonlinear Programming (Athena Scientific, Belmont, 1999)zbMATHGoogle Scholar
  2. 2.
    L. Brozovsky, V. Petricek, Recommender system for online dating service, in Proceedings of Conference Znalosti 2007, Ostrava, VSB (2007)Google Scholar
  3. 3.
    A. Cichocki, A.-H. Phan, Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. E92–A, 708–721 (2009)CrossRefGoogle Scholar
  4. 4.
    A. Cichocki, R. Zdunek, S. Amari, Hierarchical ALS algorithms for nonnegative matrix and 3d tensor factorization. Lect. Notes Comput. Sci. 4666, 169–176 (2007)CrossRefGoogle Scholar
  5. 5.
    S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, R. Harshman, Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41, 391–407 (1990)CrossRefGoogle Scholar
  6. 6.
    S. Funk, Stochastic gradient descent. (2006) simon/journal/20061211.html [Online; accessed 6-June-2012]
  7. 7.
    K. Goldberg, Jester collaborative filtering dataset. (2003) [Online; accessed 6-June-2012]
  8. 8.
    G.H. Golub, C.F. Van Loan, Matrix Computations, 3rd edn. (The Johns Hopkins University Press, Baltimore, 1996)zbMATHGoogle Scholar
  9. 9.
    L. Grippo, M. Sciandrone, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints. Oper. Res. Lett. 26(3), 127–136 (2000)zbMATHMathSciNetCrossRefGoogle Scholar
  10. 10.
    N.-D. Ho, P.V. Dooren, V.D. Blondel, Descent methods for nonnegative matrix factorization. (2008) CoRR, abs/0801.3199
  11. 11.
    R. Kannan, M. Ishteva, H. Park, Bounded matrix low rank approximation, in Proceedings of the 12th IEEE International Conference on Data Mining(ICDM-2012) (2012), pp. 319–328Google Scholar
  12. 12.
    R. Kannan, M. Ishteva, H. Park, Bounded matrix factorization for recommender system. Knowl. Inf. Syst. 39(3), 491–511 (2014)CrossRefGoogle Scholar
  13. 13.
    H. Kim, H. Park, Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23(12), 1495–1502 (2007)CrossRefGoogle Scholar
  14. 14.
    H. Kim, H. Park, Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J. Matrix Anal. Appl. 30(2), 713–730 (2008)zbMATHMathSciNetCrossRefGoogle Scholar
  15. 15.
    J. Kim, Y. He, H. Park, Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework. J. Glob. Optim. 1–35 (2013)Google Scholar
  16. 16.
    J. Kim, H. Park, Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM J. Sci. Comput. 33(6), 3261–3281 (2011)zbMATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    Y. Koren, Factorization meets the neighborhood: a multifaceted collaborative filtering model, in Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining—KDD’08 (2008), pp. 426–434Google Scholar
  18. 18.
    Y. Koren, Collaborative filtering with temporal dynamics, in Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining—KDD’09 (2009), pp. 447Google Scholar
  19. 19.
    Y. Koren, R. Bell, C. Volinsky, Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)CrossRefGoogle Scholar
  20. 20.
    D. Kuang, H. Park, C.H.Q. Ding, Symmetric nonnegative matrix factorization for graph clustering, in Proceedings of SIAM International Conference on Data Mining—SDM’12 (2012), pp. 106–117Google Scholar
  21. 21.
    A. Kyrola, G. Blelloch, C. Guestrin, Graphchi: large-scale graph computation on just a PC, in Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI’12, (USENIX Association, Berkeley, 2012) pp. 31–46Google Scholar
  22. 22.
    C.J. Lin, Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19(10), 2756–2779 (2007)zbMATHMathSciNetCrossRefGoogle Scholar
  23. 23.
    Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, J.M. Hellerstein, Graphlab: a new parallel framework for machine learning, in Conference on Uncertainty in Artificial Intelligence (UAI) (2010)Google Scholar
  24. 24.
    L.W. Mackey, D. Weiss, M.I. Jordan, Mixed membership matrix factorization, in Proceedings of the 27th International Conference on Machine Learning (ICML-10) (2010), pp. 711–718Google Scholar
  25. 25.
    I. Markovsky, Algorithms and literate programs for weighted low-rank approximation with missing data, in Approximation Algorithms for Complex Systems, ed. by J. Levesley, A. Iske, E. Georgoulis (Springer, Berlin, 2011), pp. 255–273. Chap. 12Google Scholar
  26. 26.
    Movielens dataset. (1999) [Online; accessed 6-June-2012]
  27. 27.
    A. Paterek, Improving regularized singular value decomposition for collaborative filtering, in Proceedings of 13th ACM International Conference on Knowledge Discovery and Data Mining—KDD’07 (2007), pp. 39–42Google Scholar
  28. 28.
    R. Salakhutdinov, A. Mnih, Bayesian probabilistic matrix factorization using Markov chain Monte Carlo, in ICML (2008), pp. 880–887Google Scholar
  29. 29.
    L. Xiong, X. Chen, T.-K. Huang, J.G. Schneider, J.G. Carbonell, Temporal collaborative filtering with Bayesian probabilistic tensor factorization, in Proceedings of the SIAM International Conference on Data Mining-SDM’10 (2010), pp. 211–222Google Scholar
  30. 30.
    H.-F. Yu, C.-J. Hsieh, S. Si, I.S. Dhillon, Scalable coordinate descent approaches to parallel matrix factorization for recommender systems, in Proceedings of the IEEE International Conference on Data Mining-ICDM’12 (2012), pp. 765–774Google Scholar
  31. 31.
    Y. Zhou, D. Wilkinson, R. Schreiber, R. Pan, Large-scale parallel collaborative filtering for the Netflix prize. Algorithm. Asp. Inf. Manag. 5034, 337–348 (2008)CrossRefGoogle Scholar
  32. 32.
    C.-N. Ziegler, S.M. McNee, J.A. Konstan, G. Lausen, Improving recommendation lists through topic diversification, in Proceedings of the 14th International Conference on World Wide Web-WWW’05 (2005), pp. 22–32Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Ramakrishnan Kannan
    • 1
    Email author
  • Mariya Ishteva
    • 2
  • Barry Drake
    • 1
  • Haesun Park
    • 1
  1. 1.Georgia Institute of TechnologyAtlantaUSA
  2. 2.Vrije Universiteit Brussel (VUB)BrusselsBelgium

Personalised recommendations