Abstract
Low rank approximation is the problem of finding two matrices \(\mathbf {P} \in \mathbb {R}^{m \times k}\) and \(\mathbf {Q} \in \mathbb {R}^ {k \times n}\) for input matrix \(\mathbf {R} \in \mathbb {R}^{m \times n}\), such that \(\mathbf {R} \approx \mathbf {PQ} \). It is common in recommender systems rating matrix, where the input matrix \(\mathbf {R}\) is bounded in the closed interval \([r_{min},r_{max}]\) such as [1, 5]. In this chapter, we propose a new improved scalable low rank approximation algorithm for such bounded matrices called bounded matrix low rank approximation (BMA) that bounds every element of the approximation \(\mathbf {PQ}\). We also present an alternate formulation to bound existing recommender systems algorithms called BALS and discuss its convergence. Our experiments on real-world datasets illustrate that the proposed method BMA outperforms the state-of-the-art algorithms for recommender system such as stochastic gradient descent, alternating least squares with regularization, SVD++ and bias-SVD on real-world datasets such as Jester, Movielens, Book crossing, Online dating, and Netflix.
Keywords
- Low-rank Approximation
- Recommender System Algorithms
- GraphChi
- RMSE Scores
- Hierarchical Alternating Least Squares (HALS)
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The details about this dataset can be found in Table 4.2.
References
D.P. Bertsekas, Nonlinear Programming (Athena Scientific, Belmont, 1999)
L. Brozovsky, V. Petricek, Recommender system for online dating service, in Proceedings of Conference Znalosti 2007, Ostrava, VSB (2007)
A. Cichocki, A.-H. Phan, Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. E92–A, 708–721 (2009)
A. Cichocki, R. Zdunek, S. Amari, Hierarchical ALS algorithms for nonnegative matrix and 3d tensor factorization. Lect. Notes Comput. Sci. 4666, 169–176 (2007)
S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, R. Harshman, Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41, 391–407 (1990)
S. Funk, Stochastic gradient descent. (2006) http://sifter.org/ simon/journal/20061211.html [Online; accessed 6-June-2012]
K. Goldberg, Jester collaborative filtering dataset. (2003) http://goldberg.berkeley.edu/jester-data/ [Online; accessed 6-June-2012]
G.H. Golub, C.F. Van Loan, Matrix Computations, 3rd edn. (The Johns Hopkins University Press, Baltimore, 1996)
L. Grippo, M. Sciandrone, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints. Oper. Res. Lett. 26(3), 127–136 (2000)
N.-D. Ho, P.V. Dooren, V.D. Blondel, Descent methods for nonnegative matrix factorization. (2008) CoRR, abs/0801.3199
R. Kannan, M. Ishteva, H. Park, Bounded matrix low rank approximation, in Proceedings of the 12th IEEE International Conference on Data Mining(ICDM-2012) (2012), pp. 319–328
R. Kannan, M. Ishteva, H. Park, Bounded matrix factorization for recommender system. Knowl. Inf. Syst. 39(3), 491–511 (2014)
H. Kim, H. Park, Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23(12), 1495–1502 (2007)
H. Kim, H. Park, Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J. Matrix Anal. Appl. 30(2), 713–730 (2008)
J. Kim, Y. He, H. Park, Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework. J. Glob. Optim. 1–35 (2013)
J. Kim, H. Park, Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM J. Sci. Comput. 33(6), 3261–3281 (2011)
Y. Koren, Factorization meets the neighborhood: a multifaceted collaborative filtering model, in Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining—KDD’08 (2008), pp. 426–434
Y. Koren, Collaborative filtering with temporal dynamics, in Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining—KDD’09 (2009), pp. 447
Y. Koren, R. Bell, C. Volinsky, Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)
D. Kuang, H. Park, C.H.Q. Ding, Symmetric nonnegative matrix factorization for graph clustering, in Proceedings of SIAM International Conference on Data Mining—SDM’12 (2012), pp. 106–117
A. Kyrola, G. Blelloch, C. Guestrin, Graphchi: large-scale graph computation on just a PC, in Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI’12, (USENIX Association, Berkeley, 2012) pp. 31–46
C.J. Lin, Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19(10), 2756–2779 (2007)
Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, J.M. Hellerstein, Graphlab: a new parallel framework for machine learning, in Conference on Uncertainty in Artificial Intelligence (UAI) (2010)
L.W. Mackey, D. Weiss, M.I. Jordan, Mixed membership matrix factorization, in Proceedings of the 27th International Conference on Machine Learning (ICML-10) (2010), pp. 711–718
I. Markovsky, Algorithms and literate programs for weighted low-rank approximation with missing data, in Approximation Algorithms for Complex Systems, ed. by J. Levesley, A. Iske, E. Georgoulis (Springer, Berlin, 2011), pp. 255–273. Chap. 12
Movielens dataset. (1999) http://movielens.umn.edu [Online; accessed 6-June-2012]
A. Paterek, Improving regularized singular value decomposition for collaborative filtering, in Proceedings of 13th ACM International Conference on Knowledge Discovery and Data Mining—KDD’07 (2007), pp. 39–42
R. Salakhutdinov, A. Mnih, Bayesian probabilistic matrix factorization using Markov chain Monte Carlo, in ICML (2008), pp. 880–887
L. Xiong, X. Chen, T.-K. Huang, J.G. Schneider, J.G. Carbonell, Temporal collaborative filtering with Bayesian probabilistic tensor factorization, in Proceedings of the SIAM International Conference on Data Mining-SDM’10 (2010), pp. 211–222
H.-F. Yu, C.-J. Hsieh, S. Si, I.S. Dhillon, Scalable coordinate descent approaches to parallel matrix factorization for recommender systems, in Proceedings of the IEEE International Conference on Data Mining-ICDM’12 (2012), pp. 765–774
Y. Zhou, D. Wilkinson, R. Schreiber, R. Pan, Large-scale parallel collaborative filtering for the Netflix prize. Algorithm. Asp. Inf. Manag. 5034, 337–348 (2008)
C.-N. Ziegler, S.M. McNee, J.A. Konstan, G. Lausen, Improving recommendation lists through topic diversification, in Proceedings of the 14th International Conference on World Wide Web-WWW’05 (2005), pp. 22–32
Acknowledgments
This work was supported in part by the NSF Grant CCF-1348152, the Defense Advanced Research Projects Agency (DARPA) XDATA program grant FA8750-12-2-0309, Research Foundation Flanders (FWO-Vlaanderen), the Flemish Government (Methusalem Fund, METH1), the Belgian Federal Government (Interuniversity Attraction Poles—IAP VII), the ERC grant 320378 (SNLSID), and the ERC grant 258581 (SLRA). Mariya Ishteva is an FWO Pegasus Marie Curie Fellow. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF or the DARPA.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Kannan, R., Ishteva, M., Drake, B., Park, H. (2016). Bounded Matrix Low Rank Approximation. In: Naik, G. (eds) Non-negative Matrix Factorization Techniques. Signals and Communication Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-48331-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-662-48331-2_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-48330-5
Online ISBN: 978-3-662-48331-2
eBook Packages: EngineeringEngineering (R0)