Abstract
Matrix factorization as a popular technique for collaborative filtering in recommendation systems computes the latent factors for users and items by decomposing a user-item rating matrix. Most matrix factorization methods including probabilistic matrix factorization that projects (parameterized) users and items probabilistic matrices to maximize their inner product suffer from data sparsity and result in poor latent representations of users and items. To alleviate these problems, we propose a novel deep generative model, namely Neural Variational Matrix Factorization, that incorporates side information (features) of both users and items to capture better latent representations of them for more effective collaborative-filtering recommendation. Our model consists of two end-to-end variational autoencoder neural networks, namely user neural network and item neural network respectively, that are capable of learning complex nonlinear distributed representations of users and items through our proposed variational inference. We present a Stochastic Gradient Variational Bayes estimator to estimate the intractable posterior distributions of latent factors of users and items and parameters of our model, and derive the variational evidence lower bounds of the model. Experiments conducted on three publicly available datasets show that our model significantly outperforms the state-of-the-art methods on recommendation accuracy measured by Hit Ratio and Normalized Discounted Cumulative Gain respectively.
Similar content being viewed by others
References
Zhang S, Yao L, Sun A Deep learning based recommender system: a survey and new perspectives
Koren Y, Bell R, Volinsky C Matrix factorization techniques for recommender systems. Computer
Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: NIPS, pp 556–562
Xu M, Zhu J, Zhang B (2013) Fast max-margin matrix factorization with data augmentation. In: ICML, pp 978– 986
Salakhutdinov R, Mnih A (2008) Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In: ICML, pp 880–887
Adams RP, Dahl GE, Murray I Incorporating side information in probabilistic matrix factorization with gaussian processes. arXiv:1003.4944
Park S, Kim Y-D, Choi S (2013) Hierarchical Bayesian matrix factorization with side information. In: IJCAI, pp 1593– 1599
Kim YD, Choi S (2014) Scalable variational Bayesian matrix factorization with side information, 493–502
Porteous I, Asuncion A, Welling M (2010) Bayesian matrix factorization with side information and Dirichlet process mixtures. In: AAAI, pp 563–568
Singh A, Gordon GJ (2010) A Bayesian matrix factorization model for relational data. In: UAI, pp 556–563
Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: NIPS, pp 3111–3119
He K, Gkioxari G, Dollar P, Girshick R (2017) Mask r-cnn. In: ICCV, pp 2980–2988
Hinton G, Deng L, Yu D, Dahl GE, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97
Graves A, Jaitly N (2014) Towards end-to-end speech recognition with recurrent neural networks. In: ICML, pp 1764– 1772
Li S, Kawale J, Fu Y (2015) Deep collaborative filtering via marginalized denoising auto-encoder. In: CIKM, pp 811–820
Wang H, Wang N, Yeung DY (2014) Collaborative deep learning for recommender systems, 1235–1244
Strub F, Gaudel R, Mar J (2016) Hybrid recommender system based on autoencoders. In: Proceedings of the 1st workshop on deep learning for recommender systems. ACM, pp 11–16
Dong X, Yu L, Wu Z, Sun Y, Yuan L, Zhang F (2017) A hybrid collaborative filtering model with deep structure for recommender systems. In: AAAI, pp 1309–1315
Kingma DP, Welling M Auto-encoding variational Bayes. arXiv:1312.6114
Rezende DJ, Mohamed S, Wierstra D Stochastic backpropagation and approximate inference in deep generative models. arXiv:1401.4082
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: NIPS, pp 2672–2680
Li X, She J (2017) Collaborative variational autoencoder for recommender systems. In: KDD, pp 305–314
Liang D, Krishnan RG, Hoffman MD, Jebara T Variational autoencoders for collaborative filtering. arXiv:1802.05814
Chen Y, de Rijke M (2018) A collective variational autoencoder for top-n recommendation with side information. In: Proceedings of the 3rd workshop on deep learning for recommender systems. ACM, pp 3–9
Bowman SR, Vilnis L, Vinyals O, Dai A, Jozefowicz R, Bengio S (2016) Generating sentences from a continuous space. In: Proceedings of the 20th SIGNLL conference on computational natural language learning, pp 10–21
Mnih A, Salakhutdinov RR (2008) Probabilistic matrix factorization. In: NIPS, pp 1257–1264
Wang C, Blei DM (2011) Collaborative topic modeling for recommending scientific articles. In: KDD, pp 448–456
Doersch C Tutorial on variational autoencoders
Agarwal D, Chen BC (2009) Regression-based latent factor models. In: KDD, pp 19–28
Wainwright MJ, Jordan MI, et al. (2008) Graphical models, exponential families, and variational inference. Found Trends®; Mach Learn 1(1–2):1–305
Higgins I, Matthey L, Pal A, Burgess C, Glorot X, Botvinick M, Mohamed S, Lerchner A beta-vae: learning basic visual concepts with a constrained variational framework
Singh AP, Gordon GJ (2008) Relational learning via collective matrix factorization. In: KDD, pp 650–658
He X, Liao L, Zhang H, Nie L, Hu X, Chua T-S (2017) Neural collaborative filtering. In: WWW, pp 173–182
Liang D, Charlin L, McInerney J, Blei DM (2016) Modeling user exposure in recommendation. In: Proceedings of the 25th international conference on World Wide Web. International World Wide Web Conferences Steering Committee, pp 951–961
Xiao T, Shen H (2019) Neural Variational Matrix Factorization with Side Information for Collaborative Filtering. In: PAKDD 2019 (to appear)
Acknowledgements
This work is supported by National Key R & D Program of China Project #2017YFB0203201 and Australian Research Council Discovery Project DP150104871. The corresponding author is Hong Shen.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xiao, T., Shen, H. Neural variational matrix factorization for collaborative filtering in recommendation systems. Appl Intell 49, 3558–3569 (2019). https://doi.org/10.1007/s10489-019-01469-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-019-01469-6