Neural Variational Matrix Factorization with Side Information for Collaborative Filtering

  • Teng Xiao
  • Hong ShenEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11439)


Probabilistic Matrix Factorization (PMF) is a popular technique for collaborative filtering (CF) in recommendation systems. The purpose of PMF is to find the latent factors for users and items by decomposing a user-item rating matrix. Most methods based on PMF suffer from data sparsity and result in poor latent representations of users and items. To alleviate this problem, we propose the neural variational matrix factorization (NVMF) model, a novel deep generative model that incorporates side information (features) of both users and items, to capture better latent representations of users and items for the task of CF recommendation. Our NVMF consists of two end-to-end variational autoencoder neural networks, namely user neural network and item neural network respectively, which are capable of learning complex nonlinear distributed representations of users and items through our proposed variational inference. We derive a Stochastic Gradient Variational Bayes (SGVB) algorithm to approximate the intractable posterior distributions. Experiments conducted on three publicly available datasets show that our NVMF significantly outperforms the state-of-the-art methods.


Collaborative filtering Neural network Matrix factorization Deep generative process Variational inference 



This work is supported by the National Key Research and Development Program of China (No. #2017YFB0203201) and Australian Research Council Discovery Project DP150104871.


  1. 1.
    Adams, R.P., Dahl, G.E., Murray, I.: Incorporating side information in probabilistic matrix factorization with Gaussian processes. arXiv (2010)Google Scholar
  2. 2.
    Agarwal, D., Chen, B.C.: Regression-based latent factor models. In: KDD, pp. 19–28 (2009)Google Scholar
  3. 3.
    Bowman, S.R., Vilnis, L., Vinyals, O., Dai, A., Jozefowicz, R., Bengio, S.: Generating sentences from a continuous space. In: Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pp. 10–21 (2016)Google Scholar
  4. 4.
    Chen, Y., de Rijke, M.: A collective variational autoencoder for top-n recommendation with side information. In: Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems, pp. 3–9. ACM (2018)Google Scholar
  5. 5.
    Doersch, C.: Tutorial on variational autoencoders (2016)Google Scholar
  6. 6.
    Dong, X., Yu, L., Wu, Z., Sun, Y., Yuan, L., Zhang, F.: A hybrid collaborative filtering model with deep structure for recommender systems. In: AAAI, pp. 1309–1315 (2017)Google Scholar
  7. 7.
    Goodfellow, I., et al.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)Google Scholar
  8. 8.
    He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering. In: WWW, pp. 173–182 (2017)Google Scholar
  9. 9.
    Higgins, I., et al.: beta-VAE: learning basic visual concepts with a constrained variational framework (2016)Google Scholar
  10. 10.
    Kim, Y.D., Choi, S.: Scalable variational Bayesian matrix factorization with side information, pp. 493–502 (2014)Google Scholar
  11. 11.
    Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
  12. 12.
    Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer (2009)Google Scholar
  13. 13.
    Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: NIPS, pp. 556–562 (2001)Google Scholar
  14. 14.
    Li, S., Kawale, J., Fu, Y.: Deep collaborative filtering via marginalized denoising auto-encoder. In: CIKM, pp. 811–820 (2015)Google Scholar
  15. 15.
    Li, X., She, J.: Collaborative variational autoencoder for recommender systems. In: KDD, pp. 305–314 (2017)Google Scholar
  16. 16.
    Liang, D., Charlin, L., McInerney, J., Blei, D.M.: Modeling user exposure in recommendation. In: Proceedings of the 25th International Conference on World Wide Web, pp. 951–961 (2016)Google Scholar
  17. 17.
    Liang, D., Krishnan, R.G., Hoffman, M.D., Jebara, T.: Variational autoencoders for collaborative filtering. arXiv (2018)Google Scholar
  18. 18.
    Park, S., Kim, Y.D., Choi, S.: Hierarchical Bayesian matrix factorization with side information. In: IJCAI, pp. 1593–1599 (2013)Google Scholar
  19. 19.
    Porteous, I., Asuncion, A., Welling, M.: Bayesian matrix factorization with side information and Dirichlet process mixtures. In: AAAI, pp. 563–568 (2010)Google Scholar
  20. 20.
    Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models. arXiv (2014)Google Scholar
  21. 21.
    Salakhutdinov, R., Mnih, A.: Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In: ICML, pp. 880–887 (2008)Google Scholar
  22. 22.
    Singh, A., Gordon, G.J.: A Bayesian matrix factorization model for relational data. In: UAI, pp. 556–563 (2010)Google Scholar
  23. 23.
    Singh, A.P., Gordon, G.J.: Relational learning via collective matrix factorization. In: KDD, pp. 650–658 (2008)Google Scholar
  24. 24.
    Wainwright, M.J., Jordan, M.I., et al.: Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning, pp. 1–305 (2008)Google Scholar
  25. 25.
    Wang, C., Blei, D.M.: Collaborative topic modeling for recommending scientific articles. In: KDD, pp. 448–456 (2011)Google Scholar
  26. 26.
    Wang, H., Wang, N., Yeung, D.Y.: Collaborative deep learning for recommender systems, pp. 1235–1244 (2014)Google Scholar
  27. 27.
    Zhang, S., Yao, L., Sun, A.: Deep learning based recommender system: a survey and new perspectives (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Data and Computer ScienceSun Yat-sen UniversityGuangzhouChina
  2. 2.School of Computer ScienceThe University of AdelaideAdelaideAustralia

Personalised recommendations