RACMF: robust attention convolutional matrix factorization for rating prediction

  • Biqing ZengEmail author
  • Qi Shang
  • Xuli Han
  • Feng Zeng
  • Min Zhang
Industrial and commercial application


Matrix factorization is widely used in collaborative filtering, especially when the data are extremely large and sparse. To deal with the scale and sparsity problem of data, several recommender models adopt users and items’ side information to improve the recommendation results. However, some existing works do not perform well enough for they are not effectively use the side information. For example, using bag-of-words model, topic model to gain the latent representation of words or merely utilizing items or users’ side information, leads to the result that the performance deteriorates, especially when rating dataset is extremely large and sparse. To overcome the data sparsity problem, we present a hybrid model named robust attention convolutional matrix factorization (RACMF) model, which is composed of attention convolutional neural network (ACNN) and additional stacked denoising autoencoder (aSDAE); ACNN and aSDAE are used to extract the items’ and users’ latent factors, respectively. The experimental results show that our RACMF model has good prediction ability, even when the rating data are sparse or the scale of rating data is large. What’s more, compared with the state-of-the-art model PHD, the present model RACMF increased the accuracy rate on ML-100k, ML-1m, ML-10m and AIV-6 datasets by 4.80%, 0.57%, 1.98% and 3.67%, respectively.


Convolutional neural network Attention mechanism Additional stacked denoising autoencoder Rating prediction 



This work is supported by the National Natural Science Foundation of China (61503143).


  1. 1.
    Mnih A, Salakhutdinov RR (2008) Probabilistic matrix factorization. In: Advances in neural information processing systems, pp 1257–1264Google Scholar
  2. 2.
    Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749CrossRefGoogle Scholar
  3. 3.
    Wang C, Blei DM (2011) Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 448–456Google Scholar
  4. 4.
    Wang H, Wang N, Yeung DY (2015) Collaborative deep learning for recommender systems. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 1235–1244Google Scholar
  5. 5.
    McAuley J, Leskovec J (2013) Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM conference on Recommender systems. ACM, pp 165–172Google Scholar
  6. 6.
    Li S, Kawale J, Fu Y (2015) Deep collaborative filtering via marginalized denoising auto-encoder. In: Proceedings of the 24th ACM international on conference on information and knowledge management. ACM, pp 811–820Google Scholar
  7. 7.
    Ling G, Lyu MR, King I (2014) Ratings meet reviews, a combined approach to recommend. In: Proceedings of the 8th ACM conference on recommender systems. ACM, pp 105–112Google Scholar
  8. 8.
    Dong X, Yu L, Wu Z, et al (2017) A hybrid collaborative filtering model with deep structure for recommender systems. In: AAAI, pp 1309–1315Google Scholar
  9. 9.
    Liu J, Wang D, Ding Y (2017) PHD: a probabilistic model of hybrid deep collaborative filtering for recommender systems. In: Asian Conference on machine learning, pp 224–239Google Scholar
  10. 10.
    Vincent P, Larochelle H, Lajoie I et al (2010) Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(Dec):3371–3408MathSciNetzbMATHGoogle Scholar
  11. 11.
    Seo S, Huang J, Yang H, et al (2017) Interpretable convolutional neural networks with dual local and global attention for review rating prediction. In: Proceedings of the eleventh ACM conference on recommender systems. ACM, pp 297–305Google Scholar
  12. 12.
    Seo S, Huang J, Yang H, et al (2017) Representation learning of users and items for review rating prediction using attention-based convolutional neural network. In: 3rd international workshop on machine learning methods for recommender systems (MLRec) (SDM’17)Google Scholar
  13. 13.
    Zheng L, Noroozi V, Yu PS (2017) Joint deep modeling of users and items using reviews for recommendation. In: Proceedings of the tenth ACM international conference on Web search and data mining. ACM, pp 425–434Google Scholar
  14. 14.
    Chen C, Zhang M, Liu Y, et al (2018) Neural attentional rating regression with review-level explanations. In: Proceedings of the 2018 World Wide Web conference on World Wide Web. International World Wide Web conferences Steering Committee, pp 1583–1592Google Scholar
  15. 15.
    Kim D, Park C, Oh J, et al (2016) Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM conference on recommender systems. ACM, pp 233–240Google Scholar
  16. 16.
    Kim D, Park C, Oh J et al (2017) Deep hybrid recommender systems via exploiting document context and statistics of items. Inf Sci 417:72–87CrossRefGoogle Scholar
  17. 17.
    Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 8:30–37CrossRefGoogle Scholar
  18. 18.
    Xu K, Ba J, Kiros R, et al (2015) Show, attend and tell: Neural image caption generation with visual attention. In: International conference on machine learning, pp 2048–2057Google Scholar
  19. 19.
    Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473
  20. 20.
    Luong MT, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025
  21. 21.
    Yin W, Schütze H, Xiang B et al (2016) ABCNN: attention-based convolutional neural network for modeling sentence Pairs. Trans Assoc Comput Linguist 4:259–272CrossRefGoogle Scholar
  22. 22.
    Wang Y, Huang M, Zhao L (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in Natural Language Processing, pp 606–615Google Scholar
  23. 23.
    LeCun Y, Bottou L, Bengio Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  24. 24.
    Van den Oord A, Dieleman S, Schrauwen B (2013) Deep content-based music recommendation. In: Advances in neural information processing systems, pp 2643–2651Google Scholar
  25. 25.
    Zhang F, Yuan NJ, Lian D, et al (2016) Collaborative knowledge base embedding for recommender systems. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 353–362Google Scholar
  26. 26.
    Xue H, Dai X, Zhang J, et al (2017) Deep matrix factorization models for recommender systems. In: International joint conference on artificial intelligence, pp 3203–3209Google Scholar
  27. 27.
    Zhang S, Wang W, Ford J, et al (2006) Learning from incomplete ratings using non-negative matrix factorization. In: Proceedings of the 2006 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, pp 549–553Google Scholar
  28. 28.
    Sarwar B, Karypis G, Konstan J, et al (2000) Application of dimensionality reduction in recommender system-a case study. Minnesota Univ Minneapolis Dept of Computer ScienceGoogle Scholar
  29. 29.
    Goodfellow I, Pouget-Abadie J, Mirza M, et al (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of ComputerSouth China Normal UniversityTianhe District, Guangzhou CityPeople’s Republic of China

Personalised recommendations