Multi-modal Behavioral Information-Aware Recommendation with Recurrent Neural Networks

  • Guoyong CaiEmail author
  • Nannan Chen
  • Weidong Gu
  • Jiao Pan
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 895)


Data sparsity is one of the most challenging problems in recommendation systems. In this paper, we tackle this problem by proposing a novel multi-modal behavioral information-aware recommendation method named MIAR which is based on recurrent neural networks and matrix factorization. First, an interaction context-aware sequential prediction model is designed to capture user-item interaction contextual information and behavioral sequence information. Second, an attributed context-aware rating prediction model is proposed to capture attribution contextual information and rating information. Finally, three fusion methods are developed to combine two sub-models. As a result, the MIAR method has several distinguished advantages in terms of mitigating the data sparsity problem. The method can well perceive diverse influences of interaction and attribution contextual information. Meanwhile, a large number of behavioral sequence and rating information can be utilized by the MIAR approach. The proposed algorithm is evaluated on real-world datasets and the experimental results show that MIAR can significantly improve recommendation performance compared to the existing state-of-art recommendation algorithms.


Data sparsity Multi-modal information Recommendation Recurrent neural networks 



This work is supported by the Chinese National Science Foundation (#61763007), the Guilin Science and Technology Project (20170113-6) and the Guangxi Natural Science Foundation (#2017JJD160017).


  1. 1.
    Adomavicius, G., Tuzhilin, A.: Context-aware recommender systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P. (eds.) Recommender Systems Handbook, pp. 217–253. Springer, Boston (2011)CrossRefGoogle Scholar
  2. 2.
    Cai, G., Gu, W.: Heterogeneous context-aware recommendation algorithm with semi-supervised tensor factorization. In: International Conference on Intelligent Data Engineering and Automated Learning, Guilin, pp. 232–241. Springer (2017)Google Scholar
  3. 3.
    Zhang, M., Tang, J., Zhang, X., et al.: Addressing cold start in recommender systems: a semi-supervised co-training algorithm. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 73–82. ACM, New York (2014)Google Scholar
  4. 4.
    Liu, Q., Wu, S., Wang, L.: COT: Contextual Operating Tensor for context-aware recommender systems, pp. 203–209. AAAI, Menlo Park (2015)Google Scholar
  5. 5.
    Karatzoglou, A., Amatriain, X., Baltrunas, L., et al.: Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In: Proceedings of the Fourth ACM Conference on Recommender Systems, pp. 79–86. ACM (2010)Google Scholar
  6. 6.
    Rendle, S., Gantner, Z., Freudenthaler, C., et al.: Fast context-aware recommendations with factorization machines. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 635–644. ACM, New York (2011)Google Scholar
  7. 7.
    Zheng, V.W., Cao, B., Zheng, Y., et al.: Collaborative filtering meets mobile recommendation: a user-centered approach, vol. 10, pp. 236–241. AAAI, Menlo Park (2010)Google Scholar
  8. 8.
    Levandoski, J.J., Sarwat, M., Eldawy, A., et al.: Lars: a location-aware recommender system. In: 2012 IEEE 28th International Conference on Data Engineering (ICDE), pp. 450–461. IEEE, Piscataway (2012)Google Scholar
  9. 9.
    Ren, X., Song, M., Haihong, E., et al.: Context-aware probabilistic matrix factorization modeling for point-of-interest recommendation. Neurocomputing 241, 38–55 (2017)CrossRefGoogle Scholar
  10. 10.
    Wu, S., Liu, Q., Wang, L., et al.: Contextual operation for recommender systems. IEEE Trans. Knowl. Data Eng. 28(8), 2000–2012 (2016)CrossRefGoogle Scholar
  11. 11.
    Zhang, Y., Dai, H., Xu, C., et al. Sequential click prediction for sponsored search with recurrent neural networks, pp. 1369–1375. AAAI, Menlo Park (2014)Google Scholar
  12. 12.
    Liu, Q., Wu, S., Wang, L., et al.: Predicting the next location: a recurrent model with spatial and temporal contexts, pp. 194–200. AAAI, Menlo Park (2016)Google Scholar
  13. 13.
    Wu, S., Ren, W., Yu, C., et al.: Personal recommendation using deep recurrent neural networks in NetEase. In: 2016 IEEE 32nd International Conference on Data Engineering (ICDE), pp. 1218–1229. IEEE, Piscataway (2016)Google Scholar
  14. 14.
    Okura, S., Tagami, Y., Ono, S., et al.: Embedding-based news recommendation for millions of users. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1933–1942. ACM, New York (2017)Google Scholar
  15. 15.
    Liu, Q., Wu, S., Wang, D., et al.: Context-aware sequential recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 1053–1058. IEEE, Piscataway (2016)Google Scholar
  16. 16.
    Zhao, X., Niu, Z., Chen, W., et al.: A hybrid approach of topic model and matrix factorization based on two-step recommendation framework. J. Intell. Inf. Syst. 44(3), 335–353 (2015)CrossRefGoogle Scholar
  17. 17.
    Zhao, Z.D., Shang, M.S.: User-based collaborative-filtering recommendation algorithms on hadoop. In: International Conference on Knowledge Discovery and Data Mining, pp. 478–481. IEEE, Piscataway (2010)Google Scholar
  18. 18.
    Sarwar, B., Karypis, G., Konstan, J., et al.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, pp. 285–295. ACM, New York (2001)Google Scholar
  19. 19.
    Koren, Y.: Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 426–434. ACM, New York (2008)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Guoyong Cai
    • 1
    Email author
  • Nannan Chen
    • 1
  • Weidong Gu
    • 1
  • Jiao Pan
    • 2
  1. 1.Guangxi Key Lab of Trusted SoftwareGuilin University of Electronic TechnologyGuilinChina
  2. 2.Guilin Kaige Information Technology Co. Ltd.GuilinChina

Personalised recommendations