Skip to main content
SpringerLink
Log in

Attentive Hybrid Recurrent Neural Networks for sequential recommendation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Recently, the sequential recommendation has become a hot spot. Previous works combined user long-term and short-term behavior to achieve the next item recommendation, but the previous works typically processed the user long-term sequential behavior in left-to-right order and some useful information may be overlooked in such a particular way. Moreover, these methods ignored that every user has his/her own attention on the different items. In this paper, we propose a novel hybrid model called Attentive Hybrid Recurrent Neural Networks to resolve these problems. The first module is the bidirectional long- and short-term memory network (Bi-LSTM), and the second is the GRU module, both of which are equipped with user-based attention mechanism. The hybrid model aims to grasp the user general preference as well as to capture the user latest intent. Experiment results on two public datasets suggest that our hybrid model has better performance on the next item recommendation task compared with previously reported baseline algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Bagher RC, Hassanpour H, Mashayekhi H (2017) User trends modeling for a content-based recommender system. Expert Syst Appl 87:209–219

    Article  Google Scholar 

  2. Bobadilla J, Ortega F, Hernando A, Gutiérrez A (2013) Knowledge-based systems. Recommender Syst Surv 46:109–132

    Google Scholar 

  3. Chen L, Zhang G, Zhou E (2018) Fast greedy map inference for determinantal point process to improve recommendation diversity. In: Advances in Neural Information Processing Systems, pp. 5622–5633

  4. Chen S, Xu J, Joachims T (2013) Multi-space probabilistic sequence modeling. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 865–873

  5. Cho K, van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder–decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734

  6. Choi S, Ha H, Hwang U, Kim C, Ha JW, Yoon S (2018) Reinforcement learning based recommender system using biclustering technique. arXiv preprint arXiv:1801.05532

  7. Devooght R, Bersini H (2016) Collaborative filtering with recurrent neural networks. arXiv preprint arXiv:1608.07400

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  9. He R, Kang WC, McAuley J (2017) Translation-based recommendation. In: Proceedings of the eleventh ACM conference on recommender systems, pp. 161–169

  10. He X, Chua TS (2017) Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 355–364

  11. Hidasi B, Karatzoglou A, Baltrunas L, Tikk D (2016) Session-based recommendations with recurrent neural networks. In: 4th International Conference on Learning Representations, ICLR 2016

  12. Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 263–272. Ieee

  13. Huang R, McIntyre S, Song M, Ou Z et al (2018) An attention-based recommender system to predict contextual intent based on choice histories across and within sessions. Appl Sci 8(12):2426

    Article  Google Scholar 

  14. Kabbur S, Ning X, Karypis G (2013) Fism: factored item similarity models for top-n recommender systems. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 659–667

  15. Kang WC, McAuley J (2018) Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206. IEEE

  16. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  17. Koren Y (2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 426–434

  18. Koren Y, Bell R (2015) Advances in collaborative filtering. In: Recommender systems handbook. Springer, Berlin, pp 77–118

    Chapter  Google Scholar 

  19. Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37

    Article  Google Scholar 

  20. Li J, Ren P, Chen Z, Ren Z, Lian T, Ma J (2017) Neural attentive session-based recommendation. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 1419–1428

  21. Linden, G., Smith, B., York, J.: Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing 7(1), 76–80 (2003)

  22. Ling G, Lyu MR, King I (2014) Ratings meet reviews, a combined approach to recommend. In: Proceedings of the 8th ACM Conference on Recommender systems, pp. 105–112

  23. Lu J, Wu D, Mao M, Wang W, Zhang G (2015) Recommender system application developments: a survey. Decis Support Syst 74:12–32

    Article  Google Scholar 

  24. McAuley J, Leskovec J (2013) Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM conference on Recommender systems, pp. 165–172

  25. Van den Oord A, Dieleman S, Schrauwen B (2013) Deep content-based music recommendation. In: Advances in neural information processing systems, pp. 2643–2651

  26. Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2012) Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618

  27. Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th international conference on World wide web, pp. 811–820

  28. Ruocco M, Skrede OSL, Langseth H (2017) Inter-session modeling for session-based recommendation. In: Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems, pp. 24–31

  29. Salakhutdinov R, Mnih A (2008) Bayesian probabilistic matrix factorization using markov chain monte carlo. In: Proceedings of the 25th international conference on Machine learning, pp. 880–887

  30. Salakhutdinov R, Mnih A, Hinton G (2007) Restricted boltzmann machines for collaborative filtering. In: Proceedings of the 24th international conference on Machine learning, pp. 791–798

  31. Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th international conference on World Wide Web, pp. 285–295

  32. Singhal A, Sinha P, Pant R (2017) Use of deep learning in modern recommendation system: a summary of recent works. Int J Comp Appl 975:8875–8887

    Google Scholar 

  33. Sun F, Liu J, Wu J, Pei C, Lin X, Ou W, Jiang P (2019) Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1441–1450

  34. Tang J, Wang K (2018) Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 565–573

  35. Terzi M, Rowe M, Ferrario MA, Whittle J(2014) Text-based user-knn: Measuring user similarity based on text reviews. In: International Conference on User Modeling, Adaptation, and Personalization, pp. 195–206. Springer

  36. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp. 5998–6008

  37. Wang S, Cao L, Wang Y (2019) A survey on session-based recommender systems. arXiv preprint arXiv:1902.04864

  38. Wang X, Wang Y (2014) Improving content-based and hybrid music recommendation using deep learning. In: Proceedings of the 22nd ACM international conference on Multimedia, pp. 627–636

  39. Xiao J, Ye H, He X, Zhang H, Wu F, Chua TS (2017) Attentional factorization machines: learning the weight of feature interactions via attention networks. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 3119–3125

  40. Yao L, Sheng QZ, Ngu AH, Ashman H, Li X (2014) Exploring recommendations in internet of things. In: Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, pp. 855–858

  41. Yera R, Martinez L (2017) Fuzzy tools in recommender systems: a survey. Int J Comput Intell Syst 10(1):776–803

    Article  Google Scholar 

  42. Yu F, Liu Q, Wu S, Wang L, Tan T (2016) A dynamic recurrent model for next basket recommendation. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 729–732

  43. Zhao X, Zhang L, Ding Z, Xia L, Tang J, Yin D (2018) Recommendations with negative feedback via pairwise deep reinforcement learning. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1040–1048

  44. Zhao Z, Hong L, Wei L, Chen J, Nath A, Andrews S, Kumthekar A, Sathiamoorthy M, Yi X, Chi E (2019) Recommending what video to watch next: a multitask ranking system. In: Proceedings of the 13th ACM Conference on Recommender Systems, pp. 43–51

  45. Zheng L, Noroozi V, Yu PS (2017) Joint deep modeling of users and items using reviews for recommendation. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 425–434

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China under Grant 61976178, 62076202.

Author information

Authors and Affiliations

  1. School of Computer Science, Northwestern Polytechnical University, Xi’an, 710072, Shaanxi Province, China

    Lixiang Zhang, Peisen Wang, Jingchen Li, Zhiwei Xiao & Haobin Shi

Authors
  1. Lixiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peisen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingchen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiwei Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haobin Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haobin Shi.

Ethics declarations

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Wang, P., Li, J. et al. Attentive Hybrid Recurrent Neural Networks for sequential recommendation. Neural Comput & Applic 33, 11091–11105 (2021). https://doi.org/10.1007/s00521-020-05643-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05643-7

Keywords

Search

Navigation