Advertisement

Memory-Augmented Attention Network for Sequential Recommendation

  • Cheng HuEmail author
  • Peijian He
  • Chaofeng Sha
  • Junyu Niu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11881)

Abstract

An increased interest in sequential recommendation has been observed in recent years. Many models have been proposed to leverage the sequential user-item interaction data, which includes those based on Markov Chain or recurrent neural networks. Most of these models are designed for the scenario where each historical record composed of single item. However, the records could be a subset of items (or session) such as music playlists and baskets in e-commerce applications. How to leverage the session structure to improve the effectiveness of the recommendation system is a challenge. To this end, we propose a MEmory-augmented Attention Network for Sequential recommendation (MEANS), to effectively recommend next items given the sequential session data. The most recent sessions are stored into external memory after a max-pooling operation. The long-term user preference are learned through an attention network which is stacked on the memory layer. Finally, the mixture of long-term and short-term preference is feeded into the prediction layer to make recommendations. Extensive experiments on four real datasets show that MEANS outperforms various state-of-the-art sequential recommendation models.

Keywords

Sequential recommendation Memory network Attention mechanism 

Notes

Acknowledgments

This work was supported by National Key R&D Program of China (Grant No. 2018YFB0904503), the National Natural Science Foundation of China (NFSC) under Grant No. 61572135, and Shanghai Municipal Science and Technology Commission (17511108504).

References

  1. 1.
    Adda, M., Djeraba, C.: Recommendation strategy based on relation rule mining. In: ITWP, pp. 33–40 (2005)Google Scholar
  2. 2.
    Cai, C., He, R., McAuley, J.J.: SPMC: socially-aware personalized Markov chains for sparse sequential recommendation. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, 19–25 August, 2017, pp. 1476–1482 (2017).  https://doi.org/10.24963/ijcai.2017/204
  3. 3.
    Chang, Y., Zhai, C., Liu, Y., Maarek, Y. (eds.) Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 2018, Marina Del Rey, CA, USA, 5–9 February 2018. ACM (2018).  https://doi.org/10.1145/3159652
  4. 4.
    Deshpande, M., Karypis, G.: Item-based top-N recommendation algorithms. ACM Trans. Inf. Syst. 22(1), 143–177 (2004).  https://doi.org/10.1145/963770.963776CrossRefGoogle Scholar
  5. 5.
    Donkers, T., Loepp, B., Ziegler, J.: Sequential user-based recurrent neural network recommendations. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, 27–31 August 2017, pp. 152–160 (2017).  https://doi.org/10.1145/3109859.3109877
  6. 6.
    Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. In: 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, 2–4 May 2016, Conference Track Proceedings (2016)Google Scholar
  7. 7.
    Hu, L., Cao, L., Wang, S., Xu, G., Cao, J., Gu, Z.: Diversifying personalized recommendation with user-session context. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, 19–25 August 2017, pp. 1858–1864 (2017).  https://doi.org/10.24963/ijcai.2017/258
  8. 8.
    Jannach, D., Ludewig, M.: When recurrent neural networks meet the neighborhood for session-based recommendation. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, 27–31 August 2017, pp. 306–310 (2017).  https://doi.org/10.1145/3109859.3109872
  9. 9.
    Koren, Y., Bell, R.M., Volinsky, C.: Matrix factorization techniques for recommender systems. IEEE Comput. 42(8), 30–37 (2009).  https://doi.org/10.1109/MC.2009.263CrossRefGoogle Scholar
  10. 10.
    Lang, J. (ed.): Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, 13–19 July 2018, Stockholm, Sweden. ijcai.org (2018)Google Scholar
  11. 11.
    Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., Ma, J.: Neural attentive session-based recommendation. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM 2017, Singapore, 06–10 November 2017, pp. 1419–1428 (2017).  https://doi.org/10.1145/3132847.3132926
  12. 12.
    Ludewig, M., Jannach, D.: Evaluation of session-based recommendation algorithms. User Model. User-Adapt. Interact. 28(4–5), 331–390 (2018).  https://doi.org/10.1007/s11257-018-9209-6CrossRefGoogle Scholar
  13. 13.
    Quadrana, M., Cremonesi, P., Jannach, D.: Sequence-aware recommender systems. ACM Comput. Surv. 51(4), 66:1–66:36 (2018).  https://doi.org/10.1145/3190616CrossRefGoogle Scholar
  14. 14.
    Quadrana, M., Karatzoglou, A., Hidasi, B., Cremonesi, P.: Personalizing session-based recommendations with hierarchical recurrent neural networks. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, 27–31 August 2017, pp. 130–137 (2017).  https://doi.org/10.1145/3109859.3109896
  15. 15.
    Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: bayesian personalized ranking from implicit feedback. In: UAI 2009, Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada, 18–21 June 2009, pp. 452–461 (2009)Google Scholar
  16. 16.
    Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized Markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web, WWW 2010, Raleigh, North Carolina, USA, 26–30 April 2010, pp. 811–820 (2010).  https://doi.org/10.1145/1772690.1772773
  17. 17.
    Salakhutdinov, R., Mnih, A.: Probabilistic matrix factorization. In: Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, 3–6 December 2007, pp. 1257–1264 (2007)Google Scholar
  18. 18.
    Salakhutdinov, R., Mnih, A., Hinton, G.E.: Restricted Boltzmann machines for collaborative filtering. In: Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvallis, Oregon, USA, 20–24 June 2007, pp. 791–798 (2007).  https://doi.org/10.1145/1273496.1273596
  19. 19.
    Sarwar, B.M., Karypis, G., Konstan, J.A., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the Tenth International World Wide Web Conference, WWW 10, Hong Kong, China, 1–5 May 2001, pp. 285–295 (2001).  https://doi.org/10.1145/371920.372071
  20. 20.
    Sen, S., Geyer, W., Freyne, J., Castells, P. (eds.): Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA, 15–19 September 2016. ACM (2016)Google Scholar
  21. 21.
    Tan, Y.K., Xu, X., Liu, Y.: Improved recurrent neural networks for session-based recommendations. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, DLRS@RecSys 2016, Boston, MA, USA, 15 September 2016, pp. 17–22 (2016).  https://doi.org/10.1145/2988450.2988452
  22. 22.
    Tuan, T.X., Phuong, T.M.: 3d convolutional networks for session-based recommendation with content features. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, 27–31 August 2017, pp. 138–146 (2017).  https://doi.org/10.1145/3109859.3109900
  23. 23.
    Wang, P., Guo, J., Lan, Y., Xu, J., Wan, S., Cheng, X.: Learning hierarchical representation model for next basket recommendation. In: SIGIR, pp. 403–412 (2015).  https://doi.org/10.1145/2766462.2767694
  24. 24.
    Wu, C., Ahmed, A., Beutel, A., Smola, A.J., Jing, H.: Recurrent recommender networks. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, 6–10 February 2017, pp. 495–503 (2017).  https://doi.org/10.1145/3018661.3018689
  25. 25.
    Yap, G.-E., Li, X.-L., Yu, P.S.: Effective next-items recommendation via personalized sequential pattern mining. In: Lee, S., Peng, Z., Zhou, X., Moon, Y.-S., Unland, R., Yoo, J. (eds.) DASFAA 2012. LNCS, vol. 7239, pp. 48–64. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-29035-0_4CrossRefGoogle Scholar
  26. 26.
    Ying, H., et al.: Sequential recommender system based on hierarchical attention networks. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, 13–19 July 2018, Stockholm, Sweden, pp. 3926–3932 (2018).  https://doi.org/10.24963/ijcai.2018/546
  27. 27.
    Yu, F., Liu, Q., Wu, S., Wang, L., Tan, T.: A dynamic recurrent model for next basket recommendation. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Italy, 17–21 July 2016, pp. 729–732 (2016).  https://doi.org/10.1145/2911451.2914683
  28. 28.
    Zhu, Y., et al.: What to do next: modeling user behaviors by time-LSTM. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, 19–25 August 2017, pp. 3602–3608 (2017).  https://doi.org/10.24963/ijcai.2017/504

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer Science, Shanghai Key Laboratory of Intelligence ProcessingFudan UniversityShanghaiChina

Personalised recommendations