Evaluating Session-Based Recommendation Approaches on Datasets from Different Domains

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11814)


Recommending relevant items of interest for user is the main purpose of recommendation system based on long-term user profiles. However, personal data privacy is becoming a big challenge recently. Thus, recommendation system needs to reduce the dependence on user profiles while still keeping high accuracy on recommendation. Session-based recommendation is a recent proposed approach for recommendation system to overcome the issue of user profiles dependency. The relevance of problem is quite high and has triggered interest among researchers in observing activities of users. It increased a number of proposals for session-based recommendation algorithms that aiming to make prediction of next actions. In this paper, we would like to compare the performance of such algorithms by using various datasets and evaluation metrics. The most recent deep learning approach named GRU4REC [1] and simpler methods based are included in our comparison. Six real-world datasets from three different domains are included in our experiment. Our experiments reveal that in case of numerous unpopular items dataset, GRU4REC’s performance is low. However, its performance is significantly increased after applying our proposed sampling method. Therefore, our obtained results suggested that there is still room for improving deep learning session-based recommendation algorithms.


Session-based recommendation Sequential recommendation Nearest neighbors Recurrent neural networks Recommendation systems 



Tran Khanh Dang is supported by a project with the Department of Science and Technology, Ho Chi Minh City, Vietnam (contract with HCMUT No. 42/2019/HĐ-QPTKHCN, dated 11/7/2019).


  1. 1.
    Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. In: ICLR (2016)Google Scholar
  2. 2.
    Quadrana, M., Cremonesi, P., Jannach, D.: Sequence-aware recommender systems. Comput. Surveys 54(1), 1–36 (2018)Google Scholar
  3. 3.
    Robin, D., Hugues, B.: Collaborative filtering with recurrent neural networks. CoRR, abs/1608.07400 (2016)Google Scholar
  4. 4.
    Hidasi, B., Karatzoglou, A.: Recurrent neural networks with top-k gains for session-based recommendations. CoRR abs/1706.03847 (2017). arXiv:1706
  5. 5.
    Kamehkhosh, I., Jannach, D., Ludewig, M.: A comparison of frequent pattern techniques and a deep learning method for session-based recommendation. In: TempRec Workshop at ACM RecSys 2017, Como, Italy (2017)Google Scholar
  6. 6.
    Mobasher, B., Dai, H., Luo, T., Nakagawa, M.: Using sequential and non-sequential patterns in predictive web usage mining tasks. In: ICDM 2002, pp. 669–672 (2002)Google Scholar
  7. 7.
    Bonnin, G., Jannach, D.: Automated generation of music playlists: survey and experiments. Comput. Surv. 47(2), 26:1–26:35 (2014)CrossRefGoogle Scholar
  8. 8.
    Quadrana, M., Karatzoglou, A., Hidasi, B., Cremonesi, P.: Personalizing session-based recommendations with hierarchical recurrent neural networks. In: RecSys 2017, pp. 130–137 (2017)Google Scholar
  9. 9.
    Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: WWW (2001)Google Scholar
  10. 10.
    Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009, pp. 452–461 (2009). ISBN 978-0-9749039-5-8Google Scholar
  11. 11.
    Davidson, J., Liebald, B., Liu, J., et al.: The YouTube video recommendation system. In: ACM Conference on Recommender Systems, Recsys 2010, pp. 293–296 (2010). ISBN 978-1-60558-906-0Google Scholar
  12. 12.
    Jannach, D., Lerche, L., Jugovac, M.: Adaptation and evaluation of recommendations for short-term shopping goals. In: RecSys 2015, pp. 211–218 (2015)Google Scholar
  13. 13.
    Turrin, R., Quadrana, M., Condorelli, A., Pagano, R., Cremonesi, P.: 30Music listening and playlists dataset. In: Poster Proceedings of RecSys 2015 (2015)Google Scholar
  14. 14.
    Jannach, D., Ludewig, M.: When recurrent neural networks meet the neighborhood for session-based recommendation. In: Proceedings of RecSys 2017, Como, Italy, 27–31 August 2017, 5 p. (2017)Google Scholar
  15. 15.
    Jannach, D., Zanker, M., Ge, M., Gröning, M.: Recommender systems in computer science and information systems - a landscape of research. In: Proceedings of the 13th International Conference on E-Commerce and Web Technologies, pp. 76–87 (2012)Google Scholar
  16. 16.
    Karimi, M., Jannach, D., Jugovac, M.: News recommender systems - survey and roads ahead. Inf. Proces. Manag. (2018)Google Scholar
  17. 17.
    Ludmann, C.A.: Recommending news articles in the CLEF news recommendation evaluation lab with the data stream management system odysseus. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation (2017)Google Scholar
  18. 18.
    Ludewig, M., Jannach, D.: Evaluation of session-based recommendation algorithms. CoRR abs/1803.09587 (2018)Google Scholar
  19. 19.
    Phan, T.N., Dang, T.K.: A lightweight indexing approach for efficient batch similarity processing with MapReduce. SN Comput. Sci. (SNCS) 1(1) (2020)Google Scholar
  20. 20.
    Dang, T.K., Pham, D.M.C., Ho, D.D.: On verifying the authenticity of e-commercial crawling data by a semi-crosschecking method. Int. J. Web Inf. Syst. (IJWIS) (2019)Google Scholar
  21. 21.
    Dang, T.K.: Ensuring correctness, completeness and freshness for outsourced tree-indexed data. Inf. Resour. Manag. J. (IRMJ) 21(1), 59–76 (2008)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Ho Chi Minh City University of Technology, VNU-HCMHo Chi Minh CityVietnam
  2. 2.International University, VNU-HCMHo Chi Minh CityVietnam

Personalised recommendations