Skip to main content
SpringerLink
Log in

An explanation-based approach for experiment reproducibility in recommender systems

  • Engineering Applications of Neural Networks 2018
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The offline evaluation of recommender systems is typically based on accuracy metrics such as the Mean Absolute Error and the Root Mean Squared Error for error rating prediction and Precision and Recall for measuring the quality of the top-N recommendations. However, it is difficult to reproduce the results since there are various libraries that can be used for running experiments and also within the same library there are many different settings that if not taken into consideration when replicating the results might vary. In this paper, we show that within the use of the same library an explanation-based approach can be used to assist in the reproducibility of experiments. Our proposed approach has been experimentally evaluated using a wide range of recommendation algorithms ranging from collaborative filtering to complicated fuzzy recommendation approaches that can solve the filter bubble problem, a real dataset, and the results show that it is both practical and effective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Polatidis N, Pimenidis E (2018) Reproduction of experiments in recommender systems evaluation based on explanations. In: International conference on engineering applications of neural networks. Springer, Cham, 194–200

  2. Polatidis N, Kapetanakis S, Pimenidis E, Kosmidis K (2018) Reproducibility of experiments in recommender systems evaluation. In: 14th International conference on artificial intelligence applications and innovations, AIAI 2018. IFIP AICT, 519

  3. Said A, Bellogín A (2014) Comparative recommender system evaluation. In: Proceedings of 8th ACM Conference on Recommender Systems—RecSys’14, 129–136

  4. Herlocker JL, Konstan JA, Terveen LG, Riedl JT (2004) Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst 22:5–53

    Article  Google Scholar 

  5. Jannach D, Lerche L, Gedikli F, Bonnin G (2013) What recommenders recommend–an analysis of accuracy, popularity, and sales diversity effects. In: User modeling, adaptation, and personalization, 1–13

  6. Konstan JA, Adomavicius G (2013) Toward identification and adoption of best practices in algorithmic recommender systems research. In: Proceedings of the international workshop on reproducibility and replication in recommender systems evaluation, ACM, New York, NY, USA, 23–28

  7. Bellogin A, Castells P, Said A, Tikk D (2013) Workshop on reproducibility and replication in recommender systems evaluation. In: Proceedings of the 7th ACM conference on Recommender systems—RecSys’13, 485–486

  8. Bellogin A, Castells P, Said A, Tikk D (2014) Report on the workshop on reproducibility and replication in recommender systems evaluation (RepSys). SIGIR Forum. 48:29–35

    Article  Google Scholar 

  9. Košir A, Odić A, Tkalčič M (2013) How to improve the statistical power of the 10-fold cross validation scheme in recommender systems. In: RecSys RepSys 2013: proceedings of the international workshop on reproducibility and replication in recommender systems evaluation, 3–6

  10. Said A, Bellogín A (2014) RiVal—a toolkit to foster reproducibility in recommender system evaluation. In: RecSys 2014 proceedings of 8th ACM conference on recommender systems, 371–372

  11. Hernández del Olmo F, Gaudioso E (2008) Evaluation of recommender systems: a new approach. Expert Syst Appl 35:790–804

    Article  Google Scholar 

  12. Peker S, Kocyigit A (2016) mRHR: a modified reciprocal hit rank metric for ranking evaluation of multiple preferences in top-N recommender systems. In: Dichev C, Agre G (eds) Artificial intelligence: methodology, systems, and applications: 17th international conference, AIMSA 2016, Varna, Bulgaria, September 7–10, 2016, Proceedings, Springer International Publishing, Cham, 320–329

  13. Harper FM, Konstan JA (2015) The movielens datasets. ACM Trans Interact Intell Syst 5:1–19

    Article  Google Scholar 

  14. Pariser E (2011) The filter bubble: What the internet is hiding from you. Penguin, Westminster

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, Engineering and Mathematics, University of Brighton, Brighton, BN2 4GJ, UK

    Nikolaos Polatidis

  2. Department of Civil Engineering, School of Engineering, Democritus University of Thrace, 67100, Xanthi, Greece

    Antonios Papaleonidas & Lazaros Iliadis

  3. Department of Computer Science and Creative Technologies, University of the West of England, Bristol, BS16 1QY, UK

    Elias Pimenidis

Authors
  1. Nikolaos Polatidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonios Papaleonidas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elias Pimenidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lazaros Iliadis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolaos Polatidis.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Polatidis, N., Papaleonidas, A., Pimenidis, E. et al. An explanation-based approach for experiment reproducibility in recommender systems. Neural Comput & Applic 32, 12259–12266 (2020). https://doi.org/10.1007/s00521-019-04274-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04274-x

Keywords

Search

Navigation