Advertisement

Replication and Reproduction in Recommender Systems Research - Evidence from a Case-Study with the rrecsys Library

  • Ludovik ÇobaEmail author
  • Markus Zanker
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10350)

Abstract

Recommender systems (RS) are a real-world application domain for Artificial Intelligence standing at the core of massively used e-commerce and social-media platforms like Amazon, Netflix, Spotify and many more. The research field of recommendation systems now has already a more than 20 years long tradition and issues like replication of results and reproducibility of algorithms become more important. Therefore this work is oriented towards better understanding the underlying challenges of reproducibility of offline measurements of recommendation techniques. We therefore introduce rrecsys, an open-source package in R, that implements many popular RS algorithms, expansion capabilities and has an integrated offline evaluation mechanism following an accepted methodology. In addition, we present a case study on the usability of the library along with results of benchmarking the provided algorithms with other open-source implementations.

Keywords

Recommender System Collaborative Filter Homework Assignment Recommendation Algorithm Normalize Discount Cumulative Gain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Beel, J., Breitinger, C., Langer, S., Lommatzsch, A., Gipp, B.: Towards reproducibility in recommender-systems research. User Model. User Adap. Inter. 26(1), 69–101 (2016)CrossRefGoogle Scholar
  2. 2.
    Çoba, L., Zanker, M.: rrecsys: an r-package for prototyping recommendation algorithms. In: Guy, I., Sharma, A. (eds.) Poster Track of the 10th ACM Conference on Recommender Systems (RecSys 2016) (RecSysPosters). No. 1688 in CEUR Workshop Proceedings, Aachen (2016). http://ceur-ws.org/Vol-1688/#paper-12
  3. 3.
    Eddelbuettel, D., François, R., Allaire, J., Chambers, J., Bates, D., Ushey, K.: Rcpp: Seamless r and C++ integration. J. Stat. Softw. 40(8), 1–18 (2011)CrossRefGoogle Scholar
  4. 4.
    Ekstrand, M.D., Ludwig, M., Konstan, J.A., Riedl, J.T.: Rethinking the recommender research ecosystem: reproducibility, openness, and lenskit. In: Proceedings of the Fifth ACM Conference on Recommender Systems, pp. 133–140. ACM (2011)Google Scholar
  5. 5.
    Harper, F.M., Konstan, J.A.: The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst. (TiiS) 5(4), 19 (2016)Google Scholar
  6. 6.
    Jannach, D., Lerche, L., Gedikli, F., Bonnin, G.: What recommenders recommend – an analysis of accuracy, popularity, and sales diversity effects. In: Carberry, S., Weibelzahl, S., Micarelli, A., Semeraro, G. (eds.) UMAP 2013. LNCS, vol. 7899, pp. 25–37. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38844-6_3 CrossRefGoogle Scholar
  7. 7.
    Jannach, D., Resnick, P., Tuzhilin, A., Zanker, M.: Recommender systems—: beyond matrix completion. Commun. ACM 59(11), 94–102 (2016)CrossRefGoogle Scholar
  8. 8.
    Koren, Y., Bell, R., Volinsky, C., et al.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)CrossRefGoogle Scholar
  9. 9.
    Lemire, D., Maclachlan, A.: Slope one predictors for online rating-based collaborative filtering. In: SDM, vol. 5, pp. 1–5. SIAM (2005)Google Scholar
  10. 10.
    Lops, P., De Gemmis, M., Semeraro, G.: Content-based recommender systems: state of the art and trends. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 73–105. Springer, New York (2011)Google Scholar
  11. 11.
    Pan, R., Zhou, Y., Cao, B., Liu, N.N., Lukose, R., Scholz, M., Yang, Q.: One-class collaborative filtering. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 502–511. IEEE (2008)Google Scholar
  12. 12.
    Ricci, F., Rokach, L., Shapira, B.: Introduction to recommender systems handbook. Springer, New York (2011)Google Scholar
  13. 13.
    Rossetti, M., Stella, F., Zanker, M.: Contrasting offline and online results when evaluating recommendation algorithms. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 31–34. ACM (2016)Google Scholar
  14. 14.
    Said, A., Bellogín, A.: Comparative recommender system evaluation: benchmarking recommendation frameworks. In: Proceedings of the 8th ACM Conference on Recommender systems, pp. 129–136. ACM (2014)Google Scholar
  15. 15.
    Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, pp. 285–295. ACM (2001)Google Scholar
  16. 16.
    Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 257–297. Springer, New York (2011)Google Scholar
  17. 17.
    Zanker, M., Jessenitschnig, M., Jannach, D., Gordea, S.: Comparing recommendation strategies in a commercial context. IEEE Intell. Syst. 22(3), 69–73 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Free University of Bozen-BolzanoBozen-BolzanoItaly

Personalised recommendations

Cite paper

Buy options