World Wide Web

, Volume 18, Issue 4, pp 997–1017 | Cite as

Distributed architecture for k-nearest neighbors recommender systems

  • Vreixo Formoso
  • Diego Fernández
  • Fidel Cacheda
  • Victor Carneiro


Collaborative filtering is one of the most popular recommendation techniques. While the quality of the recommendations has been significantly improved in the last years, most approaches present poor efficiency and scalability. In this paper, we study several factors that affect the performance of a k-Nearest Neighbors algorithm, and we propose a distributed architecture that significantly improves both throughput and response time. Two techniques for distributing recommender systems, user and item partition, were proposed and evaluated using that simulation model. We have found that user partition is generally better, with a faster response time and higher throughput.


Recommender systems collaborative filtering distributed systems performance simulation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Badue, C.S., Baeza-Yates, R., Ribeiro-Neto, B., Ziviani, A., Ziviani, N.: Analyzing imbalance among homogeneous index servers in a web search system. Inf. Process. Manage. 43, 592–608 (2007). CrossRefGoogle Scholar
  2. 2.
    Bennett, J., Lanning, S.: The netflix prize.In:proceedings of KDD cup and workshop, KDDCup ’07, pp 3–6 (2007). ACM, San Jose,California, USAGoogle Scholar
  3. 3.
    Breese, J.S., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering.In:proceedings of the fourteenth conference on uncertainty in artificial intelligence, UAI’98, pp 43–52. Morgan Kaufmann Publishers Inc., San Francisco (1998). URL Google Scholar
  4. 4.
    Büttcher, S., Clarke, C., Cormack, G.V.: Information retrieval: implementing and evaluating search engines. The MIT Press (2010)Google Scholar
  5. 5.
    Cacheda, F., Carneiro, V., Fernández, D., Formoso, V.: Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems. ACM Trans. Web 5, 2:1–2:33 (2011). doi: 10.1145/1921591.1921593 CrossRefGoogle Scholar
  6. 6.
    Cacheda, F., Carneiro, V., Plachouras, V., Ounis, I.: Performance analysis of distributed information retrieval architectures using an improved network simulation model. Inf. Process. Manage. 43(1), 204–224 (2007). doi: 10.1016/j.ipm.2006.06.002 CrossRefGoogle Scholar
  7. 7.
    Cöster, R., Svensson, M.: Inverted file search algorithms for collaborative filtering. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’02, pp 246–252 (2002). ACM, New York, NY, USAGoogle Scholar
  8. 8.
    Desrosiers, C., Karypis, G. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) : Handbook, Recommender Systems, pp 107–144, Springer (2011)Google Scholar
  9. 9.
    Ding, S., Suel, T.: Faster top-k document retrieval using block-max indexes. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, SIGIR ’11, pp 993–1002 (2011). ACM, New York, NY, USAGoogle Scholar
  10. 10.
    Formoso, V., Fernández, D., Cacheda, F., Carneiro, V.: Using neighborhood pre-computation to increase recommendation efficiency. In Proceedings of the fourth international conference on knowledge discovery and information retrieval (KDIR 2012). (To Appear) (2012)Google Scholar
  11. 11.
    Herlocker, J., Konstan, J.A., Riedl, J.: An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Inf. Retr. 5(4), 287–310 (2002). doi: 10.1023/A:1020443909834 CrossRefGoogle Scholar
  12. 12.
    Jiang, J., Lu, J., Zhang, G., Long, G.: Scaling-up item-based collaborative filtering recommendation algorithm based on hadoop. In Proceedings of the 2011 IEEE world congress on services, SERVICES ’11, pp 490–497. IEEE Computer Society, Washington (2011)CrossRefGoogle Scholar
  13. 13.
    Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009). doi: 10.1109/MC.2009.263 CrossRefGoogle Scholar
  14. 14.
    Linden, G., Smith, B., York, J.: recommendations: Item-to-item collaborative filtering. IEEE Internet Computing 7(1), 76–80 (2003). doi: 10.1109/MIC.2003.1167344 CrossRefGoogle Scholar
  15. 15.
    Little, M.C.: Javasim user’s guide. public release 0.3, version 1.0. (1999).
  16. 16.
    Manning, C.D., Raghavan, P., Schtze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)zbMATHCrossRefGoogle Scholar
  17. 17.
    Papagelis, M., Rousidis, I., Plexousakis, D., Theoharopoulos, E.: Incremental collaborative filtering for highly-scalable recommendation algorithms. In Proceedings of the 15th international conference on Foundations of Intelligent Systems, ISMIS’05, pp 553–561. Springer-Verlag, Berlin, Heidelberg (2005)CrossRefGoogle Scholar
  18. 18.
    Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: Grouplens: an open architecture for collaborative filtering of netnews. In Proceedings of the 1994 ACM conference on Computer supported cooperative work, CSCW ’94, pp 175–186 (1994). ACM, New York, NY, USAGoogle Scholar
  19. 19.
    Ribeiro-Neto, B.A., Barbosa, R.A.: Query performance for tightly coupled distributed digital libraries. In Proceedings of the third ACM conference on digital libraries, DL ’98, pp 182–190 (1998). ACM, New York, NY, USAGoogle Scholar
  20. 20.
    Schelter, S., Boden, C., Markl, V.: Scalable similarity-based neighborhood methods with mapreduce. In Proceedings of the sixth ACM conference on Recommender systems, RecSys ’12, pp 163–170 (2012). ACM, New York, NY, USAGoogle Scholar
  21. 21.
    Turtle, H., Flood, J.: Inf. Process. Manage. 31(6), 831–850 (1995). doi: 10.1016/0306-4573(95)00020-H CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Vreixo Formoso
    • 1
  • Diego Fernández
    • 1
  • Fidel Cacheda
    • 1
  • Victor Carneiro
    • 1
  1. 1.Department of Information and Communication TechnologiesFacultad de Informática, Campus de Elviña s/nCoruñaSpain

Personalised recommendations