A Framework for Evaluation of Information Filtering Techniques in an Adaptive Recommender System

  • John O’Donovan
  • John Dunnion
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2945)


This paper proposes that there is a substantial relative difference in the performance of information-filtering algorithms as they are applied to different datasets, and that these performance differences can be leveraged to form the basis of an Adaptive Information Filtering System. We classify five different datasets based on metrics such as sparsity, user-item ratio etc, and develop a regression function over these metrics in order to predict suitability of a particular recommendation algorithm to a new dataset, using only the aforementioned metrics. Our results show that the predicted best algorithm does perform better for the new dataset.


Recommender System Regression Function Collaborative Filter Bayesian Model Aver Recommendation Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Billsus, D., Pazzani, M.J.: Learning collaborative information filters. In: Proc. 15th International Conf. on Machine Learning, pp. 46–54. Morgan Kaufmann, San Francisco (1998)Google Scholar
  2. 2.
    Cotter, P., Smyth, B.: PTV: Intelligent personalised TV guides. In: Proceedings of the 7th Conference on Artificial Intelligence (AAAI 2000) and of the 12th Conference on Innovative Applications of Artificial Intelligence (IAAI 2000), July 30–3, pp. 957–964. AAAI Press, Menlo Park (2000)Google Scholar
  3. 3.
    Goldberg, K., Roeder, T., Gupta, D., Perkins, C.: Eigentaste: A constant time collaborative filtering algorithm. Information Retrieval 4(2), 133–151 (2001)zbMATHCrossRefGoogle Scholar
  4. 4.
    Hayes, C., Cunningham, P., Clerkin, P., Grimaldi, M.: Programme-driven music radio (2002)Google Scholar
  5. 5.
    Karypis, G.: Evaluation of item-based top-n recommendation algorithms. In: CIKM, pp. 247–254 (2001)Google Scholar
  6. 6.
    Melville, P., Mooney, R., Nagarajan, R.: Content-boosted collaborative filtering (2001)Google Scholar
  7. 7.
    O’Donovan, J., Dunnion, J.: A comparison of collaborative recommendation algorithms over diverse data. In: Proceedings of the National Conference on Artificial Intelligence and Cognitive Science (AICS), Ireland, September 17-19, pp. 101–104 (2003)Google Scholar
  8. 8.
    O’Mahony, M.P., Hurley, N., Silvestre, G.C.M.: An attack on collaborative filtering. In: Hameurlain, A., Cicchetti, R., Traunmüller, R. (eds.) DEXA 2002. LNCS, vol. 2453, pp. 494–503. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  9. 9.
    Raftery, A.E., Madigan, D., Hoeting, J.A.: Bayesian model averaging for linear regression models. Journal of the American Statistical Association 92(437), 179–191 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: Grouplens: An open architecture for collaborative filtering of netnews. In: Proceedings of ACM CSCW 1994 Conference on Computer-Supported Cooperative Work, Sharing Information and Creating Meaning, pp. 175–186 (1994)Google Scholar
  11. 11.
    Sarwar, B.M., Karypis, G., Konstan, J.A., Reidl, J.: Itembased collaborative filtering recommendation algorithms. In: World Wide Web, pp. 285–295 (2001)Google Scholar
  12. 12.
    Smyth, B., Wilson, D., O’Sullivan, D.: Improving the quality of the personalised electronic programme guide. In: Proceedings of the TV 2002 the 2nd Workshop on Personalisation in Future TV, May 2002, pp. 42–55 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • John O’Donovan
    • 1
  • John Dunnion
    • 1
  1. 1.Department of Computer ScienceUniversity College DublinIreland

Personalised recommendations