Evaluation of Attribute-Aware Recommender System Algorithms on Data with Varying Characteristics

  • Karen H. L. Tso
  • Lars Schmidt-Thieme
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3918)


The growth of Internet commerce has provoked the use of Recommender Systems (RS). Adequate datasets of users and products have always been demanding to better evaluate RS algorithms. Yet, the amount of public data, especially data containing content information (attributes) is limited. In addition, the performance of RS is highly dependent on various characteristics of the datasets. Thus, few others have conducted studies on synthetically generated datasets to mimic the user-product relationship. Evaluating algorithms based on only one or two datasets is often not sufficient. A more thorough analysis can be conducted by applying systematic changes to data, which cannot be done with real data. However, synthetic datasets that include attributes are rarely investigated. In this paper, we review synthetic datasets applied in RS and present our synthetic data generation methodology that considers attributes. Furthermore, we conduct empirical evaluations on existing hybrid recommendation algorithms and other state-of-the-art algorithms using these variable synthetic data and observe their behavior as the characteristic of data varies. In addition, we also introduce the use of entropy to control the randomness of the generated data.


Recommender System Synthetic Dataset Collaborative Filter Recommendation Algorithm User Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aggarwal, C.C., Wolf, J.L., Wu, K.-L., Yu, P.S.: Horting hatches an egg: A new graph-theoretic approach to collaborative filtering. In: Proceedings of ACMSIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York (1999)Google Scholar
  2. 2.
    Agrawl, R., Srikant, R.: Fast algorithms for mining association rules. In: Proceedings of the 20th International Conference on Very Large Data Bases (VLDB), pp. 487–499. Morgan Kaufmann, San Francisco (1994)Google Scholar
  3. 3.
    Basilico, J., Hofmann, T.: Unifying collaborative and content-based filtering. In: Proceedings of the 21st International Conference on Machine Learning, Banff, Canada (2004)Google Scholar
  4. 4.
    Basu, C., Hirsh, H., Cohen, W.: Recommendation as classification: Using social and content-based information in recommendation. In: Proceedings of the 1998 Workshop on Recommender Systems, pp. 11–15. AAAI Press, Reston (1998)Google Scholar
  5. 5.
    Claypool, M., Gokhale, A., Miranda, T.: Combining content-based and collaborative filters in an online newspaper. In: Proceedings of the SIGIR-99 Workshop on Recommender Systems: Algorithms and Evaluation (1999)Google Scholar
  6. 6.
    Deshpande, M., Karypis, G.: Item-based top-N recommendation algorithms. ACM Transactions on Information Systems 22/1, 143–177 (2004)Google Scholar
  7. 7.
    Goldberg, D., Nichols, D., Oki, B.M., Terry, D.: Using collaborative filtering to weave an information tapestry. Commun. ACM 35, 61–70 (1992)CrossRefGoogle Scholar
  8. 8.
    Good, N., Schafer, J.B., Konstan, J., Borchers, A., Sarwar, B., Herlocker, J., Riedl, J.: Combining Collaborative Filtering with Personal Agents for Better Recommendations. In: Proceedings of the 1999 Conference of the American Association of Artificial Intelligence (AAAI), pp. 439–446 (1999)Google Scholar
  9. 9.
    Herlocker, J., Konstan, J., Borchers, A., Riedl, J.: An Algorithmic Framework for Performing Collaborative Filtering. In: Proceedings of ACM SIGIR 1999, ACM Press, New York (1999)Google Scholar
  10. 10.
    Li, Q., Kim, M.: An Approach for Combining Content-based and Collaborative Filters. In: Proceedings of the Sixth International Workshop on Information Retrieval with Asian Languages (ACL), pp. 17–24 (2003)Google Scholar
  11. 11.
    Konstan, J.A., Miller, B.N., Maltz, D., Herlocker, J.L., Gordon, L.R., Riedl, J.: Group-Lens: Applying collaborative filtering to usenet news. ACM Commun. 40, 77–87 (1997)CrossRefGoogle Scholar
  12. 12.
    Marlin, B., Roweis, S., Zemel, R.: Unsupervised Learning with Non-ignorable Missing Data. In: Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (AISTATS), pp. 222–229 (2005)Google Scholar
  13. 13.
    Melville, P., Mooney, R.J., Nagarajan, R.: Content-Boosted Collaborative Filtering for Improved Recommendations. In: Proceedings of the Eighth National Conference on Artificial Intelligence(AAAI-2002), Edmonton, Canada, pp. 187–192 (2002)Google Scholar
  14. 14.
    Miller, B.N., Riedl, J., Konstan, J.A.: Experiences with GroupLens: Making Usenet useful again. In: Proceedings of the 1997 USENIX Technical Conference (1997)Google Scholar
  15. 15.
    MovieLens (2003), Available at,
  16. 16.
    Pazzani, M.J.: A framework for collaborative, content-based and demographic filtering. Artificial Intelligence Review 13(5-6), 393–408 (1999)CrossRefGoogle Scholar
  17. 17.
    Popescul, A., Ungar, L.H., Pennock, D.M., Lawrence, S.: Probabilistic models for unified collaborative and content-based recommendation in sparse-data environments. In: Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 437–444 (2001)Google Scholar
  18. 18.
    Sarwar, B.M., Karypis, G., Konstan, J.A., Riedl, J.: Analysis of recommendation algorithms for E-commerce. In: Proceedings of the 2nd ACM Conference on Electronic Commerce (EC), pp. 285–295. ACM, New York (2000)Google Scholar
  19. 19.
    Schmidt-Thieme, L.: Compound Classification Models for Recommender Systems. In: Proceedings of the IEEE International Conference on Data Mining (ICDM), New Orleans, USA, pp. 559–570 (2005)Google Scholar
  20. 20.
    Traupman, J., Wilensky, R.: Collaborative Quality Filtering: Establishing Consensus or Recovering Ground Truth? In: Proceedings of WebKDD 2004: KDD Workshop on Web Mining and Web Usage Analysis, in conjunction with the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2004), August 22-25 (2004), Seattle, WA (2004)Google Scholar
  21. 21.
    Tso, H.L.K., Schmidt-Thieme, L.: Attribute-Aware Collaborative Filtering. In: Proceedings of the 29th Annual Conference of the German Classification Society 2005, Magdeburg, Germany (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Karen H. L. Tso
    • 1
  • Lars Schmidt-Thieme
    • 1
  1. 1.Computer-based New Media Group (CGNM), Department of Computer ScienceUniversity of FreiburgFreiburgGermany

Personalised recommendations