User Involvement in Automatic Filtering: An Experimental Study

  • Annika Wærn


The aim in information filtering is to provide users with a personalised selection of information, based on their interest profile. In adaptive information filtering, this profile partially or completely acquired by automatic means. This paper investigates if profile generation can be partially acquired by automatic methods and partially by direct user involvement. The issue is explored through an empirical study of a simulated filtering system that mixes automatic and manual profile generation. The study covers several issues involved in mixed control. The first issue concerns if a machine-learned profile can provide better filtering performance if generated from an initial explicit user profile. The second issue concerns if user involvement can improve on a system-generated or adapted profile. Finally, the relationship between filtering performance and user ratings is investigated. In this particular study the initial setup of a personal profile was effective and yielded performance improvements that persisted after substantiate training. However, the study showed no correlation between users’ ratings of profiles and profile filtering performance, and only weak indications that users could improve profiles that already had been trained on feedback.

experimental study information filtering personal profile recommender systems user involvement 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Balabanovíc, M. and Shoham, Y.: 1997, Fab: Content-based, collaborative recommendation. Communications of the ACM, 40(3), 66–72.CrossRefGoogle Scholar
  2. Baudisch, P.: 1999, Joining Collaborative and Content-based Filtering CHI ‘99 Workshop on Interacting with Recommender Systems, online proceedings. Available from Scholar
  3. Baudish, P.: 1997, The Profile editor: Designing a Direct Manipulative Tool for Assembling Profiles. 5th DELOS Workshop on Filtering and Collaborative Filtering, Budapest, pp. 11–17. Available from Scholar
  4. Breese, J., Heckerman, D. and Kadie, C.: 1998, Empirical Analysis of Predictive Algorithms for Collaborative Filtering 14th Conference on Uncertainty in Artificial Intelligence, San Francisco, Morgan Kaufmann, pp. 43–52.Google Scholar
  5. Bollacker, K., Lawrence, S. and Giles, C. L.: 1998, CiteSeer: An Autonomous Web Agent for Automatic Retrieval and Identification of Interesting Publications Second International Conference on Autonomous Agents, ACM Press, New York, pp. 116–123.Google Scholar
  6. Brin, S. and Page, L.: 1998, The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems, 30(1-7), 107–117.CrossRefGoogle Scholar
  7. Brusilovsky, P., Schwarz, E. and Weber, G.: 1996, A Tool for Developing Adaptive Electronic Textbooks on WWW WebNet’96-World Conference of the Web Society, Boston, MA, pp. 64–69.Google Scholar
  8. Burke, R.: 1999, Integrating Knowledge-based and Collaborative-filtering Recommender Systems. Workshop on AI and Electronic Commerce, AAAI’99, Orlando, Florida.Available from Scholar
  9. Burke R., Hammond, K. and Young, B.: 1997, The FINDME approach to assisted browsing. IEEE Expert, 12(4), 32–40.CrossRefGoogle Scholar
  10. Claypool, M., Gokhale, A., Miranda, T., Murnikov, P., Netes, D. and Sartin, M.: 1999, Combining Content-based and Collaborative Filters in an Online Newspaper. ACM SIGIR Workshop on Recommender Systems. Available from Scholar
  11. Claypool, M., Brown, D. and Phong, M.: 2001, Inferring user interests. IEEE Internet Computing, 5(6), 32–39.CrossRefGoogle Scholar
  12. Diao, Y., Lu, H. and Wu, D.: 2000, A Comparative Study of Classification-based Personal e-mail Filtering, 4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Kyoto, JP, Springer Verlag, Heidelberg, DE, pp. 408–419.Google Scholar
  13. Grasso, A., Meunier, J-L. and Thompson, C.: 2000, Augmenting Recommender Systems by Embedding Interfaces into Practices 33rd Hawaii International Conference on System Sciences-Volume 3, Maui, Hawaii, pp. 3016. Available from Scholar
  14. Herlocker, J., Konstan, J., Borchers, A. and Riedl, J.: 1999, An Algorithmic Framework for Performing Collaborative Filtering The 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Berkeley, California, United States, pp. 230–237.Google Scholar
  15. Herlocker, J., Konstan, J. and Riedl, J.: 2000, Explaining Collaborative Filtering Recommendations ACM 2000 Conference on Computer Supported Cooperative Work, December 2000, Philadelphia, Penn. pp. 241–250.Google Scholar
  16. Hofmann, T. and Puzicha, J.: 1999, Latent Class Models for Collaborative Filtering International Joint Conference on Artificial Intelligence, Stockholm, Morgan Kaufmann Publishers, pp. 688–693.Google Scholar
  17. Karlgren, J., Höök, K., Lantz, A., Palme, J. and Pargman, D.: 1994, The Glass Box User Model for Filtering Report R94014, Departments of Computer and Systems Sciences, Computational Linguistics, and Psychology, Stockholm University. Available from Scholar
  18. Kim, J., Oard, D. and Romanik, K.: 2000, Using Implicit Feedback for User Modeling in Internet and Intranet Searching Technical Report, College of Library and Information Services, University of Maryland at College Park. Available from Scholar
  19. Konstan, J., Miller, B., Maltz, D., Herlocker, J., Gordon, L. and Riedl, J.: 1997, GroupLens: Applying collaborative filtering to usenet news. Communications of the ACM, 40(3),77–87.CrossRefGoogle Scholar
  20. Kohrs, A. and Merialdo, B.: 2000, Using Category-based Collaborative Filtering in the Active Web Museum IEEE International Conference on Multimedia and Expo, New York,pp. 351–354.Google Scholar
  21. Kolodner, J.: 1993, Case-Based Reasoning, Morgan Kaufmann Publishers, San Mateo, California.Google Scholar
  22. Macskassy, S., Hirsh, H. and Dayanik, A.: 1999, EmailValet: Learning Email Preferences for Wireless Platforms IJCAI workshop on Learning About Users, Stockholm. Available from Scholar
  23. Mobasher, B., Dai, H. and Tao, M.: 2002, Discovery and evaluation of aggregate usage profiles for web personalization. Data Mining and Knowledge Discovery, 6, 61–82.CrossRefMathSciNetGoogle Scholar
  24. Munro, A., Höök, K. and Benyon, D. (eds.) 1999, Social Navigation in Information Space, Springer Verlag, London.Google Scholar
  25. Pazzani, M.J.: 2000a, Information Filtering, Classification, and Extraction Tutorial notes, Intelligent User Interfaces Conference, New Orleans, Louisiana.Google Scholar
  26. Pazzani, M. J.: 2000b, Representation of Electronic Mail Filtering Profiles: A User Study Intelligent User Interfaces Conference, New Orleans, Louisiana, pp. 202–206.Google Scholar
  27. Popescul, A., Ungar, L., Pennock, D. and Lawrence, S.: 2001, Probabilistic Models for Unified Collaborative and Content-Based Recommendation in Sparse-Data Environments 17th Conference on Uncertainty in Artificial Intelligence, Seattle, Washington, pp. 437–444.Google Scholar
  28. Rocchio, J.: 1971, Relevance Feedback Information Retrieval. In: G. Salton, (ed.) The Smart Retrieval System, Experiments in Automatic Document Processing, Prentice-Hall, Englewood Cliffs, NJ, pp. 313–323.Google Scholar
  29. Sarwar, B., Konstan, J., Borchersm, A., Herlocker, J., Miller, B. and Riedl, J.: 1998, Using Filtering Agents to Improve Prediction Quality in the Grouplens Research Collaborative Filtering System ACM Conference on Computer Supported Cooperative Work, pp. 345–354.Google Scholar
  30. Schafer, B., Konstan, J. and Riedl, J.: 2001, E-commerce recommendation applications. Data Mining and Knowledge Discovery, 5(1/2), 115–153.CrossRefGoogle Scholar
  31. Terveen, L., Hill, W., Amento, B., McDonald, D. and Creter, J.: 1998, PHOAKS: A system for sharing recommendations. Communications of the ACM, 40(3), 59–62.CrossRefGoogle Scholar
  32. Ungar, L. H. and Foster, D. P.: 1998, Clustering Methods for Collaborative Filtering In:AAAI Workshop on Recommendation Systems, Madison, Wisconsin. Available from Scholar
  33. Waern, A., Tierney, M., Rudström, Å. and Laaksolahti, J.: 1998, ConCall: An Information Service for Researchers Based on EdInfo SICS technical report T98:04. Available as Scholar
  34. Waern, A., Tierney, M., Rudström, Å Laaksolahti, J. and Mård, T.: 1999, ConCall: Edited and Adaptive Information Filtering International Conference on Intelligent User Interfaces, Los Angeles, California, ACM, pp. 185.Google Scholar
  35. Wasfi, A.: 1999, Collecting User Access Patterns for Building User Profiles and Collaborative Filtering International Conference on Intelligent User Interfaces, Los Angeles, California, ACM, pp. 57–64.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Annika Wærn
    • 1
  1. 1.Swedish Institute of Computer Science, Homle LaboratoryKistaSweden

Personalised recommendations