Anatomy and Empirical Evaluation of an Adaptive Web-Based Information Filtering System

  • Alessandro Micarelli
  • Filippo Sciarrone
Article

Abstract

A case study in adaptive information filtering systems for the Web is presented. The described system comprises two main modules, named HUMOS and WIFS. HUMOS is a user modeling system based on stereotypes. It builds and maintains long term models of individual Internet users, representing their information needs. The user model is structured as a frame containing informative words, enhanced with semantic networks. The proposed machine learning approach for the user modeling process is based on the use of an artificial neural network for stereotype assignments. WIFS is a content-based information filtering module, capable of selecting html/text documents on computer science collected from the Web according to the interests of the user. It has been created for the very purpose of the structure of the user model utilized by HUMOS. Currently, this system acts as an adaptive interface to the Web search engine ALTA VISTATM. An empirical evaluation of the system has been made in experimental settings. The experiments focused on the evaluation, by means of a non-parametric statistics approach, of the added value in terms of system performance given by the user modeling component; it also focused on the evaluation of the usability and user acceptance of the system. The results of the experiments are satisfactory and support the choice of a user model-based approach to information filtering on the Web.

artificial neural networks case-based reasoning empirical methods information filtering user modeling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aamodt, A. and Plaza, E.: 1994, Case-based easoning: Foundational issues, methodological variations, and system approaches. AI Communications 7(1), 39–59.Google Scholar
  2. Armstrong, R., Freitag, D., Joachims, T. and Mitchell, T.: 1995, WebWatcher: A learning apprentice for the world wide web, AAAI Spring Symposium on Information Gathering from Hetereogeneus,Distributed Environments, March 1995.Google Scholar
  3. Artale, A., Magnini, B. and Strapparava, C.: 1997, WordNet for Italian and its use for Lexical Discrimination. In: M. Lenzerini (ed.): AI*IA-97: Advances in Artificial Intelligence, Lecture Notes in Artificial Intelligence, Vol. 1321. Springer Verlag, pp. 346–356.Google Scholar
  4. Asnicar, F., Di Fant, M. and Tasso, C.: 1997, User model-based information filtering. In: M. Lenzerini (ed.): AI*IA 97: Advances in Artificial Intelligence, Lecture Notes in Artificial Intelligence, Vol. 1321. Springer Verlag, pp. 242–253.Google Scholar
  5. Baclace, P. E.: 1992, Competitive agents for information filtering. Communications of theACM 35(12), 50.Google Scholar
  6. Baeza-Yates, R. and Ribeiro-Neto, B.: 1999, Modern Information Retrieval. Addison-Wesley.Google Scholar
  7. Balabanović, M.: 1998, Exploring versus exploiting when learning user models for text recommendation. User Modeling and User-Adapted Interaction 8(1-2), 71–102.CrossRefGoogle Scholar
  8. Balabanović, M. and Shoham, Y.: 1997, Fab: content-based, collaborative recommendation. Communications of the ACM 40(3), 66–72.CrossRefGoogle Scholar
  9. Brajnik, G. and Tasso, C.: 1994, A shell for developing non-monotonic user modeling systems. International Journal of Human-Computer Studies 40, 31–62.CrossRefGoogle Scholar
  10. Chakrabarti, S., Van den Berg, M. and Dom, B.: 1999, Focused crawling: a new approach to topic-specific Web resource discovery. Computer Networks 31, 1623–1640.CrossRefGoogle Scholar
  11. Chen, Q. and Norcio, A. F.: 1997, Modeling a user’s domain knowledge with neural networks. International Journal of Human-Computer Interaction 9(1), 25–40.CrossRefGoogle Scholar
  12. Chin, D. N.: 2001, Empirical evaluation of user models and user-adapted systems. User Modeling and User-Adapted Interaction 11, 181–194.CrossRefGoogle Scholar
  13. Cohen, P. R.: 1995, Empirical Methods for Artificial Intelligence. MIT Press, Cambridge, Massachusetts.Google Scholar
  14. Devore, J. L.: 1995, Probability and Statistics for Engineering and the Sciences. Brooks/Cole Publishing Company, Monterey, California, fourth edition.Google Scholar
  15. Dix, A., Finlay, J., Aboud, G. and Beale, R.: 1993,Human-Computer Interaction. Prentice-Hall.Google Scholar
  16. Doyle, J.: 1979, A truth maintenance system. Artificial Intelligence 12, 231–272.CrossRefGoogle Scholar
  17. Fellbaum, C.: 1998, WordNet. An Electronic Lexical Database. The MIT Press.Google Scholar
  18. Finin, T. W.: 1989, GUMS-A General User Modeling Shell. In: A. Kobsa and W. Wahlster (eds.): User Models in Dialog Systems, Springer, Berlin, Heidelberg, pp. 411–430.Google Scholar
  19. Forbus, K. D. and De Kleer, J.: 1993, Building Problem Solvers. The MIT Press.Google Scholar
  20. Frakes, W.: 1992, Stemming Algorithms. In: W. Frakes and R. Baeza-Yates (eds.): Information Retrieval: Data Structures & Algorithms, NJ, Prentice Hall, Englewood Cliffs, pp. 131–160.Google Scholar
  21. Gentili, G., Micarelli, A. and Sciarrone, F.: 2003, InfoWeb: An adaptive information filtering system for the cultural heritage domain. Applied Artificial Intelligence 17(8-9), 715–744.Google Scholar
  22. Goldberg, D., Nichols, D., Oki, B. M. and Terry, D.: 1992, Using collaborative filtering to weave the information tapestry. Communications of the ACM 35(12), 61–70.CrossRefGoogle Scholar
  23. Goonatilake, S. and Khebbal, S. (eds.): 1995, Intelligent Hybrid Systems. John Wiley & Sons.Google Scholar
  24. Hanani, U., Shapira, B. and Shoval, P.: 2001, Information filtering: Overview of issues, research and systems. User Modeling and User-Adapted Interaction 11, 203–259.CrossRefGoogle Scholar
  25. Harman, K. D. and Candela, G.: 1990, Retrieving records from a gigabyte of text on a minicomputer using statistical ranking. Journal of the American Society for Information Science 41(8), 581–589.CrossRefGoogle Scholar
  26. Harper, B. D. and Norman, K. L.: 1998, Improving user satisfaction: The questionnaire for user interaction satisfaction version 5.5. Proceedings of Mid Atlantic Human Factors Conference, Virginia Beach, February 1998, pp. 224–228.Google Scholar
  27. Haykin, S.: 1994, Neural Networks-A Comprehensive Foundation. Prentice Hall International.Google Scholar
  28. Jameson, A.:1999, User-adaptive systems: An integrative overview. Tutorial presented at: Seventh International Conference on User Modeling UM-99, Banff, Canada, June 20.Google Scholar
  29. Jennings, A. and Higuchi, H.: 1993, A user model neural network for a personal news service. User Modeling and User-Adapted Interaction 3(1), 1–25.CrossRefGoogle Scholar
  30. Joachims, T., Freitag, D. and Mitchell, T.: 1997, WebWatcher: A Tour Guide for the World Wide Web. Proceedings of the International Joint Conference on Artificial Intelligence IJCAI-97, pp. 770–775, August 1997.Google Scholar
  31. Kay, J.: 1995, TheUMtoolkit for cooperative user modeling. User Modeling and User-Adapted Interaction 4(3), 149–196.CrossRefGoogle Scholar
  32. Kemeney, J. G. and Snell, J. L., 1962, Mathematical Models in the Social Sciences. New York, Blaisdell.Google Scholar
  33. Kobsa, A., and Pohl, W.: 1995, The user modeling shell system BGP-MS. User Modeling and User-Adapted Interaction 4(2), 59–106.CrossRefGoogle Scholar
  34. Leuski, A. and Allan, J.: 2004, Interactive information retrieval using clustering and spatial proximity. User Modeling and User-Adapted Interaction 14(2-3), 259–288.CrossRefGoogle Scholar
  35. Maes, P.: 1994, Agents that reduce work and information overload. Communications of the ACM 37(7), 30–40.CrossRefGoogle Scholar
  36. Magnini, B. and Strapparava, C.: 2004, User modelling for news web sites with word sense based techniques. User Modeling and User-Adapted Interaction 14(2-3), 239–257.CrossRefGoogle Scholar
  37. McAllester, D. A.: 1980, An Outlook on Truth Maintenance. AI Memo 551, MIT AI Laboratory, Cambridge, Massachusetts, 924–929.Google Scholar
  38. Micarelli, A., Sciarrone, F., Ambrosini, L. and Cirillo, V.:1998, A Case-Based Approach to User Modeling. In: B. Smyth and P. Cunningham (eds.): Advances in Case-Based Reasoning, Lecture Notes in Artificial Intelligence, Vol. 1488, Springer-Verlag, Berlin, pp. 310–321.Google Scholar
  39. Mladenic, J.: 1999, Text-learning and related intelligent agents: A survey. IEEE Intelligent Systems 14(4), 44–54.CrossRefGoogle Scholar
  40. Mock, K. J. and Vemuri, V. R: 1997, Information filtering via hill climbing, wordnet and index patterns. Information Processing and Management 33(5), 633–644.CrossRefGoogle Scholar
  41. Mostafa, J., Mukhopadhyay, S., Lam, W. and Palakal, M.: 1997, A multi-level approach to intelligent information filtering: Model, system and evaluation. ACM Transactions on Information Systems 15(4), 368–399.CrossRefGoogle Scholar
  42. Narendra, K. S. and Thathachar, M. A. L.: 1989, Learning Automata-An Introduction. NJ: Prentice-Hall, Englewood Cliffs.Google Scholar
  43. Oard, D. W.: 1997, The state of the art in text filtering. User Modeling and User-Adapted Interaction 7(3), 141–178.CrossRefGoogle Scholar
  44. Pazzani, M. and Billsus, D.: 1997, Learning and revising user profiles: The identification of interesting web sites. Machine Learning 27, 313–331.CrossRefGoogle Scholar
  45. Rich, E.: 1989, Stereotypes and User Modeling. In: A. Kobsa and W. Wahlster (eds.): User Models in Dialog Systems, Springer-Verlag, pp. 35–51.Google Scholar
  46. Rumelhart, D. E. and McClelland, J. L. (eds.): 1986, Parallel Distributed Processing. MIT Press, Cambridge, Massachusetts.Google Scholar
  47. Salton, G.: 1971, The SMART Retrieval System-Experiments in Automatic Document Processing. NJ: Prentice Hall Inc., Englewood Cliffs.Google Scholar
  48. Salton, G. and Lesk, L.: 1968, Computer evaluation of indexing and text processing. Journal of the ACM 15(1), 8–36.CrossRefGoogle Scholar
  49. Shapira, B., Shoval, P. and Hanani, U.: 1997, Stereotypes in information filtering systems. Information Processing & Management 33(3), 273–287.Google Scholar
  50. Shapira, B., Shoval, P. and Hanani, U.: 1999, Experimentation with an information filtering system that combines cognitive and Sociological filtering integrated with user stereotypes. Decision Support Systems 27, 5–24.CrossRefGoogle Scholar
  51. Shneiderman, B.: 1998, Designing the User Interface-Strategies for Effective Human-Computer Interaction. Addison-Wesley, Third Edition.Google Scholar
  52. Siegel, S. and Castellan, N. J.:1988, Nonparametric Statistics for the Behavioral Sciences. New York: McGraw-Hill Inc..Google Scholar
  53. Waern, A.: 2004, User Involvement in Automatic Filtering-an experimental study. User Modeling and User-Adapted Interaction 14(2-3), 201–237.CrossRefGoogle Scholar
  54. Weibelzahl, S. and Weber, G.: 2001, A Database of empirical evaluations of adaptive systems. In: R. Klingerberg, S. Ruping, A. Fick, N. Henze, C. Herzog, R. Molitor and O. Schroeder (eds.): Proceedings of Workshop Lernen-Lehren-Wissen LLWA 01, Research Report in Computer Science nr. 763, University of Dortmund, pp. 302–306.Google Scholar
  55. Wilcoxon, F.: 1947, Probability tables for individual comparisons by ranking methods. Biometrics 3, 119–122.Google Scholar
  56. Wong, S. K., Yao, Y. Y. and Bollmann, P.: 1988, Linear Structure in Information Retrieval. Proceedings of The 18th International ACM Conference on Research and Development in Information Retrieval SIGIR-88, pp. 219–232.Google Scholar
  57. Wong, S. K. and Yao, Y. Y.: 1990, Query formulation in linear retrieval models. Journal of the American Society for Information Science 41(5), 334–341.CrossRefGoogle Scholar
  58. Yao, Y.Y.: 1995, Measuring retrieval e?ectiveness based on user preference of documents. Journal of the American Society for Information Science 46(2), 133–145.CrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Alessandro Micarelli
    • 1
  • Filippo Sciarrone
    • 1
  1. 1.Department of Computer Science and Automation, Artificial Intelligence LaboratoryUniversity of Roma TreRomeItaly

Personalised recommendations