Recommender Systems for Learning: Building User and Expert Models through Long-Term Observation of Application Use

  • Frank Linton
  • Hans-Peter Schaefer


Information technology has recently become the medium in which much professional office work is performed. This change offers an unprecedented opportunity to observe and record exactly how that work is performed. We describe our observation and logging processes and present an overview of the results of our long-term observations of a number of users of one desktop application. We then present our method of providing individualized instruction to each user by employing a new kind of user model and a new kind of expert model. The user model is based on observing the individual's behavior in a natural environment, while the expert model is based on pooling the knowledge of numerous individuals. Individualized instructional topics are selected by comparing an individual's knowledge to the pooled knowledge of her peers.

agent cluster analysis data mining instructional technology instrumentation knowledge acquisition learning logging long-term observation organizational learning OWL recommender system sequence analysis student modeling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Anderson, J., Corbett, A., Fincham, J., Hoffman, D. and Pelletire, R.: 1992, General principles for an intelligent tutoring architecture. In: J. Regian and V. Shute (eds.), Cognitive Approaches to Automated Instruction, Mawah, NJ: Erlbaum, pp. 81-106.Google Scholar
  2. Baudisch, P.: 1999, CHI '99 Workshop: Interacting with Recommender Systems, 15/16 May 1999. Part of the CHI '99 Conference, Pittsburgh, Pennsylvania, USA. Scholar
  3. Cheikes, B., Geier, M., Hyland, R., Linton, F., Riffe, A., Rodi, L. and Schaefer, H.: 1998, Embedded training for complex information systems. International Journal of Artificial Intelligence in Education 10, 314-334.Google Scholar
  4. Collins, J., Greer, J., Kumar, V., McCalla, G., Meagher, P. and Tkatch, R.: 1997, Inspectable user models for Just-In-Time workplace training. In: A. Jameson, C. Paris and C. Tasso (eds.), User Modeling: Proceedings of the Sixth International Conference, UM97, New York, Springer Verlag, pp. 327-337.Google Scholar
  5. Draper, S.: 1984, The nature of expertise in UNIX. In: B. Shackel (ed.), Proceedings of Human-Computer Interaction-INTERACT '84, Elsevier Science Publishers B. V. (North-Holland), pp. 465-471.Google Scholar
  6. Fischer, G.: 1999, User modeling: The long and winding road. In: J. Kay (ed.), UM99: User Modeling: Proceedings of the Seventh International Conference, New York, Springer Verlag, pp. 349-355.Google Scholar
  7. Fukunaga, K.: 1990, Introduction to Statistical Pattern Recognition, San Diego, Academic Press.Google Scholar
  8. Horvitz, E., Breese, J., Heckerman, D., Hovel, D. and Rommelse, K.: 1998, The Lumiere Project: Bayesian user modeling for inferring the goals and needs of software users. In: Proceedings of the FourteenthConference on Uncertainty inArtificial Intelligence, Madison, WI, July 1998, Morgan Kaufmann, San Francisco, pp. 256-265.Google Scholar
  9. Howell. D.: 1982, Statistical methods for psychology, Boston, Duxbury Press.Google Scholar
  10. Kautz, H.: 1998, Recommender Systems: Papers from the AAAI Workshop, Henry Kautz, Chair, July 27, 1998, Madison Wisconsin. Technical Report WS-98-08. Scholar
  11. Kay, J.: 1994, Lies, damned lies and stereotypes: Pragmatic approximations of users. In: B. Goodman, A. Kobsa and D. Litman (eds.), Proceedings of UM94-1994 User Modeling Conference, Boston, User Modeling Inc., pp. 175-184.Google Scholar
  12. Kay, J. and Thomas, R.: 1995, Studying long-term system use; computers, end-user training and learning, Communications of the ACM 38(7), 61-69.Google Scholar
  13. Kirkpatrick, D. L.: 1975, Techniques for evaluating training programs. In: D. Kirkpatrick (ed.), Evaluating training programs, Madison, WI, American Society for Training and Development, pp. 1-17.Google Scholar
  14. Lesgold, A., Eggan, G., Katz. S. and Rao, G.: 1992, Possibilities for assessment using computer-based apprenticeship environments. In: J. Regian and V. Shute (eds.), Cognitive Approaches to Automated Instruction, Mawah, NJ, Erlbaum, pp. 49-80.Google Scholar
  15. Linton, F., Joy, D. and Schaefer, H-P.: 1999, Building user and expert models by long-term observation of application usage. In: J. Kay (ed.), UM99: User Modeling: Proceedings of the Seventh International Conference, New York, Springer Verlag, pp. 129-138.Google Scholar
  16. Linton, F.: 1999, Dataset: Usage of Microsoft Word Commands. A repository of 70,000 rows of data at the Machine Learning for User Modeling web site: Scholar
  17. Linton, F., Joy, D., Schaefer, H-P. and Charron, A.: 2000, OWL: A recommender system for organization-wide learning. Educational Technology and Society., 62-76.Google Scholar
  18. Patton, M.: 1990, Qualitative Evaluation and Research Methods, 2nd edn, London: Sage.Google Scholar
  19. Resnick, P. and Varian, H.: 1997, Introduction to special section on recommender systems. Communications of the ACM 40(3), 56-58. Scholar
  20. Thomas, R.: 1996, Long term exploration and use of a text editor. Unpublished doctoral dissertation, University of Western Australia.Google Scholar
  21. Thomas, R.: 1998, Long term human-computer interaction: An exploratory perspective, New York, Springer Verlag.Google Scholar
  22. Triola, M.: 1983, Elementary Statistics, 2nd edn, Menlo Park, CA, Benjamin/Cummings.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Frank Linton
    • 1
  • Hans-Peter Schaefer
    • 1
  1. 1.The MITRE CorporationBedfordUSA

Personalised recommendations