Using Simulated Students for the Assessment of Authentic Document Retrieval

  • Jonathan Brown
  • Maxine Eskenazi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4053)


In the REAP system, users are automatically provided with texts to read that are targeted to their individual reading abilities and needs. To assess such a system, students with different abilities use it, and then researchers measure how well it addresses their needs. In this paper, we describe an approach using simulated students to perform this assessment. This enables researchers to determine if the system functions well enough for the students to learn the curriculum and how factors such as corpus size and retrieval criteria affect performance. We discuss how we have used simulated students to assess the REAP system and to prepare for an upcoming study, as well as future work.


Student Model Document Corpus Corpus Size Lexical Quality Simulated Student 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    VanLehn, K., Ohlsson, S., Nason, R.: Applications of simulated students: An exploration. Journal of Artificial Intelligence in Education (1994)Google Scholar
  2. 2.
    Perfetti, C.A., Britt, M.A., Georgi, M.: Text-based learning and reasoning: Studies in history. Erlbaum, Hillsdale (1995)Google Scholar
  3. 3.
    Perfetti, C.A., Hart, L.: The lexical quality hypothesis. In: Vehoeven, L., Elbro, C., Reitsma, P. (eds.) Precursors of functional literacy, Amsterdam/Philadelphia (2001)Google Scholar
  4. 4.
    Brown, J., Frishkoff, G., Eskenazi, M.: Automatic question generation for vocabulary assessment. In: Proceedings of HLT/EMNLP 2005, Vancouver, B.C (2005)Google Scholar
  5. 5.
    Collins-Thompson, K., Callan, J.: A language modeling approach to predicting reading difficulty. In: Proceedings of the HLT/NAACL 2004 Conference, Boston (2004)Google Scholar
  6. 6.
    Brown, J., Eskenazi, M.: Student, text and curriculum modeling for reader-specific document retrieval. In: Proceedings of the IASTED International Conference on Human-Computer Interaction, Phoenix (2005)Google Scholar
  7. 7.
    Brown, J., Eskenazi, M.: Retrieval of Authentic Documents for Reader-Specific Lexical Practice. In: Proceedings of InSTIL/ICALL Symposium 2004, Venice, Italy (2004)Google Scholar
  8. 8.
    Ritter, F., Avraamides, M.N., Councill, I.G.: Validating Changes to a Cognitive Architecture To More Accurately Model the Effects of Two Example Behavior Moderators. In: Proceedings of the Eleventh Computer-Generated Forces and Behavior Representation Conference, Orlando, FL (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jonathan Brown
    • 1
  • Maxine Eskenazi
    • 1
  1. 1.Language Technologies InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations