Using Simulated Students for the Assessment of Authentic Document Retrieval
In the REAP system, users are automatically provided with texts to read that are targeted to their individual reading abilities and needs. To assess such a system, students with different abilities use it, and then researchers measure how well it addresses their needs. In this paper, we describe an approach using simulated students to perform this assessment. This enables researchers to determine if the system functions well enough for the students to learn the curriculum and how factors such as corpus size and retrieval criteria affect performance. We discuss how we have used simulated students to assess the REAP system and to prepare for an upcoming study, as well as future work.
KeywordsStudent Model Document Corpus Corpus Size Lexical Quality Simulated Student
Unable to display preview. Download preview PDF.
- 1.VanLehn, K., Ohlsson, S., Nason, R.: Applications of simulated students: An exploration. Journal of Artificial Intelligence in Education (1994)Google Scholar
- 2.Perfetti, C.A., Britt, M.A., Georgi, M.: Text-based learning and reasoning: Studies in history. Erlbaum, Hillsdale (1995)Google Scholar
- 3.Perfetti, C.A., Hart, L.: The lexical quality hypothesis. In: Vehoeven, L., Elbro, C., Reitsma, P. (eds.) Precursors of functional literacy, Amsterdam/Philadelphia (2001)Google Scholar
- 4.Brown, J., Frishkoff, G., Eskenazi, M.: Automatic question generation for vocabulary assessment. In: Proceedings of HLT/EMNLP 2005, Vancouver, B.C (2005)Google Scholar
- 5.Collins-Thompson, K., Callan, J.: A language modeling approach to predicting reading difficulty. In: Proceedings of the HLT/NAACL 2004 Conference, Boston (2004)Google Scholar
- 6.Brown, J., Eskenazi, M.: Student, text and curriculum modeling for reader-specific document retrieval. In: Proceedings of the IASTED International Conference on Human-Computer Interaction, Phoenix (2005)Google Scholar
- 7.Brown, J., Eskenazi, M.: Retrieval of Authentic Documents for Reader-Specific Lexical Practice. In: Proceedings of InSTIL/ICALL Symposium 2004, Venice, Italy (2004)Google Scholar
- 8.Ritter, F., Avraamides, M.N., Councill, I.G.: Validating Changes to a Cognitive Architecture To More Accurately Model the Effects of Two Example Behavior Moderators. In: Proceedings of the Eleventh Computer-Generated Forces and Behavior Representation Conference, Orlando, FL (2002)Google Scholar