Evaluation of ERST – An External Representation Selection Tutor

  • Beate Grawemeyer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4045)


This paper describes the evaluation of ERST, an adaptive system which is designed to improve its users’ external representation (ER) selection accuracy on a range of database query tasks. The design of the system was informed by the results of experimental studies. Those studies examined the interactions between the participants’ background knowledge-of-external representations, their preferences for selecting particular information display forms, and their performance across a range of tasks involving database queries. The paper describes how ERST’s adaptation is based on predicting users’ ER-to-task matching skills and performance at reasoning with ERs, via a Bayesian user model. The model drives ERST’s adaptive interventions in two ways – by 1. hinting to the user that particular representations be used, and/or 2. by removing from the user the opportunity to select display forms which have been associated with prior poor performance for that user. The results show that ERST does improve an individual’s ER reasoning performance. The system is able to successfully predict users’ ER-to-task matching skills and their ER reasoning performance via its Bayesian user model.


Bayesian Network Adaptive System External Representation Database Query Late Trial 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brusilovsky, P., Eklund, J.: A study of user model based link annotation in educational hypermedia. Journal of Universal Computer Science, special issue on assessment issues for educational software 4, 429–448 (1998)Google Scholar
  2. 2.
    Cheng, P.C.-H.: Functional roles for the cognitive analysis of diagrams in problem solving. In: Cottrell, G.W. (ed.) Proceedings of the 18th Annual Conference of the Cognitive Science Society, pp. 207–212. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
  3. 3.
    Conati, C., Gertner, A., VanLehn, K.: Using Bayesian networks to manage uncertainly in student modeling. User Modeling & User-Adapted Interaction 12, 371–417 (2002)MATHCrossRefGoogle Scholar
  4. 4.
    Cox, R., Brna, P.: Supporting the use of external representations in problem solving: The need for flexible learning environments. Journal of Artificial Intelligence in Education 6, 239–302 (1995)Google Scholar
  5. 5.
    Cox, R., Stenning, K., Oberlander, J.: The effect of graphical and sentential logic teaching on spontaneous external representation. Cognitive Studies: Bulletin of the Japanese Cognitive Science Society 2, 5–75 (1995)Google Scholar
  6. 6.
    Cox, R.: Representation construction, externalised cognition and individual differences. Learning and Instruction 9, 343–363 (1999)CrossRefGoogle Scholar
  7. 7.
    Cox, R., Grawemeyer, B.: The mental organisation of external representations. In: European Cognitive Science Conference (EuroCogSci). Osnabrück (2003)Google Scholar
  8. 8.
    Cox, R., Romero, P., du Boulay, B., Lutz, R.: A cognitive processing perspective on student programmers’ ‘graphicacy’. In: Blackwell, A.F., Marriott, K., Shimojima, A. (eds.) Diagrams 2004. LNCS (LNAI), vol. 2980, pp. 344–346. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Day, R.: Alternative representations. In: Bower, G. (ed.) The Psychology of Learning and Motivation, vol. 22, pp. 261–305 (1988)Google Scholar
  10. 10.
    Grawemeyer, B., Cox, R.: The effects of knowledge of external representations and display selection upon database query performance. In: Proceedings of the Second International Workshop on Interactive Graphical Communication (IGC 2003), University of London, Queen Mary (2003)Google Scholar
  11. 11.
    Grawemeyer, B., Cox, R.: A Bayesian approach to modelling user’s information display preferences. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 233–238. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Grawemeyer, B., Cox, R.: Graphical data displays and database queries: Helping users select the right display for the task. In: Butz, A., Fisher, B., Krüger, A., Olivier, P. (eds.) SG 2005. LNCS, vol. 3638, pp. 53–64. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Humphreys, G.W., Riddoch, M.J.: Visual object processing: A cognitive neuropsychological approach. Lawrence Erlbaum Associates, Hillsdale (1987)Google Scholar
  14. 14.
    Jameson, A., Gromann-Hutter, B., March, L., Rummer, R.: Creating an empirical basis for adaptation decisions. In: Lieberman, H. (ed.) IUI 2000: International Conference on Intelligent User Interfaces (2000)Google Scholar
  15. 15.
    Kirby, J.R., Moore, P.J., Schofield, N.J.: Verbal and visual learning styles. Contemporary educational psychology 13, 169–184 (1988)CrossRefGoogle Scholar
  16. 16.
    Kozhevnikov, M., Hegarty, M., Mayer, R.E.: Visual/spatial abilities in problem solving in physics. In: Anderson, M., Mayer, B., Olivier, P. (eds.) Diagrammatic representation and reasoning, Springer, Heidelberg (2002)Google Scholar
  17. 17.
    Mitchell, T.M.: Machine learning. McGraw Hill, New York (1997)MATHGoogle Scholar
  18. 18.
    Norman, D.A.: Things that make us smart. Addison-Wesley, MA (1993)Google Scholar
  19. 19.
    Novick, L.R., Hurley, S.M., Francis, M.: Evidence for abstract, schematic knowledge of three spatial diagram representations. Memory & Cognition 27, 288–308 (1999)CrossRefGoogle Scholar
  20. 20.
    Novick, L.R., Hurley, S.M.: To Matrix, Network, or Hierarchy: That Is the Question. Cognitive Psychology 42, 158–216 (2001)CrossRefGoogle Scholar
  21. 21.
    Pearl, J.: Probabilistic reasoning in intelligent systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
  22. 22.
    Shute, V.J., Gawlick-Grendell, L.A., Young, R.K., Burnham, C.A.: An experimental system for learning probability: Stat Lady description and evaluation. Instructional Science 24, 25–46 (1996)CrossRefGoogle Scholar
  23. 23.
    Stenning, K., Cox, R., Oberlander, J.: Contrasting the cognitive effects of graphical and sentential logic teaching: Reasoning, representation and individual differences. Language and Cognitive Processes 10, 333–354 (1995)CrossRefGoogle Scholar
  24. 24.
    Vessey, I.: Cognitive fit: A theory-based analysis of the graphs versus tables literature. Decision Sciences 22, 219–241 (1991)CrossRefGoogle Scholar
  25. 25.
    Weibelzahl, S.: Problems and Pitfalls in Evaluating Adaptive Systems. In: Chen, S., Magoulas, G. (eds.) Adaptable and Adaptive Hypermedia Systems, pp. 285–299. IRM Press, Heshey (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Beate Grawemeyer
    • 1
  1. 1.Representation & Cognition Group, Department of InformaticsUniversity of SussexFalmer, BrightonUK

Personalised recommendations