Supporting Representational Competences Through Adaptive Educational Technologies

  • Martina A. RauEmail author
Part of the Models and Modeling in Science Education book series (MMSE, volume 11)


Helping students acquire representational competences is an important educational goal in many STEM domains. In particular, students need to acquire connection-making competences: they need to conceptually understand how different representations map to one another, and they need to be perceptually fluent in translating between representations. I present a number of experiments on instructional support for connection-making competences. The experiments were conducted in the context of intelligent tutoring systems for elementary-school fractions and undergraduate chemistry. Intelligent tutoring systems are educational technologies that adapt to the individual student’s knowledge level in real time, based on a cognitive model of their learning that is updated throughout the learning experience. Results show that the effectiveness of different types of instructional support depends on a number of student characteristics. Support for conceptual connection-making competences is most effective for students who have a basic level of prior domain knowledge but who are not yet proficient. Support for perceptual fluency in translating between representations is most effective for students with high mental rotation ability. Furthermore, the sequence in which conceptual and perceptual connection-making support should be provided depends on the students’ prior conceptual understanding of connections. These findings suggest that adaptive educational technologies might be more effective if they adaptively select the appropriate type of representational support based on a real-time assessment of the student’s current knowledge level. Such adaptive support is likely to not only result in better learning of the domain knowledge but also in better attainment of representational competences.


Connection making Multiple representations Conceptual learning Perceptual learning Intelligent tutoring systems 


  1. Ainsworth, S. (2006). Deft: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16, 183–198.CrossRefGoogle Scholar
  2. Ainsworth, S. (2008a). How should we evaluate multimedia learning environments? Understanding multimedia documents (pp. 249–265).CrossRefGoogle Scholar
  3. Ainsworth, S. (2008b). How do animations influence learning? In D. H. Robinson & G. Schraw (Eds.), Current perspectives on cognition, learning, and instruction: Recent innovations in educational technology that facilitate student learning (pp. 37–67). Charlotte: Information Age Publishing.Google Scholar
  4. Ainsworth, S. (2008c). The educational value of multiple-representations when learning complex scientific concepts. In J. K. Gilbert, M. Reiner, & A. Nakama (Eds.), Visualization: Theory and Practice in Science Education (pp. 191–208). Netherlands: Springer.CrossRefGoogle Scholar
  5. Ainsworth, S., Bibby, P., & Wood, D. (2002). Examining the effects of different multiple representational systems in learning primary mathematics. Journal of the Learning Sciences, 11, 25–61.CrossRefGoogle Scholar
  6. Arcavi, A. (2003). The role of visual representations in the learning of mathematics. Educational Studies in Mathematics, 52, 215–241.CrossRefGoogle Scholar
  7. Berthold, K., & Renkl, A. (2009). Instructional aids to support a conceptual understanding of multiple representations. Journal of Educational Research, 101(1), 70–87.Google Scholar
  8. Berthold, K., Eysink, T. H. S., & Renkl, A. (2008). Assisting self-explanation prompts are more effective than open prompts when learning with multiple representations. Instructional Science, 27, 345–363.Google Scholar
  9. Betrancourt, M. (2005). The animation and interactivity principles in multimedia Learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 287–296). New York: Cambridge University Press.CrossRefGoogle Scholar
  10. Bodemer, D., & Faust, U. (2006). External and mental referencing of multiple representations. Computers in Human Behavior, 22, 27–42.CrossRefGoogle Scholar
  11. Bodemer, D., Ploetzner, R., Feuerlein, I., & Spada, H. (2004). The active integration of information during learning with dynamic and interactive visualisations. Learning and Instruction, 14, 325–341.CrossRefGoogle Scholar
  12. Bodemer, D., Ploetzner, R., Bruchmüller, K., & Häcker, S. (2005). Supporting learning with interactive multimedia through active integration of representations. Instructional Science, 33, 73–95.CrossRefGoogle Scholar
  13. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293–332.CrossRefGoogle Scholar
  14. Charalambous, C. Y., & Pitta-Pantazi, D. (2007). Drawing on a theoretical model to study students’ understandings of fractions. Educational Studies in Mathematics, 64, 293–316.CrossRefGoogle Scholar
  15. Chi, M. T. H., Feltovitch, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152.CrossRefGoogle Scholar
  16. Cook, M., Wiebe, E. N., & Carter, G. (2007). The influence of prior knowledge on viewing and interpreting graphics with macroscopic and molecular representations. Science Education, 92, 848–867.CrossRefGoogle Scholar
  17. Corbett, A. T., Koedinger, K., & Hadley, W. S. (2001). Cognitive tutors: From the research classroom to all classrooms. In P. S. Goodman (Ed.), Technology enhanced learning:Opportunities for change (pp. 235–263). Mahwah: Lawrence Erlbaum Associates Publishers.Google Scholar
  18. Cramer, K. (2001). Using models to build an understanding of functions. Mathematics Teaching in the Middle School, 6, 310–318.Google Scholar
  19. Dori, Y. J., & Barak, M. (2001). Virtual and physical molecular modeling: Fostering model perception and spatial understanding. Educational Technology & Society, 4, 61–74.Google Scholar
  20. Dreyfus, H., & Dreyfus, S. E. (1986). Five steps from novice to expert mind over machine: The power of human intuition and expertise in the era of the computer (pp. 16–51). New York: The Free Press.Google Scholar
  21. Eilam, B., & Poyas, Y. (2008). Learning with multiple representations: Extending multimedia learning beyond the lab. Learning and Instruction, 18, 368–378.CrossRefGoogle Scholar
  22. Even, R. (1998). Factors involved in linking representations of functions. The Journal of Mathamtical Behavior, 17, 105–121.CrossRefGoogle Scholar
  23. Gegenfurtner, A., Lehtinen, E., & Säljö, R. (2011). Expertise differences in the comprehension of visualizations: A meta-analysis of eye-tracking research in professional domains. Educational Psychology Review, 23, 523–552.CrossRefGoogle Scholar
  24. Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7, 155–170.CrossRefGoogle Scholar
  25. Gibson, E. J. (1969). Principles of perceptual learning and development. New York: Prentice Hall.Google Scholar
  26. Gibson, E. J. (2000). Perceptual learning in development: Some basic concepts. Ecological Psychology, 12, 295–302.CrossRefGoogle Scholar
  27. Gilbert, J. K. (2008). Visualization: An emergent field of practice and inquiry in science education. In J. K. Gilbert, M. Reiner, & M. B. Nakhleh (Eds.), Visualization: Theory and practice in science education (pp. 3–24). Dordrecht: Springer.CrossRefGoogle Scholar
  28. Gutwill, J. P., Frederiksen, J. R., & White, B. Y. (1999). Making their own connections:Students’ understanding of multiple models in basic electricity. Cognition and Instruction, 17, 249–282.CrossRefGoogle Scholar
  29. Hegarty, M., & Waller, D. A. (2005). Individual differences in spatial abilities. In P. Shah & A. Miyake (Eds.), The Cambridge handbook of visuospatial thinking (pp. 121–169). New York: Cambridge University Press.CrossRefGoogle Scholar
  30. Holzinger, A., Kickmeier-Rust, M. D., & Albert, D. (2008). Dynamic media in computer science education; Content complexity and learning performance: Is less more? Educational Technology & Society, 11, 279–290.Google Scholar
  31. Jones, L. L., Jordan, K. D., & Stillings, N. A. (2005). Molecular visualization in chemistry education: The role of multidisciplinary collaboration. Chemistry Education Research and Practice, 6, 136–149.CrossRefGoogle Scholar
  32. de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68, 179–201.CrossRefGoogle Scholar
  33. de Jong, T., Ainsworth, S. E., Dobson, M., Van der Meij, J., Levonen, J., & Reimann, P. (1998). Acquiring knowledge in science and mathematics: The use of multiple representations in technology-based learning environments. In M. W. Van Someren, W. Reimers, H. P. A. Boshuizen, & T. de Jong (Eds.), Learning with Multiple Representations (pp. 9–41). Bingley: Emerald Group Publishing Limited.Google Scholar
  34. Kellman, P. J., & Garrigan, P. B. (2009). Perceptual learning and human expertise. Physics of Life Reviews, 6, 53–84.CrossRefGoogle Scholar
  35. Kellman, P. J., & Massey, C. M. (2013). Perceptual learning, cognition, and expertise. The psychology of learning and motivation, 558, 117–165.CrossRefGoogle Scholar
  36. Kellman, P. J., Massey, C. M., Roth, Z., Burke, T., Zucker, J., Saw, A., .Wise, J. (2008).Perceptual learning and the technology of expertise: Studies in fraction learning and algebra. Pragmatics & Cognition, 16, 356-405.CrossRefGoogle Scholar
  37. Kellman, P. J., Massey, C. M., & Son, J. Y. (2009). Perceptual learning modules in mathematics: Enhancing students’ pattern recognition, structure extraction, and fluency. Topics in Cognitive Science, 1, 285–305.Google Scholar
  38. Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19, 239–264.CrossRefGoogle Scholar
  39. Koedinger, K. R., & Corbett, A. (2006). Cognitive tutors: Technology bringing learning sciences to the classroom. New York: Cambridge University Press.Google Scholar
  40. Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012). The knowledge-learning-instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36, 757–798.CrossRefGoogle Scholar
  41. Kordaki, M. (2010). A drawing and multi-representational computer environment for beginners’ learning of programming using C: Design and pilot formative evaluation. Computers & Education, 54, 69–87.CrossRefGoogle Scholar
  42. Kozma, R., & Russell, J. (2005). Students becoming chemists: Developing representational competence. In J. Gilbert (Ed.), Visualization in science education (pp. 121–145). Dordrecht: Springer.CrossRefGoogle Scholar
  43. Kozma, R., Chin, E., Russell, J., & Marx, N. (2000). The roles of representations and tools in the chemistry laboratory and their implications for chemistry learning. The Journal of the Learning Sciences, 9, 105–143.CrossRefGoogle Scholar
  44. Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science: A Multidisciplinary Journal, 11, 65–100.CrossRefGoogle Scholar
  45. Lewalter, D. (2003). Cognitive strategies for learning from static and dynamic visuals. Learning and Instruction, 13, 177–189.CrossRefGoogle Scholar
  46. Linenberger, K. J., & Bretz, S. L. (2012). Generating cognitive dissonance in student interviews through multiple representations. Chemistry Education Research and Practice, 13, 172–178.CrossRefGoogle Scholar
  47. Massey, C. M., Kellman, P. J., Roth, Z., & Burke, T. (2011). Perceptual learning and adaptive learning technology - developing new approaches to mathematics learning in the classroom. In N. L. Stein & S. W. Raudenbush (Eds.), Developmental cognitive science goes to school (pp. 235–249). New York: Routledge.Google Scholar
  48. Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction, 13, 125–139.CrossRefGoogle Scholar
  49. van der Meij, J., & de Jong, T. (2006). Supporting students’ learning with multiple representations in a dynamic simulation-based learning environment. Learning and Instruction, 16, 199–212.CrossRefGoogle Scholar
  50. Van der Meij, J., & de Jong, T. (2011). The effects of directive self-explanation prompts to support active processing of multiple representations in a simulation-based learning environment. Journal of Computer Assisted Learning, 27, 411–423.CrossRefGoogle Scholar
  51. Moss, J. (2005). Pipes, tubes, and beakers: New approaches to teaching the rational-number system. In J. Brantsford & S. Donovan (Eds.), How people learn: A targeted report for teachers (pp. 309–349). Washington, D.C.: National Academy Press.Google Scholar
  52. NCTM. (2000). Principles and standards for school mathematics. Reston: National Council of Teachers of Mathematics..Google Scholar
  53. NCTM. (2006). Curriculum focal points for prekindergarten through grade 8 mathematics: A quest for coherence. VA: Reston.Google Scholar
  54. Özgün-Koca, S. A. (2008). Ninth grade students studying the movement of fish to learn about linear relationships: The use of video-based analysis software in mathematics classrooms. The Mathematics Educator, 18, 15–25.Google Scholar
  55. Pape, S. J., & Tchoshanov, M. A. (2001). The role of representation (s) in developing mathematical understanding. Theory into Practice, 40, 118–127.CrossRefGoogle Scholar
  56. Patel, Y., & Dexter, S. (2014). Using multiple representations to build conceptual understanding in science and mathematics. In M. Searson & M. Ochoa (Eds.), Proceedings of society for information technology & teacher education international conference 2014 (pp. 1304–1309). Chesapeake: AACE.Google Scholar
  57. Peters, M., Laeng, B., Latham, K., Jackson, M., Zaiyouna, R., & Richardson, C. (1995). A redrawn Vandenberg & Kuse mental rotations test: Different versions and factors that affect performance. Brain and Cognition, 28, 39–58.CrossRefGoogle Scholar
  58. Rau, M. A., & Evenstone, A. L. (2014). Multi-methods approach for domain-specific grounding: An ITS for connection making in chemistry. In S. Trausan-Matu, K. E. Boyer, M. Crosby & K. Panourgia (Eds.), Proceedings of the 12th International conference on intelligent tutoring systems (pp. 426–435). Berlin/Heidelberg: Springer.CrossRefGoogle Scholar
  59. Rau, M. A., & Wu, S. P. W. (2015). ITS support for conceptual and perceptual processes in learning with multiple graphical representations. In C. Conati, N. Heffernan, A. Mitrovic, & M. F. Verdejo (Eds.), Artificial intelligence in education (pp. 398–407). Switzerland: Springer International Publishing.CrossRefGoogle Scholar
  60. Rau, M. A., Aleven, V., Rummel, N., & Rohrbach, S. (2012). Sense making alone doesn’t do it: Fluency matters too! Its support for robust learning with multiple representations. In S. Cerri, W. Clancey, G. Papadourakis, & K. Panourgia (Eds.), Intelligent tutoring systems (pp. 174–184). Berlin: Springer.CrossRefGoogle Scholar
  61. Rau, M. A., Aleven, V., Rummel, N., & Rohrbach, S. (2013). Why interactive learning environments can have it all: Resolving design conflicts between conflicting goals. In Proceedings of the SIGCHI 2013 ACM conference on human factors in computing systems (pp. 109–118). New York: ACM.Google Scholar
  62. Rau, M. A., Aleven, V., & Rummel, N. (2014a). Sequencing sense-making and fluency-building support for connection making between multiple graphical representations. In J. L. Polman, E. A. Kyza, D. K. O'Neill, I. Tabak, W. R. Penuel, A. S. Jurow, K. O'Connor, T. Lee, & L. D'Amico (Eds.), Learning and becoming in practice: The international conference of the learning sciences (ICLS 2014) (pp. 977–981). Boulder: International Society of the Learning Sciences.Google Scholar
  63. Rau, M. A., Aleven, V., Rummel, N., & Pardos, Z. (2014b). How should intelligent tutoring systems sequence multiple graphical representations of fractions? A multi-methods study. International Journal of Artificial Intelligence in Education, 24, 125–161.CrossRefGoogle Scholar
  64. Rau, M. A., Michaelis, J. E., & Fay, N. (2015). Connection making between multiple graphical representations: A multi-methods approach for domain-specific grounding of an intelligent tutoring system for chemistry. Computers and Education, 82, 460–485.CrossRefGoogle Scholar
  65. Richman, H. B., Gobet, F., Staszewski, J. J., & Simon, H. A. (1996). Perceptual and memory processes in the acquisition of expert performance: The epam model. In K. A. Ericsson (Ed.), The road to excellence? The acquisition of expert performance in the arts and sciences, sports and games (pp. 167–187). Mahwah: Erlbaum Associatees.Google Scholar
  66. Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 49–69). New York: Cambridge University Press.CrossRefGoogle Scholar
  67. Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13, 141–156.CrossRefGoogle Scholar
  68. Schwonke, R., Renkl, A., Salden, R., & Aleven, V. (2011). Effects of different ratios of worked solution steps and problem solving opportunities on cognitive load and learning outcomes. Computers in Human Behavior, 27, 58–62.CrossRefGoogle Scholar
  69. Seufert, T. (2003). Supporting coherence formation in learning from multiple representations. Learning and Instruction, 13, 227–237.CrossRefGoogle Scholar
  70. Seufert, T., & Brünken, R. (2006). Cognitive load and the format of instructional aids for coherence formation. Applied Cognitive Psychology, 20, 321–331.CrossRefGoogle Scholar
  71. Siegler, R. S., Carpenter, T., Fennell, F., Geary, D., Lewis, J., Okamoto, Y., . . . Wray, J. (2010). Developing effective fractions instruction: A practice guide. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.Google Scholar
  72. Stern, E., Aprea, C., & Ebner, H. G. (2003). Improving cross-content transfer in text processing by means of active graphical representation. Learning and Instruction, 13, 191–203.CrossRefGoogle Scholar
  73. Stieff, M. (2007). Mental rotation and diagrammatic reasoning in science. Learning and Instruction, 17, 219–234.CrossRefGoogle Scholar
  74. Taber, S. B. (2001). Making connections among different representations: The case of multiplication of fractions. Paper presented at the Annual meeting of the American Educational Research Association (Seattle, WA, April 10–14, 2001).Google Scholar
  75. Talanquer, V. (2013). Chemistry education: Ten facets to shape us. Journal for Research in Mathematics Education, 90, 832–838.Google Scholar
  76. Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R., Warren, C., & Newcombe, N. S. (2013). The malleability of spatial skills: A meta-analysis of training studies. Psychological Bulletin, 139, 352–402.CrossRefGoogle Scholar
  77. Van Labeke, N., & Ainsworth, S. E. (2002). Representational decisions when learning population dynamics with an instructional simulation. In S. A. Cerri, G. Gouardères & F. Paraguacu (Eds.), Proceedings of the 6th international conference intelligent tutoring systems (pp. 831–840): Springer Verlag.Google Scholar
  78. Van Someren, M. W., Boshuizen, H. P. A., & de Jong, T. (1998). Multiple representations in human reasoning. In M. W. Van Someren, H. P. A. Boshuizen, & T. de Jong (Eds.), Learning with multiple representations (pp. 1–9). Pergamon: Oxford.Google Scholar
  79. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems and other tutoring systems. Educational Psychologist, 46, 197–221.CrossRefGoogle Scholar
  80. Vreman-de Olde, C., & De Jong, T. (2007). Scaffolding learners in designing investigation assignments for a computer simulation. Journal of Computer Assisted Learning, 22, 63–73.CrossRefGoogle Scholar
  81. Wai, J., Lubinski, D., & Benbow, C. P. (2009). Spatial ability for stem domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101, 817–835.CrossRefGoogle Scholar
  82. Wise, J. A., Kubose, T., Chang, N., Russell, A., & Kellman, P. J. (2000). Perceptual learning modules in mathematics and science instruction. In P. Hoffman & D. Lemke (Eds.), Teaching and learning in a network world (pp. 169–176). Amsterdam: IOS Press.Google Scholar
  83. Wu, H. K., & Shah, P. (2004). Exploring visuospatial thinking in chemistry learning. Science Education, 88(3), 465–492.CrossRefGoogle Scholar
  84. Wu, H. K., Krajcik, J. S., & Soloway, E. (2001). Promoting understanding of chemical representations: Students’ use of a visualization tool in the classroom. Journal of Research in Science Teaching, 38, 821–842.CrossRefGoogle Scholar
  85. Zhang, J. (1997). The nature of external representations in problem solving. Cognitive Science, 21, 179–217.CrossRefGoogle Scholar
  86. Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive Science: A Multidisciplinary Journal, 18, 87–122.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of Wisconsin, MadisonMadisonUSA

Personalised recommendations