The ‘Ins’ and ‘Outs’ of Learning: Internal Representations and External Visualizations

  • David N. Rapp
  • Christopher A. Kurby
Part of the Models and Modeling in Science Education book series (MMSE, volume 3)


Science classrooms teach complex topics by exposing students to information through a variety of methodologies, including lectures, discussions, readings, lab experiences, and representational experiences. The goal of these activities is to help students build internal representations for course content – information stored in memory that students can retrieve to generate inferences, solve problems, and make decisions. But what are these internal representations like, and what does the nature of these representations suggest for the design of learning methodologies such as external representations? This chapter is an introduction to current and contemporary work on mental representations. In particular, we emphasize theoretical and empirical views that have focused on links between perception and action, and what those links imply for learning. In this way, basic research on the nature of memory can provide pragmatic suggestions with respect to the design, implementation, and assessment of what are commonly called ‘visualizations’ (i.e., external visual representations of processes) as tools for science learning.


Mental Representation Perceptual Experience Science Classroom Abstract Word External Representation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alibali, M. W. (2005). Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation, 5, 307–331.CrossRefGoogle Scholar
  2. Anderson, J. R., & Lebiere , C. (1998). The atomic components of thought. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  3. Baddeley, A. D. (1986). Working memory. Oxford, UK: Clarendon Books.Google Scholar
  4. Baddeley, A. D. (1992). Working memory. Science, 255, 556–559.CrossRefGoogle Scholar
  5. Baddeley, A. D., & Hitch , G. J. (1974). Working memory. In G. H. Bower (Ed.), Recent Advances in learning and motivation (Vol. 8, pp. 47–90). New York: Academic Press.Google Scholar
  6. Baddeley, A. D., & Lieberman , K. (1980). Spatial working memory. In R. S. Nickerson (Ed.), Attention and performance VIII (pp. 521–539). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  7. Baddeley, A. D., & Logic, R. H. (1999). Working memory: The multiple-component model. In A. Miyake & P. Shah (Eds.), Models of working memory (pp. 28–61). Cambridge, UK: Cambridge University Press.Google Scholar
  8. Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral & Brain Sciences, 22, 577–660.CrossRefGoogle Scholar
  9. Barsalou, L. W., & Hale, C. R. (1993). Components of conceptual representation: From feature lists to recursive frames. In I. Van Mechelen, J. Hampton, R. Michalski, & P. Theuns (Eds.), Categories and concepts: Theoretical views and inductive data analysis (pp. 97–144). San Diego, CA: Academic Press.Google Scholar
  10. Behrmann, M. (2005). The mind’s eye mapped onto the brain’s matter. In B.A. Spellman & D.T. Willingham (Eds.), Current Directions in Cognitive Science. Readings from the American Psychological Society (pp. 11–18). Upper Saddle River, NJ: Pearson Education.Google Scholar
  11. Brunyé, T. T., Taylor, H. A., Rapp, D. N., & Spiro, A. B. (2006). Learning procedures: The role of working memory in multimedia learning experiences. Applied Cognitive Psychology, 20, 917–940.CrossRefGoogle Scholar
  12. Chao, L.L., & Martin, A. (2000). Representation of manipulable man-made objects in the dorsal stream. Neuroimage, 12, 478–484.CrossRefGoogle Scholar
  13. Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152.Google Scholar
  14. Chi, M. T. H., Glaser, R., & Farr. M. (Eds.). (1988). The nature of expertise. Hillsdale, NJ: LEA.Google Scholar
  15. Clark, H. H. (1996). Using language. Cambridge: Cambridge University Press.Google Scholar
  16. Dunn, K., & Dunn, R. (1978). Teaching students through their individual learning styles. Reston, VA: National Council of Principals.Google Scholar
  17. Dweck, C. (1986). Motivational processes affecting learning. American Psychologist, 41, 1040–1047.CrossRefGoogle Scholar
  18. Eley, M. G. (1991.) Selective encoding in the interpretation of topographic maps. Applied Cognitive Psychology, 5, 403–422.CrossRefGoogle Scholar
  19. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). Psychological Review, 100, 363–406.CrossRefGoogle Scholar
  20. Ferretti, T. R., McRae, K., & Hatherell, A. (2001). Integrating verbs, situation Schemas, and thematic role concepts. Journal of Memory and Language, 44, 516–547.CrossRefGoogle Scholar
  21. Fincher-Kiefer, R. (2001). Perceptual components of situation models. Memory & Cognition, 29, 336–343.Google Scholar
  22. Franco, C., & Colinvaux, D. (2000). Grasping mental models. In J. K. Gilbert & C. J. Boulter (Eds.), Developing models in science education (pp. 93–118). Boston, MA: Kluwer Academic Publishers.Google Scholar
  23. Gerlach, C., Law, I., & Paulson, O. B. (2002). When action turns into words: Activation of motor-based knowledge during categorization of manipulable objects. Journal of Cognitive Neuroscience, 14, 1230–1239.CrossRefGoogle Scholar
  24. Glenberg, A. M. (1997). What memory is for. Behavioral and Brain Sciences, 20, 1–55.CrossRefGoogle Scholar
  25. Glenberg, A. M., Gutierrez, T., Levin, J. R., Japuntich, S., & Kaschak, M .P. (2004). Activity and imagined activity can enhance young children’s reading comprehension. Journal of Educational Psychology, 96, 424–436.CrossRefGoogle Scholar
  26. Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychological Bulletin & Review, 9, 558–565.Google Scholar
  27. Glenberg, A. M., & Robertson, D. A. (1999). Indexical understanding of instructions. Discourse Processes, 28, 1–26.CrossRefGoogle Scholar
  28. Goldstone, R. L., & Sakamoto , Y. (2003). The transfer of abstract principles governing complex adaptive systems. Cognitive Psychology, 46, 414–466.CrossRefGoogle Scholar
  29. Graesser, A. C., & Clark, L. F. (1985). Structures and procedures of implicit knowledge. Norwood, NJ: Ablex.Google Scholar
  30. Grafton, S. T., Arbib, M. A., Fadiga, L., & Rizzolatti, G. (1996). Localization of grasp representations in humans by positron emission tomography – 2. Observation compared with imagination. Experimental Brain Research, 112, 103–111.CrossRefGoogle Scholar
  31. Harnad, S. (1990). The symbol grounding problem. Physica D, 42, 335–346.Google Scholar
  32. Hegarty, M. (2004). Mechanical reasoning by mental simulation. Trends in Cognitive Sciences, 8, 280–285.CrossRefGoogle Scholar
  33. Heiser, J., & Tversky, B. (2006). Arrows in comprehending and producing mechanical Diagrams. Cognitive Science, 30, 581–592.Google Scholar
  34. Hesslow, G. (2002). Conscious thought as simulation of behaviour and perception. Trends in Cognitive Sciences, 6, 242–24.CrossRefGoogle Scholar
  35. Horton, W. S., & Rapp, D. N. (2003). Out of sight, out of mind: Occlusion and the accessibility of information in narrative comprehension. Psychonomic Bulletin & Review, 10, 104–109.Google Scholar
  36. Johnson-Laird, P. N. (1980). Cognitive Science, 4, 71–115.CrossRefGoogle Scholar
  37. Kahneman, D., & Tversky, A. (1982). The simulation heuristic. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp.201–208). New York: Cambridge University Press.Google Scholar
  38. Kaschak, M. P., Madden, C. J., Therriault, D. J., Yaxley, R. H., Aveyard, M., Blanchard, A. A., et al. (2005). Perception of motion affects language processing. Cognition, 94, B79–B89.CrossRefGoogle Scholar
  39. Kellenbach, M. L., Brett, M., & Patterson, K. (2001). Large, colorful, or noisy? Attribute- and modality-specific activations during retrieval of perceptual attribute knowledge. Cognitive, Affective, & Behavioral Neuroscience, 1, 207–221.CrossRefGoogle Scholar
  40. Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge, UK: Cambridge University Press.Google Scholar
  41. Kosslyn, S. M. (1994). Image and brain: The resolution of the imagery debate. Cambridge, MA: MIT Press.Google Scholar
  42. Lakoff, G., & Johnson, M. (1980) Metaphors we live by. Chicago: University of Chicago Press.Google Scholar
  43. Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.Google Scholar
  44. Loftus, E. F., Miller, D. G., & Burns, H. J. (1978). Semantic integration of verbal information into a visual memory. Journal of Experimental Psychology: Human Learning and Memory, 4, 19–31.CrossRefGoogle Scholar
  45. Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13, 585–589.CrossRefGoogle Scholar
  46. Markman, A. B. (1999). Knowledge Representation. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  47. Martin, A., & Chao, L. L. (2001). Semantic memory and the brain: Structure and processes. Current Opinion in Neurobiology, 11, 194–201.CrossRefGoogle Scholar
  48. Martin, A., Haxby, J. V., Lalonde, F. M., Wiggs, C. L., & Ungerleider, L. G. (1995) Discrete cortical regions associated with knowledge of color and knowledge of action. Science, 270, 102–105.CrossRefGoogle Scholar
  49. Martin, A., Wiggs, C. L., Ungerleider, L. G., & Haxby, J. V. (1996) Neural correlates of category-specific knowledge. Nature, 379, 649–652.CrossRefGoogle Scholar
  50. Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.Google Scholar
  51. Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media.Learning and Instruction, 13, 125–139.CrossRefGoogle Scholar
  52. Mayer, R. E., & Anderson, R. B. (1992). The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84, 444–452.CrossRefGoogle Scholar
  53. Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93, 187–198.CrossRefGoogle Scholar
  54. Mayer, R. E., & Moreno , R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38, 43–52.CrossRefGoogle Scholar
  55. Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86, 389–401.CrossRefGoogle Scholar
  56. Minsky, M. (1975). A framework for representing knowledge. In P. H. Winston (Ed.), The psychology of computer vision (pp. 211–277). New York: McGraw Hill.Google Scholar
  57. Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358–368.CrossRefGoogle Scholar
  58. Nelson, D. L., Reed, U. S., & Walling, J. R. (1976). Picture superiority effect. Journal of Experimental Psychology: Human Learning & Memory, 2, 523–528.Google Scholar
  59. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  60. Norman, D. A. (1983). Some observations on mental models. In D. Gentner & A. L. Stevens (Eds.), Mental models (pp. 7–14). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  61. Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart & Winston.Google Scholar
  62. Paivio, A. (1983). The empirical case for dual coding. In J. C. Yulle (Ed.), Imagery, memory and cognition (pp. 307–332). Hillsdale, NJ: Erlbaum.Google Scholar
  63. Paivio, A. (1986). Mental representations: A dual-coding approach. New York: Oxford University Press.Google Scholar
  64. Paivio, A. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology, 45, 255–87.Google Scholar
  65. Paivio, A., & Csapo, K. (1973). Picture superiority effect: Imagery or dual coding? Cognitive Psychology, 5, 176–206.CrossRefGoogle Scholar
  66. Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2003). Verifying properties from different modalities for concepts produces switching costs. Psychological Science, 14, 119–124.CrossRefGoogle Scholar
  67. Pick, H. L., Heinrichs, M. R., Montello, D. R., Smith, K., Sullivan, C. N., & Thompson, W. B. (1995). Topographic map reading. In P. A. Hancock, J. M., Flach, J. Caird, & K. J. Vicente (Eds.), local applications of the ecological approach to human-machine systems (pp. 255–284). Hillsdale, NJ: Erlbaum.Google Scholar
  68. Pylyshyn, Z. W. (1981). The imagery debate: Analogue media versus tacit knowledge. Psychological Review, 88, 16–45.Google Scholar
  69. Pylyshyn, Z. W. (2002). Mental imagery: In search of a theory. Behavioral & Brain Sciences, 25, 157–238.CrossRefGoogle Scholar
  70. Rapp, D. N. (2005). Mental models: Theoretical issues for visualizations in science education. In J. K. Gilbert (Ed.), Visualization in Science Education (pp. 43–60). The Netherlands: Springer.CrossRefGoogle Scholar
  71. Rapp, D. N. (2006). The value of attention aware systems in educational settings. Computers in Human Behavior, 22, 603–614.CrossRefGoogle Scholar
  72. Rapp, D. N., Culpepper, S. A., Kirkby, K., & Morin, P. (in press). Fostering students’ comprehension of topographic maps. Journal of Geoscience Education.Google Scholar
  73. Rapp, D. N., Taylor, H. A., & Crane, G. R. (2003). The impact of digital libraries on cognitive processes: Psychological issues of hypermedia. Computers in Human Behavior, 19, 609–628.CrossRefGoogle Scholar
  74. Rapp, D. N., & Uttal, D. H. (2006). Understanding and enhancing visualizations: Two models of collaboration between earth science and cognitive science. In C. Manduca & D. Mogk (Eds.), Earth and mind: How geologists think and learn about the Earth (pp. 121–127). Boulder, CO: Geological Society of America Press.CrossRefGoogle Scholar
  75. Reimann, P., & Chi, M. T. H. (1989). Expertise in complex problem solving. In K. J. Gilhooly (Ed.), Human and Machine Problem Solving (pp. 161–192). New York: Plenum Press.Google Scholar
  76. Searle, J. R. (1980). Minds, brains, and programs. Behavioral & Brain Sciences, 3, 417–424.CrossRefGoogle Scholar
  77. Sloutsky, V. M., Kaminski, J. A., & Heckler, A. F. (2005). The advantage of simple symbols for learning and transfer. Psychonomic Bulletin & Review, 12, 508–513.Google Scholar
  78. Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs: General and Applied, 74, 1–30.Google Scholar
  79. Stanfield, R. A., & Zwaan, R. A. (2001). The effect of implied orientation derived from verbal context on picture recognition. Psychological Science, 12, 153–156.CrossRefGoogle Scholar
  80. Svensson, H., & Ziemke, T. (2004). Making sense of embodiment: Simulation theories and the sharing of neural circuitry between sensorimotor and cognitive processes. In Proceedings of the 26th annual meeting of the Cognitive Science Society.Mawhah, NJ: Erlbaum.Google Scholar
  81. Taylor, H. A., Renshaw, C. E., & Choi, E. J. (2004). The effect of multiple formats on understanding complex visual displays. Journal of Geoscience Education, 52, 115–121.Google Scholar
  82. Taylor, H. A., Renshaw, C. E., & Jensen, M. D. (1997). Effects of computer-based role-playing on decision making skills. Journal of Educational Computing Research, 17, 147–164.CrossRefGoogle Scholar
  83. Tversky, B. (in press). Mental models. In A. E. Kazdin (Ed.), Encyclopedia of Psychology. Washington, DC: APA Press.Google Scholar
  84. Tversky, B., Zacks, J. M., Lee, P. U., & Heiser, J. (2000). Lines, blobs, crosses, and arrows. In M. Anderson, P. Cheng, & V. Haarslev (Eds.), Theory and application of diagrams (pp. 221–230). Edinburgh: Springer.CrossRefGoogle Scholar
  85. Valenzeno, L., Alibali, M. W., & Klatzky, R. (2003). Teachers’ gestures facilitate students’ learning: A lesson in symmetry. Contemporary Educational Psychology, 28,187–204.CrossRefGoogle Scholar
  86. Van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press.Google Scholar
  87. Wiemer-Hastings, K., & Xu, X. (2005). Content differences for abstract and concrete concepts. Cognitive Science, 29, 719–736.Google Scholar
  88. Woolfolk, A. (2004). Educational psychology (9th ed.). Boston, MA: Allyn and Bacon.Google Scholar
  89. Zwaan, R. A. (2004). The immersed experiencer: Toward an embodied theory of language comprehension. In B. H. Ross (Ed.), The psychology of leaning and motivation, (Vol. 4, pp. 35–62). New York: Academic Press.Google Scholar
  90. Zwaan, R. A., Madden, C. J., Yaxley, R. H., & Aveyard, M. E. (2004). Moving words: Dynamic mental representations in language comprehension. Cognitive Science, 28, 611–619.CrossRefGoogle Scholar
  91. Zwaan, R. A., Stanfield, R. A., & Yaxley, R. H. (2002). Language comprehenders routinely represent the shape of objects. Psychological Science, 13, 168–171.CrossRefGoogle Scholar
  92. Zwaan, R. A., & Yaxley, R. H. (2004). Spatial iconicity affects semantic-relatedness judgments. Psychonomic Bulletin & Review, 10, 954–958.Google Scholar
  93. Zwaan R. A., & Yaxley, R. H. (2005). Lateralization of object-shape information in semantic processing. Cognition, 94, B35–B43.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  • David N. Rapp
    • 1
  • Christopher A. Kurby
  1. 1.Northwestern UniversityUSA

Personalised recommendations