How Visual Displays Affect Cognitive Processing

Abstract

We regularly consult and construct visual displays that are intended to communicate important information. The power of these displays and the instructional messages we attempt to comprehend when using them emerge from the information included in the display and by their spatial arrangement. In this article, we identify common types of visual displays and the kinds of inferences that each type of display is designed to promote. In particular, we outline different types of semantic and pictorial displays. Then, we describe four main ways in which visual displays can affect cognitive processing including selection, organization, integration, and processing efficiency and how semantic and pictorial displays support these types of processing. We conclude with seven recommendations for designing visual displays and possible directions for future research.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Ainsworth, S. (2006). DeFT: a conceptual framework for considering learning with multiple representations. Learning and Instruction, 16, 183–198.

    Article  Google Scholar 

  2. Ainsworth, S. E., & Loizou, A. T. (2003). The effects of self-explaining when learning with text or diagrams. Cognitive Science, 27(4), 669–681.

    Article  Google Scholar 

  3. Ayres, P., & Sweller, J. (2005). The split-attention principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 135–146). New York, NY: Cambridge University Press.

    Google Scholar 

  4. Baddeley, A. D. (2007). Working memory, thought and action. Oxford: Oxford University Press.

    Google Scholar 

  5. Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: knowing about knowing (pp. 185–205). Cambridge, MA: MIT Press.

    Google Scholar 

  6. Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn. Washington, DC: National Academy Press.

    Google Scholar 

  7. Butcher, K. R. (2006). Learning from text with diagrams: promoting mental model development and inference generation. Journal of Educational Psychology, 98(1), 182–197.

    Article  Google Scholar 

  8. Carbonneau, K. J., Marley, S. C., & Selig, J. P. (2013). A meta-analysis of the efficacy of teaching mathematics with concrete manipulatives. Journal of Educational Psychology, 105(2), 380–400. doi:10.1037/a0031084.

    Article  Google Scholar 

  9. Carney, R. N., & Levin, J. R. (2002). Pictorial illustrations still improve students’ learning from text. Educational Psychology Review, 14, 5–26.

    Article  Google Scholar 

  10. Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. doi:10.1080/00461520.2014.965823.

    Article  Google Scholar 

  11. Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3, 149–210.

    Article  Google Scholar 

  12. Cromley, J. G., Snyder-Hogan, L. E., & Luciw-Dubas, U. A. (2010). Cognitive activities in complex science text and diagrams. Contemporary Educational Psychology, 35, 59–74. doi:10.1016/j.cedpsych.2009.10.002.

    Article  Google Scholar 

  13. de Koning, B. B., Tabbers, H., Rikers, R. M. J. P., & Paas, F. (2009). Towards a framework for attention cueing in instructional animations: guidelines for research and design. Educational Psychology Review, 21(2), 113–140.

    Article  Google Scholar 

  14. de Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2010). Attention guidance in learning from a complex animation: seeing is understanding? Learning and Instruction, 20(2), 111–122.

    Article  Google Scholar 

  15. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning and comprehension by using effective learning techniques: promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14, 4–58.

    Article  Google Scholar 

  16. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: verbal reports as data. Cambridge, MA: MIT Press.

    Google Scholar 

  17. Fiorella, L., & Mayer, R. E. (2015). Learning as a generative activity: eight learning strategies that promote understanding. New York: Cambridge University Press.

    Google Scholar 

  18. Fyfe, E. R., McNeil, N. M., Son, J. Y., & Goldstone, R. L. (2014). Concreteness fading in mathematics and science instruction: a systematic review. Educational Psychology Review, 26, 9–25. doi: 10.1007/s10648-014-9249-3

  19. Ginns, P. (2006). Integrating information: a meta-analysis of the spatial contiguity and temporal contiguity effects. Learning and Instruction, 16(6), 511–525.

    Article  Google Scholar 

  20. Graesser, A. C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101, 371–395.

    Article  Google Scholar 

  21. Guitèrrez, K. D., & Rogoff, B. (2003). Cultural ways of learning: individual traits or repertoires of practice. Educational Researcher, 5, 19–25.

    Article  Google Scholar 

  22. Gutierrez, A. P., Schraw, G., & Stefik, A. (2015). Design principles for visual displays: past, present, and future. In M. T. McCrudden, G. Schraw, & C. Buckendahl (Eds.), Use of visual displays in research and testing: Coding, interpreting, and reporting data (pp. 17–46). Charlotte, NC: Information Age Publishing.

    Google Scholar 

  23. Hegarty, M., Canham, M., & Fabrikant, S. (2010). Thinking about the weather: how display salience and knowledge affect performance in a graphic inference task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 37–53. doi:10.1037/a0017683.

    Google Scholar 

  24. Hinze, S. R., Rapp, D. N., Williamson, V. M., Shultz, M. J., Deslongchamps, G., & Williamson, K. C. (2013). Beyond the ball-and-stick: students’ processing of novel STEM visualizations. Learning and Instruction, 26, 12–21.

    Article  Google Scholar 

  25. Höffler, T. N. (2010). Spatial ability: its influence on learning with visualizations—a meta-analytic review. Educational Psychology Review, 22, 245–269. doi:10.1007/s10648-010-9126-7.

    Article  Google Scholar 

  26. Höffler, T. N., & Leutner, D. (2007). Instructional animation versus static pictures: a meta-analysis. Learning and Instruction, 17, 722–738.

    Article  Google Scholar 

  27. Hollan, J. D., Hutchins, E. L., & Weitzman, L. (1984). Steamer: an interactive inspectable simulation-based training system. AI Magazine, 5(2), 15–27.

  28. Jairam, D., & Kiewra, K. A. (2010). Helping students soar to success on computers: an investigation of the SOAR study method for computer-based learning. Journal of Educational Psychology, 102(3), 601–614.

    Article  Google Scholar 

  29. Jonassen, D. H., Beissner, K., & Yacci, M. A. (1993). Structural knowledge. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  30. Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: individual differences in working memory. Psychological Review, 99, 122–149.

    Article  Google Scholar 

  31. Kauffman, D. F., & Kiewra, K. A. (2010). What makes a matrix so effective? An empirical test of the relative benefits of signaling, extraction, and localization. Instructional Science, 38, 679–705.

    Article  Google Scholar 

  32. Kiewra, K. A. (2012). IDEA Paper 51: using graphic organizers to improve teaching and learning. Manhattan, KS: The IDEA Center.

    Google Scholar 

  33. Kiewra, K. A., Kauffman, D. F., Robinson, D., DuBois, N., & Staley, R. K. (1999). Supplementing floundering text with adjunct displays. Journal of Instructional Science, 27, 373–401.

    Google Scholar 

  34. Kosslyn, S. M. (1985). Graphics and human information processing: a review of five books. Journal of the American Statistical Association, 80, 499–513.

    Article  Google Scholar 

  35. Kozhevnikov, M., Evans, C., & Kosslyn, S. M. (2014). Cognitive styles as environmentally sensitive individual differences in cognition: a modern synthesis and applications in education, business, and management. Psychological Science in the Public Interest, 15(1), 3–33.

    Article  Google Scholar 

  36. Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learning from animations. International Journal of Human-Computer Studies, 65, 911–930. doi:10.1016/j.ijhcs.2007.06.005.

    Article  Google Scholar 

  37. Lane, D. M. (2015). Guidelines for making graphs easy to perceive, easy to understand, and information rich. In M. T. McCrudden, G. Schraw, & C. Buckendahl (Eds.), Use of visual displays in research and testing: Coding, interpreting, and reporting data (pp. 47–81). Charlotte, NC: Information Age Publishing.

    Google Scholar 

  38. Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–100.

    Article  Google Scholar 

  39. Lowe, R. K. (2003). Animation and learning: selective processing of information in dynamic graphics. Learning and Instruction, 13(2), 157–176.

    Article  Google Scholar 

  40. Lowe, R. K., & Boucheix, J. M. (2008). Learning from animated diagrams: how are mental models built. In G. Stapleton, J. Howse, & J. Lee (Eds.), Theory and applications of diagrams (pp. 266–281). Berlin: Springer.

    Google Scholar 

  41. Marley, S. C., & Carbonneau, K. J. (2014). Future directions for theory and research with instructional manipulatives: commentary on the special issue papers. Educational Psychology Review, 26(1), 91–100. doi:10.1007/s10648-014-9259-1.

    Article  Google Scholar 

  42. Marr, D. (1982). Vision. San Francisco, CA: Freeman.

    Google Scholar 

  43. Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns. Computers & Education, 60, 95–109.

    Article  Google Scholar 

  44. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.

    Google Scholar 

  45. Mayer, R. E. (2013). Fostering learning with visual displays. In G. Schraw, M. T. McCrudden, & D. Robinson (Eds.), Learning through visual displays (pp. 47–73). Charlotte, NC: Information Age Publishing.

    Google Scholar 

  46. Mayer, R. E., & Johnson, C. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100, 380–386.

    Article  Google Scholar 

  47. McCrudden, M. T., Schraw, G., & Lehman, S. (2009). The use of adjunct displays to facilitate comprehension of causal relationships in expository text. Instructional Science, 37(1), 65–86. doi:10.1007/s11251-007-9036-3.

    Article  Google Scholar 

  48. McCrudden, M. T., Magliano, J., & Schraw, G. (2011). The effects of diagrams on online reading processes and memory. Discourse Processes, 48(2), 69–92. doi:10.1080/01638531003694561.

    Article  Google Scholar 

  49. McCrudden, M. T., Hushman, C., & Marley, S. (2014). Exploring the boundary conditions of the redundancy principle. The Journal of Experimental Education, 82(4), 537–554. doi:10.1080/00220973.2013.813368.

    Article  Google Scholar 

  50. McElhaney, K., Chang, H., Chiu, J. L., & Linn, M. C. (2015). Evidence for effective uses of dynamic visualisations in science curriculum materials. Studies in Science Education. doi:10.1080/03057267.2014.984506.

    Google Scholar 

  51. Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: the role of modality and contiguity. Journal of Educational Psychology, 91, 358–368.

    Article  Google Scholar 

  52. Nesbit, J. C., & Adelsope, O. O. (2006). Learning with concept and knowledge maps: a metaanalysis. Review of Educational Research, 76, 413–448. doi:10.3102/00346543076003413.

    Article  Google Scholar 

  53. Newcombe, N., & Shipley, T. F. (2012). Thinking about spatial thinking: new typology, new assessments. In J. S. Gero (Ed.), Studying visual and spatial reasoning for design creativity. Berlin: Springer.

    Google Scholar 

  54. Novick, L. R., & Hurley, S. M. (2001). To matrix, network, or hierarchy, that is the question. Cognitive Psychology, 42, 158–216.

    Article  Google Scholar 

  55. O’Brien, E. J., Rizzella, M. L., Albrecht, J. E., & Halleran, J. G. (1998). Updating a situation model: a memory-based text processing view. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 1200–1210.

    Google Scholar 

  56. O’Brien, E. J., Cook, A. E., & Guéraud, S. (2010). Accessibility of outdated information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 979–991.

    Google Scholar 

  57. O’Donnell, A., Dansereau, D., & Hall, R. H. (2002). Knowledge maps as scaffolds for cognitive processing. Educational Psychology Review, 14, 71–86.

    Article  Google Scholar 

  58. Ozcelik, E., Arslan-Ari, I., & Cagiltay, K. (2010). Why does signaling enhance multimedia learning? Evidence from eye movements. Computers in Human Behavior, 26, 110e117. doi:10.1016/j.chb.2009.09.001.

    Article  Google Scholar 

  59. Park, B., Flowerday, T., & Brünken, R. (2015). Cognitive and affective effects of seductive details in multimedia learning. Computers in Human Behavior, 44, 267–278.

    Article  Google Scholar 

  60. Pastor, D. A., & Finney, S. J.  (2013).  Using visual displays to enhance understanding of quantitative research.  In G. Schraw, M. T. McCrudden, & D. Robinson (Eds.), Learning through visual displays (pp. 387–415). Charlotte, NC: Information Age Publishing.

  61. Plass, J. L., Homer, B. D., & Hayward, E. (2009). Design factors for educationally effective animations and simulations. Journal of Computing in Higher Education, 21(1), 31–61.

    Article  Google Scholar 

  62. Ponce, H. R., & Mayer, R. E. (2014). Qualitatively different cognitive processing during online reading primed by different study activities. Computers in Human Behavior, 30, 121–130.

    Article  Google Scholar 

  63. Rapp, D. N., & Taylor, H. A. (2004). Interactive dimensions in the construction of mental representations for text. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 988–1001.

    Google Scholar 

  64. Robinson, D. H. (1998). Graphic organizers as aids to text learning. Reading Research and Instruction, 37, 85–105.

    Article  Google Scholar 

  65. Robinson, D. H., & Kiewra, K. A. (1995). Visual argument: graphic organizers are superior to outlines in improving learning from text. Journal of Educational Psychology, 87, 455–467.

    Article  Google Scholar 

  66. Robinson, D. H., & Schraw, G. (1994). Computational efficiency through visual argument: do graphic organizers communicate relations in text too effectively? Contemporary Educational Psychology, 19, 399–415.

    Article  Google Scholar 

  67. Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. (1976). Basic objects in natural categories. Cognitive Psychology, 8, 382–439.

    Article  Google Scholar 

  68. Sanchez, C. A., & Wiley, J. (2006). An examination of the seductive details effect in terms of working memory capacity. Memory & Cognition, 34, 344–355.

    Article  Google Scholar 

  69. Sauter, M., Uttal, D. H., Rapp, D. N., Downing, M., & Jona, K. (2013). Getting real: the authenticity of remote labs and simulations for science learning. Distance Education, 34(1), 37–47.

    Article  Google Scholar 

  70. Schnotz, W. (2002). Commentary—towards an integrated view of learning from text and visual displays. Educational Psychology Review, 14(1), 101–120.

    Article  Google Scholar 

  71. Shah, P., & Hoeffner, J. (2002). Review of graph comprehension research: implications for instruction. Educational Psychology Review, 14, 47–69.

    Article  Google Scholar 

  72. Stroet, K., Opdenakker, M., & Minnaert, A. (2015). What motivates early adolescents for school? A longitudinal analysis of associations between observed teaching and motivation. Contemporary Educational Psychology, 42, 129–140.

    Article  Google Scholar 

  73. Sweller, J., van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296.

    Article  Google Scholar 

  74. Tversky, B., Zacks, J., Lee, P., & Heiser, J. (2000). Lines, blobs, crosses, and arrows: diagrammatic communication with schematic figures. In M. Anderson, P. Cheng, & V. Haarslev (Eds.), Theory and application of diagrams (pp. 221–230). Berlin: Springer.

    Google Scholar 

  75. Van Meter, P. N., Waters, J. R., & Cameron, C. (2015, April). The effects of self-explanation prompts and diagram comprehension ability on task performance in multimedia learning. Poster presented at the annual conference of the American Educational Research Association, Chicago, Illinois.

  76. Vekiri, I. (2002). What is the value of graphical displays in learning? Educational Psychology Review, 14, 261–312.

    Article  Google Scholar 

  77. Winn, W. (1991). Learning from maps and diagrams. Educational Psychology Review, 3, 211–247.

    Article  Google Scholar 

  78. Wylie, R., & Chi, M. T. H. (2014). The self-explanation principle in multimedia learning. In R. E. Mayer (Ed.), Cambridge handbook of multimedia learning (2nd ed., pp. 413–432). New York: Cambridge University Press.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Matthew T. McCrudden.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

McCrudden, M.T., Rapp, D.N. How Visual Displays Affect Cognitive Processing. Educ Psychol Rev 29, 623–639 (2017). https://doi.org/10.1007/s10648-015-9342-2

Download citation

Keywords

  • Visual display
  • Graphic organizer
  • Visual representation
  • Cognitive processing
  • Multimedia learning