Digital Pictures, Videos, and Beyond: Knowledge Acquisition with Realistic Images

  • Stephan Schwan


While the distinction between pictures and texts is well established on theoretical grounds and has attracted much research, the differences and commonalities between realistic depictions and its real-world counterparts have received much less attention. This chapter aims to contribute to closing this gap by systematically comparing life-like images to real-world events in terms of commonalities and differences in visual appeal as well as in perception and mental processing. Based on the notion of a “dual character” of digital images, both closely resembling reality but simultaneously being systematically different, several issues regarding processes of knowledge acquisition will be discussed, including: Are viewers aware of differences between real-world information and mediated information—and do they take them into account? Do realistic images require specific competencies for comprehension? Should the realism of visual representations be maximized for learning? How do viewers deal with the informational complexity and ambiguity of realistic images?


Realistic pictures Authenticity Event cognition Video 


  1. Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review Psychology, 63, 1–29.CrossRefGoogle Scholar
  2. Dutz, S., & Schwan, S. (2014). New media use in art exhibitions: Enriching or annoying? Paper presented at the 27th Annual Visitor Studies Association Conference, Albuquerque, USA.Google Scholar
  3. Dwyer, F. M. (1968). Effect of varying amount of realistic detail in visual illustrations designed to complement programmed instruction. Perceptual and Motor Skills, 27, 351–354.CrossRefGoogle Scholar
  4. Eitel, A., & Scheiter, K. (2015). Picture or text first? Explaining sequence effects when learning with pictures and text. Educational Psychology Review, 27, 153–180.CrossRefGoogle Scholar
  5. Fischer, S., Lowe, R. K., & Schwan, S. (2008). Effects of presentation speed of a dynamic visualization on the understanding of a mechanical system. Applied Cognitive Psychology, 22, 1126–1141.CrossRefGoogle Scholar
  6. Fischer, S., & Schwan, S. (2010). Comprehending animations. Effects of spatial cueing versus temporal scaling. Learning and Instruction, 20, 465–475.CrossRefGoogle Scholar
  7. Garsoffky, B., Huff, M., & Schwan, S. (2007). Changing viewpoints during dynamic events. Perception, 36, 366–374.CrossRefGoogle Scholar
  8. Garsoffky, B., Schwan, S., & Hesse, F. W. (2002). Viewpoint dependency in the recognition of dynamic scenes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28, 1035–1050.Google Scholar
  9. Garsoffky, B., Schwan, S., & Huff, M. (2009). Canonical views of dynamic scenes. Journal of Experimental Psychology: Human Perception and Performance, 35, 17–27.Google Scholar
  10. Gerjets, P. (2017). Learning and Problem Solving with Hypermedia in the 21st Century: From Hypertext to Multiple Web Sources and Multimodal Adaptivity.Google Scholar
  11. Gibson, J. J. (1979). The ecological approach to visual perception. Boston, MA: Houghton Mifflin.Google Scholar
  12. Glaser, M., Garsoffky, B., & Schwan, S. (2009). Narrative-based learning: possible benefits and problems. The European Journal of Communication Research, 34, 429–447.Google Scholar
  13. Glaser, M., Garsoffky, B., & Schwan, S. (2012). What do we learn from docutainment? Processing hybrid television documentaries. Learning and Instruction, 22, 37–46.CrossRefGoogle Scholar
  14. Glaser, M., & Schwan, S. (2015). Explaining pictures: How verbal cues influence processing of pictorial learning material. Journal of Educational Psychology, 107, 1006–1018.CrossRefGoogle Scholar
  15. Hampp, C., & Schwan, S. (2014). Perception and evaluation of authentic objects: Findings from a visitor study. Museum Management and Curatorship, 29, 349–367.CrossRefGoogle Scholar
  16. Hampp, C., & Schwan, S. (2015). The role of authentic objects in museums of the history of science and technology: Findings from a visitor study. International Journal of Science Education Part B: Communication and Public Engagement, 5, 161–181.CrossRefGoogle Scholar
  17. Hard, B. M., Lozano, S. C., & Tversky, B. (2006). Hierarchical encoding of behavior: Translating perception into action. Journal of Experimental Psychology: General, 135, 588–608.CrossRefGoogle Scholar
  18. Hegarty, M., Kriz, S., & Cate, C. (2003). The roles of mental animations and external animations in understanding mechanical systems. Cognition and Instruction, 21, 325–360.CrossRefGoogle Scholar
  19. Hobbs, R. (2006). Non-optimal uses of video in the classroom. Learning, Media and Technology, 31, 35–50.CrossRefGoogle Scholar
  20. Hochberg, J., & Brooks, V. (1962). Pictorial recognition as an unlearned ability: A study of one child’s performance. The American Journal of Psychology, 75, 624–628.CrossRefGoogle Scholar
  21. Hudson, W. (1967). The study of the problem of pictorial perception among unacculturated groups. International Journal of Psychology, 2, 89–107.CrossRefGoogle Scholar
  22. Huff, M., Bauhoff, V., & Schwan, S. (2012). Effects of split attention revisited: A new display technology for troubleshooting tasks. Computers in Human Behavior, 28, 1254–1261.CrossRefGoogle Scholar
  23. Huff, M., Jahn, G., & Schwan, S. (2009). Tracking multiple objects across abrupt viewpoint changes. Visual Cognition, 17, 297–306.CrossRefGoogle Scholar
  24. Huff, M., & Schwan, S. (2008). Verbalizing events: Overshadowing or facilitation? Memory & Cognition, 36, 392–402.CrossRefGoogle Scholar
  25. Huff, M., & Schwan, S. (2012a). Do not cross the line: Heuristic spatial updating in dynamic scenes. Psychonomic Bulletin & Review, 19, 1065–1072.CrossRefGoogle Scholar
  26. Huff, M., & Schwan, S. (2012b). The verbal facilitation effect in learning to tie nautical knots. Learning and Instruction, 22, 376–385.CrossRefGoogle Scholar
  27. Ildirar, S., & Schwan, S. (2015). First-time viewers’ comprehension of films: Bridging shot transitions. British Journal of Psychology, 106, 133–151.CrossRefGoogle Scholar
  28. Institut für Demoskopie Allensbach (2013). Digitale Medien im Unterricht (Digital media in the classroom). Retrieved from
  29. Lee, V. R. (2010). Adaptations and continuities in the use and design of visual representations in US Middle School science textbooks. International Journal of Science Education, 32, 1099–1126.CrossRefGoogle Scholar
  30. Lowe, R., & Schnotz, W. (2014). Animation principles in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 513–546). Cambridge, MA: Cambridge University Press.Google Scholar
  31. Mager, U. I., Schwonke, R., Aleven, V., Popescu, O., & Renkl, A. (2014). Triggering situational interest by decorative illustrations both fosters and hinders learning in computer-based environments. Learning and Instruction, 29, 141–152.CrossRefGoogle Scholar
  32. Mayer, R. E. (2001). Multimedia learning. Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
  33. Merkt, M., & Schwan, S. (2014). Training the use of interactive videos: Effects on mastering different tasks. Instructional Science, 42, 421–441.CrossRefGoogle Scholar
  34. Merkt, M., & Sochatzy, F. (2015). Becoming aware of cinematic techniques in propaganda: Instructional support by cueing and training. Learning and Instruction, 39, 55–71.CrossRefGoogle Scholar
  35. Merkt, M., Weigand, S., Heier, A., & Schwan, S. (2011). Learning with videos vs. learning with print: The role of interactive features. Learning & Instruction, 21, 687–704.Google Scholar
  36. Meyerhoff, H. S., Huff, M., Papenmeier, F., Jahn, G., & Schwan, S. (2011). Continuous visual cues trigger automatic spatial target updating in dynamic scenes. Cognition, 121, 73–82.CrossRefGoogle Scholar
  37. Paivio, A. (1986). Mental representations: A dual-coding approach. New York: Oxford University Press.Google Scholar
  38. Papenmeier, F., & Schwan, S. (2016). If you watch it move, you’ll recognize it in 3D: Transfer of depth cues between encoding and retrieval. Acta Psychologica, 164, 90–95.CrossRefGoogle Scholar
  39. Peirce, C. S. (1940). The philosophy or Peirce: Selected writings. New York: Harcourt.Google Scholar
  40. Rey, G. D. (2012). A review of research and a meta-analysis of the seductive detail effect. Educational Research Review, 7, 216–237.CrossRefGoogle Scholar
  41. Salomon, G., & Cohen, A. A. (1977). Television formats, mastery of mental skills, and the acquisition of knowledge. Journal of Educational Psychology, 69, 612–619.CrossRefGoogle Scholar
  42. Sareen, P., Ehinger, K. A., & Wolfe, J. M. (2015). Through the looking-glass: Objects in the mirror are less real. Psychonomic Bulletin & Review, 22, 980–986.CrossRefGoogle Scholar
  43. Scheiter, K. (2014). The learner control principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 487–512). Cambridge, MA: Cambridge University Press.Google Scholar
  44. Scheiter, K., Gerjets, P., Huk, T., Imhof, B., & Kammerer, Y. (2009). The effects of realism in learning with dynamic visualizations. Learning and Instruction, 19, 481–494.CrossRefGoogle Scholar
  45. Scheiter, K., Schüler, A., Eitel, A. (this volume). Learning from Multimedia: Cognitive Processes and Instructional Support.Google Scholar
  46. Schnotz, W. (2002). Commentary: Towards an integrated view of learning from text and visual displays. Educational Psychology Review, 14, 101–120.CrossRefGoogle Scholar
  47. Schooler, J. W., & Engstler-Schooler, T. Y. (1990). Verbal overshadowing of visual memories: Some things are better left unsaid. Cognitive Psychology, 22, 36–71.CrossRefGoogle Scholar
  48. Schwan, S., Bauer, D., Kampschulte, J., & Hampp, C. (in press). Presentation equals representation? Photographs of objects received less attention and are less well remembered than real objects. Journal of Media Psychology. Google Scholar
  49. Schwan, S., & Garsoffky, B. (2004). The cognitive representation of filmic event summaries. Applied Cognitive Psychology, 18, 37–55.CrossRefGoogle Scholar
  50. Schwan, S., Garsoffky, B., & Hesse, F. W. (1998). The relationship between formal filmic means and the segmentation behavior of film viewers. Journal of Broadcasting & Electronic Media, 42, 237–249.CrossRefGoogle Scholar
  51. Schwan, S., Garsoffky, B., & Hesse, F. W. (2000). Do film cuts facilitate the perceptual and cognitive organization of activity sequences? Memory & Cognition, 28, 214–223.CrossRefGoogle Scholar
  52. Schwan, S., Grajal, A., & Lewalter, D. (2014). Understanding and engagement in places of science experience: Science museums, science centers, zoos, and aquariums. Educational Psychologist, 49, 70–85.CrossRefGoogle Scholar
  53. Schwan, S., & Ildirar, S. (2010). Watching film for the first time: How adult viewers interpret perceptual discontinuities in film. Psychological Science, 21, 970–976.CrossRefGoogle Scholar
  54. Schwan, S., & Papenmeier, F. (in press). Learning from animations: From 2D to 3D? In R. Plötzner & R. Lowe (Eds.), Learning from dynamic visualizations: Innovations in research and application. New York: Springer.Google Scholar
  55. Schwan, S., & Riempp, R. (2004). The cognitive benefits of interactive videos: Learning to tie nautical knots. Learning & Instruction, 14, 293–305.CrossRefGoogle Scholar
  56. Smith, T. J. (2012). The attentional theory of cinematic continuity. Projections, 6, 1–27.CrossRefGoogle Scholar
  57. Soemer, A., & Schwan, S. (2016). Task-appropriate visualizations: Can the very same visualization format either promote or hinder learning depending on the task requirements? Journal of Educational Psychology.Google Scholar
  58. Stafford, B. M. (1994). Artful science. Enlightment entertainment and the eclipse of visual education. Cambridge, MA: MIT Press.Google Scholar
  59. Tibus, M., Heier, A., & Schwan, S. (2013). Do films make you learn? Inference processes in expository film comprehension. Journal of Educational Psychology, 105, 329–340.CrossRefGoogle Scholar
  60. Töpper, J., Glaser, M., & Schwan, S. (2014). Extending social cue based principles of multimedia learning beyond their immediate effects. Learning and Instruction, 29, 10–20.CrossRefGoogle Scholar
  61. Tversky, B., Bauer Morrison, J., & Betrancourt, M. (2002). Animation: Can it facilitate? International Journal of Human-Computer Studies, 57, 247–262.CrossRefGoogle Scholar
  62. van Gogh, T. (2014). The signaling (or cueing) principle. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 263–278). Cambridge, MA: Cambridge University Press.Google Scholar
  63. Yarbus, A. L. (1967). Eye movements and vision. New York: Plenum Press.CrossRefGoogle Scholar
  64. Yasar, O., & Seremet, M. (2007). A comparative analysis regarding pictures included in secondary school geography textbooks taught in Turkey. International Research in Geographical and Environmental Education, 16, 157–187.CrossRefGoogle Scholar
  65. Yee, E., & Sedivy, J. C. (2006). Movements to pictures reveal transient semantic activation during spoken word recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 1–14.Google Scholar
  66. Zacks, J. M., & Tversky, B. (2001). Event structure in perception and conception. Psychological Bulletin, 127, 3–21.CrossRefGoogle Scholar
  67. Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R. (2007). Event perception: S mind-brain perspective. Psychological Bulletin, 133, 273–293.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Leibniz-Institut für WissensmedienTübingenGermany

Personalised recommendations