Skip to main content
SpringerLink
Log in

Mental Models and Visualization

  • Chapter
  • First Online:
Visualization Psychology

  • 321 Accesses

Abstract

Mental models are internal representations of external phenomena. During their interaction with visualizations, the users construct mental models to represent these visualizations internally, to visually reason on them and solve problems with them. This chapter synthesizes existing theories on mental models and visualization to discuss their role and relevance for the design of visualization systems. From a mental models perspective, we discuss two challenges of visualization design: (a) supporting the initial construction of mental models and (b) supporting the integration of information from multiple views by synchronous or sequential coherence techniques. We argue that the theory of mental models allows to understand visualization research and practice in a more unified fashion as an advanced model-building endeavor, operating on human computer ensembles engaged in “distributed cognition.”

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Prominent concepts include cognitive schemata, cognitive scripts, cognitive images, cognitive maps, prototypes, or cognitive frames [23, 27].

  2. 2.

    As we will argue later on, visualization theory would also benefit from integrating narrative sequences and stories into the second category of behavioral models (Sect. 3.3.2.2), so that the mental model concept can cover representations of static structures and time-oriented sequences in an equal fashion—similar to the distinction of cognitive schemata and cognitive scripts [44].

  3. 3.

    Due to the prevalence of user-oriented design, the quality of visualizations as external representations is tied back to the quality of the internal representations that they generate (e.g., the utility, efficiency, correctness, esthetic appeal, etc.). Arguably, it is this circle, which makes it relevant for visualization designers to know about cognitive principles (i.e., from Gestalt and color perception to more complex model construction and reasoning processes) to design for the effective amplification of perceptual and cognitive processes.

  4. 4.

    A large part of the basic research on mental models has been done in the context of text comprehension and with regard to subject matters, where a spatial layout of environmental data is given. In such a context, understanding an external representation (e.g., the description of a built environment) requires the construction of a mental model, for which a visual-spatial isomorphy between relevant aspects of internal and external representations should be achieved—and is relatively easy to verify. Despite the fact that (the rules of construction for) external representations preserving a spatial layout are widely known and universally established (e.g., by naturalistic images, miniature models, or instances of “scientific visualization”), it is known that the initial build-up of an internal model (i.e., internalization) is cognitively and energetically demanding. This holds even more true for the internalization of pictures which spatialize abstract or conceptual data due to the rules of a diagrammatic syntax (often summarized as techniques of “information visualization”). Especially, if the users are not familiar with the rules of construction, they face higher barriers as they have to build up both: a (structurally and behaviorally) isomorphic model from the external representation and a basic understanding of the principles or rules of image construction (visualization literacy [6]).

References

  1. D. Archambault and H. C. Purchase. Mental map preservation helps user orientation in dynamic graphs. In International Symposium on Graph Drawing, pages 475–486. Springer, 2012.

    Google Scholar 

  2. R. Arias-Hernandez, T. M. Green, and B. Fisher. From cognitive amplifiers to cognitive prostheses: Understandings of the material basis of cognition in visual analytics. Interdisciplinary science reviews, 37(1):4–18, 2012.

    Article  Google Scholar 

  3. D. P. Ausubel. The use of advance organizers in the learning and retention of meaningful verbal material. Journal of educational psychology, 51(5):267–272, 1960.

    Article  Google Scholar 

  4. M. Q. W. Baldonado, A. Woodruff, and A. Kuchinsky. Guidelines for using multiple views in information visualization. In AVI ’00: Proceedings of the working conference on Advanced visual interfaces, pages 110–119. ACM Press, 2000.

    Google Scholar 

  5. M.-J. Bludau, V. Brügemann, A. Busch, and M. Dörk. Reading traces: Scalable exploration in elastic visualizations of cultural heritage data. Computer Graphics Forum, 39(3):77–87, 2020.

    Article  Google Scholar 

  6. K. Börner, A. Bueckle, and M. Ginda. Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments. Proceedings of the National Academy of Sciences, 116(6):1857–1864, 2019. Publisher: National Acad Sciences.

    Google Scholar 

  7. S. K. Card, J. D. Mackinlay, and B. Shneiderman. Readings in information visualization: using vision to think. Morgan Kaufmann Publishers Inc., 1999.

    Google Scholar 

  8. J. M. Clark and A. Paivio. Dual coding theory and education. Educational Psychology Review, 3(3):149–210, Sept. 1991.

    Article  Google Scholar 

  9. J. W. Coffey, R. Hoffman, and A. Cañas. Concept map-based knowledge modeling: perspectives from information and knowledge visualization. Information Visualization, 5(3):192–201, 2006.

    Article  Google Scholar 

  10. C. Collins and S. Carpendale. Vislink: Revealing relationships amongst visualizations. IEEE Transactions on Visualization and Computer Graphics, 13(6):1192–1199, 2007.

    Article  Google Scholar 

  11. K. J. W. Craik. The nature of explanation. Cambridge University Press, 1943.

    Google Scholar 

  12. A. W. Crapo, L. B. Waisel, W. A. Wallace, and T. R. Willemain. Visualization and the process of modeling: a cognitive-theoretic view. In Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 218–226, 2000.

    Google Scholar 

  13. N. Elmqvist, A. Vande Moere, H.-C. Jetter, D. Cernea, H. Reiterer, and T. J. Jankun-Kelly. Fluid interaction for information visualization. Information Visualization, 10(4):327–340, Oct. 2011.

    Article  Google Scholar 

  14. M. J. Eppler. The image of insight: The use of visual metaphors in the communication of knowledge. In Proceedings of I-KNOW, volume 3, pages 2–4, 2003.

    Google Scholar 

  15. J. Gurlitt, S. Dummel, S. Schuster, and M. Nückles. Differently structured advance organizers lead to different initial schemata and learning outcomes. Instructional Science, 40(2):351–369, 2012.

    Article  Google Scholar 

  16. C. G. Healey, K. S. Booth, and J. T. Enns. High-speed visual estimation using preattentive processing. ACM Transactions on Computer-Human Interaction (TOCHI), 3(2):107–135, 1996.

    Article  Google Scholar 

  17. M. Hegarty. Diagrams in the mind and in the world: Relations between internal and external visualizations. In International Conference on Theory and Application of Diagrams, pages 1–13. Springer, 2004.

    Google Scholar 

  18. M. Hegarty. The cognitive science of visual-spatial displays: Implications for design. Topics in cognitive science, 3(3):446–474, 2011.

    Article  Google Scholar 

  19. J. Heiser and B. Tversky. Arrows in comprehending and producing mechanical diagrams. Cognitive science, 30(3):581–592, 2006.

    Article  Google Scholar 

  20. C. Held, M. Gottfried, G. Vosgerau, and M. Knauff. Mental models and the mind: current developments in Cognitive Psychology, Neuroscience and Philosophy of Mind. Elsevier, 2006.

    Google Scholar 

  21. E. Hutchins. Cognition in the Wild. MIT press, 1995.

    Google Scholar 

  22. P. N. Johnson-Laird. Mental models in cognitive science. Cognitive science, 4(1):71–115, 1980.

    Article  Google Scholar 

  23. G. Klein and R. R. Hoffman. Macrocognition, mental models, and cognitive task analysis methodology. Naturalistic decision making and macrocognition, pages 57–80, 2008.

    Google Scholar 

  24. G. Klein, J. Phillips, E. Rall, and D. Peluso. A Data-frame theory of sensemaking. In R. R. Hoffman, editor, Expertise Out of Context: Proceedings of the Sixth International Conference on Naturalistic Decision Making, pages 113–155, New York, NY, 2007. Lawrence Erlbaum Assoc Inc.

    Google Scholar 

  25. R. Kosara and J. Mackinlay. Storytelling: The next step for visualization. Computer, 46(5):44–50, 2013.

    Article  Google Scholar 

  26. Z. Liu, N. Nersessian, and J. Stasko. Distributed cognition as a theoretical framework for information visualization. IEEE transactions on visualization and computer graphics, 14(6):1173–1180, 2008.

    Article  Google Scholar 

  27. Z. Liu and J. Stasko. Mental models, visual reasoning and interaction in information visualization: A top-down perspective. IEEE transactions on visualization and computer graphics, 16(6):999–1008, 2010.

    Article  Google Scholar 

  28. E. Mayr, G. Schreder, M. Smuc, and F. Windhager. Looking at the representations in our mind: Measuring mental models of information visualizations. In Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization, BELIV ’16, page 96–103, New York, NY, USA, 2016. Association for Computing Machinery.

    Google Scholar 

  29. E. Mayr and F. Windhager. Once upon a spacetime: Visual storytelling in cognitive and geotemporal information spaces. ISPRS International Journal of Geo-Information, 7(3), 2018.

    Google Scholar 

  30. W. McCarty. Modeling: A Study in Words and Meanings. In S. Schreibman, R. Siemens, and J. Unsworth, editors, A Companion to Digital Humanities, page (chapter 19). Blackwell, Oxford, online edition, 2004.

    Google Scholar 

  31. T. F. Mcmanus. Individualizing instruction in a web-based hypermedia learning environment: Nonlinearity, advance organizers, and self-regulated learners. Journal of Interactive Learning Research, 11(2):219–251, 2000.

    Google Scholar 

  32. T. Munzner. Visualization Analysis and Design. A K Peters/CRC Press, Boca Raton, 1 edition, Dec. 2014.

    Book  Google Scholar 

  33. D. A. Norman. Some observations on mental models. Mental models, 7(112):7–14, 1983.

    Google Scholar 

  34. C. North. Toward measuring visualization insight. IEEE computer graphics and applications, 26(3):6–9, 2006.

    Article  Google Scholar 

  35. C. North and B. Shneiderman. Snap-together visualization: can users construct and operate coordinated visualizations? International Journal of Human-Computer Studies, 53(5):715–739, 2000.

    Article  MATH  Google Scholar 

  36. L. M. Padilla, S. H. Creem-Regehr, M. Hegarty, and J. K. Stefanucci. Decision making with visualizations: A cognitive framework across disciplines. Cognitive research: principles and implications, 3(1):1–25, 2018.

    Google Scholar 

  37. R. E. Patterson, L. M. Blaha, G. G. Grinstein, K. K. Liggett, D. E. Kaveney, K. C. Sheldon, P. R. Havig, and J. A. Moore. A human cognition framework for information visualization. Computers & Graphics, 42:42–58, 2014.

    Article  Google Scholar 

  38. P. Pirolli and S. Card. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, volume 5, pages 2–4. McLean, VA, USA, 2005.

    Google Scholar 

  39. M. Pohl, M. Smuc, and E. Mayr. The user puzzle—explaining the interaction with visual analytics systems. IEEE transactions on visualization and computer graphics, 18(12):2908–2916, 2012.

    Article  Google Scholar 

  40. Z. Qu and J. Hullman. Keeping Multiple Views Consistent: Constraints, Validations, and Exceptions in Visualization Authoring. IEEE transactions on visualization and computer graphics, Aug. 2017.

    Google Scholar 

  41. J. C. Roberts. State of the art: Coordinated & multiple views in exploratory visualization. In Coordinated and Multiple Views in Exploratory Visualization, 2007. CMV’07. Fifth International Conference on, pages 61–71. IEEE, 2007.

    Google Scholar 

  42. P. Ruchikachorn and K. Mueller. Learning visualizations by analogy: Promoting visual literacy through visualization morphing. IEEE transactions on visualization and computer graphics, 21(9):1028–1044, 2015.

    Article  Google Scholar 

  43. M. Scaife and Y. Rogers. External cognition: how do graphical representations work? International journal of human-computer studies, 45(2):185–213, 1996.

    Article  Google Scholar 

  44. R. C. Schank and R. P. Abelson. Scripts, plans, goals, and understanding: An inquiry into human knowledge structures. Psychology Press, 1977.

    MATH  Google Scholar 

  45. W. Schnotz and C. Kürschner. External and internal representations in the acquisition and use of knowledge: visualization effects on mental model construction. Instructional Science, 36(3):175–190, May 2008.

    Article  Google Scholar 

  46. G. Schreder, F. Windhager, M. Smuc, and E. Mayr. A mental models perspective on designing information visualizations for political communication. JeDEM-eJournal of eDemocracy and Open Government, 8(3):80–99, 2016.

    Article  Google Scholar 

  47. E. Segel and J. Heer. Narrative visualization: Telling stories with data. IEEE Transactions on Visualization and Computer Graphics, 16(6):1139–1148, 2010.

    Article  Google Scholar 

  48. B. Shneiderman. The eyes have it: a task by data type taxonomy for information visualizations. In Proceedings 1996 IEEE Symposium on Visual Languages, pages 336–343, 1996.

    Google Scholar 

  49. M. Smuc, P. Federico, F. Windhager, W. Aigner, L. Zenk, and S. Miksch. How do you connect moving dots? insights from user studies on dynamic network visualizations. In Handbook of human centric visualization, pages 623–650. Springer, 2014.

    Google Scholar 

  50. R. Spence. A framework for navigation. International Journal of Human-Computer Studies, 51(5):919–945, 1999.

    Article  Google Scholar 

  51. N. L. Stein and V. I. Kissel. Story schemata and causal structure. In Routledge Encyclopedia of Narrative Theory, pages 567–568. Taylor & Francis Group, 2010.

    Google Scholar 

  52. C. Stoiber, F. Grassinger, M. Pohl, H. Stitz, M. Streit, and W. Aigner. Visualization onboarding: Learning how to read and use visualizations. In Proceedings of VisComm 2019. OSF Preprints, 2019.

    Google Scholar 

  53. C. Tominski, G. Andrienko, N. Andrienko, S. Bleisch, S. I. Fabrikant, E. Mayr, S. Miksch, M. Pohl, and A. Skupin. Toward flexible visual analytics augmented through smooth display transitions. Visual Informatics, 2021.

    Google Scholar 

  54. B. Tversky. Cognitive maps, cognitive collages, and spatial mental models. In European conference on spatial information theory, pages 14–24. Springer, 1993.

    Google Scholar 

  55. T. A. van Dijk and W. Kintsch. Strategies of Discourse Comprehension. Academic Press, 1983.

    Google Scholar 

  56. C. D. Wickens and C. M. Carswell. The proximity compatibility principle: Its psychological foundation and relevance to display design. Human factors, 37(3):473–494, 1995.

    Article  Google Scholar 

  57. F. Windhager and E. Mayr. Cultural heritage cube. A conceptual framework for visual exhibition exploration. In 2012 16th International Conference on Information Visualisation, pages 540–545. IEEE, 2012.

    Google Scholar 

  58. F. Windhager, S. Salisu, R. A. Leite, V. Filipov, S. Miksch, G. Schreder, and E. Mayr. Many views are not enough: Designing for synoptic insights in cultural collections. IEEE Computer Graphics and Applications, 40(3):58–71, 2020.

    Article  Google Scholar 

  59. R. A. Zwaan. Five dimensions of narrative comprehension: The event-indexing model. In S. R. Goldman, A. C. Graesser, and P. van den Broek, editors, Narrative comprehension, causality, and coherence: Essays in honor of Tom Trabasso, pages 93–110. Lawrence Erlbaum Associates, 1999.

    Google Scholar 

  60. R. A. Zwaan and G. A. Radvansky. Situation models in language comprehension and memory. Psychological bulletin, 123(2):162, 1998.

    Google Scholar 

Download references

Acknowledgements

This work has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 101004825 and from the Austrian Science Fund (FWF), Project No. P28363-G24.

Author information

Authors and Affiliations

  1. Danube University for Continuing Education Krems, Krems an der Donau, Austria

    Florian Windhager & Eva Mayr

Authors
  1. Florian Windhager
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eva Mayr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Florian Windhager .

Editor information

Editors and Affiliations

  1. University of Colorado Boulder, Boulder, CO, USA

    Danielle Albers Szafir

  2. Department of Computer Science, King's College London, London, UK

    Rita Borgo

  3. OeRC, Department of Engineering Science, University of Oxford, Oxford, UK

    Min Chen

  4. Department of Public Health, Swansea University, Swansea, UK

    Darren J. Edwards

  5. School of Interactive Arts & Technology, Simon Fraser University, Surrey, BC, Canada

    Brian Fisher

  6. Northeastern University, Boston, USA

    Lace Padilla

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Windhager, F., Mayr, E. (2023). Mental Models and Visualization. In: Albers Szafir, D., Borgo, R., Chen, M., Edwards, D.J., Fisher, B., Padilla, L. (eds) Visualization Psychology. Springer, Cham. https://doi.org/10.1007/978-3-031-34738-2_3

Download citation

Publish with us

Policies and ethics

Search

Navigation