Measuring Effective Data Visualization

  • Ying Zhu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4842)


In this paper, we systematically examine two fundamental questions in information visualization – how to define effective visualization and how to measure it. Through a literature review, we point out that the existing definitions of effectiveness are incomplete and often inconsistent – a problem that has deeply affected the design and evaluation of visualization. There is also a lack of standards for measuring the effectiveness of visualization as well as a lack of standardized procedures. We have identified a set of basic research issues that must be addressed. Finally, we provide a more comprehensive definition of effective visualization and discuss a set of quantitative and qualitative measures. The work presented in this paper contributes to the foundational research of information visualization.


User Study Visualization Technique Information Visualization Task Completion Time Heuristic Evaluation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Johnson, C., Moorhead, R., Munzner, T., Pfister, H., Rheingans, P., Yoo, T.S.: NIH/NSF Visualization Research Challenges Report. IEEE Press (2006)Google Scholar
  2. 2.
    Johnson, C.R.: Top Scientific Visualization Research Problems. IEEE Computer Graphics & Applications 24, 13–17 (2004)CrossRefGoogle Scholar
  3. 3.
    Wijk, J.J.v.: The Value of Visualization. In: Proceedings of IEEE Visualization Conference Minneapolis, MN, IEEE, Los Alamitos (2005)Google Scholar
  4. 4.
    Dastani, M.: The Role of Visual Perception in DataVisualization. Journal of Visual Languages and Computing 13, 601–622 (2002)CrossRefGoogle Scholar
  5. 5.
    Wattenberg, M., Fisher, D.: Analyzing perceptual organization in information graphics. Information Visualization 3, 123–133 (2004)CrossRefGoogle Scholar
  6. 6.
    Tufte, E.R.: The Visual Display of Quantitative Information, 2nd edn. Graphics Press (2001)Google Scholar
  7. 7.
    Kosslyn, S.M.: Graphics and Human Information Processing: A Review of Five Books. Journal of the American Statistical Association 80, 499–512 (1985)CrossRefGoogle Scholar
  8. 8.
    Casner, S.M.: A task-analytic approach to the automated design of graphic presentation. ACM Transactions on Graphics 10, 111–151 (1991)CrossRefGoogle Scholar
  9. 9.
    Bertin, J.: Semiology of Graphics: University of Wisconsin Press (1983)Google Scholar
  10. 10.
    Nowell, L., Schulman, R., Hix, D.: Graphical Encoding for Information Visualization: An Empirical Study. In: Proceedings of the IEEE Symposium on Information Visualization 2002 (InfoVis) (2002)Google Scholar
  11. 11.
    Amar, R.A., Stasko, J.T.: Knowledge Precepts for Design and Evaluation of Information Visualizations. IEEE Transactions on Visualization and Computer Graphics 11, 432–442 (2005)CrossRefGoogle Scholar
  12. 12.
    Tweedie, L.: Characterizing Interactive Externalizations. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI) (1997)Google Scholar
  13. 13.
    Cox, R.: Representation construction, externalised cognition and individual differences. Learning and Instruction 9, 343–363 (1999)CrossRefGoogle Scholar
  14. 14.
    Freedman, E.G., Shah, P.: Toward a Model of Knowledge-Based Graph Comprehension. In: Hegarty, M., Meyer, B., Narayanan, N.H. (eds.) Diagrams 2002. LNCS (LNAI), vol. 2317, pp. 59–141. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    Scaife, M., Rogers, Y.: External cognition: how do graphical representations work? International Journal of Human-Computer Studies 45, 185–213 (1996)CrossRefGoogle Scholar
  16. 16.
    Vekiri, I.: What Is the Value of Graphical Displays in Learning? Educational Psychology Review 14, 261–312 (2002)CrossRefGoogle Scholar
  17. 17.
    Lohse, G.L.: The role of working memory on graphical information processing. Behaviour & Information Technology 16, 297–308 (1997)CrossRefGoogle Scholar
  18. 18.
    Marcus, N., Cooper, M., Sweller, J.: Understanding Instructions. Journal of Educational Psychology 88, 49–63 (1996)CrossRefGoogle Scholar
  19. 19.
    Sweller, J.: Visualisation and Instructional Design. In: Proceedings of the International Workshop on Dynamic Visualizations and Learning (2002)Google Scholar
  20. 20.
    Cleveland, W.S., McGill, R.: Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods. Journal of the American Statistical Association 79, 531–554 (1984)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Mackinlay, J.: Automating the Design of Graphical Presentations of Relational Information. ACM Transactions on Graphics 5, 110–141 (1986)CrossRefGoogle Scholar
  22. 22.
    Tversky, B., Agrawala, M., Heiser, J., Lee, P., Hanrahan, P., Phan, D., Stolte, C., Daniel, M.-P.: Cognitive Design Principles for Automated Generation of Visualizations. In: Allen, G.L. (ed.) Applied Spatial Cognition: From Research to Cognitive Technology, Lawrence Erlbaum Associates, Mahwah (2006)Google Scholar
  23. 23.
    Kosslyn, S.M.: Understanding Charts and Graphs. Applied Cognitive Psychology 3, 185–226 (1989)CrossRefGoogle Scholar
  24. 24.
    Petre, M., Green, T.R.G.: Learning to Read Graphics: Some Evidence that ’Seeing’ an Information Display is an Acquired Skill. Journal of Visual Languages and Computing 4, 55–70 (1993)CrossRefGoogle Scholar
  25. 25.
    Craft, B., Cairns, P.: Beyond guidelines: what can we learn from the visual information seeking mantra? In: Proceedings of the 9th IEEE International Conference on Information Visualization (IV) (2005)Google Scholar
  26. 26.
    Tory, M., Moller, T.: Evaluating Visualizations: Do Expert Reviews Work? IEEE Computer Graphics and Applications 25, 8–11 (2005)CrossRefGoogle Scholar
  27. 27.
    Chambers, J.M., Cleveland, W.S., Tukey, P.A.: Graphical methods for data analysis. Duxbury Press (1983)Google Scholar
  28. 28.
    Cleveland, W.S.: Visualizing Data. Hobart Press (1993)Google Scholar
  29. 29.
    Wilkinson, L.: The Grammar of Graphics, 2nd edn. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  30. 30.
    Senay, H., Ignatius, E.: Rules and Principles of Scientific Data Visualization. In: State of the art in data visualization, SIGGRAPH Course Notes (1990)Google Scholar
  31. 31.
    Shneiderman, B.: The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. In: Proceedings of the IEEE Conference on Visual Languages, IEEE, Los Alamitos (1996)Google Scholar
  32. 32.
    Casner, S.M., Larkin, J.H.: Cognitive Efficiency Considerations for Good Graphic Design. In: Proceedings of the Eleventh Annual Conference of the Cognitive Science Society Ann Arbor, MI (1989)Google Scholar
  33. 33.
    Lohse, G.L.: A Cognitive Model for Understanding Graphical Perception. Human-Computer Interaction 8, 353–388 (1993)CrossRefGoogle Scholar
  34. 34.
    Cox, R., Brna, P.: Supporting the use of external representation in problem solving: the need for flexible learning environments. Journal of Artificial Intelligence in Education 6, 239–302 (1995)Google Scholar
  35. 35.
    Saraiya, P., North, C., Duca, K.: An Insight-Based Methodology for Evaluating Bioinformatics Visualizations. IEEE Transactions on Visualization and Computer Graphics 11, 443–456 (2005)CrossRefGoogle Scholar
  36. 36.
    Shneiderman, B., Plaisant, C.: Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies. In: Proceedings of the AVI workshop on Beyond time and errors: novel evaluation methods for information visualization, ACM, New York (2006)Google Scholar
  37. 37.
    Zhu, Y., Suo, X., Owen, G.S.: Complexity Analysis for Information Visualization Design and Evaluation. In: Proceedings of the 3rd International Symposium on Visual Computing (ISVC). LNCS, vol. 4841, Springer, Heidelberg (2007)Google Scholar
  38. 38.
    Plaisant, C.:Information Visualization Repository (2007),
  39. 39.
    Card, S.K., Mackinlay, J.: The Structure of the Information Visualization Design Space. In: Proceedings of the IEEE Symposium on Information Visualization (InfoVis) (1997)Google Scholar
  40. 40.
    Chi, E.H.: A Taxonomy of Visualization Techniques using the Data State Reference Model. In: Proceeding of IEEE Symposium on Information Visualization (InfoVis) (2000)Google Scholar
  41. 41.
    Lohse, G.L., Biolsi, K., Walker, N., Rueter, H.H.: A Classification of Visual Representations. Communications of the ACM 37, 36–49 (1995)CrossRefGoogle Scholar
  42. 42.
    Tory, M., Möller, T.: Rethinking Visualization: A High-Level Taxonomy. In: Proceeding of the IEEE Symposium on Information Visualization (InfoVis) (2004)Google Scholar
  43. 43.
    Wehrend, S., Lewis, C.: A Problem-oriented Classification of Visualization Techniques. In: Proceedings of the IEEE Symposium on Information Visualization (InfoVis), IEEE, Los Alamitos (1990)Google Scholar
  44. 44.
    Senay, H., Ignatius, E.: A Knowledge-Based System for Visualization Design. IEEE Computer Graphics & Applications 14, 36–47 (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Ying Zhu
    • 1
  1. 1.Department of Computer Science, Georgia State University, Atlanta, GeorgiaUSA

Personalised recommendations