Skip to main content
SpringerLink
Log in

Evaluating Information Visualizations

  • Chapter
Information Visualization

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 4950))

Abstract

Information visualization research is becoming more established, and as a result, it is becoming increasingly important that research in this field is validated. With the general increase in information visualization research there has also been an increase, albeit disproportionately small, in the amount of empirical work directly focused on information visualization. The purpose of this chapter is to increase awareness of empirical research in general, of its relationship to information visualization in particular; to emphasize its importance; and to encourage thoughtful application of a greater variety of evaluative research methodologies in information visualization.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amar, R.A., Stasko, J.T.: Knowledge Precepts for Design and Evaluation of Information Visualizations. IEEE Transactions on Visualization and Computer Graphics 11(4), 432–442 (2005)

    Article  Google Scholar 

  2. Andrews, K.: Evaluating Information Visualisations. In: Proceedings of the 2006 AVI Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization, pp. 1–5 (2006)

    Google Scholar 

  3. Auerbach, C.: Qualitative Data: An Introduction to Coding and Analysis. University Press, New York (2003)

    Google Scholar 

  4. Baker, K., Greenberg, S., Gutwin, C.: Empirical Development of a Heuristic Evaluation Methodology for Shared Workspace Groupware. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work, pp. 96–105. ACM Press, New York (2002)

    Chapter  Google Scholar 

  5. Baldonado, M., Woodruff, A., Kuchinsky, A.: Guidelines for Using Multiple Views in Information Visualization. In: Proeedings of the Conference on Advanced Visual Interfaces (AVI), pp. 110–119. ACM Press, New York (2000)

    Chapter  Google Scholar 

  6. Barzun, J., Graff, H.: The Modern Researcher, 3rd edn. Harcourt Brace Jovanvich, New York (1977)

    Google Scholar 

  7. BELIV 2006, accessed http://www.dis.uniroma1.it/~beliv06/ (February 4, 2008)

  8. Bertin, J.: Semiology of Graphics (Translation: William J. Berg). University of Wisconsin Press (1983)

    Google Scholar 

  9. Bevan, N., Barnum, C., Cockton, G., Nielsen, J., Spool, J., Wixon, W.: The “Magic Number 5”: Is It Enough for Web Testing? In: CHI Extended Abstracts, pp. 698–699. ACM Press, New York (2003)

    Google Scholar 

  10. Boyatzis, R.: Transforming Qualitative Information: Thematic Analysis and Code Development. Sage Publications, London (1998)

    Google Scholar 

  11. Brereton, M., McGarry, B.: An Observational Study of How Objects Support Engineering Design Thinking and Communication: Implications for the Design of Tangible Media. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI’00), pp. 217–224. ACM Press, New York (2000)

    Chapter  Google Scholar 

  12. Chen, C., Czerwinski, M.: Introduction to the Special Issue on Empirical Evaluation of Information Visualizations. International Journal of Human-Computer Studies 53(5), 631–635 (2000)

    Article  Google Scholar 

  13. Corbin, J., Strauss, A.: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, 3rd edn. Sage Publications, Los Angeles (2008)

    Google Scholar 

  14. Creswell, J.: Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Sage Publications, London (1998)

    Google Scholar 

  15. Dix, A., Finlay, J., Abowd, G., Beale, R.: Human Computer Interaction, 2nd edn. Prentice-Hall, Englewood Cliffs (1998)

    Google Scholar 

  16. Dumais, S., Cutrell, E., Chen, H.: Optimizing Search by Showing Results In Context. In: Proc. CHI’01, pp. 277–284. ACM Press, New York (2001)

    Google Scholar 

  17. Eberts, R.E.: User Interface Design. Prentice-Hall, Englewood Cliffs (1994)

    MATH  Google Scholar 

  18. Ellis, E., Dix, A.: An Explorative Analysis of User Evaluation Studies in Information Visualization. In: Proceedings of the Workshop on Beyond Time and Errors: Novel Evaluation Methods for Information Visualization, BELIV (2006)

    Google Scholar 

  19. Ericsson, K., Simon, H.: Verbal Reports as Data. Psychological Review 87(3), 215–251 (1980)

    Article  Google Scholar 

  20. Ericsson, K., Simon, H.: Verbal Reports on Thinking. In: Faerch, C., Kasper, G. (eds.) Introspection in Second Language Research, pp. 24–54. Multilingual Matters, Clevedon, Avon (1987)

    Google Scholar 

  21. Ericsson, K., Simon, H.: Protocol Analysis: Verbal Reports as Data, 2nd edn. MIT Press, Boston (1993)

    Google Scholar 

  22. Fall, J., Fall, A.: SELES: A Spatially Explicit Landscape Event Simulator. In: Proceedings of GIS and Environmental Modeling, pp. 104–112. National Center for Geographic Information and Analysis (1996)

    Google Scholar 

  23. Forlines, C., Shen, C., Wigdor, D., Balakrishnan, R.: Exploring the effects of group size and display configuration on visual search. In: Computer Supported Cooperative Work 2006 Conference Proceedings, pp. 11–20 (2006)

    Google Scholar 

  24. Garfinkel, H.: Studies in Ethnomethodology. Polity Press, Cambridge (1967)

    Google Scholar 

  25. Gonzalez, V., Kobsa, A.: A Workplace Study of the Adoption of Information Visualization systems. In: Proceedings of the International Conference on Knowledge Management, pp. 92–102 (2003)

    Google Scholar 

  26. Gorard, S.: Combining Methods in Educational Research. McGraw-Hill, New York (2004)

    Google Scholar 

  27. Greenberg, S., Buxton, B.: Usability Evaluation Considered Harmful (Some of the Time). In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2008)

    Google Scholar 

  28. Greene, J., Caracelli, V., Graham, W.: Toward a Conceptual Framework for Mixed-Method Evaluation Design. Educational Evaluation and Policy Analysis 11(3), 255–274 (1989)

    Google Scholar 

  29. Gutwin, C., Greenberg, S.: The Mechanics of Collaboration: Developing Low Cost Usability Evaluation Methods for Shared Workspaces. In: Proceedings WETICE, pp. 98–103. IEEE Computer Society Press, Los Alamitos (2000)

    Google Scholar 

  30. Healey, C.G.: On the Use of Perceptual Cues and Data Mining for Effective Visualization of Scientific Datasets. In: Proceedings of Graphics Interface, pp. 177–184 (1998)

    Google Scholar 

  31. Heer, J., Viegas, F., Wattenberg, M.: Voyagers and Voyeurs: Supporting Asynchronous Collaborative Information Visualization. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI’07), pp. 1029–1038. ACM Press, New York (2007)

    Google Scholar 

  32. Holtzblatt, K., Beyer, H.: Contextual Design: Defining Customer-Centered Systems. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  33. Huck, S.W.: Reading Statistics and Research, 4th edn. Pearson Education Inc., Boston (2004)

    Google Scholar 

  34. Interrante, V.: Illustrating Surface Shape in Volume Data via Principal Direction-Driven 3D Line Integral Convolution. Computer Graphics, Annual Conference Series, pp. 109–116 (1997)

    Google Scholar 

  35. Isenberg, P., Carpendale, S.: Interactive Tree Comparison for Co-located Collaborative Information Visualization. IEEE Transactions on Visualization and Computer Graphics 12(5) (2007)

    Google Scholar 

  36. Isenberg, P., Tang, A., Carpendale, S.: An Exploratory Study of Visual Information Analysis. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI’08), ACM Press, New York (to appear, 2008)

    Google Scholar 

  37. Kay, J., Reiger, H., Boyle, M., Francis, G.: An Ecosystem Approach for Sustainability: Addressing the Challenge of Complexity. Futures 31(7), 721–742 (1999)

    Article  Google Scholar 

  38. Kim, S., Hagh-Shenas, H., Interrante, V.: Conveying Shape with Texture: Experimental Investigations of Texture’s Effects on Shape Categorization Judgments. IEEE Transactions on Visualization and Computer Graphics 10(4), 471–483 (2004)

    Article  Google Scholar 

  39. Kleffner, D.A., Ramachandran, V.S.: On the Perception of Shape from Shading. Perception and Psychophysics 52(1), 18–36 (1992)

    Google Scholar 

  40. Kobsa, A.: User Experiments with Tree Visualization Systems. In: Proceedings of the IEEE Symposium on Information Visualization, pp. 9–26 (2004)

    Google Scholar 

  41. Kruger, R., Carpendale, S., Scott, S.D., Greenberg, S.: Roles of Orientation in Tabletop Collaboration: Comprehension, Coordination and Communication. Journal of Computer Supported Collaborative Work 13(5–6), 501–537 (2004)

    Article  Google Scholar 

  42. Lewin, C. (ed.): Research Methods in the Social Sciences. Sage Publications, London (2004)

    Google Scholar 

  43. Lewis, C., Rieman, J.: Task-Centered User Interface Design: A Practical Introduction (1993)

    Google Scholar 

  44. Likert, R.: A Technique for the Measurement of Attitudes. Archives of Psychology 140, 1–55 (1932)

    Google Scholar 

  45. Lofland, J., Lofland, L.: Analyzing Social Settings: A Guide to Qualitative Observation and Analysis. Wadsworth Publishing Company, CA, USA (1995)

    Google Scholar 

  46. Mankoff, J., Dey, A., Hsieh, G., Kientz, J., Lederer, S., Ames, A.: Heuristic Evaluation of Ambient Displays. In: Proceedings of CHI ’03, pp. 169–176. ACM Press, New York (2003)

    Google Scholar 

  47. Mark, G., Kobsa, A., Gonzalez, V.: Do Four Eyes See Better Than Two? Collaborative Versus Individual Discovery in Data Visualization Systems. In: Proceedings of the IEEE Conference on Information Visualization (IV’02), July 2002, pp. 249–255. IEEE Press, Los Alamitos (2002)

    Chapter  Google Scholar 

  48. Mazza, R., Berre, A.: Focus Group Methodology for Evaluating Information Visualization Techniques and Tools. In: Proceedings of the International Conference on Information Visualization IV (2007)

    Google Scholar 

  49. McCarthy, D.: Normal Science and Post-Normal Inquiry: A Context for Methodology (2004)

    Google Scholar 

  50. McGrath, J.: Methodology Matters: Doing Research in the Social and Behavioural Sciences. In: Readings in Human-Computer Interaction: Toward the Year 2000, Morgan Kaufmann, San Francisco (1995)

    Google Scholar 

  51. Moggridge, B.: Design Interactions. MIT Press, Cambridge (2006)

    Google Scholar 

  52. Morris, M.R., Ryall, K., Shen, C., Forlines, C., Vernier, F.: Beyond “Social Protocols”: Multi-User Coordination Policies for Co-located Groupware. In: Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW, Chicago, IL, USA), CHI Letters, November 6-10, 2004, pp. 262–265. ACM Press, New York (2004)

    Google Scholar 

  53. Morse, E., Lewis, M., Olsen, K.: Evaluating Visualizations: Using a Taxonomic Guide. Int. J. Human-Computer Studies 53, 637–662 (2000)

    Article  MATH  Google Scholar 

  54. Nielsen, J., Mack, R.: Usability Inspection Methods. John Wiley & Sons, Chichester (1994)

    Google Scholar 

  55. North, C.: Toward Measuring Visualization Insight. IEEE Computer Graphics and Applications 26(3), 6–9 (2006)

    Article  Google Scholar 

  56. Patton, M.Q.: Qualitative Research and Evaluation Methods, 3rd edn. Sage Publications, London (2001)

    Google Scholar 

  57. Plaisant, C.: The Challenge of Information Visualization Evaluation. In: Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 109–116 (2004)

    Google Scholar 

  58. Purchase, H.C., Hoggan, E., Görg, C.: How Important Is the “Mental Map”? – An Empirical Investigation of a Dynamic Graph Layout Algorithm. In: Kaufmann, M., Wagner, D. (eds.) GD 2006. LNCS, vol. 4372, pp. 184–195. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  59. Purchase, H.C.: Effective Information Visualisation: A Study of Graph Drawing Aesthetics and Algorithms. Interacting with Computers 13(2), 477–506 (2000)

    Article  Google Scholar 

  60. Purchase, H.C.: Performance of Layout Algorithms: Comprehension, Not Computation. Journal of Visual Languages and Computing 9, 647–657 (1998)

    Article  Google Scholar 

  61. Brandenburg, F.J. (ed.): GD 1995. LNCS, vol. 1027. Springer, Heidelberg (1996)

    Google Scholar 

  62. Reilly, D., Inkpen, K.: White Rooms and Morphing Don’t Mix: Setting and the Evaluation of Visualization Techniques. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 111–120 (2007)

    Google Scholar 

  63. Saraiya, P., North, C., Duca, K.: An Insight-Based Methodology for Evaluating Bioinformatics Visualizations. IEEE Transactions on Visualization and Computer Graphics 11(4), 443–456 (2005)

    Article  Google Scholar 

  64. Scott, S.D., Carpendale, S., Inkpen, K.: Territoriality in Collaborative Tabletop Workspaces. In: Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW, Chicago, IL, USA), CHI Letters, November 6-10, 2004, pp. 294–303. ACM Press, New York (2004)

    Google Scholar 

  65. Seidman, I.: Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences. Teachers’ College Press, New York (1998)

    Google Scholar 

  66. Shneiderman, B.: The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. In: Proceedings of the IEEE Symposium on Visual Languages, pp. 336–343. IEEE Computer Society Press, Los Alamitos (1996)

    Chapter  Google Scholar 

  67. Shneiderman, B., Plaisant, C.: Strategies for Evaluating Information Visualization Tools: Multi-Dimensional In-Depth Long-Term Case Studies. In: Proceedings of the Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization, BELIV (2006)

    Google Scholar 

  68. Spence, R.: Information Visualization, 2nd edn. Addison-Wesley, Reading (2007)

    Google Scholar 

  69. Spool, J., Schroeder, W.: Testing Web Sites: Five Users is Nowhere Near Enough. In: CHI ’01 Extended Abstracts, pp. 285–286. ACM Press, New York (2001)

    Chapter  Google Scholar 

  70. Strauss, A.L., Corbin, J.: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications, London (1998)

    Google Scholar 

  71. Tang, A., Tory, M., Po, B., Neumann, P., Carpendale, S.: Collaborative Coupling over Tabletop Displays. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI’06), pp. 1181–1290. ACM Press, New York (2006)

    Google Scholar 

  72. Tang, A., Carpendale, S.: An observational study on information flow during nurses’ shift change. In: Proc. of the ACM Conf. on Human Factors in Computing Systems (CHI), pp. 219–228. ACM Press, New York (2007)

    Google Scholar 

  73. Tang, J.C.: Findings from observational studies of collaborative work. International Journal of Man-Machine Studies 34(2), 143–160 (1991)

    Article  Google Scholar 

  74. Tory, M., Möller, T.: Evaluating Visualizations: Do Expert Reviews Work. IEEE Computer Graphics and Applications 25(5), 8–11 (2005)

    Article  Google Scholar 

  75. Tufte, E.: The Visual Display of Quantitative Information. Graphics Press, Cheshire (1986)

    Google Scholar 

  76. Tufte, E.: Envisioning Information. Graphics Press, Cheshire (1990)

    Google Scholar 

  77. Tufte, E.: Visual Explanations. Images and Quantities, Evidence and Narrative. Graphics Press, Cheshire (1997)

    MATH  Google Scholar 

  78. Viegas, F.B., Wattenberg, M., van Ham, F., Kriss, J., McKeon, M.: Many Eyes: A Site for Visualization at Internet Scale. IEEE Transactions on Visualization and Computer Graphics (Proceedings Visualization / Information Visualization 2007) 12(5), 1121–1128 (2007)

    Article  Google Scholar 

  79. Ware, C.: Information Visualization: Perception for Design, 2nd edn. Morgan Kaufmann, San Francisco (2004)

    Google Scholar 

  80. Wigdor, D., Shen, C., Forlines, C., Balakrishnan, R.: Perception of Elementary Graphical Elements in Tabletop and Multi-surface Environments. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI’07), pp. 473–482. ACM Press, New York (2007)

    Google Scholar 

  81. Willett, W., Heer, J., Agrawala, M.: Scented Widgets: Improving Navigation Cues with Embedded Visualizations. In: INFOVIS 2007. IEEE Symposium on Information Visualization (2007)

    Google Scholar 

  82. Yost, B., North, C.: The Perceptual Scalability of Visualization. IEEE Transactions on Visualization and Computer Graphics 12(5), 837–844 (2006)

    Article  Google Scholar 

  83. Zuk, T.: Uncertainty Visualizations. PhD thesis. Department of Compute Science, University of Calgary (2007)

    Google Scholar 

  84. Zuk, T., Carpendale, S.: Theoretical Analysis of Uncertainty Visualizations. In: Proceedings of SPIE Conference Electronic Imaging, Vol. 6060: Visualization and Data Analysis (2006)

    Google Scholar 

  85. Zuk, T., Schlesier, L., Neumann, P., Hancock, M.S., Carpendale, S.: Heuristics for Information Visualization Evaluation. In: Proceedings of the Workshop BEyond Time and Errors: Novel Evaluation Methods for Information Visualization (BELIV 2006), held in conjunction with the Working Conference on Advanced Visual Interfaces (AVI 2006), ACM Press, New York (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Calgary, 2500 University Dr. NW, Calgary, AB, Canada, T2N 1N4

    Sheelagh Carpendale

Authors
  1. Sheelagh Carpendale
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Andreas Kerren John T. Stasko Jean-Daniel Fekete Chris North

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Carpendale, S. (2008). Evaluating Information Visualizations. In: Kerren, A., Stasko, J.T., Fekete, JD., North, C. (eds) Information Visualization. Lecture Notes in Computer Science, vol 4950. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70956-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-70956-5_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-70955-8

  • Online ISBN: 978-3-540-70956-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation