Evaluation of Visualization Heuristics

  • Ryan WilliamsEmail author
  • Jean Scholtz
  • Leslie M. Blaha
  • Lyndsey Franklin
  • Zhuanyi Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10901)


Multiple sets of heuristic have been developed and studied in the Human Computer Interaction (HCI) domain as a method for fast, lightweight evaluations for usability problems. However, none of the heuristics have been adopted by the information visualization or the visual analytics communities. Our literature review looked at heuristic sets developed by Nielsen and Molich [7] and Forsell and Johansson [1] to understand how these heuristics were developed and their intended applications. We also reviewed heuristic studies conducted by Hearst and colleagues [2] and Väätäjä and colleagues [10] to determine how individuals apply heuristics to evaluating visualization systems. While each study noted potential issues with the heuristic descriptions and the evaluator’s familiarity with the heuristics, no direct connections were made. Our research looks to understand how individuals with domain expertise in information visualization and visual analytics could use heuristics to discover usability problems and evaluate visualizations. By empirically evaluating visualization heuristics, we can identify the key ways that these heuristics can be used to inform the visual analytics design process. Further, they may help to identify usability problems that are and are not task specific. We hope to use this process to also identify missing heuristics that may apply to designs for different analytic purposes.


Heuristics Visualization Heuristic evaluation User study Visual analytics Information visualization 



The research described in this document was sponsored by the U.S. Department of Energy through the Pacific Northwest National Laboratory. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government.


  1. 1.
    Forsell, C., Johansson, J.: An heuristic set for evaluation in information visualization. In: Proceedings of the International Conference on Advanced Visual Interfaces, pp. 199–206. ACM (2010)Google Scholar
  2. 2.
    Hearst, M.A., Laskowski, P., Silva, L.: Evaluating information visualization via the interplay of heuristic evaluation and question-based scoring. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 5028–5033. ACM (2016)Google Scholar
  3. 3.
    Ishihara, S.: Tests for Colour-Blindness. Hongo Harukicho, Handaya (1917)Google Scholar
  4. 4.
    Nielsen, J.: Finding usability problems through heuristic evaluation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 373–380. ACM (1992)Google Scholar
  5. 5.
    Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 152–158. ACM (1994)Google Scholar
  6. 6.
    Nielsen, J.: 10 usability heuristics for user interface design.
  7. 7.
    Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256. ACM (1990)Google Scholar
  8. 8.
    Plaisant, C., Grinstein, G., Scholtz, J., Whiting, M., O’Connell, T., Laskowski, S., Chien, L., Tat, A., Wright, W., Görg, C., et al.: Evaluating visual analytics at the 2007 vast symposium contest. IEEE Comput. Graph. Appl. 28(2) (2008)Google Scholar
  9. 9.
    Snellen, H.: Test-types for the Determination of the Acuteness of Vision. PW van de Weijer, Utrecht (1862)Google Scholar
  10. 10.
    Väätäjä, H., Varsaluoma, J., Heimonen, T., Tiitinen, K., Hakulinen, J., Turunen, M., Nieminen, H., Ihantola, P.: Information visualization heuristics in practical expert evaluation. In: Proceedings of the Beyond Time and Errors on Novel Evaluation Methods for Visualization, pp. 36–43. ACM (2016)Google Scholar

Copyright information

© This is a U.S. government work and its text is not subject to copyright protection in the United States; however, its text may be subject to foreign copyright protection 2018

Authors and Affiliations

  • Ryan Williams
    • 1
    Email author
  • Jean Scholtz
    • 1
  • Leslie M. Blaha
    • 1
  • Lyndsey Franklin
    • 1
  • Zhuanyi Huang
    • 1
  1. 1.Pacific Northwest National LaboratoryRichlandUSA

Personalised recommendations