Evaluation of Visualization Heuristics
Multiple sets of heuristic have been developed and studied in the Human Computer Interaction (HCI) domain as a method for fast, lightweight evaluations for usability problems. However, none of the heuristics have been adopted by the information visualization or the visual analytics communities. Our literature review looked at heuristic sets developed by Nielsen and Molich  and Forsell and Johansson  to understand how these heuristics were developed and their intended applications. We also reviewed heuristic studies conducted by Hearst and colleagues  and Väätäjä and colleagues  to determine how individuals apply heuristics to evaluating visualization systems. While each study noted potential issues with the heuristic descriptions and the evaluator’s familiarity with the heuristics, no direct connections were made. Our research looks to understand how individuals with domain expertise in information visualization and visual analytics could use heuristics to discover usability problems and evaluate visualizations. By empirically evaluating visualization heuristics, we can identify the key ways that these heuristics can be used to inform the visual analytics design process. Further, they may help to identify usability problems that are and are not task specific. We hope to use this process to also identify missing heuristics that may apply to designs for different analytic purposes.
KeywordsHeuristics Visualization Heuristic evaluation User study Visual analytics Information visualization
The research described in this document was sponsored by the U.S. Department of Energy through the Pacific Northwest National Laboratory. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government.
- 1.Forsell, C., Johansson, J.: An heuristic set for evaluation in information visualization. In: Proceedings of the International Conference on Advanced Visual Interfaces, pp. 199–206. ACM (2010)Google Scholar
- 2.Hearst, M.A., Laskowski, P., Silva, L.: Evaluating information visualization via the interplay of heuristic evaluation and question-based scoring. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 5028–5033. ACM (2016)Google Scholar
- 3.Ishihara, S.: Tests for Colour-Blindness. Hongo Harukicho, Handaya (1917)Google Scholar
- 4.Nielsen, J.: Finding usability problems through heuristic evaluation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 373–380. ACM (1992)Google Scholar
- 5.Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 152–158. ACM (1994)Google Scholar
- 6.Nielsen, J.: 10 usability heuristics for user interface design. https://www.nngroup.com/articles/ten-usability-heuristics/
- 7.Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256. ACM (1990)Google Scholar
- 8.Plaisant, C., Grinstein, G., Scholtz, J., Whiting, M., O’Connell, T., Laskowski, S., Chien, L., Tat, A., Wright, W., Görg, C., et al.: Evaluating visual analytics at the 2007 vast symposium contest. IEEE Comput. Graph. Appl. 28(2) (2008)Google Scholar
- 9.Snellen, H.: Test-types for the Determination of the Acuteness of Vision. PW van de Weijer, Utrecht (1862)Google Scholar
- 10.Väätäjä, H., Varsaluoma, J., Heimonen, T., Tiitinen, K., Hakulinen, J., Turunen, M., Nieminen, H., Ihantola, P.: Information visualization heuristics in practical expert evaluation. In: Proceedings of the Beyond Time and Errors on Novel Evaluation Methods for Visualization, pp. 36–43. ACM (2016)Google Scholar