User-Centered Evaluation

Abstract

Visual analytics systems are not only complex to design and implement, they also present multiple challenges in evaluation. While it is, of course, necessary to evaluate software to ensure that it functions according to specifications, this chapter is concerned with evaluating the software from the standpoint of how well it serves the user population. ISO 9241-11 defines usability as “the effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments.” While evaluating the usability of the software is important, many additional and important questions need to be answered as well especially in evaluating the utility of new software. Usability issues need to be solved for specific tasks, but the following additional questions go beyond the usability of the software. They include questions of how well the software fits into processes in the user’s workplace, if the user can accomplish her tasks more easily with the software, how difficult it is to import the users’ data, if the visualizations are easily interpretable, how well the visualizations fit into their reasoning process, if the users can use their favorite analytic methods using this software, and if they are able to gain insights for their analysis.