A Validation Study of a Visual Analytics Tool with End Users

  • Heloisa Candello
  • Victor Fernandes Cavalcante
  • Alan Braz
  • Rogério Abreu De Paula
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8520)


In this paper we describe an user evaluation that aimed to understand how a group of endusers interpret a visual analytics tool in the context of service delivery. It is common for service factories to have an organization devoted to handle incidents. Many incident management systems have strict controls on how fast incidents should be handled, often subjected to penalties when targets are not met.We call Time-Bounded Incident Management (TBIM) those systems, which require clearly defined incident resolution times. In our project, research scientists proposed a method and a visual representation named Workload Profile Chart (WPC) that had as primary goal to understand the area of incident management in a service delivery department. The objective of this visual representation is to help characterizing the performance of TBIM systems and diagnosing major issues such as resource and skill allocation problems, abnormal behavior, and incident characteristics. Researchers wanted to understand if end-users, the quality analysts (QAs), would comprehend the charts and would be able to use them to identify problems and propose effective improvement actions related to TBIM activities. The study was conducted with ten QAs of a service delivery department of a IT company based in Brazil. The data was analyzed using descriptive statistical and qualitative methods. As a result, participants were mainly guided by the axes titles and chart legends to interpret the visualizations, and not always understood what kind of data the chart was displaying. Those results served as insights of how QAs think when analyzing TBIM information in a service delivery department and what improvements in the visual representation tool may be proposed to facilitate their activity. At last we identified evidences of how to design better visual analytics tools based on participant’s perceptions and interpretations of color differences and verbal information in chart labels and legend.


visual analytics service design user evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    de Souza, C.R.B., Pinhanez, C.S., Cavalcante, V.F.: Information Needs of System Administrators in Information Technology Service Factories. In: Proceedings of the 5th ACM Symposium on Computer Human Interaction for Management of Information Technology, CHIMIT 2011, pp. 3:1–3:10. ACM, New York (2011)Google Scholar
  2. 2.
    ITIL: Common ITIL Terms, (accessed February 03, 2013)
  3. 3.
    Card, S.K., Mackinlay, J.D., Shneiderman, B.: Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann Publishers Inc., San Francisco (1999)Google Scholar
  4. 4.
    Spence, R.: Information visualization. ACM Press books. Addison-Wesley, Harlow, England (2001)Google Scholar
  5. 5.
    Tufte, E.R.: Envisioning Information. Graphics Press, Cheshire (1990)Google Scholar
  6. 6.
    Ware, C.: Information Visualization - Perception for Design. Morgan Kaufmann, San Francisco (2004)Google Scholar
  7. 7.
    Hetzler, E.G., Turner, A.: Analysis Experiences Using Information Visualization. IEEE Computer Graphics and Applications 24(5), 22–26 (2004)CrossRefGoogle Scholar
  8. 8.
    Toker, D., Conati, C., Steichen, B., Carenini, G.: Individual User Characteristics and Information Visualization: Connecting the Dots Through Eye Tracking. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2013, pp. 295–304. ACM, New York (2013)CrossRefGoogle Scholar
  9. 9.
    Bertin, J.: Semiology of Graphics: Diagrams, Networks, Maps. Esri Press, Redlands (2011)Google Scholar
  10. 10.
    Tufte, E.R.: Visual Explanations: Images and Quantities, Evidence and Narrative. Graphics Press, Cheshire (1997)zbMATHGoogle Scholar
  11. 11.
    Viegas, F.B., Wattenberg, M., Van Ham, F., Kriss, J., McKeon, M.: Manyeyes: A site for visualization at internet scale. IEEE Transactions on Visualization and Computer Graphics 13(6), 1121–1128 (2007)CrossRefGoogle Scholar
  12. 12.
    Cavalcante, V.F., Pinhanez, C.S., de Paula, R.A., Andrade, C.S., de Souza, C.R.B.: Data-driven analytical tools for characterization of productivity and service quality issues in IT service factories. Journal of Service Research: JSR (2013)Google Scholar
  13. 13.
    Cooper, A.: The Inmates are Running the Asylum: Why High-Tech Products Drive Us Crazy and How to Restore the Sanity. SAMS, Indianapolis (1999)Google Scholar
  14. 14.
    Sharp, H., Rogers, Y., Preece, J.: Interaction Design: Beyond Human-Computer Interaction, 2nd edn. Wiley, Indianapolis (2007)Google Scholar
  15. 15.
    Ericsson, K.A., Simon, H.: Protocol analysis: Verbal reports as data. MIT Press, Cambridge (1993)Google Scholar
  16. 16.
    Nielsen, J., Macks, R.L.: Usability Inspection Methods (1994)Google Scholar
  17. 17.
    Lawson, B.: How Designers Think: The Design Process Demystified, 4th edn. (2005)Google Scholar
  18. 18.
    Creswell, J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. SAGE Publications (2009)Google Scholar
  19. 19.
    Iliinsky, N.: Choosing visual properties for successful visualizations, (accessed February 03, 2013)
  20. 20.
    Cleveland, W.S., McGill, R.: Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American Statistical Association 79(387), 531–554 (1984)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Heloisa Candello
    • 1
  • Victor Fernandes Cavalcante
    • 1
  • Alan Braz
    • 1
  • Rogério Abreu De Paula
    • 1
  1. 1.IBM ResearchBrazil

Personalised recommendations