One of the common problems associated with measuring the usability of information visualizations is understanding human factors of visual perception and cognitive processing in interacting with dynamic data graphs that are commonly used in social computing applications. In this paper, we investigate the cognitive and perceptual processes in the visual exploration process of information visualizations. Increasing interest in recent years has been focused on the development of performance-based usability metrics, (such as accuracy, speed and visual scanning strategies captured from session logs) to address this problem. However, the growing number of new terminology related to eye tracking metrics have caused considerable confusion to the information visualization community, consequently making the comparison of these metrics and the generalization of empirical results from eye tracking studies of data visualizations increasingly difficult. This paper proposes a framework of eye tracking metrics related to interacting with information visualizations which demonstrate the underlying relationships between human factors in gaze metrics and information visualization design factors. Design implications and issues relating to the investigation of these metrics are also discussed.
- Information visualization
- Human-computer interaction
The field of social computing has evolved in recent years and many technologies emerged in this field to facilitate the communication and interaction with social data . In order to create more efficient mediums to support social computing applications, information visualizations were used to enhance the cognitive process [2, 3]. On the other hand, the growing interest in the application of eye tracking methodology for measuring the usability of information visualizations in recent years can be attributed to the need for objective measures that lead to insights in user interactions with information visualizations, which would be considerably more difficult to uncover with other usability testing methods. For the transfer and creation of knowledge, information visualizations help make abstract concepts accessible to users, help to reduce complexity, amplify cognition, describe and explain causal relationships, segment and structure information. Novel approaches for information visualization are emerging that allow to illustrate higher complexity to embed factors such as social and cultural factors in knowledge-intense processes of interaction. To transfer knowledge, interactive information visualizations often (1) fascinate users whether they are interacting individually or in collaborative settings that are co-located or remotely linked, enabling interactive collaborations across time and space, and (2) facilitate interactions for representing and exploring complex scenarios or creating new insights. Increasing interest in recent years has been focused on measuring the usability of information visualizations and developing objective metrics, such as accuracy, speed and visual scanning strategies captured from session logs, to address this problem. Goldberg  provided an early discussion of applying eye tracking to information visualization. He considered the linear and radial visualizations to address variations in visual attention distribution and comprehension. However, the growing number of new terminology related to eye tracking metrics have caused considerable confusion to the information visualization community, consequently making the comparison of these metrics and the generalization of empirical results from eye tracking studies of data visualizations increasingly difficult. Our research goals are to understand the perceptual and cognitive processes in visual communication of information. Specifically, we proposes a framework of eye tracking metrics related to interacting with information visualizations which can describe the underlying relationships between human factors in gaze metrics and information visualization design factors.
2 Related Work
Kindly assure Interaction design for dynamic information visualizations is a multidisciplinary field. Usability research of information visualizations intersects visual design, human-computer interaction (HCI) studies, cognitive psychology, and computer science. Prior research has often focused on each of the above separately and from a specific point of view . In this section, we provide an overview of those different perspectives. We present an overview of eye tracking studies in information visualizations, followed by the perceptual and cognitive processes that have been found to be relevant to the design of interactive information visualizations. Eye tracking research in knowledge and information visualization often aims to understand the perceptual and cognitive processes in visual communication of information . Specifically, eye tracking methods are applied to investigate which visual elements do people actually pay attention to, when do they pay attention to these elements in the interaction, and for how long [4, 7]. Authors in  examined the visual scanning strategies of users in reading specific types of graphs (radial and line graph). A taxonomy of tasks and abstract data types was presented by Shneiderman in 1996 that was proposed to understand the cognitive processes involved in interacting with information visualization  are acceptable.
3 Exploratory Eye-Tracking Study
Inspired by research methodologies originally developed for analyzing graphic visualizations, we explore eye-tracking methods that measure various static and dynamic aspects of visual perception, and their relations to an underlying cognitive processes. The eye fixations and visual scanning patterns allow us to analyze and quantify what information visualization elements viewers encode, retrieve, and recall from memory. In this exploratory study, we look at the different ways in which we can analyze people’s eye-movements and responses as they interact with information visualizations.
A total of 23 participants (8 females, 15 males) participated in the experimental study. All of the participants were recruited from the local communities in Riyadh, Saudi Arabia with an average age of 28.09 years (SD = 5.18 years). All participants had bachelor’s degree or higher. The participants had normal color vision. Table 1 shows the demographics of participants in the study’s sample.
For the eye-tracking study of our experiment, we used a Tobii X120 stand-alone eye-tracking system with participants seated approximately 60 cm from a 23 in. monitor (resolution: 1920 × 1080 pixels). Tobii studio V 3.4 was used for recording and analysis. Sessions were conducted in an office setting where noise was reduced compared to the usual setting within the space. The process of establishing the framework of mapping gaze metrics involved merging taxonomy with the taxonomy of gaze metrics. Table 1 describes the metrics.
A single session of the experiment took between 15 to 20 min for each participant. At the beginning of an experimental session, participants were introduced to the session by giving them an overview of what eye-tracking is and the nature of the tasks they will be performing. When the session starts for each user, we conducted a randomized 5-point calibration procedure to establish the 3D coordinates of the eyes. At regular intervals, a drift check was performed and, if necessary, recalibration took place, and optional breaks were offered to participants. This exploratory study involved two sections; the first one prompts the user to perform informational tasks in which there were asked a set of questions and were asked to look for the answer using the stimuli they were exposed to. The second one involved a navigational task in which the users were asked to explore a webpage from the visualization platform.
For this exploratory study we used the data visualizations from the Observatory of Economic Complexity by MIT. The informational task exposed the user with two visualization patterns; stacked graph and a treemap. The theme of the informational questions was selected to test the findability and comparability of the visualization pattern. And to insure consistency and in the same time reduce the effect of learnability, the same set of questions were asked for each visualization pattern but for a different country. For the navigational task, the users were prompted by looking and using the mouse, and then asked to give their feedback at the end of the session as shown. Figure 1 illustrates the set of questions that were asked to the user.
Two research methods were applied in this study; observation and retrospective think aloud (RTA). The study was audio recorded to capture users’ comments as they performed the tasks. At the end of the session, we wrapped up by replaying the recorded eye tracking session to the user and asked the participant to explain if there were some clear patterns or unusual behavior using RTA (Figs. 2 and 3).
The process of establishing the framework of mapping gaze metrics involved merging taxonomy of gaze metrics in  with task by data type taxonomy in . The table in Fig. 4 describes the framework we are proposing of eye tracking metrics related to interacting with information visualizations.
4.1 Visual Exploration Patterns in Information Visualization
For this study we analyzed the eye movements of 23 participants examining visualizations to determine which graphic elements of a data visualization attract people’s attention, and patterns of visual exploration. The mapping of gaze metrics to design factors was verified by reflecting on the experimental results of our exploratory study. While the proposed framework for eye tracking research (described in the Framework section), can empirically support conventional design guidelines for information visualization, patterns in visual exploration that are aligned with the shape, form and function of data visualizations can provide a taxonomy for visual orientation and exploration for data visualization.
The findings demonstrate that there is a marked difference between the fixation patterns and their corresponding spatial distribution in different visualizations as expected; however, our understanding of which visual elements in the information visualization contribute to this difference in fixation patterns, and which human factors help explain the usability issues or the overall user experience is inadequate.
These are areas for further investigation that can be guided by the proposed framework described in the Framework section.
More illustrative examples include mean fixation duration of gaze within elements in an information visualization and across different information visualizations. In examining the mean fixation duration (i.e. the average duration a user fixated inside a specific area of the information visualization), we assume the intended target in visualization would contain the gist of information the user would need to direct his/her attention and process. This information, being the target, would be the hardest to comprehend, or the most interesting in the visualization explains the anticipated high mean fixation duration, which is proportional to said factors. A competing target would as well contain important information, yet less important than that contained in the target. The mean fixation duration over this area is therefore anticipated to be shorter. The mean fixation duration over the remaining areas of the visualization is anticipated to be shortest of the three (Fig. 5).
Our measured mean fixation duration over all users reflects the anticipated order of duration lengths. As per the discussion in the Exploratory Eye Tracking Study section, the users were presented with two different visualization patterns and were asked to find the largest of two values. In this case, the target area becomes the area in the visualization that encodes information about the correct answer, or the item with largest value. Likewise the competing target is the area where the user would find information about the item with smallest value. Across the two different visualization patterns that we presented to our participants, mean fixation durations were consistent with our predictions. The results of this experiment shows that mean fixation duration is sensitive and thus a candidate to measure cognitive processing or mental effort spent during the exposure to information visualizations (Fig. 6).
Another example is aggregate visual attention that were captured in heat maps. Animated gaze heat maps show users’ aggregate fixation locations over time. In this section, we are to elaborate on two interesting segments of the animated heat maps: orientation phase, and convergence phase. In the orientation phase (i.e. the first few seconds of users exposure to the visualization when they search for elements to help them identify key information such as units, scale, and search for their target), the animated heat maps of this phase showed an interesting distribution of gaze areas (Fig. 7).
In the treemap pattern, most gaze areas correspond to labels possibly indicating that users were searching for their target. In the convergence phase, the treemap visualization pattern presented to our participants showed gravitation of eye gaze towards the targets, and the competing target. More specifically, users aggregate eye gaze accumulated at labels of the target and the competing target (Tables 2 and 3).
Similar patterns appeared in the stacks visualization pattern. Users’ aggregate eye gaze were gravitated towards targets. However, in this case, visual elements that encode information users were expected to compare had low color contrast. In this heat map, eye gazes were attracted to labels of actual target and competing target, and as well attracted to a label of another area similar in visual properties (similar background color) with target and competing target, and attracted to the separating line between the three areas.
In this paper, we have introduced a framework for incorporating eye tracking in the usability/ux toolkit for information visualization. An exploratory user study has shown that designers and analysts can use the framework to map metrics from eye tracking research to the process of usability and user experience evaluations of dynamic information visualizations. Based on the eye tracking results presented in the preceding sections, we summarize in this section the key observations from our study. This work presents an outline of mapping eye-tracking metrics to usability metrics for investigating interactive information visualizations. The emphasis was on the dimensions of design factors, human factors of perception and cognition, and task. In addition, we characterize visualization design considerations that are related to individual and aggregate gaze metrics, including fixations, saccades and scan paths. Based on insights from our exploratory eye tracking study and previous research, we are able to offer framework to support usability evaluations based on existing conventional visualization design guidelines.
Erickson, T.: Designing visualizations of social activity: six claims. In: Extended Abstracts on Human Factors in Computing Systems, CHI 2003. ACM (2003)
Chen, C.: Information visualization. Wiley Interdisc. Rev. Comput. Stat. 2(4), 387–403 (2010)
Wang, F.-Y., et al.: Social computing: from social informatics to social intelligence. IEEE Intell. Syst. 22(2), 79–83 (2007)
Goldberg, J.H., Helfman, J.I.: Comparing information graphics: a critical look at eye shop: beyond time and errors: novel evaluation methods for information visualization, pp. 71–78. ACM (2010)
Card, S.K., Mackinlay, J.D., Shneiderman, B.: Information Visualization: Using Vision to Think. Morgan Kaufmann, San Francisco (1999)
Keller, T., Tergan, S.-O.: Knowledge and Information Visualization. Springer, New York (2005)
Goldberg, J., Helfman, J.: Eye tracking for visualization evaluation: reading values on linear versus radial graphs. Inf. Vis. (2011). doi:10.1177/1473871611406623
Shneiderman, B.: The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of IEEE Symposium on Visual Languages, 1996, pp. 336–343. IEEE (1996)
Ehmke, C., Wilson, S.: Identifying web usability problems from eye-tracking data. In: Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI… but not as we know it, vol. 1, pp. 119–128. British Computer Society (2007)
This research was supported by the King Abdulaziz City for Science and Technology (KACST) as part of the research under the Center for Complex Engineering Systems (CCES). We thank our colleagues in CCES who provided insight and expertise that greatly assisted the research, and special thanks for the director of the center, Anas Alfaris.
Editors and Affiliations
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Almahmoud, J., Albeaik, S., Alrashed, T., Almalki, A. (2017). Visual Exploration Patterns in Information Visualizations: Insights from Eye Tracking. In: Meiselwitz, G. (eds) Social Computing and Social Media. Applications and Analytics. SCSM 2017. Lecture Notes in Computer Science(), vol 10283. Springer, Cham. https://doi.org/10.1007/978-3-319-58562-8_27
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58561-1
Online ISBN: 978-3-319-58562-8