Skip to main content

Biometrics and Business Information Visualization: Research Review, Agenda and Opportunities

Part of the Lecture Notes in Computer Science book series (LNISA,volume 10923)

Abstract

Business Intelligence & Analytics’ dashboards, data visualizations, and visual analysis systems are cognitive tools that heavily rely on our understanding of subconscious-level processing, human perception and cognitive processes overall. These cognitive tools are the focal point of data visualization subfield called Business Information Visualization (BIV). While the BIV field has seen calls for an increased need of deploying physiological sensing techniques and the use of biometric data, these calls have yet to translate to a more comprehensive evaluation of human response to visual displays and systems using business data. This research identifies and describes biometric sensors researchers should consider in BIV, provides a brief review of literature at the nexus of data visualization and biometrics, and identifies a biometric data-based research agenda and opportunities for BIV.

Keywords

  • Business Information Visualization
  • Information visualization
  • Biometrics
  • Physiological sensing
  • Eye tracking
  • EEG
  • Galvanic skin response
  • GSR
  • fNIRS
  • Facial muscle movement
  • FACS

1 Introduction

Business Intelligence & Analytics’ (BI&A) dashboards, data visualizations, and visual analysis systems are cognitive tools that heavily rely on our understanding of subconscious-level processing, human perception and cognitive processes overall. These tools are the focal point of data visualization subfield called Business Information Visualization (BIV) or the use of computer-supported, interactive, visual representation of business data to amplify cognition [1] to achieve a better understanding of business (processes, data, and behaviors) to improve decision making [2]. While the literature suggests the role of cognitive processes in BIV [2, 3] and visualizations in general [4], research attempting to measure the physiological user response as correlates of those cognitive processes is scarce. The scarcity of BIV users’ physiological response research is surprising given findings from psychology research that evaluating visual interface performance without measuring cognitive processes, such as workload, may lead to incorrect conclusions about the cognitive efficiency of an interface [5,6,7]. For a field that declares to focus on cognition amplification, memory limits, preattentive attributes, and Gestalt psychology [8], the lack of more pronounced research inclusion of physiological measurements of a user response to visual capabilities of BI&A systems and processes represents a gap and an opportunity. In this research, I offer a starting path to close the gap and in the process, provide the following contributions: (i) identify and describe biometric sensors researchers should consider in BIV, (ii) provide a brief review of literature at the nexus of data visualization and biometrics, and (iii) identify a biometric-based research agenda and opportunities for BIV.

Organizations continue to make large investments in BI&A or “the techniques, technologies, systems, practices, methodologies, and applications that analyze critical business data to help an enterprise better understand its business and market and make timely business decisions” [9]. The visual layer and employee-facing interface of BI&A is often made available and delivered through visually intensive applications such as dashboards and other types of highly visual reporting and decision-supporting solutions that require insights from BIV, a field that is informed by and find its academic roots in interlinked subfields that literature labeled as Graphical display, Data Visualization, Information Visualization, and Visual Analytics. Despite BIV’s significant impact and use within organizations, research focused on users’ physiological response to BIV as a way of understanding users’ and decision makers’ perceptual and cognitive processes is still underdeveloped. The lack of progress is not limited to research only as “chartjunk” [8] design practices prevail in practice and are often enabled by both vendors and dashboard designers. Unfortunately, the visual layer of BI&A software too often gets in the way, interrupting and undermining the thinking process rather than complementing and extending it [10]. The frequent inappropriate use of information presentation formats [11] may lead to suboptimal decisions [12]. Further amplifying the practical significance of this research is the reality of users being asked to make decisions in the age of information overload, where the role of systems supporting visual analysis and data representation that filter and separate signal from noise is critical.

In the wider visualization field, others recognized the gap and offered an overview of available physiological sensing technology in evaluating information visualization systems [13], suggested the use of cognitive measures in related scientific visualization [14], evaluated research opportunities of using specific biometric sensor (eye tracking) [15], and made a call for more Information Systems research in BIV using biometrics [2]. This manuscript builds and extend their call by (i) documenting some of the progress in the last decade and (ii) suggesting BIV-system centric and user abilities centric lens that would benefit from applying those physiological sensing technologies.

The rest of the paper is organized as follows. Section 2 provides a brief historical and research summary of Business Information Visualization and introduces its critical elements. Section 3 provides an overview of biometric sensors that can provide value to BIV research and documents key literature at the nexus of broader visualization field and biometrics. Recommendations for future research design based on literature review are offered as well. Section 4 discusses the research agenda through the lens of critical BIV elements. The paper ends with concluding remarks in Sect. 5.

2 Business Information Visualization - Background

While the term Information Visualization has been coined in 1999 [1], it is important to recognize that the history of the field dates back long time before computers and information technology platforms. It was not until the 17th century and Renee Descartes [16] that two dimensional visual grids were first used purely to represent numbers. Through works of Heinrich Lambert and William Playfair (early 1800s), graphical design was at last no longer dependent on direct analogy to the physical world [17]. John Snow’s famous cholera outbreak infographic and Minard’s infographic showing Napoleon’s march to Moscow and the consequent retreat represent some of the best visualization works of that era and beyond [8]. The innovative data visualization research remained effectively dormant in the first half of the 20th century, until a ‘perfect storm’ occurred with a call for recognition of data analysis as a separate discipline [18], the birth of computer technology and Bertin’s [19] attempt to classify all graphic marks as expression of data. In 1970’s, Management Information Systems’ researchers are starting to explore presentation formats [20, 21] while statistical and quantitative themes continued in the 80s with Cleveland [22] and Tufte [8]. The cognitive perspective of information visualization came to the forefront in the same period with works by Kosslyn [23] and Tufte [24]. Some of the early notable Information Systems (IS) academic papers occurred in the same period [25,26,27] with emphasis on the analysis of computer graphics’ implications on decision making. This research led to the introduction of the cognitive fit theory [3, 28], which attempts to explain the appropriateness of emphasizing the importance of task-representation match and resulting cognitive effort. With the emergence of Information Visualization (IV) and Business Intelligence (BI), a new discipline called Business Information Visualization came to life in the last 20 years, drawing from historical experiences, events and disciplines such as the ones described above.

Most of the existing IV and BIV research adopted the lens of information representation and interaction [29]. Information representation, or spatial representations that are derived from symbolic data [1], has been researched extensively. Historically, a large part of it centered on understanding the significance of representation formats (histograms, tables, bar charts, bullet graphs, and other formats) and layout/position. More recent research is exploring the implications of interactivity, exploration, storytelling, and user & data characteristics. Bačić and Fadlalla [30] revealed an essential link between visualization and decision making support through the emphasis of human visual abilities and suggested the need for BI’s visualization components to assist human intelligence. They provided an organizing framework [2] identifying critical BIV elements aligned with five nonverbal (visual) mental abilities: interaction, exploration, business acumen, relevant data, analytics, statistics, representation, perception, cognition, cognitive effort, memory & storytelling (Table 1).

Table 1. Business information visualization elements

The next section provides a brief overview of biometric sensors researchers should consider deploying in the BIV context, followed by a review of existing research at the nexus of biometrics and BIV, and the larger data visualization field.

3 Biometrics and BIV Background

The understanding of user physiological response to a BI&A system, specifically its presentation layer is still in its infancy. While there are remaining significant challenges in interpreting physiological data, measurement validity concerns [13], and the issues stemming from BIV’s complex and interactive setting, the experiments capturing user physiological response are becoming more accessible and insightful. Several human physiological responses may be measured effectively in experimental, yet realistic setting using biometric equipment and techniques. A select few particularly promising in the BIV context are eye gaze and pupil dilation (visual attention and effort), galvanic skin response (arousal), facial muscle movement (affective states and emotional engagement), and brain activity (brain workload and cognitive engagement).

3.1 Biometrics Overview

Driven in part by the ‘eye-mind hypothesis’ [31], eye tracking is considered effective in assessing user’s attention and effort as it reveals how the user reads and scans the displayed information by capturing users’ eye fixation, saccades, blink and pupil dilation-based metrics [32]. Fixations [33] (time spent looking at a specific location), saccades (the rapid eye movements between fixations) and pupil dilation [34, 35] responses have been linked to cognitive processes indicating various forms of attention, interest, mental effort, and cognitive load [36, 37]. Eye tracking-based metrics have been used across many disciplines measuring attention in reading, psycholinguistics, website usage, online gaming, writing, and language acquisition [38]. The standard metrics can be separated into three categories: fixation-derived metrics (fixation count, duration, time to first fixation), saccade-derived metrics, and scan path-derived metrics (fixation and saccades sequence, area-of-interest (AOI) revisits) [15]. Despite a very long track record of using eye tracking across numerous fields, the eye tracking-based research, while emergent and the most common biometric technique in visualization [13], is still rare in BIV or represents a relatively small portion of system evaluation in broader data visualization field [15].

Galvanic skin response (GSR), also referred to as electrodermal activity (EDA), Skin Conductance (SC) or Psychogalvanic Reflex (PGR), is a measure of conductivity of human skin and measures sympathetic nervous system response as an indicator of general arousal. As participant’s effort, engagement, excitement, or anxiety level changes, the sympathetic nervous system reacts by releasing small amounts of moisture (sweat secretion) in the skin (sweat glands). Highly sensitive GSR devices capture skin conductance using skin electrodes and researchers deploy them in detecting the arousal level of emotions [39, 40], stress [40, 41], and cognitive load [42]. The use of GSR-based biometric data is currently minimal in the evaluation of BIV systems.

Biometrics based on the facial muscle movement offers insights into users’ affective states, more specifically, their emotional valance. Facial Action Coding System (FACS), a method designed to help classify human facial movements by their appearance on the face [43, 44] is used by facial expression technology vendors to identify and understand not only facial units but also combine them to capture emotional valence (positive, negative) and basic emotions (joy, disgust, contempt, confusion, frustration, surprise, anger) [38]. Simultaneous use of GSR and automated FACS is a particularly effective way of understanding both the valance and the arousal associated with an emotional response. The use of facial muscle movement offers promising potential in BIV, especially in assessing system’s usability (valance) and aesthetics.

While there are several ways to capture biometric data based on brain activity, electroencephalograms (EEG) and functional near-infrared spectroscopy (fNIRS) are particularly useful to BIV as they are very effective and non-invasive methodsFootnote 1. These devices detect electrical activity in the brain through the placement of electrodes on the head. EEG devices detect fields (waves) of electrical activity or neuronal oscillations of neurons firing around our brains. These waves provide a signal in the form of waves at different frequencies: delta (<3.5 Hz), theta (4–7.5 Hz), alpha (8–13 Hz), and beta (14–18 Hz) frequency bands. Researchers found that a change in specific waves correlates to several cognitive processes [45]. EEG has been proven effective in detecting alertness, cognitive workload [46, 47], cognitive overload [48], insight generation [49], memory and new information encoding [45], various levels of engagement/boredom [47], approach/avoidance motivation [50], and emotions [51]. Although relatively non-invasive, high in temporal resolution and low cost, EEG sensors can take a long time to set-up, have low spatial resolution and are sensitive to interferences. fNIRS can overcome some of those limitations as they have high spatial resolution and are less sensitive to interferences (cite) [52]. fNIRS detect the brain activity by using near-infrared light to measure levels and oxygenation of the blood in the tissue at depths of 1–3 cm. Like EEG, fNIRS was used to detect cognitive workload [7], a potentially useful metric to assess BIV efficiency and effectiveness.

3.2 Biometrics and BIV

The review of literature focused on information visualization in the business (i.e. BIV) context revealed only three research efforts [38, 53, 54] focused on using or integrating biometrics. However, since BIV is firmly rooted in IV field and the distinction between the two is not always clear or useful, this manuscript provides a brief literature summary of the research at the nexus of biometrics and broader visualization field, focused on manuscripts with findings more directly applicable to the business context.

Eye Tracking.

Eye tracking, although underutilized [55], is the dominant physiological sensing technology used to evaluate data visualizations. In the context of bar and radial chart, eye tracking was used to evaluate the influence of individuals’ perceptual speed and working memory on gaze behavior [56]. Eye tracking was applied to evaluate simple information graphics, such as bar charts & line graphs [57, 58] and tables using various sorting techniques [59, 60]. Related eye tracking study partially showed that the decision quality difference came from the changes in decision strategies while noting incapability of capturing peripheral vision as a limitation of the eye tracking method [61]. Another study used eye tracking to study how parallel coordinate visualizations are perceived, and compared the results to the optimal visual scan path required to complete the tasks [62]. While not traditionally used in business dashboards and other visual displays, node-link diagrams are becoming more common in analyzing networks in the business context as well. The research focused on understanding sequential characteristics of eye movement in node-link diagrams is robust [63,64,65,66,67,68,69] and should be considered when deploying in the BIV context.

Several studied using eye tracking focused on understanding the process of graph comprehension. A comparison of linear and radial charts found that tasks requiring value lookup on one or two dimensions are more efficiently achieved using linear graphs. In the process, the same study found evidence of a three-stage processing model leveraged by visualization readers (i) find desired data dimension, (ii) find its data point, and (iii) map the data point to its value [55]. On other hand, suggestions were made that graph comprehension proceeds in distinct stages [70] and in sequential order; readers interpret graphs by first focusing on the nodes and then by processing the lines between the nodes [71]. Others noted the complexity of graph comprehension and reported that users, instead of spending time on patterns, were spending time mostly relating graph lines to referents/data legends [72] or performing encoding-recoding strategies to compensate for memory limitations [58].

In a context of dashboards, research shows that visualizations with misuse and overuse of colors do not lead to poorer decision performance but rather decision makers using such dashboards experience more cognitive effort and take longer to make a decision [53]. Eye tracking was also used to assess several conventional qualitative dashboard design guidelines, including the role of title and the supporting text, the use of pictograms and their role in recognition, how redundancy understanding can improve recognition, and how redundancy and memorability help to effectively communicate the message [54]. The role of information scents and context clues was assessed in Hyperbolic Tree display format [73], while the efficiency or effectiveness of the two highlighting methods in the context of supporting the visual search for information in linked geo-displays [74].

The use of eye tracking in combination with interaction logs, and thinking-aloud protocols in the context of interactive visualizations and insight creation was proposed as valuable [75]. Related empirical research dealing with challenges of insight creation in visual analytics reports that think-aloud insight statement are accompanied by peaks in the pupil diameter, usually followed by a sharp drop while fixation duration are good indicators of higher cognitive effort with some slight delay in comparison with the pupil diameter [36]. Pupillary dilations were used to compare cognitive efforts across auditory and visual presentation tasks, suggesting that when that visual task presentation leads to lower cognitive load [37].

There is only study that attempted to suggest areas in which eye tracking could be useful in evaluating visual display of data and visual analytics and identify open research challenges [15]. The same study provided an overview of how eye tracking is currently used in the evaluation of visualization techniques, how cognitive models could be applied in our use of eye tracking for visual analytics, and provided a list of additional research efforts linking eye tracking with visual analytics.

Other Sensors (GSR, EEG and fNIRs).

Despite calls for more use of biometric sensors in the context of data visualization [38, 76], the use of sensors other than eye trackers is limited. No relevant research was identified attempting to assess emotional engagement and valance through facial muscle movement and FACS. Similarly, only a small study [5] using GSR (and heart rate) attempted to test the correlates of Bertin’s [19] efficiency ranking of visual variables. Five selected visual characterizations (angle, text, surface, framed rectangles and luminosity) were evaluated in an Air Traffic Control setting but the study failed to confirm the ranking. Only in the case of angle and text visual variables the difference in galvanic skin response was detected. Due to small sample, we should be very cautious in interpreting these results.

Although the use of relatively less invasive brain imaging techniques such as EEG and fNIRS in broader HCI field is not uncommon, their use in a narrower data visualization filed is still in infancy. However, promising findings from cognitive science found a unique pattern in brain activity that corresponds with the unique sensation of the “a-ha moment” [49], which was supported in a study using ManyEyes map-based visualizations where a connection between insight and frustration and excitement was detected [77]. Beyond insight generation response measurement, EEG-generated alpha and theta signals were analyzed to assess cognitive resource strain on users when using different methods of visualizing distribution data [78].

Researchers tested the possibility of measuring the impact of data representation using fNIRS [6] and confirmed the promising potential of fNIRS technology. Study participants reacted differently to pie charts and bar graphs at a cognitive level, but exhibited the same performance characteristics, raising further need to understand the impact of visual displays beyond traditional performance metrics (speed and accuracy).

Literature Review-Based Insights.

This brief literature review provides several insights and recommendations regarding the use of biometric sensors in BIV. First, there is a need to start testing and implementing findings from broader HCI and visualization fields into BIV. However, as the research moves into business context, it is critical for experiments to use very realistic business tasks, problems, scenarios and systems. Second, multidisciplinary approaches and collaborations between domain experts (business, HCI, design, psychology, neuroscience, information systems, computer science, and other contextually relevant domains) is important for appropriate experimental design, data analysis and the interpretation of the findings. More cross-discipline and collaborative research is needed. Third, eye tracking sensing technology dominates physiological research in the context of this manuscript. Given the evidence across other biometric technologies as correlates of affective, perceptual, and cognitive processes, there is a need and an opportunity to encourage research using facial muscle movement, galvanic skin response and brain imaging. Fourth, we should encourage and support research that combines multi-sensor research design. Affective, perceptual and cognitive processes are complex, task and subject dependent; using only one biometric technology in an experiment may not reveal critical insights when dealing with such complex processes. Fifth, biometric research is not the ‘silver bullet’. We should encourage research that combines simultaneous biometric, survey and behavioral data collection.

4 Business Information Visualization – Research Agenda

The framework introduced in Sect. 2 suggests that BIV, to be effective, needs to support and accentuate relevant human intelligence dimensions. This need requires software that seamlessly interacts with the brain to support and extend human cognitive and perceptual abilities [2]. Given framework’s (a) explicit recognition of human cognitive and perceptual response, (b) comprehensiveness rooted in existing visualization literature, and (c) identification of critical components, this research will adopt Bacic and Fadlalla’s framework [2] and its BIV elements as a lens through which BIV & Biometrics agenda and research opportunities will be presented.

4.1 Fluid Intelligence - Data Exploration and Interaction

Fluid reasoning/intelligence allows humans to solve new problems on the ‘spot’ through mental operations such as drawing inferences, concept formation, classification, generating and testing hypothesis, identifying relations, comprehending implications, problem-solving, extrapolating, transforming information, and recognizing patterns [79, 80]. In BIV literature, system capabilities of exploration and interaction with data emerged and are well positioned to support users’ fluid reasoning. Exploration is defined as the examination of data without having an apriori understanding of what patterns, information, or knowledge it might contain [81]. Some of the common exploratory tasks include: observing specific data point, patterns or outliers, making inferences, comparing to one’s prior knowledge, generating hypotheses and drawing analogies [82].

Interaction capabilities often facilitate exploration capabilities. The goal of interaction is to enable a user to understand information better by allowing the user to interact with the information. The extant literature offers several interaction taxonomies: low level interaction tasks (overview, zoom, filter, details-on-demand, relate, history, and extract) [83], user’s intent (select, explore, reconfigure, encode, abstract/elaborate, filter, and connect) [84], and model-based reasoning (external anchoring, information forging and cognitive offloading) [85].

Mental operations performed by users during exploration and interaction are critical to sense making and insight creation. These mental operations require well designed interfaces to support users’ visual perception and cognitive processes. Research focused on understanding the physiological response and biometric correlates to affective, perceptual and cognitive processes linked to data exploration and interaction design and deployment can significantly enhance existing knowledge and represents a promising research opportunity. Furthermore, without access to information one cannot explore nor interact with it. BIV is being delivered across multiple platforms including PDAs/smartphones, tablets, and traditional PCs. Multi-platform physiological sensing and biometric research in the context of fluid reasoning represents an opportunity for practical insights.

4.2 Domain-Specific Knowledge - Business Acumen and Data Relevancy

Domain specific knowledge has been defined as individual’s breadth and depth of acquired knowledge in specialized (demarcated) domains [80] and is modifiable “software” aspect of the cognitive system [86]. A viewpoint referred to as knowledge-is-power hypothesis is being described as “one of the most influential ideas to emerge in cognitive psychology during the past 25 years and is based on the idea that domain knowledge, and not basic/global cognitive abilities, is the main determinant of success in cognition related tasks” [86].

Literature focused on visualization is yet to approach the topic of knowledge and business acumen in similar depth as in the fields of psychology and cognition. Since it is widely accepted that a significant level of domain knowledge is needed to achieve expertise, BIV usage or deployment by users across various levels of expertize and business acumen is an underdeveloped research area independent of research methodology, let alone in biometric-based research. Similarly, the effectiveness of BIV, even if efficiently deployed in every other aspect, will be eliminated if deployed using inappropriate data. Understanding how experts comprehend, view, process, explore, interact, achieve insights, select appropriate data and eliminate noise when dealing with BIV applications are only some of the research questions biometric-inclusive research should explore. Biometric data using physiological sensors discussed in this research can provide valuable answers focused on the role of domain knowledge, expertize and business acumen in the BIV context.

4.3 Quantitative Reasoning - Analytics and Statistics

The research defines Quantitative Reasoning as a person’s wealth (breadth and depth) of acquired store of declarative and procedural quantitative knowledge [80]. In BIV, a system can enhance decision making through Quantitative Reasoning by offering ready-to-use, statistical functions and analytics capabilities. Examples include, on the fly summary statistics such as averages, medians, standard deviations, percentiles as well as more complicated algorithms, calculations, and data mining techniques such as regressions, clustering, and association analysis [2]. With the advent of more advanced analytics and its popularity in the context of Big Data, BI&A users are starting to expect higher integration of analytics capabilities and visual technologies as well. Analytics capabilities and the representation methods once reserved for individuals strong in declarative, procedural quantitative knowledge, and acquired mathematical knowledge (statisticians, modelers and data-scientists), such as box-plots, scatterplots, node-link graphs, decision trees, and tabular representations of statistical data, are increasingly being directed at business analysts [2].

Despite knowing very little how users are responding to these new capabilities, BI&A vendors are starting to incorporate significant statistical and analytics capabilities into their interfaces. Often, the new capabilities are complex and low in transparency, with a potential for users to face challenges, stress, frustration, reach insights and different levels of cognitive load. We know very little about users’ neurophysiological responses to these BIV applications especially in the light of varying degrees of expertise in analytics, statistics or how to visually communicate complex results. The trend of adding more advanced capabilities and complex visual representations will most likely continue due to ‘fashion-like’ popularity of everything labeled ‘analytics’ or ‘Big Data’ and the research focused on understanding user response using biometric data offers relevancy and practical value.

4.4 Visual-Spatial Processing - Representation, Perception, Cognition, and Cognitive Effort

Visual Spatial Processing abilities are defined as having the ability to generate, retain, retrieve, and transform well-structured visual images [87]. A significant majority of visualization literature is focused on how to enable visual-spatial processing abilities through research focused on data representation (formats and features), and how visualization choices both impact and are impacted by human perception, cognition, and cognitive effort [2].

Information representation has been researched extensively and a large part of it centered on understanding the significance of representation format (tables vs graphs or comparison between similar versions of the same display type). In addition to representation methods, researchers created a significant body of knowledge around representation elements such as color, object depth and dimensionality and layout, symbols, labels, text, icons, lines, grids, and axes [88]. The design of those elements is often informed by our knowledge of human perception or the process of interpreting and recognizing sensory information [89]. Studies of human perceptual ability including visual imagery, cognitive fit, gestalt principles and preattentive attributes, have led to many design principles [8]. The assessment of the physiological user response to representation method choices, design of representation elements (some call that ‘presentation’), the role of the gestalt principles and preattentive attributes, and the evaluation of numerous design principles advocated in research and practice community represent currently underdeveloped research area that would benefit from validating or challenging current practices in BIV.

Within greater discussion of cognition, a robust Information Systems’ visualization literature stream emerged with focus on the role of cognitive effort. Over the last 25 years, the Cognitive Fit Theory [3, 28], a dominant theoretical lens, has been used in empirical studies to suggest a significant role of cognitive effort in user efficiency and effectiveness when dealing with data representations. Yet, BIV literature is limited in capturing, measuring or discussing cognitive effort directly [2, 30]. These gaps should be addressed in future by adopting the available correlates of cognitive effort, both perceptual and physiological, such as those advocated through physiological sensors (for example, eye tracker (fixation duration, pupil dilation) and EEG (cognitive workload) or fNIRS (cognitive load)).

4.5 Working and Short-Term Memory – Memory and Storytelling

Lastly, visual designs that minimize memory limitations of human brains have been discussed in visualization literature along with more recent research on storytelling [2]. The importance of memory and the efficient use of memory when visually presenting and processing information is widely acknowledged [90] and the use of design principles leveraging memory is well documented in literature evaluating representation methods [23]. The issue of limited amount of information storable in short term memory is central to many design constraints. Consequently a way to increase the amount of information in short-term memory called “chunking” was proposed [24]. Similarly, the appropriate choice of colors [26] and symbols [19] is often executed in consultation with the relevant memory and cognition-based literature.

Most recently, the relevant literature is beginning to recognize the vital role of storytelling and narrative play. This development is partially rooted in the recognition that report designers are not always decision makers, necessitating well organized and captivating communication [91]. The convergence of computer technology, art and media is now allowing for various storytelling techniques to be deployed in business context as well [92]. These techniques include: building the picture, using comics metaphor, animating, setting mood and place in time, conflict and ambiguity resolution, intentional omission, continuity, effective redundancy, and increasing attention [2]. In the context of BI&A and dashboards, a set of requirements for enhancing BI analysis was proposed consisting of fluid transition, integration, narrative visual aids, interactive visualization, appropriate BI story templates, reuse, and option playback [93].

Both the design principles based on users’ memory limitations and the storytelling approaches can benefit from understanding comprehension, recall, insight generation, truth-telling, user affective states and the overall decision-making implications. The physiological sensing techniques and the biometric data discussed in this research can offer valuable insights into this dimension (memory) of enhancing human visual intelligence abilities in the BI&A context.

5 Conclusion

While the BIV field has seen calls for an increased need for the use of physiological sensing techniques and the resulting biometric data, these calls have yet to translate to a more extensive evaluation of human response to visual displays and systems using business data. To initiate the process of addressing the gap, this research offers the following three contributions to research and practice. First, this research identified and described human physiological responses that may be measured in experimental setting using biometric equipment and techniques in the BIV context; eye gaze and pupil dilation (visual attention and effort), galvanic skin response (arousal), facial muscle movement (affective states and emotional engagement), and brain activity (brain workload and cognitive engagement). Second, it provided a brief review of literature at the nexus of data visualization and biometrics leading to five recommendations concerning the more effective use of biometric sensors in BIV. Third, this research identified six focus areas for a more robust BIV research agenda.

In conclusion, current BIV research is manly focused on gathering behavioral and self-reported data. Despite BIV’s reliance on the need to understand subconscious-level processing, human perception and cognitive processes overall; the use of biometric data is mainly absent in the BIV. The current state-of-the-research represents a considerable opportunity for both a theoretical and practical impact. This manuscript provides an initial roadmap of how and in which context BIV research should employ physiological sensors and the resulting biometric data.

Notes

  1. 1.

    Other brain imaging techniques have been used in visualization research and can be very useful (for example, functional magnetic resonance (fMRI) and positron-emission tomography (PET)) but are excluded from this discussion due to their invasive nature and limited or no ability to test the subjects in BIV-realistic setting.

References

  1. Card, S., Mackinlay, J., Shneiderman, B.: Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann, ‎Burlington (1999)

    Google Scholar 

  2. Bačić, D., Fadlalla, A.: Business information visualization intellectual contributions: an integrative framework of visualization capabilities and dimensions of visual intelligence. Decis. Support Syst. 89, 77–86 (2016)

    CrossRef  Google Scholar 

  3. Vessey, I.: Cognitive fit: a theory-based analysis of the graphs versus tables literature. Decis. Sci. 22, 219–240 (1991)

    CrossRef  Google Scholar 

  4. Peck, E.M., Yuksel, B.F., Harrison, L., Ottley, A., Chang, R.: Towards a 3-dimensional model of individual cognitive differences: position paper. In: Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization, pp. 6:1–6:6. ACM, New York (2012)

    Google Scholar 

  5. Causse, M., Hurter, C.: The physiological user’s response as a clue to assess visual variables effectiveness. In: Kurosu, M. (ed.) HCD 2009. LNCS, vol. 5619, pp. 167–176. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02806-9_20

    CrossRef  Google Scholar 

  6. RS brain sensing to evaluate information visualization interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 473–482. ACM, New York (2013)

    Google Scholar 

  7. Hirshfield, L.M., Solovey, E.T., Girouard, A., Kebinger, J., Jacob, R.J.K., Sassaroli, A., Fantini, S.: Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2185–2194. ACM, New York (2009)

    Google Scholar 

  8. Tufte, E.: The Visual Display of Quantitative Information (1983)

    Google Scholar 

  9. Chen, H., Chiang, R.H.L., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. 36, 1165–1188 (2012)

    CrossRef  Google Scholar 

  10. Few, S.: BizViz: The Power of Visual Business Intelligence (2006). http://www.b-eye-network.com/view/2470

  11. Tractinsky, N., Meyer, J.: Chartjunk or goldgraph? effects of persenataion objectives and content desirability on information presentation: effects of presentation objectives and content desirability on information presentation. MIS Q. 23, 397–420 (1999)

    CrossRef  Google Scholar 

  12. Amer, T.S., Ravindran, S.: The effect of visual illusions on the graphical display of information. J. Inf. Syst. 24, 23–42 (2008)

    Google Scholar 

  13. Riche, N.H.: Beyond system logging : human logging for evaluating information visualization. In: 2009 CHI (2009)

    Google Scholar 

  14. Anderson, E.W.: Evaluating visualization using cognitive measures. In: Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization, pp. 5:1–5:4. ACM, New York (2012)

    Google Scholar 

  15. Kurzhals, K., Fisher, B., Burch, M., Weiskopf, D.: Eye tracking evaluation of visual analytics. Inf. Vis. 15, 340–358 (2016)

    CrossRef  Google Scholar 

  16. Descartes, R.: La Geometrie (1637)

    Google Scholar 

  17. Tufte, E.: Visual Explanations. Graphics Press, Cheshire (1997)

    MATH  Google Scholar 

  18. Tukey, J.W.: The future of data analysis. Ann. Math. Stat. 33, 1–67 (1962)

    MathSciNet  CrossRef  Google Scholar 

  19. Bertin, J.: Graphics and Graphic Information Processing. deGruyter Press, Berlin (1981)

    CrossRef  Google Scholar 

  20. Dickson, G.W., Senn, J.A., Chervany, N.L.: Research in management information systems: the minnesota experiments. Manag. Sci. 23, 913–934 (1977)

    CrossRef  Google Scholar 

  21. Benbasat, I., Schroeder, R.G.: An experimental investigation of some MIS design variables. MIS Q. 1, 37–49 (1977)

    CrossRef  Google Scholar 

  22. Cleveland, W.: Visualizing Data (1985)

    Google Scholar 

  23. Kosslyn, S.M.: Understanding charts and graphs. Appl. Cogn. Psychol. 3, 185–225 (1989)

    CrossRef  Google Scholar 

  24. Tufte, E.: Envisioning information. Graphics Press, Cheshire (1990)

    Google Scholar 

  25. Ives, B.: Graphical user interfaces for business information systems. MIS Q. 6, 15–47 (1982)

    CrossRef  Google Scholar 

  26. Benbasat, I., Dexter, A.S., Todd, P.: An experimental program investigating color-enhanced and graphical information presentation: an integration of the findings. Commun. ACM 29, 1094–1105 (1986)

    CrossRef  Google Scholar 

  27. DeSanctis, G.: Computer graphics as decision aids: directions for research*. Decis. Sci. 15, 463–487 (1984)

    CrossRef  Google Scholar 

  28. Vessey, I., Galletta, D.: Cognitive fit: an empirical study of information acquisition. Inf. Syst. Res. 2, 63–84 (1991)

    CrossRef  Google Scholar 

  29. Bačić, D., Henry, R.: The role of business information visualization in knowledge creation. In: 2012 AMCIS (2012)

    Google Scholar 

  30. Bačić, D., Fadlalla, A.: Business information visualization: a visual intelligence-based framework. In: 2013 AMCIS (2013)

    Google Scholar 

  31. Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124, 372–422 (1998)

    CrossRef  Google Scholar 

  32. Eckstein, M.K., Guerra-Carrillo, B., Singley, A.T.M., Bunge, S.A.: Beyond eye gaze: what else can eyetracking reveal about cognition and cognitive development? Dev. Cogn. Neurosci. 25, 69–91 (2017)

    CrossRef  Google Scholar 

  33. Just, M.A., Carpenter, P.A.: Eye fixations and cognitive processes. Cogn. Psychol. 8, 441–480 (1976)

    CrossRef  Google Scholar 

  34. Granholm, E., Steinhauer, S.R.: Pupillometric measures of cognitive and emotional processes. Int. J. Psychophysiol. 52, 1–6 (2004)

    CrossRef  Google Scholar 

  35. Beatty, J.D.: Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychol. Bull. 91, 276–292 (1982)

    CrossRef  Google Scholar 

  36. Smuc, M.: Just the other side of the coin? from error to insight analysis. Inf. Vis. 15, 312–324 (2016)

    CrossRef  Google Scholar 

  37. Klingner, J., Tversky, B., Hanrahan, P.: Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiology 48, 323–332 (2011)

    CrossRef  Google Scholar 

  38. Bačić, D.: Understanding business dashboard design user impact: triangulation approach using eye-tracking, facial expression, galvanic skin response and EEG sensors. In: 2017 AMCIS, Boston, MA (2017)

    Google Scholar 

  39. Nakasone, A., Prendinger, H., Ishizuka, M.: Emotion recognition from electromyography and skin conductance. In: The Fifth International Workshop on Biosignal Interpretation (BSI-05), pp. 209–2012 (2005)

    Google Scholar 

  40. Boucsein, W.: Electrodermal Activity. Springer Science & Business Media, Boston (2012). https://doi.org/10.1007/978-1-4614-1126-0

    CrossRef  Google Scholar 

  41. Villarejo, M.V., Zapirain, B.G., Zorrilla, A.M.: A stress sensor based on galvanic skin response (GSR) controlled by ZigBee. Sensors (Basel). 12, 6075–6101 (2012)

    CrossRef  Google Scholar 

  42. Shi, Y., Ruiz, N., Taib, R., Choi, E., Chen, F.: Galvanic skin response (GSR) as an index of cognitive load. In: CHI 2007 Extended Abstracts on Human Factors in Computing Systems, pp. 2651–2656. ACM, New York (2007)

    Google Scholar 

  43. Hjortsjö, C.H.: Man’s Face and Mimic Language. Studen litteratur, Lund (1969)

    Google Scholar 

  44. Ekman, P., Friesen, W.V, Hager, J.C.: Facial action coding system: the manual. Research Nexus Devision of Network Information Research Corporation. Salt Lake City (2002)

    Google Scholar 

  45. Klimesch, W.: EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res. Rev. 29, 169–195 (1999)

    CrossRef  Google Scholar 

  46. Berka, C., Levendowski, D.J., Cvetinovic, M.M., Petrovic, M.M., Davis, G., Lumicao, M.N., Zivkovic, V.T., Popovic, M.V., Olmstead, R.: Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. Int. J. Hum.-Comput. Interact. 17, 151–170 (2004)

    CrossRef  Google Scholar 

  47. Berka, C., Levendowski, D.J., Lumicao, M.N., Yau, A., Davis, G., Zivkovic, V.T., Olmstead, R.E., Tremoulet, P.D., Craven, P.L.: EEG correlates of task engagement and mental\workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78, B231–B244 (2007)

    Google Scholar 

  48. Gevins, A., Smith, M.E.: Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style (2000)

    CrossRef  Google Scholar 

  49. Bowden, E.M., Jung-Beeman, M., Fleck, J., Kounios, J.: New approaches to demystifying insight. Trends Cogn. Sci. 9, 322–328 (2005)

    CrossRef  Google Scholar 

  50. Harmon-Jones, E., Gable, P.A.: On the role of asymmetric frontal cortical activity in approach and withdrawal motivation: an updated review of the evidence. Psychophysiology 55, e12879 (2018)

    CrossRef  Google Scholar 

  51. Cernea, D., Olech, P.-S., Ebert, A., Kerren, A.: EEG-based measurement of subjective parameters in evaluations. In: Stephanidis, C. (ed.) HCI 2011. CCIS, vol. 174, pp. 279–283. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22095-1_57

    CrossRef  Google Scholar 

  52. Lee, J.C., Tan, D.S.: Using a low-cost electroencephalograph for task classification in HCI research. In: Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology, pp. 81–90. ACM, New York (2006)

    Google Scholar 

  53. Bera, P., Bera, B.Y.P.: How colors in business dashboards affect users’ decision making. Commun. ACM 59, 50–57 (2016)

    CrossRef  Google Scholar 

  54. Borkin, M.A., Bylinskii, Z., Kim, N.W., Bainbridge, C.M., Yeh, C.S., Borkin, D., Pfister, H., Oliva, A., Member, S., Oliva, A.: Beyond memorability: visualization recognition and recall. IEEE Trans. Vis. Comput. Graph. 22, 519–528 (2016)

    CrossRef  Google Scholar 

  55. Goldberg, J., Helfman, J.: Eye tracking for visualization evaluation: reading values on linear versus radial graphs. Inf. Vis. 10, 182–195 (2011)

    CrossRef  Google Scholar 

  56. Toker, D., Conati, C., Steichen, B., Carenini, G.: Individual user characteristics and information visualization : connecting the dots through eye tracking (2013)

    Google Scholar 

  57. Goldberg, J.H., Helfman, J.I.: Comparing information graphics: a critical look at eye tracking. In: Proceedings of the 3rd BELIV 2010 Workshop: BEyond Time and Errors: Novel evaLuation Methods for Information Visualization, pp. 71–78. ACM, New York (2010)

    Google Scholar 

  58. Peebles, D., Cheng, P.C.-H.: Modeling the effect of task and graphical representation on response latency in a graph reading task. Hum. Factors 45, 28–46 (2003)

    CrossRef  Google Scholar 

  59. Hur, I., Yi, J.S.: SimulSort: multivariate data exploration through an enhanced sorting technique. In: Jacko, J.A. (ed.) HCI 2009. LNCS, vol. 5611, pp. 684–693. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02577-8_75

    CrossRef  Google Scholar 

  60. Hur, I., Kim, S.-H., Samak, A., Yi, J.S.: A comparative study of three sorting techniques in performing cognitive tasks on a tabular representation. Int. J. Hum.-Comput. Interact. 29, 379–390 (2013)

    CrossRef  Google Scholar 

  61. Kim, S.H., Dong, Z., Xian, H., Upatising, B., Yi, J.S.: Does an eye tracker tell the truth about visualizations?: findings while investigating visualizations for decision making. IEEE Trans. Vis. Comput. Graph. 18, 2421–2430 (2012)

    CrossRef  Google Scholar 

  62. Siirtola, H., Laivo, T., Heimonen, T., Räihä, K.J.: Visual perception of parallel coordinate visualizations. In: 2009 13th International Conference Information Visualisation, pp. 3–9 (2009)

    Google Scholar 

  63. Burch, M., Andrienko, G., Andrienko, N., Höferlin, M., Raschke, M., Weiskopf, D.: Visual task solution strategies in tree diagrams. In: 2013 IEEE Pacific Visualization Symposium (PacificVis), pp. 169–176 (2013)

    Google Scholar 

  64. Burch, M., Heinrich, J., Konevtsova, M., Hoeferlin, M., Weiskopf, D.: Evaluation of traditional, orthogonal, and radial tree diagrams by an eye tracking study. IEEE Trans. Vis. Comput. Graph. 17, 2440–2448 (2011)

    CrossRef  Google Scholar 

  65. Huang, W., Eades, P.: How people read graphs. In: Proceedings of the 2005 Asia-Pacific Symposium on Information Visualisation, vol. 45, pp. 51–58. Australian Computer Society, Inc., Darlinghurst (2005)

    Google Scholar 

  66. Jianu, R., Rusu, A., Hu, Y., Taggart, D.: How to display group information on node-link diagrams: an evaluation. IEEE Trans. Vis. Comput. Graph. 20, 1530–1541 (2014)

    CrossRef  Google Scholar 

  67. Huang, W.: Using eye tracking to investigate graph layout effects. In: Asia-Pacific Symposium on Visualisation 2007 (APVIS), pp. 97–100, Sydney (2007)

    Google Scholar 

  68. Huang, W., Eades, P., Hong, S.-H.: A graph reading behavior: geodesic-path tendency. In: 2009 IEEE Pacific Visualization Symposium, pp. 137–144 (2009)

    Google Scholar 

  69. Pohl, M., Schmitt, M., Diehl, S.: Comparing the readability of graph layouts using eyetracking and task-oriented analysis. In: Proceedings of the Fifth Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging, pp. 49–56. Eurographics Association, Aire-la-Ville (2009)

    Google Scholar 

  70. Körner, C.: Sequential processing in comprehension of hierarchical graphs. Appl. Cogn. Psychol. 18, 467–480 (2004)

    CrossRef  Google Scholar 

  71. Körner, C.: Eye movements reveal distinct search and reasoning processes in comprehension of complex graphs. Appl. Cogn. Psychol. 25(6), 893–905 (2011)

    CrossRef  Google Scholar 

  72. Carpenter, P.A., Shah, P.: A model of the cognitive and perceptual processes in graphical display comprehension. J. Exp. Psychol. Appl. 4, 75–100 (1998)

    CrossRef  Google Scholar 

  73. Pirolli, P., Card, S.K., Van Der Wege, M.M.: Visual information foraging in a focus + context visualization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 506–513. ACM, New York (2001)

    Google Scholar 

  74. Griffin, A.L., Robinson, A.C.: Comparing color and leader line highlighting strategies in coordinated view geovisualizations. IEEE Trans. Vis. Comput. Graph. 21, 339–349 (2015)

    CrossRef  Google Scholar 

  75. Blascheck, T., Ertl, T.: Towards analyzing eye tracking data for evaluating interactive visualization systems. In: Proceedings of the 2014 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization (2014)

    Google Scholar 

  76. Ba, D., Fadlalla, A.: Business information visualization intellectual contributions: an integrative framework of visualization capabilities and dimensions of visual intelligence. Decis. Support Syst. 89, 77–86 (2016)

    CrossRef  Google Scholar 

  77. Cernea, D., Kerren, A., Ebert, A.: Detecting insight and emotion in visualization applications with a commercial EEG headset. In: 2011 SIGRAD, pp. 53–60 (2011)

    Google Scholar 

  78. Anderson, E.W., Potter, K.C., Matzen, L.E., Shepherd, J.F., Preston, G.A., Silva, C.T.: A user study of visualization effectiveness using EEG and cognitive load, vol. 30 (2011)

    CrossRef  Google Scholar 

  79. Cattell, R.B.: Theory of fluid and crystallized intelligence: a critical experiment. J. Educ. Psychol. 54, 1–22 (1963)

    CrossRef  Google Scholar 

  80. McGrew, K.S.: CHC theory and the human cognitive abilities project: standing on the shoulders of the giants of psychometric intelligence research. Intelligence 37, 1–10 (2009)

    CrossRef  Google Scholar 

  81. Tukey, J.W.: Exploratory Data Analysis. Addison-Wesley, Reading (1977)

    MATH  Google Scholar 

  82. Baker, J., Jones, D., Burkman, J.: Using visual representations of data to enhance sensemaking in data exploration tasks. J. Assoc. Inf. Syst. 10, 533–559 (2009)

    Google Scholar 

  83. Kosara, R., Hauser, H., Gresh, D.L.: An interaction view on information visualization. In: 2003 State-of-the-Art Proceedings of EUROGRAPHICS (2003)

    Google Scholar 

  84. Yi, J.S., ah Kang, Y., Stasko, J., Jacko, J.: Toward a deeper understanding of the role of interaction in information visualization. IEEE Trans. Vis. Comput. Graph. 13, 1224–1231 (2007)

    CrossRef  Google Scholar 

  85. Liu, Z., Stasko, J.: Mental models, visual reasoning and interaction in information visualization: a top-down perspective. IEEE Trans. Vis. Comput. Graph. 16, 999–1008 (2010)

    CrossRef  Google Scholar 

  86. Hambrick, D.Z., Engle, R.W.: Effects of domain knowledge, working memory capacity, and age on cognitive performance: an investigation of the knowledge-is-power hypothesis. Cogn. Psychol. 44, 339–387 (2002)

    CrossRef  Google Scholar 

  87. Lohman, D.: Spatial Ability. Macmillan, New York (1994)

    Google Scholar 

  88. Ware, C.: Information Visualization: Perception for Design. Morgan Kaufmann, San Francisco (2000)

    Google Scholar 

  89. Ashcraft, M.H.: Fundamentals of Cognition. Addison-Wesley Educational Publishers, New York (1998)

    Google Scholar 

  90. Miller, G.A.: The magical number seven, plus or minus two: some limits on our capacity for processing information (1956)

    CrossRef  Google Scholar 

  91. Kosara, R., Mackinlay, J.: Storytelling: the next step for visualization. Computer (Long. Beach. Calif.) 46, 44–50 (2013)

    Google Scholar 

  92. Gershon, N., Page, W.: What storytelling can do for information visualization. Commun. ACM 44, 31–37 (2001)

    CrossRef  Google Scholar 

  93. Elias, M., Aufaure, M.-A., Bezerianos, A.: Storytelling in visual analytics tools for business intelligence. In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds.) INTERACT 2013. LNCS, vol. 8119, pp. 280–297. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40477-1_18

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dinko Bačić .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Bačić, D. (2018). Biometrics and Business Information Visualization: Research Review, Agenda and Opportunities. In: Nah, FH., Xiao, B. (eds) HCI in Business, Government, and Organizations. HCIBGO 2018. Lecture Notes in Computer Science(), vol 10923. Springer, Cham. https://doi.org/10.1007/978-3-319-91716-0_53

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91716-0_53

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91715-3

  • Online ISBN: 978-3-319-91716-0

  • eBook Packages: Computer ScienceComputer Science (R0)