Overloaded and Biased? Using Augmented Cognition to Understand the Interaction Between Information Overload and Cognitive Biases
- 1.6k Downloads
Virtual teams are increasingly utilized in organizations, yet they often make poor decisions. Previous research has established that a primary cause of poor virtual team decision making is due to confirmation bias: team members focusing their cognitive resources on factual and normative information that supports pre-discussion preferences, rather than deeply considering information that challenges them. Building on this, the current study examines whether confirmation bias exists at varying levels of information load, establishing if confirmation bias is mediated by information overload. This study will utilize electroencephalography (EEG) and psychophysiology to examine changes in an individual’s processing of information at three levels of information load. The individual will participate in a simulated team discussion on a decision-making task. These findings will elucidate whether virtual team members use confirmation bias as a heuristic problem solving approach in response to information overload or if confirmation bias is present in all virtual team interactions.
KeywordsCognitive load and performance Confirmation bias Electroencephalography Augmented cognition Decision making Virtual teams
Virtual teams have been increasingly utilized in today’s organizations over the past decade, and they are now commonplace in today’s organizations [1, 2]. Virtual teams enable team members to be either spatially or temporally dispersed, yet able to readily communicate and solve problems . Generally, virtual teams communicate via different forms of technology [4, 5, 6, 7], including videoconferencing, email, online chat rooms, and instant messaging (IM). These different modes of communication vary in richness and synchronicity, providing flexibility in the way virtual teams communicate [8, 9]. These technologies enable certain process gains, such as the elimination of production blocking and increased anonymity . However, despite these gains, decades of research has shown that: virtual teams come to no better decisions than face-to-face teams and this problem exists at the level of individual information processing [11, 12].
Recently, a study used electroencephalography (EEG), electrodermal activity (EDA), and facial electromyography (EMG) to examine individual information processing in simulated virtual team interactions. While the individual was connected to the neurophysiological and psychophysiological equipment, they interacted with a simulator which they believed to be a real virtual team. The findings indicated that individuals processed information that conformed with their beliefs before the virtual team interaction differently than information that differed from individuals preferences before the virtual team interaction, indicating confirmation bias was present in the virtual team interaction . The study, however, did not examine whether varying levels of information load led to confirmation bias, that is, the research was unable to answer if confirmation bias was a cause of information overload or a “default” bias individuals revert to during virtual team interactions.
This study examines whether increased information load in a virtual team is related to the presence of confirmation bias. In essence, we examine whether confirmation bias is a heuristic that individuals rely on once information processing becomes overwhelming. To examine this phenomenon, we will collect heart rate data, electrodermal response (EDR) and facial electromyography (EMG). We will also collect electroencephalography (EEG) data to examine changes in cognition that indicate confirmation bias as was done in . This research will elucidate whether the confirmation bias reported in  is related to the individual’s information load in a virtual team or is present even in low information load conditions. The findings will have implications for the design of collaborative technologies, specifically whether design should focus on alleviating confirmation bias or information overload.
2 Theoretical Background
2.1 Decision Making
The decision making process is a dynamic process involving multiple decision factors . The pros and cons of each factor must be considered to reach an optimal decision . Prior research has identified unique challenges in team decision making especially due to negotiating multiple opinions [15, 16, 17]. Other factors found to affect team decision-making processes include cognitive processes [15, 18], cognitive anchors [15, 19], and the desire to stick to prior decisions [15, 20].
There are three distinct stages in the decision making process: the pre-decision stage, the decision stage, and the post-decision stage . The scope of this study is limited to the first two. The pre-decision stage involves gathering information about the decision alternatives . Teams are often formed because they have access to more information than any one decision maker acting alone . Past research has shown that on average, teams make better decisions than individuals, even when they are comprised of team members with relatively homogenous backgrounds, perhaps because they are less likely to make major errors .
The net result is what Stasser  has called a “hidden profile” task. Each team member knows some baseline information that is known to all team members, which is often called “common information” . Each team member also usually knows some “unique information” that is known only by them. This combination of information typically leads each team member to form initial pre-discussion preferences about the decision alternatives. Because these pre-discussion preferences are based on a mixture of common and unique information, it is not unusual for individual team members to develop different preferences prior to a team discussion.
During the decision stage, team members come together (face-to-face or virtually) to discuss the decision alternatives. During this process, they share the information they have, as well as their preferences for the different alternatives. Thus the information that team members contribute to and receive from the discussion can be organized into two distinct categories . The first category is factual information about the decision alternatives. Factual information is devoid of opinion (although the choice of what factual information to contribute can be shaped by opinions) . It simply states the facts and leaves the interpretation of those facts to the receiver. The second category is normative information about the preferences of the contributor. Normative information simply states which alternative(s) the contributor prefers, without providing underlying reasons .
Since most teams have a mix of common and unique information, it is not uncommon for team members to receive information they already know, as well as new information they do not know. This new information—information not considered in the pre-discussion stage—should be what a team member focuses on, because it is new and not yet considered . Such new information received from other team members may support or challenge the pre-discussion preferences that a team member has developed. In theory, new information that challenges pre-discussion preferences is the most important and should receive the most attention because it has the greatest potential to change decisions [26, 27, 28, 29]. However, prior research has shown that team decisions can be contaminated by information processing biases [14, 30, 31]. Virtual team decision making is also encumbered by these biases .
2.2 Confirmation Bias and Information Overload
One of the most common biases present in team discussions is confirmation bias. Confirmation bias is the tendency of people to ignore information that challenges their current decision preferences [32, 33, 34]. Thus not only do decision makers seek out selective information, but when they do uncover information that challenges their initial preferences, they resolve the cognitive dissonance by ignoring the contradictory information [32, 33, 35]. This can lead to the biased interpretation of new information they receive such that information that supports pre-discussion preferences is considered while information that challenges those preferences is ignored . Thus the net effect of gathering new information, even unbiased information search that gathers new information that both supports and challenges existing decision preferences, is to reinforce the existing pre-discussion preferences, rather than helping decision makers find better solutions. This hurts decision quality if the initial decision preferences are incorrect .
Most of the research on bias in decision making has focused on individual decision making not, team decision making. One would hope that involving other people in the decision process, each of whom may bring their own—and potentially different—biases to the discussion, might overcome some of the selective information search and confirmation bias observed in individual decision making. Unfortunately, selective information search also has been observed in teams that meet face-to-face to make decisions . Stasser and his colleagues have published many studies examining face-to-face decision making teams and have repeatedly found that teams engage in selective information search by focusing the discussion on common information known to all team members [23, 37, 38, 39]. Team members routinely choose not to share the unique information known only to them, and as a result, team decisions are often poor.
An individual’s processing of information will be affected by information load, such that an increase in information load results in an increase in the presence of confirmation bias.
Our goal in this study is to understand how individual team members respond to new information they receive from other team members during a text-based ICT discussion in varying information load conditions. Therefore, the unit of analysis will be the individual, not the team. Given this focus, having real participants interact with each other and measuring the data at the individual level would introduce large error variance, because each team discussion is likely to be different. Therefore, we will use a simulator [see, 12, 43, 44] designed to look and act like a real Instant Messenger (IM) tool. The study participant can type comments in the simulator and see them contributed to the discussion. The simulator will present participants with a standardized script based off of a real-world virtual team discussion. In order not to bias the outcomes, participants were not informed that they were using a simulator until the end of the experimental session.
Participants will be undergraduate students from a state university who will receive course credit for their time. We aim to collect data on 40 subjects in a repeated measures design.
We will use three hidden profile decision making tasks. In each task, we will ask the participants to select three of five fictitious student applicants to admit to the university. This task is based on one used extensively in prior research [e.g., 11, 45]. It has the advantage of being familiar to the participants, because each of them has successfully navigated the university entrance process and understands the task information (e.g., GPA, SAT). The version of the tasks we used are validated by the admissions office at the university where the experiment was conducted to ensure that the information it contained was appropriate.
Each task has two parts. First, the participant is given incomplete information about the five potential applicants and asked to make an initial decision of which three to admit. This corresponds to the pre-decision stage commonly encountered in decision making. Then the participant will be informed that they will work in a team with four other participants using a text-based discussion tool to discuss the information and reach a team decision that could be the same or different from their initial decisions. Participants will be informed that each team member has received incomplete information. Some common information is known to all team members but each team member had been given unique information that is known only to them, and that it was important to share this unique information and consider the new information contributed by others in order to reach a good decision.
The IM tool the participants will use is like most text-based discussion tools. The simulator will provide two onscreen windows. The top window displays the comments from “other team members” and the bottom window enabled the participants to type their comments (which were then displayed in the top window with the other comments). The participant was told that the other discussion participants were in different locations, and not with them in the research lab, because we were not collecting physiological data from them.
There will be three treatments: a low, medium, and high information load condition. In the low information load condition, the virtual team discussion will move slowly, providing the participant with little information to process during the discussion. The high information load condition will move quickly providing the participant much information to process. The medium information load condition will contain a moderate amount of information for the subject to process. The way participants process the information in each condition will be compared to examine whether confirmation bias is present in each condition. The conditions will be counter-balanced across participants to control for any task order effects.
The independent variable is the nature of the information contained in the simulator script displayed to the participants, with five levels explained below. The scripts were written to closely match the transcripts of discussions from student teams that had performed this task in prior experiments.
The script was designed to contain five types of information contained in target statements. The target statements contained factual information that supported the participant’s pre-discussion decision preference and factual information that challenged the participant’s pre-discussion decision preference (e.g., Don had one of the highest GPAs). We will record each participant’s pre-discussion choice so we could cross check it against the target statements to ensure that preference supporting and preference challenging information was coded appropriately. Separate target statements will also contain normative information that supports the participant’s pre-discussion decision preference and information that challenges it (e.g., I like Angela). Fifteen target statements contained irrelevant information designed to provide a baseline against which we could compare the other four types of information (e.g., I’m hungry). The irrelevant information was drawn from transcripts of teams that had performed this task in other experiments, so it commonly appears in team discussions of this type.
In the low, medium, and high information load treatments, we will vary the amount of the five types of target statements to increase information processing requirements on the participant.
Autonomic arousal will be operationalized as skin conductance level measured with disposable electrodes filled with electrically neutral gel and adhered planar surface of the foot. A Biopac MP150 system will be used to collect the skin conductance data at 1000 Hz. Emotional valence will be operationalized as the relative activation of the corrugator supercilli muscle group (facial EMG). Corrugator EMG will be measured using a pair of mini (4 mm) reusable AG/AGCL electrodes filled with electrolyte gel placed above the subject’s left eye after dead skin cells has been removed by a skin prep pad containing rubbing alcohol and pumice. The bipolar corrugator measures will be collected using the Biopac MP150 system with high pass filters set at 8 Hz. The full wave signal will be rectified and then contour integrated online at a time constant of 100 ms, and then sampled at 1000 Hz by the Biopac MP150 system.
Controls and Manipulation Checks.
It is essential to ensure that participants perceive the simulator as a real team discussion. All participants will complete a post-session questionnaires that asked if they had noticed anything unusual about the team discussion. A variety of distractor questions (e.g., satisfaction with discussion, perceived effectiveness) will be included to better ensure that manipulation check question does not stand out. Participants that indicate they did not believe the simulated team discussions were not a real will be removed from the analysis.
Markers will be inserted by the researchers into the EEG data to indicate onset and end of the simulator’s text discussion. A timestamp generated by the simulator program will be used to identify precisely when each statement is displayed. This timestamp marker for each target statement will then be matched to the EEG data.
Participants will complete the experimental procedure individually after providing informed consent approved by the university’s Institutional Review Board. The experiment will take place in an individual lab room. The entire research session will take about 90 min. The experiment will be controlled by E-Prime software.
Participants will be seated in a high back chair to minimize movement. They will be provided a standard keyboard and a mouse. The experimenter will then explain the procedure for attaching the physiological electrodes and fitting the EEG cap, answering any questions posed by the participant. After obtaining adequate impedance readings for the EDA, EMG and EEG measures, the protocol will continue with another brief handedness questionnaire.
Next the participant will receive information on the college admissions task and the pre-discussion information. They will be instructed that not only should they use this information to make their decision, but that they would be sharing the information they are given with the team. They will be given 3 min to read the information, which will be presented in the form of a data table. Then they must decide which three of five students they would choose to admit.
After making their pre-discussion decisions, the participant will be instructed on the nature of the discussion task. They will be told that the other discussion participants are located at various places across campus and not with them in the research lab because physiological data is not being collected from them. The participant will be told that they would participate in three different decision-making tasks. Each task will be a discussion that lasts for 12 min. The participant will be reminded that each team member received information known only to them. They will also be instructed that they should therefore share all the information they have and read all the information provided by other team members.
After the three virtual team decision making tasks are completed, the experimenter will remove the EEG cap and physiological sensors. The participant will then complete the post-experiment questionnaire. Finally, the participant will be debriefed, told of the deception, asked not to inform others of the deception, and thanked for their time.
3.5 Data Cleaning and Preparation
EEG data will be cleaned and analyzed using EEGLab . One limitation of EEG is that cortical bioelectrical activity is extremely small in magnitude when compared to muscle movements across the head. Therefore, participant movement introduces artifacts of high-frequency and magnitude into the EEG data. These will be removed using two methods: EEGlab probability calculations and visual inspection. The EEGLab artifact rejection algorithm uses deviations in microvolts greater than three standard deviations from the mean to reject specific trials. However, additional artifacts are also apparent to the trained eye, so visual inspection of trials is essential in artifact removal .
In addition to trial-by-trial removal of artifacts, occasionally specific EEG channels must be rejected in an individual subject’s data due to unacceptable impedance levels. This can be done in the current study using an automatic impedance detection feature of EEGLab.
Electrodermal and facial EMG data were aggregated to mean values per second using Biopac’s Acqknowledge software. Change scores will be calculated by subtracting the physiological level at the onset of each target statement during the online discussion from each subsequent second across a 6-s window.
3.6 ICA Analysis of EEG Data
The first step of the EEG analysis will be an Independent Components Analysis (ICA) at the individual level. A common problem in neurophysiology research results from the collection of large amounts of data which, based upon the Central Limit Theorem, become normally distributed. However, the brain is comprised of discrete patches of cortex that are very active at some points in time and relatively non-active at others (i.e. – activity is not normally distributed across the scalp) . ICA overcomes this problem by taking this Gaussian data and rotating it until it becomes non-Gaussian, thereby isolating independent components of activation. The ICAs are distributed patterns of activation across the 14-electrodes in the EEG system.
Initially, an EEGLab ICA performs a Principal Components Analysis (PCA). At each electrode site the program assesses which of the other electrode sites account for the most variance in the signal. Taking these weighted values it then relaxes the orthogonality constraint of PCA to isolate individual components of activation . Each ICA component then represents a pattern of activation over the entire brain, not solely the activity present at a specific electrode. The number of independent components (ICs) depends on the number of electrodes in the dataset, as the algorithm is working in an N-dimensional space (where N is the number of electrodes). Most participants in the current study are expected to generate 14 distinct ICs, since our recording device has 14 electrodes.
Finally, using the K-means component of EEGlab the independent components at the individual level will be grouped into clusters containing similar components using procedures recommended by . This procedure clusters similar ICs based upon their latency, frequency, amplitude, and scalp distribution . Relevant clusters will be identified and a time-frequency decomposition will be performed to examine changes in event-related desynchronization of the alpha rhythm.
4 Potential Implications
This study seeks to elucidate whether confirmation bias and information load are mutually exclusive or mutually related. We examine cognitive and emotional changes present at the individual level in a virtual team interaction to understand how information load affects processing of different types of information. We believe the findings of this study have several implications for understanding virtual team interactions.
First, if we find that confirmation bias and information load interact, there are several areas of interest for design. Mainly, the question poised will be: How can we alleviate information load in a virtual team decision making setting? Exploring ways to design collaborative tools to reduce information load will also alleviate confirmation bias. Conversely, if we find that confirmation bias is unrelated to information load, the question poised would be: How do we design a collaborative tool to reduce confirmation bias? Both prospects will provide future directions for further research on this topic and give designers of collaborative tools insight into how to improve their software. Overall, the findings of this study will generate many avenues for future research on decision-making, collaboration and augmented cognition.
- 1.Zuboff, S.: In the Age of the Smart Machine: The Future of Work and Power. Basic Books Inc., New York (1984)Google Scholar
- 3.O’Leary, M.B., Cummings, J.N.: The spatial, temporal, and configurational characteristics of geographic dispersion in teams. MIS Q. 31, 433–452 (2007)Google Scholar
- 5.Duarte, D.L., Tennant-Snyder, N.: Mastering Virtual Teams: Strategies, Tools, and Techniques that Succeed. Jossey-Bass, San Francisco (1999)Google Scholar
- 6.Lipnack, J., Stamps, J.: Virtual Teams: Working Across Space, Time and Organizations. Wiley, New York (1997)Google Scholar
- 7.Townsend, A.M., DeMarie, S.M., Hendrickson, A.R.: Virtual teams: technology and the workplace of the future. Acad. Manag. Exec. (1993–2005) 12, 17–29 (1998)Google Scholar
- 9.Dennis, A.R., Fuller, R.M., Valacich, J.S.: Media, tasks, and communication processes: a theory of media synchronicity. MIS Q. 32, 575–600 (2008)Google Scholar
- 16.Guzzo, R.A.: Group decision making and group effectiveness in organizations. In: P.S.G. Associates (ed.) Designing Effective Work Groups, pp. 34–71. Jossey-Bass, San Francisco (1986)Google Scholar
- 17.Hackman, J.R.: Groups That Work (and Those That Don’t). Jossey-Bass, San Francisco (1991)Google Scholar
- 18.Bazerman, M.H.: Judgement in Managerial Decision Making. Wiley, New York (1990)Google Scholar
- 20.Staw, B.M.: The escalation of commitment to a course of action. Acad. Manag. Rev. 6, 577–587 (1981)Google Scholar
- 26.Shoemaker, P.J.: Media Gatekeeping, 2nd edn. Longman, New York (1996)Google Scholar
- 30.Klayman, J.: Varieties of confirmation bias. In: Jerome Busemeyer, R.H., Douglas, L.M. (eds.) Psychology of Learning and Motivation, vol. 32, pp. 385–418. Academic Press, New York (1995)Google Scholar
- 42.Frey, D., Schulz-Hardt, S., Stahlberg, D.: Information seeking among individuals and groups and possible consequences for decision-making in business and politics. Underst. Group Behav. 2, 211–225 (2013)Google Scholar
- 44.Hilmer, K.M., Dennis, A.R.: Stimulating thinking: cultivating better decisions with groupware through categorization. J. Manag. Inf. Syst. 17, 93–114 (2000)Google Scholar
- 48.Onton, J., Makeig, S.: Information-based modeling of event-related brain dynamics. In: Christa, N., Wolfgang, K. (eds.) Progress in Brain Research, vol. 159, pp. 99–120. Elsevier, Amsterdam (2006)Google Scholar
- 49.Delorme, A., Makeig, S.: EEGLAB Wikitutorial, May 2012. http://sccn.ucsd.edu/wiki/PDF:EEGLAB_Wiki_Tutorial