Presentation of Information Uncertainty from IoBT for Military Decision Making

Abstract

The goal of this paper is to investigate uncertainty of information introduced at the final decision point and ways this information can be presented to potentially improve decision making. Uncertainty expressions and representations from different fields of study will be explored in this paper. While uncertainty is important to all types of decision making there is a focus on a simple task that is representative of a military relevant task in this paper. Supporting this effort will be the results from a recent experimental study where information from simulated Internet of Battlefield Thing (IoBT) devices were presented to Soldiers for a simplified decision making task with and without an uncertainty of information value given at preset decision making points. The presentation of this information was done in the Android Tactical Assault Kit (ATAK) which is a mapping engine that allows precision targeting, intelligence on surrounding land formations, navigation, and situational awareness. For this experimental study the software was modified to add a decision making component within the ATAK interface. Analysis of the data will help influence additional directions for future research and experimentation.

1 Introduction

The Internet of Things (IoT) is defined as “a variety of objects or things that are able to interact with each other through unique addressing schemes [1].” IoT was listed as one of six “Disruptive Civil Technologies” by the US National Intelligence Council. The strength of IoT has been its impact to everyday life and how individuals use this new capability. IoT systems are expanding from commercial application the government or DoD, in the form of Internet of Military Things or Internet of Battlefield Things (IoBT) [2]. But the use of IoBT systems to solve defense problems is not trivial or straightforward. The challenges to the realization of IoBT include the organization and management of the large and heterogeneous set of devices, as well as how individuals will make sense of the diverse data that comes from these devices. If IoBT devices provide new sources of data, then they will have a significant impact on decision making. Like all technology, the benefits come with possible uncertainty from the operation of the devices to the information that the devices generate. This leads to the question like what and how uncertainty of information will be represented and presented to decision makers, particularly in military operations.

A decision is defined as a conclusion or a resolution reached after consideration or the action or process of deciding something. While, decision making is the thought process of selecting a logical choice from the available options. When trying to make a good decision, a person must weigh the pros and cons of each option and consider all the alternatives. Ideally, a person must also be able to predict the possible outcomes of multiple options and determine which option is the best for the particular situation [3]. This includes making the decision with uncertainties in mind. Multiple theories for decision making exist, however common threads like uncertainty may be seen across the various concepts.

Whether it is Herbert Simon’s three step theory, Colonel John Boyd’s “Observe, Orient, Decide and Act” (OODA) loop, or the military decision making process (MDMP), the ability to grasp uncertainty is a key element of decision making. From a military standpoint the requirements of decision making across the echelons drives the effect of uncertainty as the time available to make a decision is compressed from months to seconds as one approaches the tactical edge. The inclusion of IoBT devices in decision making will increase the volume of data available to base decisions on, but may also cause new sources of uncertainties.

Equally important is the expression and presentation of this data along with any associated uncertainty. The uncertainty may be related to the ambiguity, limitations, and constraints linked with the devices and the device data. Critical to decision making is any uncertainty related to possible risks that shape the final decision. While uncertainty measures of information can be interjected into multiple steps prior to and after the final decision in this paper the focus is on uncertainty presented prior to the final decision or act.

2 Decisions and Decision Making

2.1 Models for Decision Making

There are several models for decision making. Three have been selected to highlight in this paper. The first was developed by Herbert Simon the Nobel Prize winner known for his decision making theory categorized in three steps: intelligence activity, design activity, and choice activity [4]. In his paper, Herbert Simon discussed those things that affected a rational decision: lack of knowledge, cognitive limitations, and time constraints which can cause uncertainty. Lack of knowledge is due to the problem that all the information that will generate the optimal decision is not always known or available to the decision maker.

The second is the OODA loop decision cycle developed by Colonel John Boyd from the United States Air Force [5]. As a military strategist, COL Boyd’s creation was primarily applied to combat operations and included five steps: observe, orient, decide, and act. In Boyd’s 1987 talk “Organic Design for Command and Control” he stated that uncertainty places the biggest part in the observe and orient steps. In these steps, uncertainty is in the managing of information including the evolution, the inconsistencies, the incompleteness, and the dependencies from the sources of the observations [5].

The third, is the military decision making process (MDMP) [6]. The MDMP is iterative, linking the commander, the staff, subordinate headquarters, and the partners’ activities to make decisions for successful missions. There are seven basic steps in the MDMP. In the MDMP, Receipt of Mission and Production of Orders are critical anchors serving as the start and end steps of the process. Between these anchor points five steps are identified: mission analysis, course of action (COA) development, course of action analysis, course of action comparison, and course of action approval. Having an idea of the uncertainty that impacts the decision is needed ideally at each step in the process.

3 Uncertainty

3.1 Presentation of Uncertainty

In the book “The Design of Everyday Things” [7] the author stated “Great precision is not required.” If this is true, then how do we represent uncertainty information? How do we present uncertainty information that positively impacts decision making, making decisions that are robust and adaptive?

Although not a new problem, presentation of uncertainty is a continual challenge. The seemingly never-ending struggle to display information from sources that may not be intuitive and that may grow exponentially from the devices and the data supports the need to continue investigating the presentation of uncertainty. The questions how much information is needed to aid decision makers and how to aid decision makers with the ability to understand the uncertainties associated with the information are always relevant in this space. Moreover, continuing the investigation of how uncertainty can be presented can be useful as new technology such as IoT and IoBT are integrated into sources of data for decision making.

Graphical representations can be extremely useful for presenting uncertainty. Quantitative analysis uses techniques such as analysis of variance, point estimates, confidence intervals, and least squares regression to determine relationships with the data’s independent and dependent variables. These techniques can use a variety of graphs such as scatter plots, histograms, probability plots, box plots, and spectrum plots. For example, in regression analysis the estimated relationship between variables can be shown between the conditional expectation of the dependent variables and fixed independent variables. If there is a linear relationship between the variable, $y_{i}$ and dependent variable, $x_{i}$, parameters, $\alpha$ and $\beta$ with error, $\varepsilon_{i}$ their relationship can be described with the following equation:

$$ y_{i} = \alpha x_{i} + \beta + \varepsilon_{i} ,i = 1, \ldots ,n $$

Applying a confidence band can represent the uncertainty of the function based on limited or noisy data. Figure 1 shows an example of a scatter plot with confidence bands with probability of 0.95 and a collection of confidence intervals constructed with coverage probability of 0.95.

Fig. 1.

Example graph of confidence bands for an example of a linear regression analysis. (wiki: https://en.wikipedia.org/wiki/Confidence_and_prediction_bands)

Uncertainty in this case can be calculated using equations of the distribution to reflect uncertainty in the estimation, probability of a poor fit, or sample noise across the data. In the case of a scatter plot, as in example in Fig. 1 where a trend line is shown that represents the function used to estimate the actual data. Seeing how the sample data varies in reference to the model or estimated function can provide a visual representation of the uncertainty, particularly if your model is being used for the prediction of future data points. Ideally, there would be no divergence between the output generated by the model and the future data but this is not always the case. A simple graph as in Fig. 2 can be used to show this divergence of data to imply that there is an increased uncertainty.

Fig. 2.

An example of data that diverges from expected values. (https://www.mathworks.com/matlabcentral/answers/249350-find-diverging-point-of-two-arrays)

Graphs that show error bars to represent variability can visually indicate error of uncertainty in measurement. Figure 3 is an example [8]. One caution is to ensure that the true value is not lost in presenting the uncertainty. Yau’s paper discusses the challenge from a data statistics point of view where standard errors, confidence intervals, and likelihoods are hard to capture graphically. In traditional approaches usually a range or confidence interval is shown where the middle is the mean or median and a line shows other possible values. In this approach there can be a loss of details in the data and explanations are sometimes needed for the confidence intervals [8].

Fig. 3.

An example graph with error bars.

Other traditional graphical representations include boxplots, histograms, and scatter plots. The boxplot presents the minimum and maximum range of data values, the upper and lower quartiles, and the median. Confidence levels are shown by changing the width of the plots. The width of the box is related to the size of the data, if notches are present they are related to the significant differences of the medians.

Another visualization option is where a distribution shows variation in a sample or spread of possible values or decisions. In this option multiple outcomes are presented as lines that represent all the different paths indicating that there is not just one outcome, but the use of too many lines or paths can overwhelm the information itself. Also, additional description is need to understand what the variations mean and the uncertainty connection to the individual options. An example graph is in Fig. 4 [8].

Fig. 4.

An example showing spread in values or decisions.

Animation, gitter gauges are also ways to show uncertainty. Even simulations where different results are shown one at a time to create an overall picture can be used but losing the thread of uncertainty across results can occur. Adding obscurity to indicate uncertainty by using transparency, color scale, or blur as the outcomes increase or as the outcomes have different uncertainties is possible. Figure 5 shows an example; this approach requires interpretation of the fuzziness or opaqueness [8].

Fig. 5.

An example visual using transparency to represent uncertainty.

Using a descriptive statement such as better than, about, we doubt, unlikely, etc. is another way to represent uncertainty. This technique requires interpretation and consensus of the meanings associated with each term [8].

Also, a numerical value for uncertainty can be calculated and presented at the time of decision making. This value can inform the decision makers on the state of information they are using, for example information from a set of IoBT devices. This number could use a scale from 0 to N to represent the level of uncertainty, where 0 is the lowest level of uncertainty. This approach was selected for the pre-pilot study discussed in the following section.

4 Uncertainty Measure in IoBT

4.1 Motivation

For the military “discovery, characterization, and tracking of relevant, available and useful things, dynamically in time and space” [2] are part of the information gathering process for making decisions. IoBT can be used to provide this information which can be categorized as sources from people, buildings, and the environment. The behaviors and characteristics of the devices and the network are definitely needed prior to making a decision and as decision changes throughout a task or a mission. In addition, the vast amounts of IoBT data, the information sharing criteria, and the mission relevance for this data needs to be considered for decision making. Along with this is the need for any uncertainty associated with this data to be provided to the decision maker. How this data is presented can influence the usefulness of the data and how it impacts the decision. As a first step towards looking at the usefulness of presenting uncertainty of this information as a numerical value a pre-pilot study was conducted and discussed in the following section.

5 Pre-pilot User Study Using the Uncertainty of Information Value

5.1 Description

The pre-pilot user study was the first attempt for us to investigate how the presentation of uncertainty can support decision making, which may be applicable to military situations. In this study, at each decision point information based on a notional IoBT device informs to the participant on which direction to take along a route. The information is a simulated notice from the IoBT device of the state of the route for a given direction. For example, if a right turn along the route is clear then the report says that this direction should be taken. Part of the study is to also present this report with an uncertainty value that indicates any uncertainty related to the information from the IoBT device. The primary research question was to determine if including information from the IoBT data to soldiers along with a number that represented the uncertainty of the data would affect the decision to take action in a helpful way. The number was presented as a value but was not identified as a probability. The higher the number, the more uncertain the information. This uncertainty value was incorporated into the interface of the software for the task.

The experiment followed a within-subjects design consisting of two blocks. Each block consisted of four trials, two with the uncertainty of information value and two without. Participants were asked to complete the simple task. All participants were shown a series of routes with multiple decision points.

A random number was assigned to the participants and no additional demographic information was collected from the participants for this pre-pilot user study. Upon arrival, participants were given an introduction describing the study. If they chose to participate, participants who volunteered to do the study were provided with consent forms and were divided into two groups. They were all seated at tables and given a tablet with the ATAK software installed. Participants were then given instructions on how to use the tablet and software (including the interface).

ATAK is a mapping engine that allows for precision targeting, intelligence on land, navigations, and situational awareness. This system is under development by a variety of government laboratories. Navigation using GPS maps can be overlaid with symbols and Cursor on Target data to show situational awareness of events. The fictional route was displayed in the ATAK software on the tablet. At selected points along the route a window displayed information describing whether the path was blocked (with and without a number that represents the uncertainty of the information). At the end of the trials the participants were debriefed and dismissed.

The modified ATAK interface consisted of three components: (I) The ATAK Map display, in which the route was shown (Fig. 6); (II) The Information Dialog Box, which presented the information from the device with and without the uncertainty values (Fig. 7); and (III) a timer indicating remaining time to complete objective.

Fig. 6.

A figure caption is always placed below the illustration. Short captions are centered, while long ones are justified. The macro button chooses the correct format automatically.

Fig. 7.

Figure showing presentation of information for the decision points in the study.

5.2 Results and Discussion

Data from two participants were taken to first determine if there was a difference in the decisions made with and without the uncertainty of information as a numerical value. For each participant there are four decision points per trail with a total of four trails. This selected sample data shows two of the decisions were the same for one of the participants but none of the decisions were the same for the other participant. From all the samples analyzed only 10% of the decisions were the same. This implies the uncertainty of information value presented numerically possibly changed the decisions. Of course, there are many influences to the decisions the participants made, from the vigilance during the study to biases when making this type of simple decision.

After completion of the task in ATAK, a questionnaire was given and one of the questions was what additional information was needed to make a better decision. Most of the participants wanted additional insights to the value in terms of what was behind the value that made it either high, medium, or low. In future user studies this information will be incorporated and presented with the value.