A maritime decision support system to assess risk in the presence of environmental uncertainties: the REP10 experiment

Abstract

The aim of this work is to report on an activity carried out during the 2010 Recognized Environmental Picture experiment, held in the Ligurian Sea during summer 2010. The activity was the first at-sea test of the recently developed decision support system (DSS) for operation planning, which had previously been tested in an artificial experiment. The DSS assesses the impact of both environmental conditions (meteorological and oceanographic) and non-environmental conditions (such as traffic density maps) on people and assets involved in the operation and helps in deciding a course of action that allows safer operation. More precisely, the environmental variables (such as wind speed, current speed and significant wave height) taken as input by the DSS are the ones forecasted by a super-ensemble model, which fuses the forecasts provided by multiple forecasting centres. The uncertainties associated with the DSS’s inputs (generally due to disagreement between forecasts) are propagated through the DSS’s output by using the unscented transform. In this way, the system is not only able to provide a traffic light map (run/not run the operation), but also to specify the confidence level associated with each action. This feature was tested on a particular type of operation with underwater gliders: the glider surfacing for data transmission. It is also shown how the availability of a glider path prediction tool provides surfacing options along the predicted path. The applicability to different operations is demonstrated by applying the same system to support diver operations.

1 Introduction

Military and civilian maritime operations are both deeply affected by METereological and OCeanographic (METOC) conditions putting human safety, involved assets and the environment at serious risk. To be successful in the dynamic maritime environment, improvement of the general situational awareness (SA) of mission planners and decision makers is a key. In particular, improving environment SA by providing information and tools to correctly assess the future impact of the weather and the ocean state on operations, deeply contributes to reducing the risk of mission failure due to environmental conditions falling outside the allowed operational constraint intervals.

A considerable number of operational failures and important accidents have been due to human factors, in particular lack of SA of operators, and due to bad environmental conditions (Grech and Horberry 2002; Clifford and Ah 2004; de la Campa 2005). The main causes of these failures and accidents span the three levels of the classical SA model of Endsley (1995, 2000, 2004), which are perception, cognition and projection of events in the future. For example, poorly designed observational sensor networks, data scarcity and data misperception are factors contributing to incorrectly perceived information by the decision maker. A large amount of time- and spatial-varying data from several sources and associated uncertainty prevents the decision maker from correctly integrating or comprehending information and from making correct projections of present trends. In general, all these factors cause humans to be unable to make informed decisions, increasing the risk of failure.

To allow METOC information to play a role in the mission planning process, a significant effort in observing and forecasting atmospheric and oceanic variables and associated uncertainty is essential, especially in those highly dynamic areas such as the littoral zone. Moreover, to further improve environmental SA, decision support systems (DSS) that are able to fuse and analyse a massive amount of METOC forecasts and measurements, collected from a number of distributed METOC centres have to be provided to the decision maker. Environmental risk assessment given mission operational constraints (that is, thresholds on METOC variables elicited by domain experts), automatic hypothesis ranking, and decision making on the course of action are value-added products that synthetically describe and predict the situation in the operational field and decrease the heavy cognitive work load on the decision maker. Exploitation of data uncertainty associated with METOC predictions and measurements makes it possible to give a level of confidence to the DSS outcomes to further improve SA.

1.1 The REP10 experiment

The 2010 Recognized Environmental Picture experiment (REP10) took place in the Ligurian Sea from 19 August to 3 September 2010. The sea trial was focused on the exploitation of a variety of observational assets, including remote sensing satellites, underwater gliders (Eriksen et al. 2001; Schofield et al. 2007), drifters, HF radar and moorings, to extract near-surface geophysical parameters and physical and bio-optical properties in the water column. All these measurements can be assimilated into bio-optical and physical METOC models providing an integrated approach for near real-time METOC data collection and modelling. The uncertainty associated with METOC measurements and forecasts has been evaluated so that it can be exploited by decision support tools that are able to propagate input uncertainty and give a confidence score to their outcomes as detailed in the following sections.

1.2 Related works on environmental DSSs

Senne and Condon (2007) presented the Lincoln Laboratory initiative in integrated sensing and decision support (ISDS), a decision support model combining key elements of the observe-orient-decide-act (OODA) loop, the military decision making process and the recognition-primed decision models. They highlighted the necessity of integrating improved observational capabilities with automatic intelligent decision support tools in order to analyze the increasing amount of information available to the decision makers and thereby improve their responsiveness and ability to make informed decisions. Examples of the use of the ISDS model in several fields were reported, including air traffic control applications in which weather forecast models were used to extend the time horizon of air-route traffic planners under severe weather conditions.

Regnier (2007) discussed main gaps between systems and end-users preventing the proper use of METOC information in DSSs and highlighted important research topics to be investigated in order to cover the gaps, including tools for decision-centric end-to-end systems, cognitive aspects of weather forecasting, production of stochastic forecasts and use of uncertainty in DSSs.

The economic value of METOC probabilistic forecasts, as predicted by ensemble techniques, was analyzed by several authors, such as Zhu et al. (2002), demonstrating through a cost-loss model, the benefit of using ensemble predictions compared to deterministic ones. The analysis was conducted without taking into account possible additional benefits provided by automatic DSSs.

Decision support tools are widely used in several fields such as operation planning, transport, medicine, finance, environmental resource planning and crisis management at strategic, tactical and operational levels (Jarre et al. 2008; Liu and Lai 2009; Acosta et al. 2010; Apipattanavis et al. 2010; Mensa et al. 2011). In these tools, several approaches to risk assessment and automatic decision making are used. Rule-based fuzzy logic expert systems (Ross 2010) have several advantages with respect to other techniques. High interpretability, the possibility to deal with objective and/or subjective knowledge elicited from experts, application in the presence of uncertainty and vagueness, are, among others, the most remarkable characteristics making fuzzy systems highly preferable.

Aiello et al. (2009), Balmat et al. (2009), and Tavana and Bourgeois (2010) are examples of interesting works on fuzzy-based DSSs for risk assessment in maritime operations in which the risk is assessed by analysing the impact of dynamical environmental and static non-environmental factors. The environmental part of their proposed architectures does not make use of stochastic forecasts and therefore predicted uncertainty on METOC input variables is not exploited to improve SA of the end user.

The NATO Undersea Research Centre (NURC) has been conducting research on tactical DSSs for assessing the impact of the environment on maritime operations since 2002. A first architecture based on fuzzy systems coupled with web-based geographical information systems (GISs) for data management and dissemination was proposed in Grasso and Giannecchini (2006). Since then, the system has been incrementally updated (Grasso 2009; Grasso et al. 2010b) up to the latest version presented in Grasso et al. (2011) and tested during REP10 as reported in the present work. Improvements with respect to previous works include fusion of METOC predictions from different centres and production of stochastic forecasts as well as efficient and explicit use of the predicted uncertainty through the unscented transform (UT) to improve performance of the decision makers.

1.3 Main contributions of this work

The DSS component of the REP10 trial allowed the demonstration of an extended concept of a recognised environmental picture, which exploits the acquired environmental knowledge at higher levels of abstraction of decision support models such as the OODA loop.

The experiment provided a rare opportunity to test the generic maritime operation DSS described in Grasso et al. (2011) during a real experiment at sea involving a fleet of underwater gliders and operational METOC forecast models from different sources. In particular, the system has been used to automatically deliver DSS products in near-real time, which are now available to the REP10 partners for post analysis and validation. Moreover, an initial investigation activity involving human experts has already started, aiming at assessing human factors in decision support systems that use METOC data. This activity is providing useful feedback, which is helping to improve the system and define new requirements.

1.4 Organisation of the paper

Section 2 of this paper describes the architecture of the DSS used during REP10 with a minimal level of technical details. Section 3 describes the part of the REP10 experiment related to the use of the DSS. Section 4 describes the experimental scenario in terms of environmental conditions and Section 5 presents the outcomes. Section 6 offers conclusions and future work.

2 System description

The architecture of the integrated DSS is depicted in Fig. 1. The DSS comprises two sub-systems:

Fig. 1

The integrated DSS architecture comprises a super-ensemble METOC forecast system and a general purpose environmental DSS based on a fuzzy/Bayesian inference engine

• A super ensemble (SE) stochastic forecast system that fuses measurements and METOC predictions from N METOC centres, and measurements and produces a forecast of METOC statistics such as mean and covariance

• A DSS to support generic operations based on a fuzzy/Bayesian rule-based expert system, which calculates the risk of performing an operation due to the environment, proposes a course of action and assigns a confidence level to decisions by propagating the input uncertainty through the entire system by the UT

The fuzzy/Bayesian DSS will be described in the following subsections, mainly reviewing the concepts outlined in Grasso et al. (2010b) and Grasso et al. (2011), with additional details, in particular those specific to the REP10 experiment.

2.1 The super-ensemble forecast system

The SE multi-model technique combines several predictions provided by different models to produce a single forecast and uncertainty. The technique is based on the optimization, in terms of distance to observations, of the weighted linear combination of the input models during a specified learning period. The weights are then used to combine the corresponding model forecasts. The 3D super-ensemble (3DSE) described in Lenartz et al. (2010), which allows spatially varying model weights, has been used. The present implementation in the Ligurian Sea provides 72-h predictions of ocean temperature, horizontal ocean currents, significant wave height, and 10-m wind velocity. The prediction of the different parameters is the result of independent 3DSE simulations. Surface temperature data from the Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA) system (Stark et al. 2007) is assimilated during the learning period, together with ocean temperature profiles from gliders and conductivity–temperature–depth stations. Mourre et al. (2011) evaluated the accuracy of these 3DSE temperature predictions, showing RMS errors of 0.40°C against future OSTIA analysis and 0.77°C against independent REP10 in situ temperature profiles. Below the sea surface, the 3DSE was found to provide similar temperature forecast skills as the most accurate of the models and the model ensemble mean, whereas the 3DSE skills were better at the surface.

The 3DSE provides an associated uncertainty forecast (Mourre et al. 2011) when observations are assimilated during the learning period, which is the case of ocean temperature in this study. This uncertainty is estimated from the product of the model predictions by the a posteriori model weight error covariances computed through the 3DSE analysis. This uncertainty, which depends on (1) the location and error statistics of the observations assimilated during the 3DSE learning period, and (2) the a priori weight error covariances (specified as distance-dependent in this study), is lower in the area spanned by the assimilated measurements. When there are no observations assimilated (as is the case for ocean currents, significant wave height and winds), the 3DSE is the mean forecast over the ensemble of models, while an estimate of the associated uncertainty is given by the standard deviation of the ensemble.

2.1.1 The forecasting models used by the 3DSE in REP10

The set of METOC operational numerical systems considered during the REP10 experiment consists of two atmospheric forecasting systems, four surface gravity wave forecasting systems and three ocean forecasting systems. All the systems provided forecasts in real time with at least 3 days of forecast range. Although some institutions release more than one forecast per day, only runs with 00:00 UTC reference time have been used in the super-ensemble. The systems are reported in Appendix 1.

2.2 The fuzzy/Bayesian DSS

The DSS for generic operations is depicted in Fig. 2. The core of the system (Grasso et al. 2010b) is a risk calculator based on a hybrid fuzzy/Bayesian inference engine (Ross 2010). This block calculates the risk of performing an action, selected from a pre-defined list of actions, under given environmental conditions described by a set of METOC variables predicted by the SE model. The risk calculator is wrapped by the UT that is used to propagate input METOC statistics (mean and covariance) and predict mean and covariance of the action risks (Grasso et al. 2011). The risk statistics are then used as inputs to the action selection rule and the confidence calculator. The former simply chooses the action having the minimum mean risk, while the latter analyses the mean and the covariance of the action risks to assess the statistical separability of the actions and give a confidence level to the final decision. Each component of the system depicted in Fig. 2 will be described in greater detail in the following sub-sections.

Fig. 2

The schematic of the fuzzy/Bayesian DSS for generic operations

2.2.1 The action risk calculator

Figure 3 depicts the action risk calculator in detail. The system couples the flexibility of a fuzzy logic system in encoding expert knowledge and dealing with vague concepts and the rigorous formulation of the Bayesian minimum risk decision theory.

Fig. 3

The schematic of the fuzzy/Bayesian action risk calculator

The fuzzy classifier assigns subjective conditional probabilities, $P\left( {{\omega_m}\left| {{\mathbf{x}}_{{ij}}^t} \right.} \right)$ (Anscombe and Aumann 1963), to a set of environmental decision classes, $\Omega \equiv \left\{ {{\omega_1}, \ldots, {\omega_M}} \right\}$, given a vector of F METOC variables, ${\mathbf{x}}_{{ij}}^t \equiv {\left[ {{x_1}, \ldots, {x_F}} \right]^T}$ (also called environmental factors, such as significant wave height and wind speed), affecting the operation to be supported. The indices (i, j) represent discrete positions on a spatial grid, which covers the geographical area of interest (in this paper the grid is in latitude and longitude on the sea surface), while t is a temporal index indicating a time instant in the future.

In the specific case of this work, the input space is partitioned into three classes, Ω ≡ {ω ₁ = favourable, ω ₂ = marginal, ω ₃ = unfavourable}. The ω ₁ class represents environmental conditions in which the operation can safely run, while ω ₃ is the class having METOC values in a range such that it is unsafe to run the operation. The ω ₂ class represents intermediate situations between the two extreme classes ω ₁ and ω ₃. It indicates METOC ranges for which the decision of running or not running an operation should be postponed until a clear understanding of the environmental condition is available. The outputs of the system are the three class subjective conditional probabilities ${\mathbf{p}}_{{ij}}^t = {\left[ {\matrix{{*{20}{c}} {P({\omega_1}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} & {P({\omega_2}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} & {P({\omega_3}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} \\ } } \right]^T}$. The rule base of the fuzzy classifier depends on the considered operation. It embeds knowledge elicited from experts of the domain (including the types of METOC variables to be considered and the operational constraints on those variables) as a series of if-then rules (Ross 2010). Appendix 2 describes the fuzzy classifier in greater detail including the type of inference used and considerations about the elicitation of system parameters.

The risk associated with an action α _l ∈ A, where A ≡ {α ₁,…,α _L }, in the Bayesian sense, is calculated (see the risk calculator in Fig. 3) as the average conditional cost of deciding α _l given ${\mathbf{x}}_{{ij}}^t$, over the set of environmental classes, Ω:

$$ r({\alpha_l}\left| {{\mathbf{x}}_{{ij}}^t} \right.) = \sum\limits_{{{\omega_m} \in \Omega }} {\lambda ({\alpha_l}\left| {{\omega_m}} \right.) \cdot P({\omega_m}\left| {{\mathbf{x}}_{{ij}}^t} \right.)}, $$

(1)

where, λ(α _l |ω _m ) is the conditional cost of deciding for the action α _l given the environmental class ω _m .

In this work, the action list A is composed of two elements, A ≡ {α ₁ = run operation, α ₂ = not run operation}. The conditional costs, λ(α _l |ω _m ), can be arranged in this case in a L = 2 by M = 3 matrix as follows:

$$ {\mathbf{\Lambda }} = \left[ {\matrix{{*{20}{c}} {\lambda ({\alpha_1}\left| {{\omega_1}} \right.)} & {\lambda ({\alpha_1}\left| {{\omega_2}} \right.)} & {\lambda ({\alpha_1}\left| {{\omega_3}} \right.)} \\ {\lambda ({\alpha_2}\left| {{\omega_1}} \right.)} & {\lambda ({\alpha_2}\left| {{\omega_2}} \right.)} & {\lambda ({\alpha_2}\left| {{\omega_3}} \right.)} \\ } } \right]. $$

(2)

The conditional cost matrix is specified by experts of the domain. It should take into account, for instance, actual costs of assets and equipment involved in the operation and subjective judgement of experts with regards to their preferences about risk.

An estimate of risk for running the operation and for not running the operation for every input METOC environmental variable is now available. The next steps include the propagation of uncertainty through the risk calculator, the action selection and the decision confidence as detailed in the following sub-sections.

2.2.2 Propagation of the input uncertainty by the UT

The risk calculator can be considered as a non-linear random vector mapping between the input space of METOC factors, ${\mathbf{x}}_{{ij}}^t \in {\mathbb{R}^F}$ and the action conditional risks, ${\mathbf{r}}_{{ij}}^t = {[r({\alpha_1}\left| {{\mathbf{x}}_{{ij}}^t} \right.), \ldots, r({\alpha_L}\left| {{\mathbf{x}}_{{ij}}^t} \right.)]^T} \in {\mathbb{R}^L}$. The UT is used to propagate the statistics of ${\mathbf{x}}_{{ij}}^t$ through the risk calculator up to the second order. In particular, the mean METOC factor vector, $\overline {\mathbf{x}}_{{ij}}^t$, and the covariance matrix, ${\mathbf{X}}_{{ij}}^t$, predicted by the SE model (see Fig. 2) are the inputs to the UT wrapping the risk calculator. The UT is an efficient way to propagate second order statistics (mean, covariance and cross-covariance) of a random variable passing through a non-linearity of any order (Julier and Uhlmann 2004; Van der Merwe 2004). The UT estimates the statistics of the output random vector knowing the statistics of the input by approximating the true probability density function of the input with a Gaussian and making use of a deterministic sampling procedure that is more efficient than classical random sampling Monte Carlo techniques. Advanced versions of the technique use information on higher order moments of the input distribution, resulting in an improved estimate of mean and covariance of the output (Julier and Uhlmann 2004). These features will be used in future work for those cases in which the input presents highly non-Gaussian characteristics.

The output of the risk calculator is evaluated for a set of so-called sigma points:

$$ \matrix{{*{20}{c}} {{\mathbf{sp}}_{{ijk}}^t \equiv \left\{ {{w_k},\chi_{{ijk}}^t} \right\}} & {k = 1,...,2F + 1} \\ } $$

(3)

with $\chi_{{ijk}}^t \in {\mathbb{R}^F}$. The UT deterministic sampling scheme generates the samples $\chi_{{ijk}}^t$ in the input space by using the METOC statistics and the weights w _k . The same weights and the risk calculator outputs evaluated for each $\chi_{{ijk}}^t$ are then used to estimate the statistics of the output by means of sample weighted averages as reported in Appendix 3.

As specified by Eq. 3, the UT makes use of S = 2 F + 1 sigma points to estimate the output statistics (in case of 4 inputs, as in the present work, the number of sigma points is just 9). In those cases for which only the input statistics are available, we need a method to resample and propagate the uncertainty through the system. In case of a large ensemble with N _e  > S (N _e being the number of members of the ensemble), calculating the ensemble statistics and then using the UT is more efficient than providing the ensemble members directly to the DSS system and then estimating the output uncertainty. Moreover, the data transfer from a remote METOC centre would benefit if only the statistics are transmitted through the network. Furthermore, in the case where an SE model with assimilated measurements is used, propagating each single model used in the SE and then estimating the output uncertainty is different than propagating the statistics of the SE which are a combination of models and measurements. This paper presents the system taking into account the more general scenario even though the number of models available to the SE is less than S and no measurements can be assimilated into the SE. The DSS concept provided in this work is valid in any case, independently of the quality of the input statistics. In case of poor quality statistics, the operator may still be interested, for example, in performing a what-if or sensitivity analysis by testing different levels of uncertainty and bias to compare different future scenarios. The user can either trust the system if the scenarios are not significantly different, for example, or decide on the basis of experience and constraints of a different nature, beyond those considered in the DSS.

The estimated risk mean vector, $\overline {\mathbf{r}}_{{ij}}^t = {[\overline {r({\alpha_1}\left| {{\mathbf{x}}_{{ij}}^t} \right.)}, \ldots, \overline {r({\alpha_L}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} ]^T} = {[r_{{_{{ij1}}}}^t,...,r_{{ijL}}^t]^T}$ and the risk covariance matrix, ${\mathbf{R}}_{{ij}}^t$, at the output of the UT (recall that for an action list of two actions, which is the case examined in this work, the risk mean vector is 2 × 1 while the covariance matrix is 2 × 2) are used in the action selection and confidence level calculation steps as described in the following sub-section.

2.2.3 Action selection and decision confidence

As originally proposed in Grasso et al. (2010b), the system chooses, according to the classical Bayesian decision theory (Duda et al. 2000), the action having the minimum conditional risk:

$$ \widehat{\alpha }_{{ij}}^t \equiv \arg \,\mathop{{\min }}\limits_{{{\alpha_l} \in A}} [r({\alpha_l}\left| {{\mathbf{x}}_{{ij}}^t} \right.)], $$

(4)

This step is postponed, as proposed in Grasso et al. (2011), until the UT (see Fig. 2) in order to propagate the input uncertainty as described in the previous section. The final architecture is then able to exploit information from multiple METOC centres and the uncertainty associated with METOC forecasts so as to improve the SA of the mission planners.

The final decision law is obtained by substituting $r({\alpha_l}\left| {{\mathbf{x}}_{{ij}}^t} \right.)$ in (4) with the mean risk vector components, $\overline {r({\alpha_l}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} = r_{{ijl}}^t$, at the output of the UT:

$$ \widehat{\alpha }_{{ij}}^t \equiv \arg \,\mathop{{\min }}\limits_{{{\alpha_l} \in A}} [\overline {r({\alpha_l}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} ]. $$

(5)

The decision confidence step detects those cases in which the actions are not statistically separable as a consequence of too much uncertainty in the METOC forecast inputs. As proposed in Grasso et al. (2011), the system estimates a decision confidence measure, $cm_{{ij}}^t$, by comparing the 95% action risk confidence intervals, which are calculated by using the diagonal elements of the risk covariance matrix, ${\mathbf{R}}_{{ij}}^t$ and the mean vector, $\overline {\mathbf{r}}_{{ij}}^t$, both estimated by the UT. The confidence measure $cm_{{ij}}^t$ is a heuristic index which was inspired by methods like the Tukey’s multiple comparisons procedure or the Gabriel’s test, which find application in testing if several groups of samples belong to the same statistical population (Hayter 1984; Benjamini and Braun 2002). Future developments of the system will improve the confidence measure by considering a formal hypothesis test procedure and correlation among risks. If $r_{{ij\,Up}}^t = r_{{ij\sigma (1)}}^t + {\Delta_{{\sigma (1)}}}$ is the upper limit of the confidence interval associated to the chosen action and $r_{{ij\,Lo}}^t = \mathop{{\min }}\limits_{{p > 1}} \left( {r_{{ij\sigma (p)}}^t - {\Delta_{{\sigma (p)}}}} \right)$ is the minimum of the lower limits associated to the remaining actions (p = 1,…,L and σ(p) is the permutation of the action indexes, l = 1,…,L, induced by the rank of the mean risks in ascending order), the difference $cm_{{ij}}^t = r_{{ij\,Up}}^t - r_{{ij\,Lo}}^t$ or its normalised value, for example by the square root of the sum of risk variances, is the confidence given to the decision. Confidence to the selected action is given for positive values of $cm_{{ij}}^t$, meaning that the confidence intervals of the action risks are not overlapped (see Fig. 4 for an example with two actions). In addition, a binary confidence map, $c_{{ij}}^t$, is produced as well by detecting negative and positive values of $cm_{{ij}}^t$:

Fig. 4

One-dimensional spatial example of action risk 95% confidence levels. In this case, $\matrix{{*{20}{c}} {\overline {r({\alpha_1}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} < \overline {r({\alpha_2}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} } & {\forall i} \\ }$ and the system selects the action α ₁ in the entire spatial region. A region with no confidence is detected due to overlapping risk confidence intervals (see shaded area)

$$ c_{{ij}}^t = \left\{ {\matrix{{*{20}{c}} 0 & {cm_{{ij}}^t \leqslant 0} \\ 1 & {cm_{{ij}}^t > 0} \\ } } \right., $$

(6)

helping the mission planner to easily locate confident and non-confident decisions in space and time. In this way, the system provides an additional level of decision, the confidence map, $c_{{ij}}^t$, to inform the user that the system itself is not able to make a final decision due to statistically contradicting information. The user, in these cases, can either accept the proposed action or distrust the system and decide on the basis of his experience, different criteria and the operational context.

2.2.4 Considerations with respect to decision theory and future improvements of the DSS

The system presented in this paper chooses the action with the minimum expected cost (or loss) with respect to the set of environmental classes. A negative action/class cost is also allowed meaning that a gain is achieved when the action is performed under that specific class. In the present work, the costs are not elicited, but are chosen to be consistent with monotonicity constraints λ(α ₁|ω ₁) < λ(α ₁|ω ₂) < λ(α ₁|ω ₃) and λ(α ₂|ω ₁) > λ(α ₂|ω ₂) > λ(α ₂|ω ₃) as reported in Subsection 3.4 (Grasso et al. 2011). In case real costs are provided (cost of assets and equipment for instance) the system is risk neutral. The cost of each action/class pair, if intended in a broader sense, could also reflect the subjective judgement of human experts even though the correct formulation to integrate user risk preferences into the system is given by the maximisation of the expected utility (Kreps 1990). Due to the flexibility of the proposed DSS architecture, several other approaches to decision under risk could easily be integrated, not only the expected utility theory, but also alternatives such as a generalisation of expected value with mode or median of payoffs (Yager 2004) and prospect theory (Kahneman and Tversky 1979). Future versions of the system may include these principles to further improve operator efficiency and effectiveness.

In order to achieve this goal, the challenging issue of modelling the decision maker’s risk attitude and eliciting his preferences as utility or prospect value functions has to be faced. Perception of risk by humans depends on the context and the application domain. In a military context, for example, in which extreme situations have to be handled, Haerem et al. (2010) observed a risk-seeking behaviour in both loss and gain domains due to self-efficacy of the decision maker, deviating from prospect theory original findings (Tversky and Kahneman 1981) in which decision makers are influenced by the gain and loss framing. In general, no final consensus has been reached on risk preferences in a military context and specifically, the authors of the study suggest further investigation to confirm their conclusions. Moreover, no clear understanding exists on how METOC operators make decisions under conditions of uncertainty.

During the REP10 experiment, a preliminary investigation (Grasso et al. 2010d) was conducted in collaboration with METOC officers of national navies with the aim of providing information on their operational working procedures and testing the present system. GIS clients were used as a tool to overlay DSS product grids. The METOC officer activity was automatically tracked using web-logging in order to have data on utilisation of METOC forecasts and DSS products, to be analyzed during the post-experiment phase. In general, DSS products, such as traffic light maps, are able to reduce the information overload, improving METOC officer performance in generating hypotheses on the course of action in space and time over the area of interest. The experiment was not able to produce definitive results on the cognitive behaviour of METOC officers and on their preferences versus risk, but it provided the foundation for further investigation. A future long-term effort is envisioned to develop requirements for future DSSs and include risk preferences of METOC operators and final decision makers. The investigation will adopt a cognitive engineering approach which includes subject matter expert interviews and knowledge audits, task and sense making analyses and collection of METOC operator performance in situ data.

3 The DSS experiment within REP10

The DSS experiment within REP10 demonstrated the architecture presented in Section 2 in a real operational context and showed how METOC information can be used to provide value-added products to the decision makers. The experiment focused on supporting underwater glider operations for several reasons. Firstly, gliders were an important component of REP10, the success of which was largely dependent on how well the vehicles were operated. In general, glider missions are at risk for several reasons and specifically designed decision support tools provided to the mission planners and vehicle pilots contribute to the mission success by mitigating the risk. In this sense, the REP10 experiment provides a valuable data set that can be exploited to study, design and implement integrated decision support systems for glider mission planning and control. Secondly, the system’s ability to support several phases typically involved in operations with gliders can be tested including deployment, recovery, navigation, surfacing and data transmission, each one affected by the environment in a different way, demonstrating the flexibility of the proposed decision support architecture. Moreover, the operational phases are affected by several critical factors such as wind velocity, sea current speed, wave height and period, rain and maritime traffic, which have to be simultaneously fused to support a particular operation phase. Supporting glider operations during REP10 provides an opportunity to show the architecture working with multiple inputs and outputs. Finally, the personnel operating the vehicles during REP10 and previous NURC experiments (Grasso et al. 2011), which includes experienced pilots and engineers, provided valuable feedback to help define the system rule base and parameters. Input from experienced pilots and engineers contributes to understanding the psychological aspects and human factors related to the use of METOC information in decision support (Grasso et al. 2010d), a topic which will be further investigated in future work.

3.1 The glider operating area

Figure 5 shows the area of interest and in particular the 60 × 90 km boundary (see the rectangle in blue) in which a fleet of seven underwater gliders was allowed to operate during the trial. The vehicles were required by the local maritime authority to navigate in the highlighted area. Given these physical limitations, it was important to monitor the mission over the entire experiment and predict possible strong currents with enough notice for the pilots to keep the gliders within the allowed area. Two gliders navigated along rectilinear transects (black lines in Fig. 5) to collect measurements to initialize and validate METOC forecast models. The rest of the fleet was exploited to adaptively sample the area and measure those regions where the METOC models were affected by a greater estimated prediction error. The measurements, when possible, were assimilated into those models to refine the forecast and reduce the prediction error.

Fig. 5

REP10 area of operations in the Ligurian Sea. The blue box is the allowed area for glider operations. Glider paths along two rectilinear transects (black) are shown in red and magenta. An example of adaptive sampling path is shown in green

3.2 Glider operations

During the experiment, a set of glider operations was considered including glider deployment, recovery and surfacing for data transmission. Two other kinds of operations, diver operations and naval refuelling, were also considered to further demonstrate the general purpose capabilities of the system.

The METOC critical factors affecting glider operations and associated operational constraints were elicited by means of an informal pre-analysis of the problem with the collaboration of domain experts. A formal knowledge elicitation process, using questionnaires submitted to a group of experts (Acosta et al. 2010), will be implemented in future works. An initial formalisation of this process was started during REP10 (Grasso et al. 2010d), but it needs further improvements and testing. For this reason, the DSS rule base and conditional cost matrix used during REP10 to support glider operations are likely to be updated in future work.

Glider deployment and recovery operations using small rubber boats either starting from a glider port or a mother ship, expose humans and equipment to serious risk. The critical factors affecting the operations are both non-METOC and METOC related. The experience of the crew and the type of the boat are typical non-METOC information that should be taken into account during the planning of the mission. METOC factors include the significant wave height, the dominant wave period, the wind speed, the rain intensity and thunderstorm probability. High values of the significant wave height are conditions that should be avoided as they significantly impact the rubber boat navigation and the operations to load and unload them from a mother ship. Additionally, the presence of high wind speed might cause the overturning of a boat, depending on the relative direction of the wind with respect to the rubber boat’s course over-ground.

Data transmission after glider surfacing allows the transfer of measurements and their near-real-time assimilation into METOC forecast models to improve prediction skills. Typically, the glider is programmed to periodically surface, for example every 3 h, and transmit data to the command and control centre. During the transmission phase, in which the glider cannot be manoeuvred except for stopping the transmission and re-starting the survey, the glider may drift for long distances if strong currents and winds are present. Moreover, depending on the volume of data, the speed of the communication link and near-real-time requirements, the glider can be at the surface for long periods (more than 30 min) and at risk of collision with other vessels. Finally, reliability of the data link may be strongly affected by the sea state.

3.3 Additional operations: naval refuelling and diver operations

Critical factors and operational constraints for naval refuelling and diver operations were gathered from specialised handbooks on standard METOC operation support, issued by national navy forces (JMH 2000).

Naval refuelling underway, for example of an aircraft carrier by a tanker ship, exposes humans and vessels to serious risk of injuries or damage. The two vessels are connected by steel cables with fuelling pipes alongside. Rough sea state makes it difficult to navigate along parallel routes and keep a secure distance, and thus increases the risk of a cable breaking with disastrous consequences for refuelling personnel.

Diver operations are significantly affected by current speed, temperature, wave height and water visibility. Swimmer capabilities are reduced in strong currents and wave height could impact the diver deployment phase. Depending on the temperature, diver performance can be significantly degraded; for example, in cold water, lack of concentration and hypothermia can occur. Vertical and horizontal water visibility affects diver detection as well as the ability of divers to operate. In general, diver operation support is mainly of interest on a local scale rather than in the open sea, such as the REP10 study area. Nevertheless, in this work, examples of products for diver operation support will be provided to demonstrate the flexibility of the system in fusing information from heterogeneous sources including remote sensing. The DSS can provide operation support at a local scale if local METOC forecast models and high resolution satellite data are available, without modifying the system architecture.

3.4 Setting up the DSS

The setup of the general purpose DSS consists of (1) specifying fuzzy sets and associated membership functions of the input and output variables, (2) populating the rule base, (3) deciding the fuzzy inference method, (4) determining the action list and (5) setting the action conditional cost matrix. These steps need to be completed for each of the supported operations, which are defined during the expert knowledge elicitation phase. The system parameter setup can be found in Grasso et al. (2010b) and Grasso et al. (2011). In particular, the cost matrix was chosen as follows:

$$ {\mathbf{\Lambda }} = \left[ {\matrix{{*{20}{c}} {10} & {70} & {200} \\ {100} & {30} & {10} \\ } } \right]. $$

(7)

The configuration of the costs is such that in the limiting cases the conditional probability vector is ${\mathbf{p}}_{{ij}}^t = {\left[ {\matrix{{*{20}{c}} {P({\omega_1}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} & {P({\omega_2}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} & {P({\omega_3}\left| {{\mathbf{x}}_{{ij}}^t} \right.)} \\ } } \right]^T} = {\left[ {\matrix{{*{20}{c}} 1 & 0 & 0 \\ } } \right]^T}$ and ${\mathbf{p}}_{{ij}}^t = {\left[ {\matrix{{*{20}{c}} 0 & 0 & 1 \\ } } \right]^T}$, the preference is given to the actions runOp and notRunOp, respectively, while for ${\mathbf{p}}_{{ij}}^t = {\left[ {\matrix{{*{20}{c}} 0 & 1 & 0 \\ } } \right]^T}$ the decision is notRunOp. When the classes are equiprobable, the preference is given to notRunOp.

The METOC variables used to support operations during REP10 are reported in Table 1 for each operation. The variables include the wind speed from the atmospheric component, the current speed and the water temperature from the oceanographic component and the significant wave height from the surface gravity wave component. The vessel traffic density was estimated from a long series of automatic identification system data. Water visibility to support diver operations was estimated from satellite multi-spectral passive sensor images.

Table 1 Operations supported during REP10 and DSS input METOC variables

As previously mentioned, METOC ensemble predictions were available each day with reference time 00:00:00 for a 72-h forecast period with a sampling interval of 1 h. The DSS was set to provide operational products on the same forecast period with a sampling interval of 6 h. DSS products were ready to be published on the REP10 web site and available to the experiment partners in less than 1 h after receiving the SE predictions. The experimental period includes 15 days from 18 August to 1 September 2010 covering the entire period of active glider operations.

4 The scenario

The previously described DSS has been tested on a subset of data extracted from the REP10 experiment. While the DSS was run for the duration of the REP10 experiment in near real-time, in this work a single case is shown and analysed. The case is considered sufficiently representative of the operating conditions and highlights most of the behaviours experienced during the entire REP10 cruise.

It is worthwhile to note that the proposed DSS assumes that the uncertainty is provided together with the METOC forecasts, no matter how they are generated. This section provides a qualitative description of the environmental scenario for the period under consideration by analysing the statistical, spatial and temporal variability of relevant METOC variables forecast by the super-ensemble. DSS results for the considered operations are then shown and interpreted in the subsequent section.

Figures 6, 7, 8 and 9 show the spatial and temporal evolution of the SE mean of a set of METOC variable forecasts during a 12-h period in the REP10 operational area. The figures also show the position at 43.83° N and 9.11° E of an Ocean Data Acquisition System (ODAS) buoy available to sample local conditions. Sea surface temperature (see Fig. 6) shows some diurnal heating and the existence of a warm pool to the east, associated with the intrusion of southern warm water masses. The temperature ranges between 23°C and 26°C with higher values (25–26°C) in the eastern part of the basin. Figure 7 shows surface currents. Qualitatively, the water circulation in the area is characterised by an overall cyclonic circulation with a main flow from the south-west (west Corsican current) turning counter-clock wise along the Italian coastline and exiting the Ligurian Sea in the northwestern part (the Ligurian current; Millot 1999). The existence of an anticyclonic eddy can also be noted to the southeast. In the forecast considered here as an example, the current speed is found to be approximately 0.2 m/s at +30 h since base time, with maximum values in the southern part of the area around 9° E reaching 0.35 m/s at +42 h. The significant wave height from the corresponding snapshots progressively grows from values less than 1 m to values greater than 1.5 m at +42 h in open sea (see Fig. 8), as southwesterlies grow in intensity (from about 5 m/s at +30 h to values larger than 15 m/s at +42, see Fig. 9).

Fig. 6

Mean sea surface temperature (degree Celcius) 3DSE forecast. Forecast times at +30, +36 and +42 h since base time 26 August 00:00:00 + 00 UTC. The red marker shows the position of the ODAS buoy at 43.83° N, 9.11° E

Fig. 7

Mean horizontal surface current velocity (meter per second) 3DSE forecast. Forecast times at +30, +36 and +42 h since base time 26 August 00:00:00 + 00 UTC. The red marker shows the position of the ODAS buoy at 43.83° N, 9.11° E

Fig. 8

Mean significant wave height (meter) 3DSE forecast. Forecast times at +30 h, +36 h and +42 h since base time 26 August 00:00:00 + 00 UTC. The red marker shows the position of the ODAS buoy at 43.83°N, 9.11°E

Fig. 9

Mean 10 m wind (meter per second) 3DSE forecast. Forecast times at +30, +36 and +42 h since base time 26 August 00:00:00 + 00 UTC. The red marker shows the position of the ODAS buoy at 43.83° N, 9.11° E

Figure 10 shows the temporal behaviour of METOC SE forecasted mean and associated 95% confidence interval for a grid point in the area of interest located at the position of the ODAS buoy (43.83° N and 9.11° E) during the entire 3-day forecast period. The sea surface temperature ranges between 24.5°C and 25.5°C with a first period of increase and a second phase of decrease starting from +15 h since base time with a peak temperature of 25.3°C. The minimum 95% confidence lower limit is about 23°C while the maximum higher limit is about 27°C. Surface current speed has an oscillating behaviour around 0.10 m/s with maximum peak-to-peak amplitude of about 0.15 m/s. Minimum lower confidence limit and maximum higher confidence limit are 0 and 0.6 m/s, respectively. The mean significant wave height (Fig. 10c) is less than 1 m and decreasing until +20 h since base time after which values increase and reach 2.0 m at +51 h, and then decrease to between 2.0 and 1.5 m. Confidence limits ranges between 0 and about 3 m. The wind speed, depicted in Fig. 10d, is correlated (based on visual inspection) with the significant wave height in Fig. 10c with values starting from 1.3 m/s and increasing to reach 10 m/s at +41 h since base time. A decreasing phase then starts with wind speed reaching 2.0 m/s at +58 h followed by a second increasing phase with a value of about 10 m/s at the end of the forecast period. The 95% confidence interval ranges between 0 and 18 m/s. Even if the full validation of SE outputs is beyond the scope of this paper, temperature and wind speed observations from the ODAS buoy, which are displayed in Fig. 10a and d, show that the SE prediction gives a reasonable estimation of the environmental variables. No co-located observations were available to undertake a similar comparison for significant wave height and current speed.

Fig. 10

Temporal graphs of super-ensemble/3DSE METOC forecasts and associated 95% confidence level at position of the ODAS buoy 43.83° N, 9.11° E (see the red marker in Figs. 6, 7, 8 and 9); a sea surface temperature (T), b horizontal surface current velocity (CS), c significant wave height (HS), d 10 m wind (WS)

Figure 11 reports the coefficient of variation (the ratio between the standard deviation and the mean) of the four METOC variables in the same spatial grid point as in Fig. 10 over the entire forecast period so as to compare the range of variability on a normalised scale. The coefficient of variation for sea surface temperature (see Fig. 11a) ranges between 0.015 and 0.04. The surface current speed shows higher variability with a mean of about 0.7 and with peaks reaching 1.25 (see Fig. 11b). The significant wave height coefficient of variation ranges roughly from 0.04 to 0.5 reaching the highest statistical variability at +40 h since base time. The wind speed coefficient of variation has a maximum value of 1.4 at +3 h since base time and values of approximately 0.4 between +20 and +30 h, and +50 and +60 h.

Fig. 11

Temporal graphs of the super-ensemble/3DSE METOC forecasts coefficient of variation at position 43.83° N, 9.11° E; a sea surface temperature (T), b horizontal surface current velocity (CS), c significant wave height (HS), d 10 m wind (WS)

Based on this analysis, the dynamic variability of the DSS output is driven mainly by the current, the significant wave height and the wind speed, as these factors present the most dynamic and uncertain statistical behaviour (within the temporal window considered). In the following section, we will try to empirically correlate the METOC behaviour with the action statistical separation and the action confidence levels.

Other data provided as input to the DSS include variables not predicted by forecast systems and non-METOC information. For example, to support diver operations no forecast models are available to predict sea water optical properties, therefore, water visibility is estimated by using the visible bands of a multi-spectral satellite sensor and updated as new satellite data are available. Figure 12 shows an example from the National Aeronautics and Space Administration Moderate Resolution Imaging Spectro-Radiometer sensor on board the satellite Aqua acquired 26 August 2010 at 12:05:08 + 00 UTC (see Fig. 12a for the false colour image). The horizontal visibility (see Fig. 12b) was estimated using the spectral beam attenuation coefficient for a wavelength in the green light region (Zaneveld and Pegau 2003). As a rule of thumb, 10% of the calculated value is used to estimate the standard deviation associated with water visibility (Zaneveld and Pegau 2003), which is then propagated through the DSS. In the future work, this error estimate may be improved by comparing simultaneous measurements of water optical properties with satellite estimates.

Fig. 12

Water visibility retrieved from multi-spectral satellite images. a MODerate resolution Imaging Spectro-radiometer aqua image of the Ligurian Sea. Acquisition time, 20100826T120508Z. b Horizontal visibility on the area of interest

Non-METOC data include information on the vessel traffic in the area of interest, which is used in support of glider operations. This information is retrieved by processing a long sequence (such as 1 year) of automatic identification system (AIS) contacts acquired in the area of interest and for a long time period (Baldacci et al. 2008). Figure 13a shows an example of an AIS contact data set acquired over the Mediterranean Sea while Fig. 13b shows the AIS coverage estimation over the area of interest (unit-less number between 0 and 1, 0 meaning no coverage, 1 maximum coverage), which is related to the vessel traffic density (Baldacci et al. 2008). As can be expected, areas of high-traffic density are located close to major ports in the SE and NW parts of the region of interest. The density map can be updated as new AIS contacts become available, but compared with the dynamic variability of involved METOC variables, this map can be considered stationary over several forecast periods.

Fig. 13

a A data set of AIS contacts on the Mediterranean Sea. b Map of vessel traffic density (normalised between 0 and 1) in the area of interest estimated from AIS data

It is important to recall that the description of the scenario in this section to evaluate the DSS products only uses forecasts, thus resembling the considerations made by the DSS user (typically a METOC officer) in so far as possible. The evaluation can in no case be considered a statistical analysis of the expected behaviour over the considered area, as this implies the use of a long historical dataset.

5 Results

This section reports examples of operational products from the fuzzy/Bayesian DSS in support of a glider surfacing for data transmission operation and a diver operation outlined in the scenario described in the previous section. During REP10, the DSS was run every day, as soon as a new SE forecast was available. At each run, hourly maps on latitude/longitude grids for 3 days ahead were generated (at +1, +2, …, +72 h). For the sake of clarity, only three sets of products are shown (those at +30, +36 and +42 h since base time 26 August 00:00:00 + 00 UTC), since they represent an interesting case study. In addition, temporal plots reporting results for a particular spatial point are provided with the full temporal resolution (1 h) over the whole forecast period of 72 h.

5.1 Glider surfacing for data transmission operation

Figure 14a and b show maps of suggested actions and maps of fused action/confidence, respectively, as provided by the DSS. At +30 h since base time, the recommended action is runOp over the entire area of interest. The system decision is confident on the whole grid except for a small region in the SW part of the area.

Fig. 14

a Action and b fused Action/Confidence maps for glider surfacing and data transmission operation support

On average, the risk associated to action runOp progressively grows from +30 to +42 h since base time (see Fig. 15a) while the risk associated to action notRunOp decreases (see Fig. 15b). Mean significant wave height and wind speed increase approaching marginal/unfavourable conditions, as modelled by fuzzy system membership functions and rules (see Appendix 2 and Fig. 26 for the specification of rules and the membership functions). Mean current and vessel traffic density values are in a favourable/marginal condition over the temporal span of 12 h.

Fig. 15

a Mean risk for action runOp, b mean risk for action notRunOp for glider surfacing and data transmission operation support

The standard deviation of the risks increases (see Fig. 16a and b) due to increasing significant wave height uncertainty as depicted in Figs. 10c and 11c (the tendency is roughly the same over the entire area of interest, the temporal plot on a single spatial point facilitates the interpretation by the reader) with average value and coefficient of variation increasing between +30 and +42 h since base time, meaning that the standard deviation increases too.

Fig. 16

a Risk standard deviation for action runOp, b risk standard deviation for action notRunOp for glider surfacing and data transmission operation support

The situation just described explains the evolution of the action map going progressively from runOp to notRunOp on the whole spatial grid (see Fig. 14a) passing through an intermediate situation at +36 h. In this intermediate situation, the confidence intervals of the action risks overlap both due to the mean risks being close to each other and the uncertainty increase. The total effect is the presence of a no confidence state over much of the western half of the area of interest (see Fig. 14b). The decision is confident again at +42 h, even if the risk standard deviation is greater than at +36 h, because the values of the action mean risks are sufficiently separated to not allow the overlap of the 95% risk confidence intervals. Across the boundary between runOp and notRunOp regions, the action mean risks are closer to each other and the risk confidence intervals naturally tend to overlap. This gives rise to a no confidence area along the borders (see Fig. 14a and b), whose width depends on the amplitude of the confidence intervals themselves.

Figure 17 shows the confidence measure used to classify the area of interest in regions with confidence and no confidence. At +30 h since base time, positive confidence is given for the whole area except for a small SW region as previously highlighted. The negative confidence area grows as expected to roughly cover the western half of the region of interest at +36 h. The confidence increases, reaching positive values over almost the entire region at +42 h. The confidence measure is extremely useful to locate regions in space and time that are characterised by imprecise and contradicting information. The index synthesises the effect of uncertainty of several input factors, an important feature toward the reduction of the user cognitive work load.

Fig. 17

Confidence measure along the 12 h time span for glider surfacing and data transmission operation support

Figure 18 shows the temporal evolution of the minimum risk associated with the recommended action over the region of interest. These maps are also useful during the operation planning phase when an area of low risk has to be identified to run the operation in the safest possible condition. In the examined case, the recommended action at +30 h since base time is runOp in the whole region, but the best conditions with lower risk are present in the eastern part toward the Italian coast. This is consistent with the input METOC variable situation as depicted in Figs. 6, 7, 8 and 9, where there are lower values (associated with most favourable operational conditions) of current speed, significant wave height and wind speed in the eastern part of the basin. The situation is symmetric when the decision is notRunOp. In this case, the minimum risk is lower when the METOC conditions are most unfavourable as in the case depicted at +42 h since base time for which the system decision is notRunOp almost everywhere in the area considered. Again, the minimum risk map is consistent with the METOC values which are higher in the western part of the basin than in the eastern.

Fig. 18

Minimum risk along the 12 h time span for glider surfacing and data transmission operation support

Figure 19 reports DSS outputs over the entire forecast period for the spatial point at 43.83° N and 9.11° E as in Figs. 10 and 11 for the input METOC variable statistics. The temporal graph is useful to better understand the effect of the input data uncertainty on action risk statistics and consequent decisions. As depicted in Fig. 19a, the recommended action is runOp from +00 to +38 h since base time; whereas the action is notRunOp for the rest of the time span considered. The decision is not confident in much of the time interval between +38 h to the end of the forecast interval as depicted in Fig. 19b. These outputs are consistent with the risk statistics and the confidence measure values reported in Fig. 20. In the first 37 h, the 95% risk confidence intervals are clearly separated giving rise to a positive confidence measure (see Fig. 20b). The mean risk associated with the runOp action is less than the risk associated to action notRunOp because the input METOC values in this period are consistent with a favourable operational environment. After +37 h since base time, the situation is inverted as the significant wave height and the wind speed evolve toward marginal/unfavourable conditions (increasing their values), and remain so until the end of the forecast period. The negative confidence between +38 and +39 h is due to the overlap of the risk confidence intervals (see Fig. 20a) occurring during a transition phase in which mean risks are close to each other and the associated standard deviation slightly increases, mainly due to an increase in the significant wave height uncertainty (see Figs. 10c and 11c). After this transition phase, risk standard deviations further increase, but confidence intervals do not overlap, giving rise to a positive confidence measure until +45 h. The risks start to decrease while the associated uncertainties increase until the confidence limits start to overlap giving rise to a negative confidence measure in most of the remaining interval.

Fig. 19

a Decided action and associated minimum risk, b fused decided action and confidence and associated minimum risk for glider surfacing and data transmission operation support at position 43.83° N, 9.11° E

Fig. 20

a Action mean risk and associated risk confidence levels, b measure of confidence in decision for glider surfacing and data transmission operation support at position 43.83° N, 9.11° E

5.1.1 Glider surfacing/transmission and integration with glider path prediction

Support to glider operations is further improved by integrating the fuzzy/Bayesian DSS with a glider path prediction system changing the context in which risk assessment and decision making have been treated from the Eulerian to the Lagrangian. This allows pilots to display the variability of fuzzy/Bayesian DSS products and input METOC variables in one dimension along predicted glider paths, decreasing cognitive burden and so improving awareness and their ability to make informed decisions. In the specific case of glider surfacing, the transmission of the large amount of acquired measurements can be planned to occur at specific locations in which the environment allows the maximum chance of successful data transfer.

In this work, a stochastic glider path prediction, first proposed in Grasso et al. (2010a, c), is used to estimate tracks of gliders operating during the REP10 experiment given a probabilistic prediction of the 3D current field over the area of interest. The system inputs consist of predicted current speed mean vector and covariance matrix from the 3DSE forecast model, mission way point list, last known glider position and glider parameters. The system is based on a three-dimensional non-linear kinematic glider model with 3D stochastic current speed as input. Currents are modelled as a bi-variate Gaussian random vector (the two components being the meridional and zonal currents) with a given mean and covariance matrix. Uncertain model parameters (such as heading as measured by an electronic compass) are modelled as Gaussian random variables with mean and standard deviation estimated from glider parameter measurements. The glider position statistics (path estimate and associated covariance matrix) are predicted from the last known GPS position by using the kinematic model input into the prediction step of an unscented Kalman filter (Julier and Uhlmann 2004; Van der Merwe 2004), which is able to deal with model non-linearity in a more efficient way than Monte Carlo-based techniques. The algorithm runs every time a new GPS position is acquired and new current forecasts are produced by the SE system.

Figure 21 shows an example of the output of the path prediction algorithm. The glider (named Laura) was programmed to run a mission along a rectilinear transect in the NW–SE direction (see the desired path in black) within the allowed REP10 area. The picture shows the mean glider path prediction (see the line in magenta) with associated 99% position error ellipses. The ellipse major axis grows from a few metres due to GPS position uncertainty to roughly 7 km after about 60 h of navigation due to accumulated uncertainty originated by the current field prediction error and glider parameters uncertainty. The second-order statistical characterisation of the glider position can be used to estimate risk indices, such as the probability of crossing a denied area, which can help glider pilots to reschedule mission parameters sufficiently in advance to avoid unsafe navigation conditions (Grasso et al. 2010c).

Fig. 21

Prediction of Laura glider track and associated 99% error ellipses within the allowed operational area. Starting simulation time, 20100826T135500Z. Starting position, 43.86°N, 9.04°E; SE model base time, 20100826T000000Z. Predicted mission duration, 57 h

Figure 22 depicts SE METOC forecasts along the predicted glider mean path with associated 95% confidence intervals. The comparison of Fig. 22 with Fig. 10, showing predicted METOC conditions for a specific point of the spatial grid as a function of time, reveals a similar temporal variability between the two cases, though each temporal sample in Fig. 22 is associated with a different spatial point along the glider path. In particular, similar to Fig. 10c and d, the significant wave height and the wind speed (see Fig. 22c and d) are strongly correlated, with values progressively increasing in the interval +20 and +50 h since the base time and transitioning from favourable/marginal environmental conditions to unfavourable ones. The variability of DSS products for the surfacing and transmission operation along the predicted glider path is expected to be similar to the case in Figs. 19 and 20 for the single spatial point.

Fig. 22

Super-ensemble/3DSE METOC forecasts along predicted glider path; a sea surface temperature (T), b horizontal surface current velocity (CS), c significant wave height (HS), d 10 m wind (WS)

Figure 23a and b show the graphs of the recommended action and the fused decision/confidence, respectively, along the predicted glider path for the surfacing and transmission operation. The decided action is runOp between +10 and +40 h since base time. From +40 h since base time to the end of the prediction temporal interval, the action is notRunOp due to values of significant wave height and wind speed, which are marginal/unfavourable conditions. The fused decision/confidence graph shows four cases in which the final decision is marked with no confidence.

Fig. 23

a Action map and b fused action/confidence map with associated minimum risk for glider surfacing and data transmission operation support along the predicted glider path

An in-depth retrospective analysis of action risk and METOC statistics can reveal main factors causing the system to not be confident about its decisions. Figure 24a and b depict mean risks associated to each action with 95% confidence intervals and the confidence measure, respectively, along the predicted path. The mean risk and the associated uncertainty between +10 and +40 h since base time are such that associated actions can be statistically discriminated, as confirmed by the confidence measure in the same interval. METOC values are compatible with a favourable environmental condition and the associated prediction is not uncertain enough to degrade the confidence measure it remains positive except for a short period between +30 and +40 h since base time. In this period, the environment is transitioning to a marginal/unfavourable condition, mainly due to increasing significant wave height and wind speed. The mean risks associated to the actions converge, and even if the risk standard deviation does not increase significantly, the risk confidence intervals overlap each other giving a negative confidence measure to the system decision. After the transition, from +40 h since base time, the decision switches to the action notRunOp having the minimum risk. Between +40 and +60 h the confidence measure is negative during short periods due to increasing risk uncertainty, which is mainly related to an increase in significant wave height and wind speed confidence intervals (see Fig. 22).

Fig. 24

a Mean risk associated to action runOp and notRunOp with 95% confidence intervals along predicted glider path and b confidence measure along the predicted glider path for glider surfacing and data transmission operation support

5.2 Diver operations

This section provides examples of DSS products to support a diver operation. The results reported are limited, for the sake of brevity, to action and fused action/confidence maps. Additional white regions within the maps are due to cloud coverage in passive multi-spectral data used to estimate water visibility.

Figure 25a and b present a temporal series of suggested actions and fused action/confidence maps, respectively, as provided by the DSS for a diver operation that could be conducted in the REP10 region. At +30 h since base time, the recommended action is runOp over the entire area and the system decision is confident on the whole grid. A transient condition between a favourable and a completely unfavourable situation is present at +36 h since base time, for which the system decides for notRunOp in a region between 8°30′ E and 9° E. This change is driven mainly by an increase in significant wave height from values around 0.5 m at +30 h to values greater than 1 m at +36 h. The confidence measure is positive over the entire area except along the border between the regions labelled runOp and notRunOp. A no-confidence label is assigned by the system in this area as depicted in Fig. 25b. Negative confidence is due to action risks being close to each other during a transition phase such as this, and the risk standard deviation increasing with respect to the condition at +30 h due to an increase in significant wave height prediction uncertainty. At +42 h since base time predicted significant wave height reaches values of approximately 2 m, which are highly unfavourable. A regime condition is reached where the decided action is notRunOp over the whole region of interest (see Fig. 25a) with high confidence over the entire domain (see Fig. 25b).

Fig. 25

a Action and b fused action/confidence maps for diver operation support. White no data available (e.g., due to overcast conditions or missing forecast)

6 Conclusions

This paper has shown the products that a recently developed DSS can provide to the decision maker during a real experiment at sea: the REP10 experiment. Key features of the DSS are (1) the ability to deal with generic operations, (2) the tolerance to uncertainties on its inputs and (3) the ability to propagate such uncertainties to its outputs (using the unscented transform). In this way, the DSS is able to provide not only a recommended action but a confidence level for that action. In particular, when the costs associated with different actions are not statistically separable, the decision maker is informed and has to make a decision based on intuition or other external factors.

A complex scenario with a number of factors playing different roles and influences has been shown. The cognitive workload that a human decision maker has to handle is significant, particularly when planning 3 days in advance. The DSS significantly simplifies this task, by providing the decision maker with a single run/not run/no confidence map. This has the potential to add a significant value to the environmental forecasts themselves by improving the responsiveness of METOC officers and decision makers through the provision of more options than those a human operator can assess manually.

The usefulness of the system has been judged positively by glider pilots and other decision makers during the experiment. However, in the future, the system needs to be further validated using human experts more extensively in upcoming experiments at sea.

Appendices

Appendix 1

The forecast models used during REP10 in the super ensemble system are described for each environmental component: atmosphere, surface gravity waves and ocean.

1.1 Atmospheric models

The IFS ECMWF atmospheric deterministic model T1279 (Miller et al. 2010; http://www.ecmwf.int/) is a global model with horizontal resolution of 0.125° and output data available every 3 h.

The atmospheric model COSMO-ME (Bonavita and Torrisi 2005): provided by the Italian Air Force National Meteorological Center (Centro Nazionale per la Meteorologia e Climatologia Aeronautica, CNMCA) COSMO-ME is a non-hydrostatic model with 7 km horizontal resolution covering the European region and nested in IFS ECMWF atmospheric model. Output data is available every 1 h.

1.2 Surface gravity wave models

The IFS ECMWF wave model (Miller et al. 2010; http://www.ecmwf.int) is based on the WAM model (Komen et al. 1994) and coupled to the IFS ECMWF atmospheric model. The horizontal resolution is 1/4°. Output data are available every 3 h.

The NETTUNO wave forecasting system is provided by UGM (http://www.meteoam.it/). It is based on the WAM model (Komen et al. 1994) and forced using COSMO-ME atmospheric model. The horizontal resolution is 1/20°. Output data are available every 3 h. The domain is the entire Mediterranean Sea.

The WW3MED wave forecasting system is provided by PREVIMER (www.previmer.org). It is based on the wave model Wave Watch III (Tolman 2009). Atmospheric forcing is provided by IFS ECMWF. The horizontal resolution is 1/10°. Output data are available every 3 h. The domain is the entire Mediterranean Sea.

The WW3MENOR wave forecasting system is provided by PREVIMER (www.previmer.org). It is based on the wave model Wave Watch III (Tolman 2009). The atmospheric forcing is provided by IFS ECMWF. The horizontal resolution is 4 km. Output data are available every 3 h. The domain covers the northwestern Mediterranean Sea.

1.3 Ocean models

The MARS3DMENOR: ocean forecasting system is provided by PREVIMER (http://www.previmer.org/). It is based on the ocean model MARS (Lazure and Dumas 2008). The horizontal resolution is 1 km. Output data available every 3 h. The domain is the northwestern Mediterranean Sea and it is nested in the Mediterranean forecasting system (MFS, Oddo et al. 2009). The surface forcing is provided by MM5 run at ACRI-ST.

The NRL NCOM operational ocean forecasting system is provided by NRL-SSC. It is based on the ocean model NCOM (Martin 2000) with set up for the REP10 trial framework only. The horizontal resolution is ∼1 km. Output data is available every 1 h. The domain is the Ligurian Sea and nested with multiple downscaling in the Mediterranean NCOM model. Meteorological forcing is provided by COAMPS model (Hodur 1997).

The NURC ROMS is an operational ocean forecasting system based on the ocean model ROMS (Haidvogel et al. 2008), set up for the REP10 trial framework only. The horizontal resolution is 2 km. Output data is available every 3 h. The domain is the Ligurian Sea and it is nested in MFS model. Meteorological forcing is provided by COSMO-ME model.

Appendix 2

A fuzzy rule-based classifier (Cordon et al. 1999) is made up of two parts: the knowledge base (KB) and the fuzzy reasoning method (FRM). The KB, in turn, is made of a data base (DB) and a rule base (RB). The DB contains the information about the fuzzy sets used to partition each input and output variable, i.e., the parameters associated with the membership functions which describe a fuzzy set. Figure 26 depicts the fuzzy sets for METOC input variables used to support a glider surfacing and transmission operation. Each variable domain is covered by three fuzzy sets with associated linguistic terms low, medium and high and trapezoidal membership functions. Each membership function ranges between 0 and 1 and specifies to which extent a value in the domain belong to a fuzzy set.

Fig. 26

Fuzzy variables and membership functions for the glider surfacing and data transmission operation. a Horizontal surface current velocity (CS), b 10 m wind speed (WS), c significant wave height (HS), d vessel traffic density (VTD)

The RB is a set of if-then rules expressed by combining linguistic terms of each input/output variable in the premise and the consequence of a rule. A rule can easily be specified by using natural language words as in the example below:

R1: If current speed = low And wind speed = low And significant wave height = low And vessel traffic density = low Then favourable = high And marginal = low And unfavourable = low.

The fuzzy inference engine combines the METOC input values (which are the UT sigma points as the UT wraps the fuzzy classifier and the risk calculator) with the fuzzy sets in the premise and then in the consequence of each rule. The rule outputs are then combined to calculate the output posterior probabilities of environmental classes.

There are many ways to design the KB, mostly distinguished by knowledge-driven and data-driven approaches. Our system supports a knowledge-driven approach, which means that it allows the user to define both the DB (i.e., the membership function values) and the RB. This approach allows the system to integrate past experience of human experts for each operation at sea, and to run the system even when few or no historical data exist.

Under the knowledge-driven approach, the user is asked to provide a DB and a RB, both consistent with the maritime operation under consideration. Once done, the system is able to perform the classification. The use of a fuzzy RB instead of a non-fuzzy RB allows a greater flexibility in defining the DB, instead of using crisp thresholds on input forecasts.

Elicitation of fuzzy set parameters was performed by first agreeing on input and output operational limits with METOC operators and glider pilots involved in the REP10 experiment, then using these limits to define fuzzy set membership functions with a certain degree of overlap as in Fig. 26. The authors are envisioning the use of a technique adapted from the Interval Approach (IA) method (Feilong and Mendel 2007) to formally elicit input and output variable limits and transform them into fuzzy sets.

With regards the fuzzy reasoning method, we have used a First Inference Then Aggregate approach based on the singleton “fuzzification” method (i.e., we have used real numbers as inputs for our fuzzy system) and the Mamdani inference method (Mamdani and Assilian 1975). More precisely, the minimum has been used to model the AND operator in if-then rules. The same operator has been used to model the implication for deducing the consequents of each rule. The output fuzzy sets obtained after implication have been aggregated using the maximum, while the posterior probability associated with each output class has been computed as the centre-of-gravity method.

Appendix 3

The UT is a technique used to propagate mean and covariance of a random vector through any kind of non-linearity with accuracy up to the second order (Julier and Uhlmann 2004; Van der Merwe 2004). Given a random vector x and a vector function f(•) the problem of estimating the mean and the covariance of y = f(x) from the mean and covariance of x is solved by using a set of so called weighted sigma points:

$$ \matrix{{*{20}{c}} {{\mathbf{s}}{{\mathbf{p}}_i} \equiv \left\{ {{w_i},{{\mathbf{\chi }}_i}} \right\}} & {i = 0,...,2F,} \\ } $$

(8)

where, w _i is a weight factor, χ _i is a point in the original input space and F is the input space dimensionality. The χ _i are deterministic points that capture the mean and the covariance of the variable x in the input space. Points and the associated weights are chosen as follows (Julier and Uhlmann 2004; Van der Merwe 2004):

$$ \matrix{{*{20}{c}} {{\chi_0} = {{\mathbf{m}}_{{\mathbf{x}}}},\,{w_0} = \frac{\varsigma }{{L + \varsigma }}} & {i = 0,} \\ } $$

(9)

$$ \matrix{{*{20}{c}} \begin{gathered} {\chi_i} = {{\mathbf{m}}_{{\mathbf{x}}}} + {\left[ {\sqrt {{\left( {L + \varsigma } \right) \cdot {{\mathbf{R}}_{{\mathbf{x}}}}}} } \right]_i} \hfill \\ {w_i} = \frac{\varsigma }{{2 \cdot \left( {L + \varsigma } \right)}} \hfill \\ \end{gathered} & {i = 1,...,F,} \\ } $$

(10)

$$ \matrix{{*{20}{c}} \begin{gathered} {\chi_i} = {{\mathbf{m}}_{{\mathbf{x}}}} - {\left[ {\sqrt {{\left( {L + \varsigma } \right) \cdot {{\mathbf{R}}_{{\mathbf{x}}}}}} } \right]_i} \hfill \\ {w_i} = \frac{\varsigma }{{2 \cdot \left( {L + \varsigma } \right)}} \hfill \\ \end{gathered} & {i = F + 1,...,2F,} \\ } $$

(11)

where, m _x and R _x are the mean and the covariance matrix of x, respectively, ς is a scaling parameter used to control the error in the estimated statistics due to higher order moments (in this work ς = 0), ${\left[ {\sqrt {{\left( {F + \varsigma } \right) \cdot {R_x}}} } \right]_i}$ is the ith column of the matrix square root of (F + ς)·R _x which is usually calculated by through the Cholesky factorization.

The χ _i points are propagated through the function f(•) to calculate the set of points γ _i in the output space:

$$ \matrix{{*{20}{c}} {{\gamma_i} = {\mathbf{f}}\left( {{\chi_i}} \right)} & {i = 0,...2F.} \\ } $$

(12)

These points, together with the weights, w _i , are subsequently used to estimate the second-order statistics of the output random vector y, that is mean ${\widehat{{\mathbf{m}}}_{{\mathbf{y}}}}$, covariance ${\widehat{{\mathbf{R}}}_{{\mathbf{y}}}}$ and cross-covariance ${\widehat{{\mathbf{R}}}_{{{\mathbf{xy}}}}}$:

$$ {\widehat{{\mathbf{m}}}_{{\mathbf{y}}}} = \sum\limits_{{i = 1}}^{{2F}} {{w_i} \cdot {{\mathbf{\gamma }}_i}}, $$

(13)

$$ {\widehat{{\mathbf{R}}}_{{\mathbf{y}}}} = \sum\limits_{{i = 1}}^{{2F}} {{w_i} \cdot \left( {{{\mathbf{\gamma }}_i} - {{\widehat{{\mathbf{m}}}}_{{\mathbf{y}}}}} \right)} \cdot {\left( {{{\mathbf{\gamma }}_i} - {{\widehat{{\mathbf{m}}}}_{{\mathbf{y}}}}} \right)^T}, $$

(14)

$$ {\widehat{{\mathbf{R}}}_{{{\mathbf{xy}}}}} = \sum\limits_{{i = 1}}^{{2F}} {{w_i} \cdot \left( {{\chi_i} - {{\widehat{{\mathbf{m}}}}_{{\mathbf{x}}}}} \right)} \cdot {\left( {{{\mathbf{\gamma }}_i} - {{\widehat{{\mathbf{m}}}}_{{\mathbf{y}}}}} \right)^T}. $$

(15)