Human–agent collaboration for disaster response

Abstract

In the aftermath of major disasters, first responders are typically overwhelmed with large numbers of, spatially distributed, search and rescue tasks, each with their own requirements. Moreover, responders have to operate in highly uncertain and dynamic environments where new tasks may appear and hazards may be spreading across the disaster space. Hence, rescue missions may need to be re-planned as new information comes in, tasks are completed, or new hazards are discovered. Finding an optimal allocation of resources to complete all the tasks is a major computational challenge. In this paper, we use decision theoretic techniques to solve the task allocation problem posed by emergency response planning and then deploy our solution as part of an agent-based planning tool in real-world field trials. By so doing, we are able to study the interactional issues that arise when humans are guided by an agent. Specifically, we develop an algorithm, based on a multi-agent Markov decision process representation of the task allocation problem and show that it outperforms standard baseline solutions. We then integrate the algorithm into a planning agent that responds to requests for tasks from participants in a mixed-reality location-based game, called AtomicOrchid, that simulates disaster response settings in the real-world. We then run a number of trials of our planning agent and compare it against a purely human driven system. Our analysis of these trials show that human commanders adapt to the planning agent by taking on a more supervisory role and that, by providing humans with the flexibility of requesting plans from the agent, allows them to perform more tasks more efficiently than using purely human interactions to allocate tasks. We also discuss how such flexibility could lead to poor performance if left unchecked.

Appendices

Appendix 1: Radiation cloud modelling

The radiation cloud diffusion process is modelled using the Smoluchowski drift-diffusion equation,

$$\begin{aligned} \frac{D \hbox {Rad}(\mathbf{z}, \tau )}{D \tau }=\kappa \triangledown ^2 \hbox {Rad}(\mathbf{z},\tau )-\hbox {Rad}(\mathbf{z},\tau )\triangledown \cdot \mathbf{w}(\mathbf{z},\tau )+\sigma (\mathbf{z},\tau ) \end{aligned}$$

(4)

where $D$ is the material derivative, $\text {Rad}(\mathbf{z},\tau )$ is the radiation cloud intensity at location $\mathbf{z}=(x,y)$ at time $\tau$, $\kappa$ is a fixed diffusion coefficient and $\sigma$ is the radiation source(s) emission rate. The diffusion equation is solved on a regular grid defined across the environment with grid coordinates $G$ (as defined in Sect. 3.1). Furthermore, the grid is solved at discrete time instances $\tau$. The cloud is driven by stochastic wind forces which vary both spatially and temporally. These forces induce anisotropy into the cloud diffusion process proportional to the local average wind velocity, $\mathbf{w}(\mathbf{z},\tau )$. The wind velocity is drawn from two independent Gaussian processes (GP), one GP for each Cartesian coordinate axis, $w_i(\mathbf{z},\tau )$, of $\mathbf{w}(\mathbf{z},\tau )$. The GP captures both the spatial distribution of the wind velocity and the dynamic process resulting from shifting wind patterns (e.g., short term gusts and longer term variations).

In our simulation, each spatial wind velocity component is modelled by an isotropic squared-exponential GP covariance function [46], $K$, with fixed input and output scales, $l$ and $\mu$, respectively (although any covariance function can be substituted),

$$\begin{aligned} K(\mathbf{z},\mathbf{z}^\prime )=\mu ^2\exp -(\mathbf{z}-\mathbf{z}^\prime )^T \mathbf{P}^{-1}(\mathbf{z}-\mathbf{z}^\prime ) \end{aligned}$$

where $\mathbf{P}$ is a diagonal covariance matrix with diagonal elements $l^2$. This choice of covariance function generates wind conditions which vary smoothly in both magnitude and direction across the terrain. Furthermore, as wind conditions may change over time we introduce a temporal correlation coefficient, $\rho$, to the covariance function. Thus, for a single component, $w_i$, of $\mathbf{w}$, defined over grid $G$ at times $\tau$ and $\tau ^\prime$, the wind process covariance function is, $\hbox {Cov}(w_i(\mathbf{z},\tau ),w_i(\mathbf{z^\prime },\tau ^\prime ))=\rho (\tau ,\tau ^\prime ) K(\mathbf{z},\mathbf{z}^\prime )$. We note that, when $\rho =1$ the wind velocities are time invariant (although spatially variant). Values of $\rho <1$ model wind conditions that change over time.

Using the above model, we are able to create a moving radiation cloud. This poses a real challenge both for the HQ ($PA$ and $H$) and the responders on the ground as the predictions they make of where the cloud will move to will be prone to uncertainty both due to the simulated wind speed and direction. While it is possible to use radiation readings provided by first responders on the ground, as they move in the disaster space, in our trials, we assumed that these readings are coming from sensors already embedded in the environment to allow the team to focus on path planning for task allocation (which is the focus of this paper) rather than for radiation monitoring. Hence, using such sensor readings, the prediction algorithm provided in Appendix 2 is then used to provide estimates of the radiation levels across the disaster space during the game. These estimates are displayed as a heat map as described in Sect. 5.

Appendix 2: Predictive model of radiation cloud

Predictions of the clouds location are performed using a latent force model (LFM) [47, 48]. The LFM is a Markov model that allows the future state of the cloud and wind conditions to be predicted efficiently from the current state. Predictions are computed using the Extended Kalman filter (EKF) which has a linear computational complexity with regard to the time interval over which the dynamics are predicted forward.¹¹ The EKF estimates provide both the mean and variance of the state of the cloud and wind conditions. Figure 6 shows example cloud simulations for slow varying (i.e. $\rho =0.99$) and gusty (i.e. $\rho =0.90$) wind conditions. The left panes in each subfigure show the ground truth simulation obtained by sampling from the LFM. The middle panes show the mean of the cloud and wind conditions and the right panes show the uncertainty in the conditions.

Fig. 6

Radiation and wind simulation ground truth and EKF estimates obtained using measurements from monitor agents (black dots). Left most panes are ground truth radiation and wind conditions, the middle panes are corresponding estimates and right most panes are state uncertainties: a invariant and b gusty wind conditions. The radiation levels are normalised to the range $[0,\ 1]$. a Slowly varying wind conditions. b Gusty wind conditions

The radiation is monitored using a number of sensors on the ground that collect readings of the radiation cloud intensity and, optionally, wind velocity every minute of the game. These monitor agents can be at fixed locations or they can be mobile agents equipped with geiger-counters that inform the user and commander of the local radiation intensity. The measurements can be folded into the EKF and this refines estimates of both the radiation cloud and wind conditions across the grid. Figure 6 shows the impact of such measurements on the uncertainty of the cloud and wind conditions. Location of two monitors are shown as black dots in the upper row of panes in both subfigures. The right most panes show the relative uncertainty in both the cloud and wind conditions as a result of current and past measurements. Figure 6a shows slow varying wind conditions in which case the radiation cloud can be interpolated accurately using sparse sensor measurements and the LFM model. Alternatively, during gusty conditions the radiation cloud model is more uncertain far from the locations where recent measurements have been taken, as shown in Fig. 6b.

Appendix 3: Simulation results of MMDP solution

Before deploying our solution (as part of $PA$) to advise human responders, it is important to test its performance to ensure it can return efficient solutions on simulations of the real-world problem. Given there is no extant solution that takes into account uncertainty in team coordination for emergency response, we compare our algorithm with a greedy and a myopic method to evaluate the benefits of coordination and lookahead. For each method, we use our path planning algorithm to compute the path for each responder. In the greedy method, the responders are uncoordinated and select the closest tasks they can do. In the myopic method, the responders are coordinated to select the tasks but have no lookahead for the future tasks (Line 8 in Algorithm 2). Table 3 shows the results for a problem with 17 tasks and 8 responders on a $50\times 55$ grid. As can be seen, our MMDP algorithm completes more tasks than the myopic and greedy methods (see Table 3). More importantly, our algorithm guarantees the safety of the responders, while in the myopic method only 25 % of the responders survive and in the greedy method all responders are killed by the radioactive cloud. More extensive evaluations are beyond the scope of this paper as our focus here is on the use of the algorithm in a field deployment to test how humans take up advice computed by the planning agent $PA$.

Table 3 Experimental results for the MMDP, myopic, greedy algorithms in simulation