Spatial Representations and Analysis Techniques

Abstract

Space plays an important role in the dynamics of collective adaptive systems (CAS). There are choices between representations to be made when we model these systems with space included explicitly, rather than being abstracted away. Since CAS often involve a large number of agents or components, we focus on scalable modelling and analysis of these models, which may involve approximation techniques. Discrete and continuous space are considered, for both models of individuals and models of populations. The aim of this tutorial is to provide an overview that supports decisions in modelling systems that involve space.

Appendix: Discrete and continuous time Markov Chains

This section briefly introduces these concepts, as they would be used in stochastic modelling both without aggregation of state and with aggregation of state (population-based Markov chains) [4, 10].

Definition 1

A discrete time Markov chain (DTMC) is a tuple ${\mathscr {M}}_D = ({\mathscr {S}},\mathbf {P})$ where

• ${\mathscr {S}}$ is a finite set of states, and

• $\mathbf {P}: {\mathscr {S}}\times {\mathscr {S}}\rightarrow [0,1]$ is a probability matrix satisfying $\sum _{S' \in {\mathscr {S}}} \mathbf {P}(S,S') = 1$ for all $S \in {\mathscr {S}}$.

A DTMC is time-abstract [4] in the sense that time is viewed as a sequence of discrete steps or clock ticks. It describes behaviour as follows: if an entity or individual is currently in state $S \in {\mathscr {S}}$ then the probability of the entity being in state $S'$ at the next time step is defined by $\mathbf {P}(S,S')$. Under certain conditions, the steady state of the DTMC can be determined and this describes when the DTMC is at equilibrium and gives the (unchanging) probability of being in any of the states of ${\mathscr {S}}$. By contrast, transient state probabilities can be determined at each point in time before steady state is achieved.

Definition 2

A continuous time Markov chain (CTMC) is a tuple ${\mathscr {M}}_C = ({\mathscr {S}},\mathbf {R})$ where

• ${\mathscr {S}}$ is a finite set of states, and

• $\mathbf {R}: {\mathscr {S}}\times {\mathscr {S}}\rightarrow \mathbb {R}_{\ge 0}$ is a rate matrix.

CTMCs are time-aware [4] since they use continuous time. If an entity is currently in state S, then $\mathbf {R}(S,S')$ is a non-negative number that defines an exponential distribution from which the duration of the time taken to transition from state S to state $S'$ can be drawn. As with DTMCs and under certain conditions, transient and steady state probabilities can be calculated which describe the probability of being in each state at a particular time t or in the long run, respectively.

Let $E(S) = \sum _{S'\in {\mathscr {S}}} \mathbf {R}(S,S')$ be the exit rate of state $S'$. Then the embedded DTMC of a CTMC has entries in its probability matrix of the form $\mathbf {P}(S,S') = \mathbf {R}(S,S')/E(S)$ if $E(S)>0$ and $\mathbf {P}(S,S') = 0$ otherwise. DTMCs and CTMCs can be state-labelled (usually with propositions) or transition-labelled (usually with actions). The research in QUANTICOL focusses on transition-labelled Markov chains. We next consider population Markov chains, both discrete time and continuous time. Instead of considering an entity with states, we now consider a vector of counts $\mathbf {X}$ that describes how many entities are in each state; thus it is a population view rather than an individual view. Our definition in the continuous-time case is slightly simpler than that appearing in another chapter in this volume [9] since transitions do not have guards and we do not parameterise the Markov chain with the population size.

Definition 3

A population discrete time Markov chain (PDTMC) is a tuple ${\mathscr {X}}_D = (\mathbf {X},{\mathscr {D}},{\mathscr {T}})$ where

• $\mathbf {X}= (X_1,\ldots ,X_n)$ is a vector of variables

• ${\mathscr {D}}$ is a countable set of states defined as ${\mathscr {D}}= {\mathscr {D}}_1 \times \ldots \times {\mathscr {D}}_n$ where each ${\mathscr {D}}_i \subseteq \mathbb {N}$ represents the domain of $X_i$

• ${\mathscr {T}}= \{ \tau _1, \ldots \tau _m \}$ is the set of transitions of the form $\tau _j = (\mathbf {v},p)$ where

  • $\mathbf {v}= (v_1,\ldots ,v_n) \in \mathbb {N}^n$ is the state change or update vector where $v_i$ describes the change in number of units of $X_i$ caused by transition $\tau _j$

  • $p : {\mathscr {D}}\rightarrow \mathbb {R}_{\ge 0}$ is the probability function of transition $\tau _j$ that defines a sub-probability distribution, namely $\sum _{\tau \in {\mathscr {T}}} p_\tau (\mathbf {d}) \le 1$ for all $\mathbf {d}\in {\mathscr {D}}$, such that $p(\mathbf {d})=0$ whenever $\mathbf {d}+ \mathbf {v}\not \in {\mathscr {D}}$

Definition 4

A population continuous time Markov chain (PCTMC) is a tuple ${\mathscr {X}}_C = (\mathbf {X},{\mathscr {D}},{\mathscr {T}})$ where

• $\mathbf {X}$ and ${\mathscr {D}}$ are defined as in the previous definition,

• ${\mathscr {T}}= \{ \tau _1, \ldots \tau _m \}$ is the set of transitions of the form $\tau _j = (\mathbf {v},r)$ where

  • $\mathbf {v}$ is defined as in the previous definition,

  • $r : {\mathscr {D}}\rightarrow \mathbb {R}_{\ge 0}$ is the rate function of transition $\tau _j$ with $r(\mathbf {d}) = 0$ whenever $\mathbf {d}+ \mathbf {v}\not \in {\mathscr {D}}$.

In both types of population Markov chain, the associated Markov chain can be obtained. In both cases, ${\mathscr {D}}$ is the state space ${\mathscr {S}}$. For the population DTMC, the probability matrix of its associated DTMC is defined as

$$\begin{aligned} \mathbf {P}(\mathbf {d},\mathbf {d}') = \sum _{\tau \in {\mathscr {T}}, \mathbf {v}_\tau =\mathbf {d}'-\mathbf {d}} p_\tau (\mathbf {d}) \text { whenever } \mathbf {d}\ne \mathbf {d}'\ \end{aligned}$$

and since probability functions define sub-probabilities then the rest of the probability mass must be accounted for by defining

$$\begin{aligned} \mathbf {P}(\mathbf {d},\mathbf {d}) = 1 \, - \sum _{\tau \in {\mathscr {T}}, \mathbf {v}_\tau \ne 0} p_\tau (\mathbf {d}). \end{aligned}$$

For the population CTMC, the rate matrix of its associated CTMC is

$$ \mathbf {R}(\mathbf {d},\mathbf {d}') = \sum _{\tau \in {\mathscr {T}}, \mathbf {v}_\tau =\mathbf {d}'-\mathbf {d}} r_\tau (\mathbf {d}) \text { whenever } \mathbf {d}\ne \mathbf {d}'$$

and if the summation is empty, then $\mathbf {R}(\mathbf {d},\mathbf {d}') = 0$.

As the size of the population increases, it has been shown [58] under specific conditions that cover a large range of models that the behaviour of an (appropriately normalised) population CTMC at time t is very close to the solution of a set of ODEs, expressed in the form $\mathbf {X}(t) = (X_1(t),\ldots ,X_n(t))$ defining a trajectory over time. The ODEs can be expressed in terms of a single vector ODE as

$$\begin{aligned} \dot{\mathbf {X}} = \frac{d\mathbf {X}}{dt} = \mathbf {f}(\mathbf {X}) \end{aligned}$$

where $\mathbf {f}(\mathbf {X})$ is a function derived from the specifics of the PCTMC (see [9] in this volume for details). It is also possible to approximate the moments of a PCTMC using the ODEs [32]

$$\begin{aligned} \frac{\mathrm d}{\mathrm d t} {E}[M(\mathbf {X} (t))] = \sum _{\tau \in \mathscr {T}} {E}[(M(\mathbf {X}(t) + \mathbf {v}_{\tau }) - M(\mathbf {X}(t)))r_{\tau }(\mathbf {X}(t))] \end{aligned}$$

where $M(\mathbf {X})$ denotes the moment to be calculated, $\mathbf {v}_{\tau }$ and $r_{\tau }(\mathbf {X}(t))$ represents the update vector and the rate of a transition $\tau$, respectively.