Network Structure and Dynamics of Biological Systems

Abstract

Many biological systems in nature can be represented as a dynamic model on a network. Examples include gene regulatory systems, neuronal networks, food webs, epidemics spreading within populations, social networks, and many others. A fundamental question when studying biological processes represented as dynamic models on networks is to what extent the network structure is contributing to the observed dynamics. In other words, how does network connectivity affect a dynamic model on a network? In this chapter, we will explore a variety of network topologies and study biologically inspired dynamic models on these networks.

Appendix

Here we provide the R code used to generate all network figures in this chapter with the exception of Figs. 1, 2, 7, 12, 13, and 14 which were created from scratch in MS PowerPoint. Networks were generated using the igraph package (version 1.0.1) in R (version 3.5.3) [18, 38]. Note that the code below will generate a realization or sample network from the given model. These are random graph models, and their generation is based on probabilistic rules (e.g., edge probabilities in the Erdős–Rényi model). Therefore, it is unlikely that you will generate the exact same sample network given in this chapter, but rather a different sample network with the same or similar network properties.

Fig. 13

An illustration of the transition state diagram corresponding to the discrete-time stochastic process example described above. The state space S = {1, 2, 3} is described by the nodes in this 3-state network, and the possible transitions between states are given by directed edges with associated (one time step) transition probabilities

Fig. 14

An illustration of the transition state diagram corresponding to the simple random walk on Z. The nodes are the integers and the only possible state transitions (in one time step) are to move by ± 1, also given by the directed edges with associated transition probabilities p or 1 − p for some p ∈ [0, 1]

We also give a brief overview of the basics of stochastic processes, including formal definitions of stochastic process, random walk, and Poisson process. We provide sample R code to simulate a simple random walk which is intended for use in Exercise 28 and can be adapted for Challenge Problem 3. Lastly, we provide R code to simulate a Poisson process for use in Exercises 33 and 34 and Research Project 3.

1.1 R Code for Figures

################################################################### # Figures in Section 3 ################################################################### # First, install the igraph package and load the igraph library library(igraph) # FIGURE 3 # Sample connected undirected network, N is number of nodes N=10; g <- erdos.renyi.game(N,0.25) iglayout1 = layout.fruchterman.reingold(g) plot(g, layout=iglayout1, vertex.size=20, vertex.label.dist=0, vertex.color="lightblue") # Sample disconnected undirected network N=10; g <- erdos.renyi.game(N,0.1) iglayout1 = layout.fruchterman.reingold(g) plot(g, layout=iglayout1, vertex.size=20, vertex.label.dist=0, vertex.color="lightblue") # FIGURE 4 # Sample directed network, weakly connected N=10; g <- erdos.renyi.game(N,0.1,directed = TRUE) iglayout1 = layout.fruchterman.reingold(g) plot(g, layout=iglayout1, vertex.size=20, vertex.label.dist=0, vertex.color="lightblue") # FIGURE 5 # Sample undirected network and adjacency matrix N=4; g <- erdos.renyi.game(N,0.5) iglayout1 = layout.fruchterman.reingold(g) plot(g, layout=iglayout1, vertex.size=20, vertex.label.dist=0, vertex.color="lightblue") # Retrieves the adjacency matrix used in the above graph adj <- get.adjacency(g) adj # FIGURE 6 # Sample directed network and adjacency matrix N=4; g <- sample_pa(N, directed = TRUE) plot(g, layout=iglayout1, vertex.size=20, vertex.label.dist=0, vertex.color="lightblue") # Retrieves the adjacency matrix used in the above graph adj <- get.adjacency(g) adj ################################################################### # Figures in Section 4 ################################################################### # FIGURE 8 # Ring network g <- make_ring(10) plot(g, layout=layout_with_kk, vertex.color="lightblue") # Complete graph g <- make_full_graph(10) plot(g, layout=layout_with_kk, vertex.color="lightblue") # FIGURE 9 # Erdos-Renyi network N=30; p=0.2 g <- sample_gnp(N,p) plot(g, vertex.size=10, vertex.color="wheat") # hist(degree_distribution(g), 20, col = "wheat") # FIGURE 10 # Small-world network: Watts--Strogatz model N <- 20 # WS network with no rewiring (q=0) g1 = sample_smallworld(1, N, 3, 0) -- sample realization mean_distance(g1) transitivity(g1, type="average") # > mean_distance(g1) # [1] 2.105263 # > transitivity(g1, type="average") # [1] 0.6 # WS network with small rewiring (q=0.05) g2 = sample_smallworld(1, N, 3, 0.05) -- sample realization mean_distance(g2) transitivity(g2, type="average") # > mean_distance(g2) # [1] 1.936842 # > transitivity(g2, type="average") # [1] 0.4841667 g3 = sample_smallworld(1, N, 3, 1) -- sample realization mean_distance(g3) transitivity(g3, type="average") # > mean_distance(g3) # [1] 1.831579 # > transitivity(g3, type="average") # [1] 0.2939432 # Plot all three networks together in one row # Use layout option to keep nodes in a circle par(mfrow = c(1,3)) plot(g1, vertex.size=10, layout=layout.circle) plot(g2, vertex.size=10, layout=layout.circle) plot(g3, vertex.size=10, layout=layout.circle) # FIGURE 11 # Scale-free network: Barabasi-Albert model # This model is directed by default # Use "directed = FALSE" to make undirected g <- sample_pa(50, directed = FALSE, out.pref = TRUE) degree_distribution(g) plot(g, vertex.size=10, vertex.color="wheat")

1.2 Basics of Stochastic Processes

Knowledge of introductory probability is assumed. In particular, we assume knowledge of the definitions of random variable and conditional probability as well as familiarity with common probability distributions (e.g., exponential and Poisson distributions). For a review of these basics, see [39].

Definition 17

A stochastic process is a sequence of random variables {X(t) : t ∈ T}, where T is an index set typically thought of as time. In particular, for each t ∈ T, X(t) is a random variable and we refer to X(t) as the state of the process at time t.

The set of possible values that the random variable X(t) can take on is called the state space, S. The time index set T can be discrete or continuous, and the same goes for the state space S. However, here we focus on discrete-state processes since we are talking about processes evolving on the nodes of a finite size network. Often, the set of nodes is the state space S, as in the examples below, but this does not have to be the case. We could, for instance, define a stochastic process to occur on each node of the network; see the neuronal spiking example in Sect. 5.3 and other examples in Sect. 5. We will consider both discrete-time and continuous-time processes in this chapter.

As an example of a discrete-time stochastic process on a finite state space, let T = {0, 1, 2, … } and S = {1, 2, 3}. Then the process {X(t) : t = 0, 1, … } starts out in state 1, 2, or 3 and moves around on these three states according to the following rules:

• If the process is in state 1 at time t (for t ∈ T), mathematically this is denoted by X(t) = 1, then the process moves to state 2 (with probability 1) in the next time step, or X(t + 1) = 2. We can denote this transition by the following conditional probability:

  $$\displaystyle \begin{aligned} P(X(t+1)=2 ~|~X(t)=1) = 1.\end{aligned} $$

  (29)

• If X(t) = 2, then the process has two choices for where to move in the next time step: move to state 3 with probability $\frac {1}{2}$ or move back to state 1 with probability $\frac {1}{2}$. Using conditional probabilities, we have

  $$\displaystyle \begin{aligned} \begin{array}{rcl} P(X(t+1)=3 ~|~X(t)=2) = \frac{1}{2} \end{array} \end{aligned} $$

  (30)
  $$\displaystyle \begin{aligned} \begin{array}{rcl} P(X(t+1)=1 ~|~X(t)=2) = \frac{1}{2}. \end{array} \end{aligned} $$

  (31)

• Finally, if X(t) = 3, then the process must move back to state 2 in the next time step. Again, we represent this state transition using a conditional probability.

  $$\displaystyle \begin{aligned} P(X(t+1)=2 ~|~X(t)=3) = 1.\end{aligned} $$

  (32)

In general, these conditional probabilities are called transition probabilities which describe the probability of a transition from the current state i to next state j in one discrete-time step. This general notation is given by

$$\displaystyle \begin{aligned} P(X(t+1)=j ~|~X(t)=i) = p_{ij}, \end{aligned} $$

(33)

where each p _ij is the (i, j)th entry in the transition probability matrix, P. For this example,

$$\displaystyle \begin{aligned} \begin{array}{rcl} P = \begin{pmatrix} 0 &\displaystyle 1 &\displaystyle 0\\ \frac{1}{2} &\displaystyle 0 &\displaystyle \frac{1}{2}\\ 0 &\displaystyle 1 &\displaystyle 0 \end{pmatrix}\!. \end{array} \end{aligned} $$

(34)

Note that each row of P sums to 1. This is true in general since the sum over all possible one-step transition probabilities from a given state i to any other state j ∈ S must be 1.

The diagram in Fig. 13 is another way of illustrating this example process: the states are the nodes, the directed edges denote the possibility of a transition from one node to another, and the numbers above the edges denote the probability of such transitions. A sample path or realization of this process up through time t = 7 is

$$\displaystyle \begin{aligned} X(0){=}1, X(1){=}2, X(2){=}1, X(3){=}2, X(4){=}3, X(5){=}2, X(6){=}1, X(7){=}2, \dots.\end{aligned} $$

If we just list the sequence of states starting in state 1 and moving forward in time, this sample path looks like

$$\displaystyle \begin{aligned}1 \rightarrow 2 \rightarrow 1 \rightarrow 2 \rightarrow 3 \rightarrow 2 \rightarrow 1 \rightarrow 2 \dots.\end{aligned} $$

Another interesting stochastic process called a random walk is defined below. We define the discrete-time random walk on a discrete state space only (such as the integers Z); see [40] for further reading on other varieties of random walks. Additionally, we focus on the simple random walk which can only move in increments of ± 1.

Definition 18

A discrete-time simple random walk on state space S = Z is a stochastic process {Y (t) : t = 0, 1, 2, … } such that the state of the process increases or decreases by 1 at each time step with probabilities p and 1 − p, respectively. These transition probabilities are given by

$$\displaystyle \begin{aligned} \begin{array}{rcl} P(Y(t+1)=i+1 ~|~Y(t)=i) &\displaystyle =&\displaystyle p \end{array} \end{aligned} $$

(35)

$$\displaystyle \begin{aligned} \begin{array}{rcl} P(Y(t+1)=i-1 ~|~Y(t)=i) &\displaystyle =&\displaystyle 1-p \end{array} \end{aligned} $$

(36)

for some p ∈ [0, 1].

See Fig. 14 for a graphical representation of the simple random walk process on Z. Suppose the simple random walk {Y (t) : t = 0, 1, 2, … } starts in state 0 at time 0. Then a sample path of the process is

$$\displaystyle \begin{aligned} Y(0){=}0, Y(1){=}-1, Y(2){=}0, Y(3){=}1, Y(4){=}2, Y(5){=}3, Y(6){=}2, Y(7){=}1,\dots. \end{aligned}$$

We provide R code at the end of the Appendix to generate a simple random walk. Using this code, we generate and plot five sample paths of the symmetric simple random walk in Fig. 15 below. The symmetric case means that the parameter p = 1∕2 = 1 − p.

Fig. 15

Sample realizations from the simple random walk simulation on the integers, symmetric case $p= \frac {1}{2}$. Initial state is 0 in all cases

Note that due to the random nature of a stochastic process such as this random walk, each simulation run will be different. In general, in order to draw meaningful conclusions about a given stochastic process, one needs to run multiple simulations with the same parameter settings and look at the distribution of outcomes.

Lastly, we define a Poisson process which is a continuous-time stochastic process that takes values in the discrete state space S = Z ^∗, the non-negative integers. This process is used in the neuronal spiking process described in Sect. 5.3. Poisson processes are commonly used to model a variety of biological processes.

Definition 19

A Poisson process on state space S = Z ^∗ (non-negative integers) is a continuous-time stochastic process {N(t) : t ≥ 0} that counts events that have occurred up to and including time t. Specifically, {N(t) : t ≥ 0} is a Poisson process with rate λ > 0 if

• N(0) = 0.

• {N(t) : t ≥ 0} has independent increments, meaning that for any 0 ≤ t ₀ < t ₁ < ⋯ < t _n < ∞, N(t ₀), N(t ₁) − N(t ₀), …, N(t _n) − N(t _n−1) are all independent random variables.

• N(s + t) − N(s) ∼Poisson(λt) for any s, t ≥ 0, meaning that the number of events in any interval of length t follows a Poisson distribution with rate λt.

For instance, let λ = 1. Then the expected number of events that will occur by time t is λt = t since λ = 1 in this case. In other words, on average, one event will occur per unit time. This follows because N(t) = N(t) − N(0) ∼Poisson(λt) and E[N(t)] = λt, a property of the Poisson distribution [40].

Now let us describe a sample path for a Poisson process {N(t) : t ≥ 0} with rate λ = 1. We assume that at time t = 0 no events have occurred (since N(0) = 0), so the first event will occur after an exponentially distributed amount of time with rate (parameter) λ. Starting at the time of the first event, the amount of time we have to wait until the second even occurs follows an exponential distribution with rate λ. It follows that the length of time between any two successive events in the Poisson process also follows an exponential distribution with rate λ. The times between events are called interarrival times, and a key property of the Poisson process is that the interarrival times follow an exponential distribution. Hence, the average length of the interarrival times is given by 1∕λ, or the mean of an exponential distribution with rate λ.

A sample path of a Poisson process {N(t) : t ≥ 0} with rate λ = 1 is given below.

$$\displaystyle \begin{aligned} \begin{array}{rcl} &\displaystyle &\displaystyle N(0)=0, N(0.062)=1, N(0.88)=2, N(1.02){=}3, N(2.38){=}4, N(3.09){=}5,\\ &\displaystyle &\displaystyle N(5.94)=6, ~N(8.04)=7, ~N(8.92)=8, ~N(9.65)=9, ~N(10.10)=10, \dots. \end{array} \end{aligned} $$

Note that the state of the process increases by 1 at each time step, and the time increments are given by random draws from the exponential distribution with rate λ = 1. The expected number of events by time t = 10 is 10, and in this sample path we have 9 events that occurred by time t = 10 since N(9.65) = 9 and N(10.10) = 10. The tenth event occurred slightly after time t = 10, specifically at t = 10.10 in this case. See Fig. 16 for a plot of this sample path, and see R code used to generate this plot below. In general, a widely used method for simulating continuous-time discrete-state stochastic processes is known as the Gillespie Stochastic Simulation Algorithm (SSA) [24]; see also the GillespieSSA package in R.

Fig. 16

A plot of the sample path detailed above from a Poisson process with rate λ = 1

For more reading on introductory stochastic processes, random walks, and Poisson processes, see [19, 40].

1.3 R Code for Selected Stochastic Processes

################################################################### # Simulate a random walk ################################################################### # Simple random walk on the integer line with probability p of # moving +1 step (right), probability 1-p of moving -1 step (left) # Parameters: n=number of time steps, p=prob increase state by 1, # x1=initial state RW.sim <- function(n,p,x1) { sim <- as.numeric(n) if (missing(x1)) { sim[1] <- 0  # initial condition: start at state 0 } else { sim[1] <- x1 } for (i in 2:n) { newstate <- sample(c(-1,1),1,prob=c(1-p,p)) + x1 sim[i] <- newstate x1 = newstate } sim } # Generate 5 sample paths and plot all on the same figure p = 0.5  # symmetric case run <- RW.sim(100,p,0) plot(run, type="l", col="blue", ylim=c(-18,18), xlab="Time step", ylab="State", main = "Simple Random Walk") run <- RW.sim(100,p,0) points(run, type="l", col="red") run <- RW.sim(100,p,0) points(run, type="l", col="darkgreen") run <- RW.sim(100,p,0) points(run, type="l", col="orange") run <- RW.sim(100,p,0) points(run, type="l", col="magenta") ################################################################### # Simulate a Poisson process ################################################################### # Install poisson package and load library library(poisson) # Simulate a homogeneous Poisson process with a given rate # Parameters: rate=rate of Poisson process, num.events=number # of events to simulate, num.sims=number of times to run the # simulation (default is 1) PP.sim <- function (rate, num.events, num.sims = 1, t0 = 0) { if (num.sims == 1) { x = t0 + cumsum(rexp(n = num.events, rate = rate)) return(c(t0,x)) } else { xtemp = matrix(rexp(n=num.events∗num.sims, rate=rate), num.events) x = t0 + apply(xtemp, 2, cumsum) return(rbind(rep(t0, num.sims), x)) } } # Run the function PP.sim rate = 1 PP.sim(rate, num.events = 10) PP.sim(rate, num.events = 10, num.sims = 5) # Run the function and plot the time series for 1 simulation rate = 1 num.events = 10 run1 <- PP.sim(rate, num.events) plot(run1, 0:num.events, xlab="Time", ylab="Number of Events", main = "Poisson Process Sample Path", pch = 19, col="blue") run1