# Linear Autonomous Compartmental Models as Continuous-Time Markov Chains: Transit-Time and Age Distributions

- 488 Downloads
- 1 Citations

## Abstract

Linear compartmental models are commonly used in different areas of science, particularly in modeling the cycles of carbon and other biogeochemical elements. The representation of these models as linear autonomous compartmental systems allows different model structures and parameterizations to be compared. In particular, measures such as system age and transit time are useful model diagnostics. However, compact mathematical expressions describing their probability distributions remain to be derived. This paper transfers the theory of open linear autonomous compartmental systems to the theory of absorbing continuous-time Markov chains and concludes that the underlying structure of all open linear autonomous compartmental systems is the phase-type distribution. This probability distribution generalizes the exponential distribution from its application to one-compartment systems to multiple-compartment systems. Furthermore, this paper shows that important system diagnostics have natural probabilistic counterparts. For example, in steady state the system’s transit time coincides with the absorption time of a related Markov chain, whereas the system age and compartment ages correspond with backward recurrence times of an appropriate renewal process. These relations yield simple explicit formulas for the system diagnostics that are applied to one linear and one nonlinear carbon-cycle model in steady state. Earlier results for transit-time and system-age densities of simple systems are found to be special cases of probability density functions of phase-type. The new explicit formulas make costly long-term simulations to obtain and analyze the age structure of open linear autonomous compartmental systems in steady state unnecessary.

### Keywords

Carbon cycle Compartmental system Phase-type distribution Pool system Regenerative process Reservoir model## 1 Introduction

Compartmental models are widely used to describe a range of biological and physical processes that rely on mass conservation principles (Anderson 1983; Jacquez and Simon 1993). In particular, most models that describe the carbon cycle can be generalized as compartmental models (Luo and Weng 2011; Sierra and Müller 2015; Rasmussen et al. 2016). These models are very important for predicting interactions between ecosystems and the global climate. However, the predictions of these models diverge widely (Friedlingstein et al. 2006, 2014) since models have different structures and parameter values. To compare diverse models, key quantities such as system age and transit time can be considered. These quantities provide relevant information about the time scales at which material is cycled in compartmental systems and facilitate comparisons among different model structures and parameterizations. Formulas for their means were recently provided by Rasmussen et al. (2016) for linear nonautonomous compartmental systems. For autonomous systems, that is, systems with constant coefficients, the existence of explicit solutions raises hope for explicit formulas not only for the means, but also for the densities of system age and transit time.

A first attempt in this direction was started by Nir and Lewis (1975) who established formulas for transit-time and age densities in dependence on a not explicitly known system response function. This response function was the basis for Thompson and Randerson (1999) to compute the desired densities numerically by long-term simulations in two carbon-cycle models. Impulsive inputs and how systems respond to them were later investigated by Manzoni et al. (2009) to obtain explicitly the system response function for a set of carbon-cycle models of very simple structure. Despite the simple structure of these systems, the derivation of the according density formulas is involved, because the system’s response needs to be transformed from the phase domain to the Laplace domain and back. In this manuscript, the dynamics of linear autonomous compartmental systems are considered as a stochastic process to show that the impulse response function is the probability density function of a phase-type distribution. To this end, open linear autonomous compartmental systems are related to absorbing continuous-time Markov chains.

This manuscript is organized as follows. In Sect. 2, linear autonomous compartmental systems are introduced, and the idea of looking at them from the perspective of one single particle is framed. The transit time of this particle through the system coincides with the absorption time of a continuous-time Markov chain. A short presentation of the basic ideas of continuous-time Markov chains follows in Sect. 3. The probability distribution of the absorption time is shown to be an example of a phase-type distribution, and its cumulative distribution function, probability density function, and moments are computed. In Sect. 4, concepts from renewal theory are used to construct a regenerative process, which is the basis for the computation of the system- and compartment-age densities. The relationship of Markov chains and linear autonomous compartmental systems is established in Sect. 5, where the previous probabilistic results are used to obtain simple explicit formulas for the densities of the system age, compartment age, and transit time. In Sect. 6, the derived formulas are applied to two autonomous terrestrial carbon-cycle models. Both models are considered in steady state, which allows the treatment of a linear model (Emanuel et al. 1981), as well as a nonlinear one (Wang et al. 2014) within the proposed framework. Relations to earlier work and some other aspects are discussed in Sect. 7, and in Sect. 8 conclusions are presented. “Appendix A” contains the proof of the main theorem from Sect. 4, and “Appendix B” covers some examples for systems with simple structure.

## 2 Linear Autonomous Compartmental Systems

Compartmental differential equations are useful tools to describe flows of material between units called compartments under the constraint of mass conservation. Following Jacquez and Simon (1993), a compartment is an amount of some material that is kinetically homogeneous. This means that the material of a compartment is at all times homogeneous; any material entering the compartment is instantaneously mixed with the material already there. Hence, compartments are always well-mixed.

*d*-dimensional linear system of differential equations

### Definition 1

- (i)
\(0\le b_{ij}\) for all \(i\ne j\),

- (ii)
\(0\le -b_{jj}<\infty \) for all

*j*, - (iii)
\(\sum _{i=1}^d b_{ij} \le 0\) for all

*j*.

If now \({\mathsf {B}}\) is a compartmental matrix, (1) is called a linear autonomous compartmental system. The term autonomous refers to the fact that both \({\mathsf {B}}\) and \(\mathbf {u}\) are independent of time *t*. A detailed treatment of such systems can be found in Anderson (1983) and Jacquez and Simon (1993).

For \(t\ge 0\) the vector \(\mathbf {x}(t)\in \mathbb {R}^d\) represents the contents of the different compartments at time *t*. The off-diagonal entries \(b_{ij}\) of \({\mathsf {B}}\) are called fractional transfer coefficients. They are the rates at which mass moves from compartment *j* to compartment *i*. For \(j=1,\ldots ,d\), the nonnegative value \(z_j=-\sum _{i=1}^d b_{ij}\) is the rate at which mass leaves the system from compartment *j*. If at least one of these output or release rates is greater than zero, system (1) is called open, otherwise it is called closed. This paper is concerned with open systems only, since closed systems with positive input accumulate mass indefinitely.

### 2.1 The One-Particle Perspective

Looking at the behavior of the entire system, the initial content \(\mathbf {x^0}\) distorts what can be seen happening to the system content. There are two ways to get rid of this disturbing influence of the initial content. One way is to consider the system after it has run for an infinite time such that all the initial content has left. Another way is to look at one single particle that just arrives at the system through new input \(\mathbf {u}\).

Since all compartments are well-mixed, this particle’s way through the system is not influenced by the presence or absence of any other particles. It enters the system at a compartment according to \(\mathbf {u}\) and then, at each time step, whether it stays or moves on is decided on the basis of its current position and its schedule. If the decision is to move on, then it can move to another compartment or leave the system, depending only on the connections of the current compartment. The particle follows a schedule and a map given by the matrix \({\mathsf {B}}\). Its diagonal entries govern how long the particle stays in a certain compartment, and the off-diagonal entries provide the connections to other compartments. By leaving the system, the particle finishes a cycle and starts a new one by reentering the system. At each cycle, the sequence of compartments to which the particle belongs at successive time steps constitutes a stochastic process called discrete-time Markov chain. Letting the size of the time steps tend to zero, the particle’s future becomes continuously uncertain. The path of the particle traveling through the system is then represented by a continuous-time Markov chain (Norris 1997).

In Sect. 2.2, the structure and properties of continuous-time Markov chains are introduced, and the reader should always have in mind the traveling particle. When the Markov chain changes its state from *j* to *i*, the particle is considered to move from compartment *j* to compartment *i*. When the Markov chain is absorbed, the particle leaves the system. The time that elapses from the moment the particle enters the system to the moment of its exit is called the particle’s transit time. To get a grasp on it, from the beginning of the cycle only a look into the future is necessary.

While the particle is still in the system, the time that has passed since the particle’s entry is called its system age. If the age of the particle is considered at a random time, a look into the past of the particle is needed, not knowing when it entered the system the last time. Consequently, it has to be assumed that the particle has ran through the system already infinitely often. Otherwise the existence of a maximum age of the particle would be implied, but any choice of this maximum value would be arbitrary and ill-founded. Therefore, a particular regenerative process will be considered: a sequence of absorbing continuous-time Markov chains.

### 2.2 From One Particle to All Particles in the System

*n*particles with system ages represented by

*n*independent and identically distributed random variables. Hence, the system age \(A_k\) of particle

*k*is assumed to have the cumulative distribution function \(F_A\) for all \(k=1,2,\ldots ,n\). The share of particles with system age less than or equal to \(y\ge 0\) equals

*n*of particles goes to infinity. Consequently, if the system contains an infinite amount of particles, the share of the total system content \(\mathbf {x}(t)\) that has system age less than or equal to

*y*is

## 3 Continuous-Time Markov Chains

Markov chains are the most important examples of random processes. Given their simple structure and high diversity, they are applied to many different scientific problems. In particular, they are the simplest mathematical models for random phenomena evolving in time. Their characteristic property is that they retain no memory on the states of the system in the past. Only the current state of the process can influence where it goes next. If the process can assume only a finite or countable set of states, it is called a Markov chain. Discrete-time Markov chains are usually defined on a set of integers, whereas continuous-time Markov chains live on a subset of the real line. In the following, the basic theory of continuous-time Markov chains is introduced along the line of Norris (1997), from which also most unproven results are taken.

A vector \({\varvec{\lambda }}\in \mathbb {R}^d\) is called a distribution if all its components are nonnegative and sum to one. Furthermore, a matrix \({\mathsf {P}}=(p_{ij})_{i,j\in J}\) is called stochastic on a finite set *J* if every column \((p_{ij})_{i\in J}\) is a distribution. Note that in standard literature on Markov chains row sums are considered. Here, column sums are used instead, because thereby the connection to compartmental matrices will become more obvious later. The definition of a Markov chain can best be formalized in terms of its corresponding transition rate matrix. A transition rate matrix is a special compartmental matrix where all columns sum to zero.

### Definition 2

*J*be a finite state space. A transition rate matrix on

*J*is a matrix \({\mathsf {Q}}=(q_{ij})_{i,j\in J}\) satisfying the conditions

- (i)
\(0\le q_{ij}\) for all \(i\ne j\),

- (ii)
\(0\le -q_{jj}<\infty \) for all

*j*, - (iii)
\(\sum _{i\in J} q_{ij} = 0\) for all

*j*.

Each off-diagonal entry \(q_{ij}\) can be interpreted as the rate of going from state *j* to state *i* and the diagonal entries \(q_{jj}\) as the rate of leaving state *j*.

### Definition 3

*J*. A stochastic process \(X=(X_t)_{t\ge 0}\) is a continuous-time Markov chain on

*J*with initial distribution \({\varvec{\lambda }}\) and transition rate matrix \({\mathsf {Q}}\) if

- (i)
\(\mathbb {P}(X_0=j) = \lambda _j\) for all \(j\in J\);

- (ii)for all \(n=0,1,2,\ldots \), all times \(0\le t_0\le t_1\le \cdots \le t_{n+1}\), and \(j_0,j_1,\ldots ,j_{n+1}\in J\),where for any quadratic matrix \({\mathsf {M}}\) the expression \(\mathrm{e}^{\mathsf {M}}\) denotes the matrix exponential$$\begin{aligned} \mathbb {P}\left( X_{t_{n+1}} = j_{n+1}\,|\,X_{t_0}=j_0,\ldots ,X_{t_n}=j_n\right) =\left( \mathrm{e}^{(t_{n+1}-t_n)\,{\mathsf {Q}}}\right) _{j_{n+1} j_n}, \end{aligned}$$$$\begin{aligned} \mathrm{e}^{\mathsf {M}} = \sum \limits _{k=0}^\infty \frac{{\mathsf {M}}^k}{k!}. \end{aligned}$$

Since \({\mathsf {Q}}\) does not depend on time, *X* is called homogeneous. Property (ii) states that the future evolution of a Markov process depends only on its current state and not on its history. This is called Markov property.

*X*be a continuous-time Markov chain on

*J*with initial distribution \({\varvec{\lambda }}\) and transition rate matrix \({\mathsf {Q}}\). Then, for \(i,j\in J\), the probability of being in state

*i*at time

*t*having started in state

*j*is equal to

*i*at time

*t*(conditioned on the initial distribution \({\varvec{\lambda }}\)) is

### 3.1 Absorbing Continuous-Time Markov Chains

*J*is assumed to be equal to \(\{1,2,\ldots ,d, d+1\}\) for some natural number \(d\ge 1\), and its transition rate matrix has the shape

*d*-dimensional column vector containing only zeros. Let \(S:=\{1,2,\ldots ,d\}\subseteq J\). The \(d\times d\)-matrix \({\mathsf {B}}=(b_{ij})_{i,j\in S}\) is assumed to meet the requirements (i) and (ii) of a transition rate matrix, but instead of property (iii) of Definition 2 it fulfills only the weaker condition

*d*-dimensional row vector comprising ones. This means that the \(z_j\) are nonnegative and denote the transition rates from

*j*to \(d+1\). The \((d+1)\)st column of \({\mathsf {Q}}\) contains only zeros. Consequently, the process

*X*cannot change its state anymore once it has reached state \(d+1\). For that reason, \(d+1\) is called the absorbing state of

*X*. The trivial case in which the process starts in its absorbing state is excluded by considering only initial distributions \({\varvec{\lambda }}\) with \(\lambda _{d+1}=0\). Then the new initial distribution of

*X*is defined as

*d*-dimensional row vector. This means for \(i,j\in S\) that

### 3.2 The Absorption Time

### Lemma 1

If \({\mathsf {B}}\) is nonsingular, then the absorption time *T* is finite with probability one.

### Definition 4

A Markov chain *X* on \(J = \{1,2,\ldots ,d,d+1\}\) whose transition rate matrix \({\mathsf {Q}}\) has the structure of Eq. (2) and that will eventually be absorbed to \(d+1\) with probability one is called absorbing. The matrix \({\mathsf {B}}\) is called its transition rate matrix, \(S=\{1,2,\ldots ,d\}\) its state space, and \(d+1\) its absorbing state. If \({\varvec{\lambda }}\) denotes the initial distribution of *X*, then \({\varvec{\beta }}\) as defined in Eq. (4) is called the initial distribution of the absorbing chain.

The following results related to absorbing continuous-time Markov chains will be used in the forthcoming.

### Remark 1

### Lemma 2

### Remark 2

Since \({\mathsf {Q}}\) is a transition rate matrix, \(\mathrm{e}^{t\,{\mathsf {Q}}}\) is stochastic. This makes all of its entries and hence all entries of \(\mathrm{e}^{t\,{\mathsf {B}}}\) nonnegative. From Eq. (6) it follows that also all entries of \(-{\mathsf {B}}^{-1}\) are nonnegative.

From now on, let \({\mathsf {B}}\) be the nonsingular transition rate matrix of an absorbing continuous-time Markov chain \(X=(X_t)_{t\ge 0}\) on \(S=\{1,2,\ldots ,d\}\) with initial distribution \({\varvec{\beta }}\) and absorbing state \(d+1\). As it turns out, the absorption time *T* follows a phase-type distribution that depends on the initial distribution \({\varvec{\beta }}\) and on the transition rate matrix \({\mathsf {B}}\).

#### 3.2.1 Probability Distributions of Phase-Type

Phase-type distributions constitute a highly versatile class of probability distributions and are closely related to the solutions of systems of linear differential equations with constant coefficients. As mixtures of exponential distributions, they generalize the Erlang, hypoexponential, and hyperexponential distribution. These are widely used in queuing theory and the field of renewal processes. An introduction to phase-type distributions and the unifying matrix formalism used in this paper can be found in Neuts (1981).

*t*, the cumulative distribution function \(F_T(t)=\mathbb {P}(T\le t)\) of the absorption time

*T*is equal to the probability of not being in any of the states \(j\in S\). Consequently, using Eq. (5) leads to

#### 3.2.2 Moments of the Phase-Type Distribution

*n*th moment of the phase-type distribution. Repeated integration by parts leads to

Note that the variance, that is the second central moment, of the phase-type distribution can be obtained from \(\sigma ^2_T = \mathbb {E}[T^2] - (\mathbb {E}[T])^2\).

#### 3.2.3 A Closure Property of the Phase-Type Distributions

The class of phase-type distributions has several closure properties, one of which is given by the following lemma (Neuts 1981, Theorem 2.2.3).

### Lemma 3

### 3.3 Occupation Time

*X*takes on different states \(j\in S\). Its occupation time of state

*j*is defined as the time the process spends in state

*j*. It is given as the nonnegative random variable

*i*before absorption, under the condition that it started in state

*j*.

### 3.4 The Last State Before Absorption

*E*denote the state from which

*X*jumps to the absorbing state. From the homogeneity of

*X*follows

*j*.

## 4 A Regenerative Process

Assume that an absorbing continuous-time Markov chain is restarted immediately every time it hits the absorbing state. The goal of this section is to determine the distribution of the age of this new process at a random time \(\tau \) drawn from the positive half-line (technical details on \(\tau \) are presented in “Appendix A”). Age means here the time that has passed since the last restart.

First, the new situation is set up, then the distribution of the age at a random time is derived, and as a last step the age distribution at a random time under the condition of being in a fixed state is computed. Important tools are elements from renewal theory such as renewal and regenerative processes. A comprehensive treatise of this topic can be found in Asmussen (2003).

### 4.1 Definition of the Regenerative Process

Let *X* be an absorbing continuous-time Markov chain on a finite state space *J* with transition rate matrix \({\mathsf {B}}\) and initial distribution \({\varvec{\beta }}\). Now, the process *X* is stopped at its absorption time *T*, an immediate restart is executed, and this procedure is repeated over and over again. This gives an infinite sequence \((X^k)_{k=1,2,\ldots }\) of independent and identically distributed cycle processes, where each cycle process behaves like the process *X* up to absorption. This sequence constitutes a regenerative process \(Z=(Z_t)_{t\ge 0}\) with \(Z_t=X^k_t\) for \(t\in [T_{k-1},T_k)\), where \(T_0:=0\). The process that counts the number of restarts is called a renewal process.

A regenerative process is called alternating, if it has only the two possible states 1 and 0. Let \(Y = (Y_t)_{t\ge 0}\) be an arbitrary alternating regenerative process with the same cycle lengths as *Z*, that is \(Y_t = Y^k_t\) for \(t\in [T_{k-1},T_k)\). The alternating process *Y* is called on at time *t* if \(Y_t=1\), otherwise it is called off. Note that \(X^k_t\) and \(Y^k_t\) are defined to be zero if \(t\notin [T_{k-1},T_k)\). Such alternating processes together with the following theorem build the main tool for the computation of the age distributions.

### Theorem 1

*Y*with the same cycle lengths as

*Z*is on at a random time \(\tau \ge 0\) equals the ratio of expected on-time during the first cycle to the average cycle length. This leads to

Despite the intuitive nature of this result, its proof requires quite some technical effort and it is postponed to “Appendix A”.

### 4.2 The Age of the Regenerative Process

Consider the age process \((A_t)_{t\ge 0}\) defined such that the random variable \(A_t\) describes the time that has passed by *t* since the last restart. It is also called backward recurrence time of the renewal process. It can be considered the age of the regenerative process *Z* at time *t*. For \(y\ge 0\) and a randomly chosen time \(\tau \), the probability \(F_{A_\tau }(y)=\mathbb {P}(A_\tau \le y)\) is to be computed.

*Y*is constructed that is on at time

*t*if and only if the time elapsed since the last restart is less than or equal to

*y*. Obviously, \(\mathbb {P}(A_\tau \le y) = \mathbb {P}(Y_\tau =1)\). Taking into account that \(Y^1_t=1\) if and only if \(t<T_1\) and \(t\le y\), it follows that

*A*instead of \(A_\tau \) and apply Theorem 1 to the alternating process

*Y*to obtain the cumulative distribution function

*Z*. Hence, this age is again phase-type distributed by Lemma 3, now with initial probability vector \({\varvec{\eta }}\).

### 4.3 The Age of the Regenerative Process in a Fixed State

*Z*given that it is in state

*j*. For \(y\ge 0\) this means

*Y*and \(\tilde{Y}\), respectively. The process

*Y*is defined to be on at time

*t*if and only if \(A_t\le y\) and \(Z_t=j\), whereas \(\tilde{Y}\) is defined to be on if and only if

*Z*is in state

*j*. Invoke Theorem 1 to

*Y*and \(\tilde{Y}\) to get

*Z*in state

*j*is given by

## 5 Compartmental Systems and Markov Chains

*j*at time \(t\ge 0\) is then given by \(\tilde{x}_j(t)=\left( \mathrm{e}^{t\,{\mathsf {B}}}\,\mathbf {u}\right) _j\).

*X*with initial distribution \({\varvec{\beta }}\) being in state \(j\in S:=\{1,2,\ldots ,d\}\) at time

*t*is by an application of Eq. (5) given by

*j*at time

*t*. Consequently, the continuous-time Markov chain

*X*describes the stochastic travel of a single particle through the compartmental system. When the traveling particle leaves the compartmental system, the process

*X*jumps to the absorbing state \(d+1\).

### 5.1 Global Asymptotic Stability

Assume the compartmental matrix \({\mathsf {B}}\) to be nonsingular. Then the constant vector \(\mathbf {x}^*= -{\mathsf {B}}^{-1}\,\mathbf {u}\) is a steady-state solution or equilibrium of system (1), that is, \(\mathrm{d}\,\mathbf {x}^*/\mathrm{d}t = \mathbf {0}\).

It is well known that system (1) is globally asymptotically stable, if \({\mathsf {B}}\) is strictly diagonally dominant, that is \(\sum _{i=1}^d b_{ij}<0\) for all \(j\in S\). However, here \({\mathsf {B}}\) is not strictly diagonally dominant (it is diagonally dominant, though), but it is a nonsingular compartmental matrix. This means that \({\mathsf {B}}\) is the transition rate matrix of an absorbing continuous-time Markov chain. For the absorbing state to be reached eventually, at least one rate \(z_j=-\sum _{i=1}^d b_{ij}\) of leaving the system from compartment *j* must be strictly greater than zero. This makes system (1) an open system (Jacquez and Simon 1993).

### 5.2 Steady-State Compartment Contents

*j*is \(x^*_j=-\left( {\mathsf {B}}^{-1}\,\mathbf {u}\right) _j\). Plugging in \({\varvec{\beta }}=\mathbf {u}/\Vert \mathbf {u}\Vert \) gives

*j*by the absorbing continuous-time Markov chain

*X*.

### 5.3 Release from the System

For \(j\in S\), the release of mass from compartment *j* to the environment is denoted by \(r_j\). As a function of time *t*, it can be computed as product of the rate \(z_j\) of mass leaving compartment *j* toward the environment and the mass \(x_j(t)\) contained in compartment *j* at time *t*. For a system in steady state, \(x_j(t)=x_j^*\) remains constant and consequently \(r_j=z_j\,x_j^*\) remains constant as well.

*X*to be absorbed through state

*j*. From Eq. (9) the probability of

*j*being the last state before absorption is \(\mathbb {P}(E=j)=-z_j\,\left( {\mathsf {B}}^{-1}\,{\varvec{\beta }}\right) _j\). Use \({\varvec{\beta }}=\mathbf {u}/\Vert \mathbf {u}\Vert \) and \(\mathbf {x}^*=-{\mathsf {B}}^{-1}\,\mathbf {u}\) to get

*j*is proportional to the probability of

*X*being absorbed through state

*j*. Since absorption is certain, \(\sum _{j\in S}\mathbb {P}(E=j) = 1\), hence \(\Vert \mathbf {r}\Vert =\sum _{j\in S} r_j = \Vert \mathbf {u}\Vert \), reflecting that in steady state total system output equals total system input.

### 5.4 Age Distribution of the System in Steady State

Suppose that the total system content in steady state has an unknown age density \(f_A\). In Sect. 4.2, it was already considered a particle that travels over and over again through the system, and its age at a random time was computed by methods from renewal theory. The according regenerative process expresses the need for an infinite history. Consequently, it reflects the behavior of the compartmental system in steady state, when all initial mass has left the system.

*A*was shown to be phase-type distributed with transition rate matrix \({\mathsf {B}}\) and initial distribution

### 5.5 Age Distribution of the Compartments in Steady State

*Z*was calculated under the condition that its state equals

*j*. This is the age that a traveling particle has when it is in compartment

*j*. From Eq. (14) and \(\mathbf {x}^*=-{\mathsf {B}}^{-1}\,\mathbf {u}\), it follows that

*n*th moments of the compartment ages. It is given by

### 5.6 Transit Time in Steady State

For compartmental systems two types of transit times can be considered (Nir and Lewis 1975). The forward transit time (\({\text {FTT}}\)) at time \(t_a\) is the time *t* a particle needs to travel through the system after it arrives at time \(t_a\). The backward transit time (\({\text {BTT}}\)) at time \(t_e\) gives the age *y* a particle has at the moment it leaves the system, that is, the time it needed to travel through the system given that it exits at time \(t_e\). For an autonomous system in steady state one would expect the two types of transit time to coincide.

*X*. As soon as the traveling particle leaves the system,

*X*hits its absorbing state. Hence, the \({\text {FTT}}\) is phase-type distributed with initial distribution \({\varvec{\beta }}\) and transition rate matrix \({\mathsf {B}}\). This makes its density function

*Z*at the time of a restart. Hence, it follows the same distribution as the cycle length and is identically distributed to the absorption time. So, from the probabilistic point of view, \({\text {FTT}}\) and \({\text {BTT}}\) are obviously the same. But the density of the \({\text {BTT}}\) can also be calculated as a weighted average of ages of particles that are leaving the system by

## 6 Application to Carbon Cycle Models

### 6.1 A Linear Autonomous Compartmental System in Steady State

*T*is phase-type distributed with probability density function (Fig. 1)

### 6.2 A Nonlinear Autonomous Compartmental System in Steady State

Assume now that system (22) is in a steady state \(\mathbf {x}^*\). From \(\mathrm{d}\,\mathbf {x}^*/\mathrm{d}t = 0\) follows that the compartment contents \(x_j\) do not change and the mapping *B* turns into a matrix \({\mathsf {B}}\) with constant coefficients. Hence, assuming the nonlinear autonomous compartmental system (22) is in a steady state, it can be treated as a linear autonomous compartmental system and the entire theory developed for those systems can be applied to it.

*B*depends on \(\mathbf {x}=\left( \begin{matrix}C_{s}&C_{b}\end{matrix}\right) ^T\) through \(\lambda \)’s dependence on \(\mathbf {x}\), which is given by

The graph of the mean system age for this model with \(\varepsilon = 0.39\) (Fig. 4) lies directly on the one of the mean transit time. The huge difference in the compartments’ steady-state contents causes very little difference in the initial distributions \({\varvec{\beta }}\) and \({\varvec{\eta }}\) for the traveling particle. This results in very similar distributions of transit time and system age.

## 7 Discussion

Overview of derived formulas for open linear autonomous compartmental systems \(\frac{\mathrm{d}}{\mathrm{d}t}\,\mathbf {x}(t)={\mathsf {B}}\,\mathbf {x}(t)+\mathbf {u}\)

Metric | Density |
| First moment |
---|---|---|---|

Transit time | \(\mathbf {z}^T\,\mathrm{e}^{t\,{\mathsf {B}}}\,\frac{\mathbf {u}}{\Vert \mathbf {u}\Vert }\) | \((-1)^n\,n!\,\mathbf {1}^T\,{\mathsf {B}}^{-n}\,\frac{\mathbf {u}}{\Vert \mathbf {u}\Vert }\) | \(-\mathbf {1}^T\,{\mathsf {B}}^{-1}\,\frac{\mathbf {u}}{\Vert \mathbf {u}\Vert }\), \(\frac{\Vert \mathbf {x}^*\Vert }{\Vert \mathbf {u}\Vert }\) |

System age | \(\mathbf {z}^T\,\mathrm{e}^{y\,{\mathsf {B}}}\,\frac{\mathbf {x}^*}{\Vert \mathbf {x}^*\Vert }\) | \((-1)^n\,n!\,\mathbf {1}^T\,{\mathsf {B}}^{-n}\,\frac{\mathbf {x}^*}{\Vert \mathbf {x}^*\Vert }\) | \(-\mathbf {1}^T\,{\mathsf {B}}^{-1}\,\frac{\mathbf {x}^*}{\Vert \mathbf {x}^*\Vert }\), \(\frac{\Vert {\mathsf {B}}^{-1}\,\mathbf {x}^*\Vert }{\Vert \mathbf {x}^*\Vert }\) |

Age vector | \(({\mathsf {X^*}})^{-1}\,\mathrm{e}^{y\,{\mathsf {B}}}\,\mathbf {u}\) | \((-1)^n\,n!\,({\mathsf {X^*}})^{-1}\,{\mathsf {B}}^{-n}\,\mathbf {x}^*\) | \(-({\mathsf {X^*}})^{-1}\,{\mathsf {B}}^{-1}\,\mathbf {x}^*\) |

### 7.1 Relation to Earlier Results

Over many years compartmental models with fixed coefficients have been studied (Anderson 1983; Bolin and Rodhe 1973; Eriksson 1971) by mainly investigating the mean age of particles in the system and the mean transit time of particles. The topic became of increasing interest again with the need for modeling the terrestrial carbon cycle.

The general situation of linear nonautonomous compartmental models was investigated by Rasmussen et al. (2016). For the special case of autonomous systems they provide the formulas (19), (18), and (20) for the mean age vector, mean system age, and mean transit time, respectively. The probabilistic approach used here showed that these formulas are just expected values of corresponding random variables with according probability distributions \(\mathbf {f_a}(y) = -({\mathsf {X}}^*)^{-1}\, \mathrm{e}^{y\,{\mathsf {B}}}\,\mathbf {u}\), \(A\sim {\text {PH}}({\varvec{\eta }},{\mathsf {B}})\), and \(T\sim {\text {PH}}({\varvec{\beta }}, {\mathsf {B}})\). In particular, the terms mean system age and mean transit time refer to the expected absorption times of related continuous-time Markov chains. A general formula for the moments of the distribution of the transit time in a compartmental system without external inputs is given by Hearon (1972). He also states a formula for the backward transit-time density in this situation.

*t*. They furthermore give a survivor function

*t*. In the present setup this means

For special cases of simple compartmental systems, they further provide formulas for the densities and means of transit time and system age. These formulas were obtained by simulating an impulsive input and considering the impulse response in the Laplace space. It turns out that those formulas are special cases of the general shape of probability density functions of phase-type. Examples are presented in “Appendix B”.

### 7.2 Singular Compartmental Matrices

In this paper, only nonsingular compartmental or transition rate matrices were considered, because they ensure that every particle that enters the system will eventually leave it. It can be shown (Neuts 1981) that the transit time of a continuous-time Markov chain with singular transition rate matrix does not lead to a certainly finite transit time. In this case, the system contains traps from which particles are not able to get out once they have entered them. So, some particles will remain in the system forever. Although this seems unrealistic, there are successful carbon-cycle models with this structure.

A famous example is the RothC model (Jenkinson and Rayner 1977). It contains an inert compartment that has neither inputs nor outputs, but it has a positive constant content. To apply the stochastic framework of the present paper to that model, it is necessary to cut this compartment out of the system and to look at the steady state of the remaining smaller system. The transit time of newly arriving particles will not be affected. However, the age distribution of the system will depend on time since the particles in the inert compartment become older and older. A detailed analysis of how to treat systems with traps can be found in Jacquez and Simon (1993) and references therein.

### 7.3 Stochastic Versus Deterministic Approach

As noted by Purdue (1979), in modeling ecological systems, there are powerful arguments for the use of stochastic processes. Even if nature is completely deterministic, ecological systems are too complex for a complete theoretical understanding or descriptive tools. This lack of complete knowledge can be accounted for by using probabilistic methods. One of the first authors to integrate stochastic behavior in biological models was Bartholomay (1958). Purdue (1979) gives an early review on stochastic compartmental models with ecological applications. He showed how compartmental models can be considered as a deterministic representation of an underlying stochastic process by establishing the link to continuous-time Markov chains. While Eisenfeld (1979) elaborates on this link and applies it to a liver model, the present paper connects the continuous-time Markov chain approach to the theory of phase-type distributions (Neuts 1981) and focuses on transit time and ages.

The deterministic theory describes the ideal or average behavior of a system, whereas stochastic models can examine the deviations from such ideal or average behavior. The stochastic approach allows to think of key quantities as random variables. Knowledge of their probability distributions provides a whole new set of tools not only to study the mean, but also higher order moments of such random variables. A high variance of the transit time, for example, expresses that it is not the usual behavior of particles to travel through the system as long as indicated by the mean transit time. There is rather a large variety of different transit times depending on the particular path taken by the particle. Long tails of the distribution mean that there are in fact particles that need very long to travel through the system and hence there are very old masses in the system. With the probability density functions in hand, it is also possible to think of quantiles and confidence intervals of transit time and system age, something not possible with the deterministic approach.

Linear autonomous compartmental carbon-cycle models relate to continuous-time Markov chains with homogeneous transition probabilities. Turning to nonautonomous models (Rasmussen et al. 2016), nonhomogeneous Markov chains would have to be considered in the probabilistic approach. In this situation the Kolmogorov equations govern the evolution of the transition probabilities. There is plenty of theory available on continuous-time Markov chains (Norris 1997; Ross 2010), and much of it might be applied to carbon-cycle models.

## 8 Conclusions

Open linear autonomous compartmental systems were modeled by absorbing continuous-time Markov chains and it was shown that the behavior of those systems is governed by the phase-type distribution. It applies for the transit time as well as for the system age, only with different initial distributions. This knowledge provides simple general formulas for the densities, means, and higher order moments of transit time, compartment ages, and system age. Furthermore, it explains the underlying structure of the intrinsic connection between system age and transit time (Bolin and Rodhe 1973).

This approach also revealed the potential and opportunities of expressing deterministic compartmental models as stochastic processes. For example, important system diagnostics in reservoir theory such as system ages and transit times have analogous counterparts in probabilistic terms. The theory of stochastic processes can further help to address important questions in reservoir theory with applications in other fields even beyond global biogeochemical cycles.

## Notes

### Acknowledgements

Open access funding provided by Max Planck Society.

### References

- Anderson DH (1983) Compartmental modeling and tracer kinetics, vol 50. Springer, BerlinGoogle Scholar
- Asmussen S (2003) Applied probability and queues, 2nd edn. Springer, BerlinGoogle Scholar
- Bartholomay AF (1958) On the linear birth and death processes of biology as Markoff chains. Bull Math Biol 20(2):97–118Google Scholar
- Billingsley P (1968) Convergence of probability measures. Wiley, HobokenGoogle Scholar
- Bolin B, Rodhe H (1973) A note on the concepts of age distribution and transit time in natural reservoirs. Tellus 25(1):58–62CrossRefGoogle Scholar
- Dudley RM (1999) Uniform central limit theorems, vol 23. Cambridge University Press, CambridgeCrossRefGoogle Scholar
- Eisenfeld J (1979) Relationship between stochastic and differential models of compartmental systems. Math Biosci 43(3):289–305CrossRefGoogle Scholar
- Emanuel WR, Killough GG, Olson JS (1981) Modelling the circulation of carbon in the world’s terrestrial ecosystems. In: Bolin B (ed) Carbon cycle modelling, SCOPE 16. Wiley, Hoboken, pp 335–353Google Scholar
- Engel KJ, Nagel R (2000) One-parameter semigroups for linear evolution equations, vol 194. Springer, BerlinGoogle Scholar
- Eriksson E (1971) Compartment models and reservoir theory. Annu Rev Ecol Syst 2:67–84CrossRefGoogle Scholar
- Friedlingstein P, Cox P, Betts R, Bopp L, Von Bloh W, Brovkin V, Cadule P, Doney S, Eby M, Fung I, Bala G, John J, Jones C, Joos F, Kato T, Kawamiya M, Knorr W, Lindsay K, Matthews HD, Raddatz T, Rayner P, Reick C, Roeckner E, Schnitzler KG, Schnur R, Strassmann K, Weaver AJ, Yoshikawa C, Zeng N (2006) Climate-carbon cycle feedback analysis: results from the \(\text{ C }^4\)MIP model intercomparison. J Clim 19(14):3337–3353CrossRefGoogle Scholar
- Friedlingstein P, Meinshausen M, Arora VK, Jones CD, Anav A, Liddicoat SK, Knutti R (2014) Uncertainties in CMIP5 climate projections due to carbon cycle feedbacks. J Clim 27(2):511–526. https://doi.org/10.1175/JCLI-D-12-00579.1 CrossRefGoogle Scholar
- Hearon JZ (1972) Residence times in compartmental systems and the moments of a certain distribution. Math Biosci 15(1):69–77CrossRefGoogle Scholar
- Jacquez JA, Simon CP (1993) Qualitative theory of compartmental systems. SIAM Rev 35(1):43–79CrossRefGoogle Scholar
- Jenkinson D, Rayner J (1977) The turnover of soil organic matter in some of the Rothamsted classical experiments. Soil Sci 123(5):298–305CrossRefGoogle Scholar
- Luo Y, Weng E (2011) Dynamic disequilibrium of the terrestrial carbon cycle under global change. Trends Ecol Evol 26(2):96–104CrossRefGoogle Scholar
- Manzoni S, Katul GG, Porporato A (2009) Analysis of soil carbon transit times and age distributions using network theories. J Geophys Res 114(G4):1–14. https://doi.org/10.1029/2009JG001070 CrossRefGoogle Scholar
- Neuts MF (1981) Matrix-geometric solutions in stochastic models: an algorithmic approach. Johns Hopkins Press, LondonGoogle Scholar
- Nir A, Lewis S (1975) On tracer theory in geophysical systems in the steady and non-steady state. Part I. Tellus 27(4):372–383. https://doi.org/10.1111/j.2153-3490.1975.tb01688.x CrossRefGoogle Scholar
- Norris JR (1997) Markov chains. Cambridge University Press, CambridgeCrossRefGoogle Scholar
- Priestley MB (1982) Spectral analysis and time series. Academic Press, CambridgeGoogle Scholar
- Purdue P (1979) Stochastic compartmental models: a review of the mathematical theory with ecological applications. In: Matis J, Patten B, White G (eds) Compartmental analysis of ecosystem models. International Co-operative Publishing House, Maryland, pp 223–260Google Scholar
- Rasmussen M, Hastings A, Smith MJ, Agusto FB, Chen-Charpentier BM, Hoffman FM, Jiang J, Todd-Brown KEO, Wang Y, Wang YP, Luo Y (2016) Transit times and mean ages for nonautonomous and autonomous compartmental systems. J Math Biol 73(6–7):1379–1398. https://doi.org/10.1007/s00285-016-0990-8 CrossRefGoogle Scholar
- Ross SM (2010) Introduction to probability models, 10th edn. Academic Press, CambridgeGoogle Scholar
- Sierra CA, Müller M (2015) A general mathematical framework for representing soil organic matter dynamics. Ecol Monogr 85(4):505–524. https://doi.org/10.1890/15-0361.1 CrossRefGoogle Scholar
- Thompson MV, Randerson JT (1999) Impulse response functions of terrestrial carbon cycle models: method and application. Glob Change Biol 5(4):371–394. https://doi.org/10.1046/j.1365-2486.1999.00235.x CrossRefGoogle Scholar
- Wang YP, Chen BC, Wieder WR, Leite M, Medlyn BE, Rasmussen M, Smith MJ, Agusto FB, Hoffman F, Luo YQ (2014) Oscillatory behavior of two nonlinear microbial models of soil carbon decomposition. Biogeosciences 11(7):1817–1831. https://doi.org/10.5194/bg-11-1817-2014 CrossRefGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.