Synchronization and Random Attractors in Reaction Jump Processes

Abstract

This work explores a synchronization-like phenomenon induced by common noise for continuous-time Markov jump processes given by chemical reaction networks. Based on Gillespie’s stochastic simulation algorithm, a corresponding random dynamical system is formulated in a two-step procedure, at first for the states of the embedded discrete-time Markov chain and then for the augmented Markov chain including random jump times. We uncover a time-shifted synchronization in the sense that—after some initial waiting time—one trajectory exactly replicates another one with a certain time delay. Whether or not such a synchronization behavior occurs depends on the combination of the initial states. We prove this partial time-shifted synchronization for the special setting of a birth-death process by analyzing the corresponding two-point motion of the embedded Markov chain and determine the structure of the associated random attractor. In this context, we also provide general results on existence and form of random attractors for discrete-time, discrete-space random dynamical systems.

1 Introduction

Stochastic models of biochemical reaction dynamics are mostly based on the theory of Markov processes [1, 26]. A central role play reaction jump processes which model a well-mixed reaction system as a continuous-time Markov process on a discrete state space. The state is given by the number of particles of each involved chemical species, and chemical reactions are modeled as stochastic events which induce jumps in the system’s state that occur after exponentially distributed sojourn times. The temporal evolution of the system’s probability distribution is in this case characterized by the chemical master equation [13]. The well-known stochastic simulation algorithm, introduced by D. Gillespie in 1976, allows to generate statistically exact realizations of such reaction-jump processes [12]. Besides such reaction jump processes, there exist also modeling approaches using discrete-time Markov chains [16, 19], stochastic differential equations (SDEs, concretely: the chemical Langevin equation) or ordinary differential equations (ODEs) [14, 21, 22], which approximate the dynamics on a macroscopic level in case of large population sizes, as well as hybrid model recombinations for multiscale reaction systems [17, 27, 33, 37]. All these approaches for describing and analyzing stochastic phenomena within biochemical or other types of applied contexts have extensively been studied in the literature [2, 32, 34].

1.1 Background and Related Work

The counterpart to stochastic processes within dynamical system theory is given by random dynamical systems (RDS). Here, the origin of uncertainties is considered somewhat differently. In simple terms, the system evolves according to deterministic maps which are chosen randomly from a stochastic law. Formally speaking, an RDS $(\theta ,\varphi )$ on a metric state space ${\mathbb {X}}$ (endowed with its Borel $\sigma$-algebra ${\mathcal {B}}({\mathbb {X}})$) and discrete time set ${\mathbb {T}}={\mathbb {N}}_0$ or ${\mathbb {T}}={\mathbb {Z}}$ consists of the following two components:

• A noise model given by a metric dynamical system $\left( \Omega ,{\mathcal {F}},{\mathbb {P}},\theta \right)$. By this we mean that $(\Omega , {\mathcal {F}},{\mathbb {P}})$ is a probability space and $\theta :=\left( \theta ^n\right) _{n\in {\mathbb {T}}}$ is a family of measurable maps $\theta ^n:\Omega \rightarrow \Omega$ for which $\theta ^{n+m}=\theta ^n\circ \theta ^m$ for all $n,m\in {\mathbb {T}}$, and which is invariant with respect to ${\mathbb {P}}$ (or, ${\mathbb {P}}$ is $\theta$-invariant). This last statement means that $\theta ^n_*{\mathbb {P}}(\cdot ):={\mathbb {P}}\left( \left( \theta ^n\right) ^{-1}(\cdot )\right) ={\mathbb {P}}(\cdot )$ for all n.

• A cocycle map $\varphi :{\mathbb {N}}_0\times \Omega \times {\mathbb {X}}\rightarrow {\mathbb {X}}$, with $(n,\omega ,x)\mapsto \varphi ^n_{\omega }(x)$, which is measurable and satisfies the cocycle property over $\theta$, that is for all $x\in {\mathbb {X}}$, $n,m\in {\mathbb {N}}_0$, and $\omega \in \Omega$

  $$\begin{aligned} \varphi ^0_\omega (x)=x, \qquad \varphi ^{n+m}_\omega (x)=\varphi ^n_{\theta ^m\omega }\circ \varphi ^m_{\omega }(x). \end{aligned}$$

  (1)

Notice that while the dynamics on the noise space $\Omega$ might be defined for both positive and negative times, this does not need to be the case for the cocycle map $\varphi$ which in our context will only be defined for times on ${\mathbb {N}}_0$. For a comprehensive theoretical background of RDS we refer to [3].

For a discrete-time system on a finite state space the maps are given by deterministic transition matrices (containing only entries zero and one), and the expectation of the matrix-valued random variable of transitions maps agrees with the stochastic transition matrix of the corresponding Markov chain. The relation between such finite-state RDS and the related Markov chains has been studied in [35, 36]. Among other things, it has been found that a given finite-state RDS induces a unique Markov chain, while one Markov chain might be compatible with several RDS [36], as already discussed in a general context by Kifer [18]. In this sense, the RDS formulation may be seen as a more refined model of stochastic dynamics than the Markov chain: the former gives a precise description of the two-point motion, comparing trajectories with different initial conditions but driven by the same noise allowing for the analysis of random attractors [7], whereas the latter characterizes the statistics of the one-point motion by means of the transition probabilities.

While RDS representations of Markov chains (discrete in space and time) or SDEs (continuous in space and time) (see e.g. [3]) have been studied in the literature, an analogous investigation for continuous-time Markov processes on discrete state spaces is still missing. In the present work, we do a first step in this direction by formulating random dynamical systems corresponding to reaction jump processes as special types of continuous-time Markov processes. Our goal is to study questions of synchronization: Given the same noise realization, will trajectories generated by Gillespie’s stochastic simulation algorithm approach each other in the course of time after starting at different initial states? Once they coincide at a certain time point, do they stay together forever? Our numerical experiments have shown that two realizations of the reaction jump process with distinct starting points (but the same driving noise) may actually resemble each other after some time period in the sense that one of the trajectories appears to be a time-delayed replicate of the other. That is, after some random initial “finding time”, the two process realizations start to wander through the same sequence of states, with identical sojourn times in each of these states, but with a certain time lag with respect to each other. Whether or not this type of trajectory replication happens seems to depend in general on the combination of chosen initial states. By means of the RDS presentation of the dynamics, we provide an analytical explanation for this intriguing phenomenon of time-shifted synchronization and its dependency on the initial conditions.

1.2 Main Results

For our analysis, we use the fact that a (continuous-time) Markov jump process $(X(t))_{t\ge 0}$ admits a discrete-time representation given by the augmented Markov chain [31] which assigns to each discrete index n the random time $T_n$ where the nth jump of the process occurs, as well as the state $X_n=X(T_n)$ entered by the process at this jump time. The random sequence $\left( X_n\right) _{n\in {\mathbb {N}}_0}$ of states, called embedded Markov chain, is a discrete-time Markov process on a countable state space. In particular, we use an explicit recursive formula for this Markov chain which immediately yields the cocycle of an RDS. We show that the time-shifted synchronization observed for trajectories of a time-homogeneous reaction-jump process is equivalent to the “normal" synchronization of the embedded Markov chain, for an appropriate subset of initial conditions. Given that the jump rate constants are time-independent, also the sojourn times within the states will agree once that the states do agree.

In more detail, we focus on two main examples, providing several general insights on random attractors for discrete state spaces on the way: a standard birth-death process and the Schlögl model [30] with their monostable and bistable structures, respectively, detecting similarities and differences in the described synchronization behavior. We obtain the following main results and insights:

• For the embedded Markov chain of general birth-death processes which admit a unique stationary distribution, we prove partial synchronization (and, by that, partial time-shifted synchronization for the reaction jump process) in the sense that common-noise trajectories with starting states of the same parity join each other in finite time.

• For general RDS corresponding with Markov chains, we relate different forms of random attractors (Theorem 9) and give conditions for the existence of a (weak) random attractor (Theorem 11). We verify these conditions for our examples (Proposition 6), prominently using the existence of a unique stationary distribution.

• We characterize the random attractor of the embedded chain of a birth-death jump process admitting a stationary distribution as a pullback and forward attractor consisting of two random points which form a random periodic orbit (Theorem 15). By that, we also find the structure of the corresponding sample measures, also called statistical equilibria (Proposition 16). In the particular case of the standard birth-death chain, these random points are within distance 1 of each other, implying that initial states with different parity lead eventually to oscillations around each other (Proposition 18 and Corollary 19).

• We illustrate numerical insights about the weak attractor of the embedded chain of the Schlögl model, which has the same structure as the standard birth-death chain, apart from the fact that the distance of the two random points is not 1, mirroring the bistability of the model.

Except for the general insights on random attractors on countable state spaces (Sects. 4.1 and 4.2), most of our analytical results are, so far, restricted to the case of birth-death processes by a monotonicity argument. However, the proof structure for these results may well be extended to more general chemical reaction networks, also with multiple reactants and corresponding random periodic orbits.

Note that the works [16, 35, 36] mentioned earlier also deal with synchronization of RDSs for Markov chains, and in [16] even partial synchronization is considered. However, the latter approach is focused on linear cocycles for random networks, using the theory of Lyapunov exponents. Our proofs deploy an analysis of the two-point motion and its consequences for the random attractor, and do not require a linear interpretation, being confronted with an infinite state space. Note that Newman’s work on synchronization for RDS [28, 29] achieves general equivalent conditions for synchronization to occur, which can typically be verified via the maximal Lyapunov exponent when the state space is a smooth manifold. For the class of examples considered in this work, the equivalence of these conditions, adjusted to the problem of partial synchronization, will automatically appear in a straight-forward manner.

1.3 Structure of the Paper

The remainder of the paper is structured as follows. In Sect. 2, we introduce reaction jump processes (Sect. 2.1), the corresponding augmented and embedded Markov chains and their interpretations as random dynamical systems (Sects. 2.2 and 2.3), and their relationship with realizations of the jump process (Sect. 2.4). Section 3 discusses the notions of synchronization and partial synchronization for RDS in discrete time and discrete space (Sect. 3.1) and contains the proof of partial synchronization for a general class of birth-death chains (Sect. 3.2). Furthermore, the result is applied to the standard and the Schlögl birth-death chain (Sect. 3.3). In Sect. 4, we discuss general properties of weak, pullback and forward attractors for the discrete setting (Sect. 4.1), show a general result on the existence of weak attractors including the setting of reaction problems (Sect. 4.2), and characterize the structure of these attractors for general birth-death chains as random periodic orbits (Sect. 4.3). In Sect. 5 we discuss implications of our results for the two-point motions of the standard birth-death chain (Sect. 5.1) and the Schlögl chain (Sect. 5.2). Finally, we provide a conclusion with outlook in Sect. 6.

2 Reaction Jump Processes and Random Dynamical Systems

In this section, we introduce the reaction network under consideration and formulate the corresponding stochastic dynamics. At first, in Sect. 2.1, the pathwise formulation of the reaction-jump process is given, including two exemplary reaction networks which will be extensively studied in this work. The random dynamical systems of the related embedded and augmented Markov chain will be formulated in Sects. 2.2 and 2.3, respectively. In Sect. 2.4, the RDS of the augmented Markov chain will be reinterpreted as a continuous-time process.

2.1 The Reaction Jump Process

We consider the standard setting of well-mixed stochastic chemical reaction dynamics [34]: There is a system of particles with $L\in {\mathbb {N}}$ different types/species ${\mathcal {S}}_1,\dotsc ,{\mathcal {S}}_L$. The particles interact by $K\in {\mathbb {N}}$ chemical reactions ${\mathcal {R}}_1,\dotsc ,{\mathcal {R}}_K$ given by

$$\begin{aligned} {\mathcal {R}}_k: \quad \sum _{l=1}^L s_{lk} {\mathcal {S}}_l \rightarrow \sum _{l=1}^L s'_{lk} {\mathcal {S}}_l, \end{aligned}$$

where the stoichiometric coefficients $s_{kl},s'_{lk}$ are non-negative integers. The state of the system is given by a vector $x=(x_l)_{l=1,\ldots ,L}\in {\mathbb {N}}_0^L$ with $x_l$ counting the number of particles of species ${\mathcal {S}}_l$. Each reaction induces a jump in the state of the form $x \mapsto x +\nu _k$, where $\nu _k=(\nu _{1k},\ldots ,\nu _{Lk})\in {\mathbb {Z}}^L$ is the state-change vector given by $\nu _{lk}:=s'_{kl}-s_{lk}$. Given a state x, the reaction ${\mathcal {R}}_k$ takes place at rate $\alpha _k(x)$, where $\alpha _k:{\mathbb {N}}_0^L \rightarrow [0,\infty )$ is the corresponding propensity function.

The resulting (continuous-time) reaction jump process (RJP) on ${\mathbb {X}}={\mathbb {N}}_0^L$ has the path-wise representation

$$\begin{aligned} X(t) = X(0) + \sum _{k=1}^K {\mathcal {U}}_k\left( \int _0^t \alpha _k(X(s)) ds\right) \nu _k, \quad t \ge 0, \end{aligned}$$

(2)

where ${\mathcal {U}}_k$ are independent unit-rate Poisson processes. This process (and equivalently a more general Markov jump process) is fully characterized by the random jump times $T_n$, $n=1,2,\ldots$, at which the jumps (here reactions) take place and the states $X_n:=X(T_n)$ that are entered at the jump times, namely by

$$\begin{aligned} X(t) = X_n \quad \text{ for } \; T_n \le t < T_{n+1}, \end{aligned}$$

(3)

with $T_0 = 0$ and $X_0=X(0)$. That is, we can consider a division of the Markov jump process into the process of jump times $(T_n)_{n \in {\mathbb {N}}_0}$ with values in $[0, \infty )$ and the process of the states $(X_n)_{n \in {\mathbb {N}}_0}$ in ${\mathbb {X}}$ which is called the embedded Markov chain. The discrete-time process $(X_n,T_n)_{n \in {\mathbb {N}}_0}$ is called the augmented Markov chain [31].

The central method to generate statistically exact trajectories of the RJP (2) is given by Gillespie’s stochastic simulation algorithm [15]. It is based on the insight that the waiting times between reactions follow exponential distributions, and the selection of the next reaction event is based on the relative rates of the reactions. More concretely,

$$\begin{aligned} T_{n+1}&= T_n + \tau (X_n) , \end{aligned}$$

(4)

$$\begin{aligned} X_{n+1}&= X_{n} + \nu _{ \kappa (X_n)} , \end{aligned}$$

(5)

with $T_0=0$, $X_0=X(0)$, where $\tau (x)$ is an exponentially distributed random variable with mean $1/\sum _{j}\alpha _j(x)$ and $\kappa (x)\in \{1,\ldots ,K\}$ is a random variable with point probabilities $\alpha _k(x)/\sum _{l=1}^K \alpha _l(x)$ for $k=1,\ldots ,K$. There exist different variants to generate the random numbers $\tau (x)$ and $\kappa (x)$ given the state $x\in {\mathbb {X}}$ of the system. In the following, we consider the direct method [12], which draws independent, uniformly distributed random numbers $r,q \sim U(0,1)$ and sets

$$\begin{aligned} \tau (x,r) = \frac{1}{\sum _{k=1}^K\alpha _k(x)}\log \left( \frac{1}{r}\right) , \end{aligned}$$

(6)

while $\kappa (x,q)$ is chosen to be the smallest integer satisfying

$$\begin{aligned} \sum _{k=1}^{\kappa (x,q)} \alpha _k(x) > q\sum _{k=1}^K \alpha _k(x). \end{aligned}$$

(7)

Unlike approximate discrete-time simulation schemes with fixed step size, Gillespie’s algorithm associates the iteration index n with the jump times that have variable distances.

Example 1

(Standard birth-death process) As a basic example which will be analyzed in detail in Sects. 3.2 and  4.3 we consider the standard birth-death process of a single species ${\mathcal {S}}$ given by $K=2$ reactions

$$\begin{aligned} {\mathcal {R}}_1: \emptyset {\mathop {\longrightarrow }\limits ^{\gamma _1}} {\mathcal {S}}, \quad {\mathcal {R}}_2: {\mathcal {S}} {\mathop {\longrightarrow }\limits ^{\gamma _2}} \emptyset . \end{aligned}$$

Here, $\gamma _1,\gamma _2>0$ are rate constants and the corresponding propensity functions are given by the law of mass action as

$$\begin{aligned} \alpha _1(x)=\gamma _1, \quad \alpha _2(x)=\gamma _2 x.\end{aligned}$$

The state space of the resulting jump process is given by ${\mathbb {X}}={\mathbb {N}}_0$. Consequently, also the state-change vectors $\nu _k$ are actually scalar and given by $\nu _1=1$ and $\nu _2=-1$. The waiting time $\tau (x)$ until the next reaction takes place, given the actual state x, is exponentially distributed with mean $1/(\gamma _1+\gamma _2x)$. Then, reaction ${\mathcal {R}}_1$ takes place with probability $\frac{\gamma _1}{\gamma _1+\gamma _2x}$ (corresponding to $\kappa (x)=1$), while ${\mathcal {R}}_2$ takes place with probability $\frac{\gamma _2x}{\gamma _1+\gamma _2x}$ (corresponding to $\kappa (x)=2$). In terms of the direct method, see (7), this may be realized by setting

$$\begin{aligned} \kappa (x,q)= {\left\{ \begin{array}{ll} 1 &{} if q<\frac{\gamma _1}{\gamma _1+\gamma _2x}, \\ 2 &{} otherwise \end{array}\right. } \end{aligned}$$

(8)

for $q\sim U(0,1)$.

Under an appropriate scaling of the propensity functions $\alpha _1$ and $\alpha _2$, one may derive the corresponding reaction rate equation governing the dynamics of the concentration $C(t) = \frac{X^V(t)}{V}$ for the large volume limit $V\rightarrow \infty$ (cf. [34]). This is an ordinary differential equation (ODE), given by

$$\begin{aligned} \frac{\textrm{d}C(t)}{\textrm{d}t} = - \gamma _2 C(t)+ \gamma _1, \end{aligned}$$

(9)

with globally attracting equilibrium at $C^* = \frac{\gamma _1}{\gamma _2}$ for which the right-hand side of (9) is zero (see Fig. 1a).

Example 2

(Schlögl model) The other main example of this work is the Schlögl model, a chemical reaction network exhibiting bistability, cf. e.g. [9, 25, 30] and see Sect. 5.2 for a detailed discussion. Again we only have one species ${\mathcal {S}}$ with the following reactions

$$\begin{aligned} \begin{array}{lcl} {\mathcal {R}}_1: \emptyset {\mathop {\longrightarrow }\limits ^{\gamma _1}} {\mathcal {S}}, &{} \quad &{}{\mathcal {R}}_2: {\mathcal {S}} {\mathop {\longrightarrow }\limits ^{\gamma _2}} \emptyset ,\\ {\mathcal {R}}_3: 2{\mathcal {S}} {\mathop {\longrightarrow }\limits ^{\gamma _3}} 3{\mathcal {S}}, &{}\quad &{}{\mathcal {R}}_4: 3{\mathcal {S}} {\mathop {\longrightarrow }\limits ^{\gamma _4}} 2{\mathcal {S}}, \end{array} \end{aligned}$$

for rate constants $\gamma _1, \gamma _2, \gamma _3, \gamma _4 > 0$ and $x \in {\mathbb {N}}_0$. The corresponding standard mass action propensities are given by

$$\begin{aligned} \begin{array}{lcl} \alpha _1(x) = \gamma _1, &{} \quad &{}\alpha _2(x)=\gamma _2 x,, \\ \alpha _3(x) = \gamma _3 x(x-1), &{}\quad &{}\alpha _4(x)=\gamma _4x(x-1)(x-2), \end{array} \end{aligned}$$

and the state-change vectors are given by the scalars $\nu _1, \nu _3=1$ and $\nu _2, \nu _4=-1$.

Fig. 1

Solution of the reaction rate equation for a the birth-death process, see Eq. (9), and b the Schlögl model, see Eq. (10), each for three different initial states C(0). For (a), the rate constants are given by $\gamma _1=10, \gamma _2=1$, and there is a globally attracting equilibrium at $C^* = \frac{\gamma _1}{\gamma _2}=10$. In contrast, for (b) the rate constants are $\gamma _1=6, \gamma _2=3.5, \gamma _3=0.4, \gamma _4=0.0105$ and there are two locally attracting equilibria at $C^*_1\approx 2.2664$ and $C^*_2\approx 26.2087$, as well as an unstable equilibrium given by $C^*_3 \approx 9.6201$

The corresponding reaction rate ODE is given by

$$\begin{aligned} \frac{\textrm{d}C(t)}{\textrm{d}t} = - \gamma _4 C(t)^3 + \gamma _3 C(t)^2 - \gamma _2 C(t)+ \gamma _1, \end{aligned}$$

(10)

which—depending on the value of the reaction rates—may exhibit two stable equilibria $C^*_1, C^*_2$ and an unstable equilibrium $C^*_3$ (see Fig. 1b).

The two examples demonstrate two fundamentally different patterns in terms of the large-volume behavior of the process: whereas in Example 1 the reaction rate equation has one globally attracting equilibrium such that all trajectories synchronize to the same concentration, in Example 2 there are two (locally) attracting equilibria separated by an unstable equilibrium such that different ODE-trajectories may or may not synchronize depending on their initial conditions, see Fig. 1.

This observation prompts interest in the synchronizing behavior of the underlying reaction jump process: Given the same random numbers but different initial states, will the jump times and states given by Eqs. (4)–(5) approach each other in the course of time? Can we observe different synchronization behavior in Examples 1 and 2? To give a systematic answer to these questions, we will analyze the reaction jump process in terms of the corresponding RDS which gives a natural approach to comparing trajectories with different initial conditions but driven by the same noise realizations.

2.2 RDS Formulation of the Embedded Markov Chain

At first, we formulate the setting of a RDS for the embedded Markov chain $(X_n)_{n\in {\mathbb {N}}_0}$, given by $X_n:=X(T_n)$, as a discrete-time stochastic process. The noise space ${\mathcal {Q}}_+$ of the RDS is chosen as

$$\begin{aligned} {\mathcal {Q}}_+ = \left\{ q=(q_n)_{n \in {\mathbb {N}}_0}: q_n \in (0,1)\right\} . \end{aligned}$$

We endow ${\mathcal {Q}}_+$ with the Borel $\sigma$-algebra $\sigma ({\mathcal {Q}}_+)$ generated by its cylinder sets, and with the infinite product probability measure ${\mathbb {P}}=\lambda ^{{\mathbb {N}}_0}$, where $\lambda$ denotes the Lebesgue measure on (0, 1). On this probability space $({\mathcal {Q}}_+, \sigma ({\mathcal {Q}}_+), {\mathbb {P}})$ we define the shift map $\theta : {\mathcal {Q}}_+ \rightarrow {\mathcal {Q}}_+$ and its iterates by

$$\begin{aligned} \theta (q_0,q_1,\ldots ) = (q_1,q_2,\ldots ), \qquad \theta ^n:=\underbrace{\theta \circ \cdots \circ \theta }_{n times }. \end{aligned}$$

(11)

Since $\theta$ is invariant with respect to ${\mathbb {P}}$, the tuple $({\mathcal {Q}}_+,\sigma ({\mathcal {Q}}_+),{\mathbb {P}},(\theta ^n)_{n\in {\mathbb {N}}_0})$ constitutes our underlying noise model. Throughout this work we will use interchangeably the short-hand notation

$$\begin{aligned} {\mathbb {P}}(S(q)) = {\mathbb {P}}(\{q\in {\mathcal {Q}}_+:S(q) holds \}), \end{aligned}$$

where S(q) is a q-dependent statement. For any $q\in {\mathcal {Q}}_+$, consider the transition map $f_q:{\mathbb {X}} \rightarrow {\mathbb {X}}$ defined by

$$\begin{aligned} f_q(x):= x + \nu _{ \kappa (x,q_0)}, \end{aligned}$$

(12)

where $f_\cdot$ takes a whole sequence $q=(q_n)_{n \in {\mathbb {N}}_0}$ as an input but only evaluates the first entry of q, namely $q_0$, in $\kappa (x, \cdot )$ which determines the index of the reaction, see Eqs. (5) and (7), where the latter defines the choice of the function $\kappa (x, \cdot )$ for a given order of the reactions. Therefore, $f_{\theta ^n q}(X_n)$ coincides with the right-hand side of the recursion (5). Given a fixed order of the reactions, the transition map $f_q$ is unique. The RDS of the embedded Markov chain $(X_n)_{n\in {\mathbb {N}}_0}$ is given by the tuple $(\theta ,\varphi )$ with the cocycle map $\varphi :{\mathbb {N}}_0 \times {\mathcal {Q}}_+ \times {\mathbb {X}} \rightarrow {\mathbb {X}}$ defined by

$$\begin{aligned} \varphi ^n_q(x) = {\left\{ \begin{array}{ll} f_{\theta ^{n-1}q}\circ \cdots \circ f_{q}(x)&{} if n \ge 1, \\ x &{} if n =0. \end{array}\right. } \end{aligned}$$

(13)

Noting that for any $n\in {\mathbb {N}}_0$ we have that $\theta ^nq = (\theta \circ ... \circ \theta ) q =(q_n,q_{n+1},\ldots )$ holds for the shift map $\theta$ given in (11), it is straightforward to verify that the cocycle property (1) holds.

For each initial state $x \in {\mathbb {X}}$ and each $q\in {\mathcal {Q}}_+$ we obtain the sequence of states $(x_n)_{n \in {\mathbb {N}}_0} =\left( \varphi _{q}^n(x)\right) _{n \in {\mathbb {N}}_0}$ from the random difference equation

$$\begin{aligned} x_{n+1}= f_{\theta ^nq}(x_n), \, n \in {\mathbb {N}}_0 \quad \text {and} \quad x_0 = x \in {\mathbb {X}}. \end{aligned}$$

(14)

Given an initial state x, we have the relation

$$\begin{aligned} {\mathbb {P}}\left( X_n \in A\;\vert \;X_0 =x\right) = {\mathbb {P}}\left( \varphi ^n_{q}(x) \in A\right) \end{aligned}$$

for any $A\in {\mathcal {B}}({\mathbb {X}})$.

Remark 1

(Dependence on simulation scheme) Note that by virtue of a fixed order of reaction indices assumed for the direct method underlying the choice of the reaction index $\kappa$ in Eq. (7), the difference equation (5) defines our RDS in a unique way. Any permutation of the set of reactions would lead to different values of $\kappa$ and to a different RDS. Also applying other versions of Gillespie’s algorithm (such as the first-reaction method [12]) would result in a different RDS. However, statistically, these RDS will all be equivalent, and the results of this work will not depend on the order of the reactions or the underlying simulation scheme. In fact, our results depend exclusively on the (positive) recurrence properties of the associated embedded Markov chain.

2.3 RDS Formulation of the Augmented Markov Chain

The RDS $(\theta ,\varphi )$ introduced before captures only the states $X_n$ entered by the reaction jump process at the jump times $T_n$. In the following, we formulate another RDS which takes account also of the jump times $T_n$ by considering the augmented Markov chain ($X_n,T_n)_{n\in {\mathbb {N}}_0}$, see Eqs. (4)–(5).

The state space of the augmented Markov chain is given by ${\mathbb {X}} \times [0,\infty )$ with the $\sigma$-algebra given by ${\mathcal {P}}({\mathbb {X}}) \otimes {\mathcal {B}}([0,\infty ))$, where ${\mathcal {P}}({\mathbb {X}})$ denotes the power set of ${\mathbb {X}}$. In analogy to $f_q$ defined in (12), we consider for any $r \in {\mathcal {Q}}_+$ the mapping $g_r:{\mathbb {X}} \times [0,\infty ) \rightarrow [0,\infty )$ with

$$\begin{aligned} g_r(x,t): = t + \tau (x,r_0) \end{aligned}$$

(15)

for $\tau$ given in (6), such that $T_{n+1} = g_{\theta ^n r}(X_n,T_n)$ in (4). However, in contrast to $f_q$, this mapping depends on both variables $t \in [0,\infty )$ and $x \in {\mathbb {X}}$. So, the corresponding cocycle map has to depend on state and time, as well as on $r \in {\mathcal {Q}}_+$ and $q \in {\mathcal {Q}}_+$. Therefore, we introduce the product noise space

$$\begin{aligned} \Omega _+={\mathcal {Q}}_+ \times {\mathcal {Q}}_+ = \left\{ \omega =(\omega _n)_{n\in {\mathbb {N}}_0}: \omega _n =(q_n,r_n), q_n,r_n \in (0,1) \right\} \end{aligned}$$

endowed with the product $\sigma$-algebra $\sigma ({\mathcal {Q}}_+) \otimes \sigma ({\mathcal {Q}}_+)$ and the product measure ${\mathbb {P}}_{\Omega _+} = \lambda ^{{\mathbb {N}}_0}\otimes \lambda ^{{\mathbb {N}}_0}$. By abusing the notation, the corresponding shift map $\theta$ acts on both entries of a $\omega \in \Omega _+$:

$$\begin{aligned} \theta \omega = \theta (q_n,r_n)_{n\in {\mathbb {N}}_0} = (q_{n+1},r_{n+1})_{n\in {\mathbb {N}}_0}. \end{aligned}$$

The transformation/time-one mapping for the augmented Markov chain is given by $h_\omega : {\mathbb {X}} \times [0,\infty ) \rightarrow {\mathbb {X}} \times [0,\infty )$ with

$$\begin{aligned} h_{\omega }(x,t):= (f_{q}(x),g_{r}(x,t)), \end{aligned}$$

(16)

where $f_q$ and $g_r$ are given in (12) and (15). The cocycle map $\psi : {\mathbb {N}}_0 \times \Omega \times {\mathbb {X}} \times [0,\infty ) \rightarrow {\mathbb {X}} \times [0,\infty )$ is given by

$$\begin{aligned} \psi ^n_\omega (x,t) = {\left\{ \begin{array}{ll} h_{\theta ^{n-1}\omega }\circ \cdots \circ h_{\omega }(x,t)&{} if n \ge 1, \\ (x,t) &{} if n =0, \end{array}\right. } \end{aligned}$$

and fulfills the cocycle property (1). We obtain

$$\begin{aligned} {\mathbb {P}}_{\Omega _+}\Big ((X_n,T_n)\in A \;\big \vert \; (X_0,T_0)=(x,t)\Big ) = {\mathbb {P}}_{\Omega _+}\left( \psi _{\omega }^n(x,t)\in A\right) , \end{aligned}$$

for $A \in {\mathcal {P}}({\mathbb {X}})\otimes {\mathcal {B}}\big ([0,\infty )\big )$ and a given initial state x and starting time t.

We note that the first component of $\psi _{\omega }^{n}$ coincides with the cocycle map $\varphi _{q}^n$ of the embedded Markov chain, i.e. we have $\left( \psi _{\omega }^{n}(x,t)\right) _1= \varphi _{q}^n(x)$ for $\varphi _{q}^n$ given in (13) and $q_n=(\omega _n)_1$, while the second component $\left( \psi _{\omega }^n(x,t)\right) _2$ referring to the time points cannot be considered separately as a cocycle.

Fig. 2

Continuous-time Gillespie realizations $(\Phi ^t_\omega (x_0))_{t\ge 0}$ defined by (17) for the standard birth-death process of Example 1, driven by the same noise $\omega$ and for two initial values with a even distance and b odd distance. In (a), the orange trajectory seems to become a time-delayed copy of the blue one, while this is not the case in (b). c Extract of (a), showing time-shifted synchronization from time index $n_0=19$ on, where both realizations reach the state $x=8$. For the process starting in $x_0=5$ this happens at time $T_{\text {syn}}=(\psi _\omega ^{n_0}(5,0))_2\approx 1.395$, while for the other realization the time point is given by $T'_{\text {syn}}=(\psi _\omega ^{n_0}(15,0))_2\approx 1.064$. I.e., the time-shift is $R:=T_{\text {syn}}-T'_{\text {syn}}\approx 0.331$ for these initial states. The rate constants are chosen as $\gamma _1=10$, $\gamma _2=1$ (Color figure online)

2.4 Continuous-Time Process Realizations

By means of the RDS $(\theta ,\psi )$ of the augmented Markov chain, we can introduce a version of the continuous-time Markov jump process $(X(t))_{t\ge 0}$ by defining the map

$$\begin{aligned} \Phi ^t_\omega (x):=\varphi ^n_q(x) \quad \text{ for } \quad (\psi ^{n}_\omega (x,0))_2 \le t < (\psi ^{n+1}_\omega (x,0))_2 \end{aligned}$$

(17)

and $\omega =(q,r)$. For a fixed $\omega$ and an initial state $x_0$ this gives us a continuous-time process realization $(\Phi ^t_\omega (x_0))_{t\ge 0}$ of the Markov jump process $(X(t))_{t\ge 0}$ starting in $X(0)=x_0$, see also the piecewise definition in Eq. (3). We refer to the realization (17) of the reaction jump process as a Gillespie realization. This is helpful for illustrating the dynamics: In Fig. 2, common noise realizations of the standard birth-death process from Example 1 are depicted for different initial states $x_0\ne y_0$. As we can see, the two realizations seem to approach each other—with a certain time-delay—given that the difference $x_0-y_0$ of the initial states is even (see Fig. 2a), while this is not the case for an odd difference $x_0-y_0$ (see Fig. 2b). This observation motivates to formulate and analyze the synchronization behavior of random dynamical systems for the reaction systems under consideration, which we will do in the following section.

Importantly, we note that the map $\Phi =\Phi ^t_\omega (x)$, as given in (17), does not satisfy the cocycle property and, hence, the continuous-time Markov jump process is itself not an RDS in this formulation. The easiest way to observe this is that there are instances of $\Phi ^t_\omega (x_0) = \Phi ^t_\omega (y_0)$ but $\Phi ^{t+s}_\omega (x_0) \ne \Phi ^{t+s}_\omega (y_0)$ for some $t,s >0$, $x_0\ne y_0$. This is due to the fact that $\Phi ^t_\omega (x_0)$ is implicitly defined via the discrete-time RDS of the augmented Markov chain. Here, the continuous time-index $t\ge 0$ stands in contrast with the discrete index of the noise terms $(w_n)_{n\in {\mathbb {N}}_0}$.

We additionally emphasize that our construction demonstrates an intriguing lack of commutativity in the following sense: the Markov jump process admits a version that corresponds to the augmented Markov chain which directly induces an RDS. This RDS can be related back to the original process via (17) giving a version of the Markov jump process which, however, does not satisfy the cocycle property and is therefore not part of a continuous-time RDS itself. In summary, the RDS structure lies in the space-time version of the reaction rate process, revealing also relevant information about this process as we will see in the following.

3 Synchronization of Reaction Jump Processes

In the following, we introduce the terms synchronization and partial synchronization for the random dynamical systems under consideration. We analyze the synchronizing properties of the birth-death process given in Example 1 as well as of the Schlögl model of Example 2.

3.1 General Formulation

Analogously to [16], we say that an RDS $(\theta ,\varphi )$ on ${\mathbb {X}}={\mathbb {N}}_0^L$ is synchronizing in ${\mathbb {S}}\subset {\mathbb {X}}$ (or, simply, synchronizing when ${\mathbb {S}}={\mathbb {X}}$) if for every two different initial states $x,y \in {\mathbb {S}}$ and ${\mathbb {P}}$-a.e. $q \in {\mathcal {Q}}_+$ there exists a number $n_0\equiv n_0(x,y, q) \in {\mathbb {N}}_0$ such that

$$\begin{aligned} \varphi _{q}^{n_0}(x) = \varphi _{q}^{n_0}(y). \end{aligned}$$

(18)

It follows from the cocycle property that if (18) holds for some $n_0\in {\mathbb {N}}$, it is true for any other $n\ge n_0$. We say that the RDS $(\theta ,\varphi )$ is partially synchronizing if there exists a partition $\xi =\{W_0, \dotsc , W_{p-1}\}$, $p\in {\mathbb {N}}, p\ge 2$, of ${\mathbb {X}}$ such that $(\theta ,\varphi )$ is synchronizing in each $W_i\in \xi$. A much general definition is given in [16], where the partition $\xi$ may depend on $q\in {\mathcal {Q}}_+$. In this paper, however, we only work with partitions independent of the noise.

For a fixed $q \in \mathcal {Q_+}$ and two different initial states $x,y \in {\mathbb {X}}$ the process $\left( \varphi _{q}^n(x), \varphi _{q}^n(y)\right) _{n \in {\mathbb {N}}_0}$ in the product space ${\mathbb {X}}^2$ is called the two-point motion. Let $\Delta$ denote the diagonal in ${\mathbb {X}}^2$, i.e.

$$\begin{aligned} \Delta :=\left\{ (x, y) \in {\mathbb {X}}^2: x= y\right\} . \end{aligned}$$

(19)

Hence the RDS is synchronizing if and only if the two-point motion reaches the diagonal $\Delta$ at a time index $n_0(x,y, q)$, ${\mathbb {P}}$-a.s.

Remark 2

Since ${\mathbb {X}}$ is discrete, in order to show that an RDS $(\theta ,\varphi )$ is synchronizing in ${\mathbb {S}}$ it suffices to show that for any two initial states $x,y\in {\mathbb {S}}$ there is a full probability measurable set ${\mathcal {Q}}_{x,y}$ in which (18) holds. Indeed, for each $x,y\in {\mathbb {X}}$ consider the measurable sets

$$\begin{aligned} {\mathcal {Q}}_{x,y}=\bigcup _{n=1}^{\infty }\left\{ q\in {\mathcal {Q}}_+: \varphi ^n_q(x)=\varphi ^n_q(y)\right\} \end{aligned}$$

and assume that ${\mathbb {P}}({\mathcal {Q}}_{x,y})=1$. We can thus take ${\mathcal {Q}}_{{\mathbb {S}}}=\bigcap _{x,y\in {\mathbb {S}}}{\mathcal {Q}}_{x,y}$ and (18) holds for every $x,y\in {\mathbb {X}}$ and $q\in {\mathcal {Q}}_{{\mathbb {S}}}$, where ${\mathbb {P}}({\mathcal {Q}}_{{\mathbb {S}}})=1$. Furthermore, if an RDS is partially synchronizing then for each $W_i\in \xi$ we consider the corresponding sets ${\mathcal {Q}}_{W_i}$. By considering ${\hat{Q}}=\bigcap _{i=0}^{p-1}Q_{W_i}$, we can always assume without loss of generality that the set is the same for each element of the partition.

3.1.1 Time-Shifted Synchronization

Let us now discuss an implication of the synchronization of the embedded chain. Let $(x_n)_{n\in {\mathbb {N}}_0}=(\varphi ^n_q(x))_{n\in {\mathbb {N}}_0}$ and $(y_n)_{n\in {\mathbb {N}}_0}= (\varphi ^n_q(y))_{n\in {\mathbb {N}}_0}$ be two trajectories of the embedded Markov chain under the same noise realization $q\in {\mathcal {Q}}_+$. Recall that the jump times $T_n$ are given by the maps $(\psi ^n_{\omega }(x,t))_2$. Considering the sojourn times $\Delta T_n:=T_{n}-T_{n-1}$ for $n\ge 1$, it follows from the recursion (4)–(5) that, in fact, they depend only on the embedded chain and the noise realization. Hence, for $n\ge 1$, $\Delta T_n$ can be represented as

$$\begin{aligned} \tau ^n_\omega (x):= (\psi ^n_{\omega }(x,t))_2-(\psi ^{n-1}_{\omega }(x,t))_2, \end{aligned}$$

by abusing the notation $\tau ^n_{\omega }(x)=\tau (\varphi ^n_q(x),r_n)$ as given in (6). Note that while $(\psi ^n_{\omega }(x,t))_2$ depends on the initial time t, the difference $\tau ^n_{\omega }(x)$ no longer explicitly depends on t. Then, the map

$$\begin{aligned} {\hat{\psi }}_\omega ^n(x,t):= \left( \varphi ^n_q(x),\tau ^n_{\omega }(x)\right) \end{aligned}$$

(20)

satisfies the cocycle property over $\theta$. Assume that $(x_n)_{n\in {\mathbb {N}}_0}$ and $(y_n)_{n\in {\mathbb {N}}_0}$ synchronize, so that for all $n\ge n_0:= n_0(x,y,q)$ we have that $x_n=y_n$. This implies in particular that the soujourn times satisfy $\tau ^n_{\omega }(x)=\tau ^n_{\omega }(y)$ for all $n\ge n_0$. In other words, we have that the RDS inducing the embedded chain $(X_n)_{n\in {\mathbb {N}}_0}$ synchronizes if and only if the RDS realization of the chain $(X_n,\Delta T_n)_{n\in {\mathbb {N}}_0}$ synchronizes. For simplicity, we refer to $(X_n,\Delta T_n)_{n \in {\mathbb {N}}_0}$, where the times $T_n$ are replaced by the sojourn times $\Delta T_n$, also as an augmented Markov chain, since here the state space is again augmented by a time component.

The synchronization of the embedded/augmented chains inherits a corresponding interpretation for the Gillespie realization in the following sense.

Definition 1

We say that the Gillespie realization $\Phi ^t_{\omega }$, cf. (17), exhibits time-shifted synchronization if its associated RDS ${\hat{\psi }}^n_\omega$ for the augmented Markov chain $(X_n,\Delta T_n)_{n\in {\mathbb {N}}_0}$ synchronizes.

Definition 1 translates as follows: for every two different initial states $x,y \in {\mathbb {X}}$, an initial time $t \in [0,\infty )$ and ${\mathbb {P}}_{\Omega _+}$-a.e. $\omega \in \Omega _+$ there exists a number $n_0:=n_0(x,y,t, \omega ) \in {\mathbb {N}}_0$ such that

$$\begin{aligned} {\left\{ \begin{array}{ll} \varphi _{q}^n(x)=\varphi _{q}^n(y) \\ \tau ^n_\omega (x)=\tau ^n_\omega (y) \end{array}\right. } \text {for all } n \ge n_0. \end{aligned}$$

(21)

Furthermore, if (21) holds for all x, y in each component of a partition of ${\mathbb {X}}$ we analogously speak of partial time-shifted synchronization. See also Fig. 3 for an overview of the terminology. We emphasize that the time-shifted synchronization is, by definition, an effect of the proposed augmented Markov chain and therefore of Gillespie’s algorithm. These associate noise with jump times and not with the real time axis (which is also the reason for the map $\Phi$ not to fulfill the cocyle property). The common noise thereby acts on different time intervals. It is possible that different versions or algorithmic implementations of the jump process might exhibit synchronization in the classical sense, or perhaps not even time-shifted synchronization at all.

As a consequence of the discussion above, for analyzing the synchronization properties of the augmented Markov chain (and with it the time-shifted synchronization properties of the realization $\Phi ^t_\omega$) it suffices to consider the corresponding embedded Markov chain. We proceed by analyzing the synchronization properties for the case of general birth-death processes. Both the standard birth-death process from Example 1 and the Schlögl model from Example 2 are special cases of this general setting for birth-death processes.

Fig. 3

Overview for the concept of time-shifted synchronization introduced in Definition 1. The map $\Phi$ is defined in (17) and refers to the Gillespie realization. The RDS $(\varphi ,\tau )$ is defined in (20), where $\varphi$ refers to the states and $\tau$ to the soujourn times

3.2 Partial Synchronization of Strict Birth-Death Chains

Let us consider a fixed RDS representation of a general class of birth-death chains $(X_n)_{n\in {\mathbb {N}}_0}$ on ${\mathbb {X}}={\mathbb {N}}_0$ given by a random map as described in the subsection above, i.e.

$$\begin{aligned} f_q(x)={\left\{ \begin{array}{ll} x+1, &{} q_0\in A_x, \\ x-1, &{} otherwise , \end{array}\right. } \end{aligned}$$

where $A_x$ is a measurable set for each $x\in {\mathbb {X}}$ such that ${\mathbb {P}}(A_x)={\mathbb {P}}\left( X_1=x+1 \;\vert \; X_0=x \right)$. We refer to birth-death chains of this type as strict birth-death chains, since stagnation at each time step is not allowed. Notice that the embedded chain of any birth-death jump process falls into this category. It is straightforward to check that if $x,y\in {\mathbb {N}}_0$ are two numbers of the same parity such that $x\le y$, then $\varphi _q^n(x)\le \varphi _q^n(y)$ for all $n\in {\mathbb {N}}_0$ and $q\in {\mathcal {Q}}_+$. From this monotonicity property we obtain the following general result, for which we recall that a Markov chain is recurrent if for any initial condition y the probability of returning to y in finite time is positive.

Lemma 2

Let $(X_n)_{n\in {\mathbb {N}}_0}$ be any recurrent strict birth-death chain. Then, for any RDS representation as given above the system partially synchronizes, with partition $\xi =\{W_0, W_1 \}$ where $W_0=\{0,2,4,\ldots \}$ and $W_1=\{1,3,5,\ldots \}$.

Proof

Let $x,y\in W_i$ for $i=0,1$ such that $x\le y$. By the monotonicity property, we have that $\varphi ^n_q(x)\le \varphi ^n_q(y)$ for all $n\in {\mathbb {N}}_0$ and $q\in {\mathcal {Q}}_+$. Let

$$\begin{aligned} n(x,y;q):=\min \big \{ n\ge 0: \varphi ^n_q(x)=\varphi ^n_q(y) \big \}. \end{aligned}$$

Observe that if $\varphi ^n_q(y)=0$ for some $n\in {\mathbb {N}}$, then $\varphi ^{n-1}_q(x)=\varphi ^{n-1}_q(y)=1$ due to monotonicity and because they must have the same parity for all n. Therefore,

$$\begin{aligned}n(x,y;q)\le m_0(y;q):=\min \{n\ge 0: \varphi ^n_q(y)=0\}.\end{aligned}$$

Since the chain is recurrent, $m_0(y;q)<\infty$ is finite for all $y\in {\mathbb {N}}_0$ ${\mathbb {P}}$-a.s., and so is n(x, y; q). $\square$

Remark 3

We note that the partial synchronization (here with splitting into $W_0$ and $W_1$) is a special feature of strict birth-death chains which, per definitions, entails jumps in each iteration step. Stagnation is not possible (in the sense that the diagonal entries of the associated transition matrix are zero), such that even and odd cannot mix up. Hence, we here consider a specific subcase of possible synchronization patterns for general Markov chains. In particular, the partition $\{W_0,W_1\}$ is independent of $\omega$.

Lemma 2 states that whenever $x,y\in W_i$ ($i=0,1$) we have for almost all $q \in {\mathcal {Q}}_+$ that $\#\varphi ^n_{q}(\lbrace x,y\rbrace )=1$ for all n sufficiently large, where $\#A$ denotes the cardinality of a set A. More generally, for each finite (deterministic) set $K\subset W_i$ we obtain that for almost all $q \in {\mathcal {Q}}_+$ there is a $n_0(K, q)\in {\mathbb {N}}$ such that $\# \varphi ^n_q(K)=1$ for all $n \ge n_0(K, q)$. This almost sure convergence implies the convergence in probability given by

$$\begin{aligned} \lim _{n\rightarrow \infty }{\mathbb {P}}\left( \# \varphi ^n_q(K)\ge 2\right) =0. \end{aligned}$$

(22)

This last statement can be extended to finite random sets as defined in the following Definition 3. For this, let $d:{\mathbb {X}}\times {\mathbb {X}}\rightarrow [0,\infty )$ be the Euclidean distance on ${\mathbb {X}}$ and define

$$\begin{aligned} d(x,B):= \inf _{y \in B}d(x,y) \end{aligned}$$

(23)

for non-empty sets $B\subset {\mathbb {X}}$.

Definition 3

Let $({\mathcal {Q}},\sigma ({\mathcal {Q}}),{\mathbb {P}})$ be an arbitrary probability space and ${\mathbb {X}}={\mathbb {N}}_0^L$. A mapping $K:{\mathcal {Q}}\rightarrow {\mathcal {P}}({\mathbb {X}})$, denoted as $q \mapsto K_q$, is a random set if the function $q \mapsto d(x, K_q)$ is measurable for each $x \in {\mathbb {X}}$.

We say that a random set $K:{\mathcal {Q}}\rightarrow {\mathcal {P}}({\mathbb {X}})$ is a finite random set if $K_q$ is nonempty and finite for every $q\in {\mathcal {Q}}$. A finite random set in ${\mathcal {P}}({\mathbb {X}})$ is contained in a deterministic finite set with high probability as indicated in the next proposition.

Proposition 4

Let $K:{\mathcal {Q}}\rightarrow {\mathcal {P}}({\mathbb {X}})$ be a finite random set. Then, for each $\varepsilon >0$ there is a finite set $F_\varepsilon \subset {\mathcal {P}}({\mathbb {X}})$ such that

$$\begin{aligned} {\mathbb {P}}\left( K_q\subset F_\varepsilon \right) \ge 1-\varepsilon . \end{aligned}$$

Proof

The statement is a particular case of a more general setting, see [5, Proposition 3.15]. $\square$

We can now generalize property (22) for strict birth-death chains and for arbitrary random finite sets in the next Proposition.

Proposition 5

Consider the setting of a recurrent strict birth-death chain. Let $K:{\mathcal {Q}}_+\rightarrow {\mathcal {P}}({\mathbb {N}}_0)$ be a random finite set such that $K_q\subset W_i$ ${\mathbb {P}}$-a.s. for some $i\in \lbrace 0,1\rbrace$. Then $\#\varphi ^n_q(K_q)\rightarrow 1$ in probability.

Proof

Since $K_q$ is nonempty ${\mathbb {P}}$-a.s., the set $\varphi ^n_q(K_q)$ has at least one element for all $n\in {\mathbb {N}}$, ${\mathbb {P}}$-a.s. On the other hand, let $\varepsilon >0$ be arbitrarily small and $F_\varepsilon \subset {\mathbb {N}}_0$ a finite set as in Proposition 4. Then,

$$\begin{aligned} {\mathbb {P}}\left( \#\varphi ^n_q(K_q)\ge 2\right) = {\mathbb {P}}\left( \#\varphi ^n_q(K_q)\ge 2 \ \wedge K_q\subset F_\varepsilon \right) + {\mathbb {P}}\left( \#\varphi ^n_q(K_q)\ge 2 \ \wedge K_q\not \subset F_\varepsilon \right) . \end{aligned}$$

Note that for any $n\in {\mathbb {N}}_0$,

$$\begin{aligned} \left\{ q\in {\mathcal {Q}}_+: \# \varphi ^n_q(K_q)\ge 2 \ \wedge \ K_q\subset F_\varepsilon \right\} \subset \left\{ q\in {\mathcal {Q}}_+: \# \varphi ^n_q(F_\varepsilon )\ge 2\right\} , \end{aligned}$$

and thus we deduce together with Proposition 4 that

$$\begin{aligned} {\mathbb {P}}\left( \# \varphi ^n_q(K_q)\ge 2\right) \le {\mathbb {P}}\left( \# \varphi ^n_q(F_\varepsilon )\ge 2\right) +\varepsilon . \end{aligned}$$

Due to (22) and since $\varepsilon$ was arbitrarily small we conclude that

$$\begin{aligned} \lim _{n\rightarrow \infty }{\mathbb {P}}\left( \#\varphi ^n_q(K_q)\ge 2\right) =0, \end{aligned}$$

and the result follows. $\square$

3.3 Partial Synchronization for the Standard and the Schlögl Birth-Death Chains

We consider the corresponding embedded Markov chains of the processes defined in Examples 1 and 2, which we call in the following standard birth-death chain and Schlögl birth-death chain, respectively. In particular, both of them are strict birth-death chains. For the standard birth-death chain, its transition map $f_q:{\mathbb {N}}_0 \rightarrow {\mathbb {N}}_0$ can be chosen as

$$\begin{aligned} f_q(x)= {\left\{ \begin{array}{ll} x+1 \quad \text {if } q_0< \frac{\gamma _1}{\gamma _1+\gamma _2 x}, \\ x-1 \quad \text {otherwise,} \end{array}\right. } \end{aligned}$$

for $q \in {\mathcal {Q}}_+$, see Eq. (8) in Example 1. The transition maps for the Schlögl birth-death chain can be obtained in a similar fashion. Different choices of $f_q$ are possible, defining an RDS with the same statistics (in terms of its transition probabilities and stationary distribution), see Remark 1.

3.3.1 Transition Probabilities

Let z be a variable which takes the values $\nu _1=1$ or $\nu _2=-1$ of the corresponding state-change vectors. For the transition probabilities of the RDS of the embedded Markov chain we set

$$\begin{aligned} P_{z} (x):= {\mathbb {P}}\Big (\varphi _{q}^{n+1}(x_0)=x+z \; \big \vert \; \varphi _{q}^n(x_0)=x\Big ) \end{aligned}$$

(24)

for an arbitrary state $x\in {\mathbb {N}}_0$ and an initial state $x_0\in {\mathbb {N}}$. This probability is independent of n since the process is time-homogeneous. The transition probabilities of the standard birth-death chain are given by

$$\begin{aligned} P_1(x) = \frac{\gamma _1}{\gamma _1+\gamma _2x}, \quad P_{-1}(x) = \frac{\gamma _2x}{\gamma _1+\gamma _2x}, \end{aligned}$$

(25)

see Example 1. Similarly, the transition probabilities for the Schlögl chain are

$$\begin{aligned} P_1(x) = \frac{\gamma _1+\gamma _3x(x-1)}{\Gamma (x)}, \quad P_{-1}(x) = \frac{\gamma _2x+\gamma _4 x(x-1)(x-2)}{\Gamma (x)} \end{aligned}$$

(26)

for

$$\begin{aligned} \Gamma (x):=\gamma _1+\gamma _3x(x-1)+\gamma _2x+\gamma _4 x(x-1)(x-2) \end{aligned}$$

(27)

and an arbitrary state $x \ge 0$.

We proceed to show that both the standard and the Schlögl birth-death chains partially synchronize. In order to use Lemma 2, we prove that both chains admit a unique stationary distribution, and thus are positive recurrent, i.e. mean return times to any point are finite [8, Theorem 6.5.6]. This is a stronger statement which will also be useful for our analysis in Sect. 4.

Proposition 6

The standard birth-death chain admits a unique stationary distribution. Therefore, the associated RDS $(\theta ,\varphi )$ partially synchronizes.

Proof

Recall that the transition probabilities for the embedded Markov chain for the birth-death process are given in (25). From [8, p. 304], we know that the birth-death chain with transition probabilities given by (25) admits a unique stationary distribution if and only if

$$\begin{aligned} \zeta :=\sum _{x=1}^{\infty }\prod _{j=0}^{x-1}\frac{P_1(j)}{P_{-1}(j+1)}<\infty . \end{aligned}$$

(28)

Consider the transition probabilities (25), and for simplicity let $\alpha :=\frac{\gamma _2}{\gamma _1} > 0$. Then,

$$\begin{aligned} \zeta&= \sum _{x=1}^{\infty }\prod _{j=0}^{x-1}\frac{1+\alpha (j+1)}{(1+\alpha j)\alpha (j+1)} = \sum _{x=1}^{\infty }\prod _{j=0}^{x-1}\left( 1+\frac{\alpha }{1+\alpha j} \right) \cdot \frac{1}{\alpha (j+1)} \nonumber \\&\quad \le \sum _{x=1}^{\infty }\prod _{j=0}^{x-1}\left( \frac{1+\alpha }{\alpha } \right) \cdot \frac{1}{j+1} =\sum _{x=1}^{\infty } \frac{\beta ^x}{x!}, \end{aligned}$$

(29)

where $\beta =(1+\alpha )/\alpha$. Recall that by Stirling’s approximation we have that $x!/\left( \sqrt{2\pi x}(\frac{x}{e})^x\right) \rightarrow 1$ as $x\rightarrow \infty$. Then, there exists $N\in {\mathbb {N}}$ such that for all $x\ge N$ we have

$$\begin{aligned} \frac{1}{x!}<\frac{2}{\sqrt{2\pi x}\left( \frac{x}{e}\right) ^x}. \end{aligned}$$

(30)

From (29) and (30), and assuming without loss of generality that $N\ge 2\pi$, it follows that

$$\begin{aligned} \zeta < S+ \sqrt{2/\pi }\sum _{x=N}^{\infty } \beta ^x\cdot \frac{1}{\sqrt{x}\left( \frac{x}{e} \right) ^x}\le S+ \sqrt{2/\pi }\sum _{x=N}^{\infty } \left( \frac{e\beta }{x}\right) ^x \end{aligned}$$

for $S:=\sum _{x=1}^{N-1}\beta ^x\cdot \frac{1}{x!}$. Hence, it suffices to show that $\sum _{x=N}^{\infty } \left( \frac{e\beta }{x} \right) ^x<\infty$. Assume again without loss of generality that N is big enough so that $\frac{e\beta }{x}<\frac{1}{e}$ for all $x\ge N$. Thus,

$$\begin{aligned} \sum _{x=N}^{\infty }{\left( \frac{e\beta }{x}\right) ^x}<\sum _{x=N}^{\infty }{e^{-x}}<\infty , \end{aligned}$$

from which we finally deduce $\zeta <\infty$. $\square$

By using a similar reasoning we show that the Schlögl chain admits a stationary distribution as well.

Proposition 7

The Schlögl birth-death chain admits a unique stationary distribution. Therefore, the associated RDS partially synchronizes.

Proof

Analogously to Proposition 6, we show that the quantity $\zeta$ in (28) is finite. Indeed,

$$\begin{aligned} \prod _{j=0}^{x-1}\frac{P_1(j)}{P_{-1}(j+1)}= \prod _{j=0}^{x-1}\left( \frac{\gamma _1+\gamma _3 j (j-1)}{\gamma _2(j+1)+\gamma _4 j(j+1)(j-1)}\right) \cdot \left( \frac{\Gamma (j+1)}{\Gamma (j)} \right) \\ =\frac{1}{x!}\prod _{j=0}^{x-1}\left( \frac{\gamma _1+\gamma _3 j (j-1)}{\gamma _2+\gamma _4 j(j-1)}\right) \cdot \left( \frac{\Gamma (j+1)}{\Gamma (j)} \right) \end{aligned}$$

for $\Gamma (x)$ given in (27). Both terms inside the product are bounded for all $j\in {\mathbb {N}}_0$. Thus, there exists $C>0$ such that

$$\begin{aligned} \zeta \le \sum _{x=1}^{\infty } \frac{1}{x!}C^x<\infty . \end{aligned}$$

The finiteness of the right-hand side follows from Stirling’s approximation similarly as in Proposition 6. $\square$

As noted the end of Sect. 3.1, the partial synchronization of the RDS $(\theta , \varphi )$ for the embedded Markov chain directly implies the partial time-shifted synchronization of the corresponding Gillespie realizations, which can easily be seen from Eq. (4). Thus, the observations from Fig. 2 for the standard birth-death process can now be confirmed and clarified: In Fig. 2a, we have $x_0\in W_0$ for both initial states, such that time-shifted synchronization as defined in Definition 1 is guarantied by Proposition 6, see Fig. 2c for a detailed look at the dynamics. In contrast, the initial states chosen in Fig. 2b are not in the same set $W_i$, and consequently, the trajectories do not synchronize. However, they are likely to stay close to each other—a property that will be further investigated in Sect. 5.1.

In Fig. 4, two Gillespie realizations of the Schlögl model are shown. As in the standard birth-death process, one can also observe a time-shifted synchronization when starting with even distance as indicated by Proposition 7, see Fig. 4a. For an odd distance in the starting points, see Fig. 4b, the separation of the trajectories is even more significant than in the standard birth-death scenario.

Fig. 4

Gillespie realizations $\Phi ^t_\omega (x_0)$ given in (17) for the Schlögl model of Example 2, driven by the same noise. Realizations for two initial values with a even distance and b odd distance. In (a), the orange trajectory becomes a time-delayed copy of the blue one, just as in Fig. 2, while in (b), the trajectories clearly separate. The rate constants are chosen as $\gamma _1=6$, $\gamma _2=3.5, \gamma _3=0.4, \gamma _4=0.0105$ (Color figure online)

4 Random Attractors of RDS Associated to Markov Chains

In this section we introduce different notions of random attractors for an RDS, providing insights into their relationships in the context of a discrete state space in Sect. 4.1. As a matter of fact, we show in Sect. 4.2 that under very mild conditions an RDS induced by the random difference equation (14) admits a weak attractor. The partial synchronization for strict birth-death chains, which was proved in Lemma 2, will be useful to describe in full detail the structure of its attractor and the related sample measures in Sect. 4.3.

4.1 General Properties of Random Attractors in Discrete Time and Discrete Space

In this and the following section (Sect. 4.2) we consider a generalized setting as compared to Sect. 2.2. Specifically, let $({\mathcal {Q}},{\mathcal {F}},{\mathbb {P}})$ be a probability space and let $\theta : {\mathcal {Q}} \rightarrow {\mathcal {Q}}$ be an invertible and ${\mathbb {P}}$-ergodic map. Furthermore, for ${\mathbb {X}}={\mathbb {N}}_0^L$ as before, let $\varphi : {\mathbb {N}}_0 \times {\mathcal {Q}} \times {\mathbb {X}} \rightarrow {\mathbb {X}}$ be a measurable cocycle (cf. property (1)) such that $(\theta , \varphi )$ is a random dynamical system. One of the characteristics of random attractors is that of invariance as described in the following.

Definition 8

Given a RDS $(\theta ,\varphi )$, we say that a random set $A:{\mathcal {Q}}\rightarrow {\mathcal {P}}({\mathbb {X}})$ (recall Definition 3) is invariant (or $\varphi$-invariant) if

$$\begin{aligned} \varphi _{q}^n\left( A_q\right) = A_{\theta ^n q}\qquad for all n\in {\mathbb {N}}, \ {\mathbb {P}}-a.s. \end{aligned}$$

(31)

Notice that without loss of generality, we can assume that (31) is satisfied everywhere by restricting ourselves to the full probability set where this is true. Indeed, if (31) holds on a measurable set $\tilde{{\mathcal {Q}}}\subset {\mathcal {Q}}$, then ${\tilde{Q}}$ is $\theta$-invariant and thus of full probability.

Remark 4

Since we deal with a discrete-time setting, condition (31) is fulfilled as soon as it is satisfied for $n=1$, that is $\varphi _q^1\left( A_q\right) =A_{\theta q}$ ${\mathbb {P}}$-a.s. Indeed, let ${\mathcal {Q}}_1\subset {\mathcal {Q}}$ be a measurable set of full probability such that (31) holds for $n=1$. By the invariance of ${\mathbb {P}}$ under $\theta$, we have that ${\mathbb {P}}( {\mathcal {Q}}_1)={\mathbb {P}}(\theta ^{-1}{\mathcal {Q}}_1)=1$. Consider then the full probability set ${\mathcal {Q}}_2=\theta ^{-1}{\mathcal {Q}}_1\cap {\mathcal {Q}}_1$, which is the set where (31) holds for $n=1,2$, and inductively for $k\in {\mathbb {N}}$ the set ${\mathcal {Q}}_{k+1}=\theta ^{-1}{\mathcal {Q}}_k\cap {\mathcal {Q}}_k$ for which (31) is satisfied for all $n\le k+1$. Hence, condition (31) holds on $\tilde{{\mathcal {Q}}}:=\bigcap _{n=1}^{\infty }{\mathcal {Q}}_n$, and ${\mathbb {P}}(\tilde{{\mathcal {Q}}})=1$.

We give the notion of attraction in terms of the Hausdorff semi-distance for non-empty sets

$$\begin{aligned} \textrm{dist}(A,B)= \sup _{x\in A} d(x,B), \quad A,B\subset {\mathbb {X}}, \end{aligned}$$

where d(x, B) is given in (23). In general, an invariant compact random set A (i.e., $A_q$ is compact for all $q\in {\mathcal {Q}}$) is called

• (i) A (strong) forward attractor, if for each compact set $B\subset {\mathbb {X}}$

  $$\begin{aligned} \lim _{n \rightarrow \infty } \textrm{dist}\left( \varphi _{q}^n(B), A_{\theta ^n q}\right) =0 \quad {\mathbb {P}}\text {-a.s.}, \end{aligned}$$

  (32)

• (ii) A (strong) pullback attractor if for each compact set $B\subset {\mathbb {X}}$

  $$\begin{aligned} \lim _{n \rightarrow \infty } \textrm{dist}\left( \varphi _{\theta ^{-n}q}^n(B), A_ q\right) =0 \quad {\mathbb {P}}\text {-a.s.}, \end{aligned}$$

• (iii) A weak attractor if for each compact set $B\subset {\mathbb {X}}$

  $$\begin{aligned} \lim _{n \rightarrow \infty } \textrm{dist}\left( \varphi _{q}^n(B), A_{\theta ^n q}\right) =0 \ \text { in probability,} \end{aligned}$$

  (33)

cf. e.g. [7]. By the discrete nature of the state space, compact sets in ${\mathbb {X}}$ are simply finite sets. Moreover, for any $q\in {\mathcal {Q}}$ and $B \subset {\mathbb {X}}$,

$$\begin{aligned} \lim _{n\rightarrow \infty } \textrm{dist}\left( \varphi ^n_{\theta ^{-n}q}(B),A_q \right) = 0 \ \text { iff } \ \exists \; N\equiv N(q,B) \text { such that } \forall n\ge N, \ \varphi ^{n}_{\theta ^{-n}q}(B)\subset A_q. \end{aligned}$$

(34)

In a similar fashion, $\lim _{n\rightarrow \infty }\textrm{dist}\left( \varphi ^n_q(B),A_{\theta ^n q}\right) =0$ if and only if there exists $N\equiv N(q,B)$ such that $\varphi ^N_{q}(B)\subset A_{\theta ^N q}$. Here, the invariance of A guarantees that $\varphi ^n_{q}(B)\subset A_{\theta ^n q}$ for all $n\ge N$.

It is known that weak attractors are unique, in the sense that two weak attractors A and ${\tilde{A}}$ coincide ${\mathbb {P}}$-a.s. [11, Lemma 1.3]. Observe also that in the weak sense “forward" and “pullback" attraction are the same since $\theta$ is measure-preserving. In other words, A is a weak attractor if and only if for each compact set $B\subset {\mathbb {X}}$

$$\begin{aligned} \lim _{n \rightarrow \infty } \textrm{dist}\left( \varphi _{\theta ^{-n}q}^n(B), A_q\right) =0 \ \text { in probability.} \end{aligned}$$

Since the state space is discrete it is straightforward to see that if the strong pullback convergence holds for all point sets, i.e. for $B=\lbrace x\rbrace$ where $x\in {\mathbb {X}}$, then it also holds for any finite B. Using the terminology from [6, 7], this would imply that strong point attractors are equivalent to strong (set) attractors as defined above. Indeed, assuming that (34) holds for all point sets $B=\{x\}$, where $x\in K$ for a given finite set K, it also ${\mathbb {P}}$-a.s. holds for $B=K$ by taking $N(q,K)=\max \left\{ N(q,x): x\in K \right\}$. In a similar spirit, we give equivalent conditions for an invariant finite random set to be a weak attractor in the next theorem.

Theorem 9

Let $A:{\mathcal {Q}}\rightarrow {\mathcal {P}}({\mathbb {X}})$ be an invariant finite random set. Then the following are equivalent.

• I. A is a weak attractor.

• II. A is a forward attractor.

• III. A is a forward point attractor, that is (32) is satisfied for all $B=\lbrace x\rbrace$ with $x\in {\mathbb {X}}$.

• IV. A is a weak point attractor, that is (33) is satisfied for all $B=\lbrace x\rbrace$ with $x\in {\mathbb {X}}$.

• V. A weakly attracts random finite sets, that is for any finite random set $K:{\mathcal {Q}}\rightarrow {\mathcal {P}}({\mathbb {X}})$ we have that

  $$\begin{aligned} \lim _{n\rightarrow \infty }\textrm{dist}\left( \varphi ^n_q(K_q),A_{\theta ^n q}\right) =0 \ \text { in probability.} \end{aligned}$$

Proof

I implies II. Assume that A is a weak attractor. Take $x\in {\mathbb {X}}$ as an arbitrary initial condition and consider the sets ${\mathcal {Q}}_n^x:=\left\{ q\in {\mathcal {Q}}: \varphi ^n_{q}(x)\in A_{\theta ^{n}q} \right\}$. Since the state space is discrete, we have

$$\begin{aligned} {\mathbb {P}}({\mathcal {Q}}_n^x) = 1-{\mathbb {P}}\left( d\left( \varphi ^n_{q}(x),A_{\theta ^{n}q} \right) \ge r \right) \end{aligned}$$

for any $r\in (0,1]$. From the convergence in probability to the attractor (cf. (33)) it follows that for an arbitrary $\varepsilon >0$ there exists $N\equiv N(x,\varepsilon )$ such that ${\mathbb {P}}({\mathcal {Q}}_n^x)\ge 1-\varepsilon$ for all $n\ge N$. Furthermore, from the $\varphi$-invariance of A we know that ${\mathcal {Q}}_n^x\subset {\mathcal {Q}}_{n+1}^x$ for all $n\in {\mathbb {N}}$.

Let ${\mathcal {Q}}^x:=\bigcup _{n\in {\mathbb {N}}}{\mathcal {Q}}_n^x=\left\{ q\in {\mathcal {Q}}: \lim _{n\rightarrow \infty } d\left( \varphi ^n_{q}(x),A_{\theta ^{n}q}\right) =0\right\}$. Notice that for any $n\ge N$

$$\begin{aligned} {\mathbb {P}}({\mathcal {Q}}^x)\ge {\mathbb {P}}({\mathcal {Q}}_n^x)\ge 1-\varepsilon , \end{aligned}$$

and since $\varepsilon$ was arbitrarily small we have that ${\mathbb {P}}({\mathcal {Q}}^x)=1$. Now, we set

$$\begin{aligned}{\mathcal {Q}}_B:=\bigcap _{x\in B} {\mathcal {Q}}^x = \left\{ q\in {\mathcal {Q}}: \lim _{n\rightarrow \infty }d\left( \varphi ^n_{q}(x),A_{\theta ^{n}q}\right) = 0 \; \forall x\in B\right\} .\end{aligned}$$

As B is finite, we have

$$\begin{aligned} {\mathcal {Q}}_B = \left\{ q\in {\mathcal {Q}}:\lim _{n\rightarrow \infty }\textrm{dist}\left( \varphi ^n_{q}(B),A_{\theta ^{n}q}\right) =0 \right\} ,\end{aligned}$$

and, using ${\mathbb {P}}({\mathcal {Q}}^x)=1$ for all x, we get ${\mathbb {P}}({\mathcal {Q}}_B)=1$ meaning that A is a forward attractor (cf. (32)).

II implies III. This follows directly by taking $B=\lbrace x \rbrace$.

III implies IV. This statement is true from the fact that convergence a.s. implies convergence in probability.

IV implies I. Assume that (33) holds for $B=\{x\}$ for any $x\in {\mathbb {X}}$, which is equivalent to

$$\begin{aligned} \lim _{n\rightarrow \infty }{\mathbb {P}}\left( \varphi ^n_q(x)\not \in A_{\theta ^n q} \right) =0. \end{aligned}$$

Let $K=\lbrace x_1,\ldots , x_m\rbrace \subset {\mathbb {X}}$ for some $m\in {\mathbb {N}}$. Since for any $n\in {\mathbb {N}}$

$$\begin{aligned} \left\{ q: \textrm{dist}(\varphi ^n_q(K),A_{\theta ^{n}q})\ge 1 \right\} = \bigcup _{i=1}^m\left\{ q: \varphi ^n_{q}(x_i)\not \in A_{\theta ^n q} \right\} , \end{aligned}$$

it follows that

$$\begin{aligned} \lim _{n\rightarrow \infty }{\mathbb {P}}\left( \textrm{dist}(\varphi ^n_q(K),A_{\theta ^{n}q})\ge 1 \right) \le \sum _{i=1}^m{\lim _{n\rightarrow \infty }{\mathbb {P}}\left( \varphi ^n_q(x_i)\not \in A_{\theta ^nq} \right) }=0, \end{aligned}$$

and the claim follows.

I if and only if V. Clearly, V implies I, since for each finite set $B\subset {\mathbb {X}}$ one can take $K_q=B$ for all q. Conversely, if K is a finite random set and $\varepsilon >0$ an arbitrarily small constant, consider $F_\varepsilon >0$ as given in Proposition 4 so that

$$\begin{aligned} {\mathbb {P}}\left( K_q\subset F_\varepsilon \right) \ge 1-\varepsilon . \end{aligned}$$

(35)

Note that for any $n,m\in {\mathbb {N}}$ we have that

$$\begin{aligned} {\mathbb {P}}\left( \varphi ^n_q(K_q)\not \subset A_{\theta ^n}q \right)= & {} {\mathbb {P}}\left( \varphi ^n_q(K_q)\not \subset A_{\theta ^n}q \wedge K_q\subset F_\varepsilon \right) \\ {}{} & {} + {\mathbb {P}}\left( \varphi ^n_q(K_q)\not \subset A_{\theta ^n}q \wedge K_q\not \subset F_\varepsilon \right) . \end{aligned}$$

Hence, it follows from (35) and the observation

$$\begin{aligned} \left\{ \varphi ^n_q(K_q)\not \subset A_{\theta ^n}q \ \wedge \ K_q\subset F_\varepsilon \right\} \subset \left\{ \varphi ^n_q(F_\varepsilon )\not \subset A_{\theta ^n}q\right\} , \end{aligned}$$

that for each $\varepsilon >0$ there is $m\in {\mathbb {N}}$ such that

$$\begin{aligned} {\mathbb {P}}\left( \varphi ^n_q(K_q)\not \subset A_{\theta ^n}q \right) \le {\mathbb {P}}\left( \varphi ^n_q(F_\varepsilon )\not \subset A_{\theta ^n}q \right) +\varepsilon . \end{aligned}$$

Since $F_\varepsilon$ is deterministic and finite, statement I implies ${\mathbb {P}}\left( \varphi ^n_q(F_\varepsilon )\not \subset A_{\theta ^n}q \right) \rightarrow 0$ for $n\rightarrow \infty$. As $\varepsilon$ was arbitrarily small, this implies ${\mathbb {P}}\left( \varphi ^n_q(K_q)\not \subset A_{\theta ^n}q \right) \rightarrow 0$ for $n\rightarrow \infty$. $\square$

Remark 5

Note that, by virtue of this theorem, weak set attractors and weak point attractors are the same for this discrete-time and discrete-space setting. Hence, the weak set attractor being a random point almost surely is equivalent to the weak point attractor having this property. In particular, this implies that the distinction between synchronization and weak synchronization defined via an attractor being a singleton, as done in [11], is not necessary here. In addition, note that, again by Theorem 9, our definition of synchronization (Sect. 3.1) coincides with the one in [11], if an attractor exists (under extension of ${\mathcal {Q}}_+$ to ${\mathcal {Q}}$).

Due to their invariance and their attractivity, it becomes relevant to understand the dynamics within the attractors. In particular, the attractor for strict birth-death chains admits a periodic behavior, as we see later in Theorem 15.

Definition 10

A random set $A:{\mathcal {Q}}\rightarrow {\mathcal {P}}({\mathbb {X}})$ is a random periodic orbit of period M for the RDS $(\theta ,\varphi )$ if for ${\mathbb {P}}$-a.e. $q \in {\mathcal {Q}}$, $A_q=\{a_0(q), \dotsc , a_{M-1}(q)\}$ such that

$$\begin{aligned} \varphi _{q}^1(a_i(q))=a_{i+1 (\bmod M)}(\theta q) \quad \text {for } i = 0, \dotsc , M-1. \end{aligned}$$

Clearly a random periodic orbit is in particular $\varphi$-invariant. Furthermore, we say that $A:{\mathcal {Q}}\rightarrow {\mathcal {P}}({\mathbb {X}})$ is a (weak) attracting random periodic orbit if it is a random periodic orbit and a weak attractor. Note that this can be seen as a discrete-time analogue to the continuous-time oriented definition of a random periodic solution [38] and its generalization [10].

4.2 Existence of Weak Attractors

In this subsection we provide general conditions for an RDS as given in the previous section to admit a weak attractor. For this purpose, we combine [7, Thm. 10] on the existence of a weak attractor with properties of the stationary distribution (in case of its existence) for Markov chains on countable state spaces (cf. [8, Chapter 6]).

Theorem 11

Consider an RDS $(\theta ,\varphi )$ on ${\mathbb {X}}$, as defined in Sect. 4.1, and assume that for every $x\in {\mathbb {X}}$ the set $\varphi ^1_{{\mathcal {Q}}}(x):=\lbrace \varphi ^1_q(x): q\in {\mathcal {Q}}\rbrace \subset {\mathbb {X}}$ is finite, and that the Markov chain associated to $(\theta ,\varphi )$ is irreducible and recurrent. If it admits a stationary distribution, then the RDS admits a weak attractor.

Proof

In order to prove the theorem we use the following (in fact, equivalent) criterion for the existence of a weak attractor from [7, Thm. 10]: For every $\varepsilon > 0$ there exists a compact set $C_{\varepsilon } \subset {\mathbb {X}}$ such that for every compact set $K \subset {\mathbb {X}}$ there is a $n_0 \in {\mathbb {N}}$ so that for all $n \ge n_0$

$$\begin{aligned} {\mathbb {P}} \left( \varphi _q^n (K) \subset C_{\varepsilon }\right) \ge 1 - \varepsilon . \end{aligned}$$

Recall that a recurrent state $x\in {\mathbb {X}}$ has period $\ell \ge 1$ if $\ell$ is the greatest common denominator of the set

$$\begin{aligned} \left\{ n\ge 1: {\mathbb {P}}\left( \varphi ^n_{\omega }(x)=x \right) >0 \right\} . \end{aligned}$$

Since the chain is irreducible, every state has the same period. Moreover, the state space admits a cyclic decomposition given as ${\mathbb {X}}=W_0\cup W_1\cup \cdots \cup W_{\ell -1}$, where for $x\in W_i$ and $i\in \lbrace 0,1,\ldots ,\ell -1\rbrace$

$$\begin{aligned} {\mathbb {P}}\left( \varphi ^1_q(x)\in W_j \right) = {\left\{ \begin{array}{ll} 1 &{} \text {if } j=i+1 \ (\text {mod } \ell ),\\ 0 &{} \text {otherwise} \end{array}\right. }, \end{aligned}$$

and the $\ell$-step process $\left( \varphi ^{\ell n}_q(x) \right) _{n\in {\mathbb {N}}_0}$ is aperiodic and irreducible in each $W_i$ (see for instance [8, Lemma 6.7.1]). Denote by $\rho$ the unique stationary distribution of the system and let $\varepsilon$ be arbitrarily small. Consider $z_\varepsilon \in {\mathbb {N}}$ large enough such that

$$\begin{aligned} \max _{0,1,\ldots \ell -1}\frac{\rho \big (({\mathbb {X}}\setminus C_\varepsilon ) \cap W_i)\big )}{\rho (W_i)}<\frac{\varepsilon }{2} \end{aligned}$$

(36)

for the set $C_{\varepsilon }:=\lbrace 0,1,2,\ldots z_\varepsilon \rbrace ^L$.

Step 1. For each $i=0,\ldots ,\ell -1$, the $\ell$-step process $\left( \varphi _q^{\ell n}(x)\right) _{n \in {\mathbb {N}}_0}$ starting in $x\in W_i$ admits a unique stationary distribution ${\tilde{\rho }}_i$ supported on $W_i$.

For $i=0,\ldots ,\ell -1$ and $A\subset {\mathbb {X}}$ set

$$\begin{aligned} {\tilde{\rho }}_i(A):=\frac{\rho (A \cap W_i)}{\rho (W_i)}. \end{aligned}$$

(37)

Then, for any $n\in {\mathbb {N}}$ we have

$$\begin{aligned} \sum _{x=0}^{\infty }{{\mathbb {P}}\left( \varphi _q^{\ell n}(x)\in A\right) \cdot {\tilde{\rho }}_i(x)}&= \frac{1}{\rho (W_i)}\sum _{x \in W_i}{{\mathbb {P}}\left( \varphi _q^{\ell n}(x)\in A\right) \cdot \rho (x)}= \\&{\mathop {=}\limits ^{(*)}} \frac{1}{\rho (W_i)}\sum _{x \in W_i}{{\mathbb {P}}\left( \varphi _q^{\ell n}(x)\in A \cap W_i\right) \cdot \rho (x)}= \\&= \frac{1}{\rho (W_i)}\sum _{x=0}^\infty {{\mathbb {P}}\left( \varphi _q^{\ell n}(x)\in A \cap W_i\right) \cdot \rho (x)}\\ {}&=\frac{\rho (A\cap W_i)}{\rho (W_i)}= {\tilde{\rho }}_i(A), \end{aligned}$$

which implies that ${\tilde{\rho }}_i$ is a stationary distribution of $(\varphi _q^{\ell n}(x))_{n \in {\mathbb {N}}_0}$ on $W_i$. In $(*)$ we used the fact that ${\mathbb {P}}\left( \varphi _q^{\ell n}(x_0)=y\right) =0$ $\forall x_0\in W_i, y \notin W_i$. As the $\ell$-step process is irreducible in each $W_i$, the stationary distribution is unique (see [8, Theorem 6.5.5]).

Step 2. Let $x\in {\mathbb {X}}$ and $\varepsilon >0$ be given as before. Then, there exists $n_1= n_1(x)\in {\mathbb {N}}$ such that ${{\mathbb {P}}\left( \varphi _q^{\ell n}(x)\not \in C_\varepsilon \right) <\varepsilon }$ for all $n\ge n_1$.

Consider $x\in W_i$ for any $i\in \lbrace 0,\ldots ,\ell -1\rbrace$. Since the $\ell$-step process $\left( \varphi _q^{\ell n}(x)\right) _{n \in {\mathbb {N}}_0}$ is irreducible and admits a stationary distribution on $W_i$, it is positive recurrent on $W_i$. Furthermore, since it is aperiodic in $W_i$, convergence ${\mathbb {P}}\left( \varphi _q^{\ell n}(x)\in \cdot \right) \rightarrow {\tilde{\rho }}_i$ in total variation holds as $n \rightarrow \infty$, see [8, Theorem 6.6.4]. This implies in particular the convergence ${\mathbb {P}}\left( \varphi _q^{\ell n}(x)\not \in C_\varepsilon \right) \rightarrow {\tilde{\rho }}_i( {\mathbb {X}}{\setminus } C_{\varepsilon } )$. Hence, there exists $n_1= n_1(x)\in {\mathbb {N}}$ such that for any $n\ge n_1$ we obtain

$$\begin{aligned} {\mathbb {P}}\left( \varphi _q^{\ell n}(x)\not \in C_\varepsilon \right)< {\tilde{\rho }}_i( {\mathbb {X}}\setminus C_{\varepsilon } )+\frac{\varepsilon }{2} {\mathop {=}\limits ^{(37)}}\frac{\rho \big (({\mathbb {X}}\setminus C_{\varepsilon })\cap W_i\big )}{\rho (W_i)}+\frac{\varepsilon }{2} {\mathop {<}\limits ^{(36)}} \varepsilon . \end{aligned}$$

(38)

Step 3. Let $x\in {\mathbb {X}}$ be an arbitrary initial condition. Then, there exists $n_2= n_2(x)\in {\mathbb {N}}$ such that for all $n\ge n_2$ we have ${{\mathbb {P}}(\varphi _q^{n}(x)\not \in C_{\varepsilon })<\varepsilon }$.

Since $\varphi ^1_{{\mathcal {Q}}}(x)$ is assumed to be finite, also $\varphi ^r_{{\mathcal {Q}}}(x):=\left\{ \varphi ^r_q(x): q\in {\mathcal {Q}} \right\}$ is finite for any $r\in {\mathbb {N}}_0$. Let $n=k\ell +r$ for some $k\in {\mathbb {N}}_0$ and $r\in \lbrace 0,1,\ldots ,\ell -1 \rbrace$. Note that

$$\begin{aligned}{\mathbb {P}}\left( \varphi _q^{\ell k +r}(x)\not \in C_\varepsilon \right) = \sum _{y\in \varphi ^r_{{\mathcal {Q}}}(x)} {\mathbb {P}}\left( \varphi ^{\ell k}_q(y)\not \in C_\varepsilon \right) \cdot {\mathbb {P}}\left( \varphi ^r_q(x)=y\right) . \end{aligned}$$

Let $n_1(y)$ be given from Step 2 above, and choose $k\ge \max \lbrace n_1(y): y\in \varphi ^r_{{\mathcal {Q}}}(x)\rbrace$. Then

$$\begin{aligned}{\mathbb {P}}\left( \varphi _q^{\ell k +r}(x)\not \in C_\varepsilon \right) {\mathop {<}\limits ^{(38)}}\varepsilon \cdot \sum _{y\in \varphi ^r_{{\mathcal {Q}}}(x)} {\mathbb {P}}\left( \varphi ^r_q(x)=y\right) \ =\varepsilon . \end{aligned}$$

So, the claim follows by taking $n_2(x):=\max \left\{ \ell \cdot n_1(y): y\in \varphi ^r_{\mathcal {Q}}(x), r\in \lbrace 0,1,\ldots ,\ell -1 \rbrace \right\}$.

Step 4. For each $K\subset {\mathbb {N}}_0$ compact (i.e. finite) there exists $n_0= n_0(K)\in {\mathbb {N}}$ such that ${\mathbb {P}}\left( \varphi _q^{n}(x)>z\right) <\varepsilon$ for all $n\ge n_0$ and $x\in K$.

Notice that Step 3 assures that the claim holds when $K=\lbrace x \rbrace$ for any $x\in {\mathbb {N}}_0$. If we consider any finite set $K=\lbrace x_1,x_2,\ldots ,x_l\rbrace$, then the result follows by taking $n_0(K)=\max _{i=1,\ldots , l}n_2( x_i)$. $\square$

4.3 Random Periodic Orbit of Positive Recurrent Strict Birth-Death Chains

Recall that the concept of a random attractor as introduced earlier only makes sense when considering an invertible dynamical system on the noise space. However, in the RDS formulation of the embedded Markov chain (see Sect. 2.2), the shift map is not invertible since for $q_0\ne {\tilde{q}}_0$ we have that

$$\begin{aligned} \theta (q_0,q_1,q_2,\ldots )=\theta ({\tilde{q}}_0,q_1,q_2)=(q_1,q_2,\ldots ). \end{aligned}$$

We can come around this inconvenience by redefining our noise space as

$$\begin{aligned}{\mathcal {Q}}:=\lbrace q=(q_n)_{n\in {\mathbb {Z}}}: q_n\in (0,1) \rbrace ,\end{aligned}$$

endowed with its Borel $\sigma$-algebra $\sigma ({\mathcal {Q}})$ and the bi-infinite product measure $\lambda ^{{\mathbb {Z}}}$. We redefine the shift map as

$$\begin{aligned} \theta q=\theta (q_n)_{n\in {\mathbb {Z}}} =(q_{n+1})_{n\in {\mathbb {Z}}} \end{aligned}$$

such that $(\theta ^{-n}q)_i = q_{i-n}$, while the cocycle map $\varphi$ remains the same, see (13). Note that the synchronization results in Sect. 3 transfer immediately to the invertible setting since they have been formulated independently from the past. By abuse of notation, we will from now on use ${\mathbb {P}}=\lambda ^{{\mathbb {Z}}}$.

Note that, by virtue of Theorem 11, the existence of the weak attractor can be derived purely by Markov chain arguments and is expected to occur in a large class of RDS derived from reaction jump processes via the embedded Markov chain approach. For instance, since chemical reaction networks are defined via a finite set of reactions, $\varphi ^1_{{\mathcal {Q}}}(x)$ is always finite for any $x\in {\mathbb {X}}$. In particular, using Theorem 11 the following corollary is a direct consequence of Propositions 6 and 7.

Corollary 12

For the standard and Schlögl birth-death chains, any RDS representation admits a weak attractor.

4.3.1 Weak Attraction

In this subsection we provide a full characterization of the weak random attractor of strict birth-death chains. We investigate the structure of the weak attractor by formulating Proposition 13 and Corollary 14 as preparatory work for Sect. 4.3.2, where we will show that the weak attractor is in fact a strong pullback attractor consisting of two random points. The crucial insight is the translation of the partial synchronization result from Lemma 2 into the random periodic structure of the random attractor. Depending on the type of partition in such a partial synchronization, the following analysis may well be understood as a blueprint for various forms of augmented Markov chains with more complicated structure than birth-death processes. However, for this section we always assume that a RDS representation $(\theta ,\varphi )$ of a strict birth-death chain is fixed.

Proposition 13

The weak attractor $q\mapsto A_q$ of a positive recurrent strict birth-death chain has two points on each fiber ${\mathbb {P}}$-a.s., that is

$$\begin{aligned} {\mathbb {P}}(\#A_q=2)=1. \end{aligned}$$

Moreover, $A_q\cap W_i\ne \emptyset$ for $i=0,1$ ${\mathbb {P}}$-a.s., where $W_0=\{0,2,4,\ldots \}$ and $W_1=\{1,3,5,\ldots \}$.

Proof

We split the proof in two parts. We first show that $A_q$ has at least two points ${\mathbb {P}}$-a.s., and then we conclude that ${\mathbb {P}}(\# A_q\ge 3)=0$.

Step 1. $A_q\cap W_i\ne \emptyset$ for $i=0,1$ ${\mathbb {P}}$-a.s.

For each $x\in {\mathbb {N}}_0$ consider the pullback limit

$$\begin{aligned} a_x^+(q):=\limsup _{n\rightarrow \infty }\varphi ^{2n}_{\theta ^{-2n}q}(x). \end{aligned}$$

Let $(n_j)_{j=0}^{\infty }$ be a strictly increasing sequence of natural numbers such that $\varphi ^{2n_j}_{\theta ^{-2n_j}q}(x)\rightarrow a_x^+(q)$ for $j\rightarrow \infty$. Since $A_q$ is a weak attractor we have

$$\begin{aligned} \lim _{j\rightarrow \infty } {\mathbb {P}}\left( d\left( \varphi ^{2n_j}_{\theta ^{-2n_j}q}(x), A_q \right) \ge 1 \right) =0. \end{aligned}$$

From the last limit we consider a subsequence $n_{j_k}$ such that $\varphi ^{2n_{j_k}}_{\theta ^{-2n_{j_k}}q}(x)\in A_q$ for all $k\in {\mathbb {N}}_0$ ${\mathbb {P}}$-a.s. We conclude that $a_x^+(q)\in A_q$ for ${\mathbb {P}}$-a.e. q, which in particular implies that $a_x^+<\infty$ ${\mathbb {P}}$-a.s. Furthermore, since $a_x^+(q)\in W_i$ if and only if $x\in W_i$, the claim follows by taking $x=0,1$, for instance.

Step 2. ${\mathbb {P}}(\# A_q\ge 3)=0$.

Since $\theta$ is ${\mathbb {P}}$-invariant, we have that for any $n\in {\mathbb {N}}$

$$\begin{aligned} {\mathbb {P}}(\# A_q\ge 3)= {\mathbb {P}}\left( \# A_{\theta ^n q}\ge 3\right) ={\mathbb {P}}\left( \# \varphi ^n_q(A_q)\ge 3\right) , \end{aligned}$$

where the last equality follows by the invariance of $A_q$. We now partition

$$\begin{aligned}\varphi ^n_q(A_q)=\varphi ^n_q(A_q\cap W_0)\cup \varphi ^n_q(A_q\cap W_1).\end{aligned}$$

Since it follows from Step 1 that for all $n\in {\mathbb {N}}$ and $i\in \lbrace 0,1\rbrace$ the sets $\varphi ^n_q(A_q\cap W_i)$ contain at least one element, we can combine these observations to give the bound

$$\begin{aligned} {\mathbb {P}}(\# A_q\ge 3)\le {\mathbb {P}}\left( \# \varphi ^n_q(A_q\cap W_0)\ge 2\right) + {\mathbb {P}}\left( \# \varphi ^n_q(A_q\cap W_1)\ge 2\right) . \end{aligned}$$

By taking the limit $n\rightarrow \infty$, Proposition 5 implies that the right-hand side tends to 0 by taking $K^i_q=A_q\cap W_i$. The result follows. $\square$

Corollary 14

For each $x\in {\mathbb {N}}_0$

$$\begin{aligned} a_x(q):=\lim _{n\rightarrow \infty }\varphi ^{2n}_{\theta ^{-2n}q}(x) \end{aligned}$$

(39)

exists ${\mathbb {P}}$-a.s. Furthermore, for $x,y\in W_i$, $i=0,1$, we have that $a_x=a_y$ ${\mathbb {P}}$-a.s. Conversely, if $x\in W_0$ and $y \in W_1$, then $a_x\ne a_y$ ${\mathbb {P}}$-a.s.

Proof

For each $x\in {\mathbb {N}}_0$ let

$$\begin{aligned} a_x^-(q):=\liminf _{n\rightarrow \infty } \varphi ^{2n}_{\theta ^{-2n}q}(x). \end{aligned}$$

Analogously to Step 1 in the proof of Proposition 13, we obtain $a_x^-(q)\in A_q$ for almost all $q \in {\mathcal {Q}}$. Since it has the same parity as $a_x^+(q)$ and $\#A_q\cap W_i=1$, $i=0,1$, we conclude that $a_x^+=a_x^-$ holds with full probability and, hence, the limit $a_x \in A_q$ exists almost surely.

Using again that $\#A_q\cap W_i=1$, $i=0,1$, we derive that $a_x$ and $a_y$ are identical (or different, respectively) in a full measure set when x and y are of the same parity (or of different parities, respectively). $\square$

4.3.2 Strong Pullback Attraction to a Random Periodic Orbit

We can now give a full characterization of the pullback attractor in the next theorem. We already know from Proposition 13 and Corollary 14 that the weak attractor $A_q$ consists almost surely of the two distinct random points $a_0(q)$ and $a_1(q)$, as given in (39). Now, we identify the strong pullback structure of this attractor.

Theorem 15

The weak attractor $A_q = \{a_0(q), a_1(q)\}$ for a positive recurrent strict birth-death chain

• (i) is a random periodic orbit of period 2, and

• (ii) is a pullback and a forward attractor.

Proof

At first, we use the cocycle property in order to show item (i), that is

$$\begin{aligned} \varphi _{q}^1(a_0(q))=a_1(\theta q) \qquad \text {and} \qquad \varphi _{q}^1(a_1(q))=a_0(\theta q) \end{aligned}$$

(40)

is satisfied ${\mathbb {P}}$-a.s. Indeed, let $i=0,1$, $q \in {\mathcal {Q}}$ be fixed. Then,

$$\begin{aligned} \varphi _{q}^1(a_i(q))=\lim _{n\rightarrow \infty }\varphi ^{2n+1}_{\theta ^{-2n}q}(i)=\lim _{n\rightarrow \infty }\varphi ^{2n}_{\theta ^{-2n}\circ \theta q}\left( \varphi _{q}^1(i) \right) =a_{i+1 (\bmod 2)}(\theta q), \end{aligned}$$

where the last equality follows directly from Corollary 14 and the fact that $\varphi _{q}^1(i)\in W_{i+1 (\bmod 2)}$.

In order to show (ii), let $q \in {\mathcal {Q}}$. It follows from the definition (39) that, for any $x\in W_i$ with $i=0,1$, there exists $N_0= N_0(x)\in {\mathbb {N}}_0$ such that for all $n\ge N_0$ we have $\varphi _{\theta ^{-2n}q}^{2n}(x)=a_i(q)\in A(q)$. On the other hand, by the cocycle property and (40) it follows that

$$\begin{aligned} \lim _{n\rightarrow \infty }\varphi _{\theta ^{-2n-1}}^{2n+1}(x)=\lim _{n\rightarrow \infty } \varphi ^1_{\theta ^{-1}q}\left( \varphi ^{2n}_{\theta ^{-2n}\theta ^{-1}q}(x) \right) = \varphi ^1_{\theta ^{-1}q}(a_{i}(\theta ^{-1}q))=a_{i+1(\bmod 2)}(q). \end{aligned}$$

Hence, there is $N_1= N_1(x)\in {\mathbb {N}}$ such that for all $n\ge N_1$ we have that $\varphi _{\theta ^{-2n-1}q}^{2n+1}(x)\in A(q)$. Combining both parts, we obtain $\varphi _{\theta ^{-n}q}^{n}(x)\in A(q)$ for all $n\ge N(x):=\max \lbrace 2N_0(x), 2N_1(x)+1\rbrace$. Recall that point pullback attractors are (set) pullback attractors due to the state space being discrete, and thus A is a pullback attractor. Owing to Theorem 9, A is also a forward attractor. $\square$

4.3.3 Sample Measures Supported on the Attractor

In this subsection we briefly describe the statistical importance of the attractor A in terms of the invariant measure for the skew-product map $\Theta :{\mathcal {Q}}\times {\mathbb {X}}\rightarrow {\mathcal {Q}}\times {\mathbb {X}}$ given by

$$\begin{aligned} \Theta (q,x):=(\theta q, \varphi _q(x)). \end{aligned}$$

Denoting by $T^*\mu$ the push forward of a measure $\mu$ by a map T, i.e. $T^* \mu (\cdot ) = \mu (T^{-1}(\cdot ))$, we adopt the classical definition of an invariant measure for the RDS (see e.g. [3, Definition 1.4.1]): A probability measure $\mu$ on ${\mathcal {Q}} \times {\mathbb {X}}$ is invariant for the random dynamical system $(\theta , \varphi )$ if

• (i) $\Theta _t^* \mu = \mu$ for all $t \in \mathbb {\mathbb {N}}_0$ ,

• (ii) the marginal of $\mu$ on ${\mathcal {Q}}$ is ${\mathbb {P}}$, i.e. $\mu$ can be factorized uniquely into

  $$\begin{aligned} \mu (\textrm{d}q, x) = \mu _{q}( x) {\mathbb {P}}(\textrm{d}q), \end{aligned}$$

  where $q\mapsto \mu _{q}$ is the sample measure (or disintegration) on ${\mathbb {X}}$, i.e., $\mu _{q}$ is almost surely a probability measure on ${\mathbb {X}}$ and $q \mapsto \mu _{q}(B)$ is measurable for all $B \subset {\mathbb {X}}$.

In particular note that, since ${\mathbb {P}}$ is given, the sample measures $\mu _q$ completely determine such an invariant measure. A specific form of such invariant measures are Markov measures, characterized by the sample measures being measurable with respect to the past: in our setting, this means that the $\mu _q$ only depend on $q_n, n < 0$ (cf. e.g. [24] or [20]).

The theory of random dynamical systems now gives us the following result on the unique invariant measure for the RDS at hand, relating it to the stationary distribution of the Markov chain:

Proposition 16

The RDS of positive recurrent strict birth-death chains possess a unique invariant Markov measure with sample measures

$$\begin{aligned} \mu _{q} = \frac{1}{2} \delta _{a_0(q)} + \frac{1}{2} \delta _{a_1(q)}, \end{aligned}$$

such that ${\mathbb {E}}[\mu _{q}] = \rho$, where $\rho$ is the unique stationary distribution from Proposition 6.

Proof

By [6, Proposition 4.5], we know that there exists a Markov measure $\mu$ such that $\mu _q(A(q)) =1$ almost surely, where $A(q)=\lbrace a_0(q),a_1(q)\rbrace$ is the attractor from Theorem 15. Its uniqueness and the fact that ${\mathbb {E}}[\mu _{q}] = \rho$ follow from the celebrated correspondence theorem, also called Ledrappier-LeJan-Crauel Theorem (see [24, Proposition 1.2.3] for a version that suffices for our situation and [20, Theorem 4.2.9] for the more general situation).

Using [23] (see also [11, Lemma 2.19]) we can directly infer that either $\mu _{q} = \frac{1}{2} \delta _{a_0(q)} + \frac{1}{2} \delta _{a_1(q)}$ almost surely or $\mu _{q} = \delta _{a_i(q)}$ almost surely for $i=0$ or $i=1$ fixed. The latter case can now be excluded by combining (40) and the invariance property $(\varphi _q^n)^* \mu _{q} = \mu _{\theta ^n q}$ [20, Proposition 1.3.27]. $\square$

5 Qualitative Features of the Two-Point Motion

In this section we present a comparison between the behavior of the two-point motion of the standard birth-death chain and the Schlögl birth-death chain, referring to the embedded Markov chains of the reaction systems given in Examples 1 and 2, respectively. As shown in Theorem 15, both birth-death chains admit an attracting two-periodic orbit. Despite this topological similarity, we further distinguish their behavior in terms of an absorbing region for the two-point motion, that is the random dynamics on ${\mathbb {X}}\times {\mathbb {X}}$ given by the cocycle map $(\varphi _{q}^n(x_0), \varphi _{q}^n(y_0))_{n \in {\mathbb {N}}_0}$ over the same two-sided sequence space $({\mathcal {Q}},{\mathcal {B}},{\mathbb {P}})$; see Fig. 5 for an example of the dynamics of two different pairs of initial states for the standard birth-death chain.

Fig. 5

Two-point motion $(\varphi _{q}^n(x_0), \varphi _{q}^n(y_0))_{n \in {\mathbb {N}}_0}$ of the birth-death process from Example 1 with different starting points (indicated by the red dots). The trajectory in (a) starts in $(x_0,y_0)=(5,15)$ with even distance $|x_0-y_0|=10$ and belongs to the realization shown in Fig. 2a. In (b), the trajectory starts in $(x_0,y_0)=(6,15)$ with coordinates with odd distance $|x_0-y_0|=9$ and refers to Fig. 2b. In (a), the two-point motion quickly ends up on the diagonal, in contrast to (b) where it ends up on the thick diagonal ${\mathbb {D}}$ defined in (41). The rate constants set to $\gamma _1=10$, $\gamma _2=1$ (Color figure online)

In particular, for the standard birth-death chain we conclude that the thick diagonal

$$\begin{aligned} {\mathbb {D}}:=\Big \{ (x,y)\in {\mathbb {N}}_0^2: y\in \lbrace x-1,x,x+1\rbrace \Big \} \end{aligned}$$

(41)

is forward invariant for the two-point motion, i.e. $\left( \varphi ^n_q(x),\varphi ^n_q(y)\right) \in {\mathbb {D}}$ for all $n\in {\mathbb {N}}$ if $(x,y)\in {\mathbb {D}}$, and absorbing; see Fig. 6 for an illustration. On the other hand, for the Schlögl chain we explore numerically the region in ${\mathbb {X}}\times {\mathbb {X}}$ where the realizations of the two-point motion remain.

5.1 Two-Point Motion of the Standard Birth-Death Chain

Just as for the one-point motion $(\varphi _q^n(x_0))_{n \in {\mathbb {N}}_0}$, we can also determine the transition probabilities for the Markovian dynamics of the two-point motion $\left( \varphi _{q}^n(x_0), \varphi _{q}^n(y_0)\right) _{n \in {\mathbb {N}}_0}$. For $z_1,z_2\in \{1,-1\}$ set

$$\begin{aligned} P_{(z_1,z_2)} (x,y):= {\mathbb {P}}\Big (\;\big (\varphi _{q}^{n+1}(x_0),\varphi _{q}^{n+1}(y_0)\big ) =(x+z_1,y+z_2) \;\Big \vert \; \big (\varphi _{q}^n(x_0), \varphi _{q}^n(y_0)\big )=(x,y)\;\Big ) \end{aligned}$$

for any $(x,y) \in {\mathbb {N}}_0^2$. Given (25) we can deduce that the transition probabilities are

$$\begin{aligned}&P_{(1,1)} (x,y) = \min \{P_1(x),P_1(y)\},&\text {(yellow)} \end{aligned}$$

(42)

$$\begin{aligned}&P_{(-1,1)} (x,y) =\max \{0,P_1(y)-P_1(x)\},&\text {(red)} \end{aligned}$$

(43)

$$\begin{aligned}&P_{(1,-1)} (x,y) =\max \{0,P_1(x)-P_1(y)\},&\text {(blue)} \end{aligned}$$

(44)

$$\begin{aligned}&P_{(-1,-1)} (x,y) = 1-\max \{P_1(x),P_1(y)\}.&\text {(green)} \end{aligned}$$

(45)

The colors refer to the transitions given by the arrows in Fig. 6. Since we have $P_{(-1,1)}(x,y)=0$ for $x<y$ and $P_{(1,-1)}(x,y)=0$ for $x>y$, it follows immediately that the set ${\mathbb {D}}$ is forward-invariant for the two-point motion.

5.1.1 First Hitting Time of ${\mathbb {D}}$

We will show that the absorbing set ${\mathbb {D}}$ is reached by the two-point motion $\left( \varphi _{q}^n(x_0), \varphi _{q}^n(y_0)\right) _{n \in {\mathbb {N}}_0}$ of the birth-death process almost surely in finite time regardless of the starting position. Formally speaking, by considering

$$\begin{aligned} \tau _{{\mathbb {D}}}(x_0,y_0,q):=\inf \left\{ n\ge 0: (\varphi _{q}^n(x_0), \varphi _{q}^n(y_0)) \in {\mathbb {D}} \right\} \end{aligned}$$

as the first hitting time of the thick diagonal ${\mathbb {D}}$, we show that

$$\begin{aligned} {\mathbb {P}}\big (\tau _{\mathbb {D}} (x_0,y_0,q) <\infty \big ) =1 \end{aligned}$$

holds for all $x_0,y_0\in {\mathbb {N}}_0$. To do so, we first define for a given $d\in {\mathbb {Z}}$ the level set

$$\begin{aligned} I_d:= \left\{ (x,y)\in {\mathbb {N}}_0^2:x-y=d \right\} \end{aligned}$$

(46)

and show in the following lemma that for $d\ne 0$ a process starting in $I_d$ will almost surely leave this set in finite time.

Fig. 6

Schematic illustration for the transitions of the two-point motion for the birth-death process. The thick diagonal ${\mathbb {D}}$ is defined in (41), the level sets $I_d$ are defined in (46). The arrows indicate the directions in which the two-point motion can move, with the colors indicating the values of the corresponding probabilities as given in (42)–(45)

Lemma 17

Let $d\in {\mathbb {Z}}\setminus \lbrace 0\rbrace$ be given. Then, for each initial state $(x_0,y_0)\in I_d$ the two-point motion $\left( \varphi _{q}^n(x_0), \varphi _{q}^n(y_0)\right) _{n \in {\mathbb {N}}_0}$ of the standard birth-death chain exits $I_d$ ${\mathbb {P}}$-a.s., i.e.,

$$\begin{aligned} {\mathbb {P}}\left( (\varphi _{q}^n(x_0), \varphi _{q}^n(y_0))\in I_d \; \forall n \ge 0 \right) =0 \quad \text {for all } (x_0,y_0) \in I_d.\end{aligned}$$

Proof

Without loss of generality we only consider the case in which $d\ge 1$. For $x \in {\mathbb {N}}_0$ consider the state $(x+d,x)\in I_d$ and define

$$\begin{aligned} p_x:= {\mathbb {P}}\left( (\varphi _{q}^n(x+d), \varphi _{q}^n(x))\in I_d \; \forall n \ge 0 \right) .\end{aligned}$$

as the probability for the two-point motion to stay forever on $I_d$ given that it starts in $(x+d,x)\in I_d$. By means of the law of total probability we have

$$\begin{aligned} p_x = P_{(1,1)}(x+d,x)\cdot p_{x+1} + P_{(-1,-1)}(x+d,x) \cdot p_{x-1} \end{aligned}$$

for $x\ge 1$, where $P_{(1,1)}(x+d,x)=\frac{1}{1+\alpha (x+d)}$ and $P_{(-1,-1)}(x+d,x)=\frac{\alpha x}{1+\alpha x}$ with $\alpha :=\frac{\gamma _2}{\gamma _1}$, see (42) and (45). From this we can deduce the second-order difference equation

$$\begin{aligned} p_{x+2} = (1+\alpha (x+d+1))\cdot \left( p_{x+1} - \frac{\alpha (x+1)}{1+\alpha (x+1)} p_x \right) \end{aligned}$$

(47)

for $x\ge 0$. Moreover, for $x=0$ we have $p_0=P_{(1,1)}(d,0)\cdot p_{1}$ such that

$$\begin{aligned} p_1=(1+\alpha d)p_0. \end{aligned}$$

(48)

Since $p_1$ is proportional to $p_0$, it follows inductively from Eq. (47) that $p_x$ is proportional to $p_0$ for all $x\in {\mathbb {N}}_0$. We prove by contradiction that the sequence $(p_x)_{x \in {\mathbb {N}}_0}$ of probabilities has to fulfill $p_x=0$ for all $x \in {\mathbb {N}}_0$. Indeed, assume that $p_0>0$. From (48) we obtain

$$\begin{aligned} p_1-p_0=\left( 1-\frac{1}{1+\alpha d}\right) p_1=\alpha d p_0>0. \end{aligned}$$

(49)

On the other hand, it follows immediately from (47) that

$$\begin{aligned} p_{x+2}-p_{x+1}=\alpha (x+d+1)p_{x+1}-\frac{\alpha (x+1)\left[ 1+\alpha (x+d+1)\right] }{1+\alpha (x+1)}p_x. \end{aligned}$$

Hence, by adding and subtracting $\frac{\alpha (x+1)\left[ 1+\alpha (x+d+1)\right] }{1+\alpha (x+1)}\cdot p_{x+1}$ we obtain

$$\begin{aligned} p_{x+2}-p_{x+1}&=\frac{\alpha d}{1+\alpha (x+1)}\cdot p_{x+1}+ \frac{\alpha (x+1)\left[ 1+\alpha (x+d+1)\right] }{1+\alpha (x+1)}\cdot (p_{x+1}-p_x) \\&\ge \alpha (x+1)\left( 1+\frac{\alpha d}{1+\alpha (x+1)}\right) (p_{x+1}-p_x). \end{aligned}$$

Let $u_{x}:=p_{x+1}-p_x$. Then, from the last inequality it follows that

$$\begin{aligned} u_{x+1}>\alpha (x+1)u_x. \end{aligned}$$

By iterating the above inequality it follows that

$$\begin{aligned} u_x>\alpha ^{x-1}\cdot x!\; u_0 \qquad x\in {\mathbb {N}}. \end{aligned}$$

Since $u_0>0$, see (49), and since $\alpha ^{x-1}\cdot x!\rightarrow \infty$ as $x\rightarrow \infty$ for any $\alpha >0$, this implies that $u_x\rightarrow \infty$. It then follows that $p_x\rightarrow \infty$, which is a contradiction since $p_x\in [0,1]$. In conclusion, we obtain $p_0=0$, and by proportionality, $p_x=0$ for all x. Noticing that all states in $I_d$ are of the form $(x+d,x)$ for some $x\in {\mathbb {N}}_0$ completes the proof. $\square$

By means of Lemma 17 we can now make the following central statement.

Proposition 18

For each $(x,y)\in {\mathbb {N}}_0^2$, the two-point motion $\left( \varphi _{q}^n(x_0), \varphi _{q}^n(y_0)\right) _{n \in {\mathbb {N}}_0}$ of the standard birth-death chain reaches the thick diagonal ${\mathbb {D}}$ almost surely in finite time.

Proof

Let $(x_0,y_0)\in I_d$ for a given $d=x_0-y_0\ne 0$. According to Lemma 17, the two-point motion $\left( \varphi _{q}^n(x_0), \varphi _{q}^n(y_0)\right) _{n \in {\mathbb {N}}_0}$ almost surely escapes from $I_d$ in finite time. Given the transition probabilities (42)–(45), it can only end up in $I_{d-2}$ when $d\ge 1$ or in $I_{d+2}$ when $d\le -1$. This happens a finite number of times until the process reaches ${\mathbb {D}}=I_{-1}\cup I_0 \cup I_1$. $\square$

Figure 7 shows the stationary distribution of the one- and two-point motion for the standard birth-death chain. The state space of the two-point motion splits up into a transient area ${\mathbb {X}}^2\setminus {\mathbb {D}}$ and two communication classes $\Delta$ and ${\mathbb {D}}{\setminus } \Delta = I_{-1}\cup I_1$ [4] where $\Delta$ is the diagonal given in (19) and ${\mathbb {D}}$ is the thick diagonal defined in (41). The respective stationary distributions $\pi _{\Delta }$ and $\pi _{{\mathbb {D}}{\setminus } \Delta }$ are plotted in Fig. 7b, c.

Fig. 7

Stationary distributions of one- and two-point motion of the birth-death chain. a Stationary distribution $\rho$ of the one-point motion and b, c stationary distributions $\pi _{\Delta }$ and $\pi _{{\mathbb {D}}{\setminus } \Delta }$ of the two-point motion, respectively. All stationary distributions are computed as the solution of their eigenvector equation $\rho ^TP=\rho ^T$ for P the corresponding transition matrix. $\gamma _1 =10, \gamma _2 = 1$

We translate Proposition 18 into a characteristic feature of the random periodic orbit for the standard birth-death chain as given in the following statement.

Corollary 19

The attracting random periodic orbit for the standard birth-death chain $A_q=\{a_0(q),a_1(q)\}$ is such that $\vert a_0(q)- a_1(q)\vert =1$, ${\mathbb {P}}$-a.s.

Proof

Consider the sets

$$\begin{aligned} {\mathcal {Q}}_n:=\left\{ q \in {\mathcal {Q}}: \varphi _{\theta ^{-2k}q}^{2k}(0)=a_0(q), \ \varphi _{\theta ^{-2k}q}^{2k}(1)=a_1(q) \ \forall k\ge n \right\} . \end{aligned}$$

Since ${\mathcal {Q}}_n\subset {\mathcal {Q}}_{n+1}$ for all $n\in {\mathbb {N}}$, and since ${\mathbb {P}}(\bigcup _{n\in {\mathbb {N}}} {\mathcal {Q}}_n)=1$, for each $\varepsilon >0$ there is N such that for all $k\ge N$ sufficiently large

$$\begin{aligned} {\mathbb {P}}\left( \left| a_0(q) - a_1(q) \right| \ge 2 \right)&\le {\mathbb {P}}\left( \big \lbrace q\in {\mathcal {Q}}: \left| a_0(q) - a_1(q) \right| \ge 2 \big \rbrace \ \cap {\mathcal {Q}}_N \right) + \frac{\varepsilon }{2} \\&= {\mathbb {P}}\left( \left| \varphi _{\theta ^{-2k}q}^{2k}(0) - \varphi _{\theta ^{-2k}q}^{2k}(1) \right| \ge 2 \right) +\frac{\varepsilon }{2} \\&= {\mathbb {P}}\left( \left| \varphi _{q}^{2k}(0) - \varphi _{q}^{2k}(1) \right| \ge 2 \right) +\frac{\varepsilon }{2} < \varepsilon , \end{aligned}$$

for k sufficiently large, due to Proposition 18. Since $\varepsilon$ was arbitrarily small, the result follows. $\square$

Remark 6

In fact, we make the following observations for positive recurrent strict birth-death chains. Recall from Proposition 16 their invariant sample measures $\mu _{q} = \frac{1}{2} \delta _{a_0(q)} + \frac{1}{2} \delta _{a_1(q)}$ Then it is easy to see that ${{\bar{\mu }}}:= {\mathbb {E}} [ \mu _{q} \times \mu _{q}]$ is a stationary distribution for the two-point motion (see e.g. [4]). In our case, we have ${\bar{\mu }}(\Delta ) = \frac{1}{2}$ and ${\bar{\mu }}({\mathbb {X}}^2 \setminus \Delta ) = \frac{1}{2}$. In particular, we may write ${{\bar{\mu }}} = \frac{1}{2}{{\bar{\mu }}}_{\Delta } + \frac{1}{2} {{\bar{\mu }}}_{{\mathbb {X}}^2 {\setminus } \Delta }$, where

$$\begin{aligned} {{\bar{\mu }}}_{\Delta }:= \frac{1}{2} {\mathbb {E}} [ \delta _{a_0(q)} \times \delta _{a_0(q)}] + \frac{1}{2} {\mathbb {E}} [ \delta _{a_1(q)} \times \delta _{a_1(q)}] \end{aligned}$$

is invariant for the two-point motion starting with even distance between the coordinates of the initial state and

$$\begin{aligned} {{\bar{\mu }}}_{{\mathbb {X}}^2 \setminus \Delta }:= \frac{1}{2} {\mathbb {E}} [ \delta _{a_0(q)} \times \delta _{a_1(q)}] + \frac{1}{2} {\mathbb {E}} [ \delta _{a_1(q)} \times \delta _{a_0(q)}] \end{aligned}$$

is invariant for the two-point motion starting with odd distance between the coordinates of the initial state. Note from Fig. 7 that for the standard birth-death chain these measure take the form ${{\bar{\mu }}}_{\Delta } = \pi _{ \Delta }$ (b) and ${{\bar{\mu }}}_{{\mathbb {X}}^2 {\setminus } \Delta } = \pi _{{\mathbb {D}} {\setminus } \Delta }$ (c).

5.2 Two-Point Motion of the Schlögl Birth-Death Chain

As opposed to the standard birth-death chain Fig. 8 shows two realizations of the two-point motion for different initial states for the Schlögl birth-death chain. In this case, the thick diagonal ${\mathbb {D}}$ is no longer absorbing, since in some region the transition probabilites of the two-point motion are non-zero also for directions pointing away from the thick diagonal (see Fig. 9).

Fig. 8

Two-point motion $(\varphi _{q}^n(x_0), \varphi _{q}^n(y_0))_{n \in {\mathbb {N}}_0}$ of the Schlögl birth-death chain from Example 2 with different starting points (indicated by the red dots). The trajectory in (a) starts in $(x_0,y_0)=(4,10)$ with even distance $|x_0-y_0|=6$ and belongs to the realization shown in Fig. 4a. In (b), the trajectory starts in $(x_0,y_0)=(5,10)$ with odd distance $|x_0-y_0|=5$ and refers to Fig. 4b. In (a), the two-point motion quickly ends up on the diagonal, in contrast to (b) where apparently the absorbing region is not straightforward to identify. The rate constants set to $\gamma _1=6$, $\gamma _2=3.5, \gamma _3=0.4, \gamma _4=0.0105$ (Color figure online)

Fig. 9

Possible transitions to the top left (red) and the bottom right (blue) for the Schlögl birth-death chain. Transitions parallel to the diagonal are not displayed (Color figure online)

The stationary distribution of the Schlögl birth-death chain is depicted in Fig. 10. While Proposition 7 and Theorem 15 also guarantee partial synchronization for the Schlögl chain and the existence of an attracting random 2-periodic orbit, respectively, numerical explorations (cf. Fig. 8b) suggest that the distance of these two random points will not be one but larger; maybe even depending on the random realization. We emphasize that this difference to the standard birth-death chain in terms of the non-synchronizing trajectories reflects the distinction between monostability (the thick diagonal in the two-point motion is absorbing) and bistability (the thick diagonal in the two-point motion has repelling parts) as seen in the large-volume limiting ODEs (cf. Fig. 1). This fact is also mirrored by the respective stationary distributions of the one- and two-point motion, as illustrated in Figs. 7 and 10 (cf. also Remark 6).

A detailed exploration of the absorbing region in the two-point motion for the Schlögl chain is much more involved than in the standard birth-death chain and will be left for future work.

Fig. 10

Stationary distributions of one- and two-point motion of the Schlögl birth-death chain. a Stationary distribution $\rho$ of the one-point motion and b, c stationary distributions $\pi _{\Delta }$ and $\pi _{{\mathbb {S}}}$ of the two-point motion, respectively. Here, the index ${\mathbb {S}}\subset {\mathbb {X}}^2 {\setminus } \Delta$ refers to the support of the stationary distribution for the two-point motion starting with odd distance between the two coordinates $x_0,y_0$ of the initial state. d the same as (c) only on a logarithmic scale. $\gamma _1=6$, $\gamma _2=3.5, \gamma _3=0.4, \gamma _4=0.0105$

6 Conclusion

We have introduced the phenomenon of (partial) time-shifted synchronization for reaction jump processes, using their description via the augmented and embedded Markov chain whose properties as a random dynamical system can be specified through the structure of the corresponding random attractor. As a first example we have given a full proof of partial synchronization for birth-death processes whose embedded chains are recurrent, finding also that the random attractor of the embedded chain is a random periodic orbit of period 2 in the case they are positive recurrent. In addition, we have shown that the two random points forming the attractor always have distance 1 for the standard birth-death chain, implied by the fact that the respective two-point motion is absorbed by the thick diagonal ${\mathbb {D}}$. In contrast, we have demonstrated numerically that for examples such as the Schlögl birth-death chain the distance between the two random points may exhibit more complicated behaviour, corresponding with a stationary distribution supported beyond ${\mathbb {D}}$. A more detailed analyis of this phenomenon is left for future work.

Depending on the dimensionality of the system, i.e. the number of different species, or for other types of reaction rates, e.g. Michaelis–Menten, we expect various forms of (partial) synchronization and random periodic behavior for general reaction systems and leave it as a research direction for the future to work towards a categorization of such processes and their corresponding chains in terms of random dynamical systems theory. The main goal of this paper has been to relate synchronization of the embedded Markov chain to time-shifted synchronization of the trajectories of the continuous-time reaction jump process as given by Gillespie’s algorithm and to give a first complete and rigorous description of a simple class of examples, namely those with a birth-death chain as their embedded Markov chain. The strategy of understanding the two-point motion, establishing the weak attractor via a unique stationary distribution and then specifying the structure of the attractor and the statistical equilibrium supported there, may well be generalizable. In particular, one may consider time-shifted synchronization for general Markov jump processes, not necessarily given by reaction networks, via the RDS description of the related space-time Markov chains. Moreover, an intriguing point of more detailed investigation will concern the quantification and statistics of the delay times found for time-shifted synchronization.

Furthermore, our work has brought up additional questions that remain open, to our knowledge. Can one find an example coming from a chemical reaction network with no (partial) synchronization at all, i.e. where each synchronization class is a singleton? May one describe bifurcations of the attractor, for example in an easy model such as Schlögl’s, via variation of the parameters? How are attractors of the described processes related to the attractors of the corresponding volume-scaled systems, i.e. the Langevin SDEs or the reaction rate ODEs? These approximate systems associate noise with real time rather than with jump times of the underlying RJP and therefore might synchronize without time-shift. Assuming that noise is a time-dependent factor, the latter situation may seem more realistic because there raise doubts about whether the same noise affects two trajectories when the time-shifted simulation commences, even though different time intervals have elapsed. In this sense, the concept of time-shift might not be observable in real-world applications, while still being a crucial feature of statistically exact numerical simulations of the RJP. In addition, one may wonder if there could be an actual cocycle, constructed over an appropriate model of the driving noise, which precisely matches the statistics of the continuous-time Markov jump process. This leads to the interesting open question if there is a general way of finding RDS versions of jump processes.

Additionally, from the RDS point of view it will also be intriguing to give general criteria for weak attractors being (strong) pullback attractors in discrete state spaces. In summary, we see this work as a first step towards a deeper structural understanding of reaction jump processes via RDS theory and, conversely, a motivation for a broader understanding of random attractors within the dichotomy between the discrete and the continuous.