On the Validity of Linear Response Theory in High-Dimensional Deterministic Dynamical Systems

Abstract

This theoretical work considers the following conundrum: linear response theory is successfully used by scientists in numerous fields, but mathematicians have shown that typical low-dimensional dynamical systems violate the theory’s assumptions. Here we provide a proof of concept for the validity of linear response theory in high-dimensional deterministic systems for large-scale observables. We introduce an exemplary model in which observables of resolved degrees of freedom are weakly coupled to a large, inhomogeneous collection of unresolved chaotic degrees of freedom. By employing statistical limit laws we give conditions under which such systems obey linear response theory even if all the degrees of freedom individually violate linear response. We corroborate our result with numerical simulations.

Appendices

Appendix A: Model Reduction for Chaotic Microscopic Sub-systems

This appendix describes how to compute the statistics of the stochastic limiting system Eq. (5) for $\gamma =\tfrac{1}{2}$, which we recall here

$$\begin{aligned} {\mathcal {Q}}_{n+1} = (A_0+A_1\zeta _n)\,{\mathcal {Q}}_n(1-{\mathcal {Q}}_n), \end{aligned}$$

(8)

of the deterministic limiting system Eq. (6) for $\gamma =1$, which we recall here

$$\begin{aligned} {\mathcal {Q}}_{n+1} = A\,{\mathcal {Q}}_n(1-{\mathcal {Q}}_n) \end{aligned}$$

(9)

with $A = A_0 + \langle {\mathbb {E}}[\phi ] \rangle A_1$, and of the stochastic finite-size system

$$\begin{aligned} {\mathcal {Q}}_{n+1} = A\,{\mathcal {Q}}_n(1-{\mathcal {Q}}_n) \end{aligned}$$

(10)

with $A = A_0 +Z_nA_1$ where $Z_n$ is given by Eq. (7), which is recalled here as

$$\begin{aligned} Z_n = \langle {\mathbb {E}}[\phi ] \rangle + \frac{1}{\sqrt{M}}\eta + \frac{1}{\sqrt{M}}\zeta _n. \end{aligned}$$

(11)

The random variable $\eta$ accounts for the random variation in the selection of the parameters $a^{(j)}$ and the random process $\zeta _n$ accounts for the dynamics of the microscopic variables. However, we remark that setting $\eta \equiv 0$, i.e. replacing it with its expectation, gives a remarkably good approximation of the invariant measure, at least in the system we consider (not shown).

In order to simulate these systems we need to estimate $\langle {\mathbb {E}}[\phi ] \rangle$ and, for the stochastic systems, also $R(m) = \langle {\mathbb {E}}[\phi _0 \phi _m] \rangle - ({\mathbb {E}}[\phi ])^2, m \in \mathbb {N}$. We describe first how we estimate these parameters from Monte Carlo simulations of the logistic map, and then describe how we sample the stochastic process $\zeta _n$ with the covariance parameters given by R(m).

1.1 Appendix A.1: Estimating Parameters

We need to estimate the expectation values for K perturbation sizes ${\varepsilon }_i$ with $i=1,\ldots ,K$. Since we set here $a_1=1$ for all microscopic variables, at each ${\varepsilon }_i$ we write the averages over the microscopic dynamics as

$$\begin{aligned} \langle {\mathbb {E}}^{\varepsilon _i} [\phi ]\rangle = \int _{{\mathbb {R}}} {\mathbb {E}}^{\alpha } [\phi (\cdot ,\alpha )] \, \nu _{0}(\alpha -\varepsilon _i) d\alpha \end{aligned}$$

(12)

and

$$\begin{aligned} \langle {\mathbb {E}}[\phi _0 \phi _m] \rangle _{\varepsilon _i} = \int _{{\mathbb {R}}} {\mathbb {E}}^{\alpha } [\phi (x_0,\alpha ) \phi (x_m,\alpha )] \, \nu _0(\alpha -\varepsilon _i) d\alpha \end{aligned}$$

(13)

for $i = 1, \ldots , K$ and $m = 1, \ldots , \infty$, where $\nu _{0}$ is the density function of the logistic map parameters and is chosen here as the raised cosine distribution

$$\begin{aligned} \nu _{0}(a) = \mathbf {1}_{[3.8,3.9]} \frac{1}{0.1} \left( 1+\cos \left( \frac{a-3.85}{0.05}\pi \right) \right) . \end{aligned}$$

From now on it is understood that all observables, expectations and so on are for a fixed parameter $\alpha$: we therefore drop the $\alpha$ and (j) superscripts for ease of exposition.

We use a trapezoidal rule to estimate the integrals in (12) and (13), using a grid of 30, 001 values of the logistic map parameters $\alpha$ evenly spaced on [3.7, 4.0] (to allow for the support of $\nu _{0}$ as well as the range of the perturbation). This is used for each $\varepsilon _i$.

The expectations (12) and (13) can be entirely determined by simulations of a standard logistic map without coupling to the expanding r-dynamics: Denote by $\varphi _n = \phi (x_n,a_0+{\varepsilon }_i)$ such that $x_{n+1}= (a_0+{\varepsilon }_i) x_n(1-x_n)$ with $x_0 = q_0$. The logistic dynamics of the q are augmented by r-dynamics so that at any time step the q will with equal probability either advance according to the logistic map or remain unchanged. The invariant measure of q is therefore identical to the one supported by a logistic map with the same parameter $\alpha$; hence ${\mathbb {E}}[\phi ] = {\mathbb {E}}[\varphi ]$. To estimate the averages of the auto-correlations (13) we define N(m) as the number of evolution steps of the q-dynamics up to physical time m which were done according to the logistic map (i.e. discarding all those instances when the r-dynamics forces q not to vary). Note that N(m) has a binomial distribution $N(m)\sim {\mathrm {B}}(m,\tfrac{1}{2})$. Hence by definition we have

$$\begin{aligned} \phi (q_m) = \varphi _{N(m)}, \end{aligned}$$

and we can write

$$\begin{aligned} {\mathbb {E}}[\phi _0 \phi _m]&= {\mathbb {E}}[\varphi _0 \varphi _{N(m)}] \\&= \sum _{i=0}^m 2^{-i} {m \atopwithdelims ()i} {\mathbb {E}}[\varphi _0 \varphi _i]. \end{aligned}$$

For regular values of $\alpha$, when the logistic map $x_n$ with parameter $\alpha$ has a stable periodic orbit, calculating the stable periodic orbit allows for an accurate evaluation of the expectation. We use the database of periodic windows given in [54] to identify regular points and stable periodic orbits.

For chaotic values of $\alpha$ we estimate expectations and lag-correlations of the logistic map with parameter $\alpha$ via Monte-Carlo simulation of the logistic map $x_{n}$, using 10 separate initialisations with 399168 time steps each. This number of time steps was chosen as it has a large number of prime factors, and therefore will give more accurate estimates for short periodic windows outside the database, or for chaotic values where the acim has multiple connected components (i.e., f is not mixing but $f^p$ is for some $p>1$).

1.2 Appendix A.2: Sampling the Stochastic Process $\varvec{\zeta _n}$

The limiting process $\zeta _n$ is a stationary Gaussian process given by lag-covariance function R(m). Assuming sufficiently fast decay of the lag-covariance function, we can write this process as a moving-average process of infinite order

$$\begin{aligned} \zeta _n = \sum _{m=0}^\infty \beta _m X_{n-m} \end{aligned}$$

with a deterministic sequence $(\beta _m)_{m\in \mathbb {N}}\in \ell _2$ and i.i.d. standard normal random variables $X_n$.

The moving average coefficients $\beta _m$ and the covariance function $R_m$ are related by

$$\begin{aligned} R(m) = \sum _{k=0}^\infty \beta _k \beta _{m+k}. \end{aligned}$$

The coefficients can be extracted from the covariance function via the generating functions

$$\begin{aligned} \mathcal {B}(z) := \sum _{m=0}^\infty \beta _m z^m \end{aligned}$$

and

$$\begin{aligned} \mathcal {R}(z) := \sum _{m=-\infty }^\infty R(|m|) z^m, \end{aligned}$$

for which the relation $\mathcal {R}(z) = \mathcal {B}(z) \mathcal {B}(z^{-1})$ holds. If we restrict to the complex unit circle, setting $z = e^{i\theta }$, we find that $\mathcal {R}(e^{i\theta }) = \mathcal {B}(e^{i\theta })\mathcal {B}(e^{-i\theta }) = |\mathcal {B}(e^{i\theta })|^2$ since $\beta _m\in \mathbb {R}$. Assuming that $\mathcal {R}(e^{i\theta }) \ne 0$, we have that

$$\begin{aligned} \frac{1}{2} \log \mathcal {R}(e^{i\theta }) = \mathfrak {R}\log \mathcal {B}(e^{i\theta }), \end{aligned}$$

and hence, we can write, using that the $\beta _m$ are real,

$$\begin{aligned} \log \mathcal {B}(z) = \sum _{m=0}^\infty b_m z^m \end{aligned}$$

with $b_m \in {\mathbb {R}}$. The $b_m$ may be calculated via Fourier cosine transform using that

$$\begin{aligned} \frac{1}{2} \log \mathcal {R}(e^{i\theta }) = \sum _{m=0}^\infty b_m \cos m\theta . \end{aligned}$$

The $b_m$ coefficients allow one to evaluate $\mathcal {B}(e^{i\theta })$, from which the moving average coefficients $\beta _m$ are obtained via an additional Fourier transform.

Appendix B: Testing for Linear Response in Finite Time Series

We summarize here briefly the quantitative goodness-of-fit test for the detectability of linear response introduced in [37]. The test quantifies the statistical significance of an observed linear response in time series of finite size.

Given a family of chaotic maps $f_\varepsilon$ that may or may not obey linear response, we test for linear response at some reference state with parameter $\varepsilon =\varepsilon _0$. Hence we seek to examine the linear dependency of the response of a bounded and continuous observable

$$\begin{aligned} \delta {\varPsi }= {\mathbb {E}}^{\varepsilon }[ {\varPsi }] - {\mathbb {E}}^{{\varepsilon }_0} [{\varPsi }] \end{aligned}$$

(14)

in terms of the perturbation size ${\varepsilon }$. To test for linearity we consider $K > 2$ different values of the perturbation parameter $\varepsilon _1,\cdots ,\varepsilon _K$, and sample N consecutive values from the perturbed maps yielding the time series $x^i_n=f_{\varepsilon _{i}}(x^i_{n-1})$ for each $i=1,\cdots , K$ and $n=1,\cdots , N$. The initial conditions $x^i_0$ are distributed according to the physical measure associated with $f_{\varepsilon _{i}}$. The lengths of the time series N is chosen that for each $i=1,\cdots ,K$ the corresponding autocorrelation function has sufficiently decayed, i.e. we choose $N_i \gg \tau _{{\varPsi },\varepsilon _i}$, where $\tau _{{\varPsi },\varepsilon _i}$ is the 1 / e-folding time of the autocorrelation function of ${\varPsi }$ under the dynamics $f_{\varepsilon _i}$. For simplicity, we choose $N_i=N$ for all i in the following.

It is well known that for a large class of chaotic dynamical systems, the sample averages of the observations

$$\begin{aligned} \bar{{\varPsi }}_i=\frac{1}{N}\sum _{n=1}^{N}{\varPsi }\left( x^i_n\right) \end{aligned}$$

(15)

obey the central limit theorem and are distributed asymptotically as $\mathcal {N}\left( \left\langle {\varPsi } \right\rangle _{\varepsilon _{i}}, \sigma _i^2/N\right)$ [55, 56] with

$$\begin{aligned} \bar{{\varPsi }}_i \approx {\mathbb {E}}^{{\varepsilon }_i}[{\varPsi }] + \frac{\sigma _i}{\sqrt{N}} \xi _i \; , \end{aligned}$$

(16)

for $i = 1,\cdots ,K$ and iid noise $\xi _i \sim \mathcal {N}(0,I)$. The variances $\sigma _i^2$ are given by the Green–Kubo formula as an infinite sum of lag-correlations of $f_{\varepsilon _{i}}$ as

$$\begin{aligned} \sigma _i^2 = C_0({\varPsi },{\varPsi }) + 2 \sum _{j=1}^{\infty } C_j({\varPsi },{\varPsi }), \end{aligned}$$

(17)

where the correlation function $C_n$ between two observables ${\varPsi }$ and ${\varOmega }$ is defined as

The variances can be efficiently estimated numerically as a Monte-Carlo estimate from (16) for large N. The results in this work were obtained with $N=40\times 10^6$.

In the case the dynamical system has linear response and provided the perturbations $\delta \varepsilon _i=\varepsilon _i-\varepsilon _0$ are sufficiently small, the following statistical model holds for $\bar{{\varPsi }}_i$ (with $o(\delta \varepsilon _i)$ error)

$$\begin{aligned} \bar{{\varPsi }}_i = \alpha _0 + \alpha _1 \, \delta \varepsilon _i + \frac{\sigma _i}{\sqrt{N}} \xi _i \, , \end{aligned}$$

(18)

with $\alpha _0={\mathbb {E}}^{{\varepsilon }_0}[{\varPsi }]$ and $\alpha _1= {\mathbb {E}}^{{\varepsilon }_0}[{\varPsi }]^\prime$ for the unperturbed reference state with $\varepsilon =\varepsilon _0$. It is pertinent to mention that the $\xi _i$ are independent since the samples from each perturbed system are generated independently.

The parameters $\alpha _0$ and $\alpha _1$ of the model (18) can be determined from time series by means of a weighted least squares fit and we obtain

$$\begin{aligned} \left( \begin{array}{c} {\hat{\alpha }}_0\\ {\hat{\alpha }}_1 \end{array}\right) = (D^T D)^{- 1} D^T Y \end{aligned}$$

with the design matrix

$$\begin{aligned} D = \left( \begin{array}{cc} 1 / \sigma _1 &{}\quad \delta \varepsilon _1 / \sigma _1\\ \vdots &{}\quad \vdots \\ 1 / \sigma _K &{}\quad \delta \varepsilon _K / \sigma _K \end{array}\right) \, , \end{aligned}$$

and the vector of scaled observations

$$\begin{aligned} Y = \left( \begin{array}{c} \bar{{\varPsi }}_1 / \sigma _1\\ \vdots \\ \bar{{\varPsi }}_K / \sigma _K \end{array}\right) \, . \end{aligned}$$

Testing for validity of linear response then amounts to testing whether the actual observations could have been generated from the linear model (18) with normally distributed errors $\xi _i\sim {\mathcal {N}}(0,I)$. To do so we choose a Pearson $\chi ^2$-test to test the goodness-of-fit with statistics

$$\begin{aligned} \chi ^2&= N\, \sum _{i=1}^K \left( Y_i - \frac{1}{\sigma _i}\left( {\hat{\alpha }}_0+{\hat{\alpha }}_1\varepsilon _i\right) \right) ^2 \nonumber \\&= N\, Y^T(I-H)Y, \end{aligned}$$

(19)

where the idempotent matrix

$$\begin{aligned} H = D (D^T D)^{-1} D^T \end{aligned}$$

maps scaled observations Y to their linear fits, i.e. $HY = D({\hat{\alpha }}_0\;\;{\hat{\alpha }}_1)^T$ [57].

If the response of the underlying dynamical system is linear, $\chi ^2$ has a $\chi ^2$-distribution with $K-2$ degrees of freedom and expectation value ${\mathbb {E}}\chi ^2_{K-2} = K-2$. Hence a measure for the breakdown of linear response can be quantified as the difference between the $\chi ^2$ test statistic for the scaled observations $Y_i= \bar{{\varPsi }}_i / \sigma _i$ and the expectation of the test statistic under the null hypothesis of linear response

$$\begin{aligned} \mathfrak {q}= \frac{1}{N}\left( \chi ^2 -{\mathbb {E}}\chi ^2_{K-2} \right) . \end{aligned}$$

(20)

The central limit theorem (16) holds independent of the existence of linear response and can be used to obtain expressions for the mean and variance of the breakdown parameter. Defining W as the vector with components $W_i={\mathbb {E}}^{{\varepsilon }_i}[{\varPsi }]/\sigma _i$, the mean is calculated as

$$\begin{aligned} {\mathbb {E}}\mathfrak {q}&= \frac{1}{N}\left( {\mathbb {E}}\chi ^2 -{\mathbb {E}}\chi ^2_{K-2}\right) \nonumber \\&= {\mathbb {E}}\bigg ((W + \frac{1}{\sqrt{N}}{\xi })^T(I-H)(W + \frac{1}{\sqrt{N}}{\xi }) - \frac{1}{N}{\mathbb {E}}\chi ^2_{K-2}\bigg ) \nonumber \\&= \Vert W-HW\Vert ^2, \end{aligned}$$

(21)

where we used that H is idempotent. Hence $\mathfrak {q}$ is a random variable whose expected value measures the difference between the actual response ${\mathbb {E}}^{{\varepsilon }_i}[{\varPsi }]$ and an assumed linear response $\alpha _0+\alpha _1\varepsilon _i$ as calculated via least square regression. The mean of the breakdown parameter ${\mathbb {E}}\mathfrak {q}$ is non-negative and is zero only for $W = HW$, i.e. if the observations stem from a dynamical system obeying linear response. The variance of the breakdown parameter $\mathfrak {q}$ is calculated as

$$\begin{aligned} {\mathbb {V}}\mathfrak {q}&= {\mathbb {E}}\bigg ((W + \frac{1}{\sqrt{N}}{\xi })^T(I-H)(W + \frac{1}{\sqrt{N}}{\xi }) -\frac{K-2}{N} - {\mathbb {E}}\mathfrak {q}\bigg )^2 \\&= \frac{1}{N} {\mathbb {E}}\left( {\xi }^T (I-H) (2W + \frac{1}{\sqrt{N}}{\xi }) - \frac{K-2}{\sqrt{N}} \right) ^2. \end{aligned}$$

Note that ${\mathbb {V}}\mathfrak {q}\rightarrow 0$ for $N\rightarrow \infty$, and hence $\mathfrak {q}$ is a consistent estimator for the mismatch ${\mathbb {E}}\mathfrak {q}= \Vert W-HW\Vert ^2$. In numerical experiments it is practical to consider Monte-Carlo estimates of the mismatch over realizations $\mathfrak {q}_j$ differing in their initial condition and set

$$\begin{aligned} {\hat{\mathfrak {q}}}=\frac{1}{K} \sum _{j=1}^K\mathfrak {q}_j\, . \end{aligned}$$

(22)

To make statements about the statistical significance of whether an observed time series of length N is classified as obeying linear response or not, we introduce a p-value testing the null hypothesis of linear response. Let us consider the case when a dynamical system does not obey linear response, i.e. when ${\mathbb {E}}\mathfrak {q}\ne 0$. Using Chebyshev’s inequality we have that for all $b < N{\mathbb {E}}\mathfrak {q}$,

$$\begin{aligned} P(N\mathfrak {q}< b)&\le P(|\mathfrak {q}-{\mathbb {E}}\mathfrak {q}| > {\mathbb {E}}\mathfrak {q}- b/N)\\&\le \frac{{\mathbb {V}}(\mathfrak {q})}{({\mathbb {E}}\mathfrak {q}- b/N)^2}\; . \end{aligned}$$

Since, as we have shown above, ${\mathbb {V}}\mathfrak {q}\rightarrow 0$ as $N\rightarrow \infty$, we conclude that $N \mathfrak {q}\rightarrow \infty$ in probability as $N\rightarrow \infty$. Hence, if F is the cumulative distribution function of the $\chi ^2_{K-2}$ distribution, the p-value obtained using the $\chi ^2$- test,

$$\begin{aligned} p = 1 - F(\chi ^2) = 1- F(K-2 +N\mathfrak {q}), \end{aligned}$$

(23)

converges quickly in probability to zero as $N\rightarrow \infty$ [57]. This implies that the probability of falsely accepting the null hypothesis of linear response at any significance level can be made arbitrarily small for sufficiently large data length N.

For completeness (although not used in this work) we show that one can define a threshold value $\mathfrak {q}_\alpha$ for the observed random variable ${\hat{\mathfrak {q}}}$ such that if ${\hat{\mathfrak {q}}} >\mathfrak {q}_\alpha$ the null hypothesis of linear response is rejected with significance level $\alpha$ (i.e. with probability $1-\alpha$); given a specified significance level $\alpha$ the threshold value can be defined as

$$\begin{aligned} \mathfrak {q}_\alpha = \frac{1}{N} \left( F^{-1}(1-\alpha )-(K-2)\right) \, . \end{aligned}$$

(24)

It is clear from this that the detectability of breakdown of linear response crucially depends on the amount of available data. As $N\rightarrow \infty$, a breakdown will always become detectable at any specified significance level $\alpha$. Conversely, if the mismatch ${\mathbb {E}}\mathfrak {q}$ between the true response of the dynamical system and the linear response is too small and there is an insufficient amount of data available, the actual response will be swamped by the sampling noise, and one will not be able to detect the breakdown of linear response with a reasonable significance level.

Fig. 9

Cubic response of an observable ${\varPsi }(Q)=Q$ for the deterministic system (1)–(3) for $\gamma =\tfrac{1}{2}$. a $M=16$. b $M=1024$. c $M=32768$. d Stochastic limit system (5). All experiments used a time series of length $N=2\times 10^5$. The error bars are estimated from $K=200$ realizations differing in the initial conditions. We used $A_0=3.91$, $A_1=0.05$

It is possible to extend this test to probe higher order response. To test for $\ell$th order response we add terms $\sum _{j=2}^\ell \alpha _j \delta {\varepsilon }_i^j$ to our statistical model (18) and then employ higher-order regression (i.e. augmenting the design matrix D). In Fig. 9 we show results for the same numerical simulations as in Fig. 5 for $\gamma =\tfrac{1}{2}$, but now showing cubic response rather than linear response. We recall that we can expect cubic response due to the distribution density of the logistic map parameters of the microscopic sub-system being three times continuously differentiable. As for linear response, the null-hypothesis of cubic response can be rejected for small values of the system size M but cannot be rejected for sufficiently large values of M. For $M=16$ the test yields a p-value of $5.3 \times 10^{-04}$. For $M=1024$ the p-value is 0.52 consistent with cubic response.

For more details on the test the interested reader is referred to [37].