Revising and Extending the Linear Response Theory for Statistical Mechanical Systems: Evaluating Observables as Predictors and Predictands

Linear response theory has developed into a formidable set of tools for studying the forced behaviour of a large variety of systems - including out of equilibrium ones. In this paper we provide a new angle on the problem, by studying under which conditions it is possible to perform predictions on the response of a given observable of a system to perturbations, using one or more other observables of the same system as predictors, and thus bypassing the need to know all the details of the acting forcing. Thus, we break the rigid separation between forcing and response, which is key in linear response theory, and revisit the concept of causality. As a result, the surrogate Green functions one constructs for predicting the response of the observable of interest may have support that is not necessarily limited to the nonnegative time axis. This implies that not all observables are equally good as predictands when a given forcing is applied, as result of the properties of their corresponding susceptibility. In particular, problems emerge from the presence of complex zeros. We derive general explicit formulas that, in absence of such pathologies, allow one to reconstruct the response of an observable of interest to N independent forcings by using as predictors N other observables. We provide a thorough test of the theory and of the possible pathologies by using numerical simulations of the paradigmatic Lorenz '96 model. Our results are potentially relevant for problems like the reconstruction of data from proxy signals, like in the case of paleoclimate, and, in general, the analysis of signals and feedbacks in complex systems where our knowledge on the system is limited, as in neurosciences. Our technique might also be useful for reconstructing the response to forcings of a spatially extended system in a given location by looking at the response in a separate location.


Introduction
Response Theory provides a powerful array of methods for predicting how the statistical properties of a system change when one or more of its parameters are changed or when new forcings are added to the system. The usefulness of the theory comes from the fact that the response operators can be constructed by considering suitably defined statistical properties of the unperturbed system. This feature is particularly prominent in the case of systems with N degrees of freedom possessing invariant measure in the unperturbed state that is absolutely continuous with respect to the corresponding Lebesgue measure in N dimension. This is the case of deterministic equilibrium physical systems, whose evolution is governed by Hamiltonian dynamics and, consequently, the volume of the phase space is preserved, and in stochastically perturbed dynamical systems. In the case of full invariant measure, the Fluctuation-Dissipation theorem allows one to construct response operators from a suitably defined correlation functions in the unperturbed state [1,2,3].
Things are more difficult in the case of deterministic chaotic systems whose dynamics is governed by equations of motions featuring, on the average, a contraction of the phase space. In such systems, which can be thought as providing good models of non-equilibrium statistical mechanical systems, the fluctuation-dissipation theorem cannot be readily applied because there is no correspondence between forced and free fluctuations. This was clarified by Ruelle, who also showed that, regardless of such a complication, it is possible to construct a rigorous response theory for Axiom A dynamical systems [4,5,6]. The response theory has been later recovered using methods based on the transfer operator formalism [7], and extended to more general dynamical systems than Axiom A ones [8]. We remark that Axiom A dynamical systems can be considered of general physical relevance despite their rather specific mathematical properties than to the chaotic hypothesis 9 , and the predictions of linear response theory have been verified numerically for systems that are not strictly Axiom A [10,11,12,13].
The lack of correspondence between forced and free fluctuations makes a careful study of the response theory extremely relevant. In the case of climate, response theory has been shown to be a very useful tool for predicting climate change [14,15] and for critically assessing geoengineering practices [16]. The prediction is performed by constructing response operators for each observable of interest (for one or more forcings) through a set of suitably planned test experiments, and then use such operators for predicting the response to a different time pattern of the same forcing. This approach is promising in the case one has full control of the system under investigation, thus acting is a sort of real or virtual lab, where forcing can be changed at pleasure, experiments can be repeated, and suitable observables can be measured with a good degree of precision. The direct construction of the response operator faces the extremely hard computational challenge of having to deal with contributions coming from the effects of the perturbations on the stable and on the unstable manifold of the unperturbed system, which have entirely different properties in terms of convergence [17].
Straightforward applications of the fluctuation-dissipation theorem in a chaotic system like the climate have had various degrees of success (see e.g. [18,19]). The problems mostly come from the fact that if we try to reconstruct the forced response from the free fluctuations we commit an error (due to the properties of the stable manifold of the system) which is hard to bound and which depends in a very nontrivial way on the specific choice of the forcing and of the observable of interest. In a recent paradigmatic analysis performed on a simplified climate model, it has been shown that whereas for a certain forcing the fluctuation-dissipation relation leads to a fairly good estimate of the response operator, for another choice of the forcing its skill is negligible. In this second circumstance, the response of the perturbed system can be interpreted as featuring climatic surprises, in the form of weather patterns that are entirely absent in the unperturbed system [20].
Other authors have instead taken the point of view of stochastic dynamics: in this case, linear response theory can be shown to hold under very weak assumptions, as the noise takes care of regularizing a lot of the pathologies that can be encountered as a result of deterministic dynamics [21]. This has been taken as starting point to further derive the validity of linear response theory for high-dimensional deterministic chaotic systems, by considering stochastic limits for macroscopic observables [22]. The latter point of view seems quite relevant in many complex systems where one, instead of looking at the details of the microscopic dynamics, is de facto interested in studying coarse grained variables and effective, coarse grained dynamics, which is stochastic as a result of the action of the hidden, subscale variables [23,24,25,26,27]. See also the rather comprehensive and informative review given in [28].
In this paper we want to explore a rather different direction, namely we would like to relate the response of different observables of the system to the applied perturbation. The goal is to understand to what extent it is possible to use one or more forced signals as surrogates for the actual external forcing, and then develop a response theory for predicting the response for an observable of interest based on such surrogates. The reason for looking at the response from this angle lies in the fact that whereas in the case of a direct problem we are usually knowledgeable of the specific properties of the system (including its evolution equations) and of the nature and spatio-temporal pattern of the applied forcing, this is not true when considering an inverse problem where we need to interpret data and have a possibly moderate control or knowledge of the time pattern of the applied forcing, but, in fact, we might access to multiple observables on top of the one we are specifically interested into.
The former situation is quite relevant in the case we are able to prepare and observe a system in a lab, and are able to operate on it by changing some of its parameters and the applied forcings. This is the classical Gedankenexperiment for which the usual response theory is relevant. The latter situation applies when our control and knowledge of the system is inherently limited, i.e. we can have access to information to a (possibly small) subset of the degrees of freedom of the system. Obvious examples of such conditions are found in neurosciences and in geosciences.
Let's expand on the specific case of geosciences, as it has provided the initial inspiration for this work. While, mathematically, one can often deal with a generic, well-behaved, observable, in natural sciences some observables are definitely more meaningful than other ones, as in the obvious case of energy when considering physical systems. Additionally, in the case of geosciences, where collecting observations and modelling the dynamics of systems is far from being trivial, our ability to observe and/or to suitably model some observables -possibly of great relevance -can be substantially more limited than for other observables. These observables can be quite different in terms of characteristic temporal (and possibly spatial) scales of variability, as when comparing atmospheric vs. oceanic fields. Such issues are particularly relevant when considering the problem of paleoclimatic reconstruction performed by using proxy data. In this case, one has access to time series (with a varying degree of uncertainty) of measurements of physical and chemical properties of natural recorders of climate variability and change (e.g. tree rings, sediments, air bubble inside ice cores, etc.). The goal is to find an approximate functional relationship between what we can actually measure -the proxy data -and what we are interested into -climate variables such as temperature, humidity, etc. [29].
Additionally, finding functional relationships between the response of different observable to climate change ('emergent constraints") and testing them against observations has been proposed as a way to assess the quality of model and reduce uncertainty on the climatic observables we have harder time to model accurately [30,31,32].
Moreover, our aim is to provide a basis for improving and extending the classical theory of feedbacks, which assumes instantaneous relations between the response of different climatic variables to forcings or, in the opposite limit, assumes that slow variables unidirectionally modulate the response of fast ones [33]. In some cases, the effect of an external forcing on a specific variable of interest is studied by looking at how it is mediated by the response of other variables and, in turn, on their influence of the variable of interest. Let's give an example: usually, in the climate literature it is said that, in global warming conditions, the increase of global precipitations is caused by the increase in the average atmospheric temperatures and, as a resultroughly speaking -via the Clausius-Clapeyron relation -of the increased ability of the atmosphere to retain water vapour [34]. Instead, both the increase of the atmospheric temperature and of the global precipitation are caused by the increase of CO 2 concentration, which is the real forcing to the system. Nonetheless, the statement makes indeed physical sense, as a result of the time scales of the processes. Another more specific example: it is generally thought that under global warming conditions one might expect stronger (but less frequent for a complex set of reasons) hurricanes mostly as a result of higher surface temperature of the ocean [35]. In order to test such a hypothesis -and isolating the effect of the surface temperature change on the statistical properties of hurricanes -a typical (yet now somewhat outdated) strategy relies on performing simulations where, instead of using coupled global climate models with prescribed forcing due to changes in the CO 2 concentration, one uses atmospheric general circulation models having as boundary conditions prescribed warmer sea surface temperature in the desired region [36].
We will try to provide a formal treatment of these problems based upon the linear response theory as discussed above, and we will try to define under which conditions the choice of an observable is suboptimal (or not so) in terms of the information it conveys. Can we distinguish between good observables, usable for predicting the response of other observables, and less useful ones? How do the time scales of the response of each observable to the applied forcing impact such property of bearing potential predictability? This entails defining, within a system, a (maybe approximate) chain of causality associated to the forcing between different observables, or, in the case of spatially extended systems, between different regions of the system.
As mentioned above, response theory has been successfully used in the context of climate problems and we hope to provide an additional contribution in this direction, even if our treatment is more general and applications can be foreseen in other areas of science as well.
The paper is organised as follows. In Section 2 we provide a rapid summary of the formalism of response theory, presenting it in the context of deterministic dynamical systems. We will always assume that the chaotic hypothesis [9], so that we can assume, de facto, that the system is close enough (for our purposes) to being uniformly hyperbolic. Nonetheless, all the subsequent results and considerations apply a fortiori in the case of rather general stochastic systems, where the conditions for the applicability of response theory are quite relaxed, as mentioned above. Section 3 provides the basic example detailing how one can use the response of one generic observable to forcing as a predictor of the response of another generic observable, partly bypassing the need to know all the details of the forcing. Explicit formulas are derived by using complex analysis and linear algebra, under assumptions -discussed later -which are needed for performing such operations. Section 4 provides the generalisation of what shown in Section 4 to the case where N forcings are applied to the system. In this case, one needs the knowledge of N generic observables to be able to predict the response of an observable of interest. Section 5 details the fundamental limitations of what discussed before. Not in all cases the operation is practically possible, as the functional relations one finds in the frequency space do not always lead to usable, causal Green functions for performing the predictions. Section 6 provides some examples of numerical experiments performed using the now classical Lorenz 96 model [37] to support our results. In Section 7 we present our conclusions and ideas for future investigations.

Response Theory Formalism
Let's consider a smooth autonomous chaotic continuous-time dynamical system acting on a smooth compact manifold ℳ evolving from an initial condition ! at time = 0. We define , ! = Π ! ! its state at a generic time , where Π ! is the of evolution operator. Let's now consider the evolution for a time of a general observable . We define the Koopman operator ! operating on the observable as follows ! = ∘ Π ! , so that ! ! = Π ! ! [38]. The corresponding set of differential equations can be customarily written as where = Π ! !!! . The evolution of measure driven by the dynamical system given in Eq. (1) is described by the Perron-Frobenius operator [38] ℒ ! , such that , = ℒ ! , 0 . The operator ℒ ! is the adjoint of the operator ! , and is defined as follows: The family of operators ℒ ! !!! , forms a one-parameter semigroup, such that ℒ !!! = ℒ ! ℒ ! and ℒ ! = 1. The same applies for the family of operators ! !!! . The invariant measure ! of the dynamical system given in Eq. (1) is the eigenvector with eigenvalue 1 for all ℒ ! !!! . Assuming strong continuity and boundedness of the semigroup given by ℒ ! !!! , we can introduce the Liouville operator L, such that ℒ ! = exp , which allows one to construct the partial differential equation for the measure mirroring the set of differential equations given in Eq. (1): We now perturb the autonomous dynamics given in Eq. (1) by one specific forcing, so that the system is modified as follows: where ! ( ) is the forcing and ! is its time modulation. Having in mind the example of climate change, one may think ! ( ) as the extra radiative forcing due to the anomaly of CO 2 concentration, with defining its time-pattern; see [10,11]. Assuming -as mentioned abovethe chaotic hypothesis, response theory says that for any sufficiently smooth (e.g. C 3 ) observable Ψ ! ( ) the (time-dependent) change in the expectation value with respect to the unperturbed case can be written in linear approximation as is the (causal) Green function. Γ ! ! ,! ! can be written in terms of the properties of the unperturbed flow and depends explicitly on Ψ ! , ! . Using Ruelle's response theory, we have: (3b) Note that in the case of a system possessing an invariant measure that is smooth with respect to Lebesgue, so that ! = ! , we can write: perturbation to the Liouville operator on the right hand side of Eq. (1c) due to the introduction of the additional forcing given by G ! ( ).
Note that Eqs.
(2)-(3) indicate that the time-dependent version of the Ruelle response theory can be used for practically constructing the time-dependent measure defining the pullback attractor [39,40,41,42] of the non-autonomous system given in Eq. (2). The results can be extended beyond linear approximation [3,43] and applies for small (in principle, infinitesimal) forcings. Response theory becomes of little utility when one is near critical transitions [44,45], which can be seen as phase transitions for -in general -non-equilibrium systems.
After taking the Fourier Transform of Eq. (3a), we obtain: is usually referred to as susceptibility. We remark that any Γ !,! , can, under suitable assumptions, be written in general as: where ! are -for all choices of Ψ, -the eigenvalues of the Liouvillian operator introduced above, while the factors ! are constructed by computing the projection of response operator on the corresponding eigenvectors. The constants ! = ! are usually referred to as Ruelle-Pollicott poles [7,46,47]. If we consider smooth enough observables, causality implies that ! do not have positive imaginary component, i.e. ℑ ! = ℜ ! < 0 ∀ [3,4,5]. As a result, the functions Γ !,! are analytic in the upper complex plane (and have a meromorphic extension on a strip including the real axis in the lower complex plane), which leads to the fact the functions Γ !,! obey Kramers-Kronig relations [48] for all choices of smooth Ψ, [4,5,6,49]. The presence of at least one Ruelle-Pollicott pole, say ! , with real part very close to zero is associated to slow decay of correlations with time t of the form ≈ exp ℜ ! in the unperturbed system and, as first discussed in [50], can lead to breakdown of linear response for small forcings; see also a detailed analysis in the case of finite dimensional Markov chains in [51]. As recently clarified in [44,45], such a condition can be considered in some cases as an effective flag for anticipating critical transitions.
One should note that the expression for the susceptibility given in Eq. (5) mirrors very closely the quantum expressions for the electric susceptibility for e.g. atoms or molecules. In this latter case, the summation involves all the pairs of the eigenstates of the unperturbed Hamiltonian operator, of the system, and in each term of the summation the imaginary part of the poles corresponds to the energy difference between the pair of considered eigenstates, the real part is the so-called line width of the transition (whose inverse is the life time), and the numerator is the so-called dipole strength for the considered transition [52]. The analytic properties of the electric susceptibility function can be associated to the fact that the system is energetically passive.
At practical level, it is important to underline that the formal similarities between linear response theory for equilibrium and nonequilibrium systems discussed in [46] allow in principle one to use algorithms developed for the analysis of electronic systems such as Vector Fitting (VF) [53] and the more advanced RKFIT [54] for the analysis of the response of multiple observables to perturbations for a general nonequilibrium system and, in particular for estimating the Ruelle-Pollicott resonances.

Observables as Predictors and Predictands: Causality, Locality, and Memory Effects
Now, let's consider a second observable Ψ ! ( ). Repeating the derivation as in Eq. (4) above, we find: By taking the ratio of Eqs. (4a) and (6a), it is easy to derive that (6b) where this relation holds regardless of the time pattern of the forcing ! , but by just assuming its presence (i.e. ! cannot be identically zero). The previous equation can now be written as or, in the time domain, after taking the inverse Fourier Transform, as which, in general, is not local in time, but indeed linear. We refer to H !,!,! ! as the surrogate susceptibility and to H !,!,! ! as the surrogate Green function. One can clearly exchange the role of Ψ ! and Ψ ! and derive We underline that the surrogate Green functions H !,!,! ! and H !,!,! ! depend on the specific choice of the forcing ! . Next, we find an expression of the integration kernels H !,!,! ! and H !,!,! ! in order to understand, e.g., whether one of the two observables can be seen as precursor of the other.
The first remark is that both functions Γ ! ! ,! ! and Γ ! ! ,! ! are analytic in the upper complex plane, and so is their ratio, unless Γ ! ! ,! ! and Γ ! ! ,! ! possess complex zeros located there. For the moment we assume that no complex zeros are found in the upper complex plane; we will provide in Sect. 5 an interpretation of the cases where such condition does not hold, and how this can affect our results.
In the rest of Section 3, we make some assumptions on the functional form of H !,!,! ! in order to derive explicit formulas that can be used to provide a more intuitive physical interpretation to our results. Nonetheless, the main findings of the paper do not depend on the approximations taken below.
If we neglect for the moment the essential spectrum of the operator L in Eq. (1c), and using Eq. (5), we propose to approximate H !,!,! ! with a generic rational function of the form where we can consider arbitrarily large values of N and M, and where all the ! 's and ! 's are -in general -distinct. Because of analyticity, all the constants ! 's have negative imaginary part. Obviously, by the same token, also all the poles of H !,!,! ! = 1/H !,!,! ! must be in the upper complex plane, so that all the constants ! 's must also have negative imaginary part. Note that Eq. (11) implies that for values of larger than then the largest value of ! 's and ! 's ,we have so that !!! must correspond to the ratio between the asymptotic behaviours of the two susceptibilities. We can then express H !,!,! ! as follows: A standard partial fraction expansion gives us the following expression for K !,!,! !
and !,! is the standard Kronecker's delta which is nonvanishing only when the two arguments are equal. We remark that the constant ! have, in general, nothing to do with the Ruelle-Pollicott poles ! = ! defined above, because here we are constructing the properties of a constrained dynamics where one of the observables acts as surrogate forcing to the other.
We recall that the inverse Fourier transform of − ! is ! , where ! indicates the p th derivative of the Dirac's distribution, and we define: . We also recall that, assuming that ! has a negative imaginary part, the inverse Fourier transform of 1/ − ! is Θ exp − ! , where Θ is the Heaviside's distribution. We then obtain: where we have separated the singular contribution !,!,! ! to the Green function from the nonsingular one, given by K !,!,! ! . After performing the convolution following Eq. (9), we derive: The terms in the first summation provide a local (in time) link between Ψ ! ! and Ψ ! ! . The remaining terms are due to the non-singular component of the Green function and provide the non-local (in time) contribution, where memory effects are relevant. We recall that, asymptotically, the decay of the memory is controlled by the pole with the smallest (in absolute value) real part. At finite time, the coefficients ! contribute to determining which terms of the previous summation are dominant.
(12)-(13) -then we have that necessarily H !,!,! ! will be, instead, of the form given in Eq. (15), with non-vanishing contributions from the Dirac's delta and its derivatives: where we have used new symbols for the coefficient in front of the various terms and have inserted the appropriate poles ! 's in the last summation. Note that we should not expect the set of poles ! and ! to be identical, and this reflects into the fact that different observables retain differently the properties of the memory of the system.
In this case, there is a clear difference between the two observables Ψ ! ! and Ψ ! ! , because local (in time) information is relevant only in one direction of the inference. Nonetheless, it is important to note that inference is possible from any pair of observables, in both directions. Furthermore, we note that, in this case also Ψ ! ! also plays the roles of a surrogate forcing, because it appears in the convolution integral. The memory terms in Eq. (16)  at previous times, and the same reversing the applies reversing the roles of the two observables, even if the non-local rest term is clearly different in the two cases, compare Eq. (15) and Eq. (16) setting N=M (in this case ! =1/ ! ). In general, since the poles ! and ! are in general different, the time scales over which the memory acts in the two directions of prediction are in general different.

Generalizing the Results for more Complex Patterns of Forcing
We now wish to generalise the results presented above by allowing for accommodating multiple forcings in the system. The price to pay will be, as shown below, the need for considering multiple observables as predictors. Let's consider the case of a more complex pattern of perturbations added to the system: where ! ( ) and ! are two different vector fields modulated by two different time patterns ! and ! . We now consider three independent observables Ψ ! , Ψ ! , and Ψ ! , whose change in the expectation value due to the introduction of the forcing can be written as: Following the same approach as before, we would like to be able to construct an effective equation for expressing the change in one of the observables as a function of the change in the other ones. Without loss of generality, we want to find a relationship where we have slightly simplified the notation for matters of readability. We now substitute the right hand side of Eqs. (18a,b,c) in Eq. (19) and obtain: and ! are arbitrary, we derive: which gives: As all the susceptibility functions are analytic in the upper complex a plane, if one excludes complex zeros in the upper complex a plane for the denominator in the fist factor on the right hand side of Eq. (22), one derives that also H !,! and H !,! are also analytic in the same region. We then derive the following integro-differential equation where H !,! = S !,! + K !,! , with S !,! indicating the singular component of the integration kernel and K !,! the non-singular one. Equation (23), similarly to Eq. (10), has the remarkable feature of applying regardless of the time patterns ! and ! , which may be hard to access in many applications. We remark that, instead, H !,! and H !,! do depend on the vector flows ! and ! .
The problem can be generalized to the case when N distinct forcings are applied: and we wish to express the response of one observable Ψ !!! ! as a linear combination of the response of other N observables as follows: is the susceptibility of the observable Ψ ! subjected to the forcing ! , and inserting this in equation (24) , we derive by equating all terms with the same factor ! , l=1,…,N: which gives as final solution: which define the N surrogate susceptibilities of the system. By applying the inverse Fourier transform to the H !!!,! functions, we can generalise Eq. (23) as follows: Equations (10), (23), and (28) provide, in increasing level of generality, a comprehensive way for reconstructing the response of one observable of the system when one knows the response of other observables. As of the derivation above, the theory seem extremely flexible, as, in principle, no constraints exist in the choice of the predictor(s) and of the predictand. An important -in fact, essential -caveat is reported below.

Remarks on the effects of the presence of complex zeros in the effective susceptibility functions
Let's give a new, critical look at Eqs. (8)- (10). Assume now that Γ ! ! ,! ! possesses a complex zero in the upper complex plane. The occurrence of complex zeros is a very non-trivial property of complex functions, see [55], and has extremely relevant impacts in standard optical retrieval techniques [48]. This amounts to saying that Ψ ! can have a zero response to specific bounded, physically realisable time patterns ! for the vector field ! , with the ensuing result that some information on response of the system is lost if one looks at it through the lens of Ψ ! ! . As a result the surrogate susceptibility H !,!,! ! is not analytic in the same domain. This marks a clear departure from the standard case investigated in response theory.
As a result, the surrogate Green function H !,!,! ! includes also non-causal contributions, so that its support is not limited to the non-negative subset of the real axis. This implies that in this case the knowledge of the past value of Ψ ! ! is not sufficient for predicting the future values of Ψ ! ! , or, in other terms Ψ ! ! is an imperfect predictor of the predictand Ψ ! ! . One can say that the reduced system we are studying does not have a purely passive nature, but is instead also active, as a result of unstable feedbacks. Whereas response theory applies for all sufficiently well-behaved observables and cannot rule out the existence of susceptibility functions featuring complex zeros in the upper complex plane, the procedure outlined here, given the presence of a specific forcing with spatial patterns ! , allows one to discriminate between observables featuring or not predictive power on the future state of other observables.
Any prediction must be based only on the past data of the predictor. Therefore, in the case H !,!,! ! includes non-causal components, we perform predictions using the modified surrogate Green function: where K′ !,!,! ! = Θ K !,!,! ! . Note that the Heaviside distribution takes care of enforcing causality 1 . In the case we are dealing with a problem where we need to reconstruct a signal (as in the case of proxy data in geosciences) rather than predict it, this problem is less relevant as the non-causal component of the surrogate Green function can also be used.
The same problems described here for the simple case of one forcing and one observable could emerge, a fortiori, in the case of N forcings and N observables, because the algebraic operations involved in constructing the inverse of the matrix in Eq. (30) can cause the emergence of poles in the upper complex plane for one or more of the surrogate susceptibilities Κ !,! . As a result, we have to replace in any actual prediction the integral kernel K !,! with its causal component K′ !,! , as described above. We will show below a case where, instead, using two observables as predictors (thus taking advantage of Eq. 23) cures the pathology associated with the presence of complex zeros in the susceptibility of one of such observables.
The fact that not all choices of predictors and predictands are equally fortunate, makes, in fact, perfect intuitive sense, as implied in the classical concept of feedback on a system. What we show above is a mathematical characterization of such an intuition: when suitable predictors are chosen for predictand, the non-causal component of the integral kernel will be as small as possible (or even identically zero).
where P indicates the principal part..

Numerical Experiments
In order to test the performance and explore potential pitfalls of the methodology proposed in this paper, we consider the now classic and widely used Lorenz 96 model [37]. This is a forced and dissipative model that represents metaphorically the main processes -advection, dissipation, and forcing -occurring in a latitudinal circle of the atmosphere. The model has rapidly become a standard testbed for many mathematical methods in geophysics [56,57,58,59], and most notably for data assimilation and parametrization schemes, and has recently gained relevance in the context of dynamical systems theory and statistical mechanics [12,60,61,62]. In its simpler, onelevel -version, the model is defined as follows: with periodic boundary conditions !! = !!! , ! = ! , and ! = !!! . One usually assumes =1 and ! = , ∀ = 1, … , . The first term on the right hand side describe the nonlinear advection, the second term describes the dissipation, and the last term describes the forcing. In the inviscid and unforced case ( = 0 and = 0), the total energy of the system = ! ! 2 ! !!! is a conserved quantity (but it is not the generator of the time translations, see [63]). When =1, the model exhibits chaotic behaviour and intensive properties for ≥ 6 and ≥ 20, approximately [62]. We select for our simulations the classical values of = 8 and = 36, which put the system well within the chaotic regime. We perform all of our simulations below taking advantage of the MATLAB2017™ software package and using an adaptive Runge-Kutta 4 th order time integrator with absolute and relative precision of 10 !! . We first run a very long simulation of the unperturbed system lasting 10 6 time units from a random initial condition after discarding the data coming from time 10 3 units of integration, in order to remove transient effects and be safely within the steady state regime. We then consider -separately -the following perturbation vector fields: • !,! = 1, ∀ = 1, … , , corresponding to a change in the constant forcing; • !,! = − ! ∀ = 1, … , , corresponding to a change in the viscosity. We investigate the response of the system in terms of changes in the expectation value of the , j=1, 2, 3; where, in particular, Ψ ! is usually referred to as momentum of the system, and Ψ ! is, as mentioned before, the energy of the system. In order to do so, we need to construct the Green functions Γ ! ! ,! ! for all the combinations above.
Following the definition given in Eq. (3b), the Green functions need to be estimated by taking an average over the unperturbed invariant measure. We approximate it by using 100000 ensemble members, each chosen every 10 time units of the unperturbed run described above. We then proceed as follows. For each member of the ensemble, we introduce the perturbative vector field ! above with the time pattern ! = ! 2 , then ! = − ! 2 , (where is the Dirac's delta) and take the difference of the two signals for each considered observable ϕ ! , for j=1, 2, 3. We compute the ensemble average of the signals and derive Γ ! ! ,! ! , for j=1, 2, 3. We use ! = 1, while not small, is well within the range of linearity.
Note that by computing the linear response as a centred difference, we eliminate the quadratic nonlinear term, as discussed in [20]. We then repeat the same procedure, using the same ensemble members, for the perturbative vector field ! , and evaluate Γ ! ! ,! ! by computing the semidifference between the response obtained the time pattern ! = ! 2 and ! = − ! 2 , We choose here ! = 0.1, which, similarly, is well within the range of linearity. The Green functions obtained according to this procedure are presented in Fig. 1, where we show that the characteristic response time of the perturbation to the viscosity and to the forcing F are comparable for all observables, as to be expected from Eq. (5). As quite intuitive from physical arguments and from the shape of the Green function, increasing the forcing has the effect of increasing the expectation value of the three considered observables, whereas the opposite holds for the viscosity.
All Green functions analysed here have non-vanishing values in the limit of going to zero from positive values. Along the lines of [13], by using Eq. (3c) and the definition of ϕ ! , we can derive the following results: See Table 1 for the numerical estimates of the expectation value of the observables ϕ ! , = 1,2,3 in the unperturbed state. Following [13], we derive that at leading order, for large values of : We then proceed to constructing the functions H !,!,! ! as defined in Eq. (8). As all the considered susceptibilities have the same asymptotic behaviour for large values of , all the functions H !,!,! ! converge asymptotically for large values of to a constant. One finds: This leads to the presence of singular contributions S !,!,! ! in the corresponding integration kernels H !,!,! ! . The singular contributions are in the form of Dirac's deltas and have the following expression: We then shift our attention to the non-local component of the surrogate Green functions, for which it is harder to derive explicit results. We first treat the case of perturbations to the forcing F of the system and we show in Fig. 2 the functions K !,!,! ! , i, j = 1,2,3. The surrogate Green functions have a slightly less trivial structure than those depicted in Fig. 1, as more complex features appear, but still broadly conform to the case of representing an exponential decay modulated by oscillations. We find that in this case it is possible to reconstruct the response of each observable from the knowledge of the present (see Table 1) and past (see Fig. 2) state of any other observable. In other terms, one can treat, e.g., ϕ ! as surrogate forcing for ϕ ! , and viceversa. As discussed above, the fact that, e.g., both K !,!,! ! and K !,!,! ! are non-vanishing proves that the neither acts as a true external forcing. We now consider the case of perturbations to the viscosity of the system and we analyze the functions K !,!,! ! , i, j = 1,2,3. We find (Fig. 3) that the two functions K !,!,! ! and K !,!,! ! have a substantial non-causal component, and an extremely slow decay of correlations for positive times. The presence of a non-causal component must result from the existence of complex zeros for the function Γ ! ! ,! ! in the upper complex plane. Note such an occurrence is due to the specific combination of the observable and of the applied perturbation, and cannot be easily ruled out (or predicted) a priori.
We want to test the impact of the presence of a non-causal component in the surrogate Green function on our ability to predict the response of the system to a perturbation. We then consider a perturbation to the viscosity of the model (thus considering the vector field !,! = − ! ∀ = 1, … , ) modulated by ! = 0.05 sin − 0.1 sin 6 7 + 0.04 sin 8 (35) and compute over an ensemble comprising of 10000 members (the first 10% of the ensemble members described before) the time dependent change in the expectation value of ϕ ! , shown by the black line in Fig. 4. Clearly, the choice of ! (just like ! below) is arbitrary. We then estimate ϕ ! ! by convolving the corresponding Green function Γ ! ! ,! ! shown in Fig. 1 with ! in Eq. (37). The output of the prediction of ordinary linear response theory compares quite well with the observed value and is plotted as a blue line in Fig. 4. We then estimate ϕ ! ! by performing the convolution of ϕ ! ! with the surrogate Green function H !,!,! ! , whose non-singular component is causal and is shown in Fig. 3, while the singular component can be reconstructed by considering what is reported in Table 1 and Eq. (36). The prediction for ϕ ! ! obtained using ϕ ! ! as predictor is extremely good and is shown by the black line in Fig. 3.
Finally, we repeat the same procedure by using ϕ ! ! as predictor. We then convolve ϕ ! ! with the causal component of the surrogate Green function H′ !,!,! ! , constructed as described in Eq. (29). Its Dirac's component can be reconstructed by looking at Table 1 and considering Eq. (36), while, for the non-singular component, the enforced causality compels us to set to zero the non-singular component K !,!,! ! reported in Fig. 3 for non-positive values of . The result of the prediction is shown as the magenta line in Fig. 4. Clearly, this latter prediction is unsuccessful, and we conclude that ϕ ! ! is an inefficient predictor for ϕ ! ! .
We then treat the case where two independent forcings act simultaneously on the system, by adding on top of the modulation to the viscosity described in Eq. (33) a perturbation to the forcing F. We then consider a perturbation vector field !,! =   Fig. 6.
Therefore, by combining two forcings and using two observables as predictors, the pathology associated to inability of the observable ϕ ! ! to be a good predictor for ϕ ! ! for the specific case of forcing to the viscosity is sorted out, because more information is available on the response of the system thanks to using both ϕ ! ! and ϕ ! ! as predictors. In Fig. 6 we also show that ϕ ! ! ends up being the most important predictor, with ϕ ! ! playing, instead, a relatively minor role. A reasonable question to ask is whether Eq. (23) might be used also in the case where, in fact, only one between or , which provide the time modulation of the perturbation vector fields and , respectively, is different from zero. In this case, as discussed earlier, we could in principle use the simpler Eq. (10) for performing the prediction. We then decide to set =0 and use and as predictors of . Note that in Fig. 4 we had shown that in this case, using Eq. (10), cannot be used to predict , while serves the scope perfectly.
We find that, if we use Eq. (23), we can predict to a very high accuracy and, more surprisingly, provides a contribution to the prediction. The results are shown in Fig. 7, where the black line indicates the actual value of (and matches the black line in Fig.  4), the red line shows the prediction performed using Eq. (23), while the blue line shows the contributions to the prediction coming from using as predictand. Clearly, , which can by itself predict using Eq. (10), lends some role in the predictability to when the more complex Eq. (23) is used. We conjecture that using multiple predictors might increase the robustness of the prediction, as we might decrease the probability of encountering pathologies as the presence of complex zeros discussed above. . Note that the difference between the black curve here and that in Fig. 6 is, clearly, the blue line in Fig. 6.

Summary and Conclusions
Response theory is a well-established set of mathematical theorems or, in some cases, heuristically motivated yet often practically applicable formulas that allow one to predict how a large class of systems -including near equilibrium as well as far from equilibrium statistical mechanical systems -respond to applied perturbations. Specifically, response formulas give a prescription on how the expectation value of a generic observable will change as a result of the forcing, where such change in expressed in terms of the statistical properties of the unperturbed system. When the system has full Lebesgue measure, this boils down -in the linear case -to the celebrated fluctuation-dissipation relation. Response theory can be developed also in the case of time-dependent forcings: here, response theory provides a practical tool for approximating the time-dependent measure supported on the pullback attractor of the system under consideration. Finally, response theory -or better, the analysis of the conditions under which it breaks downcan be key for detecting and anticipating critical transitions.
While response theory is extremely powerful and widely used, it is not practically usable in some relevant scientific cases. In many complex systems we have only partial information on the state of the system, on its dynamics, on the acting forcing, and we might want to use some observables of the system as surrogate forcing. This might be motivated by practical reasons or by the knowledge of the acting feedbacks or by time scale arguments. Applications in geosciences and neuroscience often face these challenges.
In order to address this, in this paper, we have proposed a reformulation and extension of linear response theory where we blur the distinction between forcing and response, by focusing on the definition of constitutive relations relating the response of different observables, in order to construct tools for performing prediction, partly bypassing the need for knowing the applied forcing in full detail. Our theoretical predictions are supported by extensive yet extremely simple and computationally cheap numerical simulation performed on the Lorenz '96 system using commercial software (namely, MATLAB2017™) on a commercial laptop in a matter of a couple of days. The goal of these -purposely low-tech -simulations is to show that what the mathematical aspects discussed here emerge quite naturally.
We first evaluate to what extent one can use one forced signal (the change in the expectation value of one observable) to predict another forced signal (the change in the expectation value of another observable), thus bypassing the need for knowing some details (in particular, the time pattern) of the applied forcing. This leads to defining surrogate susceptibilities and surrogate Green functions, which, in practice, amount to justifying the existence of general linear relations between the various forced signals. This point of view is very natural as it reflects the way one looks at feedbacks inside a complex systems, which often entails constructing approximate temporally ordered causal links describing how the effect of the forcing propagates between different components or parts of the system itself. We explain that, in some cases, one can predict the future state of observable ϕ ! from the past and present knowledge of observable ϕ ! , and vice-versa, so that it makes no sense to say what causes what. Additionally, once a forcing is considered, in agreement with physical intuition, we prove that in some cases an observable ϕ ! is not capable of acting as effective predictor for another generic observable ϕ ! . This results from the presence of complex zeros in the upper complex angular frequency plane of the susceptibility of ϕ ! . The presence of a complex zero implies that for some time-pattern of forcing, the change in the expectation value of the observable ϕ ! is identically zero As a result, the surrogate Green function is not causal and ϕ ! cannot be used for predicting any other observable ϕ ! . Such a loss of causality is obviously just apparent, because we are not relating the true forcing to a response. What we see is, in fact, the result of the loss of information due to using ϕ ! as surrogate for the forcing.
Finally, we show using standard linear algebra that, in absence of mathematical pathologies, as the one just mentioned above, if a system undergoes N different simultaneous forcings, it is possible to predict the response of an observable ϕ !!! using N suitably defined surrogate Green functions convoluted with N other observables ϕ ! , … , ϕ ! without the need to know the time patterns of the actually applied forcings. As shown in our numerical example, such a strategy may overcome the difficulties associated to choosing a bad observable as predictor: the use of multiple predictors seems inherently more robust.
The surrogate Green functions discussed above can be derived from specific sets of experiments where only one of the forcings is applied at a time. Following [64], this can achieved also using stochastic perturbations.
This allows for a model-assisted reconstruction of signals where we have no or partial information on the time patterns of the forcings, or even if one or more of the forcings are, in fact, even inactive. This seems extremely promising for a variety of problems in the analysis of the response of complex models to perturbations where we might be interested in finding relationships between different forced signals, or when the actual forcings might be hard to control or measure.
We foresee applications of our results in problems relevant for climate science such as the Intercomparison of climate models regarding their response to external forcing, the analysis of the relationship between forced and free variability, and, on a different note, on the reconstruction of climate readings from multiple proxy data. Additionally, our results seem relevant for studying, in spatially extended systems, the response of a part of the system to perturbations by looking at its response somewhere else. This might be relevant in fluid dynamical systems, in systems obeying diffusion laws, neural systems, where the presence of a maximum speed of propagation of the information might lead to barrier to prediction if the considered parts of the system and/or the location of the forcings are too far away.
Finally, we believe that our results might have some relevance in the context of the theory of causal networks [65], because we are able to define whether or not one can assume causal relations between different observables or different regions of a system.