Distinguishing between deterministic oscillations and noise

Abstract

Time-dependent dynamics is ubiquitous in the natural world and beyond. Effectively analysing its presence in data is essential to our ability to understand the systems from which it is recorded. However, the traditional framework for dynamics analysis is in terms of time-independent dynamical systems and long-term statistics, as opposed to the explicit tracking over time of time-localised dynamical behaviour. We review commonly used analysis techniques based on this traditional statistical framework—such as the autocorrelation function, power-spectral density, and multiscale sample entropy—and contrast to an alternative framework in terms of finite-time dynamics of networks of time-dependent cyclic processes. In time-independent systems, the net effect of a large number of individually intractable contributions may be considered as noise; we show that time-dependent oscillator systems with only a small number of contributions may appear noise-like when analysed according to the traditional framework using power-spectral density estimation. However, methods characteristic of the time-dependent finite-time-dynamics framework, such as the wavelet transform and wavelet bispectrum, are able to identify the determinism and provide crucial information about the analysed system. Finally, we compare these two frameworks for three sets of experimental data. We demonstrate that while techniques based on the traditional framework are unable to reliably detect and understand underlying time-dependent dynamics, the alternative framework identifies deterministic oscillations and interactions.

1 Introduction

A fundamental problem in the analysis of experimental time-series data is the identification and filtering of “noise”, which is considered to obscure the real underlying dominant mechanisms responsible for the functioning of the particular system from which the time-series was recorded [1,2,3]. This is therefore relevant to a huge range of scientific fields and physical problems, including, to name a few: engineering [4], quantum and particle physics [5, 6], geophysics [7], biology and medicine [8,9,10,11], power grids [12], and economics [13]. Although it is well known that there are some dynamical phenomena in which the randomness of “noise” actually itself plays an integral part in the functioning of the system [14,15,16], generally speaking one seeks to separate deterministic functioning from the fog of background processes that simply impede the visibility of the deterministic functioning (“observational noise”), or even the actual efficiency of the deterministic functioning (“dynamical noise”). Indeed, in general, it is precisely this connotation of being “present but not positively functional” that is generally conjured in the usage of the word “noise” [17,18,19,20,21,22,23,24,25,26,27].

Despite the wide relevance and significant impact of this problem, there is neither a unified mathematical characterisation nor consensus physical explanation for the existence of noise. For instance, noise has been variously defined as follows:

• 1. A random process [4, 8, 17,18,19,20,21] that can be defined by

  • (a) an uncorrelated evolution [20, 22],

  • (b) incoherence [7, 23],

  • (c) a lack of deterministic reproducibility [24].

• 2. Any behaviour that cannot be explained by consideration of the physical system itself [8, 11, 25,26,27].

In this paper, we propose an alternative physical mechanism through which the so-called “1/f noise” can be generated and a corresponding mathematical framework through which it can be understood.

Noise in its most common definition is a phenomenon whose constituent frequencies each contribute significant power, with no single frequency contributing far more than every other, and with contributions from a broad range of frequencies. We now outline the origin and development of this definition.

• In 1905 and 1906, Einstein and Smolochowski characterised Brownian motion, a paradigmatic example of noise, as due to the impact of intractably many water molecules on pollen grains [28, 29]. This is emblematic of one of the dominant understandings of the physics of noise as a very high-dimensional process.

• In the 1920s, Johnson and Schottky observed broad-frequency current fluctuations in a vacuum tube. Many explanations for this were proposed, including atomic impurities [30, 31].

• In the 1960s, Mandelbrot suggested that ‘1/f’ noise and multifractals are related to the same physical phenomena, characterised by an extension of correlations globally throughout the system (‘wildness’) and by obeying some law or symmetry that holds across scales (‘self-affinity’) [32]. It was more recently shown that ‘1/f’ noise and fractals can be unified in a self-similar hierarchy [33].

• In 1976, Hasselmann proposed that deterministic chaotic forcing of a faster timescale than the timescales of the forced system can be an origin of noise-like behaviour in climate contexts, with the forcing dynamical system being a “weather” system and the forced dynamical system being a “climate” system [34].

• In 1987, once again, the field of dynamics is applied to try and explain the origin of noise [35]. In this framework, noise is thought to arise in high-spatial-dimensional dissipative systems with self-organised critical states. Such systems form minimally stable states on a range of length scales. It was argued that small perturbations to the system, propagating exclusively within these differently scaled states, produced a response on a range of temporal scales, i.e., noise.

In all of these physical explanations for noise, no role is played by the possibility of temporal variations in the parameters of the processes underlying the mechanism of noise emergence; rather, to generate the broad-frequency spectrum, these mechanisms rely on either the superposition of many time-independent components, i.e., high-dimensionality, or on very fast-timescale aperiodic dynamics.

As a result of its inscrutability under traditional analysis techniques, noise is commonly associated with a random evolution where there is little correlation between subsequent values in time. Hence, rather than constructing a high-dimensional time-independent deterministic model, noise is often represented by (stationary) stochastic processes.

Our alternative explanation is instead based on the observation that the vast majority of real physical systems are open to matter and energy exchanges, and as a result exhibit time-dependent dynamics. We will demonstrate that models based in this principle can generate a noise-like spread of power contributions across a wide frequency range while being both low-dimensional and consisting of an entirely deterministic, highly correlative, evolution rule.

The deterministic rules by which this apparent noise arises provide much more useful information about a system than assuming it to be a stochastic process. However, the deterministic nature of this supposed noise only becomes apparent when the dimension of time is explicitly incorporated in the analysis. Hence, in systems where this mechanism is possible, which we will outline further in Sect. 2.2, accounting for it may lead to much more informative data analysis and modelling approaches.

Due to the time-dependence of this mechanism, traditional and widely used analysis techniques that are based on the assumption of time-independence, such as the power-spectral density, can produce misleading results in this context. This problem also cannot generally be resolved by methods based in the theory of nonstationary stochastic processes, as we will discuss further in the next section. We will hence outline the theoretical (in Sect. 2) and practical (in Sects. 3 and 4) need for an alternative analysis framework based in deterministically time-dependent dynamics, in addition to demonstrating its aforementioned advantages.

2 Two frameworks for describing physical systems and their output time-series

In this section, we introduce an alternative framework for analysing time-series data, named the “non-autonomous phase network dynamics” (NPND) framework. This framework assumes the networks of underlying processes responsible for producing the time-series to be deterministically non-autonomous, particularly via time-varying frequency parameters. This is in contrast to the classical analysis framework, which instead treats time-series as sample realisations of a stochastic process.

A very common mode of analysis in the classical framework is the estimation of the power-spectral density (PSD) of a time-series. This analysis may be used to identify the frequency of deterministic processes evident in the time-series (through identification of clear, isolated spectral peaks). It may alternatively be used to classify the time-series as “noise” by the absence of such peaks, with the spectral power instead conforming to a certain linear gradient on a log–log plot of PSD against frequency. For example, a time-series is considered to be “white noise” if its PSD linear gradient is constant, while it is considered “pink noise” if the power is proportional to $1/f^\beta$ for some $\beta \in (0,2)$, and “blue noise” for some $\beta \in (-2,0)$, where f is the frequency. This classical framework is described in more detail in Appendix A.

The emergence of noise in the classical framework is commonly explained by two physical origins:

• as the net effect of many individually intractable small influences (e.g., Brownian motion of pollen particles [28]);

• as a chaotic process (e.g., turbulent “subgrid” processes in climate models [34, 36]).

Our framework identifies a third possible origin for apparent noise: few, analytically tractable, non-autonomous processes, appearing like noise when analysed under the assumptions of the classical framework. Such processes can be found wherever there is a thermodynamically open system, which is to say, throughout all natural systems.

The emergence of these systems can be explained through a physical analogy of an adaptation to the classical Brownian motion experiment, which is closely identified with the idea of noise as intractably many small influences. In this analogy, presented in Fig. 1, we replace the mesoscopic water-molecule collisions with a macroscopic pump that disturbs the surface of the water with a non-strictly-periodic rhythm, modulating the motion of the pollen grains in a deterministic but temporally irregular manner. This, hence, confers a time-dependent aspect to the pollen grains’ dynamics.

Fig. 1

A physical analogy for the difference between two mechanisms for the generation of signal components that may be identified as noise by a typical analysis as described in Appendix A. On the left, representing a classical mechanism of noise emergence, we have the classical Brownian motion experiment of pollen grains on undisturbed water, performing highly complex motion due to the large number of small mesoscopic-scale interactions with water molecules. On the right, metaphorically representing our NPND framework described in Sect. 2.1, we have a less complex motion arising from water currents induced by an external pump whose strength has the freedom to modulate according to external factors. In the latter case, an entirely macroscopic-level deterministic description of the origin of the motion in terms of the time-dependent behaviour of the pump would be obtainable by suitable time-resolved time-series analysis of the motion

2.1 Non-isolated systems and the NPND framework

The classical analysis framework, as discussed above and in Appendix A, treats time-series as products of a stochastic process, but models the underlying physical system as autonomous differential equations.

This modelling choice represents a physical assumption that the system can be treated either as (a) entirely isolated from the rest of the universe, or at least as (b) unable to have any interaction with its environment besides dissipation of energy into its environment. In other words, an autonomous differential equation can only describe an unforced physical system.

However, this is a restriction that rules out all natural physical systems. If one wishes to incorporate the time-dependent external forcing that is present in all such systems into a dynamical model, as in our framework, then this model cannot be an autonomous differential equation.

A “classical” way to incorporate such external forcing is to model it as dynamical noise with a stationary probability distribution, and another approach is to model it as periodic or quasiperiodic. However, one cannot typically expect that a system open to influence from an ever-changing environment will exhibit characteristics that are time-independent or that follow some indefinite-time pattern such as quasiperiodicity.

One approach towards overcoming this issue when analysing time-series is to generalise concepts from the theory of stationary stochastic processes and estimation of their time-independent statistical parameters to the broader theory of nonstationary stochastic processes and estimation of their time-dependent statistical parameters. However, a difficulty with this approach is that one must assume that there is a time scale on which the stochastic process is approximately an ergodic stationary process. This requires that the time scale be sufficiently slow that the approximate ergodicity is manifested in individual sample realisations, and the nonstationarity itself must then take place on an even slower time scale. In short, the framework of parameter estimation for generic nonstationary processes really assumes that the process represents an adiabatically slow parameter drift through a parameter-dependent ergodic stationary stochastic process.

An example of how this framework may be applied is in the use of methods such as moving-window autocorrelation for early warning of a bifurcation-induced critical transition, when a parameter of an autonomous dynamical system subject to stationary noise is being slowly forced. A specific example is the anthropogenic forcing of climate tipping elements, when the unforced tipping element is modelled as an autonomous system subject to stationary noise [37]. In this approach, the temporal variation of external forcing, typically bound to be present in systems that are freely open to general influences from the surrounding world, is not considered.

In seeking to solve the problem of analysis of non-isolated systems, rather than adopting the classical approach for stationary processes to nonstationary stochastic processes, we re-derive the physical principles on which any such framework is based.

We consider the observation that the thermodynamic openness of natural systems implies a time-dependent forcing on these systems, which leads to bounded, non-static system behaviour. An important role will be played by cyclic or oscillatory processes within such a system. However, in contrast to the classical model of cyclic processes that posses a stable periodic orbit resulting from an autonomous dynamical system, these processes do not necessarily possess any internal strict periodicity. Instead, they progress through the cycle in a way that is inextricably linked to their ever-changing environment. Further important functions of the system will be played by interactions between these processes, which themselves may vary in time.

The basis of our framework naturally leads to a model of networks of non-autonomous oscillatory differential equations.

Assuming the system’s evolution can be modelled by a finite-dimensional differential equation, we can express our above physical description in mathematical language as follows:

$$\begin{aligned} X(t) =&\ H(\theta _1(t),\ldots ,\theta _n(t),t) \nonumber \\&\ \text {[maybe }+\text { noise, maybe }+\text { other components]} \end{aligned}$$

(1)

$$\begin{aligned} \dot{\theta }_i(t) =&\ F_i(2\pi f_i(t),\theta _1(t),\ldots ,\theta _n(t),t) \nonumber \\&\ \text {[maybe }+\text { noise]}, \quad i=1,\ldots ,n, \end{aligned}$$

(2)

where $\theta _1(t),\ldots ,\theta _n(t)$ are angles between 0 and $2\pi$ each representing the phase of a cyclic process, and $f_1(t),\ldots ,f_n(t)$ are these cyclic processes’ “time-localised internal frequency”. Note that the angular velocity $\dot{\theta }_i(t)$ is not the same concept as time-localised frequency.

A simple example of a model within this abstract framework is to take

$$\begin{aligned} H(\theta _1,\ldots ,\theta _n,t)&= \frac{1}{n} \sum _{i=1}^n A_i(t) h_i(\theta _i)\\ F_i(\omega ,\theta _1,\ldots ,\theta _n,t)&= \omega + C_i(\theta _1,\ldots ,\theta _n,t), \end{aligned}$$

where $A_i$ is a potentially time-dependent amplitude measuring how strongly the ith cyclic process influences the time-series, $h_i$ represents the shape of how the phase of the ith cyclic process appears in the time-series, and $C_i$ is a time-dependent phase coupling function, which we take as 0 if there is no coupling; so, the model becomes

$$\begin{aligned} X(t) =&\ \frac{1}{n} \sum _{i=1}^n A_i(t)h_i(\theta _i(t)) \ \text {[maybe }+\text { noise]} \end{aligned}$$

(3)

$$\begin{aligned} \dot{\theta }_i(t) =&\ 2 \pi f_i(t) + C_i(\theta _1(t),\ldots ,\theta _n(t),t) \nonumber \\&\ \text {[maybe }+\text { noise]}. \end{aligned}$$

(4)

A standard example of the observable function H is the mean field, where $A_i=1$ and $h_i=\sin$, that is

$$\begin{aligned} H(\theta _1,\ldots ,\theta _n,t) = \frac{1}{n} \sum _{i=1}^n \sin (\theta _i); \end{aligned}$$

and a standard example of a phase coupling function is the Kuramoto coupling

$$\begin{aligned} C_i(\theta _1,\ldots ,\theta _n,t) = \sum _{j \in \{1,\ldots ,n\} \setminus \{i\}} a_{ij}(t) \sin (\theta _j-\theta _i). \end{aligned}$$

The cyclic processes $\theta _1,\ldots ,\theta _n$ are governed by a non-autonomous differential equation. In contrast to autonomous differential equations, which may be considered over any given time period, including an asymptotic one, non-autonomous processes may only be considered in finite time. Furthermore, shifting this finite-time period will alter the evolution of the system itself.

The implication of this is that while the time-independent nature of autonomous processes allows for the application of stationary stochastic theory inherent to the classical analysis framework, nonstationary stochastic processes may not be analogously applied to understand non-autonomous deterministic dynamics. This point is explored in further detail in Appendix A

The theory of finite-time dynamical systems has been growing in popularity in recent decades [38,39,40,41,42,43,44,45,46]. Our framework above is particularly concerned with oscillatory dynamics on finite-time scales. A model for cell energy metabolism that already serves to exemplify our NPND framework has been developed in [47]. A finite-time framework for qualitative analysis of dynamical stability of oscillatory processes subject to slow-time-scale external influences has also been developed in [46]. Our present paper illustrates how the NPND framework can, in fact, serve as a general overarching framework for the study of complex systems in terms of interacting oscillatory components.

2.2 Reconsideration of apparent “noise” via the NPND framework

We previously mentioned two classical mechanisms for the emergence of noise from deterministic fundamental laws of physics. Considering these mechanisms in the context of autonomous networks of oscillators, expressed in the same way as in our framework above except without the time-dependence in $F_i$ and $f_i$, a signal $X(t)=H(\theta _1(t),\ldots ,\theta _n(t))$ could produce noise-like results if

• n is extremely large (i.e. the system is very high-dimensional), or

• the system of equations

  $$\begin{aligned} \dot{\theta }_i(t) = F_i(2\pi f_i,\theta _1(t),\ldots ,\theta _n(t)) \end{aligned}$$

  is chaotic.

In both cases, even though the original equation is deterministic, one cannot practically obtain a deterministic description from time-series analysis, and so, probabilistic modelling is chosen instead.

The central point of this paper is the observation that without the need either for very large n [28, 48, 49], or for chaotic behaviour [50,51,52,53], the incorporation of time-dependence into the model can lead to results that a typical PSD approach will characterise as noise. Thus, for analysis of experimental time-series data arising from a thermodynamically open system, a different methodology is needed to distinguish prominent, functionally important oscillatory components from noise.

2.3 Deterministic time–frequency analysis

Just as there is a plethora of well-known time-series analysis methods based on the traditional framework of stationarity, so there also exist many—currently not-quite-as-widely used—time-series analysis tools designed for gaining an understanding of time-dependent oscillatory dynamics; see [54] for an overview of several such methods. At the heart of many of these methods lies deterministic time–frequency analysis, which we will now describe.

We have explained how the classical framework makes use of PSD estimation as the means of separating oscillatory processes from noise. For our model in the form of Eqs. (3)–(4), the analogous natural tool to use is deterministic linear time–frequency analysis. In our phrase “deterministic linear time-frequency analysis”:

• “Time-frequency analysis” refers to time-evolving time-localised description of the frequency content of the signal. There are many different time–frequency analysers, such as the windowed Fourier transform and the continuous wavelet transform [55].

• By “deterministic”, we mean a time–frequency analyser that can be applied to an individual signal, as opposed to a time–frequency analyser defined theoretically in terms of the probabilistic law of a stochastic process from which the recorded signal is assumed to arise as a sample realisation.

• By “linear”, we mean that for signals $X_1,\ldots ,X_n$ linearly superposed to form a new signal $Y=c_1X_1 + \ldots + c_nX_n$, we have

  $$\begin{aligned}& {\textrm{Ampl}}_Y(f,t)e^{i. {\textrm{Phase}}_Y(f,t)}\\ &\qquad = \sum _{i=1}^n c_i{\textrm{Ampl}}_{X_i}(f,t)e^{i. {\textrm{Phase}}_{X_i}(f,t)}, \end{aligned}$$

  where ${\textrm{Ampl}}_X(f,t)$ and ${\textrm{Phase}}_X(f,t)$ denote, respectively, the amplitude and the phase assigned by the time–frequency analyser to the frequency f around time t for a signal X.

The key difference between classical time-independent frequency-domain representations of a signal (such as the Fourier transform and PSD estimators derived therefrom) and representations given by time–frequency analysis is that

• the former representations do not resolve in time but, in a sense, blur all time together;

• the latter representations enable a “two-dimensional resolving of the frequency content” in which the frequency decomposition is itself resolved in the time dimension.

It is precisely this distinction that plays the key role in the phenomenon that we will present in this paper, where the apparent time-domain complexity of a signal is concluded to be deterministically intractable noise by a frequency-domain representation, but is resolved into clear deterministic components by a time–frequency-domain representation. The link between our framework in the form of Eqs. (3)–(4) and deterministic linear time–frequency analysis is expounded in more detail in Appendix A.

3 Comparisons of the two frameworks with numerical models

In this section, we

• illustrate numerically how signals arising from not-very-high-dimensional oscillator networks according to our framework can be identified as $1/f^\beta$ noise when analysed by the typical procedure of computing its PSD and looking for a roughly linear downward trend in a log–log plot;

• show that applying the continuous wavelet transform resolves the true “determinism”—i.e., that the signal has a relatively small number of individually tractable components, as opposed to a large number of individually intractable components for which only statistical rather than deterministic analysis is possible;

• illustrate how the continuous wavelet transform, and higher order spectral analysis based thereon, can reveal further features of the underlying deterministic behaviour, such as the presence of coupling between oscillators.

We consider signals of the form

$$\begin{aligned} X(t) = \frac{1}{n}\sum ^n_{i=1}\sin \left( \theta _i(t)\right) , \end{aligned}$$

(5)

where the system of oscillators $\theta _i$ evolves according to the non-autonomous differential equation

$$\begin{aligned} \frac{1}{2\pi }\dot{\theta }_i = f_i(t) + \left[ \frac{A}{n}\sum ^n_{j=1}\sin \left( \theta _j-\theta _i\right) \right] , \end{aligned}$$

(6)

where $A>0$ corresponds to the presence of coupling, and $A=0$ to the absence of coupling. We take the time-dependence of $f_i(t)$ itself to be relatively slow compared to the internal time-periods $\frac{1}{f_i(t)}$ of the oscillations themselves. Note that it is precisely the addition of this new relatively slow time scale, without the need of any fast time scale, that will be responsible for causing the signal to become “noise-like” when analysed by typical traditional spectral analysis. As discussed in Sect. 2.1, time-dependence arises physically from the system being subject to influence from its ever-changing environment.

Fig. 2

Power-spectral densities and wavelet transforms of autonomous and non-autonomous oscillator models. The PSD S(f) and the magnitude |W(t, f)| of the wavelet transform W(t, f) are computed for different time-series X(t) as given by Eqs. (5)–(7) for different parameter values, all without coupling (i.e., $A=0$). All wavelet transforms are computed using the lognormal wavelet. a Ensembles of $n=10, \ 10^2$ and $10^3$ autonomous oscillators (i.e., $A_{\omega i}=0$ for all i). The frequencies $f_{0i}$ are linearly distributed in the range [1, 10] Hz. The log–log PSDs of the mean fields of the oscillator ensembles are calculated, with the exponent $\beta$ of their linear best fit indicated, where $S(f)\sim 1/f^{\beta }$. The wavelet transforms are calculated with a frequency resolution of 5. The $n=10$ and $n=10^2$ ensembles are simulated for 10,000 s, and the $n=10^3$ for 1000 s. b The same analyses as in a applied to non-autonomous ensembles of varying sizes, with centre frequencies $f_{0i}$ linearly distributed in the range [1, 10] Hz, modulation amplitude factor $A_{\omega i} = 0.2$ and modulation frequency $f_{\omega i} = 0.05$ Hz. The $n=10^2$ ensemble is simulated for 20,000 s, and the $n=10$ and $n=5$ for 200,000 s. The wavelet transform is calculated with a frequency resolution of 2. (c) $n=5$ non-autonomous oscillators, with $f_{0i}=5.05$ Hz for each $i=1,\ldots ,5$, and otherwise identical parameters to b. d $n=5$ non-autonomous oscillators with identical parameters to c, but with larger modulation amplitude $f_{0i}A_{\omega i}=4.95$ Hz

In several of our simulations, for the sake of simplicity, we take $f_i(t)$ to be a relatively low-frequency periodic function

$$\begin{aligned} f_i(t) = f_{0i}+f_{0i}A_{\omega i}\sin \left( 2\pi tf_{\omega i}\right) ; \end{aligned}$$

(7)

but (as we will see) the periodicity itself is not at all a requirement for our results.

For the simulations in this section, we numerically integrate Eq. (6) with various parameter values of the model, using a fourth-order Runge–Kutta algorithm and an integration step of 0.001 s, and with the initial phase values $\theta _1(0),\ldots ,\theta _n(0)$ of the oscillators evenly distributed in the range $[0, 2\pi ]$. The models are simulated up to different times generally corresponding to around the longest time computationally feasible. For the PSD computations, zero padding is then added symmetrically at the start and end of the signal, such that the total number of points in the resulting time-series is the nearest power of two above the original number of points before the zero padding was added. This is to make the size of the time-series equal to a power of two, and enable the PSD to be obtained from the same size for all time-series considered [56]. The wavelet transform and wavelet bispectrum computations were carried out in MODA, for which formulae can be found in [57].

3.1 Autonomous vs non-autonomous noise

First, in the absence of coupling (i.e. $A=0$), we consider the dependence of “noise-like” behaviour in a signal on the number of oscillatory components n and their range of frequencies, in the autonomous case (i.e., without time-dependence) versus the non-autonomous case, as defined by Eqs. (5)–(7) with $A_{\omega i}=0$ in the autonomous case and $A_{\omega i}>0$ in the non-autonomous case. This is examined in Fig. 2. We will first consider the PSD plots and then the wavelet transform plots. (In Fig. 2 and everywhere else in this paper, all wavelet transform plots show the magnitude |W(t, f)| of the wavelet transform W(t, f) plotted over time–frequency space.)

We see in Fig. 2a the PSDs of the autonomous models become more noise-like, with fewer discernible isolated peaks, as the number of components n is increased. This is consistent with the traditional understanding of the generation of noise. This is in contrast to the PSDs of the non-autonomous model in (b), where even for $n=5$, the PSD appears noise-like. This immediately demonstrates our central point that the introduction of time-dependent determinism can easily appear to be noise when analysed in the traditional framework, such as through a PSD, without the need for a large number of components. This is not significantly changed either by decreasing the time-dependent modulation amplitude in (c), or increasing it in (d). It is the presence of time-dependence that is the most significant factor, rather than its magnitude.

In accordance with Sect. 2.3, we now analyse the time-series produced from the above models using time–frequency analysis; specifically, we use the continuous wavelet transform because of its logarithmic frequency resolution that enables the simultaneous resolving of oscillatory components of a wide range of time scales. (We use the lognormal wavelet as the mother wavelet for the wavelet transform, since this provides particularly good time–frequency resolution [58, 59].)

These wavelet transforms provide much more information about the dynamics of all the models than the PSDs. At low numbers of components, each individual mode can be distinguished, and in the non-autonomous models, the presence of time-dependence is evident. At larger numbers of components, the modes begin to merge, but remain clearly deterministic and time-(in)dependent, in contrast to the noise-like conclusions suggested by the corresponding PSDs.

Fig. 3

PSDs, wavelet transforms, and wavelet bispectra of non-autonomous oscillator networks with varying coupling strengths. PSDs (top left), wavelet transforms (bottom left), and wavelet bispectra (right) of the mean field of networks of $n=10$ non-autonomous oscillators are plotted. Specifically, the wavelet bispectra results show the wavelet biamplitude associated with each point in frequency–frequency space after subtraction of the 95% significance critical threshold given by a surrogate test involving 59 numerically generated WIAAFT surrogate signals [60]. The wavelet transforms and wavelet bispectra are calculated using a lognormal wavelet with frequency resolution 5. Centre frequencies $f_{0i}$ of the oscillators are distributed equidistantly over the range [1, 10] Hz, with an addition of a random number between 0 and 1 Hz to reduce harmonic relationships between the frequencies. The modulation amplitude factor is $A_{\omega i}=0.3$ and the modulation frequency is $f_{\omega i}=0.05$ Hz. The network coupling strengths A are a 0 $\hbox {s}^{-1}$, b 5 $\hbox {s}^{-1}$, and c 10 $\hbox {s}^{-1}$. The models are simulated for 200,000 s

3.2 Networks of interacting components

While the uncoupled ensemble models that we have discussed thus far have already been instructive in illustrating the problem with applying the classical PSD approach of noise characterisation to systems with time-dependent characteristics, systems with multiple oscillatory components will often have some coupling between oscillatory components. Detection of coupling between components of a signal serves as a particularly strong indicator that these components are not “noise” to be discarded, but rather represent functionally significant deterministic processes. Understanding a system’s coupling relationships can also reveal much about its physical behaviour, representing the means by which systems exchange energy, as occurs throughout the physical world [61,62,63]. In our framework, we consider phase–phase coupling between oscillatory components; specifically, here, we will consider phase–phase coupling in which oscillators attract each other’s phases to their own.

We analyse non-autonomous networks consisting of $n=10$ oscillators of distinct time-dependent internal frequency, with all-to-all coupling of Kuramoto form, according to Eq. (6). In Fig. 3, we analyse such a network with three coupling strengths A: $A=0$ in (a), reducing back to the uncoupled ensemble case; $A=5$ $\hbox {s}^{-1}$ in (b), providing an intermediate-strength coupling; and $A=10$ $\hbox {s}^{-1}$ in (c), representing strong coupling.

To detect and analyse the coupling present in these models, after calculating the wavelet transform, we additionally consider the wavelet bispectrum [57, 64]. This is designed to detect non-linearities, such as coupling, within or between signals [57, 59]. Bispectra are defined over frequency–frequency space, such that the presence of a bispectral peak around a frequency pair $(f_1,f_2)$ may indicate a coupling between two oscillatory components of frequencies in the vicinity of $f_1$ and $f_2$. (However, on the diagonal, i.e., $f_1=f_2$, or more generally for rationally dependent $f_1$ and $f_2$, bispectral peaks may indicate just a single non-sinusoidal oscillatory component, with the bispectral peaks arising from harmonics of this non-sinusoidal component.) Here, in Fig. 3, and in all other bispectra plots in this paper, we calculate a biamplitude value (i.e., modulus of the time-averaged instantaneous wavelet bispectrum) at each point in frequency–frequency space as in [57], and apply a statistical significance test to decide at which points in the frequency–frequency space we deem the biamplitude value to be “significant”. More specifically, at each point in frequency–frequency space, we subtract from the wavelet biamplitude value a critical threshold (at 95% significance) calculated from the bispectra of wavelet iterative amplitude adjusted Fourier transform (WIAAFT) surrogates generated from the time-series under investigation, according to the procedure described in [60]. Hence, the strictly positive (i.e., non-grey) values shown in these bispectra plots correspond to where the bispectrum value is considered “significant”.

Let us now discuss both the wavelet transform results and the wavelet bispectrum results. In Fig. 3a, where we have reduced back to the uncoupled ensemble case, the results we expect based on the previous subsection are confirmed: the PSD is noise-like and the wavelet transform appears to indicate non-noise-like frequency modes with a deterministic time-dependence. When medium-strength coupling is introduced in (b), the PSD continues to be noise-like. The wavelet transform shows periods of asynchrony where the oscillations cluster around three dominant modes, and shorter periods of synchrony where the oscillations cluster around just one dominant mode. One of the three modes in the asynchronous case lies roughly at the frequency value indicated by the black ticks; another lies within the green-marked frequency band; and the third lies within the purple-marked frequency band (where we see the fundamental frequency of this mode and also harmonics arising from different combinations of the three frequency modes). In the periods of synchrony, the one frequency mode appears to represent the coinciding of the upper two of the three previous frequency modes, while the lowest of the three previous frequency modes vanishes in amplitude. This alternation between three dominant modes and one dominant mode is indicative of the intermittent synchronisation phenomenon of non-autonomous systems [65]. In (c), the wavelet transform shows that the network has become highly synchronised around a single frequency mode, and yet despite this coherent behaviour the PSD of this mode still appears noise-like. Therefore, we see in all these cases that non-autonomous networks of a small number of interacting oscillators can also appear to be noise-like when analysed by the classical PSD approach, even when synchronised.

Now, we discuss the wavelet bispectrum results, especially with a view to considering what we can learn from this analysis tool that would be difficult or impossible to learn from classical spectral analysis under the traditional framework. As expected, no significant bispectral content is found in (a), where there is no coupling. In (c), the full synchronisation of the system into a single mode masks the underlying coupling, leading to only one significant bispectral amplitude peak (indicated by the black-marked frequency band), which occurs on the diagonal and thus cannot indicate coupling between different frequency modes. With non-synchronising coupling in (b), however, we detect large areas of significant biamplitude, which are mostly contained within the frequency regions of the dominant modes identified in the wavelet transform. Significant biamplitude values in areas of the frequency–frequency space corresponding to two distinct such frequency regions indicate coupling between the modes represented by those frequency regions.

Results indicative of coupling or the absence thereof between frequency modes would have been obtainable from classical bispectral analysis if the network were an autonomous dynamical system (i.e., if there were no frequency modulation or other form of time-dependent characteristics); but as seen in the various PSD computations shown in this section, the presence of frequency modulation drastically alters the classical spectral properties to the point that individual oscillators look like noise.

The bispectral results shown here (and also later for experimental data) are the most basic wavelet bispectral analysis computations, which do not explicitly show the time-evolution of bispectral properties. Having performed such computations and identified significant biamplitude peaks, one can further examine the genuineness of the apparent presence of coupling, as well as further properties of such coupling and its time-dependence, using methods described in [57, 59].

Fig. 4

Tidal time-series recorded at Lancaster Quay, United Kingdom. The readings were made every 15 min from 16th–25th February 2022 (see the data availability statement for the data). The sensor is placed at 2.15 m, and unable to detect levels below this. a The time-series of the height of the water. b The PSD. c The multiscale sample entropy with $m=2$ and $r=0.15$. d The wavelet transform using the lognormal wavelet with a frequency resolution of 1.5. e The wavelet bispectrum using the lognormal wavelet with a frequency resolution of 1.5, tested (at 95% significance) against 59 WIAAFT surrogates generated from the data [60]

4 Comparisons of the two frameworks with experimental data

We now investigate the physical appropriateness of our framework as contrasted with the classical framework for three physical systems from which experimental data have been recorded; specifically, we will see how deterministic time–frequency analysis reveals important information not yielded by methods within the classical framework’s approach to time-series analysis. In particular,

• we will indicate how time–frequency analysis can avoid the mischaracterisation of deterministic functioning as “noise” that would arise from applying to these experimental time-series a traditional PSD approach to separating oscillatory components from noise;

• and furthermore, for illustrative purposes, we will show how one might derive from our wavelet transform analysis a preliminary form of approximate model of the time-series according to our framework in Sect. 2.1 as represented by Eqs. (3)–(4).

By a “preliminary” form of model, we mean that this is simply based on our present analyses of the time-series; one can work towards more accurate and precise models through a combination of (a) applying additional time-resolved analysis methods to the signal, (b) recording further signals from the system or type of system being considered and applying suitable time-resolved analysis methods to those signals as well, and (c) incorporating any relevant already-existing physical knowledge regarding the system from which the signal is recorded.

We analyse, in Fig. 4, the height of water recorded by a monitoring station located at Lancaster Quay, United Kingdom; in Fig. 5, the magnetic field strength at the Earth’s surface recorded by an observatory in Norway [66]; and in Fig. 6, the interplanetary magnetic field strength, indicative of solar wind activity, recorded by a satellite in between the Earth and the Sun. All three of these systems are thermodynamically open systems with time-variable external influences—whether they be the gravitational force of the moon or the plethora of electromagnetic forces that affect the magnetic fields at the Earth’s surface and throughout the solar system, from solar winds, to electric currents in the Earth’s atmosphere to plasma processes in the core of both the Earth and the Sun. These systems, therefore, all have the theoretical potential to have deterministic functioning that appears noise-like under a traditional PSD analysis.

Fig. 5

x-Component of the ionospheric magnetic field recorded by UiT The Arctic University of Norway’s KAR magnetogram station every hour for a period of 39 days and 23 h, from 2nd December 2021 to 10th January 2022 [66]. a The time-series of the strength of the x-component of the ionospheric magnetic field. b The PSD. c The multiscale sample entropy, with $m=2$ and $r=0.15$. d The wavelet transform using the lognormal wavelet with a frequency resolution of 3. e The wavelet bispectrum using the lognormal wavelet with a frequency resolution of 3, tested (at 95% significance) against 59 WIAAFT surrogates generated from the data [60]

The River Lune, the subject of Fig. 4, is known to be tidal at the point of measurement in Lancaster Quay. We would, therefore, expect to see the water levels oscillate with two peaks a day, and the wavelet transform therefore to show a significant oscillatory mode with a period of approximately 12 h. The time-series itself, shown in plot (a), immediately makes clear that the river is indeed tidal, oscillating through two peaks and two troughs each day. The wavelet transform [plot (c)] shows three clear frequencies present in the signal, corresponding to periods of approximately 12 h, 6 h, and 4 h, in descending order of amplitude. The 6-h and 4-h peaks correspond to twice and three times the frequency of the fundamental 12-h peak; they are harmonics arising from the fact that the 12-h-period oscillations in the time-series are not of sinusoidal shape. (The bispectrum in plot (d) shows significant peaks at rationally related frequency-pairs arising from these harmonics. A procedure to determine whether frequency modes are harmonically related in less clear-cut cases is described in [67].)

Fig. 6

Average magnitude of the interplanetary magnetic field recorded by the WIND satellite, stationed at Lagrange point 1 of the Earth’s orbit, for every hour for a period of 1 year, from 1st January 2022 to 31st December 2022 [68]. a The time-series of the strength of the magnitude of the interplanetary magnetic field. b The PSD. c The multiscale sample entropy, with $m=2$ and $r=0.15$. d The wavelet transform using the lognormal wavelet with a frequency resolution of 3. e The wavelet bispectrum using the lognormal wavelet with a frequency resolution of 3, tested (at 95% significance) against 59 WIAAFT surrogates generated from the data [60]

From looking just at the wavelet transform, one could build a preliminary form of model for the time-series as

$$\begin{aligned} X(t)&= A(t)h(\theta (t)) \\ \dot{\theta }(t)&= 2\pi /({\text {12 h}}), \end{aligned}$$

where h is a non-sinusoidal function as reflected by the presence of the harmonics of the 12-h fundamental period, and A(t) has fairly slow time-dependence on t as reflected by varying wavelet amplitude over time seen at the frequency $1/({\text {12 h}})$ (and similarly at the harmonic frequencies). Of course, in this relatively simple example, we can see the behaviour directly from the time-series itself in plot (a), but in more complicated systems this would not be so. Since there is not much time-dependence in the frequency—although there is still significant time-dependence in the amplitude—the peaks at 12, 6, and 4 h that we saw in the wavelet transform are also present in the power-spectral density, indicated on plot (b) by the dotted lines. However, such simple examples can be rare. In subsequent figures, we will consider cases with increasingly complex frequency dynamics.

Magnetic field strength fluctuations at the Earth’s surface are the subject of Fig. 5. There are many forces known to affect the magnetic field in the Earth’s atmosphere, not least solar winds. It is already known that significant variations in the atmosphere, such as during a geomagnetic storm, also lead to variations at the Earth’s surface, but the magnitude and significance of these surface variations, even during storm events, is still a matter of open research [69].

There are a variety of periodic fluctuations that have been found to take place in the geomagnetic field in absence of any solar event, primarily driven by electric currents resulting from solar winds and the moon’s gravitational field moving against the geomagnetic field [70]. Such periods include 6 h, 8 h, 12 h, 24 h, 4 months, and a year [71, 72].

In Fig. 5, we analyse magnetometer recordings over the space of 39 days and 23 h, from the IMAGE magnetometer network. The purpose of this network is to detect ionospheric auroral electrojet events, but we have chosen a period in which no such events were detected. This period is hence more characteristic of so-called ‘solar quiet’ dynamics. The time-domain of these recordings in (a) again does not immediately clarify whether the system is deterministic or noise-like, and the PSD in (b) shows no peaks. The wavelet transform in (c), while more complex than in Fig. 4, shows four frequencies where a significant amplitude is maintained for a large majority of the examined time, corresponding to periods of approximately 24 h, 12 h, 8 h, and 35 h in descending order of amplitude. Three of these, 24 h, 12 h, and 8 h, are well-known already, as mentioned above. The 35-h period, however, does not correspond to any known geomagnetic phenomenon. Besides these four frequency modes, the wavelet transform appears to show much other content in the signal that is somewhat noise-like, although with time-dependent intensity that is perhaps correlated with the amplitude of the 35-h-period and/or 24-h-period mode. The bispectrum in (d) shows very significant peaks at $(({\text {24 h}})^{-1},({\text {24 h}})^{-1})$ and $(({\text {24 h}})^{-1},({\text {12 h}})^{-1})$; but since the system contains distinct oscillatory processes of period 24 h, 12 h and 8 h, it is difficult to conclude with confidence that the 24-h component is significantly non-sinusoidal or that there is coupling between the 24-h component and the 12-h component, until first carrying out a much more detailed investigation of time-localised phase bicoherence [59] (preferably with a longer time-series recording). The other regions where the bispectrum plot shows values slightly above the critical threshold may be due to the inherently far-from-sinusoidal nature of the apparent noise, or it could be that this “noise” genuinely includes many temporally intermittent cyclic processes with couplings between them. These are all issues that could be investigated with further time-resolved analysis of this time-series and further magnetic field strength time-series recordings together with physical considerations of the system itself; but as a preliminary model according to our framework, we could model the time-series as

$$\begin{aligned} X(t)&= \sum _{i=1}^n A_i(t) \sin (\theta _i(t)) + \varepsilon _\xi (t)\xi (t) \\ \dot{\theta }_i(t)&= 2\pi (f_i + \varepsilon _i(t)), \end{aligned}$$

where the frequencies $f_1,\ldots ,f_n$ include the four frequencies $({\text {8 h}})^{-1}$, $({\text {12 h}})^{-1}$, $({\text {24 h}})^{-1}$ and approximately $({\text {35 h}})^{-1}$, the $\varepsilon _i(t)$ are functions representing the apparent slight time-dependence of the frequency in the wavelet transform, and $\varepsilon _\xi (t)\xi (t)$ represents a noise process with intensity proportional to a function $\varepsilon _\xi (t)$. Similarly to our first example in Fig. 4, the variations in amplitude $A_i(t)$ are much more substantial than the variations in frequency $\varepsilon _i(t)$. As we have indicated, it could be that $\varepsilon _\xi (t)$ itself should be taken to be proportional to one of the $A_i(t)$. Note that while our framework allows for a separation between deterministic oscillations and noise just like the traditional framework does, the classical PSD approach based on the traditional framework does not at all clearly detect the four oscillatory modes and risks leading to the whole signal being characterised as noise.

Finally, in Fig. 6, we analyse how the magnitude of the interplanetary magnetic field changes over time, measured at Lagrange point 1 in orbit of the Earth. This measures the varying strength of the solar wind, which causes the solar magnetic field to spread throughout the solar system. The wavelet transform in (c) in this case shows arguably the most complex behaviour of all three examples. There are two clear lower modes, of approximately 14.5 and 9.6 day periods, with clear time-variability. However, at shorter periods, the modes become so time-variable and similar that they are difficult to distinguish. Such a situation is another reason to next check for harmonic relations between potential modes using a harmonic finder, which will help to identify what the independent modes are, even despite their time-variability. Despite this complication of significant variability and intermittency, it remains the case that in this frequency region, the wavelet transform shows continuous deterministic modes that persist for hundreds of days. This is in contrast to the higher frequencies of the plot, which are hardly sustained at all and often spread their power vertically over many different frequencies. Both of these behaviours are indicative of temporary noisy fluctuations, and are not seen within the turquoise region. Hence, while identifying the precise dynamics in this regime is far from trivial, it is still safe to say that it is deterministic and time-varying.

The lower modes identified here can be seen from other solar wind wavelet analysis investigations, such as in [73]. The higher modes, however, are less evident and studied, with most attention being shown to much longer periods more on the timescale of the solar period of 11 years. In the bispectrum in (d), however, there is shown to be significant potential interactions between these faster modes, and even these modes and the slower ones, that could be worth further investigation. This is indicated by the many peaks of significance located within the turquoise region, and the few in the overlap between this and the purple and black regions. Indeed, while other wavelet-based approaches have been used, the use of PSDs to identify solar wind events and mechanisms is widespread [74,75,76]. In the case of Fig. 6b, we see once again that the PSD gives no indication of the complexities identified in the wavelet transform and bispectrum, instead presenting a profile very similar to pink noise. Hence, adopting a solely or predominantly PSD approach could again lead to missing potentially key aspects of solar wind dynamics.

Table 1 How different types of processes within a system may be identified as being deterministically functional versus being noise, when their manifestation as components in a time-series is analysed from the perspective of the classical framework versus from the perspective of our framework. As has been evidenced in this paper, typical power-spectral density methodology based on the classical framework may fail to distinguish the manifestation of prominent forced oscillatory processes in a time-series form the manifestation of classical noise-emergence mechanisms

We have given three examples of open systems where determinism can be missed or even misidentified as noise under the traditional PSD analysis. The time-dependent dynamics at the heart of this mischaracterisation is typical of thermodynamically open systems, where external forces influence the evolution of the system. These systems are the norm in the natural world, outside of any environment that is not artificially isolated. Therefore, it is essential that a suitably time-resolved framework of analysis is employed when analysing any such system. For one example from the field of biology, there has been much work to include noise in the Hodgkin–Huxley model on an understanding of the behaviour of cells as partially stochastic [10]. However, it may be the case that this complex dynamics could be produced by a model based in the time-dependent theory we have presented here, which may form both a more simple and experimentally accurate model. The advantages of this kind of model over high-dimensional autonomous ones in the context of cellular dynamics have already been demonstrated in [47].

5 Summary and conclusion

In this paper, we have shown that

• an ensemble of phase oscillators can produce a “$(1/f^\beta )$-noise-like” mean field time-series without the need for either a larger number of oscillators in the ensemble or the presence of chaotic dynamics, merely as a result of the presence of time-dependence in the parameters of the system (even if this time-dependence occurs on only relatively slow timescales);

• the same mischaracterisability of time-dependent, relatively non-complex, non-chaotic oscillatory dynamics as noise can occur in actual experimentally obtained time-series;

• in both cases, deterministic time–frequency analysis and related methods can be used to gain a picture of what is really going on in the behaviour of the system.

By “noise-like”, we mean that the typical basic method of investigating noise in a signal in terms of a roughly linear trend in a log–log plot of a PSD estimate—as used widely by practitioners across the sciences—will yield results that look like white ($\beta =0$) or pink ($\beta \in (0,2)$) noise. We have identified and discussed in detail the key mathematical assumptions underlying this approach to investigating noise in a signal, and the key physical assumptions giving rise to those mathematical assumptions, that are fundamentally responsible for the mischaracterisation of relatively simple deterministically functioning signal components as noise in the numerical and experimental scenarios considered in this paper. In essence, classical power-spectral density is a concept defined within the framework of stationary stochastic processes, whose application to time-series in turn presupposes that deterministic components in the time-series arise from autonomous dynamics, which itself in turn is the mathematical modelling assumption corresponding to the physical modelling assumption that the system does not have physical characteristics being modulated over time by external influences.

By contrast, systems throughout nature are thermodynamically open, exchanging matter and energy with their environment, and thus are bound to have temporally modulating characteristics. Starting from this observation, we have built a fairly general conceptual framework for describing the dynamics of thermodynamically open complex systems, in terms of non-autonomous finite-time oscillatory dynamics, represented by Eqs. (1)–(2). We have shown how deterministic time–frequency analysis can naturally arise as the appropriate methodology for analysing time-series generated by dynamics for which our framework is suitable; and we have both (a) used our framework to generate the numerical time-series in Sect. 3, and (b) in the inverse direction, used the experimental time-series in Sect. 4 to derive preliminary models within our framework, all towards illustrating what is learnt by deterministic time–frequency analysis that is profoundly obscured by classical PSD analysis (along with the entire rest of the toolbox of time-series analysis methods based on the assumption that the time-series comes from a stationary stochastic process). As discussed in some detail within Secs. 2.1 and 2.3, merely generalising the classical framework of parameter estimation for stationary stochastic processes to that of time-dependent parameter estimation for nonstationary stochastic processes is not the appropriate treatment for time-series coming from general open systems.

The contrast between how signal components may be characterised when analysed by traditional PSD analysis as motivated by the classical framework for dynamical systems and their output time-series, versus when analysed by time–frequency analysis as motivated by our framework of finite-time non-autonomous oscillatory dynamics, is summarised in Table 1. In conclusion, as we have evidenced through numerics and experimental time-series data, apparent “noise to be filtered out” in a signal may provide crucial insight into the deterministic properties of a system when analysed by methods that explicitly resolve the dimension of the progression of time such as time–frequency analysis and other methods described in, e.g., [54] and references therein.

Appendices

A: Further mathematical details of the two frameworks

The simplest kind of quantitative model of a deterministic continuous-time system is an autonomous differential equation

$$\begin{aligned} \dot{x}(t) = F(x(t)), \end{aligned}$$

(8)

where F is a deterministic vector field with no dependence on t. A time-series X(t) recorded from such a system can then be expected to take the form

$$\begin{aligned} X(t) = H(x(t)), \end{aligned}$$

where H is some observable function and x(t) is a solution of Eq. (8). Now, if we define the time “$t=0$” to be the start of the recording, then this time is typically random relative to the actual functioning of the physical system itself. Accordingly, if the solution x(t) being observed is modelled as having its long-time-asymptotic statistics as being given by some ergodic invariant probability measure $\mu$ of the system (8)—i.e., $\mu$ represents the probability distribution of where x(t) will be at a very large random time t ¹—then one may regard the initial condition x(0) as itself a random variable of distribution $\mu$, in which case the recorded time-series X(t) then becomes a sample realisation of an ergodic stationary stochastic process. The “stationarity” here means that the probabilistic law of the process is invariant under time-translations, while the “ergodicity” means that parameters of the probabilistic law of the process can be estimated just from the statistics of an individual sample realisation provided that the duration of the recorded segment of the sample is sufficiently long. In the case that x(t) is simply a stable periodic orbit, representing a fixed-frequency deterministic oscillatory process, the invariant measure $\mu$ is just the temporally uniform distribution along the orbit.

Now, one may assume that there is noise in either the behaviour of the system itself (“dynamical noise”) or the time-series measuring equipment (“observational noise”) or indeed both, so that the time-series X(t) takes a form such as (in the case of “additive noise”)

$$\begin{aligned} X(t)&= H(x(t)) + \xi _{\text {obs}}, \end{aligned}$$

(9)

$$\begin{aligned} \dot{x}(t)&= F(x(t)) + \xi _{\text {dyn}}. \end{aligned}$$

(10)

The deterministic component F in (10) still represents the functional component in the progress of the system, while both $\xi _{\text {obs}}$ and $\xi _{\text {dyn}}$ are the pseudorandom net result of background processes that affect the system or its recording. The observational noise can, once identified, simply be filtered out of the signal through linear filtering; by contrast, the dynamical noise is inseparable from the actual progression of the state of the system, leading to the branch of science and mathematics known as stochastic filtering [77]. Nevertheless, if the vector field F simply represents an oscillatory process (i.e., has a stable limit cycle) and the noise is not too strong, one can expect a basic linear frequency decomposition of X(t) to yield a clear distinction between background noise (whether observational or dynamical or both) and a relatively narrow peak representing the approximate natural frequency of the oscillatory process.

Now, if one assumes that there is noise, then one can usually expect little further harm to be done by assuming, as a reasonable approximation, that this noise is a stationary noise process (again, meaning that its law is invariant under time-translations). Hence, if the time-series X(t) is assumed to arise from a model such as (9)–(10), then it will often be natural to treat this time-series as a finite-time segment of a sample realisation of an ergodic stationary stochastic process.

Accordingly, the main traditional framework of time-series analysis—upon which a vast range of time-series analysis methods is based—is indeed to

• regard the time-series as a finite-time segment of a sample realisation of an ergodic stationary stochastic process,

• and view the aim of time-series analysis as being to estimate parameters of the probabilistic law of this underlying stochastic process.

(Such parameters can be single numbers, such as mean and standard deviation, but can also include functions of one or more variables, such as power-spectral density, which is a function of frequency.)

1.1 Noise in the classical framework

Throughout various disciplines in the natural and social sciences, when one records a time-series $(X(t): t \in [0,T])$, it is common to investigate the presence of deterministic oscillations and/or noise through computation of the squared magnitude of the Fourier transform

$$\begin{aligned} E_X(f) = \left| \int e^{2\pi i ft} \tilde{X}(t) \, {\text {d}}t \right| ^2, \end{aligned}$$

(11)

where $\tilde{X}$ is obtained from the original X through some combination of detrending and padding. Then:

• Peaks that stand out at a set of isolated frequencies are regarded as representing deterministic oscillatory processes either at those frequencies or at fundamental frequencies of which those frequencies appear as harmonics due to non-linearities.

• In a log–log plot of $E_X(f)$ against f, roughly linear trends across a large range of frequencies are taken as representing noise of a “colour” determined by the gradient of the trend.

The theory of stationary stochastic processes gives rise to the defining of various kinds of stationary noise—in particular, of various “colours” as defined by their power-spectral density (PSD). Roughly speaking, the PSD of a stationary stochastic process $(X(t):t \in \mathbb {R})$ is the function $\mathcal {P}_X$ defined on the frequency axis by

$$\begin{aligned} \mathcal {P}_X&= \lim _{T \rightarrow \infty } \mathcal {P}_X^{(T)} \\ \mathcal {P}_X^{(T)}(f)&= \mathbb {E}\!\left[ \frac{1}{T} \left| \int _0^T e^{2\pi i f t} X(t) \, {\text {d}}t \right| ^2 \right] . \end{aligned}$$

The process X is considered “white noise” if $\mathcal {P}_X$ is constant, while it is considered “pink noise” if $\mathcal {P}_X(f) \propto 1/f^\beta$ for some $\beta \in (0,2)$, and “blue noise” for some $\beta \in (-2,0)$. Such “1/f” behaviour has been observed in diverse processes [78,79,80,81,82,83], especially biological and electronic, and questions regarding the origin of this behaviour have been much-researched [35]. Assuming sufficient ergodicity properties, one can estimate the PSD from a single time-series recording, provided that the recording is of sufficiently long duration. In practice, a typical procedure is to compute the quantity inside the expectation in the definition of $\mathcal {P}_X^{(T)}$—namely

$$\begin{aligned} \frac{1}{T} \left| \int _0^T e^{2\pi i f t} X(t) \, {\text {d}}t \right| ^2, \end{aligned}$$

where $(X(t):t \in [0,T])$ is the actual time-series recording, with T the total duration of the recording (or one can first detrend and add some padding for the sake of more effective Fast Fourier Transform computation). In other words, up to a constant time-normalisation, the function $E_X$ as defined in Eq. (11) is taken as the PSD estimate; noise in the signal is then investigated by looking for a roughly linear trend in the graph of $E_X$ plotted on a log–log plot, with the $\beta$-value for the $1/f^\beta$ law of the noise being estimated by the negated gradient of a best-fit straight line.

An expert statistician may prefer somewhat more advanced PSD estimators than simply taking the time-normalised deterministic energy-spectral density $E_X$ of the entire signal recording $(X(t):t \in [0,T])$. Nonetheless, this simple approach is already perfectly reasonable as a PSD estimator: second-order spectra and second-order correlations are Fourier transform pairs (up to normalisation), and ergodicity implies that the sample autocorrelation function for the recorded time-series is a good approximation of the underlying “population” autocorrelation function for the stochastic process from which the recording is sampled.

There also exist other methods for experimental analysis of noise besides PSD estimation, such as autocorrelation functions (the Fourier counterpart to PSD) and detrended fluctuation analysis [84]. However, in this paper, we apply PSD estimation for the analysis of ‘noise’, as it is generally the most standard and widely applied method for this purpose.

1.2 Finite-time dynamics and nonstationary stochastic processes

• If the system being modelled is open to freely time-varying influence from its environment, then there is no reason at all to expect the t-dependence in $f_i(t)$ or $F_i(\omega ,\theta _1,\ldots ,\theta _n,t)$ to follow any indefinite-time pattern such as periodicity or almost-periodicity. Thus, our framework is to be understood as a framework for studying dynamics on explicitly finite-time scales, in contrast to the classical theory of autonomous dynamical systems where “qualitative analysis of dynamics” is understood in terms of coordinate-invariant long-time-asymptotic properties. Indeed, in our framework, the differential equations describing the cyclic processes $\theta _1,\ldots ,\theta _n$ need not be well defined on infinite time at all: outside the assumption of a specified form of time-dependence, a non-autonomous dynamical system can perfectly well be defined to exist only on a bounded time-interval [42, 46], just as real physical systems often only exist and operate on bounded time intervals. Such dynamical systems are called finite-time dynamical systems.

• If the time-dependence does strictly follow some indefinite-time pattern, such as periodicity or almost-periodicity, then one can extend the phase space to incorporate the forcing (physically corresponding to “taking the system and its external forcing as one larger system”), so as to give an autonomous dynamical system. A common misconception is that even outside such assumptions on the time-dependence, simply adding time itself as a new dimension in the phase space enables non-autonomous dynamical systems to be reduced to the domain of applicability of autonomous dynamical systems theory. However, the problem with this is that the theory of autonomous dynamical systems concerns bounded invariant objects (e.g., fixed points, periodic orbits, compact attractors and their natural invariant measures, etc.) rather than being concerned with systems whose state grows unboundedly; obviously, if time itself is regarded as part of the state of the system, then this “state” will grow unboundedly [85, Remark 2.5].

• We have explained that for autonomous dynamical systems, the time-independence naturally gives rise to the applicability of stationary stochastic processes theory for the analysis of a time-series, even in the complete absence of noise. However, for a non-autonomous dynamical system, changing the time of recording will change the actual law of the system relative to the time of recording, and so there would generally be no analogous logic justifying the use of nonstationary stochastic processes theory to describe a time-series taken from a deterministic non-autonomous dynamical system. When there is also noise present, then stochastic processes theory may be appropriate for studying an isolated noise component; but as we have already said, seeking to apply a framework of time-dependent parameter estimation for nonstationary stochastic processes to the actual time-series recording X(t) would be problematic.

1.3 Linking to deterministic linear time–frequency analysis

Linearity implies that, first in the absence of coupling (i.e., $C_i=0$) and with $h_i$ being sinusoidal, the individual oscillations

$$\begin{aligned} A_i(t)\sin \!\left( \theta _i(0)+2\pi \int _0^t f_i(s) \, {\text {d}}s \right) \end{aligned}$$

will appear as single modes of time-dependent frequency. Then,

• if $h_i$ is non-sinusoidal, the ith oscillator will contribute a “non-linear” oscillatory component

  $$\begin{aligned} A_i(t)h\!\left( \theta _i(0)+2\pi \int _0^t f_i(s) \, ds \right) \end{aligned}$$

  to the time-series, appearing in the linear time–frequency analysis as a fundamental mode plus harmonics that are multiples of the fundamental mode;

• if there is coupling, then as we have described, the presence of this coupling not only affects the frequency with which the individual oscillators complete their cycles, but also distorts the relative rates at which different parts of the cycle are progressed through—this makes the oscillations “non-linear”, again resulting in the presence of harmonics in the linear time–frequency analysis.

Consequently, the presence of coupling can be investigated through higher order spectral analysis such as bispectral analysis (which will also detect individual non-linear oscillatory components even without coupling). But once again, under our framework, one should not use the classical definitions defined for stationary stochastic processes (nor even generalised versions for time-locally stationary stochastic processes), but should use higher order spectra derived from deterministic time–frequency analysis [59, 64]. To infer more about the causality and origins of couplings identified by bispectral analysis, additional analysis techniques such as dynamical Bayesian inference (see, e.g., [86] and the references therein) can also be used.

The Heisenberg uncertainty principle also plays an important role in time–frequency analysis: in the time–frequency analyser, one must decide upon a suitable trade-off between precision of the frequency axis and precision of the time-axis; an arbitrarily high-precision description in time–frequency space of the time-localised frequency content of a signal is conceptually impossible.

Let us return to our earlier point that in generic deterministic non-autonomous dynamical systems, there is no basis for describing solutions in terms of nonstationary stochastic processes. Traditional time-series analysis is essentially centred around the concept of estimation of parameters from sample data, either time-independent if the time-series is assumed to come from a stationary process or time-dependent if the time-series is assumed to come from a nonstationary process. However, time-series analysis methodologies suited to our framework, such as deterministic time–frequency analysis, do not seek to “estimate” any quantities associated with some “population” of which the time-series is considered a “sample”. In the mathematics of deterministic time–frequency analysis, time itself plays a somewhat mathematically analogous role to “the underlying probability space” in the traditional theory of stochastic processes. In other words, in deterministic time–frequency analysis, the actual recorded time-series is its own population. (Deterministic time–frequency analysers can also be used to define time-dependent parameters of nonstationary stochastic processes, e.g., “wavelet power” defined as the expected squared magnitude of the continuous wavelet transform at a given frequency. However, as we have already indicated, in the absence of an assumption on the specific form that the nonstationarity takes or an assumption of extremely low variance, the ability to estimate such time-dependent parameters from a single sample realisation would require such slow variation in the time-localised frequency content that one could simply instead apply classical estimation of time-independent parameters to large-duration segments of the signal.)

Further numerical details of the two frameworks

Fig. 7

The production of noise-like behaviour by aperiodic deterministic oscillations. One non-autonomous oscillator with an aperiodically modulated natural frequency, simulated over a 200,000 s period, is analysed in its power-spectral density and wavelet transform. The wavelet transform is constructed using the lognormal wavelet with frequency resolution 2. The analysed signal is $X(t)=\sin (\theta (t))$ where $\theta (t)$ evolves according to the equation $\frac{1}{2\pi }\dot{\theta }(t)=f(t)$ with $f(t)=5.5$ Hz $+4.51$ Hz $\sin \left( 2\pi 10^{-2}t + \sin (0.13t)\right)$

Next, we demonstrate that this phenomenon is not exclusive to periodic time-dependence of the oscillators’ internal frequencies, but can readily occur for aperiodic forms of time-dependence. In Fig. 7 we analyse a single ($n=1$) aperiodically frequency-modulated oscillator, demonstrating that the power-spectral density is still noise-like. Once again, the wavelet transform resolves the true nature of the signal as being not at all like “noise” but rather a simple oscillation whose frequency is gradually modulated, with the form of frequency modulation also clearly seen in the wavelet transform.

Fig. 8

The amount of time each model requires to obtain convergence of the gradient of the PSD. The $\beta$ factor of the $1/f^{\beta }$ gradient of the PSD of a model is calculated from time-series of a range of lengths, $t_f$. The model to which each plot refers is indicated by the legend specifying the figure in which that model was first introduced

As discussed in Sect. 1, the theoretical power-spectral density is defined over an infinite time span. Thus, an approximation of the “true” PSD must be calculated over finite time, but this approximation becomes less accurate the shorter the finite-time interval. In Fig. 8, we examine the length of time required to reach a convergent gradient of the power-spectral density for every model considered in this section; the largest value shown on the time-axis in each of the plots in Fig. 8 is equal to the simulation time that was used for the PSD estimation in the corresponding figure in this section where the model was analysed (which, as we have said, corresponds to around the longest time computationally feasible). We see in Fig. 8 that for most cases, this time is sufficient to reach convergence. Therefore, the mischaracterisation of relatively simple deterministic but time-dependent dynamics as noise-like is not based on insufficient accuracy of empirical PSD estimators, but is based truly on the concept itself of using a linear trend in a log–log plot of PSD as an indicator of noise when the signal comes from a system subject to time-dependent influences in a manner that makes the framework of stationary stochastic processes inappropriate. Rather than seeking to gain a better understanding of the system by improving the supposed precision or accuracy of PSD estimation, instead, to gain even a basic correct understanding of the system, one must incorporate the time dimension explicitly into the analysis, as exemplified in the wavelet transform.

Fig. 9

The role of the frequency resolution in the wavelet transform, using the lognormal wavelet. Considered for the aperiodic model of Fig. 7. In order, from left to right along the upper row and then left to right along the lower row, a frequency resolution of $R=0.5, 1, 1.5, 2.5, 3, 3.5$ was used, rather than $R=2$ as in Fig. 7

Finally, in this subsection, we demonstrate the practical implications of the Heisenberg uncertainty principle, as discussed in Sect. 2.3, for the wavelet transform. Specifically, for this analysis method, the frequency resolution parameter determines the trade-off between precision in the time-domain and the frequency-domain. In Fig. 9, we see that for the highest frequency resolution considered, the higher frequencies are finely resolved in time–frequency space, while the lower frequencies have a more blurry representation in the time-domain; then, as the frequency resolution is decreased, the lower frequencies become less blurry in the time-domain, but the higher frequencies start to become more blurry in the frequency domain; and at the lowest frequency resolution values, all frequencies are very poorly resolved in the frequency-domain. This illustrates how, in time–frequency analysis, the parameter controlling the trade-off between time localisation and frequency resolution needs to be carefully optimised, on a case-by-case basis, to the combination of how quickly the time-dependent characteristics of oscillations change and how large the distance in frequency is between different oscillatory contributions that one seeks to resolve; otherwise, the number and the nature (e.g., noise versus oscillation) of the components present in the signal may be unnecessarily misidentified.

1.1 Entropy and autocorrelation

What we call the “traditional framework of time-series analysis”, as described in Sect. 2, can be summarised as the approach in which one regards a recorded time-series as being a finite-duration segment of a sample realisation of an ergodic stationary stochastic process, and uses this recording to seek to estimate statistical parameters or properties of the underlying stochastic process so as to help learn more about the physical system from which the time-series was measured. PSD analysis is a common tool for understanding noise that is based on the traditional framework; but as we have said, the full “toolbox” of time-series analysis methods based on the traditional framework is vast. Here, we now illustrate how including more of these methods in the analysis of a time-series does not help to gain an understanding of the underlying behaviour of the system if there is significant time-dependence in the properties of oscillatory components of the time-series, and hence, time-resolved methods such as the wavelet transform and other methods described in [54] are still needed. Specifically, as two examples of well-known analysis tools that are intended to gain certain types of information about a time-series—and yet which are ultimately based on the traditional framework and do not resolve in time—we will consider first multiscale sample entropy [87] and second autocorrelation (the Fourier counterpart to PSD).

Entropy measures are intended to give a quantification of the “complexity” of a time-series. Multiscale sample entropy is particularly designed to avoid characterising randomness as inherently a behaviour of high complexity, such that, for example, stationary white noise will appear as being of relatively low “complexity” compared to other forms of complex-looking behaviour [87,88,89]. Multiscale sample entropy generalises the established sample entropy calculation to multiple time scales, identifying over which of these the time-series may be more or less complex.

Fig. 10

Multiscale sample entropy of autonomous and non-autonomous oscillator ensembles. In a, the autonomous models in Fig. 2a; in b the same models as in (a) but with the addition of a random number between 0 and 1 Hz to each frequency $f_{0i}$; in c, the non-autonomous models in Fig. 2b; and in d, the same models as in c but with the addition of a random number between 0 and 1 Hz to each centre frequency $f_{0i}$. In each case, the multiscale sample entropy is calculated for the time-series simulated up to a duration of $T=200$ s. The parameters of the multiscale sample entropy are $m=2$ and $r=0.15$

Fig. 11

Autocorrelation of autonomous and non-autonomous oscillator ensembles analysed in Fig. 10 with $n=10$. Each plot corresponds to the same model as that of the same letter in Fig. 10. The window size is a 0.028 s, b 0.044 s, c 0.034 s, and d 0.044 s, in each case equal to the greatest entropy peak in the corresponding model in Fig. 10

To define multiscale sample entropy, we first need to define sample entropy [90], which is defined for discrete-time signals $(x_n)_{n=1,\ldots ,N}$. Heuristically, for a fixed positive-integer parameter m, the sample entropy is the negative of the natural logarithm of the probability that two randomly selected distinct length-$(m+1)$ segments of the signal will share approximately the same $(m+1)$th entry conditional on the event that they share approximately the same first m entries; thus, a larger value of the sample entropy corresponds to the time-series being “less predictable” and thus “more complex”. To be more precise: given a signal length N, an integer $m \in \{1,\ldots ,N-1\}$ (often chosen as $m=2$) and a “tolerance” parameter $\rho >0$, letting $C_{N-m}$ be the set of all pairs (i, j) of integers i and j with $1 \le i < j \le N-m$, the sample entropy of a signal $(x_n)_{n=1,\ldots ,N}$ is defined as

$${\textrm{SampEn}}(m,\rho ,N,(x_n)_{n=1,\ldots ,N}) := \log \frac{\#\{(i,j) \in C_{N-m}: d[ \, (x_i,\ldots ,x_{i+m-1}) \,, \, (x_j,\ldots ,x_{j+m-1}) \, ]< \rho \}}{\#\{(i,j) \in C_{N-m}: d[ \, (x_i,\ldots ,x_{i+m}) \ \, \ \ (x_j,\ldots ,x_{j+m}) \, ] < \rho \}},$$

where d is a function for measuring distances between vectors (in the calculation of Fig. 10, we use Euclidean distance). From a statistical point of view within the above-described traditional framework of time-series analysis, as in [90], the quantity $\textrm{SampEn}(m,\rho ,N,(x_n)_{n=1,\ldots ,N})$ is a statistic seeking to estimate—for the underlying ergodic stationary stochastic process $(X_n)_{n \in \mathbb {Z}}$ from which the time-series $(x_n)_{n=1,\ldots ,N}$ is assumed to arise as a sample—the value of the parameter

$$\begin{aligned} \lim _{N \rightarrow \infty } \mathbb {E}[{\textrm{SampEn}}(m,\rho ,N,(X_n)_{n=1,\ldots ,N})]. \end{aligned}$$

Whereas the sample entropy of a discrete-time signal is a single number, the multiscale sample entropy of a discretely sampled signal is a function whose input variable is the “time scale” s under consideration; the corresponding output is the sample entropy of the sequence of mean averages of consecutive time-windows of duration s. To be precise: for a discretely sampled signal $(x_t)_{t = \frac{1}{f_{\text {s}}},\frac{2}{f_{\text {s}}},\ldots ,\frac{f_{\text {s}}T-1}{f_{\text {s}}},T}$ of duration T and sampling frequency $f_{\text {s}}$, fixing a positive-integer parameter m and a parameter $r>0$ and setting $\rho =r\sigma$, where $\sigma$ is the standard deviation of the set of all points $x_{\frac{1}{f_{\text {s}}}},x_{\frac{2}{f_{\text {s}}}}\ldots ,x_T$ in the signal, the multiscale sample entropy associates with each “time scale” $s>0$ (where s is a multiple of $\frac{1}{f_{\text {s}}}$ that is much smaller than T) the sample entropy of the signal $(y_n)_{n=1,2,\ldots ,\lfloor \frac{T}{s} \rfloor }$ where $y_n$ is the mean over the nth time window of length s, that is

$$\begin{aligned} y_n = \frac{f_{\text {s}}}{s} \sum _{i=1}^{\frac{s}{f_{\text {s}}}} x_{(n-1)s + \frac{i}{f_{\text {s}}}}, \end{aligned}$$

and $\lfloor \cdot \rfloor$ is the rounding-down operation.

Now, for sample entropy, it is advised that one ensure a signal length N of at least $10^m$. For multiscale sample entropy, since a larger scale s corresponds to a smaller number of points $\lfloor \frac{T}{s} \rfloor$ in the downsampled signal for which the sample entropy is calculated, it is necessary to limit the scale s to achieve the above advice. Accordingly, in this paper, we present the multiscale sample entropy up to the maximum such permissible scale.

In Fig. 10, we present the multiscale sample entropy of models considered in the previous sections, and modifications thereof. Figure 10(a) shows results for the autonomous cases analysed in Fig. 2(a), and Fig. 10(c) shows results for the non-autonomous cases analysed in Fig. 2(b). The results in plot (a) are of a qualitatively fairly similar nature to those in plot (c), apart from the relatively sharp drops in entropy as a function of scale seen at certain scale values in the $n=10$ case. There is no indication from these plots that multiscale sample entropy can be used to help distinguish time-independent (i.e., autonomous) from time-dependent (i.e., non-autonomous) dynamics.

Fig. 12

The multiscale sample entropy, with $m=2$ and $r=0.15$, of the time-series analysed in a Fig. 4, b Fig. 5, and c Fig. 6

However, perhaps not too much should be deduced from plots (a) and (c) due to the rational ratios between the frequencies $f_i(t)$ of the different oscillators present in the model, which could significantly affect the entropy analysis. Accordingly, just as we added random numbers to the centre frequencies to reduce rational relationships when considering bispectra, so likewise now in plots (b) and (d) of Fig. 10, we show results for the same models as in plots (a) and (c), respectively, except with the addition of random numbers between 0 and 1 Hz to the centre frequencies $f_{0i}$ of the oscillators. Now, in this more generically representative scenario, we actually see results indicating similar conclusions to what we saw in Fig. 2 when considering power-spectral density: The sample entropy as a function of scale shows a similar picture to that seen for white noise in Fig. 3 of [87] and Fig. 1 of [89]; but,

• in the autonomous case shown in plot (b), for the relatively low-dimensional case $n=10$, the deviations away from the monotonically decaying “noise-like” curve are fairly clear, while for larger n, the result more closely follows a monotonically decaying curve resemblant of “noise-like” behaviour;

• however, in the non-autonomous case shown in plot (d), $n=10$ is already sufficient to follow remarkably closely the “noise-like” picture.

Therefore, whereas the cases shown in plots (a) and (c) merely indicate the insufficiency of multiscale sample entropy to be able to distinguish autonomous from non-autonomous dynamics, the more generically representative results shown in plots (b) and (d) indicate not only the insufficiency of multiscale sample entropy in this regard (since the results in (b) and (d) are quite similar to each other) but furthermore the potential for positive mischaracterisation of relatively low-dimensional deterministic behaviour as noise, just as has been seen for power-spectral density.

Autocorrelation analysis is also a widely applied time-series analysis method, that describes the “statistical memory” of a signal across different time scales. The sample autocorrelation function (which seeks to estimate the “population” autocorrelation function of the ergodic stationary stochastic process from which the time-series is assumed to be a sample) calculates the correlation between values in the time-series separated in time by a given “lag” (the input variable of the autocorrelation function). Here, we work with a slight adaptation of the standard definition of sample autocorrelation, that sometimes gives more suitable results that the standard definition: for a discretely sampled signal $(x_t)_{t = \frac{1}{f_{\text {s}}},\frac{2}{f_{\text {s}}},\ldots ,\frac{f_{\text {s}}T-1}{f_{\text {s}}},T}$ of duration T and sampling frequency $f_{\text {s}}$, we fix a window-length $L \in (0,T)$ independent of the lag under investigation, and then associate with each lag $k \in [-\frac{T-L}{2},\frac{T-L}{2}]$ (where k is a multiple of $\frac{1}{f_{\text {s}}}$) the cosine of the angle between the vector $(x_{\frac{i}{f_{\text {s}}}})_{\frac{i}{f_{\text {s}}} \in [\frac{T-L}{2},\frac{T+L}{2}]}$ and the vector $(x_{\frac{i}{f_{\text {s}}}+k})_{\frac{i}{f_{\text {s}}} \in [\frac{T-L}{2},\frac{T+L}{2}]}$, as calculated by the normalised dot product. (The difference from the usual definition of the sample autocorrelation function is as follows: in the usual definition, for each lag k, one considers correlation between the time-windows $[0,T-|k|]$ and [|k|, T], whereas, in our approach, one considers the correlation between the time-windows $[\frac{T-L}{2},\frac{T+L}{2}]$ and $[\frac{T-L}{2}+k,\frac{T+L}{2}+k]$.)

In Fig. 11, we calculate the autocorrelation of the models in Fig. 10 with $n=10$, using a window size L equal to the time scale at their greatest entropy peak. We see that plots (a) and (c) slightly reveal a qualitative difference, namely a constant lower envelope in the autonomous case and a slightly variable lower envelope in the non-autonomous case, but these observations are peculiar to the precise setup considered in (a) and (c) where the frequencies $f_i(t)$ have rational ratios; once we move to the much more generically representative cases in plots (b) and (d), we see no clear qualitative difference between plots (b) and (d) that might indicate that one is generated by time-independent dynamics and the other by time-dependent dynamics. In conclusion, as is to be expected, there is no clear way in which non-time-resolved autocorrelation functions can be used to help distinguish between time-independent and time-dependent dynamics.

Just like power-spectral density, the two analysis methods that we have considered in this subsection do not resolve the time dimension: just as classical power spectra are defined over the frequency axis without an axis for the progression of time, so likewise the multiscale sample entropy and the autocorrelation are defined as a function of a variable representing a “width of time-gap”, while still effectively treating the actual progression of time as of no more significance than an atemporal set of statistical re-trials (the essence of ergodicity). Accordingly, as we have evidenced, they cannot be expected to provide a suitable methodology for gaining even a basic understanding of the behaviour of systems subject to temporal modulation of characteristics (such as frequency of oscillatory components)—nor indeed can any other tools based on the classical framework of time-series analysis. Instead, time-resolved methodologies, such as deterministic time–frequency analysis, are truly necessary for successfully analysing signals recorded from such systems.

In Fig. 12, we consider the multiscale sample entropy of the experimental time-series analysed in Sect. 4. This indicates that our three examples, in the order that we have presented them, are of increasing complexity; and this would seem to agree with the pictures indicated by the wavelet transforms in Figs. 4, 5, 6. Still, we would not have been able to arrive at a basic description of what is going on in each of these examples if we had not employed time-resolved methods such as the wavelet transform and had instead restricted ourselves to classical-framework methodologies, such as power-spectral density and non-time-resolved statistical entropy measures.