Attractor and saddle node dynamics in heterogeneous neural fields

Abstract

Background

We present analytical and numerical studies on the linear stability of spatially non-constant stationary states in heterogeneous neural fields for specific synaptic interaction kernels.

Methods

The work shows the linear stabiliy analysis of stationary states and the implementation of a nonlinear heteroclinic orbit.

Results

We find that the stationary state obeys the Hammerstein equation and that the neural field dynamics may obey a saddle-node bifurcation. Moreover our work takes up this finding and shows how to construct heteroclinic orbits built on a sequence of saddle nodes on multiple hierarchical levels on the basis of a Lotka-Volterra population dynamics.

Conclusions

The work represents the basis for future implementation of meta-stable attractor dynamics observed experimentally in neural population activity, such as Local Field Potentials and EEG.

Background

Neural field models, such as [1, 2] are continuum limits of large-scale neural networks. Typically, their dynamic variables describe either mean voltage [2] or mean firing rate [1, 3] of a population element of neural tissue (see [4, 5] for recent reviews).

The present article considers the paradigmatic Amari equation [2] describing the spatiotemporal dynamics of mean potential V(x, t) over a cortical d-dimensional manifold $\Omega \subset {ℝ}^d$:

$$\begin{array}{l}\frac{\mathrm{\partial V}(x,t)}{\mathrm{\partial t}}=-V(x,t)+{\int }_{\Omega }K(x,y)S\left[V\right(y,t\left)\right]\mathrm{d}y,\end{array}$$

(1)

where K(x, y) is the spatial synaptic connectivity between site y ∈ Ω and site x ∈ Ω, and S is a nonlinear, typically sigmoidal, transfer function. This model neglects external inputs for simplicity but without constraining the generality of the subsequent results. Possible synaptic time scales are supposed to be included in the kernel function K and can be introduced by a simple scaling of time.

In general, the connectivity kernel K(x, y) fully depends on both sites x and y, which case is referred to as spatial heterogeneity. If the connectivity solely depends on the difference between x and y, i.e. K(x, y) = K(x - y), the kernel is called spatially homogeneous[2]. Furthermore, if the connectivity depends on the distance between x and y only, i.e. K(x, y) = K(||x - y||), with ||x|| as some norm in Ω, the kernel is spatially homogeneous and isotropic[6].

Spatially homogeneous (respectively isotropic) kernels have been intensively studied in the literature due to their nice analytical properties. In this case, the evolution equations have exact solutions such as bumps [2, 7], breathers [8–10] or traveling waves [7, 11]. Moreover, such kernels allow the application of the technique of Green’s functions for deriving partial neural wave equations [7, 12, 13].

The present work focuses on spatially heterogeneous neural fields which have been discussed to a much lesser extent in previous studies than homogeneous neural fields [14–22]. This study resumes these attempts by investigating stationary states of the Amari equation (1) with heterogeneous kernels and their stability. Such a theory would be mandatory for modeling transient neurodynamics as is characteristic, e.g., for human cognitive phenomena [23], human early evoked potentials [24] or, among many other phenomena, for bird songs [25].

The article is structured in the following way. In the “Results” section we present new analytical results on stationary solutions of the Amari equation (1) and their stability in the presence of heterogeneous connectivity kernels. Moreover, we present numerical simulation results for the kernel construction and its stability analysis. The “Methods” section is devoted to construct such kernels through dyadic products of desired stationary states (cf. the previous work of Veltz and Faugeras [26]). A subsequent linear stability analysis reveals that these stationary solutions could be either attractors or saddles, depending on the chosen parametrization. Finally, we present a way to connect such saddle state solutions via heteroclinic sequences [27, 28] in order to construct transient processes.

Results

In this section we present the main results of our study on heterogeneous neural fields.

Stationary states and their stability

Analytical study

The Amari equation (1) has a trivial solution V₀(x) = 0 and non-trivial solutions V₀(x) ≠ 0 that obey the Hammerstein integral equation [29]

$$\begin{array}{l}{V}_0\left(x\right)={\int }_{\Omega }K(x,y)S\left[V_0\right(y\left)\right]\mathrm{d}y.\end{array}$$

(2)

Inspired by Hebbian learning rules for the synaptic connectivity kernel K(x, y) which found successful applications, e.g., in bi-directional associative memory [30], we consider symmetric spatially heterogeneous kernels K(x, y) = K(y, x) that can be constructed from dyadic products of the system’s non-trivial stationary states

$$\begin{array}{l}K(x,y)=(V_0\otimes V_0)(x,y)=V_0\left(x\right)V_0\left(y\right).\end{array}$$

(3)

Together with Eq. (2), this choice yields the additional condition for non-trivial stationary states

$$\begin{array}{l}1={\int }_{\Omega }{V}_0\left(y\right)S\left[V_0\right(y\left)\right]\mathrm{d}y,\end{array}$$

(4)

which is a nonlinear integral equation of Fredholm type. Since 0 < S(x) < 1 for a logistic transfer function, a necessary condition for non-trivial stationary states is

$$\begin{array}{l}1<{\int }_{\Omega }{V}_0\left(y\right)\mathrm{d}y\end{array}$$

(5)

which indicates immediately a method to find a non-trivial solution numerically as shown below.

Small deviations u(x, t) = V(x, t) - V₀(x) from a non-trivial stationary state V₀(x) obey the linear integro-differential equation

$$\begin{array}{lcr}\frac{\mathrm{\partial u}(x,t)}{\mathrm{\partial t}}& =& -u(x,t)+{\int }_{\Omega }L(x,y)u(y,t)\mathrm{d}y,\hfill \end{array}$$

(6)

where $L(x,y)=K(x,y)S^{\prime }\left(y\right)$ and $S^{\prime }\left(y\right)=\mathrm{d}S\left[V_0\right(y\left)\right]/\mathrm{d}{V}_0\left(y\right)$. A linear stability analysis of (6) carried out in the next section shows that a non-trivial stationary state V₀(x) is either a fixed point attractor, or (neglecting a singular case) a saddle with one-dimensional unstable manifold. Such saddles can be connected to form stable heteroclinic sequences.

Numerical study

To gain deeper insight into possible stationary solutions of the Amari equation (1) and their stability, the subsequent section presents the numerical integration of equation (1) in one spatial dimension for a specific spatial synaptic connectivity kernel.

Since previous experimental studies [31] have revealed Gaussian distributed probability densities of neuron interactions in the visual cortex of rats, it is reasonable to look for spatially discretized stationary states in the family of Gaussian functions

$$\begin{array}{l}{V}_{0,i}=W_{0}exp\left(-{(i-i_0)}^2{\Delta }^{2}x/2{\sigma }^2\right)/\sqrt{2\pi }\sigma +\kappa {\eta }_i\end{array}$$

(7)

parameterized by the amplitude W₀, the variance σ², the noise level κ and the spatial discretization interval Δ x.

By virtue of this parametrization of the discrete Hammerstein equation, it is sufficient to fit the model parameters optimally in such a way that the Hammerstein equation holds. Figure 1(a) illustrates a noisy kernel K and (b) shows the corresponding stationary state V₀(x) for certain parameters.

Figure 1

Illustration of (a) a noisy kernel K ( x , y ) and (b) the corresponding solution of the Hammerstein equation (solid line) together with the first eigenmode e ₁ ( x ) corresponding to the eigenvalue with the maximum real part ε ₁ . Parameters are α = 0.86, W₀ = 1.76, σ = 0.15, θ = 3.0, κ = 0.2 and n = 200, L = 1.

For each stationary state, one obtains a kernel L(x, y) of the linear stability analysis whose spectrum characterizes the stability of the system in the vicinity of the stationary state. If the eigenvalue with the maximum real part ε₁ > 1, then the stationary state V₀(x) is exponentially unstable whereas ε₁ < 1 guarantees exponential stability. Figure 1(b) shows the eigenmode e₁(x) corresponding to the eigenvalue with maximum real part which has a similar shape as the stationary state.

Moreover Figure 2 presents parameters for which V₀(x) fulfills the Hammerstein equation (2), i.e. for which V₀(x) is the stationary solution. We observe that some parameter sets exhibit a change of stability, i.e. the eigenvalue with maximum real part may be ε₁ > 1 or ε₁ < 1 for certain parameter subsets.

Figure 2

Parameters which guarantee the solution of the Hammerstein equation ( 2) which are the stationary solutions of Eq. ( 1) and the line styles encode their exponential stability (stable: solid line, unstable: dashed line). Further parameters are κ = 0.0,θ = 3.0.

Taking a closer look at the stability of V₀(x), the computation of the eigenvalues ε _k , k = 1, …, n reveals a dramatic gap in the spectrum: the eigenvalue with maximum real part ε₁ is well isolated from the rest of the spectrum $\left\{{\epsilon }_{k>1}\right\}$ with |ε_k>1| < 10^-14. This is in accordance to the discussion of Eq. (17) on the linear spectrum.

Figure 3 presents the spatiotemporal evolution of the heterogeneous neural field starting close to the stable stationary state V₀(x), see point (1) in Figure 2. As expected, the field activity remains in the vicinity of the stable state.

Figure 3

Numerically simulated spatio-temporal dynamics of the heterogeneous neural field (left) and stationary states (right). Here the stationary state V₀(x) is exponentially stable with the parameters of point (1) in Figure 2. The simulation starts close to the stationary state V(x,0) = V₀(x) + ξ(x) with the random numbers ξ(x) taken from a zero-mean Gaussian distribution with variance 0.3. The plot on the left hand side shows the deviation from the stationary state V₀(x). The plot on the right hand side shows the stationary states V₀(x) (solid line) and the final field activity at large times V(x,t = 125) (dashed line) which is almost identical to V₀(x). The spatial domain has length L = 1 with n = 300.

In contrast, for the system starting close to an unstable stationary state, cf. point (2) in Figure 2, the field activity moves away from V₀(x) and approaches a new stationary state close to but different from V₀(x), cf. Figure 4. This new stationary state obeys the Hammerstein equation (2).

Figure 4

Numerically simulated spatio-temporal dynamics of the heterogeneous neural field (left) and stationary states (right). Here the stationary state V₀(x) is exponentially unstable with the parameters of point (2) in Figure 2. The plot on the left hand side shows the deviation from the stationary state V₀(x), the right hand side panel presents the stationary states V₀(x) (solid line) and the final field activity at large times V(x,t = 125) (dashed line) which is close to V₀(x). Other parameters are taken from Figure 3.

Recalling the presence of the trivial stable solution V = 0, the activity shown in Figure 4 indicates the bistability of the system for the given parameter set.

Figure 5 supports this bistability for the same parameter set but different initial conditions, which presents the jump from the unstable stationary state V₀(x) to the trivial stable stationary state V = 0. The choice whether the system approaches the upper or lower stable stationary state depends on the initial condition of the simulation and is random for random initial conditions as implemented in Figures 3, 4 and 5. Hence, this example reveals the existence of a saddle-node bifurcation in heterogeneous neural fields.

Figure 5

Numerically simulated spatio-temporal dynamics of the heterogeneous neural field (left) and stationary states (right). Here the stationary state V₀(x) is exponentially unstable with the parameters of point (2) in Figure 2. All parameters are identical to Figure 3, however the final field activity at large times V(x, t = 125) (dashed line) is close to the trivial solution V = 0. This jump of activity recalls the presence of the trivial stationary solution V = 0 and hence reflects the multi-stability of the heterogeneous neural field.

Finally, we would like to stress that the analysis presented above does not depend on the smoothness of the kernel and stationary state. For a strong noise level in the synaptic interaction kernel K, the analytical discussion above still describes the stationary state and the linear stability quite well as shown in Figure 6 for a stable stationary state V₀(x) close to the stability threshold.

Figure 6

Strongly random connectivity kernels yielding noisy exponentially stable stationary states. Panel (a) shows K(x, y), (b)$L(x,y)=K(x,y)S^{\prime }\left(y\right)$, (c) spatio-temporal simulation starting close to the stationary state V₀, (d) stationary states V₀(x) (red) and V(x, t = 25) (black). Parameters are κ = 0.5, α = 0.88, W₀ = 1.75, σ = 0.15,θ = 3.0 yielding a maximum eigenvalue ε₁ = 0.95, i.e. close to the stability threshold.

Heteroclinic orbits

The previous section has shown that heterogeneous neural fields may exhibit various stationary states with different stability properties. In particular we found that stationary states could be saddles with one-dimensional unstable manifolds that could be connected to stable heteroclinic sequences (SHS: [27, 28]) which is supported by experimental evidence [24, 32, 33]. We present in the following paragraphs our main findings on heterogeneous neural fields exhibiting heteroclinic sequences and also hierarchies of such sequences.

One level heteroclinic sequence

It is possible to expand the integral in the Amari equation (1) into a power series yielding

$$\begin{array}{l}\frac{\mathrm{\partial V}(x,t)}{\mathrm{\partial t}}+V(x,t)\\ =K_0+{\int }_{\Omega }{K}_1(x,y)V(y,t)\mathrm{d}y+\frac{1}{2}{\int }_{\Omega }{\int }_{\Omega }{K}_2(x,y,z)V(y,t)V(z,t)\mathrm{d}y\mathrm{d}z.\end{array}$$

(8)

with kernels

$$\begin{array}{lll}\hfill K_0& =& 0\\ \hfill K_1(x,y)& =& \sum _k({\sigma }_k+1)V_k^{+}\left(y\right)V_k\left(x\right)\\ K_2(x,y,z)& =& 2\sum _{\mathit{\text{kj}}}{\rho }_{\mathit{\text{kj}}}{\sigma }_j{V}_k^{+}\left(y\right)V_j^{+}\left(z\right)V_k\left(x\right).\end{array}$$

(9)

The solution of Eq. (8) represents a heteroclinic sequence that connects saddle points {V _k (x)} along their respective stable and unstable manifolds. Its transient evolution is described as winnerless competition in a Lotka-Volterra population dynamics governed by interaction weights ρ _{i k} between neural populations k and i and their respective growth rates σ _i .

In Eq. (9) the $\left\{V_k^{+}\right(x\left)\right\}$ comprise a bi-orthogonal system of the saddles {V _k (x)}. Therefore, the kernel K₁(x, y) describes a Hebbian synapse between sites y and x that has been trained with pattern sequence V _k . This finding confirms the previous result of [18]. Moreover the three-point kernel K₂(x, y, z) further generalizes Hebbian learning to interactions between three sites x, y, z ∈ Ω. Note that the kernels K _i are linear combinations of dyadic product kernels, similar to those introduced in (3). Thus, our construction of heteroclinic orbits straightforwardly results in Pincherle-Goursat kernels used in [26].

Multi-level hierarchy of heteroclinic sequences

Now we assume that the neural field supports a hierarchy of stable heteroclinic sequences in the sense of [25, 34]. For the general case one has to construct integral kernels for a much wider class of neural field equations which can be written as

$$V(x,t)={\int }_{-\infty }^t\mathrm{d}\tau G(t,\tau ){\int }_{\Omega }\mathrm{d}yK(x,y)S\left[V\right(y,\tau \left)\right]$$

(10)

where the new temporal kernel G describing synaptic-dendritic filtering is usually the Green’s function of a linear differential operator Q, such that G(t, τ) = G(t - τ) and the temporal integration in Eq. (10) is temporal convolution. Equation (10) can be simplified by condensing space x and time t into spacetime s = (x, t). Then, (10) becomes

$$V\left(s\right)={\int }_{M}H(s,s^{\prime })S\left[V\right(s^{\prime }\left)\right]\mathrm{d}{s}^{\prime }$$

(11)

with a tensor product kernel $H(s,s^{\prime })=(K\otimes G)(s,s^{\prime })$ and integration domain $M=\Omega \times ]-\infty ,t]$.

For only two levels in such a hierarchy, we obtain

$$\begin{array}{lcr}V(x,t)& =& {\int }_{\Omega }\mathrm{d}{x}_1{\int }_{-\infty }^t\mathrm{d}{t}_1{K}_1(x,x_1)V(x_1,t_1)\hfill \\ -{\int }_{\Omega }\mathrm{d}{x}_1{\int }_{\Omega }\mathrm{d}{x}_2{\int }_{-\infty }^t\mathrm{d}{t}_2{K}_2(x,x_1,x_2)V(x_1,t_1)V(x_2,t_1)\hfill \\ -{\int }_{\Omega }\mathrm{d}{x}_1{\int }_{\Omega }\mathrm{d}{x}_2{\int }_{\Omega }\mathrm{d}{x}_3{\int }_{-\infty }^t\mathrm{d}{t}_1{\int }_{-\infty }^t\mathrm{d}{t}_2\hfill \\ \times K_3(x,x_1,x_2,x_3)V(x_1,t_1)V(x_2,t_1)V(x_3,t_2)\hfill \\ +{\int }_{\Omega }\mathrm{d}{x}_1{\int }_{\Omega }\mathrm{d}{x}_2{\int }_{\Omega }\mathrm{d}{x}_3{\int }_{\Omega }\mathrm{d}{x}_4{\int }_{-\infty }^t\mathrm{d}{t}_1{\int }_{-\infty }^t\mathrm{d}{t}_2\hfill \\ \times K_4(x,x_1,x_2,x_3,x_4)V(x_1,t_1)V(x_2,t_1)V(x_3,t_2)V(x_4,t_2)\hfill \end{array}$$

(12)

with kernels

$$\begin{array}{ccc}\hfill K_1(x,x_1)& =& \sum _{k_1=1}^{n_1}\frac{{\sigma }_{k_1}^{\left(1\right)}}{{\tau }^{\left(1\right)}}{V}_{k_1}^{{\left(1\right)}^{+}}\left(x_1\right)V_{k_1}^{\left(1\right)}\left(x\right)+\sum _{k_2=1}^{n_2}\frac{{\sigma }_{k_2}^{\left(2\right)}}{{\tau }^{\left(2\right)}}{V}_{k_2}^{{\left(2\right)}^{+}}\left(x_1\right)V_{k_2}^{\left(2\right)}\left(x\right)\hfill \\ \hfill K_2(x,x_1,x_2)& =& \sum _{k_2=1}^{n_2}\sum _{j_2=1}^{n_2}\frac{{\rho }_{k_2{j}_2}^{\left(2\right)}}{{\tau }^{\left(2\right)}}{V}_{k_2}^{{\left(2\right)}^{+}}\left(x_1\right)V_{j_2}^{{\left(2\right)}^{+}}\left(x_2\right)V_{k_2}^{\left(2\right)}\left(x\right)\hfill \\ \hfill K_3(x,x_1,x_2,x_3)& =& \sum _{k_1=1}^{n_1}\sum _{j_1=1}^{n_1}\sum _{l_2=1}^{m_2}\frac{{\sigma }_{l_2}^{\left(2\right)}{r}_{k_1{j}_1{l}_2}^{\left(1\right)}}{{\tau }^{\left(1\right)}{\tau }^{\left(2\right)}}{V}_{k_1}^{{\left(1\right)}^{+}}\left(x_1\right)V_{j_1}^{{\left(1\right)}^{+}}\left(x_2\right)V_{l_2}^{{\left(2\right)}^{+}}\left(x_3\right)V_{k_1}^{\left(1\right)}\left(x\right)\hfill \\ \hfill K_4(x,x_1,x_2,x_3,x_4)& =& \sum _{k_1=1}^{n_1}\sum _{j_1=1}^{n_1}\sum _{l_2=1}^{m_2}\sum _{j_2=1}^{n_2}\frac{r_{k_1{j}_1{l}_2}^{\left(1\right)}{\rho }_{l_2{j}_2}^{\left(2\right)}}{{\tau }^{\left(1\right)}{\tau }^{\left(2\right)}}{V}_{k_1}^{{\left(1\right)}^{+}}\left(x_1\right)V_{j_1}^{{\left(1\right)}^{+}}\left(x_2\right)V_{l_2}^{{\left(2\right)}^{+}}\left(x_3\right)\hfill \\ \hfill \times V_{j_2}^{{\left(2\right)}^{+}}\left(x_4\right)V_{k_1}^{\left(1\right)}\left(x\right).\end{array}$$

Here, the $V_{k_{\nu }}^{\left(\nu \right)}\left(x\right)$ denote the k _ν -th saddle in the ν-th level of the hierarchy (containing n _ν stationary states). Saddles are chosen again in such a way that they form a system of bi-orthogonal modes $V_{k_{\nu }}^{{\left(\nu \right)}^{+}}\left(x\right)$ whose behavior is determined by Lotka-Volterra dynamics with growth rates ${\sigma }_{k_{\nu }}^{\left(\nu \right)}>0$ and time-dependent interaction weights ${\rho }_{k_{\nu }{j}_{\nu }}^{\left(\nu \right)}\left(t\right)>0$, ${\rho }_{k_{\nu }{k}_{\nu }}^{\left(\nu \right)}\left(t\right)=1$ that are given by linear superpositions of templates $r_{k_{\nu }{j}_{\nu }{l}_{\nu +1}}^{\left(\nu \right)}$. Additionally, τ^(ν) are the characteristic time scales for level ν. Levels are temporally well separated through τ^(ν)≫τ^(ν + 1)[35].

Interestingly these kernels are time-independent. Since neural field equations can be written in the same form as Eq. (12), this result shows that hierarchies of Lotka-Volterra systems are included in the neural field description. Again we point out that the resulting kernels are linear combinations of dyadic products as introduced in Eq. (3), see also the work of Veltz and Faugeras [26].

Discussion

This study considers spatially heterogeneous neural fields describing the mean potential in neural populations according to the Amari equation [2]. To our best knowledge this work is one of the first deriving the implicit conditions for stationary solutions and identifying the corresponding stability constraint as the Hammerstein integral equation. Moreover, as one of the first studies our work derives conditions for the linear stability of such stationary solutions and derives an analytical expression for stability subjected to the properties of heterogeneous synaptic interaction. The analytical results are complemented by numerical simulations illustrating the stability and instability of heterogeneous stationary states. We point out that the results obtained extend previous studies both on homogeneous neural fields and heterogeneous neural fields as other studies in this context have done before [21].

By virtue of the heterogeneity of the model, it is possible to consider more complex spatiotemporal dynamical behavior than the dynamics close to a single stationary state. We show how to construct hierarchical heteroclinic sequences built of multiple stationary states each exhibiting saddle node dynamics. The work demonstrates in detail how to construct such sequences given the stationary states and their saddle node dynamics involved. Motivated by a previous study on hierarchical heteroclinic sequences [25, 34], we constructed such sequences in heterogeneous neural fields of the Amari type. Our results indicate that such a hierarchy may be present in a single heterogeneous neural field whereas previous studies [25, 34] consider the presence of several neural populations to describe heteroclinic sequences. The kernels obtained from heteroclinic saddle node dynamics are linear superpositions of tensor product kernels, known as Pincherle-Goursat kernels in the literature [26].

Conclusion

The present work is strongly related to the literature hypothesizing the presence of chaotic itinerant neural activity, cf. previous work by [36, 37]. This concept of chaotic itinerancy is attractive but yet lacking a realistic neural model. We admit that the present work represents just a first initial starting point for further model analysis of sequential neural activity in heterogeneous neural systems. It opens up an avenue of future research and promises to close the gap between the rather abstract concept of sequential, i.e. temporally transient, neural activity and corresponding mathematical neural models.

Methods

Stationary states and their stability

In order to learn more about the spatiotemporal dynamics of a neural field, in general it is reasonable to determine stationary states and to study their linear stability. This already gives some insight into the nonlinear dynamics of the system. Since the dynamics depends strongly on the interaction kernel and the corresponding stationary states, it is necessary to work out conditions for the existence and uniqueness of stationary states and their stability.

Analytical study

For stationary solutions, the left hand side of Eq. (1) vanishes and we obtain the Hammerstein equation (2) [29]. It has a trivial solution V₀(x) = 0 and further non-trivial solutions V₀(x) ≠ 0 under certain conditions. The existence and number of solutions of Eq. (2) depends mainly on the operator $\mathcal{A}=\mathcal{I}+\mathcal{K}\mathcal{F}$[38] and its monotonicity [39], with the unity operator $\mathcal{I}$, the linear integral operator

$$\begin{array}{l}\mathcal{Ku}={\int }_{\Omega }K(x,y)u\left(y\right)\mathrm{d}y\end{array}$$

and the Nemytskij operator $\mathcal{Fu}\left(x\right)=S\left(u\right(x\left)\right)$. For instance, a simple criterion for the existence of at least a single non-trivial solution is the symmetry and positive definiteness of the kernel K(x, y) and the condition S(u) ≤ C₁u + C₂[29]. Moreover previous studies have proposed analytical [40] and numerical [41] methods to find solutions of the Hammerstein equation (2).

To illustrate the non-uniqueness of solutions of the Hammerstein equation, let us expand the stationary solution into a set of bi-orthogonal spatial modes {ϕ _n (x)}, {ψ _n (x)}

$$\begin{array}{l}{V}_0\left(x\right)=\sum _{n=0}^{\infty }{V}_n{\varphi }_n\left(x\right)=\sum _{n=0}^{\infty }{U}_n{\psi }_n\left(x\right)\end{array}$$

with constant coefficients V _n , U _n and

$$\begin{array}{l}\left({\psi }_m,{\varphi }_n\right)={\int }_{\Omega }{\psi }_m\left(x\right){\varphi }_n\left(x\right)\mathrm{d}x={\delta }_{n,m}.\end{array}$$

with the Kronecker symbol δ_n,m. Then the Hammerstein equation recasts to

$$\begin{array}{l}{V}_m=S_0{\kappa }_m+S^{\prime }\sum _{n=0}^{\infty }{K}_{m,n}{V}_n+f_m\left(\left\{V_n\right\}\right)\end{array}$$

(13)

with

$$\begin{array}{l}{\kappa }_m=\left({\psi }_m,\mathcal{K}1\right),K_{m,n}=\left({\psi }_m,\mathcal{K}{\varphi }_n\right),f_m=\left({\psi }_m,\mathcal{Kf}\right)\hfill \end{array}$$

and

$$\begin{array}{l}f\left(x\right)=\sum _{p=2}^{\infty }\frac{1}{p!}\frac{{\partial }^{p}S\left(z\right)}{\partial z^p}z=0{\left(\sum _{n=0}^{\infty }{V}_n{\varphi }_n\left(x\right)\right)}^p.\hfill \end{array}$$

Since f _m is a nonlinear function of V _n , Eq. (13) has multiple solutions {V _n } for a given bi-orthogonal basis. Hence, V₀(x) may not be unique.

Considering small deviations u(x, t) = V(x, t) - V₀(x) from a non-trivial stationary state V₀(x), these deviations obey the linear equation (6) with $L(x,y)=K(x,y)S^{\prime }\left(y\right)$ and $S^{\prime }\left(y\right)=\mathrm{d}S\left[V_0\right(y\left)\right]/\mathrm{d}{V}_0\left(y\right)$.

A solution of (6) is then $u(x,t)=exp\left(\mathrm{\lambda t}\right)e\left(x\right),\lambda \in ℂ$ with mode e(x). Inserting this solution into Eq. (6) yields the continuous spectrum of the corresponding linear operator L(x, y) determined implicitly by the eigenvalue equation

$$\begin{array}{l}(\lambda +1)e\left(x\right)={\int }_{\Omega }L(x,y)e\left(y\right)\mathrm{d}y\end{array}$$

(14)

Under the above assumption of a dyadic product kernel (3), the eigenfunctions {e _k (x)} of the kernel L(x, y), defined through

$$\begin{array}{l}{\epsilon }_k{e}_k\left(x\right)={\int }_{\Omega }L(x,y)e_k\left(y\right)\mathrm{d}y,\end{array}$$

(15)

with eigenvalues ${\epsilon }_k={\lambda }_k+1\in ℂ$ form an orthonormal system with respect to the scalar product

$$\begin{array}{l}{〈u,v〉}_S={\int }_{\Omega }u\left(x\right)v\left(x\right)S^{\prime }\left(y\right)\mathrm{d}y\end{array}$$

(16)

with weight $S^{\prime }\left(y\right)$. This follows from the dyadic product kernel (3) through

$$\begin{array}{lcr}{\int }_{\Omega }L(x,y)e_k\left(y\right)\mathrm{d}y& =& {\int }_{\Omega }K(x,y)S^{\prime }\left(y\right)e_k\left(y\right)\mathrm{d}y\hfill \\ =& {\int }_{\Omega }{V}_0\left(x\right)V_0\left(y\right)S^{\prime }\left(y\right)e_k\left(y\right)\mathrm{d}y\hfill \\ =& V_0\left(x\right){\int }_{\Omega }{V}_0\left(y\right)e_k\left(y\right)S^{\prime }\left(y\right)\mathrm{d}y\hfill \\ \hfill {\epsilon }_k{e}_k\left(x\right)& =& V_0\left(x\right){〈V_0,e_k〉}_S.\hfill \end{array}$$

(17)

From (17) we deduce two important results:

• a certain eigenmode $e_{k_0}\left(x\right)$ is proportional to the stationary state V₀(x) with scaling factor $〈V_0,e_{k_0}{〉}_S/{\epsilon }_{k_0}$, and

• other eigenmodes in the eigenbasis $e_{k\ne k_0}\left(x\right)$ are orthogonal to V₀(x) yielding 〈V₀,e _k 〉 _S  = ε _k  = 0.

Hence for dyadic product kernels (3) the spectrum of L includes one eigenmode with eigenvalue ε₁ ≠ 0, i.e. λ₁ ≠ -1, while all other eigenmodes are stable with ε_k≠1 = 0 and thus λ_k≠1 = -1. Therefore, a non-trivial stationary state could become either an asymptotically stable fixed point, i.e. an attractor, for ε _n  < 1, for all n, or a saddle with a one-dimensional unstable manifold, for ε₁ > 1 and ε_n≠1 < 1 (neglecting the singular case ε₁= 1 ).

Finally we have to justify the necessary condition (5) which follows from 0 < S(x) < 1 and

$$\begin{array}{lcr}1& =& {\int }_{\Omega }{V}_0\left(y\right)S\left[V_0\right(y\left)\right]\mathrm{d}y<{\int }_{\Omega }{V}_0\left(y\right)\mathrm{d}y,\hfill \end{array}$$

inserted into the Hammerstein equation (2) for a dyadic product kernel (3).

Numerical study

In order to investigate stationary states of the Amari equation numerically, we choose a finite one-dimensional spatial domain of length L and discretize it into a regular grid of n intervals with grid interval length Δ x = L / n. Then the kernel function is K(x = n Δ x, y = m Δ x) = K _{n m}  / Δ x and Eq. (1) reads

$$\begin{array}{l}\frac{\partial V_i\left(t\right)}{\mathrm{\partial t}}=-V_i\left(t\right)+\sum _{j=1}^n{K}_{\mathit{\text{ij}}}S\left[V_j\right(t\left)\right]\end{array}$$

(18)

where V _n (t) = V(n Δ x, t). The corresponding Hammerstein equation (2) is given by

$$\begin{array}{l}{V}_{0,i}=\sum _{j=1}^n{K}_{\mathit{\text{ij}}}S\left[V_{0,j}\right].\end{array}$$

(19)

Taking into account the insight from the discussion of Eq. (3) and its consequences for the eigenmodes, we have chosen the spatial kernel to

$$\begin{array}{l}{K}_{\mathit{\text{ij}}}=V_{0,i}{V}_{0,j}.\end{array}$$

(20)

We employ an Euler-forward integration scheme for the temporal evolution with discrete time step Δ t = 0.05.

We render the stationary state random by adding the noise term η _i which are random numbers taken from a normal distribution with zero mean and unit variance. We point out that we choose η _i such like $\left|\sum _{i=1}^n{\eta }_i\right|<0.05$ in order to not permit an amplitude increase in the dynamics by noise, but just as a modulation. The sigmoid function is chosen as $S\left(V\right)=1/(1+exp(-\alpha (V-\theta )\left)\right)$ parameterized by the slope parameter α and the mean threshold θ.

Heteroclinic orbits

The general neural field equation (10) supplies the Amari equation (1) as a special case for G(t) = e^-tΘ(t) (Θ(t) as Heaviside’s step function). Then Q = ∂ _t  + 1 is the Amari operator for the Green’s function G(t). For second-order synaptic dynamics [42] and for the filter properties of complex dendritc trees [43], more complicated kernels or differential operators, respectively, have to be taken into account.

As a first step for constructing heteroclinic sequences for (1) or (10), we expand the nonlinear transfer function S in Eq. (11) into a power series about a certain state $\bar{V}\left(s\right)$[15],

$$\begin{array}{lcr}S\left[V\right(s\left)\right]& =& \sum _{n=0}^{\infty }\frac{S^{\left(n\right)}\left(s\right)}{n!}{\left(V\right(s)-\bar{V}(s\left)\right)}^n\hfill \\ & =& \sum _{n=0}^{\infty }\frac{S^{\left(n\right)}\left(s\right)}{n!}{(-1)}^n{\left(\bar{V}\right(s)-V(s\left)\right)}^n\hfill \\ & =& \sum _{n=0}^{\infty }\frac{S^{\left(n\right)}\left(s\right)}{n!}{(-1)}^n\sum _{k=0}^n{(-1)}^k\left(\genfrac{}{}{0ex}{}{n}{k}\right)\bar{V}{\left(s\right)}^{n-k}V{\left(s\right)}^k\hfill \\ & =& \sum _{n=0}^{\infty }\sum _{k=0}^n{(-1)}^{n+k}\frac{S^{\left(n\right)}\left(s\right)}{n!}\left(\genfrac{}{}{0ex}{}{n}{k}\right)\bar{V}{\left(s\right)}^{n-k}V{\left(s\right)}^k\hfill \end{array}$$

Inserting V(s) from Eq. (11) yields

$$\begin{array}{lcrr}S\left[V\right(s\left)\right]& & =& \sum _{n=0}^{\infty }\sum _{k=0}^n{(-1)}^{n+k}\frac{S^{\left(n\right)}\left(s\right)}{n!}\left(\genfrac{}{}{0ex}{}{n}{k}\right)\bar{V}{\left(s\right)}^{n-k}{\left({\int }_{M}H(s,s^{\prime })S\left[V\right(s^{\prime }\left)\right]\mathrm{d}{s}^{\prime }\right)}^k\hfill \\ =& \sum _{n=0}^{\infty }\sum _{k=0}^n{(-1)}^{n+k}\frac{S^{\left(n\right)}\left(s\right)}{n!}\left(\genfrac{}{}{0ex}{}{n}{k}\right)\bar{V}{\left(s\right)}^{n-k}\hfill \\ \times {\int }_M\cdots {\int }_{M}H(s,s_1)H(s,s_2)\dots H(s,s_k)\hfill \\ \times S\left[V\right(s_1\left)\right]S\left[V\right(s_2\left)\right]\dots S\left[V\right(s_k\left)\right]\mathrm{d}{s}_1\cdots \mathrm{d}{s}_k\hfill \\ =& \sum _{n=0}^{\infty }\sum _{k=0}^n{(-1)}^{n+k}\frac{S^{\left(n\right)}\left(s\right)}{n!}\left(\genfrac{}{}{0ex}{}{n}{k}\right)\bar{V}{\left(s\right)}^{n-k}\hfill \\ \times {\int }_M\cdots {\int }_M\prod _{j=1}^{k}H(s,s_j)S\left[V\right(s_j\left)\right]\mathrm{d}{s}_j.\hfill \end{array}$$

By eliminating the nonlinearities S[ V(s _j )] using the power series above again, we get

$$\begin{array}{lcr}S\left[V\right(s\left)\right]& =& \sum _{n=0}^{\infty }\sum _{k=0}^n{(-1)}^{n+k}\frac{S^{\left(n\right)}\left(s\right)}{n!}\left(\genfrac{}{}{0ex}{}{n}{k}\right)\bar{V}{\left(s\right)}^{n-k}\hfill \\ \times {\int }_M\cdots {\int }_M\prod _{j=1}^{k}H(s,s_j)\left(\sum _{n_j=0}^{\infty }\frac{S^{\left(n_j\right)}\left(s_j\right)}{n_j!}{\left(V\right(s_j)-\bar{V}(s_j\left)\right)}^{n_j}\right)\mathrm{d}{s}_j.\hfill \end{array}$$

Inserting this expression into Eq. (11) leads to a generalized Volterra series

$$\begin{array}{l}V\left(s\right)=H_0\left(s\right)\hfill \\ +\sum _{m=1}^{\infty }\frac{1}{m!}{\int }_M\cdots {\int }_M{H}_m(s,s_1,\dots ,s_m)V\left(s_1\right)\dots V\left(s_m\right)\mathrm{d}{s}_1\cdots \mathrm{d}{s}_m\hfill \end{array}$$

(21)

with a sequence of integral kernels H _m that can be read off after some further tedious rearrangements [44].

One-level hierarchy

In a previous work, we have derived a one-level hierarchy of stable heterogenic sequences [44], which we briefly recapitulate here. We assume a family of n stationary states V _k (x), 1 ≤ k ≤ n that are to be connected along a heteroclinic sequence. Each state is assumed to be realized by a population in the neural field governed by (10), that is characterized by a population activity α _k (t) ∈ [0, 1]. Then the overall field quantity is obtained through an order parameter expansion [15, 45]

$$\begin{array}{l}V(x,t)=\sum _{k=1}^n{\alpha }_k\left(t\right)V_k\left(x\right).\end{array}$$

(22)

These population amplitudes result from a winnerless competition in a generalized Lotka-Volterra system approximating a Wilson-Cowan model [46, 47]

$$\begin{array}{lcr}\frac{\mathrm{d}{\xi }_k\left(t\right)}{\mathrm{d}t}& =& {\xi }_k\left(t\right)\left({\sigma }_k-\sum _{j=1}^n{\rho }_{\mathit{\text{kj}}}{\xi }_j\left(t\right)\right)\hfill \\ {\alpha }_k\left(t\right)& =& \frac{{\xi }_k\left(t\right)}{{\sigma }_k}\hfill \end{array}$$

(23)

with growth rates σ _k  > 0, and interaction weights ρ _{k j}  > 0, ρ _{k k}  = 1, that are trained by the algorithm of [27] and [28] for the desired sequence of transitions.

For the following construction, we also assume that the system of stationary states {V _k (x)} is linearly independent such that there is a bi-orthogonal system $\left\{V_k^{+}\right(x\left)\right\}$, obeying

$$\begin{array}{l}{\int }_{\Omega }{V}_j^{+}\left(x\right)V_k\left(x\right)\mathrm{d}x={\delta }_{\mathit{\text{jk}}}.\end{array}$$

(24)

Then, we obtain from (22)

$$\begin{array}{lcr}{\int }_{\Omega }{V}_j^{+}\left(x\right)V(x,t)\mathrm{d}x& =& \sum _{k=1}^n{\alpha }_k\left(t\right){\int }_{\Omega }{V}_j^{+}\left(x\right)V_k\left(x\right)\mathrm{d}x\hfill \\ =& \sum _{k=1}^n{\alpha }_k\left(t\right){\delta }_{\mathit{\text{jk}}}\hfill \end{array}$$

and hence

$$\begin{array}{lcr}{\alpha }_j\left(t\right)& =& {\int }_{\Omega }{V}_j^{+}\left(x\right)V(x,t)\mathrm{d}x\hfill \\ {\xi }_j\left(t\right)& =& {\sigma }_j{\int }_{\Omega }{V}_j^{+}\left(x\right)V(x,t)\mathrm{d}x.\hfill \end{array}$$

(25)

Next, we take the derivation of (22) with respect to time t, by exploiting (23)

$$\begin{array}{lcr}\frac{\mathrm{\partial V}(x,t)}{\mathrm{\partial t}}& =& \sum _k\frac{1}{{\sigma }_k}\frac{\mathrm{d}{\xi }_k}{\mathrm{d}t}{V}_k\left(x\right)\hfill \\ =& \sum _k\frac{1}{{\sigma }_k}\left[{\xi }_k\left({\sigma }_k-\sum _j{\rho }_{\mathit{\text{kj}}}{\xi }_j\right)\right]V_k\left(x\right)\hfill \\ =& \sum _k{\xi }_k{V}_k\left(x\right)-\sum _{\mathit{\text{kj}}}\frac{{\rho }_{\mathit{\text{kj}}}}{{\sigma }_k}{\xi }_k{\xi }_j{V}_k\left(x\right).\hfill \end{array}$$

(26)

Adding (22), we obtain

$$\begin{array}{lcrr}\frac{\mathrm{\partial V}(x,t)}{\mathrm{\partial t}}+V(x,t)& & =& \sum _k{\xi }_k{V}_k\left(x\right)-\sum _{\mathit{\text{kj}}}\frac{{\rho }_{\mathit{\text{kj}}}}{{\sigma }_k}{\xi }_k{\xi }_j{V}_k\left(x\right)+\sum _k\frac{1}{{\sigma }_k}{\xi }_k{V}_k\left(x\right)\hfill \\ & =& \sum _k\left(1+\frac{1}{{\sigma }_k}\right){\xi }_k{V}_k\left(x\right)-\sum _{\mathit{\text{kj}}}\frac{{\rho }_{\mathit{\text{kj}}}}{{\sigma }_k}{\xi }_k{\xi }_j{V}_k\left(x\right)\hfill \end{array}$$

(27)

from which we eliminate all occurrences of ξ by means of (25). This yields

$$\begin{array}{lcr}\frac{\mathrm{\partial V}(x,t)}{\mathrm{\partial t}}+V(x,t)& =& \sum _k\left(1+\frac{1}{{\sigma }_k}\right){\sigma }_k{\int }_{\Omega }{V}_k^{+}\left(y\right)V(y,t)V_k\left(x\right)\mathrm{d}y\hfill \\ & -& \sum _{\mathit{\text{kj}}}\frac{{\rho }_{\mathit{\text{kj}}}}{{\sigma }_k}{\sigma }_k{\sigma }_j{\int }_{\Omega }{\int }_{\Omega }{V}_k^{+}\left(y\right)V(y,t)V_j^{+}\left(z\right)V(z,t)V_k\left(x\right)\mathrm{d}y\mathrm{d}z\hfill \\ & =& {\int }_{\Omega }\sum _k\left({\sigma }_k+1\right)V_k^{+}\left(y\right)V_k\left(x\right)V(y,t)\mathrm{d}y\hfill \\ & -& {\int }_{\Omega }{\int }_{\Omega }\sum _{\mathit{\text{kj}}}{\rho }_{\mathit{\text{kj}}}{\sigma }_j{V}_k^{+}\left(y\right)V_j^{+}\left(z\right)V_k\left(x\right)V(y,t)V(z,t)\mathrm{d}y\mathrm{d}z.\hfill \end{array}$$

(28)

Moreover, the nonlinear spatial integral transformation in the Amari equation (1) may be written as a generalized Volterra series (21)

$$\begin{array}{l}\mathcal{J}\left[V\right(·,t\left)\right]\left(x\right)=K_0\left(x\right)\hfill \\ +\sum _{m=1}^{\infty }\frac{1}{m!}{\int }_{\Omega }\cdots {\int }_{\Omega }{K}_m(x,y_1,y_2,\dots ,y_m)V(y_1,t)V(y_2,t)\dots V(y_m,t)\mathrm{d}{y}_1\mathrm{d}{y}_2\cdots \mathrm{d}{y}_m.\hfill \end{array}$$

(29)

Comparison of the first three terms with (28) yields the result (9).

Multi-level hierarchy of heteroclinic sequences

Here we assume that the SHS are given by (possibly infinitely many) generalized Lotka-Volterra systems

$${\tau }^{\left(\nu \right)}\frac{\mathrm{d}{\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)}{\mathrm{d}t}={\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)\left({\sigma }_{k_{\nu }}^{\left(\nu \right)}-\sum _{j_{\nu }=1}^{n_{\nu }}{\rho }_{k_{\nu }{j}_{\nu }}^{\left(\nu \right)}\left(t\right){\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)\right)$$

(30)

with growth rates ${\sigma }_{k_{\nu }}^{\left(\nu \right)}>0$, and time-dependent interaction weights ${\rho }_{k_{\nu }{j}_{\nu }}^{\left(\nu \right)}\left(t\right)>0$, ${\rho }_{k_{\nu }{k}_{\nu }}^{\left(\nu \right)}\left(t\right)=1$. Again, $\nu \in \mathbb{N}$ indicates the level within the hierarchy, n _ν is the number of stationary states and τ^(ν) represents the characteristic time scale of that level. Levels are temporally well separated through τ^(ν)≫τ^(ν + 1)[35].

Following [34], the population amplitudes ${\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)$ of level ν prescribe the control parameters of the higher level ν-1 by virtue of

$$\begin{array}{l}{\rho }_{k_{\nu }{j}_{\nu }}^{\left(\nu \right)}\left(t\right)=\sum _{l_{\nu +1}=1}^{m_{\nu +1}}{\alpha }_{l_{\nu +1}}^{(\nu +1)}\left(t\right)r_{k_{\nu }{j}_{\nu }{l}_{\nu +1}}^{\left(\nu \right)}\end{array}$$

(31)

where the constants $r_{k_{\nu }{j}_{\nu }{l}_{\nu +1}}^{\left(\nu \right)}$ serve as parameter templates.

Finally, the amplitudes ${\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)$ recruit a hierarchy of modes $V_{k_{\nu }}^{\left(\nu \right)}\left(x\right)$ in the neural field such that

$$\begin{array}{l}V\left(s\right)=V(x,t)=\sum _{\nu =1}^{\infty }\sum _{k_{\nu }=1}^{n_{\nu }}{\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)V_{k_{\nu }}^{\left(\nu \right)}\left(x\right).\end{array}$$

(32)

Assuming a system of bi-orthogonal modes $V_{k_{\nu }}^{{\left(\nu \right)}^{+}}\left(x\right)$ with

$$\begin{array}{l}{\int }_{\Omega }{V}_{k_{\nu }}^{{\left(\nu \right)}^{+}}\left(x\right)V_{j_{\nu }}^{\left(\nu \right)}\left(x\right)\mathrm{d}x={\delta }_{k_{\nu }{j}_{\nu }}\end{array}$$

(33)

we obtain from (32)

$$\begin{array}{l}{\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)={\int }_{\Omega }{V}_{k_{\nu }}^{{\left(\nu \right)}^{+}}\left(x\right)V(x,t)\mathrm{d}x.\end{array}$$

(34)

Next we convert the Lotka-Volterra differential equations (30) into their corresponding integral equations by formally integrating

$${\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)=\frac{1}{{\tau }^{\left(\nu \right)}}{\int }_{-\infty }^t{\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t_{\nu }\right)\left({\sigma }_{k_{\nu }}^{\left(\nu \right)}-\sum _{j_{\nu }=1}^{n_{\nu }}{\rho }_{k_{\nu }{j}_{\nu }}^{\left(\nu \right)}\left(t_{\nu }\right){\alpha }_{j_{\nu }}^{\left(\nu \right)}\left(t_{\nu }\right)\right)\mathrm{d}{t}_{\nu }$$

(35)

and obtain a recursion equation after inserting (31)

$${\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)=\frac{1}{{\tau }^{\left(\nu \right)}}\underset{-\infty }{\overset{t}{\int }}{\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t_{\nu }\right)\left({\sigma }_{k_{\nu }}^{\left(\nu \right)}-\sum _{j_{\nu }=1}^{n_{\nu }}\sum _{l_{\nu +1}=1}^{m_{\nu +1}}{r}_{k_{\nu }{j}_{\nu }{l}_{\nu +1}}^{\left(\nu \right)}{\alpha }_{l_{\nu +1}}^{(\nu +1)}\left(t_{\nu }\right){\alpha }_{j_{\nu }}^{\left(\nu \right)}\left(t_{\nu }\right)\right)\mathrm{d}{t}_{\nu }.$$

(36)

Eventually, we insert all results of the recursion of (36) into the field equation (32) and eliminate the amplitudes ${\alpha }_{k_{\nu }}^{\left(\nu \right)}\left(t\right)$ by means of (34) in order to get a series expansion of V(s) in terms of spatiotemporal integrals. This series must equal the generalized Volterra series (21), such that the kernel H(s,s^′) can be reconstructed to solve the neural field inverse problem [44]. For more mathematical details on the derivation of the two-level hierarchy, see Additional file 1.