Finite-size and correlation-induced effects in mean-field dynamics

Abstract

The brain’s activity is characterized by the interaction of a very large number of neurons that are strongly affected by noise. However, signals often arise at macroscopic scales integrating the effect of many neurons into a reliable pattern of activity. In order to study such large neuronal assemblies, one is often led to derive mean-field limits summarizing the effect of the interaction of a large number of neurons into an effective signal. Classical mean-field approaches consider the evolution of a deterministic variable, the mean activity, thus neglecting the stochastic nature of neural behavior. In this article, we build upon two recent approaches that include correlations and higher order moments in mean-field equations, and study how these stochastic effects influence the solutions of the mean-field equations, both in the limit of an infinite number of neurons and for large yet finite networks. We introduce a new model, the infinite model, which arises from both equations by a rescaling of the variables and, which is invertible for finite-size networks, and hence, provides equivalent equations to those previously derived models. The study of this model allows us to understand qualitative behavior of such large-scale networks. We show that, though the solutions of the deterministic mean-field equation constitute uncorrelated solutions of the new mean-field equations, the stability properties of limit cycles are modified by the presence of correlations, and additional non-trivial behaviors including periodic orbits appear when there were none in the mean field. The origin of all these behaviors is then explored in finite-size networks where interesting mesoscopic scale effects appear. This study leads us to show that the infinite-size system appears as a singular limit of the network equations, and for any finite network, the system will differ from the infinite system.

Appendices

A An alternative derivation of the moment equations

In this appendix we show that the moment equation corresponding to the Markov chain governed by Eq. (2) can be derived from a Rodriguez–Tuckwell moment expansion on the Langevin approximation of the Markov chain. Following Kurtz approach in Kurtz (1976), we know that the dynamics of the our rescaled variables p _i  = n _i /N _i , which is a Markov chain in the initial setting, approaches as N goes to infinity a continuous diffusion process (X _t ) _t ≥ 0 satisfying the equation:

$$ \begin{array}{rll} dX_t^i&=\Big(-\alpha_i\,X^i_t + f_i(s_i(t,X_t))\Big)\;dt\\&{\kern6pt}\quad+ \displaystyle\frac 1 N \sqrt{\alpha_i\,X^i_t + f_i(s_i(t,X_t))}\;dW^i_t \end{array} $$

(13)

where $S_i(t,X_t)=\sum_{j=1}^N w_{ij}X^j_t + I_i(t)$.

Following the works of Rodriguez and Tuckwell (Rodriguez and Tuckwell 1996, 1998), we derive from this equation the dynamical system governing the approximate moments of X. Denoting by m _j the mean value of X _j and by C _ij the correlation between X _i and X _j , a direct application of Rodriguez and Tuckwell formula applied to our particular form of dynamical system yields:

$$ \begin{cases} \displaystyle\frac{\text{d}m_j}{\text{d}t}&=-\alpha_j m_j +f_j(s_j) + \displaystyle\frac 1 2 f_j^{\prime\prime}(s_j)\sum_{l=1}^N \sum_{p=1}^N w_{jp}w_{jl} C_{lp}\\ \displaystyle\frac{\text{d}C_{ij}}{\text{d}t}&=-(\alpha_i+\alpha_j)C_{ij}+ f_i^{\prime}(s_i)\sum_{l=1}^N w_{il} C_{lj} \\&{\kern5pt}+ f_j^{\prime}(s_j)\sum_{l=1}^N w_{jl} C_{li} + \delta_{ij} \Big[\alpha_i m_i +f_i(s_i) \\ &{\kern5pt}+ \displaystyle\frac 1 {N_i N_j} f_i^{\prime\prime}(s_i)\sum_{k,l}w_{ik}w_{il} C_{kl}\Big] \end{cases} $$

Where $s_i=\sum_{j=1}^N w_{ij}m^j_t + I_i(t)$. These equations therefore appear as a perturbation of the BCC equations with an additional nonlinear term in the dynamics or order two. Truncation at order 1 in the small parameter 1/N yields exactly BCC equations. Moreover, it is interesting to note that these equations again correspond in the limit N→ ∞, to the infinite model described in Section 3.

B Kronecker algebra: some useful properties

In this appendix, we review and prove some useful properties of Kronecker products of matrixes. We recall the definition of the the function Vect transforming a M × N matrix into a M N-dimensional column vector, as defined in Neudecker (1969):

$$ \mathrm{Vect}: \begin{cases} \mathbb{R}^{M\times N} &\mapsto \mathbb{R}^{M\,N}\\ X & \mapsto [X_{11}, \ldots, X_{M1}, X_{12}(t), \ldots, X_{M2}(t), \\ & {\kern6pt}\ldots X_{1N}(t), \ldots, X_{MN}(t)]^T \end{cases} $$

Let us now denote by ⊗ the Kronecker product defined for A ∈ ℝ^m×n and B ∈ ℝ^r×s as the (m r) ×(n s) matrix:

$$ A\otimes B := \left( \begin{array}{cccc} a_{11} B & a_{12} B & \cdots & a_{1n}B\\ a_{21} B & a_{22} B & \cdots & a_{2n}B\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} B & a_{m2} B & \cdots & a_{mn}B\\ \end{array} \right) $$

For standard definitions and identities in the field of Kronecker products, the reader is referred to Brewer (1978). We recalled in the main text the following identities for A, B, D, G, X ∈ ℝ^M×M, I _M be the M × M identity matrix and A · B or A B denote the standard matrix product:

$$ \begin{cases} \mathrm{Vect} (AXB) = (B^T \otimes A ) \mathrm{Vect}(X)\\ A\oplus A =A\otimes I_M + I_M \otimes A\\ (A \otimes B)\cdot (D \otimes G) = (A\cdot D) \times (B\cdot G) \end{cases}. $$

(14)

The relationship ⊕ is called Kronecker sum.

Proposition 2

Let A and B in ℝ^M×M , and assume that A has the eigenvalues {λ _i ; i = 1,..., M} and B the eigenvalues {μ _i ; i = 1,..., M}. Then we have:

• A ⊗ B has the eigenvalues $\{\lambda_i\mu_j;\;(i,j)\in \{1,\ldots,M\}^2\}$ .

• A ⊕ B has the eigenvalues $\{\lambda_i+\mu_j;\;(i,j)\in \{1,\ldots,M\}^2\}$ .

Proof

Let λ _i (resp.μ _j ) be an eigenvalue of A (resp. B) with eigenvector u (resp. v), and define the matrix z = u·v ^T (i.e. z _ij  = u _i v _j ). We have:

$$ \begin{array}{rll} \Big(A\oplus B \Big) \, \mathrm{Vect} (z) &= \mathrm{Vect}\Big(A \cdot u\cdot v^T + u\cdot v^T \cdot B^T\Big)\\ &= \mathrm{Vect}\Big(\lambda_i u \cdot v^T + u \cdot (B\cdot v)^T\Big) \\ &= \lambda_i \mathrm{Vect}(z) + \mathrm{Vect}\Big(u (\mu_j v^T)\Big)\\ &= (\lambda_i + \mu_j) \mathrm{Vect}(z). \end{array} $$

which entails that Vect(z) is an eigenvector of A(ν) ⊕ A(ν) associated with the eigenvalue λ _i  + λ _j . Similarly, we have for z = v · u ^T:

$$ \begin{array}{rll} \Big(A\otimes B \Big) \, \mathrm{Vect} (z) &= \mathrm{Vect}\Big(B \cdot v\cdot u^T A^T\Big)\\ &= \mathrm{Vect}\Big((\mu_j v) \cdot (\lambda_i \,u^T) \Big) = \lambda_i\mu_h \mathrm{Vect}(z) \end{array} $$

The dimension of A ⊕ B and A ⊗ B is M ² and there are exactly M ² linearly independent matrices z possible built in the proposed fashion, therefore we identified all possible eigenvalues. □

Theorem 3

Let Φ(t) be the solution of the matrix differential equation:

$$ \begin{cases} \frac{\text{d}\Phi(t)}{\text{d}t} &= A(t)\Phi(t)\\ \Phi(0) &= I_M \end{cases} $$

and Ψ(t) the solution of:

$$ \begin{cases} \frac{\text{d}\Psi(t)}{\text{d}t} &= (A(t)\oplus A(t))\Psi(t)\\ \Psi(0) &= I_{M^2} \end{cases} $$

We have Ψ(t) = Φ(t) ⊗ Φ(t).

Proof

Indeed, we have, using the basic properties of the Kronecker product recalled in Eq. (8)

$$ \begin{array}{lll} &&{\kern-6pt}\frac{\text{d}\Phi(t)\otimes \Phi(t)}{\text{d}t} \\&&= \frac{\text{d}\Phi(t)}{\text{d}t} \otimes \Phi(t) + \Phi(t) \otimes \frac{\text{d}\Phi(t)}{\text{d}t}\\ &&= \big(A(\nu(t))\cdot \Phi(t)\big)\otimes \Phi(t) + \Phi(t) \otimes \big(A(\nu(t))\cdot \Phi(t)\big) \\ &&= \big(A(\nu(t))\cdot \Phi(t)\big)\otimes \big(I_M\cdot\Phi(t)\big) + \big(I_M\cdot\Phi(t)\big) \otimes \big(A(\nu(t))\cdot \Phi(t)\big)\\ &&= (A(\nu(t))\otimes I_M)\cdot (\Phi(t)\otimes \Phi(t)) \\ &&{\kern6pt}+ (I_M\otimes A(\nu(t))) \cdot (\Phi(t)\otimes \Phi(t))\\ &&= \big(A(\nu(t))\otimes I_M + I_M\otimes A(\nu(t))\big) \cdot \big(\Phi(t)\otimes\Phi(t)\big)\\ &&= \big(A(\nu(t))\oplus A(\nu(t))\big) \cdot \big(\Phi(t)\otimes\Phi(t)\big) \end{array} $$

Therefore Φ(t) ⊗ Φ(t) satisfies the same differential equation as Ψ(t) and moreover, $\Phi(0)\otimes \Phi(0)=I_M\otimes I_M=I_{M^2}$, and therefore by existence and uniqueness of the resolvent, Ψ(t) = Φ(t) ⊗ Φ(t). □

C Genericity of the Hopf bifurcation found

In this appendix we derive the expression of the first Lyapunov exponent of the bifurcation, which proves that the existence of the Hopf bifurcation exhibited in Section 4.1.2. In that section, we derived the expression of the Jacobian matrix at the considered fixed point:

$$ J=\left (\begin{array}{cc} -\alpha + w\, f^{\prime}(s) & \displaystyle\frac 1 2 \,f^{\prime\prime}(s)\,w^2\\ 2f^{\prime}(s)\displaystyle\frac w N & 2(-\alpha+w\, f^{\prime}(s)) \end{array}\right) $$

At the bifurcation point, we have − α ^* + w f ^′(s) = 0, and therefore at this point the Jacobian matrix reads:

$$ J_0=\left (\begin{array}{cc} 0 & \displaystyle\frac 1 2 \,f^{\prime\prime}(I)\,w^2\\ 2f^{\prime}(I)\displaystyle\frac w N & 0 \end{array}\right) $$

The eigenvalues of this matrix under the assumptions of Section 4.1.2 are ±i ω ₀ where $\omega_0=\sqrt{-f^{\prime}(I)f^{\prime\prime}(I)w^3/N}$. We define q the right eigenvector of J ₀ associated with i ω ₀:

$$ q=\Big(-\displaystyle\frac{i}{\sqrt{2\,f^{\prime}(I)\,w/N}} \qquad 1\Big)^T $$

and p the right eigenvector of $J_0^T$ associated with the eigenvalue iω ₀:

$$ p=\Big(\displaystyle\frac{i}{\sqrt{-\displaystyle\frac 1 2 f^{\prime\prime}(I)\,w^2}} \qquad 1\Big)^T $$

For the sake of simplicity, we also name the components of the vector field of the system:

$$ \begin{cases} f_1(\binom{\nu}{C}, \alpha^*) &= w\,f^{\prime}(I)\,nu + f(w\nu+I)\\&{\kern6pt}+\displaystyle\frac 1 2 f^{\prime\prime}(w\nu+I)\,w^2\,C\\ f_2(\binom{\nu}{C}, \alpha^*) &= -2\,w\,f^{\prime}(I)\,C +\\&{\kern6pt} 2\,f^{\prime}(w\nu+I)\,w\,\big(C+\displaystyle\frac{\nu}{N}\big)\\ \end{cases}. $$

Following Kuznetsov (1998), we define $B(\binom{x_1}{y_1}, \binom{x_2}{y_2})$ and $C(\binom{x_1}{y_1}, \binom{x_2}{y_2},\binom{x_3}{y_3})$ the second and third derivatives of the vector field, which are bi- and tri-linear forms. We have the following expressions for these multilinear functions:

$$ \begin{cases} B_1(\binom{x_1}{y_1}, \binom{x_2}{y_2}) &= w^2\,f^{\prime\prime}(I)\,x_1\,x_2 + \displaystyle\frac 1 2\,f^{\prime\prime}(I)\,w^2\\&{\kern6pt}\times(x_1\,y_2 +y_1\,x_2)\\ B_2(\binom{x_1}{y_1}, \binom{x_2}{y_2}) &= 2\,f^{\prime\prime}(I)\,w^2\,(x_1\,y_2+y_1\,x_2) \\&{\kern6pt}+ \displaystyle\frac 4 N\,f^{\prime\prime}(I)\,w^2\,x_1 \, x_2\\ C_1(\binom{x_1}{y_1}, \binom{x_2}{y_2}, \binom{x_3}{y_3}) &= f^{(3)}(I)\,w^3\,x_1\,x_2\,x_3 + \displaystyle\frac 1 2 \, f^{(4)}(I)\,w^4\\&{\kern6pt}\times(x_1\,x_2\,y_3+x_1\,y_2\,x_3+y_1\,x_2\,x_3)\\ C_2(\binom{x_1}{y_1}, \binom{x_2}{y_2}, \binom{x_3}{y_3}) &= \displaystyle\frac 6 N\,f^{(3)}(I)\, w^3 \, x_1\,x_2\,x_3 + 2\,f^{(3)}(I)\,w^3 \\&{\kern6pt}\times(x_1\,x_2\,y_3+x_1\,y_2\,x_3+y_1\,x_2\,x_3) \end{cases} $$

We are now in position to compute the first Lyapunov exponent l ₁(0) using the formula:

$$ \begin{array}{lll} &&{\kern-18pt}l_1(0)\\&{\kern3pt}=&\displaystyle\frac 1 {2\omega_0} \textrm{Re}\Big ( \langle p, C(q,q,\overline{q})\rangle - 2 \langle p, B(q, J_0^{-1} B(q, \overline{q}))\rangle \\ &&{\kern3pt}+ \langle p, B(\overline{q}, (2\,i\,\omega_0 Id - J_0)^{-1}B(q,q))\rangle \Big)\\ &{\kern3pt}=& \displaystyle\frac 1 {2\omega_0} (\mathcal{A}-2\mathcal{B}+\mathcal{C}) \end{array} $$

where $\langle x, y \rangle$ denotes the complex inner product $\overline{x}^T\cdot y$ and the sum of three terms denoted $\mathcal{A}$, $\mathcal{B}$ and $\mathcal{C}$ are the real parts of the terms involved in the expression of the Lyapunov exponent. After straightforward but tedious calculations (that can be conveniently performed using a formal calculation tool such as Maple), we obtain:

$$ \begin{cases} \mathcal{A} &= \displaystyle\frac{w^2 \, N}{f^{\prime}(I)^2} f^{(3)}(I)\,f^{\prime}(I)\\ \mathcal{B} &= \displaystyle\frac{w^2 \, N}{f^{\prime}(I)^2} \left( -\displaystyle\frac{f^{\prime\prime}(I)^2}{\omega_0} - \displaystyle\frac 1 2 \displaystyle\frac{f^{(3)}(If^{\prime}(I))}{\omega_0} + 2 f^{\prime\prime}(I)^2\right)\\ \mathcal{C} &= -\displaystyle\frac{w^2\, N }{f^{\prime}(I)^2}\;\displaystyle\frac{2 f^{\prime\prime}(I)^2}{3} \end{cases} $$

which yields the expression for the Lyapunov exponent:

$$ \begin{array}{rll} l_1(0) &=& \displaystyle\frac{1}{2\,\omega_0} \left [ \mathcal{A} -2\,\mathcal{B}+\mathcal{C}\right ]\\ & =& \displaystyle\frac{w^2\,N}{2\,\omega_0\,f^{\prime}(I)^2} \left [ f^{(3)}(I)f^{\prime}(I) \left(1+\displaystyle\frac {1}{\omega_0}\right)\right.\\ &&\left.\vphantom{\left(1+\displaystyle\frac {1}{\omega_0}\right)}+ f^{\prime\prime}(I)^2\left(\displaystyle\frac{2}{\omega_0} - \displaystyle\frac{14}{3}\right)\right] \end{array} $$

D Finite-size effects in BCC two-populations model I

In this appendix, we study the two-populations BCC system corresponding to Model I and show that the finite-size effects are closely related to what is observed in Bressloff model as studied in Section 4.2. BCC finite-size equations read:

$$ \begin{cases} a_1^{\prime}&=-\alpha a_1+f(s_1)+\frac{1}{2}\,f^{\prime\prime}(s_1)\\&{\kern6pt}\times(w_{11}^2\,c_{11}+w_{12}^2\,c_{22}+2\,w_{12}\,w_{11}\,c_{12})\\ a_2^{\prime}&=-\alpha a_2+f(s_2)+\frac{1}{2}\,f^{\prime\prime}(s_2)\\&{\kern6pt}\times(w_{22}^2\,c_{22}+w_{21}^2\,c_{11}+2\,w_{22}\,w_{21}\,c_{12})\\ c_{11}^{\prime}&=-2\alpha c_{11}+2\,f^{\prime}(s_{1})\,(w_{11}\,c_{11}+w_{12}\,c_{12})\\&{\kern6pt}+2\,f^{\prime}(s_1)\,a_1\,w_{11}\,n_e\\ c_{22}^{\prime}&=-2\alpha c_{22}+2\,f^{\prime}(s_{2})\,(w_{21}\,c_{12}+w_{22}\,c_{22})\\&{\kern6pt}+2\,n_i\,f^{\prime}(s_2)\,w_{22}\,a_2\\ c_{12}^{\prime}&=-2\alpha c_{12}+f^{\prime}(s_{1})\,(w_{11}\,c_{12}+w_{12}\,c_{22}+a_2\,w_{12}\,n_i) \ldots \\ & \qquad + f^{\prime}(s_2)\,(w_{21}\,c_{11}+w_{22}\,c_{12}+w_{21}\,a_{1}\,n_e)\\ \end{cases} $$

(15)

where we denoted:

$$ \begin{cases} s_1 &= w_{11}\,a_1+w_{12}\,a_2+i_1\\ s_2 &= w_{21}\,a_2+w_{22}\,a_2+i_2 \end{cases} $$

Similarly to Bressloff case, BCC model features two families of limit cycles (see Fig. 12). One of these branches corresponds exactly to the branch of limit cycles of WC system starting from a Hopf bifurcation and disappearing through a homoclinic bifurcations. Two additional Hopf bifurcations appear related to the family of periodic orbit corresponding to the correlation-induced cycle evidence in the analysis of the infinite-size system. This branch of limit cycles exist whatever n, and loses stability through a Neimark-Sacker bifurcation as the number of neurons increases. This bifurcation generates chaos for large enough networks, which clearly does not exists in WC system.

Fig. 12

Bifurcation diagram for the BCC system. Blue lines represent the equilibria, pink lines the extremal values of the cycles in the system. Bifurcations of equilibria are denoted with a red star, LP represents a saddle-node bifurcation (Limit Point), H a Hopf bifurcation. The four Hopf bifurcations share two families of limit cycles. The branch corresponding to the smaller values of i ₁ undergoes two fold of limit cycles, and the other branch of limit cycle a Neimark Sacker (Torus) bifurcation

This family of limit cycles presents stability for networks containing between 100 and 17.000 neurons, corresponding to a finite size effect that appears only in the region of interest of the cortical columns. As the number of neurons increase, the system keep the same number and type of bifurcations, and the infinite model appears to be a singular case where different bifurcations meet (see Fig. 13), and very large networks lose the property of presenting a stable cycle, and present a chaotic behavior.

Fig. 13

Codimension two bifurcations and chaotic behavior as the number of neuron increases