Encoding certainty in bump attractors

Abstract

Persistent activity in neuronal populations has been shown to represent the spatial position of remembered stimuli. Networks that support bump attractors are often used to model such persistent activity. Such models usually exhibit translational symmetry. Thus activity bumps are neutrally stable, and perturbations in position do not decay away. We extend previous work on bump attractors by constructing model networks capable of encoding the certainty or salience of a stimulus stored in memory. Such networks support bumps that are not only neutrally stable to perturbations in position, but also perturbations in amplitude. Possible bump solutions then lie on a two-dimensional attractor, determined by a continuum of positions and amplitudes. Such an attractor requires precisely balancing the strength of recurrent synaptic connections. The amplitude of activity bumps represents certainty, and is determined by the initial input to the system. Moreover, bumps with larger amplitudes are more robust to noise, and over time provide a more faithful representation of the stored stimulus. In networks with separate excitatory and inhibitory populations, generating bumps with a continuum of possible amplitudes, requires tuning the strength of inhibition to precisely cancel background excitation.

Appendices

Appendix A: Stability analysis of stationary bumps

The stability of a stationary bump solution U(x) is determined by writing

$$ u(x,t) = U(x) + \bar{\psi}(x,t),$$

(A.1)

where $||\bar {\psi }(x,t)|| \ll 1$ and expanding Eq. (2.1) to first-order in $\bar {\psi }(x,t)$. This leads to the linear equation

$$\begin{array}{@{}rcl@{}} \frac{\partial \bar{\psi}(x,t)}{\partial t} = - \bar{\psi}(x,t) + \int_{- \pi}^{\pi} w(x-y) f'(U(y)) \bar{\psi}(y,t) \mathrm{d} y.\\ \end{array}$$

(A.2)

Since Eq. (A.2) is linear, we can use separation of variables to characterize all of its solutions (Ermentrout 1998; Sandstede 2002; Folias and Bressloff 2004; Veltz and Faugeras 2010; Bressloff 2012). Plugging the ansatz $\bar {\psi }(x,t) = b(t) \psi (x)$ into Eq. (A.2), we find

$$\begin{array}{@{}rcl@{}} b'(t) \psi (x) = - b(t) \psi(x,t){\kern50pt}\\ + b(t) \int_{- \pi}^{\pi} w(x-y) f'(U(y)) \psi(y) \mathrm{d} y.{}\end{array}$$

After rearranging terms, we have

$$\begin{array}{@{}rcl@{}} \frac{b'(t)}{b(t)} = - 1 + \frac{1}{\psi(x)} \int_{- \pi}^{\pi} w(x-y) f'(U(y)) \psi(y) \mathrm{d} y, \end{array}$$

(A.3)

meaning that both the left and right hand sides of Eq. (A.3) equal a constant λ. Therefore b′(t) = λb(t) implying b(t) = e ^λt, and

$$\begin{array}{@{}rcl@{}} (\lambda + 1) \psi (x) = \int_{- \pi}^{\pi} w(x-y) f'(U(y)) \psi(y) \mathrm{d} y, \end{array}$$

(A.4)

which is an eigenvalue problem characterizing the stability of bump solutions to Eq. (2.1). Since U(x) < 0 over some portion of the domain, the function f′(U(x)) will have jump discontinuities, but since the domain − π, π is periodic, we can always rearrange the limits of integration to ensure these jump discontinuity points are on the interior of the integral.To characterize the eigensolutions to Eq. (A.4), we decompose the function ψ(x) into a Fourier series on the domain − π, π given by

$$\begin{array}{@{}rcl@{}} \psi(x) = \sum\limits_{k=1}^{N}\mathcal{A}_{k}\cos{kx}+\sum\limits_{k=1}^{N}\mathcal{B}_{k}\sin{kx}, \end{array}$$

(A.5)

where N is directly determined by the number of terms in the Fourier expansion of w(x). The associated coefficients of the expansion in Eq. (A.5) are then determined by the linear system

$$\begin{array}{@{}rcl@{}} \mathcal{A}_{k} &=& w_{k} \int_{- \pi}^{\pi} \cos (kx) f'(U(x)) \psi (x) \mathrm{d} x,\\ \mathcal{B}_{l} &=& w_{l} \int_{- \pi}^{\pi} \sin (lx) f'(U(x)) \psi (x) \mathrm{d} x, \end{array}$$

where k, l = 1, . . . , N. Solutions of this system, along with the associated λ are eigensolutions of Eq. (A.4). We can directly compute the eigenvalues associated with the stability of bumps in the case of the weight function in Eq. (2.2) so that

$$\begin{array}{@{}rcl@{}} (\lambda+1)\psi(x) = w_{1}\int_{-\pi}^{\pi}\cos{(x-y)}f'(U(y))\psi(y)dy. \\ \end{array}$$

(A.6)

Analyzing solutions (λ, ψ) of Eq. (A.6) is equivalent to determining the elements of the spectrum of the linear system in the vicinity of the bump. We are mainly interested in the point spectrum of the linear operator in Eq. (A.6), since the sign of the real part of λ for these solutions will determine the associated stability of stationary bump solutions (see Coombes and Owen 2004; Veltz and Faugeras 2010) for detailed discussions of the partitioning of spectra in neural field models. In particular, we examine the stability of stationary bump solutions of the form U(x) = A cosxwhen the firing rate function has the form given in Eq. (2.3). Hence,

$$\begin{array}{@{}rcl@{}} (\lambda+1)\mathcal{A}_{1} &=& \left\{\begin{array}{ll} \mathcal{A}_1sw_{1}\left[\frac{\pi}{2}-\cos^{-1}\left(\frac{1}{sA}\right)-\frac{1}{sA}\sqrt{1-\left(\frac{1}{sA}\right)^{2}}\right], & \indent \text{for}\;\; sA > 1, \\ \mathcal{A}_1sw_{1}\frac{\pi}{2}, & \indent \text{for}\;\; sA \le 1, \end{array}\right. \end{array}$$

(A.7)

$$\begin{array}{@{}rcl@{}} (\lambda+1)\mathcal{B}_{1} & = & \left\{\begin{array}{ll} \mathcal{B}_1sw_{1}\left[\frac{\pi}{2}-\cos^{-1}\left(\frac{1}{sA}\right)+\frac{1}{sA}\sqrt{1-\left(\frac{1}{sA}\right)^{2}}\right], & \indent \text{for}\;\; sA > 1 ,\\ \mathcal{B}_1sw_{1}\frac{\pi}{2}, & \indent \text{for}\;\; sA \le 1, \end{array}\right. \end{array}$$

(A.8)

and $\mathcal {A}_{k} = \mathcal {B}_{k} = 0$ for k ≠ 1 and $\mathcal {A}_{0} = 0$. Therefore, only sinx and cosx are eigenfunctions of the linearized system. All other Fourier modes cos(kx) and sin(lx) are linear combinations of functions associated with the essential spectrum λ = − 1, given by $\psi (x) = \cos (kx) - \mathcal {C}_{k} \cos (x)$ and $\psi (x) = \sin (lx) - \mathcal {D}_{k} \sin (x)$ where

$$\begin{array}{@{}rcl@{}}</p><p class="noindent">\mathcal{C}_k&=&\frac{\displaystyle\int_{- \pi}^{\pi} \cos y f'(U(y)) \cos (ky) \d y}{\displaystyle \int_{- \pi}^{\pi} \cos^{2} y f'(U(y)) \mathrm{d} y},\\ \mathcal{D}_k&=&\frac{\displaystyle\int_{- \pi}^{\pi} \sin y f'(U(y)) \sin (ky) \mathrm{d} y}{\displaystyle\int_{- \pi}^{\pi} \sin^{2} y f'(U(y)) \mathrm{d} y} \end{array}$$

as well as the eigenfunctions cos(x) and sin(x). Now, bump solutions of Eq. (2.1) will be neutrally stable to both even and odd perturbations when parameters in Eqs. (A.7–A.8) are such that some solutions have Reλ = 0 and others have Reλ < 0. When A > 1/s, we find

$$\begin{array}{@{}rcl@{}} \lambda_{o} &=& \frac{2}{\pi}\left(-\cos^{-1}\left(\frac{1}{sA}\right)-\frac{1}{sA}\sqrt{1-\left(\frac{1}{sA}\right)^{2}}\right) < 0, \\ \lambda_{e} & = & \frac{2}{\pi}\left(-\cos^{-1}\left(\frac{1}{sA}\right)+\frac{1}{sA}\sqrt{1-\left(\frac{1}{sA}\right)^{2}}\right) < 0. \end{array}$$

(A.9)

Appendix B: Existence and stability of bumps in two-population network

To find stationary bump solutions to the excitatory-inhibitory network defined by Eq. (2.4) with synaptic weights given by Eq. (2.5), we make the ansatz

$$\begin{array}{@{}rcl@{}} u(x,t) =& U(x) = A_{0} + A_{1}\cos{x}, \\ v(x,t) =& V(x) = M_{0} + M_{1}\cos{x}. \end{array}$$

(B.1)

and substitute the v equation into the u equation in Eq. (2.4) to generate

$$\begin{array}{@{}rcl@{}} U(x) = (w_{ee}(x)-w_{ei}(x)*w_{ie}(x))*f(U(x)), \end{array}$$

(B.2)

where $f(x)*g(x) = \int _{-\pi }^{\pi }f(x-y)g(y)dy$. Therefore, stationary solutions to Eq. (2.4) are the same as stationary solutions to Eq. (2.1) by assigning

$$\begin{array}{@{}rcl@{}} w(x) &=& w_{ee}(x)-w_{ei}(x)*w_{ie}(x)\\ &=& \bar{w}_{ee}-2\pi\bar{w}_{ei}\bar{w}_{ie}+\bar{w}_{ee}\cos{x}.\\ \end{array}$$

(B.3)

Note that Eq. (B.3) is equivalent to Eq. (2.2) by setting $w_{0} = \bar {w}_{ee}-2\pi \bar {w}_{ei}\bar {w}_{ie}$ and $w_{1} = \bar {w}_{ee}$. Therefore, under an appropriate change of variables, solving Eq. (B.2) is equivalent to solving Eq. (3.1). Therefore, our results concerning the existence of a continuum of amplitudes concerning Eq. (3.1) should hold here as well. This means that in order to obtain a line attractor of bump amplitudes, we must have that A ₀ = 0 and $\bar {w}_{ee}=2\pi \bar {w}_{ei}\bar {w}_{ie}$ (i.e., w ₀ = 0). However, we can still have M ₀ ≠ 0. Additionally, analogous to the single network in Eq. (3.1), we must require that $\bar {w}_{ee} = \frac {2}{\pi s}$. Again, we have

$$\begin{array}{@{}rcl@{}} A =\left\{\begin{array}{ll} sA\bar{w}_{ee}\left[\frac{\pi}{2}-\cos^{-1}\left(\frac{1}{sA}\right)-\frac{1}{sA}\sqrt{1-\left(\frac{1}{sA}\right)^{2}}\right]+2\bar{w}_{ee}\sqrt{1-\left(\frac{1}{sA}\right)^{2}}, & \indent \text{for}\;\; sA > 1, \\ sA\bar{w}_{ee}\frac{\pi}{2}, & \indent \text{for}\;\; sA \le 1, \end{array}\right. \end{array}$$

(B.4)

and, for the v equation

$$\begin{array}{@{}rcl@{}}</p><p class="noindent">V(x) = \bar{w}_{ei}\int_{-\pi}^{\pi}(1+\cos{(x-y)})f(U(y))dy, \end{array}$$

so that

$$\begin{array}{@{}rcl@{}} M_{0} &=& \left\{\begin{array}{ll} 2s\bar{w}_{ei} A\left[1-\sqrt{1-\left(\frac{1}{sA}\right)^{2}}\right] \\{\kern3pc} +2\bar{w}_{ei}\cos^{-1}\left(\frac{1}{sA}\right), & {\kern3.5pc} \text{for}\;\; sA > 1, \\ 2s\bar{w}_{ei} A, & {\kern3.5pc} \text{for}\;\; sA \le 1, \end{array}\right. \\ M_{1} & = & \left\{\begin{array}{ll} sA\bar{w}_{ei}\left[\frac{\pi}{2}-\cos^{-1}\left(\frac{1}{sA}\right)-\frac{1}{sA}\sqrt{1-\left(\frac{1}{sA}\right)^{2}}\right]\\{\kern3pc}+2\bar{w}_{ei}\sqrt{1-\left(\frac{1}{sA}\right)^{2}}, &{} \text{for}\;\; sA > 1, \\ sA\bar{w}_{ei}\frac{\pi}{2}, &{} \text{for}\;\; sA \le 1. \\ \end{array}\right.\\ \end{array}$$

(B.5)

Again, we have a continuum of values for $A \in [0, \frac {\pi \bar {w}_{ee}}{2}]$ that are fixed points, and the coefficients for v will depend on A, and upon substituting values for s we obtain

$$M_{0} = \frac{4\bar{w}_{ie}}{\pi\bar{w}_{ee}} A, \text{\indent} M_{1} = \frac{\bar{w}_{ie}}{\bar{w}_{ee}}A.$$

To study the way in which the line attractor globally organizes dynamics, we consider effects of breaking this balance condition in two ways: excess excitation or excess inhibition. As we shall see, too much inhibition leads to no stable bump solutions whereas too much excitation leads to only a single stable bump solution. To do this, we define the quantity $\bar {w} = \bar {w}_{ee} - 2\pi \bar {w}_{ei}\bar {w}_{ie}$ and simply consider when w̄ > 0 (excess excitation) and w̄ < 0 (excess inhibition).

First let w̄ < 0 (excess inhibition) and consider when U(x) < 1/s. Then, similar to Section 3.1, we find that

$$\begin{array}{@{}rcl@{}}A_{0} & = & 2s\bar{w}[aA_0+\sin{a}A_{1}] \\ A_{1} & = & s\bar{w}_{ee}[\sin{a}A_0+aA_{1}] \end{array}$$

(B.6)

where $a = \cos ^{-1}\left (-\frac {A_{0}}{A_{1}}\right )$ and |A ₀| ≤ |A ₁| . We must consider the cases when A ₀ > 0, A ₀ < 0 and A ₀ = 0. If A ₀ > 0, then since 0 ≤ a ≤ π we know that sina ≥ 0. Also, we impose that A ₁ ≥ 0 so that the peak of the bump corresponds to the remembered location of the stimulus. Then, since w̄ < 0, Eq. (B.6) implies that A ₀ equals something negative, which is a contradiction. Now assume that A ₀ < 0. Then Eq. (B.6) implies that

$$a \ge \frac{A_{1}}{|A_0|}\sin{a}, \qquad \text{and} \qquad a \ge \frac{|A_0|}{A_{1}}\sin{a},$$

which implies that |A ₀| = A ₁. Then our only choices are U(x) = A ₁(cosx − 1) or U(x) ≡ 0. However, if the former were true, then f(u) ≡ 0 which forces U(x) ≡ 0. Finally it is easy to see that if A ₀ = 0 then A ₁ = 0 for w̄ ≠ 0.

Now assume that w̄ > 0 (excess excitation). In the case U(x) < 1/s, we find that the only solution is U(x) ≡ 0. When U(x) > 1/s for some x, then Eq. (B.6) becomes

$$\begin{array}{@{}rcl@{}} A_{0} & = & 2s\bar{w}[(a-b)A_{0} + (\sin{a}-\sin{b})A_{1}] + 2\bar{w}b, \\ A_{1} & = & s\bar{w}_{ee}[(\sin{a}-\sin{b})A_{0} + (a-b)A_{1}] + 2\bar{w}_{ee}\sin{b},\\ \end{array}$$

(B.7)

where $b = \cos ^{-1}\left (\frac {1-sA_{0}}{sA_{1}}\right )$ such that U(b) = 1/s. To simplify the analysis, we will let $s = \frac {2}{\pi \bar {w}_{ee}}$ as was the condition for the line attractor.

We now perform a stability analysis on the fixed bump solution in Eq. (B.1). We consider the set of parameters A ₀ = 0 and A ₁ = A ∈ [0, 1/s] that leads to a line attractor of amplitudes. Similar to Section 3.2, we study the temporal evolution of perturbations to the original bump solutions by plugging in the linear expansion

$$\begin{array}{@{}rcl@{}} u(x,t) & = & U(x) + \bar{\psi}(x,t), \\ v(x,t) & = & V(x) + \bar{\phi}(x,t), \end{array}$$

(B.8)

where $||\bar {\psi }(x,t)||,||\bar {\phi }(x,t)|| \ll 1$. As before, we can show that these solutions are separable, so that $\bar {\psi }(x,t) = \mathrm {e}^{\lambda t} \psi (x)$ and $\bar {\phi }(x,t) = \mathrm {e}^{\lambda t} \phi (x)$. Therefore, by plugging the expansion Eq. (B.8) into Eq. (2.4), and noting separability, we find

$$\begin{array}{@{}rcl@{}} (\lambda+1)\psi(x) & = & w_{ee}{\ast}(f'(U(x))\psi(x))- w_{ie}{\ast}\phi(x), \\ (\tau\lambda+1)\phi(x) & = & w_{ei}{\ast}(f'(U(x))\psi(x)). \end{array}$$

(B.9)

We then expand both spatial functions in Fourier series

$$\begin{array}{@{}rcl@{}} \psi(x) & = & \sum\limits_{k=0}^{N}\mathcal{A}_{k}\cos{(kx)}+\sum\limits_{k=1}^{N}\mathcal{B}_{k}\sin{(kx)}, \\ \phi(x) & = & \sum\limits_{k=0}^{N}\mathcal{M}_{k}\cos{(kx)}+\sum\limits_{k=1}^{N}\mathcal{N}_{k}\sin{(kx)}. \end{array}$$

(B.10)

Similar to Section 3, we analyze the solutions (λ, ψ, ϕ) to determine the stability of the perturbations by observing the sign of the real part of λ. By plugging Eq. (B.10)intoEq. (B.9), we see that when using the weight functions in Eq. (2.5) we have the system

$$\begin{array}{@{}rcl@{}} (\lambda + 1)\mathcal{A}_{0} & = & \bar{w}_{ee}\int_{-\pi}^{\pi}(\mathcal{A}_{0} + \mathcal{A}_{1}\cos{y}+\mathcal{B}_{1}\sin{y})f'(U(y))dy\\ &&-\bar{w}_{ie}\int_{-\pi}^{\pi}(\mathcal{M}_0+\mathcal{M}_{1}\cos{y}+\mathcal{N}_{1}\sin{y})dy, \\ (\lambda+1)\mathcal{A}_{1} & = & \bar{w}_{ee}\int_{-\pi}^{\pi}\cos{y}(\mathcal{A}_{0} + \mathcal{A}_{1}\cos{y}+\mathcal{B}_{1}\sin{y})f'(U(y))dy, \\ (\lambda+1)\mathcal{B}_{1} & = & \bar{w}_{ee}\int_{-\pi}^{\pi}\sin{y}(\mathcal{A}_{0} + \mathcal{A}_{1}\cos{y}+\mathcal{B}_{1}\sin{y})f'(U(y))dy, \\ (\tau\lambda+1)\mathcal{M}_{0} & = & \bar{w}_{ei}\int_{-\pi}^{\pi}(\mathcal{A}_{0} + \mathcal{A}_{1}\cos{y}+\mathcal{B}_{1}\sin{y})f'(U(y))dy, \\ (\tau\lambda+1)\mathcal{M}_{1} & = & \bar{w}_{ei}\int_{-\pi}^{\pi}\cos{y}(\mathcal{A}_{0} + \mathcal{A}_{1}\cos{y}+\mathcal{B}_{1}\sin{y})f'(U(y))dy, \\ (\tau\lambda+1)\mathcal{N}_{1} & = & \bar{w}_{ei}\int_{-\pi}^{\pi}\sin{y}(\mathcal{A}_{0} + \mathcal{A}_{1}\cos{y}+\mathcal{B}_{1}\sin{y})f'(U(y))dy,\\ \end{array}$$

(B.11)

where $\mathcal {A}_{k} = \mathcal {B}_{k} = 0$ for k ≠ 0, 1. When τ ≠ 0, we can compute the integrals and set conditions of the parameters for the line attractor to find that the system in Eq. (B.11) is equivalent to the linear system

$$\begin{array}{@{}rcl@{}} \lambda \left(\begin{array}{l} \mathcal{A}_{0} \\ \mathcal{A}_{1} \\ \mathcal{B}_{1} \\ \mathcal{M}_{0} \\ \mathcal{M}_{1} \\ \mathcal{N}_1 \end{array}\right) = \left(\begin{array}{cccccc} 1 & \frac{4}{\pi} & 0 & -2\pi\bar{w}_{ie} & 0 & 0 \\ \frac{4}{\pi} & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ \frac{\displaystyle 1}{\displaystyle \pi\bar{w}_{ie}\tau} & \frac{\displaystyle 2}{\displaystyle \pi^{2}\bar{w}_{ie}\tau} & 0 & -\frac{\displaystyle 1}{\displaystyle \tau} & 0 & 0 \\ \frac{\displaystyle 2}{\displaystyle \pi^{2}\bar{w}_{ie}\tau} & \frac{\displaystyle 1}{\displaystyle 2\pi\bar{w}_{ie}\tau} & 0 & 0 & -\frac{\displaystyle 1}{\displaystyle \tau} & 0 \\ 0 & 0 & \frac{\displaystyle 1}{\displaystyle 2\pi\bar{w}_{ie}\tau} & 0 & 0 & -\frac{\displaystyle 1}{\displaystyle \tau} \end{array}\right) \left(\begin{array}{l} \mathcal{A}_{0} \\ \mathcal{A}_{1} \\ \mathcal{B}_{1} \\ \mathcal{M}_{0} \\ \mathcal{M}_{1} \\ \mathcal{N}_{1} \end{array}\right). \end{array}$$

The associated matrix has the characteristic equation

$$\lambda^{2}(\tau\lambda+1)^{2}\left(\tau\lambda^2+(1-\tau)\lambda+1-\frac{16}{\pi^{2}}\tau\right)=0$$

from which we obtain only two zero eigenvalues corresponding to odd perturbations $\left (0,0,1,0,0,\frac {\bar {w}_{ei}}{\bar {w}_{ee}}\right )$ and even perturbations $\left (0,1,0,\frac {4\bar {w}_{ei}}{\pi \bar {w}_{ee}},\frac {\bar {w}_{ei}}{\bar {w}_{ee}},0\right )$. Thus we see that obtaining a zero eigenvalue associated with even perturbations does not depend on the speed of inhibition, τ. However, neutral stability still does depend on τ, as it is possible that other eigenvalues associated with even perturbations may have positive real part. Looking at the other eigenvalues, we have two negative ones, $\lambda _{-} = -\frac {1}{\tau }$, corresponding to perturbations in $\mathcal {M}_{1}$ and $\mathcal {N}_{1}$. Therefore, if we only perturb the inhibitory network, then solutions will be attracted back toward the fixed bump solutions. The final two eigenvalues can be analyzed by examining

$$\begin{array}{@{}rcl@{}} \lambda_{\pm} = \frac{1}{2}\left(1-\frac{1}{\tau}\right) \pm \frac{1}{2\tau}\sqrt{\left(1+\frac{64}{\pi^{2}}\right)\tau^{2} - 6\tau + 1}. \end{array}$$