What can neurons do for their brain? Communicate selectivity with bursts

Abstract

Neurons deep in cortex interact with the environment extremely indirectly; the spikes they receive and produce are pre- and post-processed by millions of other neurons. This paper proposes two information-theoretic constraints guiding the production of spikes, that help ensure bursting activity deep in cortex relates meaningfully to events in the environment. First, neurons should emphasize selective responses with bursts. Second, neurons should propagate selective inputs by burst-firing in response to them. We show the constraints are necessary for bursts to dominate information-transfer within cortex, thereby providing a substrate allowing neurons to distribute credit amongst themselves. Finally, since synaptic plasticity degrades the ability of neurons to burst selectively, we argue that homeostatic regulation of synaptic weights is necessary, and that it is best performed offline during sleep.

Appendices

Appendix 1: Proof of Theorem  2

Given distribution p(x) on X, let $H(p)=-\sum _{x\in X}p(x)\cdot \log p(x)$ denote the entropy.

Theorem 2

Let $p_1(y|x)$ and $p_2(z|y)$ be Markov matrices on finite sets $X$, $Y$ and $Z$. Let $p_{12}(z|x)=\sum _{y\in Y}p_1(y|x)\cdot p_2(z|y)$ denote the composite channel. Then

$$ei\left(X\mathop{\rightarrow}\limits_{}^{n_2\circ n_1}z\right) \leq \sum_{y\in Y}c(y|z)\cdot ei\left(X\mathop{\rightarrow}\limits_{}^{n_1}y\right),\hbox {where}\; c(y|z):= p_2(z|y)\cdot\frac{p_{1}(y)}{p_{12}(z)}.$$

Proof

As usual, $p_1(y):=\sum _{x\in X}p_1(y|do(x))\cdot p_{\text{ unif}}(x)$ and $p_{12}(z):=\sum _{x\in X}p_{12}(z|do(x))\cdot p_{\text{ unif}}(x)$. The actual repertoire is

$$\begin{aligned} \hat{p}_{12}(x|z)&=\frac{p_{12}(z|do(x))\cdot p_{\text{ unif}}(x)}{p_{12}(z)} \\ & = \frac{\sum _{y\in Y}p_1 (y|do(x))\cdot p_2(z|y)\cdot p_{\text{ unif}}(x)}{p_{12}(z)}. \end{aligned}$$

(8)

Observe that

$$\begin{aligned} \hat{p}_{1}(x|y)=\frac{p_1(y|do(x))\cdot p_{\text{ unif}}(x)}{p_1(y)}. \end{aligned}$$

(9)

Combining Eq.  (8) and (9) obtains

$$\begin{aligned} \hat{p}_{12}(x|z) = \sum _{y\in Y}c(y|z)\cdot \hat{p}_1(x|y). \end{aligned}$$

It is easy to check that $c(y|z)$ induces a probability distribution on $Y$ so that, by convexity of relative entropy Cover and Thomas (2006),

$$\begin{aligned} \underbrace{H\left[\sum _{y\in Y}c(y|z)\cdot \hat{p}_1(X|y)\left\Vert\sum _{y\in Y}c(y|z)\cdot p_{\text{unif}}(X)\right]\right.}_{ei(X\mathop{\rightarrow}\limits_{}^{n_2\circ n_1} z)} \le \sum _{y\in Y} c(y|z)\cdot \underbrace{H\Big [\hat{p}_1(X|y)\Big \Vert p_{\text{unif}}(X)\Big ]}_{ei(X\mathop{\rightarrow}\limits_{}^{n_1}y)}. \end{aligned}$$

$\square$

Appendix 2: Effective information for Fig. 3

Effective information for the two detectors can be computed by exhaustively perturbing each with all possible configurations. However, since we understand their mechanisms, it is easy to replace exhaustive perturbation of the elements with combinatorics. First, since we have imposed the condition that every configuration contains 4 distinct dots in the $8\times 8$ grid, it follows that the number of potential input patterns is $635,376={{8\times 8}\atopwithdelims (){4}}$.

Effective information for the vertical rectangle: $n_2$. The number of configurations fitting inside a vertical rectangle of width 7 is computed as follows. 4 dots can fit inside a rectangle of width 7 in ${8\times 7}\atopwithdelims ()4$ ways. Excluding configurations that fit inside a smaller rectangle of width 6, we find that there are ${8\times 7\atopwithdelims (){4}} -2{8\times 6\atopwithdelims (){4}} +{8\times 5\atopwithdelims (){4}}$ configurations that fit inside a rectangle of width 7, but not a rectangle of width 6 (we add ${8\times 5}\atopwithdelims (){4}$ to compensate for double counting). Finally, there are two ways a rectangle of width 7 can fit inside the grid, so there are $139,040=2\left[{8\times 7\atopwithdelims (){4}} -2{8\times 6 \atopwithdelims (){4}} +{8\times 5 \atopwithdelims (){4}}\right]$ configurations that fit only inside a 7 pixel wide rectangle. Effective information is 2.2 bits.

Effective information for the horizontal rectangle: $n_1$. The computation is similar to that above. There are ${{8\times 2}\atopwithdelims (){4}}-2{{8}\atopwithdelims (){4}}$ ways a configuration of 4 dots can fit inside a rectangle of height 2, without fitting inside a smaller rectangle of height 1. There are 7 different ways a rectangle of height 2 can be placed inside the grid, so there are $11,760=7\left[{8\times 2\atopwithdelims (){4}}-2{8\atopwithdelims (){4}}\right]$ configurations only fit a 2 pixel high rectangle. Effective information is $5.8$ bits.

Appendix 3: Applying Theorem 2 to Figs. 4 and 5

We apply equation

$$\begin{aligned} ei\left(X\mathop{\rightarrow}\limits_{}^{n_2\circ n_1}z\right) \le \sum _{y\in Y}c(y|z)\cdot ei\left(X\mathop{\rightarrow}\limits_{}^ {n_1}y\right), \text{ where}\,c(y|z):= p_2(z|y)\cdot \frac{p_{1}(y)}{p_{12}(z)} \end{aligned}$$

to the three cases in turn.

AND-gates.

$$\begin{aligned} 4=\log _2\frac{16}{1}=ei(X\mathop{\rightarrow}\limits_{}^{\text{AND}^2\circ \text{AND}}1)&\le \sum _{y\in \{11\}}1\cdot ei(X\mathop{\rightarrow}\limits_{}^ {\text{AND}^2}11) \\&= \sum _{y\in \{11\}} \left[ei(X\mathop{\rightarrow}\limits_{}^{\text{AND}}1) + ei(X\mathop{\rightarrow}\limits_{}^ {\text{AND}}1)\right]\\&= 2+2 \end{aligned}$$

Distribution $c(y|z)$ is concentrated on $y=11$. The channel $n_1=AND^2$ decomposes into two AND-gates, generating 2 bits of information each. The only input on the bottom layer that causes a spike on the top layer is 1111.

OR-gates.

$$\begin{aligned} 0.1=\log _2\frac{16}{15}=ei(X\mathop{\rightarrow}\limits_{}^{\text{ OR}^2\circ OR}1)&\le& \sum _{y\in \{01,10,11\}}c(y|z)\cdot ei(X\mathop{\rightarrow}\limits_{}^{\text{ OR}^2}y) \\&=& \frac{16}{15}\cdot \frac{1}{4}\cdot \frac{3}{4}\cdot \left[ei(X\mathop{\rightarrow}\limits_{}^{\text{ OR}}0) + ei(X\mathop{\rightarrow}\limits_{}^{\text{ OR}}1)\right]\\&&\quad +\frac{16}{15}\cdot \frac{3}{4}\cdot \frac{1}{4}\cdot \left[ei(X\mathop{\rightarrow}\limits_{}^{\text{ OR}}1) + ei(X\mathop{\rightarrow}\limits_{}^{\text{ OR}}0)\right]\\&&\quad +\frac{16}{15}\cdot \frac{3}{4}\cdot \frac{3}{4}\cdot \left[ei(X\mathop{\rightarrow}\limits_{}^{\text{ OR}}1) + ei(X\mathop{\rightarrow}\limits_{}^{\text{ OR}}1)\right]\\&=& \frac{3}{15}(2+0.415) + \frac{3}{15}(0.415+2)+\frac{9}{15}(0.415+0.415)\\&=& 1.02 \end{aligned}$$

The distribution $c(y|z)$ points to three potential causes: 01, 10, and 11, and is therefore less concentrated than in the case of an AND-gate. Moreover, each of these three potential inputs is less informative than in the preceding case.

The set of potential inputs on the bottom layer that cause a spike on the top layer includes everything except 0000.

Finally, note that in this case the upper-bound is very loose.

NOR-gates.

$$\begin{aligned} 0.83 = \log _2\frac{16}{9}= ei\left(X\mathop{\rightarrow}\limits_{}^ {\text{NOR}^2\circ \text{ NOR}}1\right)&\le \sum _{y\in \{00\}}1\cdot ei(X\mathop{\rightarrow}\limits_{}^ {\text{ NOR}^2}00) \\&= \sum _{y\in \{00\}}\left[ei(X\mathop{\rightarrow}_{}^ {\text{ NOR}}0) + ei(X\mathop{\rightarrow}_{}^ {\text{ NOR}}0)\right]\\&= 0.415 + 0.415 \end{aligned}$$

As for the AND-hierarchy, distribution $c(y|z)$ is concentrated. However, it points to input $00$, for which NOR-gates are uninformative.

The nine potential inputs causing output 1 on the top layer are $\{0101, 1001, 0110, 1010, 1011, 0111, 1101, 1110, 1111\}$.