Short-term depression and transient memory in sensory cortex
Abstract
Persistent neuronal activity is usually studied in the context of short-term memory localized in central cortical areas. Recent studies show that early sensory areas also can have persistent representations of stimuli which emerge quickly (over tens of milliseconds) and decay slowly (over seconds). Traditional positive feedback models cannot explain sensory persistence for at least two reasons: (i) They show attractor dynamics, with transient perturbations resulting in a quasi-permanent change of system state, whereas sensory systems return to the original state after a transient. (ii) As we show, those positive feedback models which decay to baseline lose their persistence when their recurrent connections are subject to short-term depression, a common property of excitatory connections in early sensory areas. Dual time constant network behavior has also been implemented by nonlinear afferents producing a large transient input followed by much smaller steady state input. We show that such networks require unphysiologically large onset transients to produce the rise and decay observed in sensory areas. Our study explores how memory and persistence can be implemented in another model class, derivative feedback networks. We show that these networks can operate with two vastly different time courses, changing their state quickly when new information is coming in but retaining it for a long time, and that these capabilities are robust to short-term depression. Specifically, derivative feedback networks with short-term depression that acts differentially on positive and negative feedback projections are capable of dynamically changing their time constant, thus allowing fast onset and slow decay of responses without requiring unrealistically large input transients.
Keywords
Positive feedback network Derivative feedback network Balanced network Sensory memory Persistent neuronal activity Short-term depression1 Introduction
Persistence of activity after the removal of a stimulus has been observed in many areas of cortex. In the prefrontal and parietal lobes, persistent activity can last for tens of seconds during working memory tasks, for a review see (Curtis and D’Esposito 2003). Shorter decaying memory traces have recently been found in early sensory areas including visual, auditory and somatosensory cortex, and the olfactory bulb (Super et al. 2001; Shuler and Bear 2006; O’Herron and von der Heydt 2009; Nikolić et al. 2009; O’Herron and von der Heydt 2011; Petreanu et al. 2012; Patterson et al. 2013). These traces rise over tens of milliseconds and decay over seconds. For instance, Shuler and Bear (Shuler and Bear 2006) observed decaying activity in primary visual cortex which was tied to the learned time of reward. O’Herron and von der Heydt (2009) showed that activity representing border ownership relationships in cortical area V2 lasts for a time period of about one second after the end of stimulus presentation.
Physiologically plausible computational models of cortical activity in networks driven by recurrent connectivity have previously shown the ability to produce widely varying time constants (Hardy and Buonomano 2016). These models fall into two categories: those which maintain activity through positive feedback, e.g. Gavornik et al. (2009) and Gavornik and Shouval (2011), or through derivative feedback (Lim and Goldman 2013). Both types of networks depend on their recurrent connections to maintain activity after stimulus offset. Previous studies of the decay times of both network types have not taken into account history dependent changes of synaptic strength (Gavornik et al. 2009; Lim and Goldman 2013; 2014), even though the strengths of individual synapses in cortex change constantly due to both short-term depression (STD) and short-term facilitation (STF). Specifically, it is known that the strengths of synapses in early sensory cortex are dominated by STD (Beck et al. 2005; Tsodyks and Markram 1997; Petersen 2002). We therefore examine how STD impacts the transient activity of both positive feedback and derivative feedback networks. We show that the decay times of positive feedback networks are highly sensitive to STD on their recurrent connections, whereas the decay times of derivative feedback networks have decay times which are robust within the physiological range of STD parameters.
A defining characteristic of any memory mechanism is that memories are acquired much faster than they decay (other persistent activity patterns exist where this is not the case, e.g. sensory after-effects, but those are usually not considered memories). Such behavior cannot be explained in linear systems whose dynamics are symmetric with respect to rise and decay. Most current memory models create the asymmetry by positive feedback which makes activity patterns fall into particular attractor states (Wang 1999; Compte et al. 2000; Major and Tank 2004; Barak and Tsodyks 2007). Such models can explain working memory behavior but not sensory memory since, in the latter, when the stimulus is removed activity decays to the pre-stimulus level. Furthermore, our results show that even small contributions of STD in the recurrent connections, a feature commonly observed in sensory cortex, eliminates persistent activity of positive feedback networks.
A class of models whose activity does return to pre-stimulus levels are negative-derivative networks (Lim and Goldman 2013), which for simplicity we will call derivative feedback networks in the following. This is a special case of balanced networks that have been proposed as implementing a computing paradigm of much higher efficiency and representational power than conventional rate coding (Denève and Machens 2016). Recurrent activity in these networks approximates a negative derivative with respect to time which is obtained from the combination of fast inhibitory feedback followed immediately by slower excitatory feedback (Lim and Goldman 2013). This negative derivative feedback is mathematically equivalent to a friction/damping term in a mechanical system which slows down the decay of activity after a transient excursion. The defining characteristic of derivative feedback networks is that the response time of the network is strongly dependent on the difference in time course between the excitatory-to-excitatory (EE) and excitatory-to-inhibitory (IE) projections.
We have shown previously (Gillary and Niebur 2016) that small changes in the relative timing can produce disproportionately larger (by two orders of magnitude) changes in the response time of the network. We will show below that, different from positive feedback models, derivative feedback models are robust to decreases in synaptic strength which are due to STD on their recurrent connections. However, previously described derivative feedback networks cannot explain the observed persistence of responses in sensory cortex. We will show that to produce the relative difference in onset vs. offset timing that is observed in cortex, derivative feedback networks require very large onset transients. One way to produce such large transients is through STD on the afferents, the feedforward connections to the population of interest. We show that for derivative feedback networks, STD on the afferent must be stronger than what is generally observed in cortex to produce a sufficiently strong onset transient. Therefore, neither positive feedback networks nor the derivative feedback networks described so far can explain the vast differences between onset and offset time courses.
Understanding the observed neuronal behavior requires a slight generalization of the previously discussed derivative networks which, at the same time, makes the model more biophysically realistic. We have, so far, adopted the commonly made simplification that STD can be described by a single dynamic variable that describes the activity-related depletion of synaptic resources, e.g. by using up synaptic vesicles at a higher rate than they can be renewed. We call such networks “homogeneous” derivative networks. While this is a reasonable point of departure, it is unlikely that all synapses will have completely identical STD characteristics. It is more likely that different synapses, neurons and projections express some range of activity-related synaptic depression (on longer time scales which are better understood, it is known that synaptic strength even can switch signs, from long term depression to long term potentiation and vice-versa, when calcium concentration or intracellular voltage change (Tsumoto and Yasuda 1996; Hansel et al 1997; Cormier et al 2001). While the assumption of completely identical STD characteristics in all synapses is unlikely to be true, few details about the range and distribution of STD parameters are known. We therefore explore the assumption that the distribution of STD takes on the simplest non-homogeneous form. That is, we assume that one synaptic population has stronger STD than the other, with no further differentiation within these populations. We call derivative networks with this property “heterogeneous” networks.
After we have relaxed the unphysiological restriction to homogeneous networks, we are now in a position to explain the observed neuronal behavior by making one additional assumption. Glutamatergic pathways in cortex typically have synapses which are composed of a mixture of α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) and N-methyl-D-aspartate (NMDA) receptors, a property that we make use of in our model to allow for a variety of time courses in the recurrent connections. Different time courses are due to different relative contributions from synapses with different ratios of fast (AMPA) and slow (NMDA) receptors. We approximate the possible range of relative contributions of AMPA and NMDA by two synapse types, one which is AMPA-dominated with fast kinetics and another which is NMDA-dominated, with slow kinetics. These appear in the two recurrent pathways of our network, excitatory-to-excitatory and excitatory-to-inhibitory. The assumption that we will make is that the two types of STD (stronger and weaker) are assigned specifically to the fast and slow pathways, as described in more detail below.
While it is unlikely that the effect of STD is identical in different pathways, we want to make clear that, at this time, the specific assignment of STD to the fast and slow pathways as it is described below Eqs. (16-18) is a prediction of our model. There are many ways in which STD could differ between pathways. While we are not aware of any experimental evidence arguing against our assumption, we also do not know of direct evidence that would turn our assumption into a known fact. We make this prediction because it provides a simple and, we feel, plausible mechanism to explain the existence of sensory memory in cortical networks, with fast rise and slow decay of neuronal activity. We also note that slightly more general mechanisms than those we adopted can explain the behavior. We have chosen the simplest and most parsimonious model requiring the smallest number of free parameters but so-far unknown biophysical or physiological constraints may require a generalization of the model while still allowing it to explain the occurrence of sensory memory.
If STD decreases the strength of, say, the faster component of a projection more than that of its slower component, the overall time constant of the projection will change (become longer, in this case) over the course of a stimulus presentation. As we have shown in previous work (Gillary and Niebur 2016), changes in the relative time courses of positive and negative feedback connections in this balanced network lead to proportionally larger changes in the response time course of the network as a whole. If the EE projection slows relative to the IE projection, the speed of response of the network can also change significantly over a single stimulus presentation. With appropriate starting parameters these networks are capable of rising quickly at stimulus onset and then decaying slowly after stimulus offset. Simulation results (below) demonstrate that our conclusions hold for not only for rate models but also for networks of spiking neurons.
Finally, since STD causes the strength of individual connections to change over the course of a stimulus presentation and is also dependent upon the firing rate of the afferent projection, one might expect the decay times of such networks to change with different stimulus presentation lengths and for different steady state firing rates. Contrary to this expectation, previous experimental work has shown that the decay time of networks with persistent activity does not depend strongly on either parameter (O’Herron and von der Heydt 2009). Indeed, we show that the decay time of a network with heterogeneous feedback connections is well maintained over a range of stimulus presentation lengths as well as for different baseline activity levels. This is due to the reduced synaptic strengths being balanced by a slowing of the EE projection relative to the IE projection. In contrast, derivative feedback networks with homogeneous STD have network decay times which are much more strongly dependent on both the stimulus length and steady state firing rate than the network with heterogeneous STD.
2 Methods
2.1 Positive feedback model
We use the terms “rise time” and “decay time” to describe the speed of network response to the onset and offset of a stimulus respectively. We define the rise time as the time for R_{e} to go from 10% to 90% of its steady state value \(R_{e}^{ss}\) (computed below) when the stimulus is present, and vice-versa for the decay time. We refer to rise time and decay time generally as response times of the network.
2.2 Derivative feedback model
Smnl in Eqs. (11)-(13) is the synaptic activation level of the projection from population n to population m where l can be either ampa or nmda for AMPA or NMDA receptors on an excitatory projection, or gaba for GABA (γ −Amino-Butyric Acid) receptors on inhibitory projections where τ^{ampa} = 5 ms, τ^{nmda} = 100 ms and τ^{gaba} = 10 ms.
Projections from the inhibitory populations, Eq. (13), have only a single type of receptor, l = gaba, and do not have STD. Conversely, excitatory projections have multiple receptor types, l ∈{ampa, nmda}, which may occur in different ratios across synapses. They also have STD whose parameters may have different values across synapses allowing multiple possible combinations. We define a particular combination of AMPA/NMDA ratio, \(q_{mn}^{j}\), and set of STD, \(x_{mn}^{j}\), parameters by the index j and call it a synapse type.N is the number of synapse types for each excitatory projection, defined below. Note that Eq. (11) represents two equations, one for the (AMPA-)receptors on the projection from the excitatory population to itself (m = n = e), and one from the excitatory to the inhibitory projection (m = i, n = e). The analogous comment applies to Eqs. (12)-(14).
For projections from the excitatory population, \(q_{mn}^{j}\) defines the fraction of total synaptic strength for synapse type j from population n to population m that is represented by NMDA receptors. \(x_{mn}^{j}\) is the STD variable for synapse type j on the projection from populations n to m with baseline recovery time constant τ_{r}, and usage rate u, as computed in Eq. (14). We set τ_{r} = 500 ms for all simulations. We add an additional variable \(p_{mn}^{j}\) which defines a percentage shift from the baseline STD parameters.
We define two different models of STD, “homogeneous” and “heterogeneous”. For “homogeneous STD”, STD is the same for all excitatory projections, \(p_{mn}^{j}=0\) in Eq. (14). For “heterogeneous STD,” \(p_{mn}^{j}\) has multiple multiple non-zero values on each excitatory projection, see below for details. Note that in the heterogeneous STD model, STD is modified τ_{r} and through u, and that in both cases the dependence on \(p_{mn}^{j}\) is the same (both have a \((1+p^{j}_{mn})\) multiplicator). We make this choice for the sake of simplicity and parsimony but, if needed, it is possible to modify STD dynamics by changing only one of these parameters, or selecting different functions of \(p_{mn}^{j}\) for one or both of these parameters.
2.3 Leaky integrate and fire networks
We use j to define different types of synapses. For excitatory projections, a synapse is defined by its ratio of NMDA to AMPA receptors, \(q^{j}_{mn}\), and its STD parameter \(p_{mn}^{j}\). The index l separates the types of receptors and their activations in Eqs. (31)-(32) such that each synapse indexed by m and n has a type j which sets the relationship between AMPA and NMDA receptors indexed by l, see Eqs. (34)-(35). For all LIF networks the excitatory projections are evenly split between synapses which are 25% NMDA (\(q^{fast}_{mn}=0.25\)) and those which are 75% NMDA (\(q^{slow}_{mn}=0.75\)) so that the total amount of current through NMDA receptors is the same as through AMPA receptors. We define Δτ as in the rate based model using Eqs. (15) and (18). For inhibitory synapses there is only a single receptor type and no STD so Eq. (36) fully defines the inhibitory synapses.
The overall connectivity pattern of each network is the same as in the rate based simulations (see Fig. 1a for the positive feedback network and Fig. 1b for the derivative feedback network). For the positive feedback network with LIF neurons, the strength of recurrent connections is defined by a single overall parameter w and there is no inhibitory population. For the derivative feedback networks, excitatory projections have maximal strength w and inhibitory projection have strength kw.
For all networks without STD \(x^{j}_{mn}=1\) for all synapses and Eq. (33) can be ignored as in the rate based networks. For the positive feedback network with STD and the derivative feedback network with homogeneous STD, STD is the same at all synapses such that \(p^{j}_{mn}=0\) for every synapse in Eq. (33).
2.4 Summary of model parameters
Summary of model parameters
Model | Parameters |
---|---|
Positive Feedback: | |
no STD | w and I_{ff} |
with Recurrent STD | w, u, τ_{r} and I_{ff} |
Derivative Feedback: | |
no STD | w, k, Δτ and I_{ff} |
with homogeneous STD | w, k, Δτ, u, τ_{r} and I_{ff} |
with homogeneous STD and feedforward STD | w, k, Δτ, u, τ_{r}, u_{ff}, \(\tau _{r}^{ff}\), w_{ff} and R_{ff} |
with heterogeneous STD | w, k, Δτ, u, τ_{r}, p and I_{ff} |
with heterogeneous STD and feedforward STD | w, k, Δτ, u, τ_{r}, p, u_{ff}, \(\tau _{r}^{ff}\) and R_{ff} |
All Matlab and Python code used in this paper is available at: https://github.com/llgigll/FastRiseSlowDecay.
3 Results
3.1 The impact of STD on positive feedback networks
In this section we examine the impact of short-term depression on the decay time of positive feedback networks. The attractor states of such networks have previously been examined in the context of STD and their decay times have also been examined, however without STD (Barak and Tsodyks 2007; Gavornik et al. 2009). Here we examine how transient decay times of positive feedback networks are influenced by STD of their recurrent projections. A positive feedback network can be represented simply by an excitatory population with a single recurrent projection, as in Fig. 1a. We define transient decay as a network returning to baseline activity after the offset of a stimulus. We will show that for positive feedback networks, with STD on the recurrent connections, the time constant of decay is significantly shorter than without STD, and that it is more dependent on the overall strength of STD than on the starting synaptic weights.
The dynamics of a rate-based network with a single excitatory recurrent connection are determined predominantly by the strength of that recurrent connection, x(t)w, see Fig. 1a. As a reminder, w is the maximal strength of the recurrent connection and x is the effect of short-term depression which is parameterized by the usage rate u and the recovery rate τ_{r}. We will look at the response of the network to a step input for a range of values of w and u. We hold τ_{r} constant but its impact on the network is similar to changing u. We require w ≤ 1 which ensures that the network state decays back to zero after stimulus offset (see Eq. (8) with I = 0).
Figure 1b shows the results of simulations of the network in Fig. 1a for four different strengths of STD but with the same steady state activity. With no STD on the recurrent connection the network has a decay time of 25 s. We chose 25 s arbitrarily to show the significant impact of STD on the decay time. Note that since the stimulus lasts for only 2 s the simulation without STD does not come close to reaching its steady state activity before the offset of the stimulus. Even relatively weak STD yields a dramatic decrease in the decay time of the network. When u = 0.05, which is at the low end of usage rates observed in cortex (Beck et al. 2005), the decay time of the network decreases to a few hundred milliseconds. This is a change of two orders of magnitude and the decrease becomes even larger for usage rates of 0.1 and 0.2 which are more consistent with those observed in early sensory areas (Beck et al. 2005). We note that the rise time of the network also decreases as STD becomes stonger. A reduction in overall response times to both the onset and offset of a stimulus could be one reason that early sensory areas tend to be dominated by STD since fast reaction to changing sensory input is a likely priority. We focus on decay time here but will consider rise time in a subsequent section.
Although this network is nonlinear, the change in the decay time constant of the network can be understood by considering the decay time constant of a linear network (without recurrent STD) as a function of the synaptic strength at each time point in the simulations in Fig. 1b. In this case the linear network would have an instantaneous recurrent strength w_{lin} = x(t)w. Figure 1c shows the decay times for linear networks with recurrent strength w_{lin} at each time point in Fig. 1b. By definition, the network with no STD has a constant decay time. In contrast, networks with STD show a precipitous decrease in decay time after stimulus onset. This decline occurs because the decay time of a linear positive feedback network is proportional to 1/(w − 1) (seen directly from a simplified linear version of Eq. (1)). Figure 1d shows this relationship between decay time and w for the network without STD. Decay time for each value of w is computed directly from a simulation. Irrespective of the starting value of w, STD decreases the effective synaptic strength and thus drives each network toward the left hand side of Fig. 1d, compressing its decay time. When the network dynamics show long time constant decay, then very small changes in w produce large changes in decay time (imagine decreasing w in Fig. 1d when w ≈ 1).
Figure 1e shows that the compression of decay time that we would expect from the linear model does actually occur for networks with recurrent STD. We see that the addition of even weak STD, u = 0.05, reduces the decay time to approximately 700 ms or less for all values of w. Stronger STD reduces network decay times even further. In addition to decreasing the decay time of the network, recurrent STD also makes networks with w > 1 stable. However, when w > 1 in a positive feedback network with STD the network will decay to a non-zero steady state after the offset of a stimulus. Since we are considering networks without attractor like dynamics this provides an effective upper limit on the network decay time at w = 1. Additionally, increasing w past unity does not significantly increase the decay time of the network, even to an attractor state (Barak and Tsodyks 2007). Figure 1f shows the maximum decay time of the network (for w = 1) for a range of usage rates. The figure shows that it is very strongly dependent on u. When u < 0.05 then the network approaches the behavior of the linear system.
We conclude that in networks with STD and physiological usage rates, the positive feedback network has a decay time less than 500 ms. This implies that persistence in early sensory areas, where STD dominates, must be caused by a mechanism other than positive feedback. Figure 1g shows the same dynamics as in Fig. 1a but for a network made up of LIF neurons. The addition of STD has the same impact on decay time as it does in the rate based network implying that these concepts also hold for stochastic spiking networks. In the next section we will consider derivative feedback networks and show that they are far more robust to the addition of STD on their excitatory connections.
3.2 Derivative feedback networks with homogeneous STD
Recurrent networks with negative derivative feedback have been discussed as a model for working memory, mainly in central areas like prefrontal cortex (Lim and Goldman 2013). Recent work has shown that derivative feedback networks with STD can also produce persistent activity that decays to the baseline value over the course of up to several seconds (Gillary and Niebur 2016). Return to the baseline value is a property needed in circuits representing sensory information where, different from short-term memory circuitry, attractor dynamics are undesirable. Figure 2a shows an example of a derivative feedback network where w represents the baseline strength of excitatory connections and kw is the strength of inhibitory projections. In such networks the decay time depends strongly on both the overall strength of the network connections and on the difference in average synaptic time constants between the EE and IE projections. In our model the average synaptic time constant is determined by the relative proportion of AMPA and NMDA receptors on the synapses in a projection. In Fig. 2a we have only one type of synapse on each projection but each projection has a different proportion of AMPA and NMDA receptors. We define projections with relatively more NMDA as slow projections and color those synapses red while projections with relatively more AMPA receptors are termed fast projections and colored green. When the EE projection slows relative to the IE projection then the response time of the network increases. We call this difference in average synaptic time constant between excitatory projections Δτ (see Eqs. (15), (18)) where positive Δτ indicates a relatively slower EE projection when compared to the IE projection. Stability of this and related systems is analyzed by Gillary and Niebur (2016).
As in the positive feedback network we are interested in how STD on the excitatory connections impacts the decay time of the network. In this section we consider derivative feedback networks with homogeneous STD on all excitatory projections (EE and IE), where STD decreases the overall strength of excitatory connections. Figure 2b shows simulations of this network for four different STD strengths. Strikingly, although we selected the parameters of the network with no STD to give the same response times as the positive feedback network with no STD shown in Fig. 1b, the decrease in decay time of the derivate feedback network due to STD is much less pronounced. This can also be seen in the decay times computed for the linear systems as a function of STD at each time point in the simulations, Fig. 2c, to be compared to the much steeper decay in Fig. 1c. Although there is a significant reduction in the decay time of the linear systems the reductions are much smaller than in the positive feedback network for the same STD parameters.
The relative robustness of the derivative feedback network is due to the linear relationship between the strength of the network projections, w, and the decay time of the linear derivative feedback network, Fig. 2d. This is in stark contrast to the approximately geometric relationship between w and the decay time constant in the linear positive feedback network, Fig. 1d. When STD is added to the derivative feedback network the relationship between w and decay time remains linear, Fig. 2e. The strength of STD determines the slope of the relationship between w and decay time. As can be seen from the plot, the derivative feedback network with STD can produce decay times which are more than long enough to replicate the persistence observed in cortex (Shuler and Bear 2006). This is in part due to the fact that there is not an upper limit on the value of w as there is in the positive feedback network where w > 1 results in either instability for the positive feedback network without STD or an attractor state for the positive feedback network with STD. Similar dynamics are also obtained for spiking networks, Fig. 2f.
We have thus shown that derivative feedback networks can develop persistent activity that is far more robust to reductions in synaptic strength due to STD than positive feedback based networks. This robustness to STD implies that, for early sensory systems where STD dominates, derivative feedback networks are significantly more capable of providing the persistence observed in experimental work. Although we have considered only the decay of the derivative feedback network, for a memory system to work, it must also be able to encode information on time scales much shorter than its memory. In the next section we examine how the rise time of the network is impacted by STD on both the recurrent connections as well on the feedforward input to the network.
3.3 Rise time in the derivative feedback network with STD
A defining characteristic of any memory mechanism is that memories are acquired much faster than they decay. However, such dual time constant responses are not possible in linear networks. This is because responses of such linear networks are symmetric to step increases and decreases of equal and opposite magnitude in the input. Therefore, if we change the parameters to yield persistent activity after stimulus offset we also slow down the response of the network to stimulus onset. This is not necessarily the case any more for nonlinear systems, e.g. when we introduce STD. Figure 3a shows three simulations of the derivative feedback network with STD for different values of Δτ (see Eq. (18)). As Δτ increases we see an increase in both the rise time and the decay time of the network, similar in tendency to what we would expect in a linear system. The STD-associated nonlinearity on the excitatory connections, does, however break the exact symmetry between the offset and onset response of the network. Figure 3d shows that for a large range of Δτ and different STD strengths the rise time of the network is only about 30–50 % of the decay time. Therefore, a network with a decay time of ∼1 second will have a rise time of ∼300–500 milliseconds. While the decay times are comparable to those observed in sensory cortex, the rise time is far longer than observed. Furthermore, rise time and decay time are strongly correlated across the range of Δτ’s. In cortex, a correlation between the rise time and the decay time of the network is not observed (Shuler and Bear 2006; O’Herron and von der Heydt 2009).
In order to decouple the response times at the onset of the stimulus from those at the offset we require some additional form of nonlinearity. In previous work this decoupling was obtained through strong input transients to the network on a base level of much lower steady state input (Gavornik et al. 2009; Lim and Goldman 2013). Such a pulse-step approach can be accomplished with STD on the feedforward input to the network. Figure 4a-c depicts how STD produces a step input for three different feedforward usage rates, u_{ff}, see Eq. (21). The overall input to the excitatory population is I_{in}(t) = w_{ff}x_{ff}(t)R_{ff}(t) where w_{ff} is the (constant) starting strength of the feedforward projection, x_{ff} is the STD influence, and R_{ff} is the firing rate of the input. The step input to the excitatory population is produced by a step change in R_{ff} which rises at t = 0 s and returns to baseline at t = 2 s. The step input drives STD on the afferent synapses, Fig. 4b. The combination of decreasing synaptic strength and step change in input yield a large initial transient current followed by a smaller steady state input, Fig. 4c. The largest usage rate, u_{ff} = 0.5, see Fig. 4c, approaches the tail end of the distribution of usage rates that is observed in cortex (Beck et al. 2005). For this value, the input current attains a peak that is approximately three times the steady state value. Such a ratio is consistent with what has been observed in thalamic input to layer 4 cells in mouse primary visual cortex (Reinhold et al. 2015).
In Fig. 4d we chose Δτ such that the network has a 1 second long decay time. This network receives the three different transient inputs depicted in Fig. 4c. Although the transients do shorten the rise time of the network, all three values produce long rising transients peaking after 350 milliseconds. These transients are much longer than those observed for cortical networks with persistent activity. We conclude that, although STD on the input to the network decreases rise time relative to decay times, it is not sufficient to produce the observed fast onset transients and, at the same time, persistence over the observed range. In the following section, we show that heterogeneous STD on the recurrent connections can change the underlying time constant of the network from fast to slow over the course of a stimulus presentation yielding the observed difference in response time.
3.4 Derivative feedback networks with heterogeneous STD
Figure 5b shows the evolution of strong and weak STD for a step input at t = 0. The STD decreases the overall strength of the input to the population by ≈ 60% over the course of the stimulus presentation while the difference in average synaptic time constant between the projections, Δτ, increases by 8 ms, Fig. 5c. Looking back at Fig. 3c we see that a change Δτ in the derivative feedback network of 8 ms can change the decay time of the network by seconds. Therefore, if STD causes an increase in the average synaptic time constant on the EE projection and a decrease on the IE projection, then Δτ for the network will increase by a large margin. We call networks with STD of this form derivative feedback networks with heterogeneous STD, see Fig. 5a.
The initial fast response of the network is due to the negative starting value of Δτ. However, the network will also produce a fast response with larger starting values of Δτ if there is STD on the input, as in Fig. 4. Figure 6e shows simulations of the derivative network with heterogeneous STD and, in addition, with STD on the afferent projection. These simulations still show long decay times and short rise times, see also Fig. 6f. Additionally, we see that the dip following stimulus onset seen in Fig. 6a is reduced or removed when there is STD on the afferent synapses.
What Fig. 6 essentially shows is that the combination of STD and derivative feedback networks can reproduce a wide variety of dynamics that are observed in cortex by altering parameters in the local network. This is in contrast to both positive feedback networks and derivative feedback networks with homogeneous STD in which fast rise and slow decay can only be achieved by assuming unphysiologically high rates of STD on the input and virtual absence of STD on the recurrent connections.
In Fig. 6g,h we compare the behavior of our network with neuronal activity in secondary visual cortex (area V2) of two awake behaving monkeys. We use the data in Fig. 2a from O’Herron and von der Heydt (2009) which shows the difference of firing rates of border-ownership selective neurons between responses to an object in their preferred and non-preferred locations. In the experiment, a stimulus is presented either in the preferred or non-preferred border ownership location from t = 0 to t = 0.5 s, after which the object is replaced by an edge in the receptive field of the neuron that is devoid of any border ownership information, see O’Herron and von der Heydt (2009) for details. The blue curves in Fig. 6g,h show this difference for the two monkeys. It rises very fast (within less than 50 ms after stimulus onset), is maintained while the stimulus has border ownership information (until t = 0.5 s), and then decays slowly to zero over the course of about one second. It is believed that the modulation is produced by populations of neurons (“grouping cells”) in a downstream cortical area that represent the grouping of object features (Craft et al. 2007; Mihalas et al. 2011; Sugihara et al. 2011; Russell et al. 2014). We model grouping cell populations as derivative feedback networks with heterogeneous STD and plot their responses (orange lines) along with experimental data (blue lines) in Fig. 6g,h. The only difference in the fits between the two plots is a slightly higher value of p, in Fig. 6g than in Fig. 6h, chosen to match the decay time of each network. The full parameter set is listed in the figure caption. The agreement of simulation results with the experimental data for both monkeys is very good, suggesting that dynamics generated by a derivative network with heterogeneous STD may be responsible for the generation of the observed modulation of neuronal activity.
The network can be re-balanced without the anti-symmetric condition. Instead, we can slightly reduce the strength of STD on the IE projection such that the network maintains the balance of positive and negative feedback after activation. In Fig. 8b we set \(p_{ee}^{fast}=p\), \(p_{ee}^{slow}=-p\) and \(p_{ie}^{fast}=p_{ie}^{slow}=p_{bal}\) where p_{bal} is a reduction in the strength of STD on the IE projection that balances the positive and negative feedback but produces no change in the timing of positive and negative feedback. We use values of p_{bal} = {0, −0.0015, −0.0061, −0.0135} for each respective value of p = {0, 0.05, 0.1, 0.15}. The values of p_{bal} were calculated by matching the steady state strength of the EE and IE projections for a firing rate of 25 Hz for each value of p. Figure 8b shows that this way of balancing the network maintains the temporal dynamics while preventing the network from approaching an attractor state.
We note that there is an overall reduction in the range of possible decay times as compared to previous simulations. This reduction occurs because p_{bal} yields no difference in timing on the IE projection thereby reducing Δτ. It should also be noted that the purple trace (p = 0.15) does have a longer decay time than the yellow trace which is lost due to the purple trace failing to reach steady state before the offset of the input. These simulations show that although the anti-symmetric condition simplifies the study of the heterogeneous network, it is not a necessity to get both fast rise and slow decay. The removal of the anti-symmetric condition simply requires that STD parameters are calculated such that positive and negative feedback are balanced. Consequently, Fig. 8 does show that our model requires that the balance between positive and negative feedback is maintained, something to be expected in any network with strong recurrent connections. A balance between excitation and inhibition is also required by other models as well as observed during in-vivo recordings in visual cortex (Rubin et al. 2015; Lim and Goldman 2013; Xue et al. 2014).
3.5 Robustness of the heterogeneous network response
While both positive feedback models and derivative feedback network models with homogeneous STD predict strongly decreasing decay times for longer stimulus presentations, the derivative feedback network with heterogeneous STD shows an increasing decay time for longer stimulus lengths, as seen in Fig. 10b. Such behavior has been recently observed in the olfactory bulb (Patterson et al. 2013). Additionally, adaptation aftereffects in perception generally show the same correlation: longer adaptation produces longer lasting aftereffects (Greenlee et al. 1991; Leopold et al. 2005). This prediction could easily be tested more generally in any experimental paradigm yielding persistent activity after stimulus offset.
Figure 10c shows simulations for the same derivative feedback network as Fig. 10a but with different starting feed-forwards weights, w_{ff}, to yield a range of steady state firing rates, \(R_{e}^{ss}\), for the excitatory population. Furthemore, now the input duration is the same for all three cases. As in the simulations with different stimulus lengths in panel a the different trials do not show appreciably different rise times. There is, however, an inverse correlation between \(R_{e}^{ss}\) and the decay time of the network, dashed line Fig. 10d. For this choice of parameters the decay time of the network with heterogeneous STD decreases with a slope of 30 ms/Hz between 10 Hz and 50 Hz. However, the homogeneous network, solid line, has an average slope of 75 ms/Hz between 10 Hz and 50 Hz, far larger than for the network with heterogeneous STD. Overall Fig. 10 shows that, in addition to being able to replicate the fast rise and slow decay observed in cortex, the network response, when decay times are driven by heterogeneous STD, is also significantly more robust to changes in input than networks whose decay time is determined by a static Δτ.
4 Discussion
Persistent decaying activity has been observed in a number of early sensory areas. Most previous work on neuronal persistence has been on understanding short-term memory. Substantial progress has been made by the study of networks with attractor dynamics. Such dynamics are, however, not suitable for sensory coding whose task it is to efficiently and faithfully represent environmental variables. In addition (and possibly for this reason), synapses show much stronger short-term depression effects in early sensory cortex than in more central areas (Hempel et al. 2000). Here we examine the impact of STD on the transient dynamics of two classes of circuit models, positive feedback and derivative feedback networks. Both network types have been used to produce either slow integration or slow decay at the onset and offset of neural signals (Lim and Goldman 2014).
We first show that the dynamics for positive feedback models are strongly influenced by the presence of STD on the recurrent excitatory connections. Even for relatively weak STD, such networks can only produce fast onset and offset dynamics but not the observed persistent activity. Conversely, we show that the range of transient dynamics for derivative feedback networks is robust to STD. However, as with positive feedback networks, derivative feedback networks which decay slowly at stimulus offset also respond slowly to stimulus onset. Such networks can only produce a fast onset response while maintaining a level of persistence if they are driven by a large initial transient input. While onset transients are commonly observed in sensory cortex, their magnitude is much lower than what is required to generate the large difference between the time courses of rise and decay of network activity found experimentally. The derivative feedback networks considered at this point therefore do not explain observed neuronal behavior either. Therefore, we look for a way to produce fast rise times and slow decay times by altering the intrinsic time constant of the network rather than through the shape of the input.
It turns out that removing a simplification we (and others) had originally made in the construction of the model solves this problem. We had adopted the standard model of STD (Tsodyks et al. 1998) in which synaptic strength is governed by a single variable (the case \(N=1, p_{mn}^{j}=0\) in Eqs. (11)–(14) and discussed in Section 3.1). In reality, most networks have a range of STD values (Castro-Alamancos and Connors 1997). We approximate this situation in possibly the simplest way by allowing the excitatory projections (EE and IE) to have synapses with different strengths of short-term depression. Different from the homogeneous STD model considered earlier, we now assume that each projection has two types of synapse, one fast-acting and one slow-acting, as an approximation for the more general case in which a whole range of time courses exists. As for the homogeneous model, and in agreement with physiological data (Myme et al. 2003), each of these projections consists of a mixture of NMDA-type and AMPA-type receptors.
The fast and slow components are naturally obtained through synapses with different proportions of fast (AMPA) and slow (NMDA) channel dynamics. We assume, somewhat arbitrarily that the ratio is three-to-one in both cases, Eq. (16) (but show that our results do not depend critically on this choice, Fig. 7). The crucial modification from the homogeneous model is that we assume that these two components have different short term depression behavior. Little quantitative information is available about the details of the distributions of STD characteristics, therefore we make the simplifying assumption that STD is the same on the fast EE connection and the slow IE connection, and equal but opposite on the other two connections, see Eq. (17). While the resulting reduction of free parameters, from four to one, makes the network easier to understand, we emphasize that this constraint can be easily relaxed as soon as more information about the relative and absolute strength of STD in different recurrent connections becomes available. In Fig. 8b we show one particular example of how this constraint might be relaxed. However, in a network with many synaptic connections there would also be many possible ways to maintain the balance between excitation and inhibition while also increasing Δτ. Likewise, the assumptions on the relative strengths of slow and fast synapses, Eq. (16), which result in the same reduction of the number of free parameters, can be easily generalized, as can the number of contributing components, by increasing the value of N in Eqs. (11), (12).
This generalization from the homogeneous network allows us to change the intrinsic time constant of the network dynamically, rather than through the shape of the input, thus generating fast rise times and slow decay times. The reason is that the response times of derivative feedback networks depend on the difference in timing across the excitatory projections. The response time of the network over a stimulus presentation is now changed by the stimulus itself as it changes the effective AMPA to NMDA ratio of synaptic strength differently on the different excitatory projections (EE and IE). We show that in the model with heterogeneous STD parameters on the excitatory projections the average synaptic time constant of those projections changes over the course of the stimulus presentation. Such a network can respond quickly to stimulus onset and decay slowly (over a time course that can reach seconds) after stimulus offset. We also show that the model with STD driven persistence is more robust than the derivative feedback network with homogeneous STD to changes in both the stimulus length and changes in the steady state firing rate of the network. We note that the specific relationship between STD and AMPA/NMDA ratios postulated by our model has not previously been studied in the context of networks in sensory cortex showing long time constant decay. Our model therefore makes the prediction that, in such networks, synapses in the EE projection with a higher ratio of AMPA receptors have stronger STD than synapses with more NMDA receptors, and that the opposite is the case for the IE projection.
A central result of our work is that the addition of STD dramatically alters the transient dynamics of local networks. For networks whose dynamics depend on positive feedback, STD narrows the range of possible response times while, for derivative feedback networks, STD allows a network to show two substantially different response times at the onset and offset of a stimulus. The transient responses of these networks are usually considered in the context of connectivity matrices for linear networks with particular connection strengths between either populations or neurons (Ganguli et al. 2008). STD constantly alters the state of these connectivity matrices, effectively yielding a different network for every point in time. Therefore, STD acts to change the state space of the network and can dramatically alter its dynamics depending on the starting network state and the distribution of STD. This implies that the distribution of STD may be as important to the response of the network as the underlying structure and baseline synaptic strengths.
Although we only consider STD, the addition of short-term facilitation (STF) on the excitatory projections would expand the range of dynamics available to both types of network. For example, positive feedback networks dominated by facilitation can produce persistent activity which decays to baseline level rather than approaching an attractor different from the baseline (Mi et al. 2014). However, STF tends to slow the onset response of a network and prevents the network from producing the fast responses generally observed in early sensory areas (Barak and Tsodyks 2007). In future work it would be interesting to examine how weak facilitation on the excitatory projections may impact the transient responses of both positive feedback and derivative feedback networks. We also note that STF has been observed on inhibitory synapses. While our work has focused on the impact of short-term potentiation on excitatory synapses, the changing strength of inhibitory synapses may also affect the network (Gupta et al. 2000; Reyes 2011).
While stimulus dependent changes in temporal structure have been observed in early sensory areas, their impact on computation is currently unclear. It has been proposed that such activity may be used in networks that learn the temporal structure of stimuli (Johnson et al. 2010). Additionally, in reward timing paradigms cholinergic mechanisms are believed to train networks to represent the temporal relationship between a stimulus and a reward (Chubykin et al. 2013). Such persistence may also act like a transient memory after the offset of a stimulus (Pasternak and Greenlee 2005; O’Herron and von der Heydt 2009). In addition to changing the intrinsic time constant of the network during a stimulus presentation, our network could also change the intrinsic time constant of a network between stimulus presentations. Such a change would be driven by differences in the recovery rate of the synapses and could allow changes in the timing of response to stimuli across presentations.
In addition to the already discussed prediction of the specific distribution of STD strengths in the fast and slow synapses of EE and IE projections in balanced derivative networks, our work makes a number of more general predictions concerning how networks dominated by STD may produce long time constant decay. For positive feedback networks, reducing the strength of STD would be the primary method to increase the network’s decay time. Derivative feedback networks, on the other hand, have a number of different ways that the persistence of decay activity can be altered. An overall increase in w, the strength of recurrent connections, would increase the decay time. As in the positive feedback network, reducing the strength of STD would also have a similar although less drastic effect. Each of these parameters provides a way to change the response time of the network although only heterogeneous STD driven persistence can change the decay time of the network without significantly altering the rise time.
Notes
Acknowledgements
We thank Daniel Jeck and Brian Hu for many useful discussions. We would also like to thank Philip O’Herron for help in accessing the physiological data.
Compliance with Ethical Standards
Conflict of interests
The authors declare that they have no conflict of interest.
References
- Barak, O., & Tsodyks, M. (2007). Persistent activity in neural networks with dynamic synapses. PLoS Computational Biology, 3(2), e35.CrossRefPubMedPubMedCentralGoogle Scholar
- Beck, O., Chistiakova, M., Obermayer, K., & Volgushev, M. (2005). Adaptation at synaptic connections to layer 2/3 pyramidal cells in rat visual cortex. Journal of Neurophysiology, 94(1), 363–376.CrossRefPubMedGoogle Scholar
- Castro-Alamancos, M.A., & Connors, B.W. (1997). Distinct forms of short-term plasticity at excitatory synapses of hippocampus and neocortex. Proceedings of the National Academy of Sciences, 94(8), 4161–4166.CrossRefGoogle Scholar
- Chubykin, A.A., Roach, E.B., Bear, M.F., & Shuler, M.G.H. (2013). A cholinergic mechanism for reward timing within primary visual cortex. Neuron, 77(4), 723–735.CrossRefPubMedPubMedCentralGoogle Scholar
- Compte, A., Brunel, N., Goldman-Rakic, P.S., & Wang, X.J. (2000). Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model. Cerebral Cortex, 10(9), 910–923.CrossRefPubMedGoogle Scholar
- Cormier, R., Greenwood, A., & Connor, J. (2001). Bidirectional synaptic plasticity correlated with the magnitude of dendritic calcium transients above a threshold. Journal of Neurophysiology, 85(1), 399–406.PubMedGoogle Scholar
- Craft, E., Schütze, H., Niebur, E., & von der Heydt, R. (2007). A neural model of figure-ground organization. Journal of Neurophysiology, 97(6), 4310–26. PMID17442769.CrossRefPubMedGoogle Scholar
- Curtis, C.E., & D’Esposito, M. (2003). Persistent activity in the prefrontal cortex during working memory. Trends in cognitive sciences, 7(9), 415–423.CrossRefPubMedGoogle Scholar
- Denève, S., & Machens, C.K. (2016). Efficient codes and balanced networks. Nature Neuroscience, 19(3), 375.CrossRefPubMedGoogle Scholar
- Ganguli, S., Huh, D., & Sompolinsky, H. (2008). Memory traces in dynamical systems. Proceedings of the National Academy of Sciences, 105(48), 18:970–18:975.CrossRefGoogle Scholar
- Gavornik, J.P., & Shouval, H.Z. (2011). A network of spiking neurons that can represent interval timing: mean field analysis. Journal of Computational Neuroscience, 30(2), 501–513.CrossRefPubMedGoogle Scholar
- Gavornik, J.P., Shuler, M.G.H., Loewenstein, Y., Bear, M.F., & Shouval, H.Z. (2009). Learning reward timing in cortex through reward dependent expression of synaptic plasticity. Proceedings of the National Academy of Sciences, 106(16), 6826–6831.CrossRefGoogle Scholar
- Gillary, G., & Niebur, E. (2016). The edge of stability: Response times and delta oscillations in balanced networks. PLoS Computational Biology, 12(9), e1005,121.CrossRefGoogle Scholar
- Greenlee, M.W., Georgeson, M.A., Magnussen, S., & Harris, J.P. (1991). The time course of adaptation to spatial contrast. Vision Research, 31(2), 223–236.CrossRefPubMedGoogle Scholar
- Guo, K., Mahmoodi, S., Robertson, R.G., & Young, M.P. (2006). Longer fixation duration while viewing face images. Experimental Brain Research, 171(1), 91–98.CrossRefPubMedGoogle Scholar
- Gupta, A., Wang, Y., & Markram, H. (2000). Organizing principles for a diversity of gabaergic interneurons and synapses in the neocortex. Science, 287(5451), 273–278.CrossRefPubMedGoogle Scholar
- Hansel C, Artola A, & Singer W (1997). Relation between dendritic Ca2+ levels and the polarity of synaptic long-term modifications in rat visual cortex neurons. European Journal of Neuroscience, 9(11), 2309–2322.CrossRefPubMedGoogle Scholar
- Hardy, N.F., & Buonomano, D.V. (2016). Neurocomputational models of interval and pattern timing. Current Opinion in Behavioral Sciences, 8, 250–257.CrossRefPubMedPubMedCentralGoogle Scholar
- Hempel, C.M., Hartman, K.H., Wang, X.J., Turrigiano, G.G., & Nelson, S.B. (2000). Multiple forms of short-term plasticity at excitatory synapses in rat medial prefrontal cortex. Journal of Neurophysiology, 83(5), 3031–3041.PubMedGoogle Scholar
- Johnson, H.A., Goel, A., & Buonomano, D.V. (2010). Neural dynamics of in vitro cortical networks reflects experienced temporal patterns. Nature Neuroscience, 13(8), 917–919.CrossRefPubMedPubMedCentralGoogle Scholar
- Leopold, D.A., Rhodes, G., Müller, K. M., & Jeffery, L. (2005). The dynamics of visual adaptation to faces. Proceedings of the Royal Society of London B: Biological Sciences, 272(1566), 897–904.CrossRefGoogle Scholar
- Lim, S., & Goldman, M.S. (2013). Balanced cortical microcircuitry for maintaining information in working memory. Nature Neuroscience, 16(9), 1306–1314.CrossRefPubMedPubMedCentralGoogle Scholar
- Lim, S., & Goldman, M.S. (2014). Balanced cortical microcircuitry for spatial working memory based on corrective feedback control. The Journal of Neuroscience, 34(20), 6790–6806.CrossRefPubMedPubMedCentralGoogle Scholar
- Major, G., & Tank, D. (2004). Persistent neural activity: prevalence and mechanisms. Current Opinion in Neurobiology, 14(6), 675–684.CrossRefPubMedGoogle Scholar
- Mi, Y., Li, L., Wang, D., & Wu, S. (2014). A synaptical story of persistent activity with graded lifetime in a neural system. In Advances in Neural Information Processing Systems (pp. 352–360).Google Scholar
- Mihalas, S., Dong, Y., von der Heydt, R., & Niebur, E. (2011). Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects. Proceedings of the National Academy of Sciences, 108(18), 7583–8. PMC3088583.CrossRefGoogle Scholar
- Murphy, B., & Miller, K. (2009). Balanced amplification: A new mechanism of selective amplification of neural activity patterns. Neuron, 61(4), 635–648.CrossRefPubMedPubMedCentralGoogle Scholar
- Myme, C.I., Sugino, K., Turrigiano, G.G., & Nelson, S.B. (2003). The NMDA-to-AMPA ratio at synapses onto layer 2/3 pyramidal neurons is conserved across prefrontal and visual cortices. Journal of Neurophysiology, 90 (2), 771–779.CrossRefPubMedGoogle Scholar
- Nikolić, D, Häusler, S, Singer, W., & Maass, W. (2009). Distributed fading memory for stimulus properties in the primary visual cortex. PLoS Biology, 7(12), e1000, 260.CrossRefGoogle Scholar
- O’Herron, P., & von der Heydt, R (2009). Short-term memory for figure-ground organization in the visual cortex. Neuron, 61(5), 801–809. PMC2707495.CrossRefPubMedPubMedCentralGoogle Scholar
- O’Herron, P., & von der Heydt, R. (2011). Representation of object continuity in the visual cortex. Journal of Vision, 11(2). PMC3160770.Google Scholar
- Pasternak, T., & Greenlee, M.W. (2005). Working memory in primate sensory systems. Nature Reviews Neuroscience, 6(2), 97–107.CrossRefPubMedGoogle Scholar
- Patterson, M.A., Lagier, S., & Carleton, A. (2013). Odor representations in the olfactory bulb evolve after the first breath and persist as an odor afterimage. Proceedings of the National Academy of Sciences, 110(35), E3340–E3349.CrossRefGoogle Scholar
- Petersen, C.C. (2002). Short-term dynamics of synaptic transmission within the excitatory neuronal network of rat layer 4 barrel cortex. Journal of neurophysiology, 87(6), 2904–2914.PubMedGoogle Scholar
- Petreanu, L., Gutnisky, D.A., Huber, D., Xu, N.L., O’Connor, D.H., Tian, L., Looger, L., & Svoboda, K. (2012). Activity in motor-sensory projections reveals distributed coding in somatosensation. Nature, 489(7415), 299–303.CrossRefPubMedPubMedCentralGoogle Scholar
- Reinhold, K., Lien, A.D., & Scanziani, M. (2015). Distinct recurrent versus afferent dynamics in cortical visual processing. Nature Neuroscience, 18(12), 1789–1797.CrossRefPubMedGoogle Scholar
- Reyes, A.D. (2011). Synaptic short-term plasticity in auditory cortical circuits. Hearing research, 279(1), 60–66.CrossRefPubMedPubMedCentralGoogle Scholar
- Rubin, D.B., Van Hooser, S.D., & Miller, K.D. (2015). The stabilized supralinear network: a unifying circuit motif underlying multi-input integration in sensory cortex. Neuron, 85(2), 402–417.CrossRefPubMedPubMedCentralGoogle Scholar
- Russell, A.F., Mihalas, S., von der Heydt, R., Niebur, E., & Etienne-Cummings, R. (2014). A model of proto-object based saliency. Vision Research, 94, 1–15.CrossRefPubMedGoogle Scholar
- Shuler, M.G., & Bear, M.F. (2006). Reward timing in the primary visual cortex. Science, 311(5767), 1606–1609.CrossRefPubMedGoogle Scholar
- Sugihara, T., Qiu, F.T., & von der Heydt, R. (2011). The speed of context integration in the visual cortex. Journal of neurophysiology, 106(1), 374–385. PMC3129740.CrossRefPubMedPubMedCentralGoogle Scholar
- Super, H., Spekreijse, H., & Lamme, V. (2001). A neural correlate of working memory in the monkey primary visual cortex. Science, 293, 120–124.CrossRefPubMedGoogle Scholar
- Tsodyks, M., Pawelzik, K., & Markram, H. (1998). Neural networks with dynamic synapses. Neural Computation, 10, 821– 835.CrossRefPubMedGoogle Scholar
- Tsodyks, M.V., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proceedings of the National Academy of Sciences, 94, 719–23.CrossRefGoogle Scholar
- Tsumoto, T., & Yasuda, H. (1996). A switching role of postsynaptic calcium in the induction of long-term potentiation or long-term depression in visual cortex. In Seminars in Neuroscience, (Vol. 8 pp. 311–319): Elsevier.Google Scholar
- Wang, X.J. (1999). Synaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory. The Journal of Neuroscience, 19(21), 9587–9603. PMID10531461.PubMedGoogle Scholar
- Xue, M., Atallah, B.V., & Scanziani, M. (2014). Equalizing excitation-inhibition ratios across visual cortical neurons. Nature, 511(7511), 596.CrossRefPubMedPubMedCentralGoogle Scholar