Phenomenological models of synaptic plasticity based on spike timing

Many other experimental features could be added to this list, e.g., the role of intracellular calcium, of NMDA receptors, etc., but we will not do so; see Bliss and Collingridge (1993) and Malenka and Nicoll (1993) for reviews. We emphasize that, given the heterogeneity of synapses between different brain areas (plasticity has mainly been studied in visual or somatosensory cortex and hippocampus) and between different neuron and synapse types, we cannot expect that a single theoretical model can account for all experimental facts. In the next section, we will instead consider which theoretical principles could guide our search for suitable plasticity rules.

These items are not necessarily exclusive, and the relative importance of a given aspect may vary from one subsystem to the next; for example, synaptic memory maintenance might be more important for a long-term memory system than for primary sensory cortices. There is so far no rule which exhibits all of the above properties; moreover, theoretical models which reproduce some aspects of experimental findings are generally incompatible with other findings. For example, traditional learning rules that have been proposed as an explanation of receptive field development (Bienenstock et al. 1982; Miller et al. 1989), exhibit a spontaneous separation of synaptic weights into two groups, even if the input shows no or only weak correlations. This is difficult to reconcile with experimental results in visual cortex of young rats where a unimodal distribution was found (Sjostrom et al. 2001). Moreover model neurons that specialize early in development on one subset of features cannot readily re-adapt later on. On the other hand, learning rules that do produce a unimodal distribution of synaptic weights (van Rossum et al. 2000; Rubin et al. 2001; Gütig et al. 2003; Morrison et al. 2007) do not lead to long-term stability of synaptic changes, as the trajectories of individual synaptic weights perform random walks. Hence it appears that long-term stability of memory requires a multimodal synapse distribution (Toyoizumi et al. 2007; Billings and van Rossum 2008) or additional mechanisms to stabilize the synaptic weights contributing to the retention of a memory item.

For A < 1 the amount of increase gets smaller as the variable x before the update (denoted by x_) approaches its maximal value of 1. Hence the variable x stays bounded in the range 0 ≤ x ≤ 1. An extreme case of saturation is given by A = 1, in which case the reset is always to the value of 1, regardless of the value of x just before. In this case, the value of the trace x depends only on the time since the most recent spike (see Fig. 1, bottom). We will see in the following sections that the idea of traces left by pre- or postsynaptic spikes plays a fundamental role in algorithmic formulations of short-term and long-term plasticity. For example, in the case of Hebbian long-term potentiation, traces left by presynaptic spikes need to be combined with postsynaptic spikes, whereas short-term plasticity can be seen as induced by traces of presynaptic spikes, independent of the state of the postsynaptic neuron.

Open image in new window

In principle, voltage dependence could be treated in a similar fashion, see, e.g., Brader et al. (2007), but we will focus in the following on learning rules for short-term and long-term plasticity that use spike timing as the relevant variable for inducing postsynaptic changes.

One well-established phenomenological model for fast synaptic dynamics was originally formulated for depression only in Tsodyks and Markram (1997) and later extended to facilitating dynamics in Markram et al. (1998). Here, we discuss the formulation of the model presented in Tsodyks et al. (2000).

If neuron i receives a synapse from neuron j (see Fig. 2b), the synaptic current (or conductance) in neuron i isw_ijy_ij(t), wherew_ij is the absolute strength and y_ij(t) is a scaling factor that describes themomentary input to neuron i. Dropping the indices for the rest of this discussion, y evolves according to:

$${{{\rm{d}}x} \over {{\rm{d}}t}} = {z \over {{\tau _{{\rm{rec}}}}}} - {u_ + }{x_ - }\delta (t - t_j^f)$$

(3)

$${{{\rm{d}}x} \over {{\rm{d}}t}} = {y \over {{\tau _{\rm{I}}}}} - {u_ + }{x_ - }\delta (t - t_j^f)$$

(4)

$${{{\rm{d}}z} \over {{\rm{d}}t}} = {y \over {{\tau _{\rm{I}}}}} - {z \over {{\tau _{{\rm{rec}}}}}}$$

(5)

where x,y and z are the fractions of synaptic resources in the recovered, active, and inactive states respectively, t_j^f gives the timing of presynaptic spikes, τ_I is the decay constant of PSCs and τ_rec is the recovery time from synaptic depression. These equations describe the use of synaptic resources by each presynaptic spike—a fraction u₊ of the available resources x is used by each presynaptic spike. The variable u₊ therefore describes the effective use of the synaptic resources of the synapses, which is analogous to the probability of release in the model described in (Markram et al. 1998). The notation x₋ in the update equations (3) and (4) is intended to remind the reader that the value of x just before the update is used. In facilitating synapses, u₊ is not a fixed parameter, but derived from a variable u which is increased with each presynaptic spike and returns to baseline with a time constant τ_fac:

$${{{\rm{d}}u} \over {{\rm{d}}t}} = {u \over {{\tau _{{\rm{fac}}}}}} + U(1 - u\_)\delta (t - t_j^f)$$

(6)

where the parameter U determines the increase in u with each spike. We note that the update is equivalent to the saturated trace (2). The notation u₋ indicates that the value of u is taken just before the update caused by presynaptic spike arrival. However, in (3) and (4) we use the value u₊ just after the update of the variable u. If τ_fac → 0, facilitation is not exhibited, and u₊ is identical to U after each spike, as is the case with depressing synapses between excitatory pyramidal neurons (Tsodyks and Markram 1997). The model described by Eqs. 3–6 gives a very good fit to experimental results: compare Fig. 2a and c. However, it should be noted that the values for the model parameters, including the time constants, are quite heterogeneous, even within a single neural circuit (Markram et al. 1998). The biophysical causes of the heterogeneity are still largely unclear.

Open image in new window

Note that the updated value of u is used to update the variables x and y. This reflects the assumption that the effectivity of resource use is determined not just by the history of the synapse but also by the arrival of the new presynaptic spike, thus ensuring a non-zero response to the first spike (Tsodyks et al. 1998).

In many simulation systems synapse models are constrained to transmit a synaptic weight rather than a continuous synaptic current. In such cases, the synaptic weight transmitted to the postsynaptic neuron is w_ijy₀, assuming the postsynaptic neuron reproduces the dynamics of the y variable. It is not necessary for the neuron to reproduce the y dynamics for each individual synapse; due to the linearity of y between increments, all synapses with the same τ_I can be lumped together. This is the implementation used in NEST (Gewaltig and Diesmann 2007). If the postsynaptic neuron also implements an exact integration scheme (for a worked example see Morrison et al. 2007), the dynamics of y can be incorporated into the propagator of the dynamics of the postsynaptic neuron.

A pair-based update rule can be easily implemented with two local variables: one for a low-pass filtered version of the presynaptic spike train and one for the postsynaptic spike train. The concept is illustrated in Fig. 3. Let us consider the synapse between neuron j and neuron i. Suppose that each spike from presynaptic neuron j contributes to a trace x_j at the synapse:

$${{{\rm{d}}{x_j}} \over {{\rm{d}}t}} = - {{{x_j}} \over {{\tau _x}}} + \mathop \Sigma \limits_{t_j^f} \delta \left( {t - t_j^f} \right)$$

(11)

, where t_m^f denotes the firing times of the presynaptic neuron. In other words, the variable is increased by an amount of one at themoment of a presynaptic spike and decreases exponentially with time constant τ_x afterwards; see the discussion of traces in Sect. 2.3. Similarly, each spike from postsynaptic neuron i contributes to a trace y_i:

$${{{\rm{d}}{y_i}} \over {{\rm{d}}t}} = - {{{y_i}} \over {{\tau _y}}} + \mathop \Sigma \limits_{t_j^f} \delta \left( {t - t_i^f} \right)$$

(12)

, where t_i^f denotes the firing times of the postsynaptic neuron. On the occurrence of a presynaptic spike, a decrease of the weight is induced proportional to the momentary value of the postsynaptic trace y_i. Likewise, on the occurrence of a postsynaptic spike a potentiation of the weight is induced proportional to the trace x_j left by previous presynaptic spikes:

$$\Delta w _{ij}^ + \left( {t_i^f} \right) = {F_ + }({w _{ij}}){x_j}(t_i^f)$$

(13)

$$\Delta w _{ij}^ - \left( {t_j^f} \right) = - {F_ - }({w _{ij}}){y_j}(t_j^f)$$

(14)

, or alternatively:

$${{{\rm{d}}{w _{ij}}} \over {{\rm{d}}t}} = - {F_ - }({w _{ij}}){y_i}(t)\delta (t - t_j^f) + {F_ + }({w _{ij}}){x_j}(t)\delta (t - t_i^f)$$

(15)

. A pseudo-code algorithm along these lines for simulating arbitrary pair-based STDP update rules that is suitable for distributed computing is given in Morrison et al. (2007).

Open image in new window

Depending on the definition of the trace dynamics (accumulating or saturating, see Sect. 2.3), different spike pairing schemes can be realized. Before we turn to the consequences of these subtle differences (Sect. 4.1.2) and the implementation of synaptic delays (Sect. 4.1.3), we now discuss the choice of the factors F₊ (w) and F₋ (w), i.e. the weight dependence of STDP.

4.1.1 4.1.1 Weight dependence of STDP

The clearest experimental evidence for the weight dependence of STDP can be found in Bi and Poo (1998), see Fig. 4a. Unfortunately, it is difficult to interpret this figure accurately, as the unit of the ordinate is percentage change, and thus not independent of the value on the abscissa. An additional confounding factor is that the timing interval used in the spike pairing protocol varies considerably across the data. However, even given these drawbacks, the rather flat dependence of the percentage weight change for depression (Δw/w ≈ constant) suggests a multiplicative dependence of depression on the initial synaptic strength (Δw ∞ w). For potentiation the picture is less clear.

Open image in new window

Instead of plotting the percentage weight change, Fig. 4b shows the absolute weight change in double logarithmic representation. The exponent of the weight dependence can now be determined from the slope of a linear fit to the data, see Morrison et al. (2007) for more details. A multiplicative update rule (F₋(w) α w) is the best fit to the depression data but a poor fit to the potentiation data. The best fit to the potentiation data is a power law update (F₊(w) α w^μ). The quality of an additive update (F₊(w) = A₊) fit is between the power law fit and the multiplicative fit.

4.1.1.1 4.1.1.1 Unimodal versus bimodal distributions

The choice of update rule can have a large influence on the equilibrium weight distribution in the case of uncorrelated Poissonian inputs. This was first demonstrated by Rubin et al. (2001), see Fig. 5. Here, the behavior of an additive STDP rule (F₊(w)=λ, F₋(w)=λα, where λ ≪ 1 is the learning rate and α an asymmetry parameter) is compared with the behavior of a multiplicative STDP rule (\({F_ + }(w ) = \lambda (1 - w )\), F₋(w)=λαw, with w in the range [0, 1). In the lowest histograms, the equilibrium distributions are shown for a neuron receiving 1,000 uncorrelated Poissonian spike trains at 10 Hz. In the case of additive STDP, a bimodal distribution develops, whereas in the case of multiplicative STDP, the equilibrium distribution is unimodal. Experimental evidence currently suggests that a unimodal distribution of synaptic strengths is more realistic than the extreme bimodal distribution depicted in Fig. 5a, see, for example, Turrigiano et al. (1998) and Song et al. (2005). Gütig et al. (2003) extended this analysis by regarding additive and multiplicative STDP as the two extrema of a continuous spectrum of rules: F₋(w)=λαw^μ, \({F_ + }(w ) = \lambda {(1 - w )^\mu }\). A choice of μ = 0 results in additive STDP, a choice of μ = 1 leads to multiplicative STDP, and intermediate values result in rules which have an intermediate dependence on the synaptic strength.

Open image in new window

Gütig et al. (2003) further demonstrated that the unimodal distribution is the rule rather than the exception for update rules of this form. A bimodal distribution is only produced by rules with a very weak weight dependence (i.e. μ ≪ 1). Moreover, the critical value for μ at which bimodal distributions appear decreases as the the effective population size Nrτ increases, where N is the number of synapses converging onto the postsynaptic neuron, r is the rate of the input spike trains in Hz and τ is the time constant of the STDP window (assumed to be equal for potentiation and depression). Figure 5c shows the equilibrium distributions as a function of μ for N = 1,000, r = 10Hz and τ = 0.02 s. μ_crit is already very low for this effective population size. Because of the high connectivity of the cortex, we may expect that the effective population size in vivo would be an order of magnitude greater, and so the region of bimodal stability would be vanishingly small according to this analysis. It is worth noting that in the case that a sub-group of inputs is correlated, a bimodal distribution develops for all values of μ, whereby the synaptic weights of the correlated group become stronger than those of the uncorrelated group (data not shown—see Gütig et al. 2003). In contrast to a purely additive rule, the peaks of the distributions are not at the extrema of the permitted weight range. Moreover, the bimodal distribution does not persist if the correlations in the input are removed after learning. A unimodal distribution for uncorrelated Poissonian inputs and an ability to develop multimodal distributions in the presence of correlation is also exhibited by the additive/multiplicative update rule proposed by van Rossum et al. (2000): F₊(w)=λ, F₋(w)=λθw; and by the power law update rule proposed by Morrison et al. (2007) and also Standage et al. (2007): F₊(w) α λw^μ, F₋(w) α λαw.

4.1.1.2 4.1.1.2 Fixed point analysis of STDP update rules

An insight into the similarity of behavior of all of these formulations of STDP with the exception of the additive update rule can be obtained by considering their fixed point structure. Equation (10) gives the updates of an individual synaptic weight. If the pre- and postsynaptic spike trains are stochastic, the weight updates can be described as a random walk. Using Fokker-Planck mean field theory, the average rate of change of synaptic strength corresponds to the drift of the random walk, which can be expressed in terms of the correlation between the pre- and postsynaptic spike trains (Kempter et al. 1999; Kistler and van Hemmen 2000; Kempter et al. 2001; Rubin et al. 2001; Gütig et al. 2003). Writing the presynaptic spike train as \({\rho _j} = {\Sigma _{t_j^f}}\delta \left( {t - t_j^f} \right)\) and the postsynaptic spike train as \({\rho _i} = {\Sigma _{t_i^f}}\delta \left( {t - t_i^f} \right)\), the mean rates are ν_i/j=〈ρ_i/j〉. Assuming stationarity, the raw cross-correlation function is given by

$${\Gamma _{ji}}(\Delta t) = {\left\langle {{\rho _j}(t){\rho _i}(t + \Delta t)} \right\rangle _t}$$

(16)

i.e. averaging over t while keeping the delay Δt between the two spike trains fixed. The synaptic drift is obtained by integrating the synaptic weight changes given by (10) over Δt weighted by the probability, as expressed by (16), of the temporal difference Δt occurring between a pre- and post-synaptic spike:

$$\dot w = - {F_ - }(w)\int\limits_{ - \infty }^0 {{\rm{d}}\Delta t{K_ - }(\Delta t){\Gamma _{ji}}(\Delta t) + {F_ + }(w)} \int\limits_0^\infty {{\rm{d}}\Delta t{K_ + }(\Delta t){\Gamma _{ji}}(\Delta t)} $$

(17)

, where \({K_ \pm }(\Delta t) = \exp ( - \left| {\Delta t} \right|/{\tau _ \pm })\), the window function of STDP.

As we are only interested in the qualitative structure of fixed points rather than their exact location, we will simplify the analysis by assuming that the pre- and postsynaptic spike trains are independent Poisson processes with the same rate, i.e. <ρ_i>=<ρ_j>=ν and Γ_ji(Δt) = ν². We can therefore write:

$$\dot w = - {F_ - }(w ){\tau _ - }{\nu ^2} + {F_ + }(w ){\tau _ + }{\nu ^2}$$

.

In general, the rate ν of a neuron is dependent on the weight of its incoming synapses and so the right side of this equation cannot be easily determined. However, we can reformulate the equation as:

$${{\dot w } \over {{\nu ^2}}} = - {F_ - }(w){\tau _ - } + {F_ + }(w){\tau _ + }$$

(18)

.

The fixed points of the synaptic dynamics are given by definition by .w = 0, and therefore also by ˙w/ν² = 0. Figure 6 plots (18) for a range of w and a variety of STDP models. In all cases except for additive STDP the curves pass through ˙w/ν² = 0 at an intermediate value of w and with a negative slope, i.e. for weights below the fixed point there is a net potentiating effect, and for weights above the fixed point there is a net depressing effect, resulting in a stable fixed point which is not at an extremum of the weight range. In the case of additive STDP there is no such fixed point, stable or otherwise.

Open image in new window

The behavior of the additive model can be assessed more accurately by relaxing the assumption that pre- and postsynaptic spike trains can be described by independent Poisson processes. Instead, we consider a very simple neuron model in which the output spike train is generated by an inhomogeneous Poisson process with rate \({\nu _i}({u_i}) = {[\alpha {u_i} - {\nu _0}]_ + }\) with scaling factor α, threshold ν₀ and membrane potential \({u_i}(t) = {\Sigma _j}{w _{ij}}\varepsilon (t - t_j^f)\), where ∈(t) denotes the time course of an excitatory postsynaptic potential generated by a presynaptic spike arrival. The notation [x]₊ denotes a piecewise linear function: [x]₊ = x for x > 0 and zero otherwise. In the following we assume that the argument of our piecewise linear function is positive so that we can suppress the square brackets. Assuming once again that all input spike trains are Poisson processes with rate ν, the expected firing rate of the postsynaptic neuron is simply:

$${\nu _i} = - {\nu _0} + \alpha \nu \bar \varepsilon \matrix{ \Sigma \cr j \cr } {w _{ij}}$$

, where ¯ε=ε∈(s)ds, the total area under an excitatory postsynaptic potential. The conditional rate of firing given an input spike at time t_j^f is given by

$${\nu _i}(t) = - {\nu _0} + \alpha \nu \bar \varepsilon \matrix{ \Sigma \cr j \cr } {w _{ij}} + \alpha {w _{ij}}\varepsilon (t - t_j^f)$$

, thus the postsynaptic spike train is correlated with the presynaptic spike trains. This term shows up as additional spike-spike correlations in the correlation function Γ_ji. Hence, in addition to the terms in (18), the synaptic dynamics contains a term of the form αvw F₊(w) εK₊(s)ƒ(s)ds that is linear rather than quadratic in the presynaptic firing rate (Kempter et al. 1999, Kempter et al. 2001). With this additional term, (18) becomes

$${{\dot w } \over \nu } = {\nu _i}[ - {F_ - }(w){\tau _ - } + {F_ + }(w){\tau _ + }] + \alpha w {F_ + }(w)\smallint {K_ + }(s)\varepsilon (s){\rm{d}}s$$

(19)

. For the multiplicative models the argument hardly changes, but for the additive model it does. For \(\eqalign{ & s \cr & C = {F_ - }(w ){\tau _ - } - {F_ + }(w ){\tau _ + } > 0 \cr} \), the additive model has a fixed point which we find by setting the right-hand side of (19) to zero, i.e.

$$0 = - C{\nu _i} + \alpha w{C_{{\rm{ss}}}}$$

, where C_ss = F₊(w)εK₊(s)∈(s)ds denotes the contribution of the spike-spike correlations. In contrast to the curves in Fig. 6, the slope at the zero-crossing is now positive, indicating instability of the fixed point. This instability leads to the formation of a bimodal weight distribution that is typical for the additive model. Despite the instability of individual weights (which move to their upper or lower bounds), the mean firing rate of the neuron is stabilized (Kempter et al. 2001). To see this we consider the evolution of the output rate dν_i/dt = αν¯εΣ_jdw_ij/dt. Since \(d{w _{ij}}/{\rm{d}}t = - C{\nu _i}\nu + \alpha \nu {w_{ij}}{C_{{\rm{ss}}}}\) and \(\alpha \nu \bar \varepsilon {\Sigma _j}{w _{ij}} = {\nu _i} + {\nu _0}\), we can write:

$${{d{\nu _i}} \over {{\rm{d}}t}} = - \alpha {\nu ^2}\bar \varepsilon NC{\nu _i} + \alpha \nu ({\nu _i} + {\nu _0}){C_{{\rm{ss}}}}$$

where N is the number of synapses converging on the post-synaptic neuron. Thus we have a dynamics of the form:

$${{d{\nu _i}} \over {{\rm{d}}t}} = {{({\nu _i} - {\nu _{{\rm{FP}}}})} \over {{\tau _\nu }}}$$

with a fixed point given by:

$${\nu _{{\rm{FP}}}} = {{{C_{{\rm{ss}}}}{\nu _0}} \over {NC\nu \bar \varepsilon - {C_{{\rm{ss}}}}}}$$

(20)

and a time constant \({\tau _\nu } = {(\alpha \nu [NC\nu \bar \varepsilon - {C_{{\rm{ss}}}}])^{ - 1}}\). Note that stabilization at a positive rate requires that ν₀ > 0 andC > 0. The first condition states that, in the absence of any input, the neuron does not show any spontaneous activity, and this is trivially true for all standard neuron models, including the integrate-and-fire model. The latter condition is equivalent to the requirement that the integral over the STDP curve be negative: \(\smallint {\rm{d}}s[{F_ + }(w){K_ + }(s) - {F_ - }(w){K_ - }(s)] = - C < 0\). Exact conditions for stabilization of output rates are given in Kempter et al. (2001). Since for constant input rates ν we have \({\nu _i} = {\nu _0} + \alpha \nu \bar \varepsilon {\Sigma _j}{w_{ij}}\), stabilization of the output rate implies normalization of the summed weights. Hence STDP can lead to a control of total presynaptic input and of the postsynaptic firing rate — a feature that is usually associated with homeostatic processes rather than Hebbian learning per se (Kempter et al. 1999, Kempter et al. 2001; Song et al. 2000).

Note that the existence of a fixed point and its stability does not crucially depend on the presence of soft or hard bounds on the weight. Equations (18) and (19) can equate to zero for hard-bounded or or unbounded rules.

4.1.1.3 4.1.1.3 Consequences for network stability

Results on the consequences of STDP in large-scale networks are few and far between, and tend to contradict each other. Part of the reason for the lack of simulation papers on this important subject is the fact that simulating such networks consumes huge amounts of memory, is computationally expensive, and potentially requires extremely long simulation times to overcome transients in the weight dynamics which can be of the order of hundreds of seconds of biological time. A lack of theoretical papers on the subject can be explained by the complexity of the interactions between the activity dynamics of the network and the weight dynamics, although some progress is being made in this area (Burkitt et al. 2007).

It was recently shown that power law STDP is compatible with balanced random networks in the asynchronousirregular regime (Morrison et al. 2007), resulting in a unimodal distribution of weights and no self-organization of structure. This result was verified for Gütig et al. (2003) STDP for an intermediate value of the exponent (μ = 0.4). Although it has not yet been possible to perform systematic tests, it seems likely that all the formulations of STDP with the fixed point structure discussed in Sect. 4.1.1.1 would give qualitatively similar behavior. The results for additive STDP seem to be more contradictory. Izhikevich et al. (2004) reported self-organization of neuronal groups, whereas the chief feature of the networks investigated by Iglesias et al. (2005) seems to be extensive withering of the synaptic connections. In the former case, it is the existence of many strong synapses which defines the network, in the latter, the presence of many weak ones. This discrepancy may be attributable to different choices for the effective stabilized firing rates (20) in combination with different choices of delays in the network, see Sect. 4.1.3.

4.1.2 4.1.2 Spike pairing scheme

There are many possible ways to pair pre- and postsynaptic spikes to generate a weight update in an STDP model. In an all-to-all scheme, each presynaptic spike is paired with all previous postsynaptic spikes to effect depression, and each postsynaptic spike is paired with all previous presynaptic spikes to effect potentiation. This is the interpretation used for the fixed point analysis in Sect. 4.1.1.1 and can be implemented using local variables as demonstrated in Sect. 4.1. In a nearest neighbor scheme, only the closest interactions are considered. However, there are multiple possible interpretations of nearest neighbor, as can be seen in Fig. 7. Nearest neighbor schemes can also be realized in terms of appropriately chosen local variables. The symmetric nearest-neighbor scheme shown in Fig. 7a can be implemented by pre- and postsynaptic traces that reset to 1, rather than incrementing by 1 as is the case for the all-to-all scheme. In the case of the presynaptic centered interpretation depicted in Fig. 7B, the postsynaptic trace resets to 1 as in the previous example, but the presynaptic trace must be implemented with a slightly more complicated dynamics:

$${{{{\rm{d}}_{xj}}} \over {{\rm{d}}t}} = - {{{x_j}} \over {{\tau _x}}} + \sum\limits_{t_j^f} {(1 - x_j^ - )\delta (t - t_j^f) - } \sum\limits_{t_j^f} {x_j^ - \delta (t - t_i^f)} $$

, where t_j^f and t_i^f denote the firing times of the pre- and postsynaptic neurons respectively, and x_j⁻ gives the value of x_j just before the update. In other words, the trace is reset to 1 on the occurrence of a presynaptic spike and reset to 0 on the occurrence of a postsynaptic spike. Similarly, the reduced symmetric interpretation shown in Fig. 7c can be implemented by pre- and postsynaptic ‘doubly resetting’ traces of this form.

Open image in new window

It is sometimes assumed that the scheme used makes no difference, as the ISI of cortical network models is typically an order of magnitude larger than the time constant of the STDP window. However, this is not generally true (Kempter et al. 2001; Izhikevich and Desai 2003; Morrison et al. 2007). For a review of a wide variety of schemes and their consequences, particularly with respect to selectivity of higher-frequency inputs, see Burkitt et al. (2004). Experimental results on this issue suggest limited interaction between pairs of spikes. Sjostrom et al. (2001) found that their data was best fit by a nearest neighbor interaction similar to Fig. 7c but giving precedence to LTP, i.e. a postsynaptic spike can only contribute to a post-before-pre pairing if it has not already contributed to a pre-before-post pairing. However, this result may also be due to the limitations of pair-based STDP models to explain the experimentally observed frequency dependence, see Sect. 4.2. More recently, Froemke et al. (2006) demonstrated that the amount of LTD was not dependent on the number of presynaptic spikes following a postsynaptic spike, suggesting nearest-neighbor interactions for depression as in Fig. 7c. However, the amount of LTP was negatively correlated with the number of presynaptic spikes preceding a postsynaptic spike. This suggests that multiple spike pairings contribute to LTP, but not in the linear fashion of the all-to-all scheme, which would predict a positive correlation between the number of spikes and the amount of LTP. Again, these results are good evidence for the limitations of pair-based STDP rules.

4.1.3 4.1.3 Synaptic delays

Up until now we have referred to Δt as the temporal difference between a pre- and a postsynaptic spike, i.e. \(\Delta t = t_i^f - t_j^f\). However, many classic STDP experiments are expressed in terms of the temporal difference between the start of the EPSP and the postsynaptic spike (Markram et al. 1997; Bi and Poo 1998). In fact, when a presynaptic spike is generated at t_j^f, it must first travel down the axon before arriving at the synapse, thus arriving at \(t_j^s = t_j^f + {d_{\rm{A}}}\), where d_A is the axonal propagation delay. Similarly, a postsynaptic spike at t_i^f must backpropagate through the dendrite before arriving at the synapse at \(t_i^s = t_i^f + {d_{{\rm{BP}}}}\), where d_BP is the backpropagation delay. Consequently, the relevant temporal difference for STDP update rules is \(\Delta {t^s} = t_i^s - t_j^s\) as initially suggested by Gerstner et al. (1993) and Debanne et al. (1998). Senn et al. (2002) showed that under fairly general conditions, STDP may cause adaptation in the presynaptic and postsynaptic delays in order to optimize the effect of the presynaptic spike on the postsynaptic neuron. In order to calculate the synaptic drift as in (17), we therefore need to integrate the synaptic weight changes over Δt^s, weighted by the raw cross-correlation function at the synapse. With \({\Gamma _{ji}}(\Delta t) = {\Gamma _{ji}}(\Delta {t^s} + ({d_{\rm{A}}} - {d_{{\rm{BP}}}}))\), we reformulate (17) as:

$$\dot w = - {F_ - }(w)\int\limits_{ - \infty }^0 {{\rm{d}}\Delta {t^s}{K_ - }(\Delta {t^s}){\Gamma _{ji}}(\Delta {t^s} + ({d_{\rm{A}}} - {d_{{\rm{BP}}}})) + {F_ + }(\omega )} \int\limits_{ - \infty }^0 {(\Delta {t^s}){\Gamma _{ji}}(\Delta {t^s} + ({d_{\rm{A}}}} - {d_{{\rm{BP}}}}))$$

(21)

.

In the case of independent Poisson processes as in Sect. 4.1.1.1, the shift of the raw cross-correlation function by (d_A − d_BP) has no effect, as Γ_ji(Δt)} is constant. Generally, however, this is not the case. For example, networks of neurons, both in experiment and simulation, typically exhibit oscillations with a period several times larger than the synaptic delay, even when individual spike trains are irregular (see Kriener et al. 2008, for discussion). If the axonal delay is the same as the backpropagation delay, i.e. d_A = d_BP = d/2, where d is the total transmission delay of the spike, the raw cross-correlation function at the synapse is the same as the raw cross-correlation at the soma:

$$\Gamma _{ji}^s(\Delta {t^s}) = {\Gamma _{ji}}(\Delta {t^s} + ({d_{\rm{A}}} - {d_{{\rm{BP}}}})) = {\Gamma _{ji}}(\Delta t)$$

. This situation is depicted in Fig. 8b. Let w₀ be the synaptic weight for which the synaptic drift given in (21) is 0, i.e. the fixed point of the synaptic dynamics for the cross-correlation shown. If the axonal delay is larger than the backpropagation delay, this results in a shift of the raw cross-correlation function to the left. This is shown in Fig. 8a for the extreme case of d_A = d, d_BP = 0, resulting in a net shift of d. This increases the value of the first integral in (21) and decreases the second integral, such that˙w< 0 atw₀. Conversely, if the axonal delay is smaller than the backpropagation delay, the raw cross-correlation function is shifted to the right (Fig. 8c, for the extreme case of d_A = 0, d_BP = d). This decreases the value of the first integral in (21) and increases the second integral, such that ˙w > 0 at w₀. Therefore, a given network dynamics may cause systematic depression, systematic potentiation or no systematic change at all to the synaptic weights, depending on the partition of the synaptic delay into axonal and dendritic contributions. Systematic synaptic weight changes can in turn result in qualitatively different network behavior. For example, in Morrison et al. (2007) small systematic biases in the synaptic weight dynamics were applied to a network with an equilibrium characterized by a unimodal weight distribution and medium rate (< 10 Hz) asynchronous irregular activity dynamics. Here, a small systematic depression led to a lower weight, lower rate equilibrium also in the asynchronous irregular regime, whereas a systematic potentiation led to a sudden transition out of the asynchronous irregular regime: the activity was characterized by strongly patterned high-rate peaks of activity interspersed with silence, and the unimodal weight distribution splintered into several peaks.

Open image in new window

There is considerable evidence that the pair-based rules discussed above cannot give a full account of STDP. Specifically, they reproduce neither the dependence of plasticity on the repetition frequency of pairs of spikes in an experimental protocol, nor the results of recent triplet and quadruplet experiments.

STDP experiments are usually carried out with about 60 pairs of spikes. The temporal distance of the spikes in the pair is of the order of a few to tens of milliseconds, whereas the temporal distance between the pairs is of the order of hundreds of milliseconds to seconds. In the case of a facilitation protocol (i.e. pre-before-post), standard pair-based STDP models predict that if the repetition frequency is increased, the strength of the depressing interaction (i.e. post-before-pre) becomes greater, leading to less net potentiation. This prediction is independent of whether the spike pairing scheme is all-to-all or nearest neighbor (see Sect. 4.1.2). However, experiments show that increasing the repetition frequency leads to an increase in potentiation (Sjostrom et al. 2001). Other recent experiments employed multiple-spike protocols, such as repeated presentations of symmetric triplets of the form pre-post-pre and post-pre-post (Bi and Wang 2002; Froemke and Dan 2002; Wang et al. 2005; Froemke et al. 2006). Standard pair-based models predict that the two sequences should give essentially the same results, as they each contain one pre-post pair and one post-pre pair. Experimentally, quite different results are observed.

Here we review two examples of simple models which account for these experimental findings. For other models which also reproduce frequency dependence or multiple-spike protocol results, see Abarbanel et al. (2002), Senn (2002) and Appleby and Elliott (2005).

4.2.1 4.2.1 Triplet model

One simple approach to modeling STDP which addresses these issues is the triplet rule developed by Pfister and Gerstner (2006). This model is based on sets of three spikes (one presynaptic and two postsynaptic). As in the case of pair-based rules, the triplet rule can be easily implemented with local variables as follows. Similarly to pair-based rules, each spike from presynaptic neuron j contributes to a trace x_j at the synapse:

$${{{{\rm{d}}_{xj}}} \over {{\rm{d}}t}} = - {{{x_j}} \over {{\tau _x}}} + \sum\limits_{t_j^f} {\delta (t - t_j^f)} $$

, where t_j^f denotes the firing times of the presynaptic neuron. Unlike pair-based rules, each spike from postsynaptic neuron i contributes to a fast trace y_i¹ and a slow trace y_i² at the synapse:

$${{{\rm{d}}y_i^1} \over {{\rm{d}}t}} = - {{y_i^1} \over {{\tau _1}}} + \delta (t - t_i^f)$$

$${{{\rm{d}}y_i^2} \over {{\rm{d}}t}} = - {{y_i^2} \over {{\tau _2}}} + \delta (t - t_i^f)$$

, where τ₁ < τ₂, see Fig. 9. LTD is induced as in the standard STDP pair model given in (13), i.e. the weight change is proportional to the value of the fast postsynaptic trace y{sri/1} evaluated at the moment of a presynaptic spike. The new feature of the rule is that LTP is induced by a triplet effect: the weight change is proportional to the value of the presynaptic trace x_j evaluated at the moment of a postsynaptic spike and also to the slow postsynaptic trace y_i² remaining from previous postsynaptic spikes:

$$\Delta w_{ij}^ + \left( {t_i^f} \right) = {F_ + }({w_{ij}}){x_j}(t_i^f)y_i^2(t_i^{f - })$$

where t_i^f− indicates that the function y_i² is to be evaluated before it is incremented due to the postsynaptic spike at t_i^f. Analogously to pair-based models, the triplet rule can also be implemented with nearest-neighbor rather than all-to-all spike pairings by an appropriate choice of trace dynamics, see Sect. 4.1.2.

Open image in new window

The triplet rule reproduces experimental data from visual cortical slices (Sjostrom et al. 2001) that increasing the repetition frequency in the STDP pairing protocol increases net potentiation (Fig. 10). It also gives a good fit to experiments based on triplet protocols in hippocampal culture (Wang et al. 2005). The main functional advantage of such a triplet learning rule is that it can be mapped to a Bienenstock-Cooper-Munro learning rule (Bienenstock et al. 1982): if we assume that the pre- and postsynaptic spike trains are governed by Poisson statistics, the triplet rule exhibits depression for low postsynaptic firing rates and potentiation for high postsynaptic firing rates. If we further assume that the triplet term in the learning rule depends on the mean postsynaptic frequency, a sliding threshold between potentiation and depression can be defined. In this way, the learning rule matches the requirements of the BCM theory and inherits the properties of the BCM learning rule such as input selectivity. From BCM properties, we can immediately conclude that the model should be useful for receptive field development. Note that earlier efforts to show that STDP maps to the BCM model (Izhikevich and Desai 2003; Senn et al. 2000) demonstrated neither an exact mapping nor a sliding threshold. The exact relationship between the above triplet model and other models is discussed in Pfister and Gerstner (2006).

Open image in new window

Traditional LTP/LTD experiments employ the following induction paradigm: the postsynaptic neuron is held at a fixed depolarization while one or several presynaptic neurons are activated. Often a presynaptic pathway is stimulated extracellularly, so that several presynaptic neurons are activated. Depending on the level of the postsynaptic membrane potential, the activated synapses increase their efficacy while other non-activated synapses do not change their weight (Artola et al. 1990; Artola and Singer 1993). More recently, depolarization has also been combined with STDP experiments. In particular, Sjostrom et al. (2004) showed a dependence of synaptic weight changes on the synaptic membrane potential just before a postsynaptic spike.

There is an ongoing discussion whether the voltage dependence is more fundamental than the dependence on postsynaptic spiking. Indeed, voltage dependence alone can generate STDP-like behavior (Brader et al. 2007), as the membrane potential behaves in a characteristic way in the vicinity of a spike (high shortly before a spike, and low shortly after). Alternatively, a dependence on the slope of the postsynaptic membrane potential has also been shown to reproduce the characteristic STDP weight change curve (Saudargiene et al. 2003). The voltage effects caused by back-propagating spikes is implicitly contained in the mechanistic formulation of STDP models outlined above. In particular, the fast postsynaptic trace y₁ in the above triplet model could be seen as an approximation of a back-propagating action potential. However, the converse is not true: a pure STDP rule does not automatically generate a voltage dependence. Moreover, synaptic effects caused by subthreshold depolarization in the absence of postsynaptic firing cannot be modeled by standard STDP or triplet models.

We stress that all the above models concern induction of potentiation and depression, but not their maintenance. The induction of LTP may take only a few seconds: for example, stimulation with 50 pairs of pre- and postsynaptic spikes given at 20Hz takes less than 3 s. However, afterwards the synapse takes 60 min or more to consolidate these changes, and this process may also be interrupted (Frey and Morris 1997). During this time synapses are ‘tagged’, that is, they are ready for consolidation. Consolidation is thought to rely on a different molecular mechanism than that of induction. Simply speaking, gene transcription is necessary to trigger the building of new proteins that increase the synaptic efficacy.

Supervised plasticity has been demonstrated experimentally by Fregnac and Schulz (2006): the behavior of a (cortical) neuron can be changed by pairing some class of stimuli with an (artificial) increase of neural activity while pairing another class of stimuli with a decrease of responsiveness. Theoretical studies have demonstrated that a teacher-forced STDP approach can be used to learn precise spike times (Legenstein et al. 2005; Pfister et al. 2006). In a natural situation, this would mean that a few strong neural inputs can drive the neuron and therefore drive learning of other inputs. If these strong inputs are controlled in a task-specific way, they act as a teacher for the postsynaptic neuron. For a practical realization of this idea see Brader et al. (2007).

If neuronal activity leads to actions, feedback may arise from the environment in forms of reward (a piece of pizza) or punishment (burnt fingers). It is thought that success of an action is signaled by neuromodulators—a top candidate is dopamine (Schultz et al. 1997). Dopamine signals are closely related to a quantity in reinforcement learning known as δ, that can be interpreted as the difference between the received reward and the expected reward. Here ‘reward’ means current or future rewards that can be reliably predicted. In reinforcement learning, the difference between actual and expected rewards plays an important role for the update of weights in Q-learning, SARSA, and related variants of temporal difference learning (Sutton and Barto 1998).

Under a suitable interpretation of the role of pre- and postsynaptic neurons, the weight update rules can be derived from an optimality framework (Pfister et al. 2006). The learning rule can be interpreted as a Hebbian learning based on joined pre- and postsynaptic activity, but conditioned on the presence of a global reward signal. Variants of such reinforcement rules for spiking neurons have been developed (Seung 2003; Pfister et al. 2006; Izhikevich 2007; Florian 2007).