Experimentally reported STDP curves vary qualitatively depending on the system and the neuron type—see Abbott and Nelson (2000) and Bi and Poo (2001) for reviews. It is therefore obvious that we cannot expect that a single STDP rule, be it defined in the framework of temporal traces outlined above or in a more biophysical framework, would hold for all experimental preparations and across all neuron and synapse types. The first spike-timing experiments were perform by Markram and Sakmann on layer 5 pyramidal neurons in neocortex (Markram et al. 1997). In the neocortex, the width of the negative window seems to vary depending on layer, and inhibitory neurons seem to have amore symmetric STDP curve. The standard STDP curve that has become an icon of theoretical research on STDP (Fig. 1 in Bi and Poo 1998) was originally found for pyramidal neurons in rat hippocampal cell culture. Inverted STDP curves have also been reported, for example in the ELL system in electric fish. This gives rise to different functional properties (Bell et al. 1997).
4.1 Pair-based STDP rules
Most models of STDP interpret the biological evidence in terms of a pair-based update rule, i.e. the change in weight of a synapse depends on the temporal difference between pairs of pre- and postsynaptic spikes:
$$\eqalign{ & \Delta {w ^ + } = {F_ + }(w ) \cdot \exp ( - \left| {\Delta t} \right|/{\tau _ + }){\rm{ if}}\Delta t > 0 \cr & \Delta {w ^ - } = - {F_ - }(w ) \cdot \exp ( - \left| {\Delta t} \right|/{\tau _ - }){\rm{ if}}\Delta t \le 0 \cr} $$
(10)
where \(\Delta t = t_j^f - t_j^f\) is the temporal difference between the post- and the presynaptic spikes, and F
±(w) describes the dependence of the update on the current weight of the synapse. A pair-based model is fully specified by defining: (i) the form of F
±(w); (ii) which pairs are taken into consideration to perform an update. In order to incorporate STDP into a neuronal network simulation, it is also necessary to specify how the synaptic delay is partitioned into axonal and dendritic contributions.
A pair-based update rule can be easily implemented with two local variables: one for a low-pass filtered version of the presynaptic spike train and one for the postsynaptic spike train. The concept is illustrated in Fig. 3. Let us consider the synapse between neuron j and neuron i. Suppose that each spike from presynaptic neuron j contributes to a trace x
j
at the synapse:
$${{{\rm{d}}{x_j}} \over {{\rm{d}}t}} = - {{{x_j}} \over {{\tau _x}}} + \mathop \Sigma \limits_{t_j^f} \delta \left( {t - t_j^f} \right)$$
(11)
, where t
f
m
denotes the firing times of the presynaptic neuron. In other words, the variable is increased by an amount of one at themoment of a presynaptic spike and decreases exponentially with time constant τ
x
afterwards; see the discussion of traces in Sect. 2.3. Similarly, each spike from postsynaptic neuron i contributes to a trace y
i
:
$${{{\rm{d}}{y_i}} \over {{\rm{d}}t}} = - {{{y_i}} \over {{\tau _y}}} + \mathop \Sigma \limits_{t_j^f} \delta \left( {t - t_i^f} \right)$$
(12)
, where t
f
i
denotes the firing times of the postsynaptic neuron. On the occurrence of a presynaptic spike, a decrease of the weight is induced proportional to the momentary value of the postsynaptic trace y
i
. Likewise, on the occurrence of a postsynaptic spike a potentiation of the weight is induced proportional to the trace x
j
left by previous presynaptic spikes:
$$\Delta w _{ij}^ + \left( {t_i^f} \right) = {F_ + }({w _{ij}}){x_j}(t_i^f)$$
(13)
$$\Delta w _{ij}^ - \left( {t_j^f} \right) = - {F_ - }({w _{ij}}){y_j}(t_j^f)$$
(14)
, or alternatively:
$${{{\rm{d}}{w _{ij}}} \over {{\rm{d}}t}} = - {F_ - }({w _{ij}}){y_i}(t)\delta (t - t_j^f) + {F_ + }({w _{ij}}){x_j}(t)\delta (t - t_i^f)$$
(15)
. A pseudo-code algorithm along these lines for simulating arbitrary pair-based STDP update rules that is suitable for distributed computing is given in Morrison et al. (2007).
Depending on the definition of the trace dynamics (accumulating or saturating, see Sect. 2.3), different spike pairing schemes can be realized. Before we turn to the consequences of these subtle differences (Sect. 4.1.2) and the implementation of synaptic delays (Sect. 4.1.3), we now discuss the choice of the factors F
+ (w) and F
− (w), i.e. the weight dependence of STDP.
4.1.1 Weight dependence of STDP
The clearest experimental evidence for the weight dependence of STDP can be found in Bi and Poo (1998), see Fig. 4a. Unfortunately, it is difficult to interpret this figure accurately, as the unit of the ordinate is percentage change, and thus not independent of the value on the abscissa. An additional confounding factor is that the timing interval used in the spike pairing protocol varies considerably across the data. However, even given these drawbacks, the rather flat dependence of the percentage weight change for depression (Δw/w ≈ constant) suggests a multiplicative dependence of depression on the initial synaptic strength (Δw ∞ w). For potentiation the picture is less clear.
Instead of plotting the percentage weight change, Fig. 4b shows the absolute weight change in double logarithmic representation. The exponent of the weight dependence can now be determined from the slope of a linear fit to the data, see Morrison et al. (2007) for more details. A multiplicative update rule (F
−(w) α w) is the best fit to the depression data but a poor fit to the potentiation data. The best fit to the potentiation data is a power law update (F
+(w) α w
μ). The quality of an additive update (F
+(w) = A
+) fit is between the power law fit and the multiplicative fit.
4.1.1.1 Unimodal versus bimodal distributions
The choice of update rule can have a large influence on the equilibrium weight distribution in the case of uncorrelated Poissonian inputs. This was first demonstrated by Rubin et al. (2001), see Fig. 5. Here, the behavior of an additive STDP rule (F
+(w)=λ, F
−(w)=λα, where λ ≪ 1 is the learning rate and α an asymmetry parameter) is compared with the behavior of a multiplicative STDP rule (\({F_ + }(w ) = \lambda (1 - w )\), F
−(w)=λαw, with w in the range [0, 1). In the lowest histograms, the equilibrium distributions are shown for a neuron receiving 1,000 uncorrelated Poissonian spike trains at 10 Hz. In the case of additive STDP, a bimodal distribution develops, whereas in the case of multiplicative STDP, the equilibrium distribution is unimodal. Experimental evidence currently suggests that a unimodal distribution of synaptic strengths is more realistic than the extreme bimodal distribution depicted in Fig. 5a, see, for example, Turrigiano et al. (1998) and Song et al. (2005). Gütig et al. (2003) extended this analysis by regarding additive and multiplicative STDP as the two extrema of a continuous spectrum of rules: F
−(w)=λαw
μ, \({F_ + }(w ) = \lambda {(1 - w )^\mu }\). A choice of μ = 0 results in additive STDP, a choice of μ = 1 leads to multiplicative STDP, and intermediate values result in rules which have an intermediate dependence on the synaptic strength.
Gütig et al. (2003) further demonstrated that the unimodal distribution is the rule rather than the exception for update rules of this form. A bimodal distribution is only produced by rules with a very weak weight dependence (i.e. μ ≪ 1). Moreover, the critical value for μ at which bimodal distributions appear decreases as the the effective population size Nrτ increases, where N is the number of synapses converging onto the postsynaptic neuron, r is the rate of the input spike trains in Hz and τ is the time constant of the STDP window (assumed to be equal for potentiation and depression). Figure 5c shows the equilibrium distributions as a function of μ for N = 1,000, r = 10Hz and τ = 0.02 s. μcrit is already very low for this effective population size. Because of the high connectivity of the cortex, we may expect that the effective population size in vivo would be an order of magnitude greater, and so the region of bimodal stability would be vanishingly small according to this analysis. It is worth noting that in the case that a sub-group of inputs is correlated, a bimodal distribution develops for all values of μ, whereby the synaptic weights of the correlated group become stronger than those of the uncorrelated group (data not shown—see Gütig et al. 2003). In contrast to a purely additive rule, the peaks of the distributions are not at the extrema of the permitted weight range. Moreover, the bimodal distribution does not persist if the correlations in the input are removed after learning. A unimodal distribution for uncorrelated Poissonian inputs and an ability to develop multimodal distributions in the presence of correlation is also exhibited by the additive/multiplicative update rule proposed by van Rossum et al. (2000): F+(w)=λ, F−(w)=λθw; and by the power law update rule proposed by Morrison et al. (2007) and also Standage et al. (2007): F+(w) α λwμ, F−(w) α λαw.
4.1.1.2 Fixed point analysis of STDP update rules
An insight into the similarity of behavior of all of these formulations of STDP with the exception of the additive update rule can be obtained by considering their fixed point structure. Equation (10) gives the updates of an individual synaptic weight. If the pre- and postsynaptic spike trains are stochastic, the weight updates can be described as a random walk. Using Fokker-Planck mean field theory, the average rate of change of synaptic strength corresponds to the drift of the random walk, which can be expressed in terms of the correlation between the pre- and postsynaptic spike trains (Kempter et al. 1999; Kistler and van Hemmen 2000; Kempter et al. 2001; Rubin et al. 2001; Gütig et al. 2003). Writing the presynaptic spike train as \({\rho _j} = {\Sigma _{t_j^f}}\delta \left( {t - t_j^f} \right)\) and the postsynaptic spike train as \({\rho _i} = {\Sigma _{t_i^f}}\delta \left( {t - t_i^f} \right)\), the mean rates are ν
i/j
=〈ρ
i/j
〉. Assuming stationarity, the raw cross-correlation function is given by
$${\Gamma _{ji}}(\Delta t) = {\left\langle {{\rho _j}(t){\rho _i}(t + \Delta t)} \right\rangle _t}$$
(16)
i.e. averaging over t while keeping the delay Δt between the two spike trains fixed. The synaptic drift is obtained by integrating the synaptic weight changes given by (10) over Δt weighted by the probability, as expressed by (16), of the temporal difference Δt occurring between a pre- and post-synaptic spike:
$$\dot w = - {F_ - }(w)\int\limits_{ - \infty }^0 {{\rm{d}}\Delta t{K_ - }(\Delta t){\Gamma _{ji}}(\Delta t) + {F_ + }(w)} \int\limits_0^\infty {{\rm{d}}\Delta t{K_ + }(\Delta t){\Gamma _{ji}}(\Delta t)} $$
(17)
, where \({K_ \pm }(\Delta t) = \exp ( - \left| {\Delta t} \right|/{\tau _ \pm })\), the window function of STDP.
As we are only interested in the qualitative structure of fixed points rather than their exact location, we will simplify the analysis by assuming that the pre- and postsynaptic spike trains are independent Poisson processes with the same rate, i.e. <ρ
i
>=<ρ
j
>=ν and Γ
ji
(Δt) = ν
2. We can therefore write:
$$\dot w = - {F_ - }(w ){\tau _ - }{\nu ^2} + {F_ + }(w ){\tau _ + }{\nu ^2}$$
.
In general, the rate ν of a neuron is dependent on the weight of its incoming synapses and so the right side of this equation cannot be easily determined. However, we can reformulate the equation as:
$${{\dot w } \over {{\nu ^2}}} = - {F_ - }(w){\tau _ - } + {F_ + }(w){\tau _ + }$$
(18)
.
The fixed points of the synaptic dynamics are given by definition by .w = 0, and therefore also by ˙w/ν
2 = 0. Figure 6 plots (18) for a range of w and a variety of STDP models. In all cases except for additive STDP the curves pass through ˙w/ν
2 = 0 at an intermediate value of w and with a negative slope, i.e. for weights below the fixed point there is a net potentiating effect, and for weights above the fixed point there is a net depressing effect, resulting in a stable fixed point which is not at an extremum of the weight range. In the case of additive STDP there is no such fixed point, stable or otherwise.
The behavior of the additive model can be assessed more accurately by relaxing the assumption that pre- and postsynaptic spike trains can be described by independent Poisson processes. Instead, we consider a very simple neuron model in which the output spike train is generated by an inhomogeneous Poisson process with rate \({\nu _i}({u_i}) = {[\alpha {u_i} - {\nu _0}]_ + }\) with scaling factor α, threshold ν
0 and membrane potential \({u_i}(t) = {\Sigma _j}{w _{ij}}\varepsilon (t - t_j^f)\), where ∈(t) denotes the time course of an excitatory postsynaptic potential generated by a presynaptic spike arrival. The notation [x]+ denotes a piecewise linear function: [x]+ = x for x > 0 and zero otherwise. In the following we assume that the argument of our piecewise linear function is positive so that we can suppress the square brackets. Assuming once again that all input spike trains are Poisson processes with rate ν, the expected firing rate of the postsynaptic neuron is simply:
$${\nu _i} = - {\nu _0} + \alpha \nu \bar \varepsilon \matrix{ \Sigma \cr j \cr } {w _{ij}}$$
, where ¯ε=ε∈(s)ds, the total area under an excitatory postsynaptic potential. The conditional rate of firing given an input spike at time t
f
j
is given by
$${\nu _i}(t) = - {\nu _0} + \alpha \nu \bar \varepsilon \matrix{ \Sigma \cr j \cr } {w _{ij}} + \alpha {w _{ij}}\varepsilon (t - t_j^f)$$
, thus the postsynaptic spike train is correlated with the presynaptic spike trains. This term shows up as additional spike-spike correlations in the correlation function Γ
ji
. Hence, in addition to the terms in (18), the synaptic dynamics contains a term of the form αvw F
+(w) ε
K
+(s)ƒ(s)ds that is linear rather than quadratic in the presynaptic firing rate (Kempter et al. 1999, Kempter et al. 2001). With this additional term, (18) becomes
$${{\dot w } \over \nu } = {\nu _i}[ - {F_ - }(w){\tau _ - } + {F_ + }(w){\tau _ + }] + \alpha w {F_ + }(w)\smallint {K_ + }(s)\varepsilon (s){\rm{d}}s$$
(19)
. For the multiplicative models the argument hardly changes, but for the additive model it does. For \(\eqalign{ & s \cr & C = {F_ - }(w ){\tau _ - } - {F_ + }(w ){\tau _ + } > 0 \cr} \), the additive model has a fixed point which we find by setting the right-hand side of (19) to zero, i.e.
$$0 = - C{\nu _i} + \alpha w{C_{{\rm{ss}}}}$$
, where C
ss = F
+(w)εK
+(s)∈(s)ds denotes the contribution of the spike-spike correlations. In contrast to the curves in Fig. 6, the slope at the zero-crossing is now positive, indicating instability of the fixed point. This instability leads to the formation of a bimodal weight distribution that is typical for the additive model. Despite the instability of individual weights (which move to their upper or lower bounds), the mean firing rate of the neuron is stabilized (Kempter et al. 2001). To see this we consider the evolution of the output rate dν
i
/dt = αν¯εΣ
j
dw
ij
/dt. Since \(d{w _{ij}}/{\rm{d}}t = - C{\nu _i}\nu + \alpha \nu {w_{ij}}{C_{{\rm{ss}}}}\) and \(\alpha \nu \bar \varepsilon {\Sigma _j}{w _{ij}} = {\nu _i} + {\nu _0}\), we can write:
$${{d{\nu _i}} \over {{\rm{d}}t}} = - \alpha {\nu ^2}\bar \varepsilon NC{\nu _i} + \alpha \nu ({\nu _i} + {\nu _0}){C_{{\rm{ss}}}}$$
where N is the number of synapses converging on the post-synaptic neuron. Thus we have a dynamics of the form:
$${{d{\nu _i}} \over {{\rm{d}}t}} = {{({\nu _i} - {\nu _{{\rm{FP}}}})} \over {{\tau _\nu }}}$$
with a fixed point given by:
$${\nu _{{\rm{FP}}}} = {{{C_{{\rm{ss}}}}{\nu _0}} \over {NC\nu \bar \varepsilon - {C_{{\rm{ss}}}}}}$$
(20)
and a time constant \({\tau _\nu } = {(\alpha \nu [NC\nu \bar \varepsilon - {C_{{\rm{ss}}}}])^{ - 1}}\). Note that stabilization at a positive rate requires that ν
0 > 0 andC > 0. The first condition states that, in the absence of any input, the neuron does not show any spontaneous activity, and this is trivially true for all standard neuron models, including the integrate-and-fire model. The latter condition is equivalent to the requirement that the integral over the STDP curve be negative: \(\smallint {\rm{d}}s[{F_ + }(w){K_ + }(s) - {F_ - }(w){K_ - }(s)] = - C < 0\). Exact conditions for stabilization of output rates are given in Kempter et al. (2001). Since for constant input rates ν we have \({\nu _i} = {\nu _0} + \alpha \nu \bar \varepsilon {\Sigma _j}{w_{ij}}\), stabilization of the output rate implies normalization of the summed weights. Hence STDP can lead to a control of total presynaptic input and of the postsynaptic firing rate — a feature that is usually associated with homeostatic processes rather than Hebbian learning per se (Kempter et al. 1999, Kempter et al. 2001; Song et al. 2000).
Note that the existence of a fixed point and its stability does not crucially depend on the presence of soft or hard bounds on the weight. Equations (18) and (19) can equate to zero for hard-bounded or or unbounded rules.
4.1.1.3 Consequences for network stability
Results on the consequences of STDP in large-scale networks are few and far between, and tend to contradict each other. Part of the reason for the lack of simulation papers on this important subject is the fact that simulating such networks consumes huge amounts of memory, is computationally expensive, and potentially requires extremely long simulation times to overcome transients in the weight dynamics which can be of the order of hundreds of seconds of biological time. A lack of theoretical papers on the subject can be explained by the complexity of the interactions between the activity dynamics of the network and the weight dynamics, although some progress is being made in this area (Burkitt et al. 2007).
It was recently shown that power law STDP is compatible with balanced random networks in the asynchronousirregular regime (Morrison et al. 2007), resulting in a unimodal distribution of weights and no self-organization of structure. This result was verified for Gütig et al. (2003) STDP for an intermediate value of the exponent (μ = 0.4). Although it has not yet been possible to perform systematic tests, it seems likely that all the formulations of STDP with the fixed point structure discussed in Sect. 4.1.1.1 would give qualitatively similar behavior. The results for additive STDP seem to be more contradictory. Izhikevich et al. (2004) reported self-organization of neuronal groups, whereas the chief feature of the networks investigated by Iglesias et al. (2005) seems to be extensive withering of the synaptic connections. In the former case, it is the existence of many strong synapses which defines the network, in the latter, the presence of many weak ones. This discrepancy may be attributable to different choices for the effective stabilized firing rates (20) in combination with different choices of delays in the network, see Sect. 4.1.3.
4.1.2 Spike pairing scheme
There are many possible ways to pair pre- and postsynaptic spikes to generate a weight update in an STDP model. In an all-to-all scheme, each presynaptic spike is paired with all previous postsynaptic spikes to effect depression, and each postsynaptic spike is paired with all previous presynaptic spikes to effect potentiation. This is the interpretation used for the fixed point analysis in Sect. 4.1.1.1 and can be implemented using local variables as demonstrated in Sect. 4.1. In a nearest neighbor scheme, only the closest interactions are considered. However, there are multiple possible interpretations of nearest neighbor, as can be seen in Fig. 7. Nearest neighbor schemes can also be realized in terms of appropriately chosen local variables. The symmetric nearest-neighbor scheme shown in Fig. 7a can be implemented by pre- and postsynaptic traces that reset to 1, rather than incrementing by 1 as is the case for the all-to-all scheme. In the case of the presynaptic centered interpretation depicted in Fig. 7B, the postsynaptic trace resets to 1 as in the previous example, but the presynaptic trace must be implemented with a slightly more complicated dynamics:
$${{{{\rm{d}}_{xj}}} \over {{\rm{d}}t}} = - {{{x_j}} \over {{\tau _x}}} + \sum\limits_{t_j^f} {(1 - x_j^ - )\delta (t - t_j^f) - } \sum\limits_{t_j^f} {x_j^ - \delta (t - t_i^f)} $$
, where t
f
j
and t
f
i
denote the firing times of the pre- and postsynaptic neurons respectively, and x
−
j
gives the value of x
j
just before the update. In other words, the trace is reset to 1 on the occurrence of a presynaptic spike and reset to 0 on the occurrence of a postsynaptic spike. Similarly, the reduced symmetric interpretation shown in Fig. 7c can be implemented by pre- and postsynaptic ‘doubly resetting’ traces of this form.
It is sometimes assumed that the scheme used makes no difference, as the ISI of cortical network models is typically an order of magnitude larger than the time constant of the STDP window. However, this is not generally true (Kempter et al. 2001; Izhikevich and Desai 2003; Morrison et al. 2007). For a review of a wide variety of schemes and their consequences, particularly with respect to selectivity of higher-frequency inputs, see Burkitt et al. (2004). Experimental results on this issue suggest limited interaction between pairs of spikes. Sjostrom et al. (2001) found that their data was best fit by a nearest neighbor interaction similar to Fig. 7c but giving precedence to LTP, i.e. a postsynaptic spike can only contribute to a post-before-pre pairing if it has not already contributed to a pre-before-post pairing. However, this result may also be due to the limitations of pair-based STDP models to explain the experimentally observed frequency dependence, see Sect. 4.2. More recently, Froemke et al. (2006) demonstrated that the amount of LTD was not dependent on the number of presynaptic spikes following a postsynaptic spike, suggesting nearest-neighbor interactions for depression as in Fig. 7c. However, the amount of LTP was negatively correlated with the number of presynaptic spikes preceding a postsynaptic spike. This suggests that multiple spike pairings contribute to LTP, but not in the linear fashion of the all-to-all scheme, which would predict a positive correlation between the number of spikes and the amount of LTP. Again, these results are good evidence for the limitations of pair-based STDP rules.
4.1.3 Synaptic delays
Up until now we have referred to Δt as the temporal difference between a pre- and a postsynaptic spike, i.e. \(\Delta t = t_i^f - t_j^f\). However, many classic STDP experiments are expressed in terms of the temporal difference between the start of the EPSP and the postsynaptic spike (Markram et al. 1997; Bi and Poo 1998). In fact, when a presynaptic spike is generated at t
f
j
, it must first travel down the axon before arriving at the synapse, thus arriving at \(t_j^s = t_j^f + {d_{\rm{A}}}\), where d
A is the axonal propagation delay. Similarly, a postsynaptic spike at t
f
i
must backpropagate through the dendrite before arriving at the synapse at \(t_i^s = t_i^f + {d_{{\rm{BP}}}}\), where d
BP is the backpropagation delay. Consequently, the relevant temporal difference for STDP update rules is \(\Delta {t^s} = t_i^s - t_j^s\) as initially suggested by Gerstner et al. (1993) and Debanne et al. (1998). Senn et al. (2002) showed that under fairly general conditions, STDP may cause adaptation in the presynaptic and postsynaptic delays in order to optimize the effect of the presynaptic spike on the postsynaptic neuron. In order to calculate the synaptic drift as in (17), we therefore need to integrate the synaptic weight changes over Δt
s, weighted by the raw cross-correlation function at the synapse. With \({\Gamma _{ji}}(\Delta t) = {\Gamma _{ji}}(\Delta {t^s} + ({d_{\rm{A}}} - {d_{{\rm{BP}}}}))\), we reformulate (17) as:
$$\dot w = - {F_ - }(w)\int\limits_{ - \infty }^0 {{\rm{d}}\Delta {t^s}{K_ - }(\Delta {t^s}){\Gamma _{ji}}(\Delta {t^s} + ({d_{\rm{A}}} - {d_{{\rm{BP}}}})) + {F_ + }(\omega )} \int\limits_{ - \infty }^0 {(\Delta {t^s}){\Gamma _{ji}}(\Delta {t^s} + ({d_{\rm{A}}}} - {d_{{\rm{BP}}}}))$$
(21)
.
In the case of independent Poisson processes as in Sect. 4.1.1.1, the shift of the raw cross-correlation function by (d
A − d
BP) has no effect, as Γ
ji
(Δt)} is constant. Generally, however, this is not the case. For example, networks of neurons, both in experiment and simulation, typically exhibit oscillations with a period several times larger than the synaptic delay, even when individual spike trains are irregular (see Kriener et al. 2008, for discussion). If the axonal delay is the same as the backpropagation delay, i.e. d
A = d
BP = d/2, where d is the total transmission delay of the spike, the raw cross-correlation function at the synapse is the same as the raw cross-correlation at the soma:
$$\Gamma _{ji}^s(\Delta {t^s}) = {\Gamma _{ji}}(\Delta {t^s} + ({d_{\rm{A}}} - {d_{{\rm{BP}}}})) = {\Gamma _{ji}}(\Delta t)$$
. This situation is depicted in Fig. 8b. Let w
0 be the synaptic weight for which the synaptic drift given in (21) is 0, i.e. the fixed point of the synaptic dynamics for the cross-correlation shown. If the axonal delay is larger than the backpropagation delay, this results in a shift of the raw cross-correlation function to the left. This is shown in Fig. 8a for the extreme case of d
A = d, d
BP = 0, resulting in a net shift of d. This increases the value of the first integral in (21) and decreases the second integral, such that˙w< 0 atw
0. Conversely, if the axonal delay is smaller than the backpropagation delay, the raw cross-correlation function is shifted to the right (Fig. 8c, for the extreme case of d
A = 0, d
BP = d). This decreases the value of the first integral in (21) and increases the second integral, such that ˙w > 0 at w
0. Therefore, a given network dynamics may cause systematic depression, systematic potentiation or no systematic change at all to the synaptic weights, depending on the partition of the synaptic delay into axonal and dendritic contributions. Systematic synaptic weight changes can in turn result in qualitatively different network behavior. For example, in Morrison et al. (2007) small systematic biases in the synaptic weight dynamics were applied to a network with an equilibrium characterized by a unimodal weight distribution and medium rate (< 10 Hz) asynchronous irregular activity dynamics. Here, a small systematic depression led to a lower weight, lower rate equilibrium also in the asynchronous irregular regime, whereas a systematic potentiation led to a sudden transition out of the asynchronous irregular regime: the activity was characterized by strongly patterned high-rate peaks of activity interspersed with silence, and the unimodal weight distribution splintered into several peaks.
4.2 Beyond pair effects
There is considerable evidence that the pair-based rules discussed above cannot give a full account of STDP. Specifically, they reproduce neither the dependence of plasticity on the repetition frequency of pairs of spikes in an experimental protocol, nor the results of recent triplet and quadruplet experiments.
STDP experiments are usually carried out with about 60 pairs of spikes. The temporal distance of the spikes in the pair is of the order of a few to tens of milliseconds, whereas the temporal distance between the pairs is of the order of hundreds of milliseconds to seconds. In the case of a facilitation protocol (i.e. pre-before-post), standard pair-based STDP models predict that if the repetition frequency is increased, the strength of the depressing interaction (i.e. post-before-pre) becomes greater, leading to less net potentiation. This prediction is independent of whether the spike pairing scheme is all-to-all or nearest neighbor (see Sect. 4.1.2). However, experiments show that increasing the repetition frequency leads to an increase in potentiation (Sjostrom et al. 2001). Other recent experiments employed multiple-spike protocols, such as repeated presentations of symmetric triplets of the form pre-post-pre and post-pre-post (Bi and Wang 2002; Froemke and Dan 2002; Wang et al. 2005; Froemke et al. 2006). Standard pair-based models predict that the two sequences should give essentially the same results, as they each contain one pre-post pair and one post-pre pair. Experimentally, quite different results are observed.
Here we review two examples of simple models which account for these experimental findings. For other models which also reproduce frequency dependence or multiple-spike protocol results, see Abarbanel et al. (2002), Senn (2002) and Appleby and Elliott (2005).
4.2.1 Triplet model
One simple approach to modeling STDP which addresses these issues is the triplet rule developed by Pfister and Gerstner (2006). This model is based on sets of three spikes (one presynaptic and two postsynaptic). As in the case of pair-based rules, the triplet rule can be easily implemented with local variables as follows. Similarly to pair-based rules, each spike from presynaptic neuron j contributes to a trace x
j
at the synapse:
$${{{{\rm{d}}_{xj}}} \over {{\rm{d}}t}} = - {{{x_j}} \over {{\tau _x}}} + \sum\limits_{t_j^f} {\delta (t - t_j^f)} $$
, where t
f
j
denotes the firing times of the presynaptic neuron. Unlike pair-based rules, each spike from postsynaptic neuron i contributes to a fast trace y
1
i
and a slow trace y
2
i
at the synapse:
$${{{\rm{d}}y_i^1} \over {{\rm{d}}t}} = - {{y_i^1} \over {{\tau _1}}} + \delta (t - t_i^f)$$
$${{{\rm{d}}y_i^2} \over {{\rm{d}}t}} = - {{y_i^2} \over {{\tau _2}}} + \delta (t - t_i^f)$$
, where τ
1 < τ
2, see Fig. 9. LTD is induced as in the standard STDP pair model given in (13), i.e. the weight change is proportional to the value of the fast postsynaptic trace y{sri/1} evaluated at the moment of a presynaptic spike. The new feature of the rule is that LTP is induced by a triplet effect: the weight change is proportional to the value of the presynaptic trace x
j
evaluated at the moment of a postsynaptic spike and also to the slow postsynaptic trace y
2
i
remaining from previous postsynaptic spikes:
$$\Delta w_{ij}^ + \left( {t_i^f} \right) = {F_ + }({w_{ij}}){x_j}(t_i^f)y_i^2(t_i^{f - })$$
where t
f−
i
indicates that the function y
2
i
is to be evaluated before it is incremented due to the postsynaptic spike at t
f
i
. Analogously to pair-based models, the triplet rule can also be implemented with nearest-neighbor rather than all-to-all spike pairings by an appropriate choice of trace dynamics, see Sect. 4.1.2.
The triplet rule reproduces experimental data from visual cortical slices (Sjostrom et al. 2001) that increasing the repetition frequency in the STDP pairing protocol increases net potentiation (Fig. 10). It also gives a good fit to experiments based on triplet protocols in hippocampal culture (Wang et al. 2005). The main functional advantage of such a triplet learning rule is that it can be mapped to a Bienenstock-Cooper-Munro learning rule (Bienenstock et al. 1982): if we assume that the pre- and postsynaptic spike trains are governed by Poisson statistics, the triplet rule exhibits depression for low postsynaptic firing rates and potentiation for high postsynaptic firing rates. If we further assume that the triplet term in the learning rule depends on the mean postsynaptic frequency, a sliding threshold between potentiation and depression can be defined. In this way, the learning rule matches the requirements of the BCM theory and inherits the properties of the BCM learning rule such as input selectivity. From BCM properties, we can immediately conclude that the model should be useful for receptive field development. Note that earlier efforts to show that STDP maps to the BCM model (Izhikevich and Desai 2003; Senn et al. 2000) demonstrated neither an exact mapping nor a sliding threshold. The exact relationship between the above triplet model and other models is discussed in Pfister and Gerstner (2006).
4.2.2 Suppression model
An alternative model to address the inability of standard pair-based models to account for data obtained from triplet and quadruplet spike protocols was developed by Froemke and Dan (2002). They observed that in triplet protocols of the form pre-post-pre, as long as the intervals between the spikes were reasonably short (< 15 ms), the timing of the pre-post pair was a better predictor for the change in the synaptic strength than either the timing of the post-pre pair or of both timings taken together. Similarly, in post-pre-post protocols, the timing of the first post-pre pairing was the best predictor for the change of synaptic strength. On the basis of this observation, they proposed a model in which the synaptic weight change is not just dependent on the timing of a spike pair, but also on the efficacy of the spikes. Each spike of presynaptic neuron j sets the presynaptic spike efficacy ∈
j
to 0 whereafter it recovers exponentially to 1 with a time constant τ
j
. The efficacy of the nth presynaptic spike is given by:
$$\varepsilon _j^n = 1 - \exp \left( { - \left( {t_j^n - t_j^{n - 1}} \right)/{\tau _j}} \right)$$
, where t
n
j
denotes the nth spike of neuron j. In other words, the efficacy of a spike is suppressed by the proximity of a previous spike. Similarly, the postsynaptic spike efficacy is reset to 0 by each spike of postsynaptic neuron i, recovering exponentially to 1 with time constant τ
i
. The model can be implemented with local variables as follows. Each presynaptic spike contributes to an efficacy trace ∈
j
(t) with dynamics:
$${{{\rm{d}}{\varepsilon _j}} \over {{\rm{d}}t}} = - {{{\varepsilon _j} - 1} \over {{\tau _i}}} - \sum\limits_{t_j^f} {\varepsilon _j^ - \delta (t - t_i^f)} $$
, where ∈
−
j
denotes the value of ∈
j
just before the update. The standard presynaptic trace x
j
given in (11) is adapted to take the spike efficacy into account:
$${{{\rm{d}}{x_j}} \over {{\rm{d}}t}} = - {{{x_j}} \over {{\tau _x}}} - \sum\limits_{t_j^f} {\varepsilon _j^ - \delta (t - t_i^f)} $$
, i.e. each presynaptic spike increments x
j
by the value of the spike efficacy before the update. Similarly, each postsynaptic spike contributes to an efficacy trace ∈
i
(t) with dynamics:
$${{{\rm{d}}{\varepsilon _i}} \over {{\rm{d}}t}} = - {{{\varepsilon _i} - 1} \over {{\tau _i}}} - \sum\limits_{t_j^f} {\varepsilon _i^ - \delta (t - t_i^f)} $$
, and a postsynaptic trace y
i
with increments weighted by the postsynaptic spike efficacy:
$${{{\rm{d}}{y_i}} \over {{\rm{d}}t}} = - {{{y_i}} \over {{\tau _y}}} + \sum\limits_{t_j^f} {\varepsilon _i^ - \delta (t - t_i^f)} $$
The weight updates on the occurrence of a post- or presynaptic spike are therefore given by:
$$\Delta w _{ij}^ + \left( {t_j^f} \right) = {F_ + }({w _{ij}}){x_j}(t_j^f)\varepsilon _i^ - (t_i^f)$$
$$\Delta w _{ij}^ - \left( {t_j^f} \right) = - {F_ - }({w _{ij}}){y_j}(t_j^f)\varepsilon _j^ - (t_j^f)$$
.
This model gives a good fit to triplet and quadruplet protocols in visual cortex slice, and also gives a much better prediction for synaptic modification due to natural spike trains (Froemke and Dan 2002). However, it does not predict the increase of LTP with the repetition frequency observed by Sjostrom et al. (2001). A revised version of the model (Froemke et al. 2006) also accounts for the switch of LTD to LTP at high frequencies by modifying the efficacy functions.
4.3 Voltage dependence
Traditional LTP/LTD experiments employ the following induction paradigm: the postsynaptic neuron is held at a fixed depolarization while one or several presynaptic neurons are activated. Often a presynaptic pathway is stimulated extracellularly, so that several presynaptic neurons are activated. Depending on the level of the postsynaptic membrane potential, the activated synapses increase their efficacy while other non-activated synapses do not change their weight (Artola et al. 1990; Artola and Singer 1993). More recently, depolarization has also been combined with STDP experiments. In particular, Sjostrom et al. (2004) showed a dependence of synaptic weight changes on the synaptic membrane potential just before a postsynaptic spike.
There is an ongoing discussion whether the voltage dependence is more fundamental than the dependence on postsynaptic spiking. Indeed, voltage dependence alone can generate STDP-like behavior (Brader et al. 2007), as the membrane potential behaves in a characteristic way in the vicinity of a spike (high shortly before a spike, and low shortly after). Alternatively, a dependence on the slope of the postsynaptic membrane potential has also been shown to reproduce the characteristic STDP weight change curve (Saudargiene et al. 2003). The voltage effects caused by back-propagating spikes is implicitly contained in the mechanistic formulation of STDP models outlined above. In particular, the fast postsynaptic trace y
1 in the above triplet model could be seen as an approximation of a back-propagating action potential. However, the converse is not true: a pure STDP rule does not automatically generate a voltage dependence. Moreover, synaptic effects caused by subthreshold depolarization in the absence of postsynaptic firing cannot be modeled by standard STDP or triplet models.
4.4 Induction versus maintenance
We stress that all the above models concern induction of potentiation and depression, but not their maintenance. The induction of LTP may take only a few seconds: for example, stimulation with 50 pairs of pre- and postsynaptic spikes given at 20Hz takes less than 3 s. However, afterwards the synapse takes 60 min or more to consolidate these changes, and this process may also be interrupted (Frey and Morris 1997). During this time synapses are ‘tagged’, that is, they are ready for consolidation. Consolidation is thought to rely on a different molecular mechanism than that of induction. Simply speaking, gene transcription is necessary to trigger the building of new proteins that increase the synaptic efficacy.
4.4.1 Functional consequences
Long-term stability of synapses is necessary to retain memories that have been learned, despite ongoing activity of presynaptic neurons. A simple possibility used in many models is that plasticity is simply switched off once the neuron has learned what it should. This approach makes sense in the context of reward-based learning: the learning rate goes to zero once the actual reward equals the expected reward and learning stops automatically (see Sect. 5.2). It also makes sense in the framework of supervised learning (see Sect. 5.1). Learning is normally driven by the difference between desired output and actual output. However, in the context of unsupervised learning it is inconsistent to switch off the dynamics. Nevertheless, receptive field properties should be retained for a fairly long time even if the stimulation characteristic changes.
4.4.2 Bistability model
A simple model of maintenance has been proposed by Fusi et al. (2000). The basis of the model is a hidden variable that has an unstable fixed point (threshold). If the variable has a value above threshold it converges towards 1; otherwise towards 0. To stay within the framework of the previous sections, let us suppose that the weight w is calculated by one of the STDP or short-term plasticity models. Maintenance is implemented by adding on top of the STDP dynamics a slow bistable dynamics (Gerstner and Kistler 2002):
$${{{\rm{d}}w} \over {{\rm{d}}t}} = \alpha (w) = - w(1 - w)({w_\theta } - w)/{\tau _\alpha }$$
, where τ
a is a time constant of consolidation in the range of several minutes of biological time. The result is that in the absence of any stimulation, individual synapses evolve towards binary values of 0 or 1 which are intrinsically stable fixed points of the slow dynamics. As a result, rather strong stimuli are necessary to perturb the synaptic dynamics.
4.4.3 Biological evidence
Whether single synapses themselves are binary or continuous is a matter of intense debate. Some experiments have suggested that synapses are binary (Petersen et al. 1998; O’Connor et al. 2005). However, this would seem to result in a bistable distribution of weights which is at odds with the unimodal distribution reported by other studies (Turrigiano et al. 1998; Sjostrom et al. 2001; Song et al. 2005), and with the finding that the magnitude of LTP/LTD increases with the number of spike pairs in a protocol until saturation is reached (Froemke et al. 2006).
Some possibilities to reconcile these findings include: (i) since pairs of neurons form several contacts with each other, it is likely that in standard plasticity experiments several synapses are measured at the same time; (ii) LTP and STDP results are typically reported as pooled experiments over several pairs of neurons. Under the assumption that the upper bound is not the same for all synapses, a broad distribution could result; (iii) both unimodal distribution and bimodal distributions could be stable. Untrained neurons would show a unimodal distribution whereas neurons that have learned to respond to a specific pattern would develop a bimodal distribution of synaptic weights (Toyoizumi et al. 2007); (iv) all synapses are binary, but the efficacy of the ‘strong’ state is subject to short-term plasticity and homeostasis; (v) some synapses are binary and some are not. Potentially a combination of several of these possibilities must be considered in order to explain the experimental findings.