Skip to main content
Log in

Variable synaptic strengths controls the firing rate distribution in feedforward neural networks

  • Published:
Journal of Computational Neuroscience Aims and scope Submit manuscript

Abstract

Heterogeneity of firing rate statistics is known to have severe consequences on neural coding. Recent experimental recordings in weakly electric fish indicate that the distribution-width of superficial pyramidal cell firing rates (trial- and time-averaged) in the electrosensory lateral line lobe (ELL) depends on the stimulus, and also that network inputs can mediate changes in the firing rate distribution across the population. We previously developed theoretical methods to understand how two attributes (synaptic and intrinsic heterogeneity) interact and alter the firing rate distribution in a population of integrate-and-fire neurons with random recurrent coupling. Inspired by our experimental data, we extend these theoretical results to a delayed feedforward spiking network that qualitatively capture the changes of firing rate heterogeneity observed in in-vivo recordings. We demonstrate how heterogeneous neural attributes alter firing rate heterogeneity, accounting for the effect with various sensory stimuli. The model predicts how the strength of the effective network connectivity is related to intrinsic heterogeneity in such delayed feedforward networks: the strength of the feedforward input is positively correlated with excitability (threshold value for spiking) when firing rate heterogeneity is low and is negatively correlated with excitability with high firing rate heterogeneity. We also show how our theory can be used to predict effective neural architecture. We demonstrate that neural attributes do not interact in a simple manner but rather in a complex stimulus-dependent fashion to control neural heterogeneity and discuss how it can ultimately shape population codes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. In refractory, the other variables are governed by their ODEs

  2. In Fig. 4E, the effective correlation for a given ϕ is the Pearson’s correlation calculated on the set of (𝜃 j ,q j ) weighted by the number of presynaptic inputs.

  3. Although ignoring the refractory period could be problematic for large firing rates, we emphasize that the purpose of our this analysis is not for quantitative matching but rather for an analytic explanation. A similar calculation with the refractory period was performed (not shown) but the results were not insightful.

References

  • Abbott, L.F., & van Vreeswijk, C. (1993). Asynchronous states in networks of pulse-coupled oscillators. Physical Review E, 48, 1483–1490.

    Article  CAS  Google Scholar 

  • Ahn, J., Kreeger, L., Lubejko, S., Butts, D., & MacLeod, K. (2014). Heterogeneity of intrinsic biophysical properties among cochlear nucleus neurons improves the population coding of temporal information. Journal of Neurophysiology, 111(11), 2320–2331.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Apfaltrer, F., Ly, C., & Tranchina, D. (2006). Population density methods for stochastic neurons with realistic synaptic kinetics: firing rate dynamics and fast computational methods. Network: Computation in Neural Systems, 17, 373–418.

    Article  Google Scholar 

  • Averbeck, B., Latham, P., & Pouget, A. (2006). Neural correlations, population coding and computation. Nature Reviews Neuroscience, 7, 358–366.

    Article  CAS  PubMed  Google Scholar 

  • Azouz, R., & Gray, C.M. (2000). Dynamic spike threshold reveals a mechanism for synaptic coincidence detection in cortical neurons in vivo. Proceedings of the National Academy of Sciences, 97(14), 8110–8115.

    Article  CAS  Google Scholar 

  • Barna, G., Gröbler, T., & Érdi, P. (1998). Statistical model of the hippocampal CA3 region, ii. The population framework: model of rhythmic activity in CA3 slice. Biological Cybernetics, 79, 309–321.

    Article  CAS  PubMed  Google Scholar 

  • Bastian, J. (1986). Gain control in the electrosensory system mediated by descending inputs to the electrosensory lateral line lobe. Journal of Neuroscience, 6(2), 553–562.

    CAS  PubMed  Google Scholar 

  • Bastian, J., & Nguyenkim, J. (2001). Dendritic modulation of burst-like firing in sensory neurons. Journal of Neurophysiology, 85(1), 10–22.

    Article  CAS  PubMed  Google Scholar 

  • Bastian, J., Chacron, M., & Maler, L. (2004). Plastic and nonplastic pyramidal cells perform unique roles in a network capable of adaptive redundancy reduction. Neuron, 41(5), 767–779.

    Article  CAS  PubMed  Google Scholar 

  • Berman, N., & Maler, L. (1999). Neural architecture of the electrosensory lateral line lobe: adaptations for coincidence detection, a sensory searchlight and frequency-dependent adaptive filtering. Journal of Experimental Biology, 202(10), 1243–1253.

    CAS  PubMed  Google Scholar 

  • Bol, K., Marsat, G., Harvey-Girard, E., Longtin, A., & Maler, L. (2011). Frequency-tuned cerebellar channels and burst-induced ltd lead to the cancellation of redundant sensory inputs. The Journal of Neuroscience, 31(30), 11,028–11,038.

    Article  CAS  Google Scholar 

  • Bremaud, A., West, D., & Thomson, A. (2007). Binomial parameters differ across neocortical layers and with different classes of connections in adult rat and cat neocortex. Proceedings of the National Academy of Sciences, 104, 14,134–14,139.

    Article  CAS  Google Scholar 

  • Brown, E., Moehlis, J., & Holmes, P. (2004). On the phase reduction and response dynamics of neural oscillator populations. Neural Computation, 16, 673–715.

    Article  PubMed  Google Scholar 

  • Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrate-and-fire neurons with low firing rates. Neural Computation, 11, 1621–1671.

    Article  CAS  PubMed  Google Scholar 

  • Brunel, N., & Latham, P. (2003). Firing rate of the noisy quadratic integrate-and-fire neuron. Neural Computation, 15, 2281–2306.

    Article  PubMed  Google Scholar 

  • Brunel, N., Chance, F., Fourcaud, N., & Abbott, L. (2001). Effects of synaptic noise and filtering on the frequency response of spiking neurons. Physical Review Letters, 86, 2186–2189.

    Article  CAS  PubMed  Google Scholar 

  • Bruno, R., & Simons, D. (2002). Feedforward mechanisms of excitatory and inhibitory cortical receptive fields. The Journal of Neuroscience, 22(24), 10,966–10,975.

    CAS  Google Scholar 

  • Chacron, M., Maler, L., & Bastian, J. (2005). Feedback and feedforward control of frequency tuning to naturalistic stimuli. The Journal of Neuroscience, 25(23), 5521–5532.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Chacron, M.J., & Bastian, J. (2008). Population coding by electrosensory neurons. Journal of Neurophysiology, 99(4), 1825–1835.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chelaru, M., & Dragoi, V. (2008). Efficient coding in heterogeneous neuronal populations. Proceedings of the National Academy of Sciences, 105, 16,344–16,349.

    Article  CAS  Google Scholar 

  • Cohen, M., & Kohn, A. (2011). Measuring and interpreting neuronal correlations. Nature Neuroscience, 14, 811–819.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Doiron, B., Longtin, A., Turner, R.W., & Maler, L. (2001). Model of gamma frequency burst discharge generated by conditional backpropagation. Journal of neurophysiology, 86(4), 1523– 1545.

    Article  CAS  PubMed  Google Scholar 

  • Doiron, B., Chacron, M.J., Maler, L., Longtin, A., & Bastian, J. (2003). Inhibitory feedback required for network oscillatory responses to communication but not prey stimuli. Nature, 421(6922), 539–543.

    Article  CAS  PubMed  Google Scholar 

  • Doiron, B., Litwin-Kumar, A., Rosenbaum, R., Ocker, G., & Josić, K. (2016). The mechanics of state-dependent neural correlations. Nature Neuroscience, 19(3), 383–393.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Ecker, A., Berens, P., Tolias, A., & Bethge, M. (2011). The effect of noise correlations in populations of diversely tuned neurons. The Journal of Neuroscience, 283(40), 14,272–14,283.

    Article  Google Scholar 

  • Faul, F., Erdfelder, E., Lang, A.G., & Buchner, A. (2007). G* power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175–191.

    Article  PubMed  Google Scholar 

  • Faul, F., Erdfelder, E., Buchner, A., & Lang, A.G. (2009). Statistical power analyses using g* power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.

    Article  PubMed  Google Scholar 

  • Ferster, D., & Miller, K. (2000). Neural mechanisms of orientation selectivity in the visual cortex. Annual Review of Neuroscience, 23(1), 441–471.

    Article  CAS  PubMed  Google Scholar 

  • Fourcaud, N., & Brunel, N. (2002). Dynamics of the firing probability of noisy integrate-and-fire neuron. Neural Computation, 14, 2057–2110.

    Article  PubMed  Google Scholar 

  • Frank, K., & Becker, M. Nastuk, W. (Ed.). (1964). Microelectrodes for recording and stimulation. New York: Academic.

  • Georgopoulos, A., Schwartz, A., & Kettner, R. (1986). Neuronal population coding of movement direction. Science, 233(4771), 1416–1419.

    Article  CAS  PubMed  Google Scholar 

  • Gerstner, W. (2000). Population dynamics of spiking neurons: fast transients, asynchronous states, and locking. Neural Computation, 12, 43–90.

    Article  CAS  PubMed  Google Scholar 

  • Gussin, D., Benda, J., & Maler, L. (2007). Limits of linear rate coding of dynamic stimuli by electroreceptor afferents. Journal of Neurophysiology, 97(4), 2917–2929.

    Article  PubMed  Google Scholar 

  • Hermann, G., & Touboul, J. (2012). Heterogeneous connections induce oscillations in large-scale networks. Physical Review Letters, 109, 018,702.

    Article  Google Scholar 

  • Hertäg, L., Durstewitz, D., & Brunel, N. (2014). Analytical approximations of the firing rate of an adaptive exponential integrate-and-fire neuron in the presence of synaptic noise. Frontiers in Computational Neuroscience, 8.

  • Hoenig, J.M., & Heisey, D.M. (2001). The abuse of power: the pervasive fallacy of power calculations for data analysis. The American Statistician, 55(1), 19–24.

    Article  Google Scholar 

  • Huertas, M.A., & Smith, G.D. (2006). A multivariate population density model of the dLGN/PGN relay. Journal of Computational Neuroscience, 21, 1573–6873. https://doi.org/10.1007/s10,827-006-7753-2. ISSN: 929–5313 (Paper), Online.

    Article  Google Scholar 

  • Hunsberger, E., Scott, M., & Eliasmith, C. (2014). The competing benefits of noise and heterogeneity in neural coding. Neural Computation, 26(8), 1600–1623.

    Article  PubMed  Google Scholar 

  • Josić, K., Shea-Brown, E., Doiron, B., & de la Rocha, J. (2009). Stimulus-dependent correlations and population codes. Neural Computation, 21, 2774–2804.

    Article  PubMed  Google Scholar 

  • Kay, S. (1993). Fundamentals of statistical signal processing, volume 1: Estimation theory. Prentice Hall PTR.

  • Knight, B.W. (1972). The relationship between the firing rate of a single neuron and the level of activity in a population of neurons. Experimental evidence for resonant enhancement in the population response. Journal of General Physiology, 59, 767–778.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Litwin-Kumar, A., Chacron, M., & Doiron, B. (2012). The spatial structure of stimuli shapes the timescale of correlations in population spiking activity. PLoS Computational Biology, 8(9), e1002,667.

    Article  CAS  Google Scholar 

  • Ly, C. (2015). Firing rate dynamics in recurrent spiking neural networks with intrinsic and network heterogeneity. Journal of Computational Neuroscience, 39, 311–327.

    Article  PubMed  Google Scholar 

  • Ly, C., & Ermentrout, B. (2010). Analysis of recurrent networks of pulse-coupled noisy neural oscillators. SIAM Journal on Applied Dynamical Systems, 9, 113–137.

    Article  Google Scholar 

  • Ly, C., & Tranchina, D. (2007). Critical analysis of dimension reduction by a moment closure method in a population density approach to neural network modeling. Neural Computation, 19, 2032–2092.

    Article  PubMed  Google Scholar 

  • Ly, C., Middleton, J., & Doiron, B. (2012). Cellular and circuit mechanisms maintain low spike co-variability and enhance population coding in somatosensory cortex. Frontiers in Computational Neuroscience, 6, 1–26. https://doi.org/10.3389/fncom.2012.00007.

    Article  Google Scholar 

  • Maler, L. (2007). Neural strategies for optimal processing of sensory signals. Progress in Brain Research, 165, 135–154.

    Article  PubMed  Google Scholar 

  • Maler, L. (2009). Receptive field organization across multiple electrosensory maps. i. columnar organization and estimation of receptive field size. Journal of Comparative Neurology, 516(5), 376–393.

    Article  PubMed  Google Scholar 

  • Maler, L., Sas, E., Johnston, S., & Ellis, W. (1991). An atlas of the brain of the electric fish apteronotus leptorhynchus. Journal of Chemical Neuroanatomy, 4(1), 1–38.

    Article  CAS  PubMed  Google Scholar 

  • Marder, E., & Goaillard, J. (2006). Variability, compensation and homeostasis in neuron and network function. Nature Reviews Neuroscience, 7, 563–574.

    Article  CAS  PubMed  Google Scholar 

  • Marsat, G., & Maler. (2012). Preparing for the unpredictable: adaptive feedback enhances the response to unexpected communication signals. Journal of Neurophysiology, 107(4), 1241–1246.

    Article  PubMed  Google Scholar 

  • Marsat, G., & Maler, L. (2010). Neural heterogeneity and efficient population codes for communication signals. Journal of Neurophysiology, 104, 2543–2555.

    Article  PubMed  Google Scholar 

  • Marsat, G., Longtin, A., & Maler, L. (2012). Cellular and circuit properties supporting different sensory coding strategies in electric fish and other systems. Current Opinion in Neurobiology, 22(4), 686–692.

    Article  CAS  PubMed  Google Scholar 

  • Marsat, G., Hupé, G.J., & Allen, K. (2014). Heterogeneous response properties in a population of sensory neurons are structured to efficiently code naturalistic stimuli. Neuroscience Meeting Planner Program # (181.20).

  • Mehaffey, W., Maler, L., & Turner, R. (2008). Intrinsic frequency tuning in ell pyramidal cells varies across electrosensory maps. Journal of Neurophysiology, 99(5), 2641–2655.

    Article  PubMed  Google Scholar 

  • Mejias, J., & Longtin, A. (2012). Optimal heterogeneity for coding in spiking neural networks. Physical Review Letters, 108, 228,102.

    Article  CAS  Google Scholar 

  • Mejias, J., & Longtin, A. (2014). Differential effects of excitatory and inhibitory heterogeneity on the gain and asynchronous state of sparse cortical networks. Frontiers in Computational Neuroscience, 8.

  • Mejias, J., Marsat, G., Bol, K., Maler, L., & Longtin, A. (2013). Learning contrast-invariant cancellation of redundant signals in neural systems. PLoS Computational Biology, 9(9), e1003,180.

    Article  CAS  Google Scholar 

  • Middleton, J., Longtin, A., Benda, J., & Maler, L. (2009). Postsynaptic receptive field size and spike threshold determine encoding of high-frequency information via sensitivity to synchronous presynaptic activity. Journal of Neurophysiology, 101(3), 1160–1170.

    Article  PubMed  Google Scholar 

  • Moreno-Bote, R., & Parga, N. (2006). Auto- and crosscorrelograms for the spike response of leaky integrate-and-fire neurons with slow synapses. Physical Review Letters, 96, 028,101.

    Article  Google Scholar 

  • Nesse, WH., Borisyuk, A., & Bressloff, P. (2008). Fluctuation-driven rhythmogenesis in an excitatory neuronal network with slow adaptation. Journal of Computational Neuroscience, 25, 317– 333.

    Article  PubMed  Google Scholar 

  • Nicola, W., Ly, C., & Campbell, S.A. (2015). One-dimensional population density approaches to recurrently coupled networks of neurons with noise. SIAM journal on applied mathematics (in press):–.

  • Noonan, L., Doiron, B., Laing, C., Longtin, A., & Turner, R. (2003). A dynamic dendritic refractory period regulates burst discharge in the electrosensory lobe of weakly electric fish. The Journal of Neuroscience, 23 (4), 1524–1534.

    CAS  PubMed  Google Scholar 

  • Nykamp, D., & Tranchina, D. (2000). A population density approach that facilitates large-scale modeling of neural networks: analysis and an application to orientation tuning. Journal of Computational Neuroscience, 8, 19–50.

    Article  CAS  PubMed  Google Scholar 

  • Omurtag, A., Knight, B.W., & Sirovich, L. (2000). On the simulation of large populations of neurons. Journal of Computational Neuroscience, 8, 51–63.

    Article  CAS  PubMed  Google Scholar 

  • Oswald, A., Doiron, B., Rinzel, J., & Reyes, A. (2009). Spatial profile and differential recruitment of gabab modulate oscillatory activity in auditory cortex. The Journal of Neuroscience, 29, 10,321–10,334.

    Article  CAS  Google Scholar 

  • Padmanabhan, K., & Urban, N. (2010). Intrinsic biophysical diversity decorrelates neuronal firing while increasing information content. Nature Neuroscience, 13, 1276–1282.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Parker, D. (2003). Variable properties in a single class of excitatory spinal synapse. The Journal of Neuroscience, 23(8), 3154–3163.

    CAS  PubMed  Google Scholar 

  • Pouille, F., & Scanziani, M. (2001). Enforcement of temporal fidelity in pyramidal cells by somatic feed-forward inhibition. Science, 293(5532), 1159–1163.

    Article  CAS  PubMed  Google Scholar 

  • Priebe, N., & Ferster, D. (2008). Inhibition, spike threshold, and stimulus selectivity in primary visual cortex. Neuron, 57(4), 482–497.

    Article  CAS  PubMed  Google Scholar 

  • Richardson, M. (2007). Firing-rate response of linear and nonlinear integrate-and-fire neurons to modulated current-based and conductance-based synaptic drive. Physical Review E, 76, 021,919.

    Article  Google Scholar 

  • Richardson, M. (2008). Spike-train spectra and network response functions for non-linear integrate-and-fire neurons. Biological Cybernetics, 99, 381–392.

    Article  PubMed  Google Scholar 

  • Saunders, J., & Bastian, J. (1984). The physiology and morphology of two types of electrosensory neurons in the weakly electric fish apteronotus leptorhynchus. Journal of Comparative Physiology A, 154(2), 199–209.

    Article  Google Scholar 

  • Shamir, M., & Sompolinsky, H. (2006). Implications of neuronal diversity on population coding. Neural Computation, 18, 1951–1986.

    Article  PubMed  Google Scholar 

  • Simmonds, B., & Chacron, M.J. (2015). Activation of parallel fiber feedback by spatially diffuse stimuli reduces signal and noise correlations via independent mechanisms in a cerebellum-like structure. PLOS Computational Biology, 11(1), e1004,034.

    Article  Google Scholar 

  • Thomas, L. (1997). Retrospective power analysis. Conservation Biology, 11(1), 276–280.

    Article  Google Scholar 

  • Tranchina, D. (2009). Population density methods in large-scale neural network modelling. In Laing, C, and Lord, G (Eds.) Stochastic Methods in Neuroscience, Chap 7: Oxford University Press.

  • Tripathy, S., Padmanabhan, K., Gerkin, R., & Urban, N. (2013). Intermediate intrinsic diversity enhances neural population coding. Proceedings of the National Academy of Sciences, 110, 8248–8253.

    Article  CAS  Google Scholar 

  • Wilbur, W., & Rinzel, J. (1982). An analysis of Stein’s model for stochastic neural excitation. Biological Cybernetics, 45, 107– 114.

    Article  CAS  PubMed  Google Scholar 

  • Xue, M., Atallah, B.V., & Scanziani, M. (2014). Equalizing excitation-inhibition ratios across visual cortical neurons. Nature, 511, 596–600.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank David J. Edwards (VCU) for help regarding Power analysis of our data. This work was supported by a grant from the Simons Foundation (#355173, Cheng Ly), and by a grant from the National Science Foundation (NSF-ISO #1557846, Gary Marsat).

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Cheng Ly or Gary Marsat.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Action Editor: Nicolas Brunel

Cheng Ly and Gary Marsat contributed equally to this work.

Appendices

Appendix A: Asymptotic calculation for the relative standard deviation of firing rate distribution

The asymptotic calculations were all based on an expression for the firing rate of an individual neuron via the PDF or Population Density framework that has been commonly used in spiking models in cortex models (Knight 1972; Wilbur and Rinzel 1982; Fourcaud and Brunel 2002; Brunel and Latham 2003; Tranchina 2009) and other areas (Barna et al., 1998; Brown et al., 2004; Huertas and Smith 2006). In addition to the firing rate, this framework has been useful in calculating many statistical quantities of the spike train (Brunel et al., 2001; Richardson 2007; 2008) and to study the stability of coupled networks (Knight 1972; Abbott and van Vreeswijk 1993; Brunel and Hakim 1999; Gerstner 2000; Ly and Ermentrout 2010). It can also be employed as a time saving computational tool (Nykamp and Tranchina 2000; Omurtag et al., 2000; Apfaltrer et al., 2006; Ly and Tranchina 2007). We focus on the standard deviation of firing rates and use the framework to gain analytic insight into the dynamics.

To employ the framework and for feasibility, we make some technical assumptions:

  1. (i)

    the (average) population firing rate is a good approximation to the presynaptic input rate with random connectivity

  2. (ii)

    a single p.d.f. function is adequate to describe a single population’s activity

  3. (iii)

    the heterogeneity is driven by (q j ,𝜃 j ) only

The complexities that arise in a recurrent network (nonlinear PDF equation) never came about (Ly 2015) in our specific analysis because of the networks here are delayed feedforward networks. Moreover, the resulting PDF equations have lower dimensions than a recurrent network. We begin by writing down the probability density function for both (E and I) presynaptic populations that provide delayed feedforward input.

$$\begin{array}{@{}rcl@{}} &&\rho^{E}(v,\eta,t) \,dv d\eta\\ &&\quad\quad\,\,= \Pr\left( \text{presyn E cell}\in\left\{(v,v+dv)\cap(\eta,\eta+d\eta)\right\} \right) \end{array} $$
$$\begin{array}{@{}rcl@{}} &&\rho^{I}(v,\eta,t) \,dv d\eta\\ &&\quad\quad\quad= \Pr\left( \text{presyn I cell}\in\left\{(v,v+dv) \cap(\eta,\eta+d\eta)\right\} \right) \end{array} $$

The PDF equations for the presynaptic E population are:

$$\begin{array}{@{}rcl@{}} \frac{\partial \rho^{E}}{\partial t} &=& -\frac{\partial}{\partial v}\left\{ J_{V}(v,\eta,t) \right\} -\frac{\partial}{\partial \eta}\left\{J_{\eta}(v,\eta,t) \right\} \\ J_{V}(v,\eta,t) &=& \frac{1}{\tau_{m}} \left[ I_{aff}(t)-v + \sigma_{F} \eta \right]\rho^{E} \end{array} $$
(8)
$$\begin{array}{@{}rcl@{}} J_{\eta}(v,\eta,t) &=& \frac{1}{\tau_{n}}\left[-\eta\rho^{E} - \frac{1}{2}\frac{\partial\rho^{E}}{\partial \eta} \right] \end{array} $$
(9)
$$\begin{array}{@{}rcl@{}} J_{V}(1,\eta,t) &=&J_{V}(0,\eta,t+\tau_{fref}) \end{array} $$
(10)
$$\begin{array}{@{}rcl@{}} r^{E}(t) &=& {\int}_{-\infty}^{\infty} J_{V}(1,\eta,t)\,d\eta \end{array} $$
(11)

where r E(t) is the population firing rate. The presynaptic I population is similar but lacks direct sinusoidal input:

$$\begin{array}{@{}rcl@{}} \frac{\partial \rho^{I}}{\partial t} &=& -\frac{\partial}{\partial v}\left\{ J_{V}(v,\eta,t) \right\} -\frac{\partial}{\partial \eta}\left\{J_{\eta}(v,\eta,t) \right\} \\ J_{V}(v,\eta,t) &=& \frac{1}{\tau_{m}} \left[ -v - g_{EE}(t)(v-\mathcal{E}_{E}) + \sigma_{F} \eta \right]\rho^{I} \end{array} $$
(12)
$$\begin{array}{@{}rcl@{}} J_{\eta}(v,\eta,t) &=& \frac{1}{\tau_{n}}\left[-\eta\rho^{I} - \frac{1}{2}\frac{\partial\rho^{I}}{\partial \eta} \right] \end{array} $$
(13)
$$\begin{array}{@{}rcl@{}} J_{V}(1,\eta,t) &=& J_{V}(0,\eta,t+\tau_{fref}) \end{array} $$
(14)
$$\begin{array}{@{}rcl@{}} r^{I}(t) &=& {\int}_{-\infty}^{\infty} J_{V}(1,\eta,t)\,d\eta \end{array} $$
(15)

For simplicity, we assume the synaptic conductances of the target (superficial pyramidal) cells can be averaged in time (see Eq. (2)):

$$g_{EE}(t) = s_{EE} \int r^{E}(t-t') K(t') \,dt' $$
$$g_{e}(t) = s_{e} \int r^{E}(t-t') K(t') \,dt' $$
$$g_{i}(t) = s_{i} \int r^{I}(t-t') K(t') \,dt' $$

where K is the alpha function kernel:

$$K(t) = H(t)\frac{\alpha}{\frac{\tau_{d}}{\tau_{r}}-1}\left[ e^{-t/\tau_{d}} - e^{-t/\tau_{r}} \right]$$

and H(t) is the Heaviside step function and τ r < τ d (a common assumption in models of synapses). This kernel is unconventional in that \({\int }_{-\infty }^{\infty } K(t)\,dt=\alpha \tau _{r}\) and ≠ 1.

Finally, the PDF for the target cells:

$$\begin{array}{@{}rcl@{}} &&\rho(v,\eta,t) \,dv d\eta\\ && \quad\quad= \Pr\left( \text{pyramidal cell}\in\{(v,v+dv)\cap(\eta,\eta+d\eta)\} \right) \end{array} $$

are described by:

$$\begin{array}{@{}rcl@{}} \frac{\partial \rho}{\partial t} &=& -\frac{\partial}{\partial v}\left\{ J_{V}(v,\eta,t) \right\} -\frac{\partial}{\partial \eta}\left\{J_{\eta}(v,\eta,t) \right\} \\ J_{V}(v,\eta,t) &=& \frac{1}{\tau_{m}} \left[ I_{aff}(t)-v - q_{j} g_{e}(t-\tau_{del})(v-\mathcal{E}_{E})\right.\\ &&\left.-q_{j} g_{i}(t-\tau_{del})(v-\mathcal{E}_{I}) + \sigma_{P} \eta \right]\rho \end{array} $$
(16)
$$\begin{array}{@{}rcl@{}} J_{\eta}(v,\eta,t) &=&\frac{1}{\tau_{n}}\left[-\eta\rho - \frac{1}{2} \frac{\partial\rho}{\partial \eta} \right] \end{array} $$
(17)
$$\begin{array}{@{}rcl@{}} J_{V}(1,\eta,t) &=& J_{V}(0,\eta,t+\tau_{ref}) \end{array} $$
(18)
$$\begin{array}{@{}rcl@{}} r_{j}(t) &=& {\int}_{-\infty}^{\infty} J_{V}(\theta_{j},\eta,t)\,d\eta. \end{array} $$
(19)

The firing rate r j (t) is not a population firing rate, but rather the firing rate of the j th neuron in the population. We implicitly assume that the only difference between cells is given by the two heterogeneous parameters: (𝜃 j ,q j ).

Our goal is not to capture the time-varying firing rates (which are still difficult even with these assumptions because of the three coupled PDEs that each have 2 spatial dimensions and time), but rather we are interested in the trial- and time-averaged firing rates. This enables a compact expression for how the heterogeneity (𝜃,q) relationship controls the heterogeneity of steady-state firing rates. We have:

$$ \nu_{j} := \langle r_{j}(t) \rangle_{t} $$
(20)

We approximate

$$ \left\langle {\int}_{-\infty}^{\infty} I_{aff}(t)\rho(v,\eta,t)\,d\eta \right\rangle_{t} \approx {\int}_{-\infty}^{\infty} \langle I_{aff}(t) \rangle_{t} \langle \rho(v,\eta,t)\rangle_{t} \,d\eta, $$
(21)

and similarly for the expressions with g e/i . This leads to:

$$\begin{array}{@{}rcl@{}} \nu_{j} &\approx& \frac{1}{\tau_{m}} \left[ I_{0} - \theta_{j} + \bar{g}_{E} (\mathcal{E}_{E}-\theta_{j}) - \bar{g}_{I} (\theta_{j}-\mathcal{E}_{I}) \right] f(\theta_{j})\\ &&+ \frac{\sigma_{P}}{\tau_{m}} {\int}_{-\infty}^{\infty} \eta \rho(\theta_{j},\eta)\,d\eta \end{array} $$
(22)

where \(f(v)=\int \rho (v,\eta )\,d\eta \) is the steady-state marginal voltage distribution (equal to the time-average, assuming ergodic theorems apply), and \(\bar {g}_{E}=\alpha \tau _{r} \langle r^{E}(t)\rangle _{t}, \bar {g}_{I}=\alpha \tau _{r} \langle r^{I}(t)\rangle _{t}\). One could simply numerically simulate these equations, but there is not much analytic insight gained in understanding how (𝜃,q) and ρ(𝜃,q) alter the firing rate standard deviation. In applying dimension reduction methods, there are issues that arise in trying to accurately capture the firing rate (Ly and Tranchina 2007). Thus, we apply a simple (quantitatively inaccurate) dimension reduction method where we assume η is frozen and average over the resulting firing rate (Moreno-Bote and Parga 2006; Nesse et al., 2008; Hertäg et al., 2014; Nicola et al., 2015; Ly 2015). We also ignore the effects of the refractory period τ r e f .Footnote 3 The firing rate is then simply:

$$\begin{array}{@{}rcl@{}} \nu_{j}(\theta_{j},q_{j}) &\approx& {\int}_{-\infty}^{\infty} \nu_{det}(\theta_{j},q_{j}; \eta) \frac{e^{-\eta^{2}}}{\sqrt{\pi}} \,d\eta \end{array} $$
(23)
$$\begin{array}{@{}rcl@{}} \nu_{det}(\theta_{j},q_{j}; \eta) &=& \left\{\begin{array}{lll} 0, & \text{if} \frac{q(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I})+\sigma_{P}\eta+I_{0}}{1+q(\bar{g}_{E}+\bar{g}_{I})} \leq \theta_{j} \\ \frac{1+q(\bar{g}_{E}+\bar{g}_{I})}{\tau_{m} \log\left( \frac{q(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I})+\sigma_{P}\eta+I_{0}} {q(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I})+\sigma_{P}\eta+I_{0}-\theta(1+q(\bar{g}_{E}+\bar{g}_{I}))}\right)}, & \text{if} \frac{q(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I})+\sigma_{P}\eta+I_{0}}{1+q(\bar{g}_{E}+\bar{g}_{I})} > \theta_{j} \end{array}\right. \end{array} $$
(24)

The parameters (𝜃 j ,q j ) determine how one ν j differs from another; to see how the combined effects of threshold heterogeneity and synaptic variability alter ν j , we consider a specific limit. That is, the simulations indicate that thethe net conductance are large in the fitted model, thus, we consider the large firing rate limit of the term in the integrand ν d e t , to get:

$$\begin{array}{@{}rcl@{}} \tau_{m} \nu_{det}(\theta_{j},q_{j}) &=&\frac{1+q_{j}(\bar{g}_{E}+\bar{g}_{I})}{\log\left( \frac{q_{j}(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I})+\sigma_{p}\eta+I_{0}} {q_{j}(\bar{g}_{E}\mathcal{E}+\bar{g}_{I}\mathcal{E}_{I}) +\sigma_{p}\eta+I_{0} - \theta_{j}(1+q_{j}(\bar{g}_{E}+\bar{g}_{I}))} \right)} \end{array} $$
(25)
$$\begin{array}{@{}rcl@{}} &=& \frac{q_{j}}{\theta_{j}}(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I})+\frac{\sigma_{p}\eta+I_{0}}{\theta_{j}}-\frac{1}{2}(1+q_{j}(\bar{g}_{E}+\bar{g}_{I}))\\ & &-\frac{(1+q_{j}(\bar{g}_{E}+\bar{g}_{I}))^{2}\theta_{j}}{12[q_{j}(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I})+\sigma_{p}\eta+I_{0}-\theta_{j}(1+q_{j}(\bar{g}_{E}+\bar{g}_{I}))]} \\ & &+O\left( z^{2}(1+q_{j}(\bar{g}_{E}+\bar{g}_{I}))\right) \end{array} $$
(26)
$$\begin{array}{@{}rcl@{}} \text{where} z &:=& \theta_{j} \frac{1+q_{j}(\bar{g}_{E}+\bar{g}_{I})}{q_{j}(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I})+\sigma_{p}\eta+I_{0}-\theta_{j}(1+q_{j}(\bar{g}_{E}+\bar{g}_{I}))} \end{array} $$
(27)

This calculation is very similar to the one in Ly (2015). The key term is the first term in Eq. (26),

$$\frac{q_{j}}{\theta_{j}}(\bar{g}_{E}\mathcal{E}_{E}+\bar{g}_{I}\mathcal{E}_{I}) $$

which is the dominant term assuming ν d e t is large. Substituting the expansion (26) into the integral approximation(23) only changes terms with η in them (i.e., the dominant term does not change and the term σ P η/𝜃 j evaluates to 0). This shows analytically that the term q j /𝜃 j is the dominant source of firing rate heterogeneity, and that we can approximate:

$$ \sigma\left( \boldsymbol{\nu}\right) \approx C \sqrt{ \frac{1}{N-1} \sum\limits_{j = 1}^{N} \left( \frac{q_{j}}{\theta_{j}} - \mu\left( \frac{\boldsymbol{q}}{\boldsymbol{\theta}}\right) \right)^{2}} $$
(28)

where \(\mu \left (\frac {\boldsymbol {q}}{\boldsymbol {\theta }}\right )=\frac {1}{N}\displaystyle \sum\limits_{j = 1}^{N} \frac {q_{j}}{\theta _{j}}\).

Appendix B: Details of network connectivity and model in Section 3.3

We used two networks, which we generically labeled as Network 1 and Network 2 (see Fig. 4), to help demonstrate the utility of the theory for firing rate standard deviations with different architectures than random (Erdős-Rényi graph). We first describe how we selectively activate the granule cells that provide delayed feedforward input to the pyramidal cells, and then describe the connectivity rules for each network.

Selective activation of granule cells

Instead of providing constant sinusoidal drive to each of the 2N f presynaptic granule cells, the afferent stimuli \(I_{aff}(t)=I_{0}+\mathcal {A}\sin (2\pi \phi t)\) is scaled by a parameter C(l,ϕ):

$$I_{aff}(t) = C(l,\phi) \left\lceil I_{0} + \mathcal{A}\sin(2\pi\phi t) \right\rceil^{+}$$

that depends on both frequency ϕ and the index of cell l ∈{1, 2,…,N f } (2N f total because there are both excitatory and inhibitory presynaptic cells). The afferent stimuli to the target pyramidal cells is the same as before (see Eq. (1)). Before providing the formula for C(l,ϕ), we note that the strength of the sinusoidal drive will follow a (scaled) beta distribution where the location of the maximum value increases as ϕ increases. We use:

$$ C(l,\phi) = 1.5\frac{x(l)^{19}(1-x(l))^{21.52*\frac{125}{\phi} -21}}{\max_{1\leq l \leq N_{f}} \beta(l,\phi)} $$
(29)

where

$$\begin{array}{@{}rcl@{}} \beta(l,\phi) &=& x(l)^{19}(1-x(l))^{21.52*\frac{125}{\phi} -21} \\ x(l) &=& \frac{2.5\,\text{Hz}}{125\,\text{Hz}}(\lceil l/4 \rceil - 1/2) \end{array} $$

The term β(l,ϕ) is simply the numerator of the fraction in C(l,ϕ) so that C(l,ϕ) ∈ (0, 1.5). The variable x(l) is the end result of mapping the l th presynaptic neuron to one of 50 frequencies (equally spaced by 2.5 Hz from 1.25 Hz to 123.75 Hz) and normalizing by 125 Hz. Here we are assuming ϕ ∈ [0, 125] Hz so that x(l) ∈ (0, 1); other frequencies can easily be incorporated with minor adjustments to the above formulas. Finally, notice that in x(l), we have the term: ⌈l/4⌉, which denotes rounding up after dividing by 4; this essentially groups presynaptic neurons into groups of size 4 that receive the same sinusoidal drive. Figure 5A illustrates how the sinusoidal drive to the presynaptic cells, indexed by l, vary with several fixed frequencies ϕ.

Connectivity rules for Network 1

To illustrate the usefulness of the theory, we implemented a static delayed feedforward network with fixed (𝜃,q) and certain connectivity rules (see below). The presynaptic granule cells are indexed as before via l, and the target pyramidal cells by j. Recall that both E and I cells in the presynaptic population have the same connectivity for simplicity.

Each pyramidal cell j has an associated (𝜃 j ,q j ) (Fig. 4B,C), and since each presynaptic l cell’s sinusoidal drive depends on frequency, the probability of connection is specified so that the effective ρ(𝜃,q) results in firing rate heterogeneity consistent with the data (i.e., ρ > 0 for low frequencies and ρ < 0 for high frequencies). The connection probability is closely related to Fig. 4B; that is: Low frequencies activate a subset of presynaptic cells, the probability that those presynaptic cells are connected to target cells with (𝜃j, q j ) in a region that gives ρ > 0 is high (Fig.  4 B, red); the probability is lower with the other target cells (blue). Similar connection probability rules apply for high frequencies. In both networks we considered, the connectivity scheme assumes that as the afferent sinusoidal frequency increases, ρ decreases monotonically. Again, this is a questionable assumption that we have made only to provide a proof of principle for how our theory can be used. In Network 1, different effective ρ values are obtained by lines with slopes proportional to ρ. Of course, there are an infinite number of ways to arrive at a desired ρ value. The probability of a connection is:

$$\begin{array}{@{}rcl@{}} P(l\text{ is connected to} j) &=& e^{-100 d(l,j)} \\ \text{where} d(l,j) &=& \min_{(\theta_{0},q_{0})\in\mathbb{B}(l)} \sqrt{ (\theta_{j}-\theta_{0})^{2}+(q_{j}-q_{0})^{2}} \\ \mathbb{B}(l) &=& \left\{ (\theta,q) \vert m_{l}(q-\bar{q})-0.1 < \theta-\bar{\theta} < m_{l}(q-\bar{q})+ 0.1 \right\} \\ m_{l} = \left( 1-2x(l) \right)\frac{\max_{j} \theta_{j} - \min_{j} \theta_{j}}{\max_{j} q_{j} - \min_{j} q_{j}}; & \hspace{.2in} &\bar{q}=\frac{1}{2}\left( \max_{j} q_{j} + \min_{j} q_{j} \right); \hspace{.2in} \bar{\theta}=\frac{1}{2}\left( \max_{j} \theta_{j} + \min_{j} \theta_{j} \right); \end{array} $$

x(l) ∈ (0, 1) was defined above (i.e., scaled frequency that drives l well). The function d(l,j) is the Euclidean distance in (𝜃,q) space to the band \(\mathbb {B}(l)\), which consists of (𝜃,q) values that give the desired ρ. Note that the slope of the band m l goes from positive to negative as l increases.

Connectivity rules for Network 2

The rules in this network are similar in spirit to Network 1 but the region that gives an effective ρ value is no longer rectangular but rather a wedge (compare Fig. 4B and C). The probability of a connection is:

$$\begin{array}{@{}rcl@{}} P(l\text{is connected to} j) &=& e^{-100 d(l,j)} \\ \text{where} d(l,j) &=& \min_{(\theta_{0},q_{0})\in\mathbb{W}(l)} \sqrt{ (\theta_{j}-\theta_{0})^{2}+(q_{j}-q_{0})^{2}} \\ \mathbb{W}(l) &=& \mathbb{W}_{1}(l) \cup \mathbb{W}_{2}(l) \\ \mathbb{W}_{1}(l) &=& \left\{ (\theta,q) \left\vert (n_{l} - 0.4)(q-\bar{q}) < \theta-\bar{\theta} < (n_{l}+ 0.4)(q-\bar{q})\right. \right\} \\ \mathbb{W}_{2}(l) &=& \left\{(\theta,q) \left\vert (n_{l} + 0.4)(q-\bar{q}) < \theta-\bar{\theta} < (n_{l} - 0.4)(q-\bar{q})\right. \right\} \\ n_{l} = \left( 0.8-1.6x(l) \right)\frac{\max_{j} \theta_{j} - \min_{j} \theta_{j}}{\max_{j} q_{j} - \min_{j} q_{j}}; & \hspace{.2in} &\bar{q}=\frac{1}{2}\left( \max_{j} q_{j} + \min_{j} q_{j} \right); \hspace{.2in} \bar{\theta}=\frac{1}{2}\left( \max_{j} \theta_{j} + \min_{j} \theta_{j} \right); \end{array} $$

x(l) ∈ (0, 1) has the same definition as before, but the function d(l,j) is now the Euclidean distance in (𝜃,q) space to the region \(\mathbb {W}(l)\) which are 2 triangular wedges (Fig. 4C).

The resulting number of connections for both Network 1, 2 are not random but rather has structure (Fig. 5B).

Appendix C: More data and model figures

Here we provide supplemental figures for completeness; we chose not to include them in the main manuscript for exposition purposes.

Figure 6 shows the entire experimental data set used in this paper. Figure 6A shows the 5 Hz sinewave stimulus (blue), and the PSTH averaged over all 15 superficial/intermediate pyramidal cells in the intact network (black) and with parallel fiber input blocked (red). Parts B and C show the individual cell PSTH (cycle histogram) for both intact (black) and blocked (red) conditions with solid lines; the raster plots for all trials are shown in the background (same format as Fig. 1B).

Fig. 6
figure 6

Responses of the 15 pyramidal cells in our data set. Parallel fiber input was intact (black) or blocked (red). A One second excerpt of the responses to the 5 Hz AM stimulus (blue). Responses were converted to instantaneous firing rates (i.e. convolved with a 10 ms Gaussian) and averaged across the 15 cells of our data set. B PSTH (cycle histogram) or responses of each cell to 5 Hz stimuli. Raster plots of spiking for each cycle are shown in the background. C Same as B but with high frequency input (120 Hz)

Figure 7 shows the firing rate of the entire population of N = 1000 (target) cells as a function of the chosen thresholds and synaptic variability parameters, for both low and high frequency stimuli. Each panel contains relevant correlation values ρ(q,𝜃).

Fig. 7
figure 7

Resulting firing rate as a function of heterogeneous parameter values. A Relationship between firing rate and threshold 𝜃 for all 1000 LIF neurons with representative correlation ρ(q,𝜃) ∈{− 0.2,0,0.9} values for ϕ = 5 Hz. B Relationship between firing rate and q for all 1000 LIF neurons with representative correlation ρ values for ϕ = 5 Hz. C and D similar to A and B but with ϕ = 120 Hz input frequency. Although the firing rate is strongly related to the threshold, the synaptic variability has a weaker although positive correlation with the resulting firing rate

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ly, C., Marsat, G. Variable synaptic strengths controls the firing rate distribution in feedforward neural networks. J Comput Neurosci 44, 75–95 (2018). https://doi.org/10.1007/s10827-017-0670-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10827-017-0670-8

Keywords

Navigation