Abstract
Implicit expectations induced by predictable stimuli sequences affect neuronal response to upcoming stimuli at both single cell and neural population levels. Temporally regular sensory streams also phase entrain ongoing low frequency brain oscillations but how and why this happens is unknown. Here we investigate how random recurrent neural networks without plasticity respond to stimuli streams containing oddballs. We found the neuronal correlates of sensory stream adaptation emerge if networks generate chaotic oscillations which can be phase entrained by stimulus streams. The resultant activity patterns are close to critical and support history dependent response on long timescales. Because critical network entrainment is a slow process stimulus response adapts gradually over multiple repetitions. Repeated stimuli generate suppressed responses but oddball responses are large and distinct. Oscillatory mismatch responses persist in population activity for long periods after stimulus offset while individual cell mismatch responses are strongly phasic. These effects are weakened in temporally irregular sensory streams. Thus we show that network phase entrainment provides a biologically plausible mechanism for neural oddball detection. Our results do not depend on specific network characteristics, are consistent with experimental studies and may be relevant for multiple pathologies demonstrating altered mismatch processing such as schizophrenia and depression.
Similar content being viewed by others
Introduction
In predictably structured sensory streams like music, speech and visual input during motion the brain doesn’t simply respond to external events but anticipates them1,2,3,4. Studies of temporal expectation, also known as implicit timing5, demonstrate the importance of predictability of when stimuli will occur. Task performance is enhanced if stimuli occur at expected times according to an established rhythm6,7,8. The sensory streams used are also found to phase entrain low-frequency brain oscillations9,10,11,12. Phase entrainment increases with the temporal regularity of the sensory stream and correlates with behavioural performance. It is not limited to sensory cortices13; activity patterns in larger scale cortical14,15 and subcortical networks16 are also modulated by temporal expectation.
Related studies show that implicit expectations of upcoming stimulus itself can also strongly affect neural response. Stimulus specific adapatation (SSA) occurs in single cells17 and in multi-unit activity18. Here the response to a stimulus is in general suppressed after multiple presentations while responses to oddball stimuli may be enhanced19. SSA occurs in both excitatory and inhibitory cells20,21 and in multiple areas from midbrain to cortex22. Type expectation also occurs at the neuronal population level as oscillatory mismatch responses (MMR) after stimulus offset in EEG/MEG23,24,25,26,27, ECoG19, and LFP28. Most importantly adaptation can occur to whole stimulus sequences on multiple time scales simultaneously18,29 while the effects are stronger when streaming is temporally regular30. Mismatch responses are altered in pathologies, notably schizophrenia, traumatic brain injury and Alzheimers disease, and can be used as diagnostic tools31,32,33,34,35.
Whether population and single cell modulations originate from the same underlying mechanism is controversial36,37,38,39. SSA is thought to depend on single cell adaptation but studies have also implicated network mechanisms21,40,41 including recurrent inhibition42,43. MMR has been suggested to involve synaptic adaptation44 as well as memory45 and predictive coding46 based mechanisms. Intriguing recent investigations have highlighted a possible interaction of SSA and MMR through recurrent network inhibition20,47.
Neural information processing may be embedded in recurrent network dynamics48,49,50,51,52,53 which are also thought to underlie ongoing brain oscillations54,55,56. Recent modeling work57 has demonstrated entrainment of low frequency stable network oscillations by transcranial magnetic stimulation. However low frequency brain oscillations may also be chaotic58,59,60,61,62. We find that the neuronal correlates of sensory stream adaptation emerge when unstructured recurrent networks generate fairly low dimensional chaotic dynamics. We show that this is because entrainment of chaotic oscillations63 results in the complex close to critical dynamics64,65 which supports the history dependent response on long behaviourally relevant timescales necessary for sensory stream adaptation66,67.
Results
Sensory stream adaptation is a general phenonemon observed across multiple brain regions on multiple timescales. Therefore to demonstrate the hypothesis in the simplest way possible we use numerical simulations of a very standard random network of firing rate units without any synaptic plasticity or facilitation mechanisms (see Methods.) We are primarily interested in low frequency oscillations 1 ~ 10 Hz in population measures such as LFP and EEG. We therefore use a model derived from neurotransmitter fluctuations which average over spikes. The model of low frequency oscillations used in57 also employs adapting membrane potentials. Although highly simplified, rate networks can provide useful insight into elementary principles of real brain network dynamics60,68,69,78,79,80 where an individual rate unit reflects the activity of a local circuit including multiple cells of several types when spiking is irregular and asynchronous80,81.
The activity of each unit in the network depends on its net synaptic input transformed by the transfer function of a type I cell. The precise values of the model parameters, as well as the form of the transfer function itself, are not crucial. The important point is that the network should generate fairly low dimensional chaotic dynamics. Both purely inhibitory and excitatory-inhibitory rate networks60,68 show stable and chaotic dynamical regimes as network parameters, for example the connection probability, are varied. Such weakly chaotic dynamics occurs in fairly sparsely connected networks, (connection probability ~0.15) and in the close to balanced but inhibition dominated regime in excitatory-inhibitory networks. Despite the absence of timescale parameters exceeding 50 ms such networks can generate large slow coherent oscillatory population fluctuations (Fig. 1(A,B)) on the extended behaviourally relevant timescales used in the sensory streaming tasks we investigate here.
Illustration of chaos stabilization by sensory streaming in a particular inhibitory network simulation. (A,C) Mean neuroelectric activity averaged across all 500 units in (A) the autonomous network simulation when continuously driven by the background input X, and (C) the network driven by the AB sensory stream where the ITI following A is always 415 ms and the ITI following B is always 1185 ms. (B,D) Firing rate for several individual units from (B) the autonomous simulation in (A), and (D) the sensory stream driven simulation in (C). (C,D) Ticks indicate the 50 ms sensory stimulus presentations, A (green) and B (blue) in the sensory stream. During ITI periods between stimulus presentations the network is driven by the same background input X as in (A,B). (A–D) All simulations are deterministic.
Here we focus mainly on inhibitory networks however the capacity for more general larger excitatory-inhibitory networks to show similar behaviour is also demonstrated. Indeed several recent studies have highlighted network inhibition as crucial in deviance detection and its interaction with slow oscillations20,21,47. Inhibition is important in sensory network dynamics82 and may modulate the interaction of internal dynamics and sensory input by controlling network stability59. As a simple proxy for the LFP/EEG we use the total absolute size of the average input current to a rate unit57,70,83,84, which we term the ‘neuroelectric activity’57.
The sensory streams we investigate are made up of several stimuli applied in sequence. A stimulus represents a static visual noise patch or an auditory stimulus composed of random frequencies, for example. Each stimulus is modeled as a fixed set of excitatory driving inputs (see Methods) one of which is applied to each of the units in the network for the duration of the stimulus. The driving inputs which make up any given stimulus are random, drawn independently from a single given distribution once and then fixed for the duration of the simulation. Therefore all stimuli are identical statistically (see Methods).
Most of the sensory streams we investigate include only two sensory stimuli, denoted A and B. They are always applied to the network for brief 50 ms periods, as is common in sensory streaming experiments. During inter-trial-intervals (ITIs) between stimuli the network is driven by a different input we denote ‘background’ input X. Input X can represent external sensory input during ITI periods or alternatively purely internally generated excitatory driving or a combination of the two. In order to demonstrate that our results do not require that sensory stimuli provide stronger levels of excitation than the background input we also draw input X from the same distribution as the sensory stimuli A and B. The ITIs in a sensory stream can be of fixed or variable length but the mean length investigated is usually 800 ms. This generates sensory streams with stimulus presentation frequencies typically used in experiments.
Simulations (except those for demonstration of dynamics and for calculations of Lyapunov exponents (see below)) include noise. The noise is not necessary for the results here. It has been included to show that our results are resilient to its presence (see Methods). In the first part of this paper we illustrate how chaos entrainment works to generate reliable, distinct and history dependent stimulus response in this model. In the second part we show how these properties can explain several experimental results in sensory stream adaptation.
Sensory streams entrain chaotic network activity
First we illustrate how sensory streams entrain chaos in a particular exemplar 500 unit inhibitory network simulation. Figure 1(A,B) show the spontaneous activity generated by the network when it is constantly driven by the background input X. (In the following we refer to the activity endogenously generated by the network under the constant driving of the background input X as the ‘autonomous’ activity.) The random looking oscillations in the mean neuroelectric activity (Fig. 1(A)) and in the individual units activity (Fig. 1(B)) suggest that the autonomous activity is chaotic.
We use principal component analysis to visualize the network chaotic attractor. The grey line in Fig. 2(A) shows the trajectory of the autonomous activity projected in the space of its first two principal components. One particular trajectory is shown. However there are multiple other trajectories winding around the attractor, tangled up with the one shown. Which trajectory the system follows depends on the choice of initial condition. Each trajectory gradually fills the attractor as time progresses but trajectories also diverge from each other on shorter timescales. Due to this mixing property initially nearby points eventually become spread across the whole attractor.
Comparison of principal component projections for the network simulation studied in Fig. 1 under different sensory streams composed of two 50 ms stimuli, A and B. (A) A and B alternate, ITI following A 415 ms. ITI following B 1185 ms. (B) A and B alternate, ITI following A 500 ms. ITI following B 1100 ms. (C) A and B alternate, ITI following A 521 ms. ITI following B 1079 ms. (D) AAABB sensory stream with all ITIs fixed at 800 ms. (E) A and B sequence is scrambled. ITIs are drawn from an exponential distribution with mean 800 ms. (F) A and B alternate. ITIs are composed of a fixed 700 ms plus an exponentially distributed period of mean 100 ms. (A–F) Coloured line segments show the trajectory of the sensory stream driven activity projected onto the first two principal components of the autonomous activity. Segments are coloured according to the epoch within the sensory stream. Lines show trajectory segments, symbols show trajectory segment endpoints (see key.) In (C) for clarity only four trajectory segments are shown, although all trajectory segment endpoints are shown. In (D) stimulus epoch endpoints are labelled by their order in the AAABB stream. (E) Only segment endpoints are shown. (A–E) Grey line shows the trajectory of the first two principal components of the autonomous activity. All simulations are deterministic of length 150 secs. Driven Lyapunov exponents λ D are (A) −0.001, (B) −0.0001 (C) 0.0002, (D) −0.001, (E) −0.0006 (F) −0.0003.
This trajectory divergence is quantified by the maximal Lyapunov exponent. It measures the rate at which nearby dynamical trajectories diverge or converge. When the maximal Lyapunov exponent is positive network activity is chaotic and unstable to small perturbations, like noise, which can knock the system from one trajectory onto another. On the other hand in networks with stable dynamical attractors, like limit cycles and fixed points, small perturbations to trajectories decay and the maximal Lyapunov exponent is negative or zero. In this network simulation example (Figs 1(A) and 2(A, grey line)) the maximal Lyapunov exponent of the autonomous activity, denoted λ U , (see Methods) is positive, λ U = 0.0012, confirming the presence of chaos.
Figure 1(C,D) shows what happens when the same network simulation is driven by a particular sensory stream, denoted the AB sensory stream. Here the two 50 ms stimuli, A and B, are presented in alternating sequence. The ITI following stimulus A is always 415 ms while that after B is always 1185 ms. Evidently the irregular variations in activity in the autonomous network simulation (Fig. 1(A,B)) become locked to the sensory stream in the driven simulation. The solid lines in Fig. 2(A) show the sensory stream driven trajectory projected onto the principal components of the autonomous activity. For visualization the trajectory has been divided into different sections coloured according to the epoch in the sensory stream. The trajectory appears to circulate in the vicinity of the autonomous attractor and is periodic with period of the sensory driving (here and in the following we use the terminology ‘periodic sensory stream’ to denote a sensory stream which repeats exactly at fixed time intervals). This driven network trajectory is fully stable as confirmed by calculation of the driven maximal Lyapunov exponent, denoted λ D (see Methods) which is negative, λ D = −0.0013, for this simulation in this particular sensory stream. This example therefore demonstrates the complete stabilization of autonomous rate chaos by the sensory stream.
Entrainment of chaos can also be partial. Figure 2(B,C) show examples from the same network simulation under the alternating AB sensory stream with the same stimuli A and B except that the ITI lengths have been changed slightly, while the mean ITI remains fixed at 800 ms. In Fig. 2(B) the driven trajectory is again fully stable, λ D < 0, but it is now ‘mode locked’ with double the period of the driving, a behaviour also shown by stable network oscillators57. In Fig. 2(C) chaos is only partially stabilized, λ U > λ D > 0. Although the driven trajectory never quite repeats it is strongly restricted within a ‘bundle’ of much lower dimension than the chaotic attractor (grey). This is a general characteristic of chaotic synchronization75,76,85,86,87 and, if the sensory stream is periodic, phase synchronization63. Here phase synchronization is evident because any point in the stream occurs roughly at the same fixed phase around the trajectory on each cycle.
Stabilized chaotic trajectories are strongly history dependent
In sensory stream adaptation experiments repeat presentations of a given stimulus must generate different distinct responses which are however reliable when the stream itself is repeated51,52,74,77,88,89,90,91. Here we illustrate why (fully and partially) stabilized chaos provides the ideal substrate for this92,93. Figure 2(D) shows the trajectory of the same network driven by the same stimuli A and B but now composed in a different sensory stream where the sequence AAABB is presented repeatedly with fixed 800 ms ITIs between all stimulus presentations. The trajectory is again stable, λ D < 0, but stimulus responses are totally different to those under the AB stream, Fig. 2(A). The response to any given stimulus is strongly history dependent and adapted to the periodic sensory stream. Three distinct stimulus A responses and two distinct stimulus B responses can be seen. Such history dependent neural response is known in various task settings and brain regions. 94provides a recent example.
The distinctiveness of stimulus responses depends on how chaotic the autonomous network dynamics is, given by λ U . Points on trajectories which are preceded by more similar sensory streams tend to be closer in the state space. This is visible in Fig. 2(D) where the stimulus response endpoints (green circles) for the second and third presentations of A are closer together than they are to the first stimulus A endpoint. However as the strength and dimension of autonomous chaos increases ITI period trajectories become more divergent and the driven trajectory becomes spread over a larger region. Therefore the responses generated by individual stimulus presentations reflect longer and longer sequences of preceding stimuli.
On the other hand the reliability of a stimulus response depends on two factors. First, if chaos is not completely suppressed, Fig. 2(C), or if it is mode-locked, Fig. 2(B), repeat presentations of the same stimulus can generate slightly different responses, even though the dynamics is completely deterministic. This type of reliability is described by λ D and in general decreases as it increases. It is evident from these examples that the effects of unsupressed chaos on response reliability can be very weak. This is because the ‘widths’ of the trajectory ‘bundles’ are small compared to the total variation around the trajectory phase. In fact partially stabilized chaos is particularly relevant for sensory stream adaptation since it combines fairly high reliability with high distinctiveness (as illustrated below.) A much stronger effect on stimulus response reliability originates directly from the sensory stream itself. In fully and partially stabilized chaotic networks, λ D < λ U , response reliabilities for given stimuli decrease as the multiplicity of sensory streams which precede individual presentations of that stimulus increases. For example in the AAABB sensory stream, Fig. 2(D), stimuli of type A generate much less reliable responses than stimuli of type B.
Chaos stabilization only weakly depends on stream predictability
Since stabilized chaos facilitates sensory stream adapted response it is important to investigate how chaos stabilization depends on network properties as well as on the properties of the sensory stream.
In Fig. 3 driven Lyapunov exponents λ D are plotted versus autonomous Lyapunov exponents λ U for many inhibitory network simulations generated with various network connection probabilities (see Methods.) Fig. 3(A) shows λ D for streams where stimuli A and B are presented in alternating sequence separated by fixed and equal ITIs, for various ITI lengths. Chaotic network activity is generally stabilized somewhat by the sensory driving since λ D < λ U when λ U > 0. The magnitude of stabilization, λ U − λ D , is largest for networks with λ U >≈ 0 and decreases to zero as networks become more strongly chaotic. The higher the stimulus presentation frequency the greater the stabilization, as also found in95,96. We find some stabilization occurs for ITI periods as short as 100 ms and as long as several seconds. These are natural timescales often employed in sensory streaming studies6,9,10,18,19,25,98,99,100,101.
Maximal Lyapunov exponent analysis of stabilization of chaos by sensory streaming. Driven maximal Lyapunov exponent, λ D , versus the autonomous maximal Lyapunov exponent, λ U for multiple different network simulations under (A) a temporally regular sensory stream where stimulus type, A or B, alternates and all ITIs are fixed and equal, for streams of different ITI lengths (see key, lines show 7 point moving averages each over 71 network simulations) and (B) the temporally regular sensory stream with 800 ms ITI and a fully irregular sensory stream where the stimulus type sequence is scrambled and ITIs are drawn independently from a exponential distribution with expectation 800 ms (see key, lines show 20 point moving averages). Green line indicates λ D = λ U . Chaos is partially stabilized for simulations in the upper right quadrant below the green line, since here λ U > λ D > 0. Chaos is fully stabilized in simulations in the lower right quadrant since λ U > 0 > λ D . Pink vertical line where the moving averages cross λ D = 0 indicates the onset of full stabilization of chaos on average across multiple simulations. (A,B) All simulations are deterministic.
Figure 3(B) compares λ D for the alternating AB sensory stream with fixed 800 ms ITI and a maximally irregular sensory stream where ITI intervals are exponentially distributed with mean 800 ms and where stimuli A and B are applied in scrambled sequence. Although for any particular simulation λ D can depend on the stream (as demonstrated by the example in Fig. 2), on average both periodic and random streams stabilize chaos roughly equally (Fig. 3(B) lines), (although there is a weak effect of randomness). The somewhat surprising finding that even completely random streams suppress chaos as much as periodic ones is important because it means stimuli in random streams also have the capacity to show stream history dependent response. If this were not the case stimulus response would become fully unreliable suddenly at some level of stream randomness. The point is illustrated in Fig. 2(E,F) where the trajectories under the maximally irregular and a weakly irregular sensory stream are compared. Stimulus responses and ITI period trajectories are much more reliable when activity is less irregular. Since both trajectories are almost equally stable relative response reliabilities are determined by the regularity of the sensory stream.
There have been multiple investigations of chaos control in neural networks, for example53,77,92,95,97,102. Here we don’t address it in detail. However simulations show large negative deflections in λ D at stimulus onset and offset (see Supplemental Fig. S8 and Supplemental Text). This suggests that the strong dependence of stabilization on stimulus presentation frequency and weak dependence on randomness may be very roughly explained by a simple mechanism. When the autonomous Lyapunov exponent, λ U , generated when the network is driven by the background input X, is small there are only a few unstable directions which are confined within a low dimensional attractor manifold within the much higher dimensional space. During stimulus presentations the input X generated attractor is replaced by a different stimulus dependent one. This causes the activity state to move away from the X attractor region towards the new attractor region and then rapidly return after stimulus offset. Both these transient attractor switching pieces of the driven trajectory are stable. If the stabilizing effect of these kicks is sufficient to overcome the divergence on the X attractor then the driven trajectory will become stable overall, regardless of the temporal regularity or type of sensory stimulus presentations.
In the following we denote the approximate regime where chaos is fully or partially stabilized, 0 < λ U < 0.003, for sensory streams with mean ITI around 800 ms, typically used in experiments the ‘suppressed chaos regime’. Here we focus on the role of network spontaneous activity in sensory stream adaptation, rather than how this activity arises from network structure. However for completeness in Supplemental Fig. S1 we investigate this regime in more detail. It is quite broad in biologically realistic regimes of network parameter space, spanning recurrent connection probabilities from about 0.1 to 0.25, while networks including excitatory units are inhibition dominated. The point where chaos is just suppressed in fully inhibitory networks has connection probability around 0.17 and mean synpatic conductance around 0.47 nS/cm 2 when average firing rates are 1 Hz (see Methods and Supplemental Methods.)
Reliable and distinct deviancy level dependent response in roving sensory streams
Multiple studies show that a given stimulus produces different neuronal responses depending on whether the stimulus appears as implicitly expected according to the previous streaming7,18,19,23,24,25,26,27,28,29,30,39,99,101,103,104,105,106. To investigate this we employ a ‘roving’ sensory stream, (Fig. 4), often used in experimental studies. Again two stimuli, A and B, are used but each one is repeated several, ‘n’, times before switching to the other. The first stimulus after a switch is the most ‘deviant’ of that stimulus type, denoted ‘D1’, the second after a switch is the second most deviant, denoted ‘D2’, and so on. The final stimulus before a switch, ‘Dn’, is the ‘standard’, also denoted ‘S’. (We also sometimes use ‘D(n-1)’ and ‘D(n-2)’ as the standard (see below.)) Roving sensory streams switch between the two ‘single stimulus sensory streams’ which we denote AA and BB.
Schematic of roving sensory streams. Temporally regular and irregular roving sensory streams with n = 4. 50 ms stimuli A and B are separated by ITI periods of mean length 800 ms. In the irregular stream individual ITIs are chosen at random from the range 500–1100 ms in 100 ms steps. Successive presentations of a given stimulus type are labeled by their level of deviance.
We first illustrate the population activity. Figure 5(A) shows differential peri-stimulus time histograms (dPSTH) of the neuroelectric activity for the network simulation investigated above (Figs 1,2) under the n = 4 roving sensory stream with ITI 800 ms. dPSTH are generated by subtracting the activity generated by the standard S (D4) presentation of a given stimulus type, A or B, from the activity generated by deviant presentations of the same stimulus type (see Methods). Each stimulus type is shown separately. D1 stimuli show strong differential responses at stimulus presentation and modulations which persist throughout the ITI period.
Network response to deviant stimuli presentations in the network simulation shown in Figs 1 and 2 under the n = 4 roving sensory stream with ITI 800 ms. (A) Differential dPSTH of neuroelectric activity for D1 presentations of both stimulus types (see key). Bars show SEM in the dPSTH. Vertical brown dotted lines show stimulus onset and offset. Simulation includes 10% noise. (B–D) Coloured line segments (see keys) show the trajectories of (B) the n = 4 roving sensory stream, (C) the AA and BB single stimulus sensory streams. Driven Lyapunov exponents λ D are −0.0002 and −0.0027 respectively for these two streams. (D) The three sensory streams shown in (B) and (C) overlaid, projected onto the first two principal components of the autonomous activity. Lines show trajectory segments, circles show trajectory segment endpoints. The D1 stimuli and the ITI periods subsequent to them are shown dashed. Grey line shows the trajectory of the first two principal components of the autonomous activity. Simulations are deterministic.
Differential modulations are highly significant, as can be seen from the small error bars. This is because individual presentations of a given stimulus at a given deviance level are always preceded by identical sensory streams in this roving paradigm and because chaos is stabilized, λ D < 0 < λ U . Given stimuli therefore generate distinct and reliable response at different deviancy levels. This is readily appreciated from the sensory stream driven activity projected in the principal components of the autonomous activity, Fig. 5(B).
Oscillatory differential ITI period modulations, (Figs 6(A,B) and 7(A)) which are qualitatively as observed in population level studies19,23,24,25,26,27 occur because the low dimensional chaotic dynamics is dominated by low frequency oscillatory components. As illustrated in Fig. 5(B) D1 deviants produce responses which are relatively close to, but shifted away from the standards. This results in deviant and standard ITI period trajectories which rotate in a roughly concentric way generating the differential oscillations in the neuroelectric activity dPSTH, Fig. 5(A).
Stimulus locking of neuroelectric activity in a particular inhibitory network simulation which is only partially stabilized by the roving n = 12 sensory stream. (A,B) Differential dPSTH for one particular stimulus (stimulus B) at various levels of deviancy (see key) under (A) regularly and (B) irregularly timed streams. (C) PSTH of phase at the streaming frequency (see Methods) of the induced activity for several levels of deviancy (see key) and both stimuli. (D) Time-frequency plot of ITC (see Methods) in the induced activity with stimulus onset at zero averaged across all presentations, (except the first D1 and last D12), for stimulus B. (A–D) λ D = 0.0004, λ U = 0.001. Simulations include 10% noise. (A–C) Bars show SEM in the PSTH. Vertical brown lines show stimulus onset and offset.
Examples of stimulus responses at different levels of deviancy (see keys) in an exemplar excitatory-inhibitory network simulation in which only half the excitatory and half the inhibitory units are variably driven by the sensory stream and which is only partially stabilized under the n = 12 roving sensory stream. (A) Differential neuroelectric activity dPSTH from inhibitory units not directly driven by the sensory stream for one stimulus type (stimulus B). (B) Mean neuroelectric activity PSTH from excitatory units directly driven by the sensory stream for both stimuli. (C–F) Firing rate PSTH for (C) an excitatory and (D,E) inhibitory units not directly driven by the sensory stream and (F) an inhibitory unit directly driven by the sensory stream. (A–F) Vertical orange dashed lines show stimulus onset and offset. λ U = 0.0023, λ D = 0.0006. Network connection probability 0.2. Relative excitatory to inhibitory input strength 0.74. Simulation includes 10% noise.
Responses adapt slowly in the suppressed chaos regime
Experiments show that differential responses adapt gradually over surprisingly large numbers of stimulus presentations (e.g19,25.). Evidently as the number of repeat presentations of a given stimulus, D2, D3, …, increases, the activity adapts to the trajectories generated by the corresponding single stimulus sensory streams, AA and BB, shown in Fig. 5(C) and overlaid on the n = 4 roving stream in Fig. 5(D). In general if the AA or BB trajectory is stable the timescale of relaxation after D1 deviant stimuli is given by \(|1/{\lambda }_{D}|\) where here λ D refers to the driven Lyapunov exponent of the appropriate single stimulus trajectory. The slower decay to the less stable AA trajectory compared to the BB one is visible in Fig. 5(D).
Furthermore due to the bundling of trajectories described above we find a similar gradual relaxation process occurs in the regime where chaos is only partially suppressed. Figure 6(A) shows neuroelectric activity dPSTH from a different purely inhibitory network simulation driven by the n = 12 roving sensory stream while Fig. 7(A) shows an example from a network including both excitatory and inhibitory units and in which only half of the units are directly driven by the sensory stream. In both cases driven activity is only partially stabilized, λ D > 0, and adaptation occurs slowly over five or six stimulus presentations. These observations are summarized in Fig. 8(A) which shows the absolute size of differential modulations, \(\langle |{\rm{dPSTH}}|\rangle \), (|..| denotes absolute value and 〈..〉 denotes average) averaged across the whole 850 ms stimulus and ITI period for various levels of deviancy in inhibitory networks driven by the n = 12 roving sensory stream. Only in the suppressed chaos regime do deviants generate large differential responses, peaking close to where chaos is just stabilized, λ D = 0. The strongest dependence on repetition number is also shown in this regime.
Large differential responses and slow adaptation in inhibitory networks under roving sensory streams. (A,B) Size of neuroelectric activity modulations, \(\langle |{\rm{dPSTH}}|\rangle \), (|x| denotes absolute x) for various levels of deviancy (see key) averaged across the whole 850 ms stimulus and ITI period under (A) regularly and (B) irregularly timed sensory streams. 20 point moving averages. (C,D) Instantaneous responses averaged over only the 50 ms stimulus presentations. (C) Expected size of differential instantaneous stimulus responses \(({{\rm{dR}}}_{A}^{D1}+{{\rm{dR}}}_{B}^{D1})/2\) (solid) and \(\surd {({{\rm{dR}}}_{A}^{D1}{{\rm{dR}}}_{B}^{D1})}_{\pm }\) (dashed) (see text). 15 point moving averages (sey key.) (D) Mean stimulus specific adaptation index, SI, calculated in three different ways (see Methods.) Black: all active units. Red: all units which respond phasically to either or both the D1 or S presentations. Green: all units which show no phasic response to either of the D1 or S presentations (see key.) (A–C) n = 12 roving sensory stream. (D) n = 4 roving sensory stream. (A–D) Results plotted versus the autonomous Lyapunov exponent λ U . Averages are over 112 network simulations calculated from standardized neuroelectric activity. Pink and brown lines show λ D = 0 and λ U = 0 as in Fig. 3(B). Simulations include 10% noise. Bars show SEM over the local moving averages (exemplar results only for figure clarity.) Both stimulus types A and B are averaged.
The above example indicates the following explanation for these results. As λ U increases responses at different levels of deviancy become more distinct and differential fluctuations (signal) larger. When λ D increases past zero unreliability due to unsuppressed chaos (noise) increases. Because the dPSTH are calculated from standardized neuroelectric activity time series, \(\langle |{\rm{dPSTH}}|\rangle \) reflect the relative size of signal to noise which decreases as the chaotic noise increases. Differential dPSTH depend on deviancy level throughout the suppressed chaos regime because stimulus response continues to reflect the preceding sensory stream even when chaos is only partially suppressed.
Differential modulations decrease in magnitude when streaming is irregular
Differential responses are also known to depend on the temporal regularity of the adapting sensory stream30. Deviants in irregular sensory streams generate differential responses of smaller magnitude than their counterparts in regular streams. Here we investigate this using an irregularly timed roving sensory stream (see Fig. 4).
Differential responses to deviants under the irregularly timed stream (Fig. 6(B)) have similar form to their counterparts under the regularly timed stream (Fig. 6(A)) but are weaker and the error bars larger. This is confirmed in Fig. 8(B) which shows the magnitude of differential modulations, \(\langle |{\rm{dPSTH}}|\rangle \), versus λ U for inhibitory network simulations under the irregularly timed n = 12 roving sensory stream. The reduced size of the D1 differential response compared to Fig. 8(A) again occurs because the relative size of the signal component is reduced compared to noise. However in contrast to the case of temporally regular streaming described above where unsuppressed chaos was the sole source of unreliability, here unreliability is also caused by variability in the preceding streaming even when chaos is fully suppressed, λ D < 0. In particular the increased variability of the S response (compare D7 in Fig. 8(A) and (B)) prevents adaptation after the third repetition, D3, and reduces the size of the D1 differential response. These observations are in qualitative agreement with the experimental studies (see Discussion.)
Time-frequency responses
Multiple studies show that sensory streams phase entrain brain oscillations at the streaming frequency, and that this is weakened by temporal irregularity, for example9,10,11,12,13,14,15,16,107. Standard and deviant presentations of stimuli are also known to show characteristic responses31,33,107,108 and differences19,35,109,110 in their neuroelectric activity spectrograms. Stimulus presentations sometimes generate bursts of coherence in the theta and alpha bands associated with phase resetting of endogenous oscillations35,108. Here we investigate these issues in the n = 12 roving sensory stream. We employ two manipulations to remove the evoked stimulus response and reveal the underlying ‘induced’ activity.
The first ‘weaker’ manipulation is sometimes employed in experimental studies. From each time point τ ms after the onset of a given stimulus the mean value for that time point τ averaged across all presentations of that stimulus is subtracted (see Methods). We term this the ‘reduced’ activity. Supplemental Fig.S3(A) shows inter-trial phase coherence (ITC) across trials of a given deviancy level D of this reduced activity in a narrow band around the streaming frequency 1/0.85 Hz under the regularly and irregularly timed n = 12 roving streams versus the autonomous Lyapunov exponent λ U . Phase coherence peaks around the point where chaos is just stabilized for all levels of deviancy and is weaker when streaming is irregular. However it has a non-monotonic dependence on deviancy level D. This is because phase coherence is mainly due to the stimulus response which, due to response adaptation, is not fully removed by this manipulation. Indeed phase coherence at first decreases with increasing deviancy and then increases again, as expected if the D1 response is larger than the subtracted average and the D12 response smaller than the subtracted average. Phase coherence is however very low in stable networks demonstrating their absence of adaptation.
Supplemental Figs. S4 and S5 show time frequency plots of excess ITC and excess log power, respectively, in deviant D1 stimulus presentations compared to standard D11 stimulus presentations (see Methods) calculated from the reduced activity. Results are averaged across both stimuli and all network simulations with autonomous Lyapunov exponent falling within in prescribed ranges. Network simulations in the suppressed chaos regime, Supplemental Figs. S4(C), S5(C), show strong bursts of excess coherence and power locked to stimulus presentation, while these effects are weaker in more strongly chaotic and in stable networks (note different colour scales.) Such responses, which last several hundred milliseconds after stimulus offset and can extend up to around 80 Hz, are sometimes seen in experimental studies19,31,33,35,107,108,109,110.
Of more interest in the current study is a second ‘stronger’ manipulation. Here from each time point τ ms after the onset of a given stimulus of given deviancy level D we subtract the mean value for that time point τ averaged across all presentations of that stimulus of that deviancy level D. We term this the ‘induced’ activity. It reveals the underlying chaotic ‘noise fluctuations’. Figs. 6(C,D) illustrate this manipulation applied to the network simulation investigated in Fig. 6(A). PSTH of induced phase at the streaming frequency, Fig. 6(C), is highly similar across stimulus B presentations of different levels of deviancy, but less so for stimulus A. Phase for stimulus B appears to have twice the frequency of the sensory stream indicating higher mode locking for this particular network and stimulus. The phase PSTH for the two stimuli peak at different time points after stimulus offset, a common experimental observation. The time frequency plot of ITC, Fig. 6(D), averaged across all stimulus presentations (except the D1 and D12 stimuli, see Methods) for stimulus B shows a strong burst of ITC locked to stimulus onset around the theta to alpha frequency band (3–15 Hz), also often observed experimentally19,31,33,35,107,108,109,110.
In Fig. 9(A) we investigate how ITC in the induced neuroelectric activity at the streaming frequency depends on the autonomous Lyapunov exponent λ U . In common with the reduced activity, Supplemental Fig. S3(A), this quantity peaks around the point where chaos is just stabilized and is weakened in irregular sensory streams. In contrast to the reduced activity it does not strongly depend on the level of deviancy. The fact that ITC in the induced activity does not strongly depend on deviancy does not mean that stimuli of different deviancy levels necessarily have similar phase PSTH profiles. Since phase PSTH can depend on the stimulus, as shown in Fig. 6(C), we expect it may take several stimulus presentations for the shift in phase locking to occur after the stimulus is switched in the roving sensory stream. To investigate this we calculate the distance between the phase distribution for deviancy level D and that for the standard D11 stimulus. This quantity is shown in Fig. 9(B) versus deviancy level D averaged across both stimuli and all network simulations within certain prescribed ranges of autonomous Lyapunov exponent λ U . It shows a gradual decay only in the suppressed chaos regime 0 < λ U < 2, confirming the slow adaptation to a new phase locked state in the induced activity after a stimulus switch. This behaviour is also found in the reduced activity, Supplemental Fig. S3(B). Time frequency ITC Fig. 9(C–F) averaged across both stimuli and all network simulations within certain prescribed ranges of autonomous Lyapunov exponent λ U show a strong burst of theta to alpha coherence at stimulus onset, especially in the suppressed chaos regime, Fig. 9(D).
ITC (see Methods) across stimulus presentations of given deviancy level in the induced neuroelectric activity for the period between 300 and 600 ms after stimulus offset under the n = 12 roving sensory stream. (A) Phase coherence versus autonomous Lyapunov exponent for several deviancy levels in regular and irregular streams (see key.) 10 point moving average, bars show SEM in this average. (B) Distance between the phase distribution for stimuli of deviancy level D and that of standard D11 stimulus averaged across all simulations with autonomous Lyapunov exponent λ U in the ranges [−4, −2], [−2, 0], [0, 2], [4, 6], (see key). Bars show SEM from the 300 and 600 ms averaging period. (C–F) Time frequency plots of ITC averaged across all stimulus presentations, except the first, D1, and last, D12, (see Methods) for simulations with autonomous Lyapunov exponent λ U in the ranges (C) [−2, 0], (D) [0, 2], (E) [2, 4], (F) [4, 6]. (A–E) All results averaged across both stimuli A and B.
Several studies show that low frequency brain oscillations can become phase entrained by sensory streams even when the streams are composed of multiple different stimuli9,30,99,100,101,104. This is possible in the current model because even streams composed of different stimuli can stabilize chaos by attractor switching as described above. To demonstrate this we apply the model to a simplified version of the task recently investigated by9. Participants discriminated the orientation of target visual stimuli embedded within streams of ‘random’ visual stimuli termed ‘distractors’ (see Supplemental Fig. S6(A)). The authors found phase entrainment was strong at the streaming frequency and increased with the temporal regularity of the sensory stream.
We investigate phase coherence of the ‘induced’ neuroelectric activity. To generate this the stimulus conditioned PSTH is subtracted from the neuroelectric activity for each different stimulus type separately, including targets and distractors, in the same way as described above (see Methods). We find phase coherence at the streaming frequency peaks (Supplemental Fig. S6(B)) close to the point where chaos is just stabilized at this ITI of 400 ms (compare Fig. 3(A)) when distractors are identical. A strong peak is also shown when distractors are random but similar to each other (clustered.) The peak is weakened by temporal irregularity. However the peak is absent when stimuli are totally random. Indeed while totally random stimuli streams can stabilize chaos (so that the trajectory of the driven network activity is independent of the initial network state at the onset of streaming) they do not phase synchronize the activity. The distractor stimuli used in experimental studies like9 should be considered ‘clustered’ since although different they have many features in common.
Deviants generate larger instantaneous population responses than standards
We often find in inhibition dominated networks that stimulus presentations generate large negative instantaneous deflections in the neuroelectric activity. This can seen from the PSTH in the network simulation with both excitatory and inhibitory units, Fig. 7(B), as well as in the purely inhibitory network, Fig. 1(C). Here we investigate whether the magnitude of these instantaneous responses adapts with repetition number as often found in studies18,28,47. This effect is complicated by the possible presence of mode-locking. In networks where this occurs responses to odd numbered deviants (D1, D3, D5, …) may be larger than responses to even numbered deviants (D2, D4,…) for example. We also often find that D1 deviants generate instantaneous population responses of larger magnitude than standards for one stimulus type but not the other stimulus type in two stimuli roving sensory streams. The opposing character of the differential responses for the two stimulus types in the purely inhibitory network, Fig. 1(C), is apparent in the 50 ms stimulus presentation epoch in Fig. 5(A).
We measure the instantaneous stimulus response size, \({R}_{Z}^{D}\) to stimulus type Z = A,B of deviancy level D as the difference between the mean neuroelectric activity during the 50 ms stimulus presentation epoch and the 50 ms immediately preceding the stimulus presentation (see Methods). The differential response size for D stimuli is then defined as \({{\rm{dR}}}_{Z}^{D}=|{R}_{Z}^{D}|-|{R}_{Z}^{S}|\), where again |..| denotes absolute value and S = D12 is the standard stimulus. The stimulus averaged differential response, \(({{\rm{dR}}}_{A}^{D}+{{\rm{dR}}}_{B}^{D})/2\), is shown in Fig. 8(C, solid lines) versus λ U for inhibitory network simulations, while the opposing nature of the differential response sizes is illustrated by the quantity \(\surd {({{\rm{dR}}}_{A}^{D}{{\rm{dR}}}_{B}^{D})}_{\pm }\), Fig. 8(C, dashed line). Here √(x)± denotes sign (x)√(|x|). Throughout the suppressed chaos regime this latter quantity is large and negative for D1 stimuli. This effect is much weaker for later presentations after D1 (not shown). Although D1 differential responses sizes are often opposing in the two stimulus types, deviants generate larger magnitude responses in the neuroelectric activity than standards on average over both stimuli throughout the suppressed chaos regime, an effect which relaxes slowly over about four stimulus presentations, Fig. 8(C, solid lines). In fact both these effects can also be found in a simple phenomenological model where activity does not fully relax to an equilibrium state between stimulus presentations (see Supplemental). To show these effects the phenomenological model relaxation timescale parameter must be of the order of the time between stimulus presentations. In contrast here long timescales are created when network chaos is (partially) stabilized while network model timescale parameters do not exceed 50 ms and remain biologically reasonable. Thus the experimental observations of large excess instantaneous deflections in LFP18,28,47 for deviant stimuli compared to standards are qualitatively reproduced (see Discussion) only in the suppressed chaos regime.
Single units also show adaptation
Stimulus response adaptation is also know to occur in multi-unit18,28 and single unit17,111 activity in both excitatory and inhibitory cells20,21. Figure 7(C–F) shows examples of single unit PSTH for standards and deviants from the network simulation including excitatory and inhibitory units under the n = 12 roving sensory stream. As can be seen the dependence on repetition number can be quite complex, both in the initial stimulus response and in the later modulations during the ITI. Stimulus presentations can generate phasic facilitating or suppressing responses to one or other stimulus type or both, Fig. 7(C,D). Stimulus induced modulations of tonically active units are also seen, Fig. 7(E,F). Stimulus induced modulations occur in both excitatory Fig. 7(C) and inhibitory units Fig. 7(D–F) and in units not directly driven by the sensory stream Fig. 7(C–E) as well as those that are, Fig. 7(F). Notice that this network simulation is almost completely stablized and the D11 and D12 responses are very similar.
These phasic responses result from the stimulus presentation induced ‘kicks’ described above. When the stimuli are presented the trajectory transiently moves away from the chaotic attractor region and then returns. During this period some single units silent during ITI periods are briefly activated. These units can suppress the ones tonically active during the ITI period resulting in negative deflections like those shown in Fig. 7(E,F) which may also appear in the population neuroelectric activity, Fig. 7(B). According to the above results we expect deviants to produce larger single unit responses than standards for one stimulus type but not the other, while deviant responses will be on average larger. On the other hand due to the presence of phasic activations in some units and suppression of tonic activity in others this may not show up if firing rates during stimulus presentation are simply averaged across all units.
To confirm these observations we calculate stimulus specific adaptation indices (SI) for single units, as in17,111, (see Methods.) SI vary from 1 to −1, depending on whether a unit is more active during the D1 or S presentation of a stimulus. The black line in Fig. 8(D) shows SI averaged across all the active units (see Methods) for many inhibitory network simulations under the n = 4 roving sensory stream versus λ U . Both stimulus types, A and B, are combined. When calculated in this way SI are close to zero throughout the range of λ U and do not show a strong dependence on it. However when the SI average is restricted to include only units which show positive phasic responses to either the D1 or S stimulus presentations (see Methods), or both, Fig. 8(D, red), SI is significantly positive but only in the suppressed chaos regime. When the calculation includes only active units which do not show positive phasic responses to either D1 or S stimulus presentations, Fig. 8(D, green), the opposite is true. Thus phasic units are more strongly activated on average by deviants than standards while tonic units are on average more strongly suppressed by deviants than standards.
Discussion
We investigated how stimulus streams interact with spontaneously generated dynamics in neural networks without any explicit learning mechanisms. Sensory stream adaptation may be a general property common to many different brain areas13,22 embedded in multiple possible neural substrates on different spatial and temporal scales during various behavioural task settings. To illustrate how a simple mechanism can unify a range of observations we have necessarily sacrificed model specificity for model generality. The model is the simplest one may imagine; a random network of identical units, each one described by a single dynamical variable with a single timescale driven by completely random stimuli. Despite this its emergent behaviour is so complex that it should be applied to each experimental setting individually to understand the related results. It can therefore provide deep insight into the origin of many seemingly contradictory experimental results. It will be particularly relevant in music, language and birdsong however repetitive sensory patterns may be generated by the motion of the animal itself. In particular chaotic network oscillations may occur as part of central pattern generators.
Remarkably we showed that stimulus sequences produce realistic repetition effects on neural response even without synaptic modification. Deviant stimuli generate long lived differential oscillatory modulations such as mismatch negativity and repetition positivity in population activity19,23,24,25,26,27. They are automatic, independent of the behavioural task and of attention27. Quantitative comparisons in a model of this generality are not possible since the form of the modulations depends on the task and brain area under study and also varies across individuals. We found good qualitative agreement with studies (e.g19,25.) when chaos is (partially) stabilized by stimulus streams. For greater agreement with experiment in future work the delay between the time of the physical stimulus and the time the signal reaches cortex, not included in the present model, should be taken into account, as part of more detailed modeling of the pre-cortical sensory processing. We showed partial stabilization allowed stimuli to generate distinct and reliable deviancy-level dependent responses in periodic roving streams.
We also found that deviants generate strong differential responses in temporally irregular streams30,112 but the response magnitude was weaker, in striking agreement with30. We showed this was because PSTH variability reflects stream irregularity in (partially) stabilized networks. Actually the effects of different possible forms of sensory stream variability on this model are complex. In the future this model will be profitably applied case by case to understand neural response in sensory streaming experiments with interacting factors. For example when stimulus contrast and stimulus novelty interact (e.g38.), or where temporal and type expectations interact99,100,101.
We also investigated the stimulus response specifically. In inhibitory networks D1 deviants on average generated larger population responses in neuroelectric activity than standards in one stimulus type. This tendency was only weakly shown for later repetitions. The average neuroelectric response size over both stimulus types was also larger for deviants than standards as is often observed in LFP studies18,28,47. Single units showed complex repetition dependent modulations both in their initial stimulus induced phasic activations and for longer periods during the ITI which are intriguingly similar to those found in recent studies20,21,47 and thought to result from recurrent network inhibition. On average single units showed stronger phasic responses than standards in striking agreement with studies at the multi-unit18,28 and single unit17,111 level.
In roving sensory streams we found adaptation to the new single stimulus stream after a deviant can happen very slowly over multiple trials, in agreement with multiple studies (e.g25.). In contrast to models based on synaptic mechanisms113,114,115 where adaptation timescales reflect synaptic time constants here slow adaptation occurs because stabilization of chaos brings the driven dynamics close to critical λ D ≈ 0. The resulting adaptation timescales are around 20 times longer than the 50 ms synaptic time constant. There are several observations which provide convincing support for this which can’t be easily reproduced with purely synaptic mechanisms. First it accounts for the dependence of adaptation on temporal regularity of stimulus presentation30. Second it accounts for the generation of multiple adaptation timescales and adaptation to whole stimulus sequences29,111. Third it accounts for the fact that deviant stimuli may produce stronger responses than unadapted frequency matched controls19,28,40,41. However we do not expect network adaptation to operate in isolation from synaptic effects. Rather we hope recognition of the important fact that networks can adapt in a biologically plausible way will provide exciting insight into recent studies of how recurrent local inhibition and synaptic modifications interact20,21,42,43,47
Our results are in good agreement with experiments that demonstrate that low frequency brain oscillations become phase locked to stimuli presentations in streaming perceptual discrimination tasks9,10,11,14,15,106,116,117,118. Phase locking occurs across a broad range of cortical areas14, while locking outside the streaming frequency band is mostly limited to primary sensory areas. Stimuli presentations are associated with increased ITC and power predominantly in the theta to alpha band31,33,107,108, while deviants also generate differential responses in roving paradigms19,35,109,110, in qualitative agreement with this model.
Ongoing brain oscillations are often thought to be emergent properties of network mediated population dynamics54,55,56,119,120 close to a phase transition121,122. Several experimental studies have convincingly demonstrated they may be chaotic123,124. Interestingly low dimensional chaos may be stronger during slow-wave sleep than REM or wakefulness58,125,126, consistent with our hypothesis of chaos suppression by sensory driving. Intriguingly several studies have also found direct evidence for unstable periodic orbits in neural dynamics127,128,129, which are related to phase entrainment of chaos, while other studies directly demonstrate phase synchronization130.
Recurrent networks provide a natural substrate for various forms of neural computation and sequence learning48,49,50,51,52,131,132. SORN networks133 utilize plasticity to modify network structure and are therefore based on completely different principles to the current model. Like other studies64,65,89,134,135,136 in various task settings we find optimal performance in a critical state or at the edge of chaos. However we also show that consistency with experiment does not require this. We found stream history dependence in a wide regime where chaos is partially suppressed. This is surprising given the inherent instability of chaos79 but other studies find that significantly chaotic dynamics is consistent with neural information processing77. In the partially stabilized regime driven trajectories remain ‘bundled’ together so that the separation between them tends to a constant finite level. This is characteristic of chaotic synchronization75,76,85 and phase synchronization63. Phase stabilized chaotic trajectories combine divergence (high signal) with reliability (low internal noise) and therefore facilitate stream adapted response. Furthermore we found that the amount of stabilization, while strongly dependent on stimulus presentation frequency95, was only weakly dependent on the temporal or type predictability of stimuli in the stream. This allowed stimulus response reliabilites to be strongly determined by the variability of the preceding sensory stream. Finally we demonstrated this property emerged because stimuli move the network state away from its low dimensional chaotic attractor region. Thus we expect the results presented here to generally apply to chaotic networks in parameter regions where they can be stabilized, irrespective of some details of network physiology and structure.
Methods
Network simulations
We use a rate network model,
where f i (t) are the firing rates for the N firing rate units i. I i (t) is the input current to unit i given by,
This equation models the variations of synaptic neurotransmitters which control cell firing rates. The synaptic timescale is set as τ g = 50 ms. The term \(\gamma {\sqrt{(I(t)-{I}_{\theta })}}_{+}\), (\(\sqrt{I(t)-{I}_{\theta }}\) for I(t) − I θ > 0 and zero otherwise) is the dependence of firing rate on input current close to saddle-node on invariant circle bifurcation to firing from quiescence as occurs in ‘Type I’ cell models. γ = 0.09 is a fixed parameter. V Z,K = 60.0,5.0 mV are fixed parameters. I θ = 4.51 μ A/cm 2 is the current at firing threshold. These firing rate units may be used to represent single cells or alternatively local populations of cells, as in69 for example.
k ij is the fixed network connection matrix. We investigate two network models (i) a 500 unit purely inhibitory network and (ii) a 1200 unit network with 500 inhibitory units and 700 excitatory units. Most simulations for statistical calculations are performed on the 500 unit network. However we also demonstrate that the larger network has the capacity to show similar behaviour (see Supplemental Methods for more details.)
\({g}_{i}^{Z}\) is the driving excitation for unit i during sensory stimulus Z. Simulations generally include two sensory stimuli, denoted A and B. These stimuli are always applied to the network for 50 ms time periods separated by longer inter-trial-intervals (ITIs). During ITIs a different excitatory input, denoted X, is applied in place of the sensory stimuli. The input \({g}_{i}^{Z}\) is drawn randomly from the same given distribution for each unit i and each stimulus Z in a simulation, whether it be A, B or X, and fixed for the duration of the simulation. The parameters are set so that the average input current to each unit i is just above firing threshold I θ (see Supplemental Methods for more details.) All units i in the network are continuously driven above firing threshold by the external driving excitation by all stimuli Z. Only network inhibition can cause them to become quiescent for periods.
For generality we also investigate an excitatory-inhibitory network simulation where only half of the excitatory units and only half of the inhibitory units are directly driven by the sensory stimuli while the other half receive constant unvarying excitatory driving input. In this case the sensory stimuli A, B and background input X are determined as above but then \({g}_{i}^{A}\) and \({g}_{i}^{B}\) for half of the units i are set to \({g}_{i}^{X}\).
As a proxy for the LFP/EEG we use the absolute synaptic input current per unit \((1/N)|{V}_{K}{\sum }_{ij}{k}_{ij}{f}_{j}(t)|\) where |x| denotes the absolute value of x 57,70. In the simulation where only half of the units are directly driven by the sensory stream we divide this into four separate contributions, originating from the excitatory and inhibitory units directly and not directly driven by external sensory input.
In Eq. 1 η i (t) are iid Gaussian distributed noise terms with 〈η i (t)〉 = 0 and 〈η i (t 1)η j (t 2)〉 = δ ij δ(t 1 − t 2). The parameter α controls the strength of this stochastic term and is always set to α = 0.1 so that fluctuations have the expected size 10% of the rates f i , except in Lyapunov exponent calculations and the time series examples where α is set to zero. The noise represents fluctuations arising from multiple sources including incoherent spike arrival times which have been averaged out in the present rate dynamics network model. Noise has no dynamical role in the results presented here; it is included purely to demonstrate their robustness.
All simulations of the noisy rate network were performed with a stochastic weak second order Runge-Kutta integrator71 while deterministic simulations are performed with a fourth order Runge-Kutta. All simulations are performed with integration time step 1 ms.
Sensory stream experiments
Simulations of the AB and AAABB sensory streams were performed for 180 seconds after discarding a 30 second transient. Simulations of the roving sensory streams were performed until 120 repeats had occurred. This required 2390 second network simulations for n = 12 and 797 second network simulations for n = 4.
Maximal Lyapunov exponent calculations
In this paper we make use of conditional maximal Lyapunov exponents72,73,74,75,76,77 which are calculated numerically. Their values depend on the temporally varying driving input as well as the network structure. In principle they also depend on the choice of initial conditions. All Lyapunov exponent calculations are made on the deterministic version of the model, Eq. 1, where the parameter α = 0, (see Supplemental Methods for more details of calculation).
Principal component analysis
Principal components were calculated from the N × N covariance matrix obtained from the rate time series of all N units in a network simulation of length 180 seconds after discarding a 30 second transient. Principal components were calculated from autonomous network simulations constantly driven by the background input X. The grey trajectories in Figs 2 and 5 are time series of the autonomous network activity projected onto these first two principal component vectors. The coloured trajectory segments are the projections of the driven activity of the identical network simulation under the sensory streams onto the same first two principal component vectors of the autonomous activity. Sensory stream driven trajectories are divided into differently coloured segments according to the changes in driving input, A (green), ITI following A (driven by input X) (red), B (blue), ITI following B (driven by input X) (black). Segment endpoints are circled. Simulations under the sensory streams were performed until 88 presentations of each of the two stimulus types had occurred, resulting in 88 × 4 segments. This took approximately 150 seconds. All simulations were sampled at 5 ms intervals.
Peri-stimulus time histograms
Stimulus and deviancy level conditioned peri-stimulus time histograms (PSTH) H Z,D(τ) in the roving sensory streams are averages over the neuroelectric network activity, Fig. 7(B) or firing rate activity Fig. 7(C–F) time series sampled at 5 ms intervals at a particular time lag τ after the onset of the D th repetition of a particular stimulus Z = A,B for a particular network simulation. Differential dPSTH, ΔH Z,D(τ) = H Z,D(τ) − H Z,12(τ), are shown in Figs 6(A,B) and 7(A) under the n = 12 roving sensory streams. Figure 5(A) shows differential dPSTH, ΔH Z,D(τ) = H Z,D(τ) − H Z,4(τ) under the n = 4 roving sensory stream. Stimulus conditioned PSTH H Z(τ) are calculated the same way except all deviancies D are included.
The mean absolute differential \(\langle |{\rm{dPSTH}}|\rangle \) shown in Fig. 8(A,B) for each network simulation, stimulus Z and repetition D are given by \((850/5){\sum }_{\tau }|\Delta {H}^{Z,D}(\tau )|\) where |x| denotes the absolute value of x. dPSTH are calculated from standardized mean activity time series under the n = 12 roving sensory stream. The sum runs over all time lags τ from stimulus onset to the onset of the following stimulus.
Figure 8(C) shows results based on instantaneous neuroelectric activity responses R Z,D under the n = 12 roving sensory stream. \({R}^{Z,D}=(50/5){\sum }_{\tau }({H}^{Z,D}(\tau )-{H}^{Z,D}(-\tau ))\) where the sum over τ runs over only the 50 ms stimulus presentation epoch and −τ runs over the 50 ms time epoch immediately preceding stimulus presentation in 5 ms increments. Then differential instantaneous response is given by dR Z = |R Z,1| − |R Z,12|. Figure 8(C) shows (dR A + dR B)/2 and \({\rm{sign}}(d{R}^{A}d{R}^{B})\sqrt{|d{R}^{A}d{R}^{B}|}\).
Spectral analysis
First reduced and induced activity time series are created by subtracting the stimulus, Z, conditioned PSTH, H Z(τ), and the stimulus Z and deviancy level D conditioned PSTH, H Z,D(τ), respectively, from the n = 12 roving sensory stream neuroelectric activity time series. Irregular roving streams are treated the same way where PSTH are calculated from the same irregular neuroelectric activity.
To calculate time-frequency plots a Hanning window of length 640 ms is moved along the time series in 5 ms steps, t. From the complex coefficients r(t, f) (e iθ(t, f)), log power, 2log(r(t, f)), and complex phase e iθ(t, f) are computed and averaged to create stimulus onset locked power PSTH and inter-trial coherence (ITC). The absolute value of the mean complex phase is taken for the ITC which varies between zero and unity. PSTH are created in the same way as described above, for both stimuli A and B separately and all deviancy levels D separately. The time-frequency plot shown in Fig. 6(D) is the ITC averaged across stimulus B presentations of all deviancy levels except the first and last. These are omitted to avoid any end-effects originating from the stimulus switching in the 640 ms time window. The time-frequency ITC plots shown in Fig. 9(C–F) are created the same way except both stimuli and all network simulations within given ranges of λ U are averaged. To create excess ITC and excess log power PSTH plots in Supplemental Fig. S4 and Supplemental Fig. S5 time-frequency spectrograms calculated from D11 stimuli of given type are subtracted from D1 stimuli of the same type. Then both stimuli and all network simulations within given ranges of λ U are averaged. Again the D12 stimulus is not used to avoid possible effects originating from the stimulus switch.
To investigate phase coherence specifically at the streaming frequency in Figs 6(C) and 9(A,B), and Supplemental Fig. S3 a more frequency specific method is used. First the reduced or induced neuroelectric is divided into sections. Each section starts 480 ms after the offset of a D12 presentation of a given stimulus, say A, and ends 520 ms after the offset of the subsequent D12 presentation of the other stimulus type, say B. Thus each section includes presentations of only one stimulus type. Since the sampling period is 5 ms each section includes exactly 2048 grid points. In the irregular sensory stream 2048 grid point sections starting 480 ms after the offset of a D12 presentation are used. If a 2048 grid point section extends past the point of first presentation of the other stimulus type it is zero-padded from that point. Each section is band passed in a narrow band around the streaming frequency, (1/0.85)±0.3 Hz and the phase time series, θ(t), extracted using Hilbert transform. To calculate stimulus locked phase coherence, the distribution of phase, p(θ|τ, Z, D), τ ms after the onset of stimulus type Z of deviancy level D is obtained from multiple sections. For phase coherence we use the modulation index137 which is defined as 1 minus the normalized entropy of this phase distribution, \(1-{\sum }_{i}p({\theta }_{i})\mathrm{log}\,p({\theta }_{i})/log(N)\) where N is the number of bins i. This quantity is averaged across the period 350 < τ < 650 ms and over both stimuli Z to obtain the phase coherence for each deviancy D. For the irregular stream we only include ITIs longer than 600 ms in this calculation. To calculate the difference between phase coherence for different deviancy levels in Fig. 9(B), Supplemental Fig. S3(B) the KS distance between the phase distributions is calculated. This is \({\sum }_{i}|P({\theta }_{i}|\tau ,Z,D)-P({\theta }_{i}|\tau ,Z,D11)|\) where P is the cumulative distribution, ∑ runs over the whole distribution and |..| denotes absolute value. This is averaged across 350 < τ < 650 ms and over both stimuli Z to obtain KS(D).
Phase coherence at the streaming frequency in Supplemental Fig. S6 is calculated in a similar way. First ‘induced’ activity is obtained from the neuroelectric activity by subtracting the stimulus conditioned PSTH H Z(τ), where stimulus Z refers to the targets A and B and each of the six distractors, a total of eight stimuli. Next sections of length 16384 grid points (with time period of 16384 × 5 ms) are extracted, band passed in a narrow band around the streaming frequency (1/0.45) ± 0.4 Hz and the phase time series generated by Hilbert transform. Next the distribution of phase, p(θ|τ, Z), τ ms after the onset of target stimulus type Z = A,B is obtained from multiple target stimulus presentations. Finally the modulation index is calculated from the distribution and averaged across the period 300 < τ < 440 ms after stimulus onset and over both targets A,B. Irregular sensory streams are treated exactly the same way because each target is always followed by a 400 ms ITI.
Single unit phasic analysis
Each unit was defined as active for a particular epoch if its mean firing rate exceeded 10−5 Hz during the epoch, for example during the final 400 ms of the ITI period following a D1 stimulus of type A. This minimum firing rate could be varied without change in results however some minimum firing rate is necessary for the calculation of stimulus specific adaptation indices SI 17. These are defined by \(S{I}_{i}^{Z}=(SE{R}_{i}(Z,D\mathrm{1)}-SE{R}_{i}(Z,S))/(SE{R}_{i}(Z,D\mathrm{1)}+SE{R}_{i}(Z,S))\) for a unit i and stimulus type Z = A,B where ‘stimulus epoch rates’ SER i (Z,D1) and SER i (Z,S) are the firing rates of unit i during the 50 ms stimulus presentation epoch for deviant D1 and standard S presentations of stimulus type Z. For a unit to have an \(S{I}_{i}^{Z}\) it must be active in either stimulus presentation, i.e. SER i (Z, D1) > 10−5 or SER i (Z, S) > 10−5.
The late epoch rates, LER i (Z, D1) and LER i (Z, S) for cell i and stimulus type Z are defined as the mean firing rate during the final 400 ms of the ITI period following the D1 or S presentations of stimulus type Z. The associated stimulus responsivities, SR i (Z, D1) and SR i (Z, S), are defined by SER i (Z, D1)/LER i (Z, D1) and SER i (Z, S)/LER i (Z, S). Supplemental Fig. S7 shows SR i (Z, D1) versus LER i (Z, D1) and SR i (Z, S) versus LER i (Z, S) for all 500 units in both D1 and S presentations of both stimulus types Z, (a total of 2000 points), for the partially stabilized network simulation shown in Fig. 8(A) under the n = 4 sensory stream. Single units are divided mainly into two distinct clusters. The cluster with SR close to unity and LER around 20 Hz contains the units tonically (or intermittently) activated by backgound input X during ITI periods. The cluster with LER around 10−3 Hz and SR around 10−3 contains the units phasically activated by D1 presentations (black plusses) and S presentations (red crosses).
A unit i is denoted ‘phasically responsive’ for a particular stimulus type Z and presentation type, D1 or S, if the associated SR i > 10 and LER i > 10−5. These phasically responsive units are the ones in the upper right quadrant demarcated by the pink dashed lines in Supplemental Fig. S7.
References
Arnal, L. H. & Giraud, A. L. Cortical oscillations and sensory predictions. Trends in cognitive sciences 16(7), 390–398 (2012).
Schroeder, C. E. & Lakatos, P. Low-frequency neuronal oscillations as instruments of sensory selection. Trends in neurosciences 32(1), 9–18 (2009).
Friston, K. & Frith, C. A duet for one. Consciousness and cognition 36, 390–405 (2015).
Henry, M. J. & Herrmann, B. Low-frequency neural oscillations support dynamic attending in temporal context. Timing & Time Perception 2(1), 62–86 (2014).
Nobre, A. C., Correa, A. & Coull, J. T. The hazards of time. Current opinion in neurobiology 17(4), 465–470 (2007).
Barnes, R. & Jones, M. R. Expectancy, attention, and time. Cognitive psychology 41(3), 254–311 (2000).
Sanabria, D. & Correa, Ã. Electrophysiological evidence of temporal preparation driven by rhythms in audition. Biological psychology 92(2), 98–105 (2013).
Large, E. W. & Jones, M. R. The dynamics of attending: How people track time-varying events. Psychological review 106(1), 119 (1999).
Cravo, A. M., Rohenkohl, G., Wyart, V. & Nobre, A. C. Temporal expectation enhances contrast sensitivity by phase entrainment of low-frequency oscillations in visual cortex. Journal of Neuroscience 33(9), 4002–4010 (2013).
Lakatos, P., Karmos, G., Mehta, A. D., Ulbert, I. & Schroeder, C. E. Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 320(5872), 110–113 (2008).
Mathewson, K. E., Fabiani, M., Gratton, G., Beck, D. M. & Lleras, A. Rescuing stimuli from invisibility: Inducing a momentary release from visual masking with pre-target entrainment. Cognition 115(1), 186–191 (2010).
Lakatos, P. et al. The spectrotemporal filter mechanism of auditory selective attention. Neuron 77(4), 750–761 (2013).
Coull, J. T. & Nobre, A. C. Dissociating explicit timing from temporal expectation with fMRI. Current opinion in neurobiology 18(2), 137–144 (2008).
Besle, J. et al. Tuning of the human neocortex to the temporal dynamics of attended events. Journal of Neuroscience 31(9), 3176–3185 (2011).
Fujioka, T., Trainor, L. J., Large, E. W. & Ross, B. Internalized timing of isochronous sounds is represented in neuromagnetic beta oscillations. Journal of Neuroscience 32(5), 1791–1802 (2012).
Grahn, J. A. & Brett, M. Rhythm and beat perception in motor areas of the brain. Journal of cognitive neuroscience 19(5), 893–906 (2007).
Ulanovsky, N., Las, L. & Nelken, I. Processing of low-probability sounds by cortical neurons. Nature neuroscience 6(4), 391 (2003).
Yaron, A., Hershenhoren, I. & Nelken, I. Sensitivity to complex statistical regularities in rat auditory cortex. Neuron 76(3), 603–615 (2012).
Takaura, K. & Fujii, N. Facilitative effect of repetitive presentation of one stimulus on cortical responses to other stimuli in macaque monkeys-a possible neural mechanism for mismatch negativity. European Journal of Neuroscience 43(4), 516–528 (2016).
Chen, I. W., Helmchen, F. & Lütcke, H. Specific early and late oddball-evoked responses in excitatory and inhibitory neurons of mouse auditory cortex. Journal of Neuroscience 35(36), 12560–12573 (2015).
Natan, R. G. et al. Complementary control of sensory adaptation by two types of cortical interneurons. Elife 4, e09868 (2015).
Malmierca, M. S., Sanchez-Vives, M. V., Escera, C., & Bendixen, A. Neuronal adaptation, novelty detection and regularity encoding in audition. Frontiers in systems neuroscience, 8 (2014).
Tiitinen, H., May, P., Reinikainen, K. & NÃ, R. Attentive novelty detection in humans is governed by pre-attentive sensory memory. Nature 372(6501), 90 (1994).
Javit, D. C., Steinschneider, M., Schroeder, C. E., Vaughan, H. G. & Arezzo, J. C. Detection of stimulus deviance within primate primary auditory cortex: intracortical mechanisms of mismatch negativity (MMN) generation. Brain research 667(2), 192–200 (1994).
Haenschel, C., Vernon, D. J., Dwivedi, P., Gruzelier, J. H. & Baldeweg, T. Event-related brain potential correlates of human auditory sensory memory-trace formation. Journal of Neuroscience 25(45), 10494–10501 (2005).
Sams, M., Paavilainen, P., Alho, K. & Näätänen, R. Auditory frequency discrimination and event-related potentials. Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section 62(6), 437–448 (1985).
Näätänen, R., Paavilainen, P., Rinne, T. & Alho, K. The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clinical neurophysiology 118(12), 2544–2590 (2007).
Taaseh, N., Yaron, A. & Nelken, I. Stimulus-specific adaptation and deviance detection in the rat auditory cortex. PLoS One 6(8), e23369 (2011).
Costa-Faidella, J., Grimm, S., Slabu, L., Díaz-Santaella, F., & Escera, C. Multiple time scales of adaptation in the auditory system as revealed by human evoked potentials. Psychophysiology 48(6), 774–783.
Costa-Faidella, J., Baldeweg, T., Grimm, S. & Escera, C. Interactions between “what” and “when” in the auditory system: temporal predictability enhances repetition suppression. Journal of Neuroscience 31(50), 18590–18597 (2011).
Kaser, M. et al. Oscillatory underpinnings of mismatch negativity and their relationship with cognitive function in patients with schizophrenia. PloS one 8(12), e83255 (2013).
Baldeweg, T. & Hirsch, S. R. Mismatch negativity indexes illness-specific impairments of cortical plasticity in schizophrenia: a comparison with bipolar disorder and Alzheimer’s disease. International Journal of Psychophysiology 95(2), 145–155 (2015).
MartÃnez, A. et al. Neural oscillatory deficits in schizophrenia predict behavioral and neurocognitive impairments. Frontiers in human neuroscience, 9 (2015).
Lakatos, P., Schroeder, C. E., Leitman, D. I. & Javitt, D. C. Predictive suppression of cortical excitability and its deficit in schizophrenia. Journal of Neuroscience 33(28), 11692–11702 (2013).
Lee, M. et al. Neural mechanisms of mismatch negativity dysfunction in schizophrenia. Molecular Psychiatry, (2017).
Escera, C. & Malmierca, M. S. The auditory novelty system: an attempt to integrate human and animal research. Psychophysiology 51(2), 111–123 (2014).
Nelken, I. Stimulus-specific adaptation and deviance detection in the auditory system: experiments and models. Biological cybernetics 108(5), 655–663 (2014).
Herrmann, B., Henry, M. J., Fromboluti, E. K., McAuley, J. D. & Obleser, J. Statistical context shapes stimulus-specific adaptation in human auditory cortex. Journal of neurophysiology 113(7), 2582–2591 (2015).
Nelken, I. & Ulanovsky, N. Mismatch negativity and stimulus-specific adaptation in animal models. Journal of Psychophysiology 21(3–4), 214–223 (2007).
Javitt, D. C., Steinschneider, M., Schroeder, C. E. & Arezzo, J. C. Role of cortical N-methyl-D-aspartate receptors in auditory sensory memory and mismatch negativity generation: implications for schizophrenia. Proceedings of the National Academy of Sciences 93(21), 11962–11967 (1996).
Farley, B. J., Quirk, M. C., Doherty, J. J. & Christian, E. P. Stimulus-specific adaptation in auditory cortex is an NMDA-independent process distinct from the sensory novelty encoded by the mismatch negativity. Journal of Neuroscience 30(49), 16475–16484 (2010).
Duque, D., Malmierca, M. S. & Caspary, D. M. Modulation of stimulus-specific adaptation by GABAA receptor activation or blockade in the medial geniculate body of the anaesthetized rat. The Journal of physiology 592(4), 729–743 (2014).
Pérez-González, D., Hernández, O., Covey, E. & Malmierca, M. S. GABAA-mediated inhibition modulates stimulus-specific adaptation in the inferior colliculus. PLoS One 7(3), e34297 (2012).
May, P. J. & Tiitinen, H. Mismatch negativity (MMN), the deviance-elicited auditory deflection, explained. Psychophysiology 47(1), 66–122 (2010).
Nätänen, R. et al. The mismatch negativity (MMN)–“a unique window to disturbed central auditory processing in ageing and different clinical conditions. Clinical Neurophysiology 123(3), 424–458 (2012).
Friston, K. A theory of cortical responses. Philosophical Transactions of the Royal Society of London B: Biological Sciences 360(1456), 815–836 (2005).
Hamm, J. P. & Yuste, R. Somatostatin interneurons control a key component of mismatch negativity in mouse visual cortex. Cell reports 16(3), 597–604 (2016).
Amit, D. J. & Brunel, N. Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral cortex 7(3), 237–252 (1997).
Buonomano, D. V. & Merzenich, M. M. Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267(5200), 1028–1030 (1995).
Rabinovich, M., Huerta, R., & Laurent, G. Transient dynamics for neural processing. Science, 48–50 (2008).
Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural computation 14(11), 2531–2560 (2002).
Jaeger, H. The “echo state” approach to analysing and training recurrent neural networks-with anerratum note. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report 148(34), 13 (2001).
Sussillo, D. & Abbott, L. F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63(4), 544–557 (2009).
Varela, F., Lachaux, J. P., Rodriguez, E. & Martinerie, J. The brainweb: phase synchronization and large-scale integration. Nature reviews. Neuroscience 2(4), 229 (2001).
Singer, W. Cortical dynamics revisited. Trends in cognitive sciences 17(12), 616–626 (2013).
Tognoli, E. & Kelso, J. S. The metastable brain. Neuron 81(1), 35–48 (2014).
Herrmann, C. S., Murray, M. M., Ionta, S., Hutt, A. & Lefebvre, J. Shaping intrinsic neural oscillations with periodic stimulation. Journal of Neuroscience 36(19), 5328–5337 (2016).
Korn, H. & Faure, P. Is there chaos in the brain? II. Experimental evidence and related models. Comptes rendus biologies 326(9), 787–840 (2003).
Pachitariu, M. et al. Inhibitory control of shared variability in cortical networks. bioRxiv, 041103 (2016).
Harish, O. & Hansel, D. Asynchronous rate chaos in spiking neuronal circuits. PLoS computational biology 11(7), e1004266 (2015).
Sompolinsky, H., Crisanti, A. & Sommers, H. J. Chaos in random neural networks. Physical Review Letters 61(3), 259 (1988).
Skarda, C. A. & Freeman, W. J. How brains make chaos in order to make sense of the world. Behavioral and brain sciences 10(2), 161–173 (1987).
Pikovsky, A., Zaks, M., Rosenblum, M., Osipov, G. & Kurths, J. Phase synchronization of chaotic oscillations in terms of periodic orbits. Chaos: An Interdisciplinary Journal of Nonlinear Science 7(4), 680–687 (1997).
Bertschinger, N. & Natschläger, T. Real-time computation at the edge of chaos in recurrent neural networks. Neural computation 16(7), 1413–1436 (2004).
Legenstein, R. & Maass, W. Edge of chaos and prediction of computational performance for neural circuit models. Neural Networks 20(3), 323–334 (2007).
Ponzi, A., & Wickens, J. R. Temporal expectation in chaotic balanced networks. Society for Neuroscience Annual Meeting abstract, (2014).
Ponzi, A., & Wickens, J. R. How recurrent networks respond to complex stimulus sequences. Society for Neuroscience Annual Meeting abstract, (2015).
Ponzi, A. & Wickens, J. R. Optimal balance of the striatal medium spiny neuron network. PLoS computational biology 9(4), e1002954 (2013).
Hennequin, G., Vogels, T. P. & Gerstner, W. Optimal control of transient dynamics in balanced networks supports generation of complex movements. Neuron 82(6), 1394–1406 (2014).
Mazzoni, A. et al. Computing the local field potential (LFP) from integrate-and-fire network models. PLoS computational biology 11(12), e1004584 (2015).
Burrage, K. & Platen, E. Runge-Kutta methods for stochastic differential equations. (Online, 1990).
Ott, E. Chaos in dynamical systems. (Cambridge university press, 2002).
Geist, K., Parlitz, U. & Lauterborn, W. Comparison of different methods for computing Lyapunov exponents. Progress of theoretical physics 83(5), 875–893 (1990).
Manjunath, G. & Jaeger, H. Echo state property linked to an input: Exploring a fundamental characteristic of recurrent neural networks. Neural computation 25(3), 671–696 (2013).
Carroll, T. L. & Pecora, L. M. Synchronizing chaotic circuits. IEEE Transactions on circuits and systems 38(4), 453–456 (1991).
Pecora, L. M. & Carroll, T. L. Synchronization in chaotic systems. Physical review letters 64(8), 821 (1990).
Lajoie, G., Lin, K. K. & Shea-Brown, E. Chaos and reliability in balanced spiking networks with temporal drive. Physical Review E. 87(5), 052901 (2013).
Rajan, K., Harvey, C. D. & Tank, D. W. Recurrent network models of sequence generation and memory. Neuron 90(1), 128–142 (2016).
Laje, R. & Buonomano, D. V. Robust timing and motor patterns by taming chaos in recurrent neural networks. Nature neuroscience 16(7), 925–933 (2013).
Rubin, D. B., Van Hooser, S. D. & Miller, K. D. The stabilized supralinear network: a unifying circuit motif underlying multi-input integration in sensory cortex. Neuron 85(2), 402–417 (2015).
Ermentrout, G. B., & Terman, D. H. Mathematical foundations of neuroscience. (Springer Science & Business Media, 2010).
Haider, B., Häusser, M. & Carandini, M. Inhibition dominates sensory responses in awake cortex. Nature 493(7430), 97 (2013).
Mazzoni, A., Whittingstall, K., Brunel, N., Logothetis, N. K. & Panzeri, S. Understanding the relationships between spike rate and delta/gamma frequency bands of LFPs and EEGs using a local cortical network model. Neuroimage 52(3), 956–972 (2010).
Mazzoni, A., Panzeri, S., Logothetis, N. K. & Brunel, N. Encoding of naturalistic stimuli by local field potential spectra in networks of excitatory and inhibitory neurons. PLoS computational biology 4(12), e1000239 (2008).
Boccaletti, S., Kurths, J., Osipov, G., Valladares, D. L. & Zhou, C. S. The synchronization of chaotic systems. Physics reports 366(1), 1–101 (2002).
Heagy, J. F., Carroll, T. L. & Pecora, L. M. Desynchronization by periodic orbits. Physical Review E 52(2), R1253 (1995).
Pyragas, K. Weak and strong synchronization of chaos. Physical Review E, 54 ( 5 ), R4508 (1996).
Jaeger, H. & Haas, H. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004).
Mayer, N. M. Input-anticipating critical reservoirs show power law forgetting of unexpected input events. Neural computation.(2015).
Lukoševičius, M. & Jaeger, H. Reservoir computing approaches to recurrent neural network training. Computer Science Review 3(3), 127–149 (2009).
Schmidhuber, J., Wierstra, D., Gagliolo, M. & Gomez, F. Training recurrent networks by evolino. Neural computation 19(3), 757–779 (2007).
Goedeke, S., Schuecker, J. & Helias, M. Noise dynamically suppresses chaos in neural networks. arXiv preprint arXiv. 1603, 01880 (2016).
Wainrib, G. & Galtier, M. N. A local Echo State Property through the largest Lyapunov exponent. Neural Networks 76, 39–45 (2016).
Morcos, A. S. & Harvey, C. D. History-dependent variability in population dynamics during evidence accumulation in cortex. Nature neuroscience 19(12), 1672 (2016).
Rajan, K., Abbott, L. F. & Sompolinsky, H. Stimulus-dependent suppression of chaos in recurrent neural networks. Physical Review E 82(1), 011903 (2010).
Mirus, K. A. & Sprott, J. C. Controlling chaos in low-and high-dimensional systems with periodic parametric perturbations. Physical Review E 59(5), 5313 (1999).
Molgedey, L., Schuchhardt, J. & Schuster, H. G. Suppressing chaos in neural networks by noise. Physical review letters 69(26), 3717 (1992).
Jones, M. R., Moynihan, H., MacKenzie, N. & Puente, J. Temporal aspects of stimulus-driven attending in dynamic arrays. Psychological science 13(4), 313–319 (2002).
Lange, K. Brain correlates of early auditory processing are attenuated by expectations for time and pitch. Brain and cognition 69(1), 127–137 (2009).
Doherty, J. R., Rao, A., Mesulam, M. M. & Nobre, A. C. Synergistic effect of combined temporal and spatial expectations on visual attention. Journal of Neuroscience 25(36), 8259–8266 (2005).
Rimmele, J., Jolsvai, H. & Sussman, E. Auditory target detection is affected by implicit temporal and spatial expectations. Journal of cognitive neuroscience 23(5), 1136–1147 (2011).
Spiegler, A. et al. Modeling Brain Resonance Phenomena Using a Neural Mass Model. PLoS Computational Biology. 7(12), e1002298 (2011).
Lange, K. Can a regular context induce temporal orienting to a target sound? International Journal of Psychophysiology 78(3), 231–238 (2010).
Grimm, S., Escera, C., Slabu, L. & Costa-Faidella, J. Electrophysiological evidence for the hierarchical organization of auditory change detection in the human brain. Psychophysiology 48(3), 377–384 (2011).
Bendixen, A., SanMiguel, I. & Schröger, E. Early electrophysiological indicators for predictive processing in audition: a review. International Journal of Psychophysiology 83(2), 120–131 (2012).
Mathewson, K. E. et al. Making waves in the stream of consciousness: entraining oscillations in EEG alpha and fluctuations in visual awareness with rhythmic visual stimulation. Journal of cognitive neuroscience 24(12), 2321–2333 (2012).
Kizuk, S. A., & Mathewson, K. E. Power and phase of alpha oscillations reveal an interaction between spatial and temporal visual attention. Journal of Cognitive Neuroscience, (2017).
Makeig, S. et al. Dynamic brain sources of visual evoked responses. Science 295(5555), 690–694 (2002).
Ko, D. et al. Theta oscillation related to the auditory discrimination process in mismatch negativity: oddball versus control paradigm. Journal of Clinical Neurology 8(1), 35–42 (2012).
Fuentemilla, L. L., Marco-Pallarés, J., Münte, T. F. & Grau, C. Theta EEG oscillatory activity and auditory change detection. Brain research 1220, 93–101 (2008).
Ulanovsky, N., Las, L., Farkas, D. & Nelken, I. Multiple time scales of adaptation in auditory cortex neurons. Journal of Neuroscience 24(46), 10440–10453 (2004).
Pearce, M. T., Ruiz, M. H., Kapasi, S., Wiggins, G. A. & Bhattacharya, J. Unsupervised statistical learning underpins computational, behavioural, and neural manifestations of musical expectation. NeuroImage 50(1), 302–313 (2010).
Mill, R., Coath, M., Wennekers, T. & Denham, S. L. A neurocomputational model of stimulus-specific adaptation to oddball and Markov sequences. PLoS computational biology 7(8), e1002117 (2011).
Goudar, V. & Buonomano, D. V. A model of order-selectivity based on dynamic changes in the balance of excitation and inhibition produced by short-term synaptic plasticity. Journal of neurophysiology 113(2), 509–523 (2015).
May, P. J., Westö, J. & Tiitinen, H. Computational modelling suggests that temporal integration results from synaptic adaptation in auditory cortex. European Journal of Neuroscience 41(5), 615–630 (2015).
Lakatos, P. et al. The leading sense: supramodal control of neurophysiological context by attention. Neuron 64(3), 419–430 (2009).
Stefanics, G. et al. Phase entrainment of human delta oscillations can mediate the effects of expectation on reaction speed. Journal of Neuroscience 30(41), 13578–13585 (2010).
Spaak, E., de Lange, F. P. & Jensen, O. Local entrainment of alpha oscillations by visual stimuli causes cyclic modulation of perception. Journal of Neuroscience 34(10), 3536–3544 (2014).
Buzsáki, G. & Draguhn, A. Neuronal oscillations in cortical networks. Science 304(5679), 1926–1929 (2004).
Fries, P. A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends in cognitive sciences 9(10), 474–480 (2005).
Engström, D. A., Kelso, J. S. & Holroyd, T. Reaction-anticipation transitions in human perception-action patterns. Human movement science 15(6), 809–832 (1996).
Wallenstein, G. V., Kelso, J. S. & Bressler, S. L. Phase transitions in spatiotemporal patterns of brain activity and behavior. Physica D: Nonlinear Phenomena 84(3–4), 626–634 (1995).
Röschke, J. & Başar, E. The EEG is not a simple noise: strange attractors in intracranial structures in Dynamics of sensory and cognitive processing by the brain (203–216). (Springer, Berlin, Heidelberg, 1988).
El Boustani, S. & Destexhe, A. Brain dynamics at multiple scales: can one reconcile the apparent low-dimensional chaos of macroscopic variables with the seemingly stochastic behavior of single neurons? International Journal of Bifurcation and Chaos 20(06), 1687–1702 (2010).
Babloyantz, A., Salazar, J. M. & Nicolis, C. Evidence of chaotic dynamics of brain activity during the sleep cycle. Physics Letters A 111(3), 152–156 (1985).
Mayer-Kress, G. et al. Dimensional analysis of nonlinear oscillations in brain, heart, and muscle. Mathematical Biosciences 90(1–2), 155–182 (1988).
So, P. et al. Extracting unstable periodic orbits from chaotic time series data. Physical Review E 55(5), 5398 (1997).
Lesher, S. et al. Evidence for unstable periodic orbits in intact swimming lampreys, isolated spinal cords, and intermediate preparations. Annals of the New York Academy of Sciences 860(1), 486–491 (1998).
Schiff, S. J. et al. Controlling chaos in the brain. Nature 370(6491), 615–620 (1994).
Tass, P. et al. Detection of n: m phase locking from noisy data: application to magnetoencephalography. Physical review letters 81(15), 3291 (1998).
Barak, O., Sussillo, D., Romo, R., Tsodyks, M. & Abbott, L. F. From fixed points to chaos: three models of delayed discrimination. Progress in neurobiology 103, 214–222 (2013).
Sussillo, D. & Barak, O. Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural computation 25(3), 626–649 (2013).
Lazar, A., Pipa, G., & Triesch, J. SORN: a self-organizing recurrent neural network. Frontiers in computational neuroscience, 3 (2009).
Toyoizumi, T. & Abbott, L. F. Beyond the edge of chaos: Amplification and temporal integration by recurrent networks in the chaotic regime. Physical Review E 84(5), 051908 (2011).
Schrauwen, B., Büsing, L. & Legenstein, R. A. On computational power and the order-chaos phase transition in reservoir computing in Advances in Neural Information Processing Systems (1425–1432) (2009).
Boedecker, J., Obst, O., Lizier, J. T., Mayer, N. M. & Asada, M. Information processing in echo state networks at the edge of chaos. Theory in Biosciences 131(3), 205–213 (2012).
Tort, A. B., Komorowski, R., Eichenbaum, H. & Kopell, N. Measuring phase-amplitude coupling between neuronal oscillations of different frequencies. Journal of neurophysiology 104(2), 1195–1210 (2010).
Acknowledgements
I would like to thank Jeff Wickens for many valuable contributions to previous versions of this work.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing Interests
The authors declare that they have no competing interests.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Ponzi, A. Sensory Stream Adaptation in Chaotic Networks. Sci Rep 7, 16844 (2017). https://doi.org/10.1038/s41598-017-16478-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-017-16478-z
- Springer Nature Limited