# Some Joys and Trials of Mathematical Neuroscience

## Authors

- First Online:

- Received:
- Accepted:

DOI: 10.1007/s00332-013-9191-4

- Cite this article as:
- Holmes, P. J Nonlinear Sci (2014) 24: 201. doi:10.1007/s00332-013-9191-4

- 2 Citations
- 666 Views

## Abstract

I describe the basic components of the nervous system—neurons and their connections via chemical synapses and electrical gap junctions—and review the model for the action potential produced by a single neuron, proposed by Hodgkin and Huxley (HH) over 60 years ago. I then review simplifications of the HH model and extensions that address bursting behavior typical of motoneurons, and describe some models of neural circuits found in pattern generators for locomotion. Such circuits can be studied and modeled in relative isolation from the central nervous system and brain, but the brain itself (and especially the human cortex) presents a much greater challenge due to the huge numbers of neurons and synapses involved. Nonetheless, simple stochastic accumulator models can reproduce both behavioral and electrophysiological data and offer explanations for human behavior in perceptual decisions. In the second part of the paper I introduce these models and describe their relation to an optimal strategy for identifying a signal obscured by noise, thus providing a norm against which behavior can be assessed and suggesting reasons for suboptimal performance. Accumulators describe average activities in brain areas associated with the stimuli and response modes used in the experiments, and they can be derived, albeit non-rigorously, from simplified HH models of excitatory and inhibitory neural populations. Finally, I note topics excluded due to space constraints and identify some open problems.

### Keywords

AccumulatorAveragingCentral pattern generatorDecision makingBifurcationDrift-diffusion processMean field reductionOptimalityPhase reductionSpeed-accuracy tradeoff### Mathematics Subject Classification

34Cxx34Dxx37Cxx37N2560H1091E1092C20## 1 Introduction

Neuroscience is currently generating much excitement and some hyperbole (a recent review of a popular book referred to “neuromania” (McGinn 2013)). This is largely due to recent advances in experimental techniques and associated methods for analysis of “big data.” Striking examples are the CLARITY method that allows imaging of entire neural circuits and captures subcellular structural detail (Chung et al. 2013), and Connectomics (Seung 2012), which aims to determine neural connectivity and hence function at the cellular level. In announcing the US Brain Initiative in April 2013, President Obama spoke of “giving scientists the tools they need to get a dynamic picture of the brain in action and better understand how we think and how we learn and how we remember” (Insel et al. 2013). Such tools are not solely experimental (Abbott 2008). Computational approaches already play a substantial rôle in neuroscience (De Schutter 2008, 2009), and they are becoming more ambitious: the European Blue Brain project (Markram 2006) (http://bluebrain.epfl.ch/) proposes to simulate all the cells and most of the synapses in an entire brain, thereby hoping to “challenge the foundations of our understanding of intelligence and generate new theories of consciousness.”

In this article I have a more modest goal: to show how mathematical models and their analyses are contributing to our understanding of some small parts of brains and central nervous systems. I will describe how reductions of biophysically based models of single cells and circuits to low-dimensional dynamical systems can reveal mechanisms that might otherwise remain hidden in massive data analyses and computer simulations. In this regard mathematics does not merely enable numerical simulation and motivate experiments, it provides an analytical complement without which they can lose direction and lack explanatory power.

Mathematical treatments of the nervous system began in the mid 20th century. An early example is Norbert Wiener’s “Cybernetics,” published in 1948 and based on work with the Mexican physiologist Arturo Rosenblueth (Wiener 1948). Weiner introduced ideas from dissipative dynamical systems, symmetry groups, statistical mechanics, time series analysis, information theory, and feedback control. He also discussed the relationship between digital computers (then in their infancy) and neural circuits, a theme that John von Neumann subsequently addressed in a book published in the year following his death (von Neumann 1958). While developing one of the first programmable digital computers (JONIAC, built at the Institute for Advanced Study in Princeton in the late 1940s), von Neumann “tried to imitate some of the known operations of the live brain” (von Neumann 1958, Preface). In developing cybernetics, Wiener drew on von Neumann’s earlier works in analysis, ergodic theory, computation and game theory, as well his own studies of Brownian motion (a.k.a. Wiener processes). Some of these ideas appear in Sect. 4 of the present paper.

These books (Wiener 1948; von Neumann 1958) were directed at the brain and nervous system *in toto*, although much of the former was based on detailed studies of heart and leg muscles in animals. The first cellular-level mathematical model of a single neuron was developed in the early 1950s by the British physiologists Hodgkin and Huxley (1952d). This work, which won them the Nobel Prize in Physiology in 1963, grew out of a long series of experiments on the giant axon of the squid *Loligo* by themselves and others, as noted in Sect. 2 (also see Huxley’s obituary (Mackey and Santillán 2013)). Since their pioneering work, mathematical neuroscience has grown into a subdiscipline, served worldwide by courses long and short (e.g. Kopell et al. 2009, Whittington et al. 2009), textbooks (e.g. Wilson 1999, Dayan and Abbott 2001, Keener and Sneyd 2009, Ermentrout and Terman 2010, Gabbiani and Cox 2010), and review articles (recent examples include Wang 2010, Kopell et al. 2010, McCarthy et al. 2012, Deco et al. 2013). The number of mathematical models must now exceed the catalogue of brain areas by several orders of magnitude. I can present but few examples here, inevitably biased toward my own interests.

Models can be of two broad types: *empirical* (also called *descriptive* or *phenomenological*), or *mechanistic*. The former ignore (possibly unknown) anatomical structure and physiology, and seek to reproduce input–output or stimulus–response relationships of the system under study. Mechanistic models attempt to describe structure and function in some detail, reproducing observed behaviors by appropriate choice of model components and parameters and thereby revealing mechanisms responsible for those behaviors. Models can reside throughout a continuum from molecular to organismal scales, and many are not easily classifiable, but one common feature is nonlinearity. Unlike much of physical science and engineering, biology is inherently nonlinear. For example, the functions describing ion channels opening in cells in response to transmembrane voltage increase or characterizing neural firing rate dependence on input current are typically bounded above and below, and often modeled by sigmoids.

The first part of this article covers mechanistic models, beginning in Sect. 2 with the Hodgkin–Huxley (HH) equations for the generation and propagation of a single action potential (AP, or spike); it then discusses dimensional reductions that are easier to analyze and extensions of HH to describe neurons that emit bursts of spikes, and introduces models for synapses. Section 3 considers small neural circuits found in central pattern generators for locomotion, and shows how HH models of them can be simplified to phase oscillators. While mathematical methods such as averaging and dimensional reduction via time scale separation are used to simplify coupled sets of HH equations in these cases, the models are all based on cellular biophysiology.

In Sect. 4 I change scale to introduce empirical models of activity in brain areas that may contain millions of neurons. Focusing on simple binary decisions in which a noisy stimulus must be identified, I show how a pair of competing nonlinear stochastic accumulators can model the integration of noisy evidence toward a threshold, triggering a response. Linearizing and considering a limiting case, this model reduces to a scalar drift-diffusion (DD) process, which is in turn a continuum limit of the sequential probability ratio test (SPRT). The SPRT is known to be optimal in that it renders decisions of specified accuracy in the shortest possible time. The tractability of the DD process allows one to derive an explicit optimal speed-accuracy tradeoff, against which human and animal behavior can be assessed. Behavioral experiments reveal both approximations to and deviations from optimality, and further analyses of the model and data suggest three potential reasons for the latter: avoidance of errors, poor time estimation, and minimization of the cost of cognitive control.

Section 5 sketches computations based on mean-field theory which start with pools of spiking neurons having distinct “tuning curves” that respond differently to the two stimuli and lead to stochastic accumulator models like those of Sect. 4. While this is neither rigorous nor as complete as the reduction methods of Sect. 3, it provides further support for such models by connecting them to simplified neuron models of HH type. It also suggests a fourth, physiological reason for suboptimality, namely, nonlinear dynamics. Section 6 contains a brief discussion, provides references some of the many topics omitted due to space limitations, and notes some open problems.

## 2 The Components: Neurons, Synapses and the Hodgkin–Huxley Equations

The basic components of the nervous system are neurons: electrically active cells that can generate and propagate signals over distance. These signals are *action potentials* (APs, or spikes): voltage fluctuations of \(\mathcal{O}(100)\) mV, each lasting 1–5 msec, across the cell membrane. Structurally, neurons come in many shapes and sizes, but all share the basic features of a *soma* or cell body, *dendrites*: multiply branching extensions that receive signals from other neurons, and an *axon*, a cable-like extension that may also be branched, along which APs propagate to other neurons.^{1} The connections between axons and dendrites are called *synapses*, and they may be electrical, communicating voltage differences, or chemical, releasing neurotransmitters upon the arrival of an AP from the presynaptic cell. Functionally, neurons are either *excitatory* or *inhibitory*, tending to increase or depress the transmembrane voltage of postsynaptic cells to which they connect. In this section we describe models for single neurons and for synapses.

### 2.1 The Hodgkin–Huxley Equations

Before starting it is important to know that ionic transport across cell membranes occurs through ion-specific channels and pores. It is driven passively by concentration and potential differences and by active pumps that exchange sodium for potassium and remove calcium from the cell. The Nernst–Planck equation, from biophysics, relates transmembrane flux, concentration and potential differences for each ionic species, and allows one to compute equilibrium conditions consistent with zero flux (Keener and Sneyd 2009, Sect. 2.6). At this resting potential, sodium concentrations are higher outside the cell than inside, while potassium concentrations are higher inside it.

^{+}) flowing in from their higher extracellular concentration, and that the outward current was due to potassium ions (K

^{+}) leaving the cell. They also included a passive leak current, due primarily to chloride ions (Cl

^{−}). These three currents appear in (1a) as \(\bar{g}_{\mathrm{Na}}m^{3} h(v-v_{\mathrm{Na}})\), \(\bar{g}_{\mathrm{K}} n^{4}(v-v_{\mathrm{K}})\), and \(\bar{g}_{\mathrm{L}}(v-v_{\mathrm{L}})\) μA/cm

^{2}respectively, along with an externally applied current

*I*, corresponding to Kirchhoff’s law which describes the rate of change of transmembrane voltage

*v*in the circuit of Fig. 1. The barred parameters in the sodium and potassium conductances denote constant values that multiply time-dependent functions of

*n*(

*t*),

*m*(

*t*) and

*h*(

*t*) to form “dynamical” conductances \(\bar{g}_{\mathrm{K}} n^{4}\) and \(\bar{g}_{\mathrm{Na}}m^{3} h\). Voltage dependencies of the ion channels are also characterized by the Nernst

*reversal potentials*

*v*

_{K}=−12 mV,

*v*

_{Na}=115 mV and

*v*

_{L}=10.6 mV; as the name suggests, the currents change direction as

*v*crosses these values.

The rôle of each ionic species was revealed by experiments in which all but one active species were removed and the transmembrane voltage held constant and then stepped from one value to another, while the current *I*(*t*) required to maintain that voltage was recorded. This *voltage clamp* method determined each ionic conductance as a function of voltage. Moreover, by examining transient responses following steps of given sizes, Hodgkin and Huxley could fit sigmoids to the six functions *α*_{m}(*v*),…*β*_{h}(*v*) across the relevant voltage range. (Note that (1b–1d) are linear for fixed *v*.) They postulated a single gating variable *n*(*t*)∈[0,1] to describe potassium activation and noted that while conductance dropped sharply from higher levels following a downward step in *v*, it rose gently from zero after a step increase. This led to the fourth power in the potassium conductance \(\bar{g}_{\mathrm{K}} n^{4}\) (cf. Hodgkin and Huxley (1952d, Fig. 1)). Sodium dynamics proved more complicated, involving a rapid increase in conductance followed by slower decrease (Hodgkin and Huxley 1952d, Figs. 2 and 3), a non-monotonic response that required two variables *m*(*t*),*h*(*t*)∈[0,1] to describe activation and deactivation, producing the *m*^{3}*h* term in \(\bar{g}_{\mathrm{Na}}m^{3} h\).

*α*and

*β*functions are

^{2}. To emphasize the equilibrium potential

*n*

_{∞}(

*v*) at which

*n*remains constant, and the time scale

*τ*

_{n}(

*v*), the gating equations may be rewritten as follows:

*m*and

*h*. See Hodgkin and Huxley 1952d, Fig. 6 for graphs of

*α*

_{n}(

*v*),

*β*

_{n}(

*v*),

*n*

_{∞}(

*v*), etc.

- (1)
rapid increase in

*v*and*m*as sodium conductance rises towards the sodium Nernst potential in a brief*depolarized*AP. - (2)
At higher voltages

*h*decreases, lowering sodium conductance, and*n*increases, increasing potassium conductance and driving*v*down towards the potassium potential. - (3)
During the ensuing

*refractory*period*m*falls quickly to its resting value, but*n*stays high and*h*remains low because their equations have longer time constants, thus holding*v*down (*hyperpolarized*) and preventing APs. - (4)
As

*n*and*h*return to values that allow an AP, the cell enters its recovery phase.

The variables *m*,*n* and *h* can be interpreted as probabilities that gates in the corresponding ionic channels are open, and the exponents in the conductances as the numbers of gates that must be open. It is now known that potassium channels contain tetrameric structures that must cooperate for ions to flow, in agreement with the empirical *n*^{4} fit.

### 2.2 Two-Dimensional Reductions of HH

*m*(

*t*) changes relatively rapidly because its timescale

*τ*

_{m}=1/(

*α*

_{m}+

*β*

_{m})≪

*τ*

_{n},

*τ*

_{h}in the relevant voltage range (cf. (2–5) and Fig. 2). We may therefore assume that it is almost always equilibrated so that \(\dot{m} \approx0\), implying that

*n*(

*t*) and

*h*(

*t*) are approximately anti-correlated in that throughout the AP and recovery phase their sum remains almost constant:

*h*+

*n*≈

*a*. Thus

*m*and

*h*may be replaced by

*m*

_{∞}(

*v*) and

*a*−

*n*and dropped as state variables, reducing the system to

*I*=0, features a sink near

*v*=0 along with a saddle and a source. In the right-hand plot, for

*I*=15, a limit cycle has appeared. Figure 3 displays the spiking threshold

*v*

_{th}at a local minimum of the \(\dot{v} = 0\) nullcline. When the leftmost fixed point lies to the left of

*v*

_{th}it is stable, as for

*I*=0. In this

*excitable*state spikes can occur due to perturbations that push

*v*past

*v*

_{th}, but absent further perturbations the state returns to the sink. When the fixed point moves to the right of

*v*

_{th}(

*I*=15) it loses stability and solutions repeatedly cross threshold, yielding periodic spiking in the manner of a

*relaxation oscillator*(Hirsch et al. 2004).

Here, to illustrate the rich dynamics that a planar system with nonlinear nullclines can exhibit, we have chosen *I* values for which (7a)–(7b) has three fixed points; for others, it has only one (as do the original H–H equations) (Guckenheimer and Holmes 1983, Sect. 2.1).

*v*, but the relative durations of the depolarized and hyperpolarized episodes are approximately equal, unlike the HH dynamics of Fig. 2. The reason for this becomes clear when we examine the nullclines shown in Fig. 4.

First note that the basic behavior of the reduced HH equations (7a)–(7b) is preserved: for low *I* (on the left), there is a stable sink, while for higher *I* (on the right), there is a stable limit cycle. However, unlike Fig. 3, the cubic \(\dot{v} = 0\) nullcline is symmetric about *v*=0, so that the slow orbit segments are similar in duration. Moreover, since the slope of the \(\dot{r} = 0\) nullcline (1.25) exceeds the maximum slope of the \(\dot{v} = 0\) nullcline (1), (8a)–(8b) has a single fixed point for all *I*. It can be shown that this loses stability in a supercritical Hopf bifurcation Wilson (1999, Sect. 8.3) as *I* increases, creating the limit cycle, and that the limit cycle vanishes in a second Hopf bifurcation at a higher *I*, where the fixed point restabilizes, corresponding to persistent depolarization of the neuron. This bifurcation sequence also occurs for the full HH equations, but in that case the first Hopf bifurcation is *subcritical*, in which an unstable limit cycle converges on the stable hyperpolarized fixed point. The unstable cycle appears in a saddle-node bifurcation of periodic orbits (Guckenheimer and Holmes 1983, Sect. 3.4), along with a stable limit cycle at a slightly lower *I*. The FN simplification loses both quantitative and fine qualitative detail, but is nonetheless popular among applied mathematicians due to its analytical tractability.

### 2.3 Integrate-and-Fire Models

*v*exceeds a threshold

*v*

_{th}, followed by reset to a resting potential

*v*

_{r}, possibly after a fixed refractory period. The model was first introduced in 1907 in studying the sciatic nerve of leg muscles in frogs (Guckenheimer and Holmes 1983, Sect. 3.5), but further studies came decades later (Lapicque 1907; Brunel and van Rossum 2007). The linear IF model retains only the leak and applied currents of (1a) and is written

*δ*(

*t*−

*t*

_{k}) is inserted and voltage reset if

*v*reaches

*v*

_{th}at

*t*=

*t*

_{k}, making (9) a

*hybrid dynamical system*(Stein 1965; Knight 1972a,b). Without resets, all solutions would approach the sink at \(v_{\mathrm{ss}} = v_{\mathrm{L}} + I / \bar{g}_{\mathrm{L}}\) as they do for

*v*

_{ss}≤

*v*

_{th}, but if

*v*

_{ss}>

*v*

_{th}repetitive spiking occurs as shown in Fig. 5.

The decelerating subthreshold voltage profile of the linear IF model differs from the acceleration characteristic of more realistic models (cf. Fig. 2(a)). This can be repaired by using nonlinear functions, common choices being quadratic (Back et al. 1993; Guckenheimer and Johnson 1995) or exponential (Ermentrout and Kopell 1986; Latham et al. 2000; Latham and Brunel 2003). The reset upon reaching threshold prevents orbits escaping to infinity in finite time. See (Foucaud-Trocme et al. 2003; Foucaud-Trocme and Brunel 2005) for comparisons and Izhikevich (2004) for model reviews.

Interspike intervals and hence firing rates are easily computed for scalar IF models, but it is difficult to obtain explicit results for all but the simplest multi-unit circuits because one must compute threshold arrival times for every cell and paste together the intervening orbit segments to obtain the flow map. Nonetheless, IF models are in wide use for large-scale numerical simulations of cortical circuits; an example appears below in Sect. 5.

### 2.4 A Model for Bursting Neurons

^{++}) release introduces long time scales that can interact with the medium and short timescales of periodic APs to produce bursts of spikes followed by refractory periods. This is characteristic of motoneurons, and more generally of cells involved in generating rhythmic activity Burkitt (2006a,b). Let

*c*(

*t*) denote a slow gating variable governed by

*n*, as in (7a)–(7b). The voltage equation analogous to (7a) now contains an ionic current depending on

*c*, but since \(\dot{c} = \mathcal{O}(\epsilon)\), we may appeal to perturbation methods (Chay and Keizer 1983; Sherman et al. 1988) and regard

*c*as a “frozen” parameter. Changes in

*c*can cause bifurcations in the two-dimensional (

*v*,

*n*) system that lead from quiescence (a stable fixed point), to periodic spiking, as in Fig. 3, and the slow dynamics of (10) can drive the full system periodically between these states.

*c*the (

*v*,

*n*) system has a source surrounded by a stable limit cycle, and for high

*c*a single sink, which continues to the lower saddle-node bifurcation point. The upper saddle node creates the source and a saddle point. Below the \(\dot{c} = 0\) nullcline,

*c*decreases, moving the state along the lower, stable branch of equilibria during the refractory period. At the lower saddle node, the state jumps to the limit cycle, which lies above \(\dot{c} = 0\), so that

*c*now increases. However, before reaching the upper saddle-node the limit cycle collides with the saddle and vanishes in a homoclinic loop bifurcation Ghigliazza and Holmes (2004b, Fig. 11).

More on bursting mechanisms and their classification via the fast subsystem’s bifurcations as the slow *c* variable drifts back and forth can be found in Ghigliazza and Holmes (2004b) and (Guckenheimer and Holmes 1983, Sect. 6.1).

### 2.5 Neural Connectivity: Synapses and Gap Junctions

*gap junctions*, small protein structures where the cells make close contact. They are generally modeled as linear resistors, so that the voltage equations for cells

*i*and

*j*become

*I*

_{i,ion}(…) denotes the internal ionic currents of cell

*i*. Electrical synapses appear in escape reflexes: e.g., the tail-flip giant neuron in goldfish connects to sensors via a gap junction, allowing rapid responses to threatening stimuli. Gap junctions can also connect groups of small cells, causing them to spike together, as in the synchronization of ink release in certain marine snails.

Chemical synapses involve the release of neurotransmitter from a presynaptic neuron and its reception at a postsynaptic neuron. The cells are separated by synaptic clefts between *boutons*, protrusions on the presynaptic axon that contain vesicles of neurotransmitter molecules, and postsynaptic dendritic spines. After an AP arrives, calcium influx causes vesicles to fuse with the cell membrane and release their contents, which diffuse across the synaptic cleft to reach postsynaptic receptors that open ion channels and generate excitatory or inhibitory postsynaptic potentials (EPSPs, IPSPs). A single EPSP is usually too small to drive a hyperpolarized postsynaptic cell across threshold, but multiple EPSPs can evoke a spike. IPSPs drive its voltage down to delay or prevent spiking.

The amino acids acetylcholine (ACh), glutamate and *γ*-aminobutyric acid (GABA) are major neurotransmitters, as are the monoamines dopamine (DA), norepinephrine (NE) and serotinin (SE). Their effects are determined by ionotropic and metabotropic receptors; the former open channels quickly, the latter act via a slower cascade of messengers. GABA activates both ionotropic and metabotropic inhibitory receptors and 2-amino-3-hydroxy-5-methyl-isoxazolepropanoic acid (AMPA) and N-methyl-D-aspartic acid (NMDA) are excitatory ionotropic receptor types for glutamate, with AMPA exhibiting significantly faster activation and deactivation than NMDA.

Chemical synapses are considerably slower than gap junctions, but allow more complicated behavior. They exhibit *synaptic plasticity* which is crucial to learning, since it allows connections among cells (and hence brain areas) to weaken or strengthen in response to experience. They can amplify signals by releasing large numbers of neurotransmitter molecules, which open many ion channels and thereby depolarize a much larger cell than is possible with gap junctions. Neurotransmitter and receptor time constants span two orders of magnitude and their interaction can lead to reverberations that sustain neural activity in working memory: see Wilson (1999, Chap. 10) and Sect. 5 below.

*P*

_{s}, of postsynaptic ion channels being open. This process, and the closure of channels as the transmitter unbinds from receptors, can be modeled like the gating variables in the HH equations:

*α*

_{s}and

*β*

_{s}determine the rates at which channels open and close, effectively encoding the neurotransmitter time scales: see Keener and Sneyd (2009, Chap. 9) and Wang (1999, 2010), Wong and Wang (2006). Opening is typically faster than closure, so

*α*

_{s}≫

*β*

_{s}, and

*β*

_{s}is often assumed constant, but

*α*

_{s}depends on neurotransmitter concentration in the synaptic cleft, and thus on the presynaptic voltage

*v*

_{i}. Again, a sigmoid provides an acceptable model:

*C*

_{NT,max}represents the maximal neurotransmitter concentration, \(v^{\mathrm{pre}}_{\mathrm{syn}}\) sets the voltage at which vesicles begin to open,

*k*

_{pre}sets the “sharpness” of the switch, and the scale factor \(\bar{\alpha}_{\mathrm{s}}\) allows one to lump the effects of all the synapses between the two cells.

*j*due to an AP in cell

*i*involves a reversal potential, \(v^{\mathrm{post}}_{\mathrm{syn}}\), and is scaled by a maximal conductance, \(\bar{g}_{\mathrm{syn}}\), so that the voltage equation for the postsynaptic cell is

*v*

_{i}, acting via (13), makes

*α*

_{s}(

*v*

_{i}) behave like a rectangular pulse with duration of the AP and height \(\bar{\alpha}_{\mathrm{s}} C_{\mathrm{NT},\mathrm{max}}\). Equation (12) may then be solved explicitly during and following the AP and the resulting exponentials matched to produce a piecewise-smooth rising and falling pulse. Alternatively, this may be approximated as a sum of two exponentials or as an “alpha” function:

*P*

_{max}at

*t*=

*τ*

_{s}, and then decays back to zero with time constant

*τ*

_{s}. For further discussions of synaptic mechanisms, see (Destexhe et al. 1999; Dayan and Abbott 2001), Ghigliazza and Holmes (2004a) and Dayan and Abbott (2001).

As noted in Sect. 2.1, Hodgkin and Huxley modeled the propagation of APs along an axon by adding a diffusive spatial term to (1a) Keener and Sneyd (2009, Chap. 8). More complex geometries including branching dendrites and axons are often represented by multiple compartments (sometimes in the hundreds). This leads to large sets of ODEs for each cell, but allows one to capture subtle effects that influence intercellular communication. Not only do dendrite sizes affect their conductances Johnston and Wu (1997, Chaps. 11–15) and transmission delays occur in dendritic trees, but EPSPs arriving at nearby synapses interact to produce less excitation than their sum predicts (Hodgkin and Huxley 1952d). Nonlinear interactions due to shunting inhibition that changes membrane conductance can also reduce excitatory currents (Rall 1959). See (Rall et al. 1967) for reviews of such “dendritic computations.”

## 3 Central Pattern Generators and Phase Reduction

Central pattern generators (CPGs) are networks in the spinal cords of vertebrates and invertebrate thoracic ganglia, capable of generating muscular activity in the absence of sensory feedback (Rall 1964), cf London and Häusser (2005). CPGs drive many rhythmic actions, including locomotion, scratching, whisking (e.g. in rats), moulting (in insects), chewing and digestion. The stomato-gastric ganglion in lobster is perhaps the best-understood example (Cohen et al. 1988; Getting 1988; Pearson 2000; Marder 2000; Ijspeert 2008). Experiments are typically done in isolated *in vitro* preparations, with sensory and higher brain inputs removed (Wilson 1999, Chaps. 12–13), but it is increasingly acknowledged that an integrative approach, including muscle and body-limb dynamics, environmental reaction forces and proprioceptive feedback, is needed to fully understand their function (Marder and Bucher 2007). Indeed, without reaction forces, animals would go nowhere! CPGs nonetheless provide examples of neural networks capable of generating interesting behaviors, but small enough to allow the study of detailed biophysically based models.

After introducing a phase reduction method that is particularly useful for such systems and applies to any ODE with a hyperbolic limit cycle and showing how it leads to systems of coupled phase oscillators via averaging theory, I describe a model of an insect CPG. For an early review of CPG models that use phase reduction and averaging, see (Cohen et al. 1988; Grillner 1999).

### 3.1 Phase Reduction and Phase Response Curves

*Γ*

_{0}of period

*T*

_{0}for

*ϵ*=0, and let \({\bf{x}}_{0}(t)\) denote a solution lying in

*Γ*

_{0}. Invariant manifold theory Hoppensteadt and Izhikevich (1997, Chap. 9) guarantees that, in a neighborhood

*U*of

*Γ*

_{0}, the state space splits into a phase variable

*φ*∈[0,2

*π*) along the closed curve

*Γ*

_{0}and a smooth foliation of transverse

*isochrons*Ermentrout and Terman (2010, Chap. 8). Each isochron is an (

*n*−1)-dimensional manifold

*M*

_{φ}with the property that any two solutions starting on the same leaf \(M_{\varphi_{i}}\) are mapped by the flow to another leaf \(M_{\varphi_{j}}\) and hence approach

*Γ*

_{0}with equal asymptotic phases as

*t*→∞: see Fig. 7. For points \({\bf{x}}\in U\), phase is therefore defined by a smooth function \(\varphi({\bf{x}})\) and the leaves

*M*

_{φ}⊂

*U*are labeled by the inverse function \({\bf{x}}(\varphi)\). Moreover, this structure persists for small

*ϵ*>0, so

*Γ*

_{0}perturbs to a nearby limit cycle

*Γ*

_{ϵ}.

*ϵ*=0:

*ϵ*≪1) allows approximation of their effects by evaluating \({\bf{g}}({\bf{x}}, \ldots)\) along

*Γ*

_{0}.

*phase response curve*(PRC) describes how an impulsive perturbation advances or retards the next spike as a function of the phase during the cycle at which it acts. PRCs may be calculated using a finite-difference approximation to the derivative:

*V*→

*V*+Δ

*V*at \({\bf{x}}^{*} \in\varGamma_{0}\): see Fig. 7. PRCs may also be found from adjoint equations (Guckenheimer 1975).

### 3.2 Weak Coupling, Averaging and Half-Center Oscillators

*half-center oscillator*: a pair of units, often identical and hence bilaterally (or reflection-) symmetric and sometimes each containing several neurons, connected via mutual inhibition to produce an alternating rhythm Brown et al. (2004). See Brown et al. (2004) and (Ermentrout and Terman 2010, Chap. 8) for examples. Phase reduction provides a simple expression of this architectural subunit, which can be written as a system on the torus:

*ϵδ*

_{j}are allowed and the \(\mathcal{O}(\epsilon^{2})\) terms neglected. Transformation to slow phases

*ψ*

_{i}=

*φ*

_{i}−

*ω*

_{0}

*t*removes the common frequency

*ω*

_{0}and puts (20a)–(20b) in a form to which the averaging theorem for periodically forced ODEs can be applied (Ermentrout and Terman 2010, Sect. 9.6):

*T*

_{0}, the averages of the terms on the RHS of (21a)–(21b) are

*τ*=

*ψ*

_{j}+

*ω*

_{0}

*t*, so that \(dt = \frac{d \tau}{\omega_{0}} = \frac{T_{0} \, \mathrm{d} \tau}{2 \pi}\), and using the fact that the

*H*

_{i}are 2

*π*-periodic, the integral of (22) becomes

*π*-periodic and depend only on phase difference

*θ*=

*ψ*

_{1}−

*ψ*

_{2}. Equations (24) may therefore be subtracted to yield

For mutually symmetric coupling between identical units, *h*_{2}(*φ*_{2},*φ*_{1})=*h*_{1}(*φ*_{1},*φ*_{2}) in (20a)–(20b). Integration preserves this symmetry under permutation of *φ*_{1} and *φ*_{2}, so that the averaged functions satisfy \(\overline{H_{2}(-\theta)} = \overline{H_{1}(\theta)} \stackrel{\rm{def}}{=}\overline{H(\theta )}\). In this case, since *H* is 2*π*-periodic, \(G(\pi) = \overline{H(\pi)} - \overline{H(-\pi)} = \overline{H(\pi)} - \overline{H(\pi)} = 0\) and \(G(0) = \overline{H(0)} - \overline{H(0)} = 0\). Equation (25) therefore has fixed points at *θ*=0,*π*, corresponding to in-phase and anti-phase solutions, *regardless of the precise form of*\(\overline{H}\). Moreover, *G* is odd and its derivative *G*′(*θ*) is even (see Fig. 9 below). Additional fixed points *θ*^{e} such that *G*(*θ*^{e})=0 may also exist, depending on \(\overline{H}\). Nonsymmetric pairs generally do not have exact in- and anti-phase solutions.

Under the averaging theorem Hill et al. (2001), Daun-Gruhn et al. (2009), Doloc-Mihu and Calabrese (2011), hyperbolic fixed points of (24) correspond to *T*_{0}-periodic orbits of the original system (20a)–(20b). Since *θ*=*ψ*_{1}−*ψ*_{2}, fixed points *θ*^{e} of (25) appear as circles in the toroidal phase space of (24), and their linearization necessarily has a zero eigenvalue with eigenvector \((1,1)^{\rm{T}}\). This lack of hyperbolicity derives from the transformation *ψ*_{i}=*φ*_{i}−*ω*_{0}*t*, so the circles are *T*_{0}-periodic orbits in the original *φ*_{i} variables (note that *φ*_{1}−*φ*_{2}=*θ*^{e}). Provided that the other eigenvalue is nonzero, with eigenvector transverse to \((1,1)^{\rm{T}}\), it follows that the original system has a periodic orbit whose phases maintain the difference *θ*^{e} to \(\mathcal{O}(\epsilon)\). The dynamics are necessarily only neutrally stable to perturbations that equally advance or retard the phases of both units.

### 3.3 A CPG Model for Insect Locomotion

*Periplaneta americana*(Guckenheimer and Holmes 1983, Sect. 4.1–2). It uses bursting neuron and synapse models of the types described in Sects. 2.4 and 2.5 for which PRCs and averaged coupling functions were computed numerically (Guckenheimer and Holmes 1983, Sect. 4.1–2) to derive a phase reduced model. The network, including motoneurons, is shown in Fig. 8.

Cockroaches run over much of their speed range with a double tripod gait, in which left front, left rear and right middle legs (the L tripod) alternate with right front, right rear and left middle legs (the R tripod) to provide stance support. Motoneurons activating depressor and extensor muscles that drive the power stroke during stance are alternately active in the L and R tripods, and neighboring legs on the same side (ipsilateral) and across the body (contralateral) operate approximately in anti-phase. In Fig. 8 cells 1, 2, 3 drive the L tripod and 4, 5, 6 drive the R tripod. Extensors spike persistently to support the animal’s weight when standing still: they must be deactivated to swing the leg in running; in contrast, flexors must shut off during stance. As proposed in (Ghigliazza and Holmes 2004a), during its burst a single CPG interneuron can simultaneously inhibit an extensor and excite a flexor; during the interneuron’s refractory phase the extensor can resume spiking and the excitable flexor remain hyperpolarized and inactive. The model simplifies by allowing excitatory and inhibitory synapses on the same axon; in reality at least one disynaptic path would be necessary.

Little is known about architecture and neuron types in the cockroach, but representation of each leg unit by a single bursting cell, as in Fig. 8, is certainly minimal. For example, hemisegments of lamprey spinal cord each contain three different cell types as well as motoneurons Ghigliazza and Holmes (2004b,a), and in stick insects separate oscillators with multiple interneurons have been identified for each joint on a single leg Kukillaya et al. (2009).

*H*(−

*θ*) and

*H*(

*θ*): Fig. 9. Their subtraction produces an odd function

*G*(

*θ*) with zeroes at

*θ*=0 and

*π*, as noted in Sect. 3.2, that is also remarkably close to a simple sinusoid, as assumed in an earlier phase oscillator model for lamprey CPG Kukillaya et al. (2009), although this was not justified for some 25 years Pearson (1972), Pearson and Iles (1973).

*ψ*

_{1}=

*ψ*

_{2}=

*ψ*

_{3}≡

*ψ*

_{L}(

*t*),

*ψ*

_{4}=

*ψ*

_{5}=

*ψ*

_{6}≡

*ψ*

_{R}(

*t*), (26) collapse to the pair of ODEs

*ψ*

_{R}=

*ψ*

_{L}+

*π*and

*ψ*

_{R}=

*ψ*

_{L}are fixed points of (27),

*independent*of the precise form of

*H*. For this argument to hold, note that the sums on the right-hand sides of the first three and last three equations of (26) must be identical when evaluated on the tripod solutions; hence, net inputs to each cell from its synaptic connections must be equal. Also, since for \(\bar{g}_{\mathrm{syn}}> 0\) we have

*G*′(0)>0>

*G*′(

*π*) (Fig. 9(b)), so that we

*expect*the in-phase solution to be unstable and the anti-phase one to be stable. To confirm this in the full six-dimensional phase space we compute the 6×6 Jacobian matrix:

*H*′ evaluated at appropriate phase differences

*π*or 0. The anti-phase tripod solution

*ψ*

_{L}−

*ψ*

_{R}=

*π*has one zero eigenvalue with eigenvector \((1,1,1,1,1,1)^{\rm{T}}\); the remaining eigenvalues and eigenvectors are

*ψ*

_{L}(

*t*)≡

*ψ*

_{R}(

*t*)) is unstable.

This CPG model was created in the absence of information on coupling strengths among different hemisegments, and symmetry assumptions were made for mathematical convenience, allowing the reduction to a pair of tripod oscillators. Recent experiments support bilateral symmetry, but indicate that descending connections are stronger than ascending ones (Daun-Gruhn 2011; Daun-Gruhn and Toth 2011; Toth and Daun-Gruhn 2011). Similar rostral-caudal asymmetries have been identified in the lamprey spinal cord Ghigliazza and Holmes (2004a, Fig. 7). The model is currently being modified to fit the data.

In introducing this section it was noted that integrated neuro-mechanical models are needed to better understand the rôle of CPGs in producing locomotion. Examples of these for the cockroach appear in (Cohen et al. 1982) and, with proprioceptive feedback, in (Várkonyi et al. 2008). Models for lamprey swimming can be found in Ghigliazza and Holmes (2004a).

## 4 Models of Perceptual Decisions

I now move to a different topic and scale to consider decision making, specficially two-alternative forced-choice (2AFC) tasks and stochastic accumulator models that describe average activities of large groups of cortical neurons. These belong to a general class of connectionist neural networks Ghigliazza and Holmes (2004a), which, while not directly connected to cellular-level descriptions such as the HH equations, are still biologically plausible. Specifically, in nonhuman primates performing perceptual decisions, intracellular recordings in oculomotor regions such as the lateral intraparietal area (LIP), frontal eye fields, and superior colliculus show that spike rates evolve like sample paths of a stochastic process, rising to a threshold prior to response (Fuchs et al. 2011). In special cases accumulators reduce to drift-diffusion (DD) processes, which have been used to model reaction time distributions and error rates in 2AFC tasks for over 50 years, e.g. (Hagevik and McClellan 1994; Ayali et al. 2007). Subsequently, Sect. 5 sketches how biophysically based neural networks can be reduced to nonlinear leaky competing accumulator (LCAs), providing a path from biological detail to tractable models. For a broad review of decision models, see Kukillaya and Holmes (2007, 2009).

### 4.1 Accumulators and Drift-Diffusion Processes

*x*

_{1},

*x*

_{2}) represent pools of neurons selectively responsive to two stimuli Proctor and Holmes (2010), Kukillaya et al. (2009). These mutually inhibit via functions that express neural activity (e.g., short-term firing rates) in terms of inputs that include constant currents representing mean stimulus levels and i.i.d. Wiener processes modeling noises that pollute the stimuli and/or enter the local circuit from other brain regions, as described by the stochastic differential equations:

*γ*,

*β*are leak and inhibition rates and

*μ*

_{j},

*σ*the means and standard deviation of the noisy stimuli. A decision is reached when the first

*x*

_{j}(

*t*) exceeds a threshold

*x*

_{j,th}. See McMillen and Holmes (2006), McMillen et al. (2008), Tytell et al. (2010) for background on related connectionist networks, and (Rumelhart and McClelland 1986) on the equivalence of different integrator models. For stochastic ODEs, see (Schall 2001; Gold and Shadlen 2001; Shadlen and Newsome 2001; Roitman and Shadlen 2002; Ratcliff et al. 2003, 2006; Mazurek et al. 2003).

*g*and bias

*b*(30–31) without noise (

*σ*=0) can have one or two stable equilibria, separated by a saddle point. In the noisy system these correspond to “choice attractors,” and if

*γ*and

*β*are sufficiently large, a one-dimensional, attracting curve exists that contains the equilibria and orbits connecting them: see Doya and Shadlen (2012) and (Usher and McClelland 2001). Hence, after rapid transients decay following stimulus onset, the dynamics relax to that of a nonlinear Ornstein–Uhlenbeck (OU) process Rumelhart and McClelland (1986), Grossberg (1988), Usher and McClelland (2001). The dominant terms are found by linearizing (30–31) and subtracting to yield an equation for the difference

*x*=

*x*

_{1}−

*x*

_{2}:

*β*=

*γ*), and initial data are unbiased, appropriate when stimuli appear with equal probability and have equal reward values, (33) becomes a DD process

*A*=

*μ*

_{1}−

*μ*

_{2}denotes the drift rate. Responses are given when

*x*first crosses a threshold ±

*x*

_{th}; if

*A*>0 then crossing of +

*x*

_{th}corresponds to a correct response and crossing −

*x*

_{th}to an incorrect one. Here

*x*is the logarithmic likelihood ratio Miller and Fumarola (2012), measuring the difference in evidence accumulated for the two options. The error rate and mean decision time are

*A*,

*σ*and

*x*

_{th}reduce to two:

*η*≡(

*A*/

*σ*)

^{2}, signal-to-noise ratio (SNR), having units of inverse time, and

*θ*≡|

*x*

_{th}/

*A*|, threshold-to-drift ratio, the decision time for noise-free drift

*x*(

*t*)=

*At*. Accuracy may be adjusted by changing

*x*

_{th}or the initial condition

*x*(0), see Sect. 4.7 below.

The DD process (34) is a continuum limit of the sequential probability ratio test Feng et al. (2009, Fig. 2) which, for statistically stationary signal detection tasks, yields decisions of specified average accuracy in the shortest possible time Brown et al. (2005). This property leads to an optimal speed-accuracy tradeoff that maximizes reward rate, enabling the experiments and analyses described below.

### 4.2 An Optimal Speed-Accuracy Tradeoff

*D*

_{RSI}) remain constant across each block of trials and block durations are fixed, optimality corresponds to maximising

*reward rate*: average accuracy divided by average trial duration:

*T*

_{0}is the part of reaction time due to non-decision-related sensory and motor processing. Since

*T*

_{0}and

*η*also typically remain (approximately) constant for each participant, we may substitute (35) into (36) and maximize

*RR*for fixed

*η*,

*T*

_{0}and

*D*

_{RSI}, obtaining a unique threshold-to-drift ratio

*θ*=

*θ*

_{op}for each pair (

*η*,

*D*

_{tot}):

*θ*

_{op},

*η*in (37) can be replaced by the performance measures,

*p*(err) and 〈

*DT*〉, yielding a unique, parameter-free relationship describing the speed-accuracy tradeoff that maximizes

*RR*:

*optimal performance curve*(OPC) (Gold and Shadlen 2002; Bogacz et al. 2006): Fig. 10(a). Different points on the OPC represent

*θ*

_{op}’s and corresponding speed-accuracy trade-offs for different values of difficulty (

*η*) and timing (

*D*

_{tot}): lower or higher thresholds, associated with faster or slower responses, yield lower rewards (diamonds in Fig. 10(a)). The OPC’s shape is intuitively understood by noting that very noisy stimuli (

*η*≈0) contain little information, so that, if they are equally likely, it is optimal to choose at random, giving

*p*(err)=0.5 and 〈

*DT*〉=0 (SNR=0.1 at the right of Fig. 10(a)). As

*η*→∞, stimuli become so easily discriminable that both 〈

*DT*〉 and

*p*(err) approach zero (SNR=100). Intermediate SNRs require some integration of evidence (SNR=1,10). Being parameter free, the OPC can be used to compare performance with respect to optimality across conditions, tasks, and individuals, irrespective of differences in difficulty or timing.

### 4.3 Experimental Evidence: Failures to Optimize

Two 2AFC experiments Busemeyer and Townsend (1993) tested whether humans optimize reward rate in accord with the OPC. In the first, 20 participants viewed motion stimuli Bogacz et al. (2006, Appendix) and were rewarded for correct responses. Trials were grouped in 7-minute blocks with different *D*_{RSI}’s fixed through each block. In the second experiment, 60 participants discriminated if the majority of 100 locations on a static display were filled with stars or empty. Two difficulty conditions were used in 4-minute blocks. Participants were told to maximize total earnings, and practice blocks were administered prior to testing.

*DT*〉’s were estimated by fitting the DD model to reaction time distributions; the 0–50 % error-rate range was divided into 10 bins, and 〈

*DT*/

*D*

_{tot}〉 were computed for each bin by averaging over those results and conditions with error rates in that bin. This yields the open (tallest) bars in Fig. 10(b); the shaded bars derive from similar analyses restricted to subgroups of participants ranked by their total rewards accrued over all different conditions. The top 30 % group performs close to the OPC, achieving near-optimal performance, but a majority of participants are significantly suboptimal due to longer decision times (Wald and Wolfowitz 1948). This suggests two possibilities:

- (1)
Participants seek to optimize another criterion, such as accuracy, instead of, or as well as, maximizing reward.

- (2)
Participants seek to maximize reward, but systematically fail due to constraint(s) on performance and/or other cognitive factors. We now address these.

### 4.4 A Preference for Accuracy?

*RA*) subtracts a fraction of error rate from

*RR*; the second (modified reward rate,

*RR*

_{m}) penalizes errors by reducing previous winnings. In both the parameter

*q*∈(0,1) specifies a weight placed on accuracy. Increasing

*q*drives the OPC upward (Bogacz et al. 2006), consistent with the empirical observations of Fig. 10(b), suggesting that participants assume that errors are explicitly penalized.

However, alternative accounts of the data preserve the assumption of reward maximization. Specifically, timing uncertainty may degrade *RR* estimates, systematically causing longer decision times, or participants may allow for costs of cognitive control required for changing parameters, especially if these yield small increases in *RR* (cf. diamonds in Fig. 10(a)).

### 4.5 Robust Decisions in the Face of Uncertainty?

In the analyses of Sects. 4.2–4.4 an objective function is maximized, given known task parameters. However, accurate values may not be available: *RR* depends on inter-trial delays and SNR, both of which may be hard to estimate. Information-gap theory (Bogacz et al. 2006, 2010) assumes that parameters lie in a bounded *uncertainty set* and uses a *maximin strategy* to optimize a worst case scenario.

*D*

_{tot}lies in a set \(U_{p}(\alpha_{p}; \tilde{D}_{\mathrm{tot}}) = \{ D_{\mathrm{tot}} > 0: | D_{\mathrm{tot}} - \tilde{D}_{\mathrm{tot}} | \le\alpha_{p} \tilde{D}_{\mathrm{tot}} \}\), of size proportional to the actual delay \(\tilde{D}_{\mathrm{tot}}\), with uncertainty

*α*

_{p}analogous to the coefficient of variation in scalar expectancy theory (Maddox and Bohil 1998; Bohil and Maddox 2003). Instead of the optimal threshold of (37), the maximin strategy selects the threshold

*θ*

_{MM}that maximizes the worst possible

*RR*for \(D_{\mathrm{tot}} \in U_{p}(\alpha_{p}; \tilde{D}_{\mathrm{tot}})\), yielding a one-parameter family of

*maximin performance curves*(MMPCs) (Bogacz et al. 2006):

*p*(err) as uncertainty increases. An alternative strategy yields

*robust-satisficing performance curves*(RSPCs) (Bogacz et al. 2006, Fig. 13) that provide poorer fits and are not described here.

*RA*and

*RR*

_{m}of (40), to MMPCs for timing uncertainty and SNR, and to RSPCs for timing uncertainty. While there is little difference among fits to the top 30 %, data from the middle 60 % and lowest 10 % subgroups exhibit patterns that distinguish among the theories. Maximum likelihood computations show that MMPCs for uncertainties in delays provide the best fits, followed by RSPCs for uncertainties in delays and

*RA*(Buhusi and Meck 2005). Thus, greater accuracy can emerge as a

*consequence*of maximizing

*RR*under uncertainty rather than an

*objective*of optimization.

### 4.6 Practice, Timing Uncertainty, or the Cost of Control?

To test whether deviations from the OPC are better explained by an emphasis on accuracy or by timing uncertainty, (Gibbon 1977) conducted a 2AFC experiment with motion stimuli encompassing a range of discriminabilities (moving dots Zacksenhouse et al. (2010) with 0 %,4 %,8 %,16 % and 32 % coherences fixed in each block), and administered interval timing tests in parallel (Gibbon 1977). 17 participants completed at least 13 sessions in each condition, increasing the likelihood of achieving optimal performance by providing extensive training and allowing the study of practice effects (Zacksenhouse et al. 2010). There were four main findings.

Second, the accuracy-weighted objective function *RR*_{m} of (40) outperforms the OPC in fitting decision times, with accuracy weight decreasing monotonically through sessions 1–9 and thereafter remaining at *q*≈0.2 (not shown here, see Zacksenhouse et al. (2010)), suggesting that participants may initially favor accuracy, but that this diminishes with practice.

Third, timing inaccuracy throughout all but the first session, independently assessed by participants’ coefficients of variation in a peak-interval task, is significantly correlated with their distances from the OPC Zacksenhouse et al. (2010). Moreover, this provides a better account of deviations from the OPC than weighting accuracy by the parameter *q* in *RR*_{m}, supporting the hypothesis that humans can learn to maximize rewards by devaluing accuracy, with a deviation from optimality inversely related to their timing ability. However, even after long practice, MMPCs based on timing uncertainty fail to capture performance for the two lowest coherences (Fig. 12(a)), suggesting that other factors may be involved, including the cost of cognitive control (Zacksenhouse et al. 2010).

To test this fourth possibility, Balci et al. (2011) computed the single optimal threshold over all coherence conditions. Figure 12(b) shows that the resulting curve fits the full range of data for later sessions (6–13), suggesting that, given practice, participants adopted such a threshold. Rewards for this single threshold differed little from those for thresholds optimized for each coherence condition, suggesting that participants may seek one threshold that does best over all conditions, avoiding estimation of coherences and control of thresholds from block to block. Control costs are discussed in (Britten et al. 1993).

### 4.7 Prior Expectations and Trial-to-Trial Adjustments

Given prior information on the probabilities of observing each stimulus in a 2AFC task, a DD process can be optimized by appropriately shifting the initial condition *x*(0); rewards that favor one response over the other can be treated similarly (Buhusi and Meck 2005). Comparisons of these predictions with human behavioral data were carried out in (Dutilh et al. 2009; Petrov et al. 2011), finding that participants achieved 97–99 % of maximum reward. A related study of monkeys used a fixed stimulus presentation period that obviates the need for a speed-accuracy tradeoff, but in which differences in rewards for the two responses were signaled before each trial and motion coherences varied randomly between trials. The animals achieved 98 % and 99.5 % of maximum rewards Balci et al. (2011), and fits of LIP recordings to an accumulator model Balci et al. (2011) indicated that this was also done by shifting initial conditions. Human behavioral studies revealed similar near-optimal shifts in response to biased rewards Balci et al. (2011, Fig. 9).

Humans also exhibit adjustment effects in response to repetition and alternation patterns that necessarily occur, given stimuli chosen with equal probability (Balci et al. 2011). Accumulator models developed in (Posner and Snyder 1975) indicate that this is also due to initial condition shifts, presumably due to expectations that patterns will persist even when stimuli are purely random. In fact pattern recognition is advantageous in natural situations, allowing prior beliefs to adapt to match stationary or slowly changing environments Balci et al. (2011).

The work described in this section depends on simple models that at first reproduce previous data, then predict outcomes of new experiments, and finally admit analyses and modifications that account for differences between predictions and the new data. The explicit OPC expression (39) is crucial here; it seems unlikely that such predictions could readily emerge from computational simulations alone. These models are certainly useful, but they do not immediately connect to cellular-level descriptions such as those of Sects. 2–3. We now discuss this connection.

## 5 Connecting the Levels

The accumulator models of Sect. 4 address optimality constraints at the systems level, but they are too abstract to identify mechanisms or constraints arising from underlying neural circuits. To do this the abstract models must be related to biophysical aspects of neural function. For example, spiking-neuron models can be reduced in dimension by averaging over populations of cells Holmes and Cohen (2014), allowing them to include the effects of synaptic time constants and neurotransmitters that can change cellular excitability and synaptic efficacy (Bogacz et al. 2006), effectively adjusting gains *g* in frequency–current functions (32) Simen et al. (2009). I describe one such reduction in this section. For a review of spiking models for decision making, see (Feng et al. 2009).

### 5.1 Reduction of a Spiking Model to Accumulators

*v*

_{j}(

*t*) and synaptic activities

*s*

_{AMPA,j}(

*t*),

*s*

_{NMDA,j}(

*t*) and

*s*

_{GABA,j}(

*t*), described by the following ODEs:

*I*

_{syn,j}(

*t*)=−∑

_{type,k}

*g*

_{type,k}

*s*

_{type,k}(

*v*

_{k}−

*v*

_{E}), type=AMPA, GABA, or AMPA-ext,

*T*

_{type}is the time constant for that synapse type,

*k*ranges over all cell

*j*’s presynaptic neurons, and superscripts

*l*index times \(t_{j}^{l}\) at which the

*j*th cell crosses a threshold

*v*

_{th}, emits a delta function and is reset to

*v*

_{r}for a refractory period

*τ*

_{ref}, cf. Sect. 2.3 and Fig. 5. The NMDA dynamics require two ODEs to model fast rise followed by slow decay (Yu and Cohen 2009), cf. Sect. 2.5:

*s*

_{AMPA-ext,j}(

*t*), modeled by OU processes driven by Gaussian noise of mean

*μ*and standard deviation

*σ*, enter all cells:

*μ*(1±

*E*) to the selective cells with appropriately adjusted variances

*σ*

_{j}, where

*E*∈[0,1] denotes stimulus discriminability (

*E*=1 for high SNR;

*E*=0 for zero SNR). Neuromodulation is represented by multiplying excitatory and inhibitory conductances

*g*

_{AMPA,k},

*g*

_{NMDA,k}and

*g*

_{GABA,k}by factors

*γ*

_{E},

*γ*

_{I}. Eliminating irrelevant

*s*

_{type,j}’s (excitatory neurons do not release GABA, inhibitory neurons do not release AMPA or NMDA), (42–46) constitute a 9200-dimensional hybrid, stochastic dynamical system that is analytically intractable and computationally demanding.

Following the mean-field method of (Brunel and Wang 2001; Wong and Wang 2006; Eckhoff et al. 2011; Deco et al. 2013), the network is first reduced to four populations by assuming a fixed average voltage \(\bar{v} = (v_{r}+v_{\mathrm{th}}) / 2\) to estimate synaptic currents. These are multiplied by the appropriate numbers *N*_{j} of presynaptic cells in each population and by averaged synaptic variables \(\bar{s}_{\mathrm{type}, j}\), and summed to create input currents *I*_{type,j} to each population (the index *j*∈{1,2,3,4} now denotes the population). Individual, evolving cell voltages are replaced by population-averaged, time-dependent firing rates determined by frequency–current (f–I) curves *φ*_{j}(*I*_{syn,j}), analogous to the input–output function of (32). This yields an 11-dimensional system described by 4 firing rates *ν*_{j}(*t*), one inhibitory population-averaged synaptic variable \(\bar{s}_{\mathrm{GABA}}(t)\), and two variables \(\bar{s}_{\mathrm{AMPA},j}(t)\) and \(\bar{s}_{\mathrm{NMDA},j}(t)\) for each excitatory subpopulation (6 in all).

*T*

_{AMPA}=2 ms,

*T*

_{GABA}=5 ms), while that for NMDA decay is slow (

*T*

_{NMDA}=100 ms); \(\bar{s}_{\mathrm{AMPA},j}(t)\) and \(\bar{s}_{\mathrm{GABA},j}(t)\) therefore rapidly approach quasi-steady states, as in Rinzel’s reduction of HH in Sect. 2.2, This eliminates 3 ODEs for the excitatory populations and 1 for the inhibitory population. Firing rates likewise track values set by the f–I curves, since they are determined by

*T*

_{AMPA}:

*ν*

_{j}(

*t*)≈

*φ*

_{j}(

*I*

_{syn,j}(

*t*)) for the non-selective and interneuron populations and the ODEs for

*ν*

_{3}and

*ν*

_{I}drop out. Also, with stimuli on, the non-selective population typically has a less variable firing rate than the selective populations, so that \(\bar{s}_{\mathrm{NMDA},3}\) can be fixed, leaving four ODEs for the synaptic variables and firing rates of the selective populations:

^{2}The \(\bar{s}_{j}(t)\)’s and the firing rates

*ν*

_{j}(

*t*) correspond to the activity levels

*x*

_{j}(

*t*) in the LCA (30–31), and white noise is added as in those SDEs.

To complete the reduction, currents must be estimated self-consistently. This is complicated by the fact that *I*_{syn,j} in (49) contains terms that depend on both \(\bar{s}_{j}\) and *φ*_{j}(*I*_{syn,j}), so that the vector field is defined recursively. Ideally, we seek relationships of the form \(I_{\mathrm{syn},j} = \alpha_{j1} \bar{s}_{1} +\alpha_{j2} \bar{s}_{2} +\beta_{j1} \nu_{1} + \beta_{j2} \nu_{2} + I_{\mathrm{const},j}\), as in (Servan-Schreiber et al. 1990). Piecewise-smooth f–I curves help here Wang (2008), since they predict critical currents beyond which firing rates rise linearly. The parameters *γ*_{E},*γ*_{I} enter via the AMPA, NMDA and GABA components of the currents *I*_{const,j} and coefficients *α*_{jk},*β*_{jk}. See Eckhoff et al. (2009, 2011) for details.

*φ*

_{j}(

*I*

_{syn,j}), as described in Eckhoff et al. (2009). In the top two and bottom left panels of Fig. 16 (

*γ*

_{E},

*γ*

_{I})=(1,1) (on the

*RR*ridge of Fig. 14(b)) cases without stimulus, with identical stimuli

*E*=0,SNR=0 and with

*E*=0.128,SNR>0 are shown. States with both \(\bar{s}_{j}\)’s low represent lack of response, possibly awaiting stimulus appearance. With stimuli present, the basin of attraction of the low-low sink shrinks, allowing noisy trials to reach the choice attractors \(\bar{s}_{1} \gg \bar{s}_{2}\) and \(\bar{s}_{2} \gg\bar{s}_{1}\), as in Eckhoff et al. (2009); the basin of the correct attractor is larger for

*E*>0 (bottom left panel). Finally, for (

*γ*

_{E},

*γ*

_{I})=(2,1.2) (lower right region in Fig. 14(b) and Fig. 16, bottom right panel), \(\bar{s}_{1}, \bar{s}_{2}\) both high corresponds to impulsive behavior in which near-random choices occur.

### 5.2 Physiological Constraints to Optimality?

The global dynamics of the reduced system (48–49), with its multiple attractors (Fig. 16), differs qualitatively and quantitatively from the optimal DD process of (34). While the dynamics near a saddle point approximate the approach of orbits to a drift along one-dimensional DD dynamics, acceleration away from the saddle and deceleration toward an attractor cause suboptimal integration. Moreover, even if an attracting one-dimensional subspace exists, deviations from it effectively blur the decision thresholds, and firing rate bounds preclude negative and arbitrarily high activations, preventing perfect subtraction as in (33) (Wang 2002; Wong and Wang 2006). Adjustments in baseline activity and gain can keep accumulator states in near-linear dynamical ranges (Eckhoff et al. 2011), but the fact that nonlinear dynamics emerge from a biophysically based model suggests physiological obstructions to optimality, especially when task conditions span a wide range Brunel and Wang (2001), Renart et al. (2003), Wong and Wang (2006).

In this regard it is worth noting that choice attractors such as those of Fig. 16, which can persist in the absence of stimuli (Fig. 15(a)), have been identified with short-term working memory states (Jones 1994; Guckenheimer and Holmes 1983). Working memory is clearly important for delayed-response tasks such as those of Wong and Wang (2006) and in many other aspects of our lives. It is therefore plausible that cortical circuits have evolved to allow flexible multi-attractor dynamics that are inconsistent with optimal strategies in artificially simple tasks such as 2AFC.

## 6 Discussion: Some Omissions and Open Problems

In this article I have outlined some mathematical models in neuroscience that draw on dynamical systems theory. Specifically, Sects. 2–3 describe mechanistic, cellular-scale models for the generation of action potentials (spikes) and for electrical and chemical synapses, discuss methods for reducing their dimension and hence simplifying their analysis, and illustrate with a model of a central pattern generator for legged locomotion. This represents the dominant approach to modeling among neuroscientists and applied mathematicians. In contrast, Sect. 4 takes a high level perspective, using stochastic ODEs and a drift-diffusion (DD) model to represent accumulation of evidence in brain areas and to predict a strategy for maximizing reward rate in binary perceptual choice tasks, enabled by the simplicity of the DD model. Such models exemplify the connectionist neural networks in wide use among cognitive psychologists. They can be justified empirically by their ability to fit behavioral data, and, more directly, from observations of spike rates in animal studies. They can also be derived, albeit non-rigorously, from cellular-scale spiking models, as sketched in Sect. 5.

These models illustrate the range of scales and models in mathematical neuroscience, but as noted in Sect. 1, many important ideas and approaches are missing from this account. While spinal and thoracic CPGs generate functional motor rhythms, brains exhibit cortical oscillations over a wide frequency range (2–100 Hz), as detected via electroencephalography (scalp electrodes) and extracellular electrodes recording local field potentials (Eckhoff et al. 2011). There is much debate about the mechanisms and functions of such oscillations Eckhoff et al. (2011), including their rôles in “binding” different sensory modalities Eckhoff et al. (2011) and in diseased states Eckhoff et al. (2011), and their generation by and effects on spikes from individual cells Eckhoff et al. (2011). Further modeling, with phase oscillators as well as HH-type equations, could shed light on these cortical rhythms.

The study of how organisms learn about and adapt to changing environments is a major area in which reinforcement learning (RL) Eckhoff et al. (2011) and the extended notion of hierarchical reinforcement learning Eckhoff et al. (2011) draw on studies of the dopamine neuromodulation system Wong and Wang (2006) to propose discrete dynamical updates following rewarded behaviors. Dimension reduction can occur in RL (van Ravenzwaaij et al. 2012, Figs. 5–7). Goal-directed planning and searching also employ iterative models in which different strategies are explored (Servan-Schreiber et al. 1990; Cohen et al. 1990). On the mechanistic level, cortical rhythms also may be important in learning (Deco et al. 2013).

Probabilistic ideas are also widely used, perhaps more widely than those from dynamical systems. An influential subculture considers probabilistic computations based on Bayes’ rule (Usher and Cohen 1999; Wang 1999; Renart et al. 2003; Wong and Wang 2006; Deco et al. 2013) that offer normative accounts of task performance as dynamic updating of priors. Models have been developed for sensori-motor control, e.g. Zhang and Barash (2004), Feng et al. (2009), Rorie et al. (2010), Gao et al. (2011), and proposed to describe stimulus identification, decision making and learning (Buzsaki 2006; Baker 2007; Fries et al. 2007). These models are mostly empirical, but there is increasing evidence that the brain can code probability distributions and perform Bayesian computations, possibly via cortico-basal ganglia circuits (Kopell et al. 2009; Whittington et al. 2009; Wang 2010). Information theory (Gray et al. 1989; Gray and Singer 1995; Fries 2005), originally suggested by Weiner as a descriptor for sensory receptors (McCarthy et al. 2011, 2012; Uhlhaas and Singer 2010) has also been used to analyze spike trains and quantify their information content in attempts to understand neural coding (Wang 2010) as well as learning (Sutton 1988; Sutton and Barto 1998). More generally, probabilistic methods including hidden Markov models are useful in analyzing multi-electrode recordings in terms of transitions among brain states, e.g. (Botvinick et al. 2009; Botvinick 2012).

In closing I note some open problems, focusing on ones that arise from the models discussed in Sects. 2–5. A persistent difficulty is in identifying brain areas and neural substrates in which specific “computations” are done. Excepting some specialized sensory and sensori-motor circuits, such as those dedicated to reflexive actions, most computations appear to activate multiple brain areas. For example, while neural firing rates in area LIP are similar to DD processes (Schultz et al. 1997, 2000), pharmacological inactivation of LIP may not deprive an animal of the ability to identify stimuli and respond appropriately (Swinehard and Abbott 2006) (although it may slow him down). Computational models involving multiple brain areas have been constructed for over 30 years (Solway and Botvinick 2012), but there have been few attempts to analyze them mathematically (an exception is (Torta et al. 2009), which shows that the multi-layer network of (Bayes 1763) can be reduced to a DD process with time-varying input). Moreover, until recently simultaneous neural recordings from multiple areas have not been available to constrain multi-area models.

Improved theory and analytical methods for hybrid dynamical systems, especially (large) networks of integrate-and-fire cells.

Better descriptions of, and methods for extracting macroscopic activity states in averaging over large cell populations.

Nested sets of models for simple tasks, fitted to electrophysiological and behavioral data.

Further use of time scale separation in creating and analyzing models of cognitive dynamics.

Analyses of iterative learning algorithms as dynamical systems.

The joys celebrated herein are chiefly in the works of others, the trials come in my attempts to understand them, and to contribute new ideas.

## Acknowledgements

This article draws upon work done since 2000, variously supported by NSF under DMS-0101208 and EF-0425878, AFOSR under FA9550-07-1-0537 and FA9550-07-1-0528, NIH under P50 MH62196, the US-Israel Binational Science Foundation under BSF 2011059, and the J. Insley Pyne Fund of Princeton University. The material in Sects. 2–3 was adapted from notes for a course taught at Princeton since 2006 to which Philip Eckhoff contributed much. An extended version of Sect. 4 appears in Kording et al. (2004), Kording and Wolpert (2006), Wolpert (2007). The author thanks Fuat Balci for providing Fig. 12 and the anonymous reviewers for their suggestions, and gratefully acknowledges the contributions of many other collaborators, not all of whose work could be cited here.