Neural network model of the primary visual cortex: From functional architecture to lateral connectivity and back

  • Barak Blumenfeld
  • Dmitri Bibitchkov
  • Misha Tsodyks
Open Access
Article

DOI: 10.1007/s10827-006-6307-y

Cite this article as:
Blumenfeld, B., Bibitchkov, D. & Tsodyks, M. J Comput Neurosci (2006) 20: 219. doi:10.1007/s10827-006-6307-y

Abstract

The role of intrinsic cortical dynamics is a debatable issue. A recent optical imaging study (Kenet et al., 2003) found that activity patterns similar to orientation maps (OMs), emerge in the primary visual cortex (V1) even in the absence of sensory input, suggesting an intrinsic mechanism of OM activation. To better understand these results and shed light on the intrinsic V1 processing, we suggest a neural network model in which OMs are encoded by the intrinsic lateral connections. The proposed connectivity pattern depends on the preferred orientation and, unlike previous models, on the degree of orientation selectivity of the interconnected neurons. We prove that the network has a ring attractor composed of an approximated version of the OMs. Consequently, OMs emerge spontaneously when the network is presented with an unstructured noisy input. Simulations show that the model can be applied to experimental data and generate realistic OMs. We study a variation of the model with spatially restricted connections, and show that it gives rise to states composed of several OMs. We hypothesize that these states can represent local properties of the visual scene.

Keywords

Visual cortex Spontaneous activity Orientation selectivity Neural network model Optical imaging 

1. Introduction

Processing of visual information in the mammalian primary visual cortex (V1) is thought to arise from an interplay between the pattern of external projections and intrinsic cortical dynamics. Even though the intrinsic processing has been a subject of an extensive research, its role and the mechanisms underlying it remain poorly understood. A powerful tool for understanding the intrinsic processing is the study of spontaneous cortical activity, that is, activity not evoked by visual stimulation. Even in the absence of visual stimulation, V1 exhibits a rich and complex spontaneous activity (Lampl et al., 1999; Tsodyks et al., 1999; Kenet et al., 2003; Fiser et al., 2004). This activity is thought to originate from mechanisms intrinsic to V1. Thus, spontaneous activity can reveal the underlying V1 architecture and shed light on its function.

In this study, we focus on the link between spontaneous activity and the activity evoked by visual stimulation. Since the work of Hubel and Wiesel (1959) it is believed that one of the main functions of V1 is to encode and process oriented stimuli. A major progress in studying the functional architecture of V1 was achieved by optical imaging (Blasdel and Salama, 1986; Grinvald et al., 1986). In a widely used experimental protocol (Grinvald et al., 1999), an animal is shown full field moving gratings of different orientations. Each orientation yields a single condition orientation map (OM), a two-dimensional representation of the neuronal activity across the cortical sheet evoked by the stimulus. In cats, as well in many other mammals (including primates, ferrets, tree shrews and sheep, but not rodents; see Van Hooser et al., 2005) neurons with similar orientation preference tend to be clustered. Thus, OMs are characterized by patches of high neuronal activity separated by regions of low neuronal activity. The location and shape of the patches change with the gratings’ orientation.

A recent work (Kenet et al., 2003) used optical imaging with voltage sensitive dye to study spontaneous activity in V1 of an anaesthetized cat, measured when the animal’s eyes were closed. Even though no visual stimulus was presented, some of the spontaneous cortical states were similar to the OMs of the same region. When such spontaneous OMs emerged, they spanned several hypercolumns, and were often followed by states similar to OMs of a proximal orientation. It was speculated that the spontaneous activity reflects the internal state of the brain, which might be influenced by context, attention, or perceptual memory. For example, the spontaneous state can reflect an expectation of the orientation of a forthcoming input, based on prior activity. In any case, the striking similarity between spontaneous and evoked states suggests that there exists a mechanism intrinsic to V1 capable of generating the OMs. If such mechanism exists, the evoked OMs originate from an interplay between the intrinsic mechanism and the pattern of afferent LGN input, known to have some degree of orientation tuning.

What is the intrinsic V1 mechanism underlying the spontaneous emergence of OMs? Many experimental studies suggest an important role for the pattern of lateral connectivity in V1. A large body of anatomical and electrophysiological research has shown that the V1 intracortical connectivity is correlated with orientation preference. The general trend is that long range lateral projections connect neurons with similar orientation preference (Gilbert and Wiesel, 1989; Malach et al., 1993; Weliky et al., 1995; Bosking et al., 1997; Kisvárday et al., 1997; Buzás et al., 1998). As the data is highly variable, this trend is true only on average, and the exact relation between intracortical connectivity, orientation preference, and other functional maps remains a subject of active research (Kisvárday et al., 2002; Ben-Shahar and Zucker, 2004). Direct evidence for the involvement of the lateral connections in orientation tuning comes from studies in which inactivation of GABAergic neurons altered the orientation tuning curves of remotely located neurons (Crook et al., 1998; Kisvárday et al., 2000). Developmental studies in newborn animals have shown that the overall geometry of the OMs is stable from a very early stage, and that it is resistant to manipulations of the visual input (Crair et al., 1998; Löwel et al., 1998). On the other hand, cortical development involves sharpening of the OMs with a time course that closely matches the time course for expression and refinement of long range lateral connections (Gödecke et al., 1997). Developmental experiments in which the visual input was rerouted into the auditory cortex (Sharma et al., 2000), yielded an auditory cortex that was similar, in many aspects, to V1. In particular, it exhibited both OMs and long range lateral projections that preferentially connected cells with similar orientation preference. Taken together, these developmental studies provide further support for the strong link between OMs and lateral connections.

From a theoretical point of view, V1 can be viewed as a complex, nonlinear, dynamical system. Several theoretical studies suggested that OMs are attractor states of the cortical dynamics (Somers et al., 1995; Sompolinsky and Shapley, 1997; Ernst et al., 2001). This line of thought is consistent with the spontaneous emergence of orientation maps because OMs can still be attractors of the intracortical dynamics even without the stimulus-encoding afferent input. In this family of models the pattern of lateral connectivity plays an important role in the formation of the attractor landscape, consistent with the experimental findings listed above. A well studied example of such a model is the ring model (Ben-Yishai et al., 1995; Hansel and Sompolinsky, 1998). This model considers a population of neurons in one hypercolumn in V1. Each neuron is characterized by a preferred orientation, and the overall distribution of preferred orientations is uniform over the entire range of possible orientations. The neurons receive both afferent LGN inputs and lateral recurrent inputs. When a stimulus is presented to the network via the afferent connections, the network develops an activity profile that peaks at the neuron whose preferred orientation matches the stimulus’ orientation. However, the precise shape of the stationary profile is determined, to a large extent, by recurrent connections. The ring model therefore suggests a mechanism that links the cortical response evoked by an oriented stimulus and the pattern of lateral connections.

Moreover, if the recurrent connections are strong enough, states similar to those evoked by a stimulus emerge even if a uniform input (representing lack of an external visual stimulus) is applied. This occurs because the uniform solution becomes unstable, and a continuum of states that are similar to the states evoked by the different stimulus orientations emerges as an attractor of the network’s dynamics. Collectively, these states are known as the ring attractor. Thus, the ring model suggests a mechanism that explains how states similar to OMs can emerge spontaneously. This mechanism was suggested to underlie the patterns of spontaneous activity observed in Kenet et al. (2003).

The ring model, however, cannot directly explain the generation of experimental OMs. It is inherently a one-dimensional model, proposed as a model for one hypercolumn, and cannot account for the generation of the 2-dimensional OMs. One can of course consider a straightforward 2-dimensional implementation of the ring model, by simply placing the neurons on a 2-dimensional sheet (Goldberg et al., 2004); however, the attractor states of such implementation do not necessarily match the experimental OMs. We will further discuss the limitations of the ring model in Section 2.

This leaves the following question open: what type of lateral connectivity can lead to the spontaneous emergence of the OMs? In this contribution we answer this question by suggesting a neural network model with a simple connectivity rule that supports a ring attractor; this ring attractor is composed of states similar to the experimental OMs. To construct this network we first consider an approximation to the OMs (Section 2). Then, we introduce the model and solve it analytically (Section 3). Later, we take an experimental data set, and apply to it the theory we developed in the previous section (Section 4).We continue by studying the effect of the limited spatial extent of the lateral connections (Section 5), and conclude with a discussion of our results and predictions of the models (Section 6).

2. The polar map

As a first step toward constructing the network, we will consider an approximation to the OMs that is based on the “polar map” (PM; Bonhoeffer and Grinvald, 1993). The PM is a functional map that assigns each location x, a complex number, zx. The values of zx are typically calculated from the OMs as follows. Let φ1, φ2,…, φp be p equidistant orientations presented to the animal.1 Let Sxφj denote the response at location x for the OM evoked by orientation φj. Then, zx is defined by:
$${z_x} \equiv {r_x}{e^{i\theta x}} = {2 \over p}\sum\limits_{j = 1}^p {s_x^{{\varphi _j}}{r^{i{\varphi _j}}}} $$
where \(i = \sqrt { - 1} \) Equation (1) can be viewed as a summation of p 2-dimensional vectors whose angle, φj, represents the orientation of the stimulus, and their length, Sxφj represents the magnitude of the cortical response to that orientation. This view gives rise to the standard interpretation of the angle of zx, i.e. θx, as the preferred orientation at location x. The magnitude of zx, i.e. rx, measures the degree to which the response at location x is modulated by the stimulus’ orientation. Following other authors, we will refer to this variable as the selectivity of location x
Throughout the article we will use the data set obtained in Kenet et al. (2003) to demonstrate and validate the theory we develop. This data set contained 8 OMs obtained from an anaesthetized cat using voltage-sensitive dye optical imaging. After a two-fold downsampling of the original data set, each map contained 42 × 17 = 714 pixels, corresponding to an area of approximately 11.7 mm2 (the downsampling was preformed so that we can run efficient simulations in the later stages of the analysis). The PM calculated from the 8 OMs using Eq. (1) is depicted in Fig. 1(A). The color coding represents the above polar interpretation of zx. The color of each pixel represents the preferred orientation, θx at that location, and the brightness represents the selectivity, rx. Pinwheels are represented as dark pixels, i.e. pixels where rx is close to zero, around which all preferred orientations are represented. A closer examination reveals that when moving from a point far from the pinwheel (“linear zone”) towards the pinwheel, the selectivity is gradually reduced, resulting in a large range of selectivity values. The variability of the selectivities can be also observed in Fig. 1(B), where distribution of the values of zx over the complex plane is depicted. In this figure each pixel in the PM is represented by a single point. The distance of each point from the origin represents the selectivity of the pixel, and the angle it makes with the real axis represents its preferred orientation. It can be seen that for each preferred orientation there is a wide range of selectivities. This variability is not captured by previous models such as the ring model, which implicitly assume a constant selectivity. As a result, these models are limited in their ability to account for realistic OMs.
Fig. 1

(A) The polar map (PM) for the data obtained in Kenet et al. (2003). The preferred orientation at each pixel is encoded by its color according to the scheme at the bottom of the figure. The selectivity is encoded by brightness (a dark pixel represents low selectivity, a bright pixel represents high selectivity; The brightness scale is linearly stretched between zero selectivity and the maximal selectivity in this PM) (B) Distribution over the complex plane of the values of zx of the PM in (A)

Equation (1) can also be viewed as the (discrete) Fourier transform of the “tuning curve” of location x, where the term “tuning curve” here refers to the optical imaging response, Sxφj, as a function φj. Thus, zx is simply the first Fourier coefficient of the tuning curve of location x. Since these tuning curves tend to be unimodal and centered around the pixel’s preferred orientation, we can expect the first Fourier component to contain much of the tuning curve’s energy. Therefore, the tuning curve of each location can be approximated with this single Fourier component. This approximation can be written explicitly by defining the approximated OM for orientation φ as:
$$M_x^\varphi = {r_x}\cos ({\theta _x} - \varphi )$$
and SxφjMxφj, based on the arguments presented above. We suggest two ways of measuring the quality of this approximation and apply them to the Kenet et al. (2003) data set. First, the similarity between each of the experimental OMs and its approximated version can be we quantified. A standard measure of similarity between OMs is their Pearson correlation coefficient. For the experimental data set, the average correlation between the 8 pairs of experimental and approximated OMs was 0.815 (min: 0.746, max: 0.887). These are high correlations that validate the approximation. Another measure for the quality of the approximation is the percent of variance explained, which we denote by γ. With rx defined by Eq. (1), γ can be defined as:
$$\gamma = {{\sum\nolimits_x {{r_x}^2} } \over {2\sum\nolimits_x {{\sigma _x}^2} }}$$
where σx2 is the variance of the cortical response over all orientations at location x. The values that γ takes are between 0 and 1, with γ = 1 if and only if the approximation is precise. For the experimental data set, γ = 0.89, again validating the usage of the approximation. We obtained similar results for other data sets as well.

From the above analysis we conclude that experimental OMs can be well-approximated using the PM. For the purpose of this contribution, that is, constructing a neural network with OMs as being its attractor states, this approximation offers two advantages. First, by associating each location with only two real numbers (rx, θx), rather than the entire tuning curve, the dimensionality of the problem is greatly reduced. Second, the approximation suggests a natural way to interpolate OMs for orientations other than the ones presented during the experiment. We can thus consider a ring of approximated OMs, formally defined by assigning arbitrary values to φ in Eq. (2) Our modeling goal can be now reformulated as constructing a model whose attractor states will coincide with this ring of approximated OMs. This goal is achieved by the model we introduce in the next section.

3. Model

Let mx (t) denote the average firing rate of the cortical minicolumn at location x at time t. Since OMs tend to be smooth functions of space, we use a continuous approximation and let x take real numbers rather than discrete values. The evolution of mx is described by the standard rate equation (Wilson and Cowan, 1973):
$$\tau {{\dot m}_x} = - {m_x} + {\left[ {I_x^{{\rm{rec}}} + I_x^{{\rm{aff}}} - T} \right]_ + }$$
where x is the time derivative of mx, Ixrec is the input to location x due to the recurrent connections, Ixaff is the afferent input to that location, τ is a time constant, T is the firing threshold, [ ]+ denotes the ramp gain function ([ξ]+ = ξ if ξ > 0 and [ξ]+ = 0 otherwise), and the time argument has been suppressed for brevity. The recurrent input, Ixrec, is defined by:
$$I_x^{{\rm{rec}}} = {1 \over A}\int {dy{W_{xy}}{m_y}} $$
The integral is taken over the whole region of V1 that is modeled, and A is the area of that region. A is used as a normalization factor that simplifies the mathematical formulations. The synaptic weights connecting locations x and y, Wxy, are defined by:
$${W_{xy}} = {J_2}{r_x}{r_y}\cos ({\theta _x} - {\theta _y}) + {J_0}$$

The information about the PM is given by the term rxrycos(θxθy). The parameter J2 > 0, is a global scaling factor of this term. Since scaling the values of r is equivalent to scaling J2, we assume that the values of r are normalized so that the mean of rx2 over the area is 1. The parameter J0 represents global excitation (if J0 > 0) or global inhibition (if J0 < 0). It should be noted that the connectivity given by Eq. (6) is similar to the connectivity of the ring model (Ben-Yishai et al., 1995). However, unlike in the ring model, the connectivity we suggest depends not only on the preferred orientations of the pre- and post- synaptic locations, but also on their selectivities.

The connectivity pattern induced by Eq. (6) is illustrated in Fig. 2(A). The connectivity matrix was calculated using the PM of the Kenet et al. (2003), and setting J0 = −2, J2 = 5, as used in subsequent simulations. The connectivity strength between one specific location (black dot) and all other locations is visualized by color coding, with shades of green representing positive weights (“excitatory” connections), shades of red representing negative weights (“inhibitory” connections), and white representing connections close to zero. Note the patchy structure of the connectivity pattern, which follows directly from the patchy nature of the PM. Moreover, the “excitatory patches” correspond to regions with a similar orientation preference; Thus, the patchy pattern in Fig. 2(A) reflects the patchy layout of long range connections in V1 found in many anatomical studies (Gilbert and Wiesel, 1989; Malach et al., 1993; Bosking et al., 1997; Kisvárday et al., 1997). However, in contrast with known anatomy, the connectivity does not depend on the distance between the pre-and post-synaptic locations. This is a simplification we make at this point to allow an analytical investigation. Later, in Section 5, we discuss a model with spatially restricted connections and show that it is similar, in many aspects, to the model we consider here. The spatially restricted connections are illustrated in Fig. 2(B), with the corresponding parameters used in subsequent simulations.
Fig. 2

Connectivity pattern of the model. (A) Connection strength between one location, marked by the black dot, and all other locations. The connection strength was calculated by Eq. (6), using the data of Kenet et al. (2003), with J0 = −2, J2 = 5. (B) Connection strength for the model with spatially restricted connections. The connection strength was calculated by Eq. (48), using the same data set, with J0 = 3, J2 = 3.5, σ = 0.6 mm

We next consider the system given by Eqs. (4)–(6) for two types of afferent input: one corresponding to spontaneous activity (Section 3.1), and the other representing the afferent input evoked by the gratings stimulus (Section 3.2).

3.1. Spontaneous activity

To model spontaneous activity, we take the afferent input to be constant for all locations, i.e., Ixaff. We also assume C > T, otherwise all locations are below threshold, and the network does not develop any activity.

3.1.1. Reduced dynamics

We start the analysis of the model by reducing the dynamics given in (4) to the dynamics of two order parameters. To this end, we observe that the synaptic weights defined by Eq. (6) can be written as J2 Re(~zxzy)+J0, with zx, zy defined by Eq. (1). Thus, Eq. (5) can rewritten as:
$$I_x^{{\rm{rec}}} = {J_2}{\mathop{\rm Re}\nolimits} ({{\bar z}_x}{1 \over A}\int {dy} {z_y}{m_y}) + {J_0}{1 \over A}\int {dy{m_y}} $$
This form leads to the following definitions of the order parameters μ and Z:
$$\mu = {1 \over A}\int {dx{\rm{ }}{m_x}} $$
$$Z \equiv \rho {e^{i\psi }} = {1 \over A}\int {dx{r_x}{e^{i{\theta _x}}}{m_x}} $$
These variables represent global properties of the network activity: μ is the average firing rate and Z is the inner product of the firing rates with the PM. Equation (7) can be now rewritten as:
$$I_x^{{\rm{rec}}} = {J_2}{\mathop{\rm Re}\nolimits} ({{\bar z}_x}Z) + {J_0}\mu = {J_2}\rho {r_x}\cos ({\theta _x} - \psi ) + {J_0}\mu $$
To obtain equations for the evolution of μ and Z, we differentiate Eqs. (8) and (9) with respect to time, and then express x as a function of μ and Z using Eqs. (4) and (10). With these substitutions, we replace the spatial integration, i.e., the integration over y, with an integration over r and θ. As a result, the following equations are obtained:
$$\tau \dot \mu = - \mu + \int {{{\int {drd\theta {\rm{P}}\left( {r,\theta } \right)\left[ {{J_2}\rho r\left( {\theta - \psi } \right) + {J_0}\mu + C - T} \right]} }_{_ + }}} $$
$$\tau \dot Z = - Z + \int\!\!\!\int {drd\theta {\rm{P}}\left( {r,\theta } \right)r{e^{i\theta }}{{\left[ {{J_2}\rho r\cos \left( {\theta - \psi } \right) + {J_0}\mu + C - T} \right]}_ + }} $$
where the integrals are taken over all possible values of r and θ (i.e., r ∈ [0,∞], θ ∈ [−π, π]) and P(r, θ) is the joint probability density function of r and θ. Equations (11) and (12) are closed equations for the dynamics of μ and Z that do not depend directly on the particular values of mx. However, in general, these equations still depend on the angle of Z, i.e., ψ. Nevertheless, if isotropy is assumed, that is, the distribution of θ is uniform and independent of the distribution of r, this dependency upon ψ is lost. The validity of these assumptions is discussed later, in Section 4. In this case, we can rewrite Eqs. (11) and (12) as:
$$\tau \dot \mu = - \mu + \int\!\!\!\int {drd\theta {{{\rm{P}}\left( r \right)} \over {2\pi }}{{\left[ {{J_2}\rho r\cos \theta + {J_0}\mu + C - T} \right]}_ + }} $$
$$\tau \dot \rho = - \rho + \int\!\!\!\int {drd\theta {{{\rm{P}}\left( r \right)} \over {2\pi }}r\cos \theta {{\left[ {{J_2}\rho r\cos \theta + {J_0}\mu + C - T} \right]}_ + }} $$
$$\tau \rho \psi = 0$$
As usual, to find the fixed points for the dynamics, we set the time derivative in Eqs. (13) – (15) to zero. We discuss uniform (with ρ = 0), and non-uniform (with ρ > 0) solutions separately.

3.1.2. Uniform solutions

In the case of a uniform solution, Eq. (14) implies that the time derivative of ρ is zero for any μ. Equation (13) implies \(\mu = {\left[ {{J_0}\mu + C - T} \right]_ + }\) which can be solved for any J0 < 1. The solution for the uniform fixed point is therefore:
$$\mu = {{C - T} \over {1 - {J_0}}}$$
$$\rho = 0$$
The stability analysis is straightforward, as all neurons are above threshold, and the system near the fixed point is linear. We find that the fixed point is stable in the region J0 < 1 and J2 < 2.

3.1.3. Non-uniform solutions

To find the non-uniform solutions, we pull ρ out of the integrals in Eqs. (13) and (14) and rewrite them as:
$$\tau \dot \mu = - \mu + {J_2}\rho {F_0}\left( X \right)$$
)
$$\tau \dot \rho = - \rho + {J_2}\rho {F_2}\left( X \right)$$
with,
$$X = {{{J_0}\mu + C - T} \over {{J_2}\rho }}$$
and,
$${F_0}\left( X \right) = \int\!\!\!\int {drd\theta {{{\rm{P}}\left( r \right)} \over {2\pi }}{{\left[ {r\cos \theta + X} \right]}_ + }} $$
$${F_2}\left( X \right) = \int\!\!\!\int {drd\theta {{{\rm{P}}\left( r \right)} \over {2\pi }}r\cos \theta {{\left[ {r\cos \theta + X} \right]}_ + }} $$
In order to find the fixed point solutions for μ and ρ using Eqs. (18) and (19), it is necessary to know the functions F0 and F2 that in turn depend on the concrete choice of the selectivities’ distribution P(r) (Eqs. (21) and (22)). To study the properties of these functions, we first plot them for 3 different choices of P(r) (Fig. 3). In these examples, we see that the precise values that F0 and F2 take indeed depend on P(r). However, in all 3 cases, F0 is a non-decreasing function of X, approaching zero for large negative X, and increasing with a slope of 1 for large positive X. Similarly, in all 3 cases, F2 is a sigmoid-shaped function that saturates at 0 and 1/2. In fact, these observations about F0 and F2 are true in the general case. Formally, we show a set of properties of F0 and F2 that hold for any P(r). These properties are listed in Table 1, and their proof is given in the appendix. As we will show, they allow us to obtain solutions for the fixed point values of μ and ρ for an arbitrary P(r).
Fig. 3

The functions F0 and F2. (A) F0 plotted for 3 choices of P(r); Dashed: constant r, P(r) = δ(r − 1). Dash-dotted: exponential distribution, \({\rm{P}}(r) = {\sqrt {2e}^{ - \sqrt {2r} }}\). Solid: P(r) according to the isotropy-adjusted experimental PM (see Section 4 and Fig. 7 for details). (B) F2 plotted for the same distributions of r

No

Property

(i)a

F0(X) ≥ 0

(ii)b

F0(X) ≥ X

(iii)a

F0(X) ≥ 0

(iv)b

F0(X) ≤ 1

(v)a

F2(X) ≥ 0

(vi)b

F2(X) ≤ ½

(vii)c

F2(X) ≥ 0

(viii)

F0(X) − XFo(X) = F2(X)

(ix)b

F2(X) − XF2(X) ≤ ½

(x)a

F0(X)F2(X) − F0(X)F2(X) ≥ 0

The inequalities hold with equality if and only if:

ar is bounded and X ≤ −rmax or r is unbounded and X → −∞

br is bounded and Xrmax or r is unbounded and X → ∞

cr is bounded and |X| ≥ rmax or r is unbounded and |X| → ∞

We first find the fixed point value of X by setting the time derivative in Eq. (19) to zero. We find that the fixed point value of X is defined by:
$$1 - {J_2}{F_2}\left( X \right) = 0$$
Property (vi) in Table 1 directly implies that when J2 < 2, Eq. (23) cannot be satisfied, therefore a non-uniform solution does not exist (see also Fig. 4(C)). On the other hand, properties (v), (vi), (vii) imply that when J2 > 2 Eq. (23) has a unique solution (Fig. 4(D)). Therefore, when J2 > 2 there is a unique solution for the fixed point value of X. Assuming this solution is known, the following expressions for the fixed point value of µ and ρ in terms of the fixed point value of X can be derived form Eqs. (18), (20), and (23):
$$\mu = {{C - T} \over {X - {J_0}{F_0}\left( X \right)}}{F_0}\left( X \right)$$
$$\rho = {{C - T} \over {X - {J_0}{F_0}\left( X \right)}}{F_2}\left( X \right)$$
Since μ and ρ are by definition positive, it must hold that:
$$X - {J_0}{F_0}\left( X \right) > 0$$
otherwise, the solution diverges. Again, we can use the properties in Table 1 to find the conditions under which this inequality is satisfied. Property (ii) implies that inequality (26) never holds when J0 > 1 (Fig. 4(A)). On the other hand, if J0 < 1, XJ0F0(X) is a monotonically increasing function of X (properties (iii), (iv)), taking negative values for large negative values of X, and positive values for large positive values of X (properties (i), (ii)). Therefore, when J0 < 1, XJ0F0(X) = 0 has a unique solution. We denote this solution by X0 (Fig. 4(B)), and the solution of Eq. (23) by X2 (Fig. 4(D)). Using these notations, we find that inequality (26) holds for X = X2 if and only if X0 < X2. Since X0 is a function of J0 and X2 is a function of J2, the condition X0 < X2, together with the conditions J0 < 1, J2 > 2, define a region in the J0J2 plane, where the non-uniform fixed point exists. Stability analysis given in the appendix shows that when this fixed point exists, it is always stable.
Fig. 4

Solution and phase diagram of the spontaneous activity model. (A, B) The function XJ0F0(X) (see Eq. (26)). In (A), J0 = 1.5, representing the case where J0 > 1. The function XJ0F0(X) is negative for all X. In (B), J0 = −2, representing J0 < 1. The function XJ0F0(X) crosses zero at a unique point, which is denoted by X0. (C, D) The function 1 − J2F2(X) (see Eq. (23)). In (C) J2 = 1.5, representing the case where J2 < 2. The function 1 − J2F2(X) is positive for all X. In (D) J2 = 2.5 representing J2 > 2. The function 1 − J2F2(X) crosses zero at a single point, which is denoted by X2. (E) Phase diagram for the spontaneous activity. The border of the linear phase is given by the lines J0 = 1, J2 = 2. The line separating the marginal phase from the amplitude instability is determined by the condition X0 = X2. In all examples, F0 and F2 were calculated using the experimental P(r) (solid lines in Fig. 3)

3.1.4. Phase diagram

We turn to summarize the different regimes of the model and relate the fixed points in these regimes to the OMs. Since the voltage-sensitive dye optical imaging signal corresponds to the instantaneous post synaptic membrane potential rather than firing rate (Sterkin et al., 1999; Grinvald et al., 1999; Sharon and Grinvald, 2002; Petersen et al., 2003), we compare OMs to the total synaptic input, denoted by Itot = Irec + Iaff. The shape of Itot is given by (see Eq. (10)):
$$I_x^{{\rm{tot}}} = {J_2}\rho {r_x}\cos ({\theta _x} - \psi ) + {J_0}\mu + C$$
with μ, ρ, and ψ taking their fixed point values. The different phases of the model are summarized in the phase diagram (Fig. 4(E)). In the linear phase, J0 < 1, J2 < 2, the only solution is ρ = 0, and it is stable. In this case, Eq. (27) implies that Ixtot is uniform and does not correspond to any OM. However, in the marginal phase, J0 < 1, J2 > 2, X0 < X2 (where X0 is a function of J0 and X2 is a function of J2), the solution with ρ = 0 is unstable, but solutions withρ > 0 are stable. In fact, there are infinitely many solutions, because the value of ψ is arbitrary (Eq. (15)). By comparing Eq. (27) to Eq. (2), we see that in this case Ixtot equals a scaled version of the approximated OM of orientation ψ, up to an additive constant shift. Thus, in the marginal phase, OMs of arbitrary orientations emerge as fixed points of the network dynamics. Collectively, these fixed points form a ring attractor, that is, a ring of states where each state is an attractor of the dynamics.

Outside the regions of linear phase and marginal phase no stable fixed point exists and the network develops amplitude instability. Unlike the boundaries of the linear phase, the line X0 = X2 that separates the marginal phase from the amplitude instability region depends on P(r). However, some of its properties do not depend on P(r). In particular it can be checked that it always passes through the points (J0 = 1, J2 = 2), and (J0 = 0, J2 = 4).

3.2. Evoked activity

The case of a non-uniform input is more complex than the uniform input case, because the steady state solutions are sensitive to the fine structure of the input. However, some general observations about the fixed point can still be made. Since the synaptic input Irec is still given by Eq. (10), it equals a scaled/shifted version of one of the approximated OMs, and the total synaptic input is the sum of that map, and the afferent input, Iaff. The orientation of the approximated OM, and its scaling and translation parameters are still determined by ψ, ρ, and μ. Thus, the role of Iaff reduces to selecting the values of μ, ρ, and ψ, which will put the system in a steady state.

In this section we study a simple structure of the input; we add to the uniform input we studied in the previous section a small modulation that is proportional to one of the approximated OM. Formally we define:
$$I_x^{{\rm{aff}}} = C(1 + \varepsilon {r_x}\cos ({\theta _x} - {\psi _{{\rm{aff}}}}))$$
where ψaff is the orientation encoded in the input and ɛ is the modulation of the orientation encoding term. With this form, each location receives maximal afferent input when the orientation of the stimulus coincides with its preferred orientation. The main motivation for choosing this particular form is that it allows a relatively simple analytical study. Using simulations, we also studied different variations of the above input, and obtained essentially the same results. An example of such variation, is given in Section 4.

3.2.1. Reduced dynamics

Rewriting the right-hand side of Eq. (28) as \(C + {\mathop{\rm Re}\nolimits} ({{\bar z}_x}{Z_{{\rm{aff}}}})\) with \({Z_{{\rm{aff}}}} = C\varepsilon {e^{i{\psi _{{\rm{aff}}}}}}\), we find that:
$$I_x^{{\rm{rec}}} + I_x^{{\rm{aff}}} = {\mathop{\rm Re}\nolimits} ({{\bar z}_x}({J_2}Z + {Z_{{\rm{aff}}}})) + {J_0}\mu + C$$
where μ and Z are defined by the same equations given for the spontaneous activity (Eqs. (8) and (9)). Equation (29) leads to the following definition:
$${Z_{{\rm{tot}}}} = {\rho _{{\rm{tot}}}}{e^{i{\psi _{{\rm{tot}}}}}} = {J_2}Z + {Z_{{\rm{aff}}}}$$
and with these notations:
$$\tau {{\dot m}_x} = - {m_x} + {[{\rho _{{\rm{tot}}}}{r_x}\cos ({\theta _x} - {\psi _{{\rm{tot}}}}) + {J_0}\mu + C - T]_ + }$$
As in the case of spontaneous activity, Eq. (31) can be used to derive dynamical equations for μ and Z:
$$\tau \dot Z = - Z + {e^{i{\psi _{{\rm{tot}}}}}}\int\!\!\!\int {drd\theta {{{\rm{P}}(r)} \over {2\pi }}r{e^{i(\theta - {\psi _{{\rm{tot}}}})}}{{[{\rho _{{\rm{tot}}}}r\cos (\theta - {\psi _{{\rm{tot}}}}) + {J_0}\mu + C - T]}_ + }} $$
$$\tau \dot \mu = - \mu + \int\!\!\!\int {drd\theta {{{\rm{P}}(r)} \over {2\pi }}{{[{\rho _{{\rm{tot}}}}r\cos (\theta - {\psi _{{\rm{tot}}}}) + {J_0}\mu + C - T]}_ + }} $$
This can be rewritten as:
$$\tau \dot \mu = - \mu + {\rho _{{\rm{tot}}}}{F_0}(X)$$
$$\tau \dot \rho = - \rho + {\rho _{{\rm{tot}}}}\cos ({\psi _{{\rm{tot}}}} - \psi ){F_2}(X)$$
$$\tau \rho \dot \psi = {\rho _{{\rm{tot}}}}\sin ({\psi _{{\rm{tot}}}} - \psi ){F_2}(X)$$
with,
$$X = {{{J_0}\mu + C - T} \over {{\rho _{{\rm{tot}}}}}}$$
In this formulation, it is assumed that ρtot > 0. However it can be easily checked that ρtot = 0 is not a fixed point solution, therefore the case of the uniform solution can be disregarded.

3.2.2. Solution

At fixed point, Eq. (36) implies that sin(ψtotψ) = 0. Therefore, it must hold that either ψtot = ψ or ψtot = ψ + π. However, in the second case, Eq. (35) implies ρ = −ρtotF2(X), which cannot hold because both ρ and ρtot are positive, and F2(X) is non-negative (property (v) in Table 1). Thus, as follows from Eq. (30), the possible solutions are ψtot = ψ = ψaff, and ψtot = ψ = ψaff + π. We focus on the first case, which corresponds, as the analysis in the appendix shows, to the only stable fixed point. In this case, ρtot = J2ρ + C∈. Setting \(\dot \mu = 0\), \(\dot \rho = 0\) we rewrite Eqs. (34), (35), (37) as:
$$\mu = ({J_2}\rho + C\varepsilon ){F_0}(X)$$
$$\rho = ({J_2}\rho + C\varepsilon ){F_2}(X)$$
$$X = {{{J_0}\mu + C - T} \over {({J_2}\rho + C\varepsilon )}}$$
Thus, a fixed point requires that this system of 3 equations for the 3 variables μ, ρ, and X has a solution with positive μ and ρ. Solving this system, we find that the fixed point value of X is determined by:
$${{1 - {J_2}{F_2}(X)} \over {X - {J_0}{F_0}(X)}} = {\rm{\gamma }}$$
where ϒ is the effective stimulus tuning (Hansel and Sompolinsky, 1998), defined by:
$${\rm{\gamma }} = {{\varepsilon C} \over {C - T}}$$
The solution for μ and ρ is given by the same equations we derived for the spontaneous activity (Eqs. (24) and (25)), and, as it was in that case, these equations imply that inequality (26) must hold. In addition, in the evoked case, because ϒ > 0, it must also hold that:
$$1 - {J_2}{F_2}(X) > 0$$
Taken together, a fixed point solution requires finding X that satisfies Eq. (41) and inequalities (26), (43). Thus, we check the existence of such a solution in the different regions of the J0J2 plane.
As in the case of the spontaneous activity, when J0 > 1, inequality (26) never holds, and no fixed point exists. In the region corresponding to the linear phase of the spontaneous activity, i.e., J0 < 1, J2 < 2, inequality (43) holds for any X, and inequality (26) holds for any X > X0, where X0 is a function of J0 (Section 3.1.3). On the interval X > X0 the left hand side of (41) is a monotonically decreasing function of X, approaching ∞ at X = X0, and 0 at X → ∞ (Fig. 5(A)). Therefore, in this region, Eq. (41) has a unique solution for any (positive) ϒ, and a unique fixed point exists.
Fig. 5

Solution and phase diagram of the evoked activity model. (A, B) The function \({{1 - {J_2}{F_2}(X)} \over {X - {J_0}{F_0}(X)}}\) (see Eq. (41)). In (A) J0 = −3, J2 = 1.5, representing the case where J0 < 1 and J2 < 2. On the interval X > X0 the function takes all possible positive values, and a unique solution for X exists for every value of ϒ. In (B) J0 = −3, J2 = 3.5, representing the case where J0 < 1, J2 > 2, and X0 < X2. On the interval X0 < X < X2 the function takes all possible positive values, and a unique solution for X exists for every value of ϒ as well. (C) Phase diagram for the evoked activity. The line separating the linear phase from the non-linear phase is plotted for few values of ϒ (0.05, 0.1, 0.15, 0.2, 0.25, bottom-right to top-left). In all examples, F0 and F2 were calculated with the experimental P(r) (solid lines in Fig. 3)

In the region corresponding to the marginal phase of the activity, i.e., J0 < 1, J2 > 2, X0 < X2, where X0 is a function of J0 and X2 is a function of J2, (Section 3.1.3) inequalities (26) and (43) imply X0 < X < X2. On the interval X0 < X < X2, the left hand side of Eq. (41) is monotonically decreasing of X, approaching ∞ at X = X0, and taking the value of 0 at X = X2 (Fig. 5(B)). Therefore, in this region, Eq. (41) has a unique solution for any (positive) ϒ, and a unique fixed point exists. Finally, in the region J0 < 1, J2 > 2, X0 > X2, inequalities (26) and (43) cannot be satisfied simultaneously, therefore, no fixed point exists.

3.2.3. Phase diagram

In sum, the above analysis shows that the region where a fixed point exists for the evoked activity, coincides precisely with the regions where a fixed point exists for the spontaneous activity, i.e., the regions of the linear and marginal phases of the spontaneous activity. The stability analysis in the appendix shows that this fixed point is stable, and that any other fixed point in this region is unstable. Similar arguments to those presented for the marginal phase of the spontaneous activity imply that the total synaptic current is a scaled/shifted version of an OM. However, unlike the case of the spontaneous activity where the orientation of the OM was arbitrary, for the evoked activity, the orientation of the OM is bound to match the orientation of the afferent input.

Although Itot is always a scaled/shifted OM, when firing rates are taken into account, we find that two types of solutions exist. The fixed point solution for the firing rate is given by:
$${m_x} = {\rho _{{\rm{tot}}}}{[{r_x}\cos ({\theta _x} - {\psi _{{\rm{aff}}}}) + X]_ + }$$
Thus, if the selectivities (rx) are bounded, i.e., there exists rmax with P(r > rmax) = 0, and X is large enough, all locations are above threshold, and the system at fixed point is effectively linear. Otherwise, at least some locations are below threshold, and the system is non-linear. This distinction between linear and non linear solutions is important because linear solutions are associated with broad tuning curves, whereas non-linear solutions are associated with narrow tuning curves (Hansel and Sompolinsky, 1998). If J2 > 2, the system is non-linear for any ϒ, because the fixed point solution for X is smaller than X2, which in turn is smaller that rmax (property (vi)). However, if J2 < 2, the fixed point solution for X can take arbitrarily large values, and both linear and non-linear solutions are possible. The line separating the linear regime from the non linear one for the evoked case, can be obtained by plugging X = rmax in Eq. (41). Using the fact that F0(rmax) = rmax and F2(rmax) = ½ (properties (ii), (vi)) we find that:
$${J_0} - 1 = {{{J_2} - 2} \over {2{\rm{\gamma }}{r_{\max }}}}$$
Thus, we find that this line always passes through the point (J{nu0} = 1, J2 = 2) and its slope (where the line is viewed as a function of J2) is inversely proportional to ϒ and rmax. This line for a few choices of ϒ is drawn in the phase diagram of the evoked activity (Fig. 5(C)). In the limit ϒ → 0, this line becomes J2 = 2, i.e., the line separating the linear phase from the marginal phase of the spontaneous activity. On the other hand, when ϒ → g8 the system switches from linear solutions to non-linear ones, for any choice of J0 and J2.

3.3. Energy function

The connectivity matrix of our model is symmetric; therefore, an energy function for the model can be constructed. The energy function decreases during the evolution of the system, and the fixed points are local minima of the energy function. The appropriate energy function is given by Hopfield, (1984) and Hertz et al. (1991):
$$H = - {1 \over {2{A^2}}}\int\!\!\!\int {dxdy{W_{xy}}{R_x}{R_y} + {1 \over {2A}}\int {dxR_x^2} } - {1 \over A}\int {dx{R_x}(I_x^{{\rm{aff}}}} - T)$$
where \({R_x} = {[I_x^{{\rm{rec}}} + I_x^{{\rm{aff}}} - T]_ + }\). Using a derivation similar to the one we used to reduce the dynamics of the system to the dynamics of the order parameters, Eq. (46) can be rewritten as a function of the order parameters μ and Z:
$$H = - {1 \over 2}\rho _{{\rm{tot}}}^2({J_0}{F_0}{(X)^2} + {J_2}{F_2}{(X)^2}) + {1 \over 2}({\rho _{{\rm{tot}}}}({J_0}\mu - C + T){F_0}(X) + ({({J_2}\rho )^2} - {(\varepsilon C)^2}{F_2}(X))$$
with ρtot and X being functions of μ and Z, according to the definitions given for the evoked activity (Eqs. (30) and (37)), and the spontaneous activity treated as a particular case with = 0.
Usually, energy functions of neural networks are high dimensional and cannot be directly visualized. However, Eq. (47) reveals that the energy function for this system is a function of one complex and one real variable (Z and μ). In the case of J0 = 0 the energy is a function of Z only, and therefore can be easily visualized. Figure 6 summarizes the main results we obtained in previous sections, by showing how the energy function looks like in the different regimes of the system. In Fig. 6(A)(C) the energy is drawn as a function of Z for spontaneous activity. In Fig. 6(A), the model is in the linear phase, and the energy function is similar to a cone. Only one local minimum exists and it is at the center of the “cone”, i.e., at Z = 0. This local minimum corresponds to the uniform solution with ρ = 0. In Fig. 6(B), the model is in the marginal phase, and the energy function is similar to a “mexican hat”. The ring of local minima corresponds to the infinite number of solutions with ρ > 0. In Fig. 6(C), no local minima exist, and the system undergoes amplitude instability. In Fig. 6(D), the case of evoked activity is depicted. The energy function is similar to the energy function of the spontaneous activity in the marginal phase, Fig. (6B), but “tilted” in the direction defined by the angle ψaff. Thus, the continuum of local minima is lost, and only one local minimum remains. While the afferent input we used to generate Fig. 6(D) is of the type we studied in Section 3.2, qualitatively similar results can be obtained for many other types of inputs. This is because the ring attractor, as depicted in Fig. 6(B), is singular; any small anisotropy in the afferent input will destroy its prefect symmetry, and will cause the network to converge to particular values of Z, representing specific OMs.
Fig. 6

Energy function. In the examples given in this figure we assume J0 = 0, and the energy, H, depends only on the complex order parameter Z. (A, B, C) The energy as a function of Z for spontaneous activity. (A) J2 = 1, linear phase. (B) J2 = 3, marginal phase. (C) J2 = 5, amplitude instability. (D) The energy as a function of Z for evoked activity with J2 = 3, = 0.04. In all examples F0 and F2 were calculated with the experimental P(r) (solid lines in Fig. 3). The values of other parameters were C = 3, T = 1. For clarity the energy was drawn only for values of Z with |Z| < 3.3

4. Simulations

To illustrate the theoretical results obtained in the previous section and validate their applicability to experimental data, we applied the model to the Kenet et al. (2003) data set. An important assumption we made in the previous section is that of isotropy of orientation preference (Section 3.1.1). The isotropy assumption requires the distribution of the values of zx to be rotationally symmetric about the origin. The experimental distribution of zx, which is shown in Fig. 1(B), does not seem to deviate strongly form the rotational symmetry assumption. This is consistent with some previous optical imaging studies in cats that either did not find any systematic over-representations of specific orientations (Bonhoeffer and Grinvald, 1993; Kenet et al., 2003), or found small, albeit significant in some cases, over representation of cardinal orientations (Müller et al., 2000; Yu and Shou, 2000; Dragoi et al., 2001b; Wang et al., 2003). However, there are many factors that might distort the precise distribution and mask the true extent of the anisotropy. In particular, the fact that only a small fraction of V1 is imaged can increase the anisotropy in the imaged map compared to the true one, whereas measurement noise of the optical imaging data, can artificially induce an opposite effect.

In this work, we avoid the precise characterization of the anisotropy in the cat data. Instead, we first adjusted the polar map (PM) to match the isotropy assumption, and then quantified the deviation between the experimental and the adjusted PMs. The adjustment was performed in two steps. In the first step, we adjusted the values of rx such that for any angle θx the conditional distribution of rx given θx will match, approximately, the marginal distribution of rx. This was achieved by considering for every pixel x, the set of pixels ⨑ub;y⫂ub; such that \(\left| {\left. {{\theta _x} - {\theta _y}} \right| \le {\pi \over {10}}} \right.\). Next, we calculated px, defined as the percent of pixels with ryrx. Then, rx was set to the px-th percentile of the distribution of the original values of rx of all the pixels. In the second step, the pixels were sorted according to the value of rx and divided into 12 groups of approximately equal size. The values of r at each group were set to the average value at the group. Then, the pixels at each group were sorted according to θx, and the values of θx were adjusted such that difference between two adjacent θs was made constant. The distribution of the values of zx after the adjustment is shown in Fig. 7(B). This distribution is rotationally symmetric, and meets the isotropy assumption.
Fig. 7

(A) PM for the data obtained in Kenet et al. (2003) after isotropy adjustment. The figure is displayed on the same brightness scale as in 1A to facilitate comparison. (B) Distribution over the complex plane of the values of zx of the PM in (A)

The PM after isotropy adjustment is shown in Fig. 7(A) and it is very similar to the experimental PM shown in Fig. 1(A). Since our main motivation of using the PM is its ability to provide an approximation to the OMs (Eq. (2)), we quantified the distortion caused by the isotropy adjustment by calculating for every orientation the correlation between the OM approximated with the experimental PM and the OM approximated with the adjusted PM. The average correlation over all possible orientations was 0.971 (min: 0.968, max: 0.973). These high correlations indicate that the original map was not very far from meeting the isotropy assumption in the first place. The average correlation between the 8 experimental OMs and the OMs approximated with the adjusted PM was high as well (0.791; min: 0.725, max: 0.855). We therefore used the adjusted PM to construct the connectivity matrix (Eq. (6)) and tested the model in the spontaneous activity and the evoked activity regimes.

Our analysis of the spontaneous activity in the marginal phase showed that the steady state synaptic input (Irec + C) is a scaled/shifted version of an approximated OM. The orientation of theOMis arbitrary and therefore depends only on the initial state, i.e., mx at t = 0. We thus performed 10000 simulations with random initial states. The simulations were performed using MATLAB®, using first order Euler method to solve Eq. (4). Each simulation was let run until convergence. The parameters of the model were fixed for all simulations, with J0 and J2 selected to be in the marginal phase (see Fig. 8 for a full list of parameters’ values).
Fig. 8

Simulations of spontaneous activity. (A) Initial state in one simulation. (B) The total synaptic input at steady state for the same simulation. (C) The experimental OM of 90°, which is very similar to the state in (B). (D) Distribution of the orientations of the approximated OMs at steady state in 10000 simulations. The number of bins was 8. The following values of parameters were used in all simulations: J0 = −2, J2 = 5, τ = 10, C = 2, T = 1. The random initial state was taken from a gaussian distribution with mean 1 and variance 0.25. Equation (4) was solved using the first order Euler method with Δt = 1. The simulation was run until t = 500, a value that was large enough to guarantee the convergence to a fixed point. Grayscale range (arbitrary units): (A) 0.2–2, (B) −9–7, (C) −2.4–2.7

An example of a single simulation is presented in Fig. 8(A)–8(C). Figure 8(A) shows the initial state. Figure. 8(B) shows the total synaptic input, Irec + C, at the fixed point. It is a linear transformation of the approximated OM of 90°. The highly similar experimental OM of 90° is presented in Fig. 8(C) for comparison (correlation = 0.85). The steady states of all simulations where similar to one of the experimental OMs (correlation >0.7). The high similarity between the simulated and experimental maps demonstrates that the model produced a good approximation for the experimental OM. It also implies that any violations of the approximation assumptions, (ability to approximate OMs with the PM and isotropy) did not cause any significant distortion of the OM. In Fig. 8(D) we show the distribution of the orientations of the steady state OMs for all 10000 simulations. The orientation was calculated by taking the angle of the projection of the synaptic input at the fixed point on the PM. Consistently with our theoretical analysis, this figure shows that the OM at the fixed point was arbitrary, with equal probability for all orientations.

We next tested the model with a tuned input, with = 0.1 and ψaff selected randomly from a uniform distribution. In order to test the model with a more complex afferent input than the one we analyzed in the previous section, we added to the afferent input gaussian noise with zero mean and a standard deviation of 0.1. Thus, noise in the afferent input was equal in amplitude to . The other parameters of the simulation were identical to those of the spontaneous activity simulation, and the simulation was also performed 10000 times. The results are shown in Fig. 9. Figures 9(A)(C) showan example of a single simulation. Figure 9(A) shows the afferent input. The orientation used to construct the input was 0°. The steady state is shown in Fig. 9(B). It is very similar to the experimental OM of 0° degrees which is shown in 9C. Note that the noise we added in the afferent input is filtered out by the recurrent connections. Finally, in Fig. 9(D), we show the distribution of the difference between the afferent input orientation and the steady state orientation. The narrow distribution around zero (SD = 2.2°) shows that the input orientation and the steady state orientation are very close to one another despite the noise. A comparison of Fig. 8(D) and Fig. 9(D) demonstrates the main difference between the spontaneous and evoked conditions in the marginal phase. In the former, the OM is chosen randomly whereas in the latter, it is chosen by the input. However, in both cases, the steady states are similar to OMs.
Fig. 9

Simulations of evoked activity. (A) Afferent input in one simulation. The input is constant, with a small modulation in the shape of the approximated OM of 0°( = 0.1,ψaff = 0) and a random gaussian noise (mean= 0, std= 0.1). (B) The total synaptic input at steady state for the same simulation. The 0δ OM is extracted from the afferent input. (C) The experimental OM of 0δ, which is very similar to the state in (B). (D) Distribution of the difference between the orientation of the approximated OM at steady state and the orientation encoded in the afferent input for 10000 simulations. The number of bins was 100. The parameters for these simulations were identical to the parameters used for the spontaneous activity simulations. Grayscale range (arbitrary units): (A) 1.5–2.5, (B) −9–7, (C) −2.4–2.7

5. Model with spatially restricted connections

The connectivity pattern we considered so far (Eq. (6)) allowed a thorough mathematical analysis, which we presented in Section 3. However, this pattern is not realistic when large cortical areas are considered because the synaptic weights do not fall off with the distance between the pre- and postsynaptic locations. In contrast, lateral cortical projections in the cortex are spatially restricted. In the cat V1 these projections do not exceed 3.5 mm, with most axons projecting up to a distance of 0.5 mm (Gilbert and Wiesel, 1989; Kisvárday et al., 1997; Yousef et al., 2001). In this section we extend the model by imposing a restriction on the spatial extent of the lateral connections. This restriction is achieved by multiplying the synaptic strength given by Eq. (6) with a spatially isotropic gaussian of the distance between the pre- and postsynaptic locations, i.e.:
$${W_{xy}} = ({J_2}{r_x}{r_y}\cos ({\theta _x} - {\theta _y}) + {J_0}){G_\sigma }(\left\| {\left. {x - y} \right\|} \right.)$$
where σ denotes the standard deviation of the gaussian that determines the spatial range of the connections, Gσ denotes a 2-dimensional gaussian function (\(G\sigma (\xi ) = {1 \over {2\pi {\sigma^2}}}{e^{ - {\xi^2}/(2{\sigma^2})}}\)) and ‖xy‖ is the distance between locations x and y. An example of the connectivity pattern induced by Eq. (48) for the Kenet et al. (2003) data set, is given in Fig. 2(B). Like the connectivity pattern induced by the spatially unrestricted model (Eq. (6), Fig. 2(A)), it exhibits a patchy structure, with patches corresponding to regions with similar orientation preference. However, now the patches are spatially restricted, and their density decays as a function of the distance between the interconnected locations. Both properties have been observed experimentally (Gilbert and Wiesel, 1989; Malach et al., 1993; Bosking et al., 1997; Kisvárday et al., 1997).
While the connectivity given by Eq. (48) is more realistic than the one given by Eq. (6), the behavior of the model is now much more complicated. This is because it depends not only on the distribution of preferred orientations and selectivities, but also on the spatial structure of the PM. One implication of this additional complexity is the loss of isotropy. While the PM can be still isotropic over the entire region simulated, each location is now connected to only a small subregion; in this subregion isotropy does not necessarily hold. One might expect that this would disrupt the formation of the ring attractor, and indeed simulations of the spatially restricted model with a uniform input yielded only one or two attractor states, depending on the model parameters (e.g., J2, σ). These attractor states were usually similar to OMs, with the identity of the attractor-OM also depending on the precise selection of the model parameters. A similar phenomenon, in which the ring attractor collapses into few isolated attractors, occurs in the ring model when some degree of anisotropy is introduced, i.e. when the distribution of preferred orientations is not uniform (Tsodyks and Sejnowski, 1995; Zhang, 1996; Renart et al., 2003; Goldberg et al., 2004). Thus, the spatially restricted model does not support a ring attractor in the sense we discussed so far. However, if the spatial range of the interactions is large enough, such that the recurrent input received by each location is approximately isotropic, the network has an unlimited number of states that have energy levels similar to the attractors. We therefore expected that these states could be revealed by adding a random noise to the uniform input. To explore this possibility we simulated the network with many realizations of the random noise. The random noise was taken to be a gaussian noise with small spatial correlations induced by filtering it with a 2-dimensional spatially isotropic gaussian kernel with a small standard deviation (0.1 mm) (see a similar approach in Goldberg et al. (2004)). An example of two realizations of the input is given in Fig. 10(A), 10(D). To allow comparison between the spatially restricted and unrestricted models under the same conditions, we repeated the simulations of spontaneous activity in the spatially unrestricted model (Section 4) using the noisy input we used for the spatially restricted model. The results for the unrestricted model with the noisy input were very similar to the results with the uniform input, implying that our characterization of spontaneous activity in Section 4 holds also for the noisy input.
Fig. 10

Simulations of the model with spatially restricted connectivity. (A, D) Afferent input for two simulations of spontaneous activity. (B, E) The total synaptic input at steady state for the same simulations. (C, F) Mosaics of experimental OMs that are highly similar to the steady states in (B) and (E). (G) Afferent input in a simulation of evoked activity encoding an orientation of 45δ. (H) Steady state for the same simulation. (I) The experimental OM of 45δ. The small bars on the right-hand side of the figure indicate the orientations of the OMs in (C), (F), (I). The values of the parameters were J0 = −3, J2 = 3.5, σ = 0.6 mm, C = 2, T = 1. The noise in the input was random gaussian with zero mean, 0.1 standard deviation, and it was spatially filtered with a spatially isotropic gaussian with a standard deviation of 0.1 mm. Grayscale range (arbitrary units): (A, D, G) 1.1–2.9, (B, E) −0.8–4.6, (H) −1.3–4.3, (C, F, I) −2.4–2.7

For the spatially restricted model we constructed the lateral connectivity with Eq. (48), using the same PM we used for the simulations in Section 4. We took σ = 0.6 mm, which is approximately the diameter of a hypercolumn. It has been estimated that in cats most projections are approximately within this distance (Kisvárday et al., 1997; Schummers et al., 2002). The global parameters of the lateral connections were J0 = −3 and J2 = 3.5, which correspond to the marginal phase of the spatially unrestricted model. Any single realization of the input, such as the ones in Fig. 10(A) and (D), yielded one particular steady state solution. However, for different realizations of the input, the network had an unlimited number of attractors states. Some of these states were similar to OMs, like in the case of the spatially unrestricted model. Still, other steady states were not highly correlated with OMs. An example of two such states is given in Fig. 10(B) and (E). A close examination of these states revealed that they were mosaics of OMs, i.e., they were composed of different OMs in different regions of the modeled area. The structure of these states is demonstrated in Fig. 10(C), 10(F). For example, in Fig. 10(C), the OM of 22.5° is presented in the upper part, and the OM of 67.5° is presented in the lower part. This combination of OMs presented in Fig. 10(C) produces a state that is very similar to the steady state shown in Fig. 10(B). We conclude that with noisy afferent input the model with spatially restricted connections is qualitatively similar to the model of spontaneous activity we considered in Section 3.1, except that steady states can be mosaics of OMs, rather than a single OM.

We also tested this model in the evoked activity case, by adding a small modulation in the shape of an approximated OM to the noisy input described above (similarly to our approach in Section 4). An example of such input is presented in Fig. 10(F) where the orientation of the OM was 45°. The steady state for this simulation is presented in Fig. 10(G), and the experimental OM for 45° is presented in 10H. The steady state is similar to the OM encoded in the input, and the noise is filtered out. This was the case for all simulations. Thus, for evoked activity, the behavior of the model with spatially restricted connections is very similar to the behavior of the model with unrestricted connections.

6. Discussion

The emergence of OMs during spontaneous activity in anaesthetized cats has led to the suggestion that OMs are attractors of the intracortical network (Kenet et al., 2003). Adopting this view, we posed the following question: what type of intracortical dynamics and connectivity can lead to the formation of attractors at the OMs? In this work, we suggested a rate model endowed with a simple connectivity rule (Eq. (6)), and showed that it yields attractor states that are highly similar to OMs. Specifically, we showed analytically that in the proper parameter regime (marginal phase), and given a uniform input, the model has a ring attractor formed by approximated OMs of arbitrary orientations. This property explains the formation of OMs during spontaneous activity where the afferent input is assumed to be unstructured. We also considered the case where the activity is evoked by a visual stimulus and showed how a structured afferent input can select the OM that matches the stimulus’ orientation. The model therefore suggests that OMs are encoded in the lateral connections, and that these connections can generate OMs both when the activity is spontaneous and when it is evoked by a visual stimulus.

6.1. Selectivity

The core of our model is its connectivity pattern (Eqs. (6), (48)). This connectivity pattern depends on three properties of the interconnected sites: the selectivity, the preferred orientation, and the distance between the pre- and post- synaptic locations. The main theoretical innovation in our model is the explicit incorporation of the selectivity variable into the connectivity pattern. We treat the selectivity as a structural attribute of every cortical location, similarly to how the preferred orientation is dealt with by many models (including ours). We have shown that this approach is sufficient for explaining the correspondence between OMs and intrinsically preferred states of the cortical network. The model therefore predicts a different connectivity pattern for neurons near pinwheels and neurons in linear zones. There are some experimental evidence in favor of this view. Yousef et al. (2001) reported that connections of neurons near pinwheels have shorter lateral extent that neurons in linear zones, and that these connections are not as orientation-specific as those of neurons in linear zones. The latter result is consistent with our connectivity matrix, in which the orientation-dependent term in the equation of synaptic strength (Eq. (6)) is multiplied by the selectivity.

In the context of studying orientation selectivity, it is important to distinguish between two types tuning curves: of membrane potential tuning curves, on which we focused in this contribution, and commonly considered firing rate tuning curves. These two types of tuning curves give rise to two different measures of selectivity, and the closely related measures of tuning width. In our model, the shape of the membrane potential tuning curve is the same for all locations (a cosine shape). The different selectivity of different locations comes from different scaling of the tuning curve. As for firing rate tuning curves, the picture is more complicated. Equation (44) implies that the selectivity and width of firing rate tuning curves vary with membrane potential selectivity. The precise characteristics of the firing rate tuning curve depends on the model parameters (J0, J2), and both broad and narrow tuning curves are possible. In the linear regime, all locations have broad tuning curves. In the non-linear regime, both broad and narrow tuning curves are possible.

Several experimental studies used recordings of single neurons to study the dependence of selectivity and other tuning curve properties on the cortical location. Before we compare our model with these studies, it is important to note that the low selectivity near pinwheels seen with optical imaging, which is at the heart of our modeling approach, is difficult to interpret in terms of single neuron properties. This is because it can arise from either lack of orientation selectivity of single neurons near the pinwheel, or from averaging highly selective neurons with a highly variable orientation preference. The second option received support form extra-cellular recordings (Maldonado et al., 1997; Dragoi et al., 2001a) that found that as far as firing rates are concerned, neurons near pinwheels are as sharply tuned as neurons in linear zones. However, intracellular recordings (Schummers et al., 2002) have shown that when tuning curves of sub-threshold membrane potential are considered, the selectivity of neurons is indeed reduced with the selectivity given by the PM. Our model, in which selectivity varies both for synaptic-input and firing-rate tuning curves, is therefore consistent with membrane potential tuning curves, but not with firing rate tuning curves. As suggested by Schummers et al. (2002), this difference between the tuning of membrane potential and the tuning of firing rate might be accounted for by different parameters of the gain function (e.g., firing thresh-old) for neurons near pinwheels. This suggestion can be easily incorporated into our model. Other mechanisms were suggested, some of them, like sensitivity to the temporal structure of the membrane potential fluctuations (Volgushev et al., 2002), are beyond the framework of the rate models we discussed here.

The notion of tuning curves, is clearly associated with evoked activity. However, the principles laid down in this work can lead to the definition of “spontaneous tuning curves”. In an experimental setup, if optical imaging of spontaneous activity is performed simultaneously with single unit recordings, a spontaneous tuning curve can be defined as the average firing rate (measured electrophysiologically) as a function of the orientation encoded by the cortical state (measured by the optical imaging). It would be interesting to characterize spontaneous tuning curves and compare them with the classical, evoked, tuning curves. In one particular scenario our model predicts a qualitative difference between spontaneous and evoked tuning curves. A comparison between the phase diagrams for spontaneous and evoked activity (Figs. 4(E), 5(C)) implies that if V1 operates near the phase transition between the linear and marginal phases of spontaneous activity, (J2 close to 2), spontaneous tuning curves are expected to be significantly broader than the evoked ones. This prediction can be tested experimentally.

6.2. Orientation preference

Similarly to other models, our connectivity pattern suggests a more excitatory (or less inhibitory) coupling between neurons with similar orientation preference. This is in agreement with many experimental works that have shown that neurons tend to make connections with other neurons with a similar orientation preference. It has been argued that this is only a weak trend that is true only on average (Kisvárday et al., 1997). This does not necessarily contradict our model because the model considers populations of neurons and not single neurons. In addition, both in the linear and in the marginal phases, the ratio between the preferred-orientation dependent and independent components can be made arbitrarily small, by taking large negative value for J0.

Another challenge for our model is the fact that the dependence of the connectivity on orientation preference is primarily true for long range connections, i.e. connections between different hypercolumns. Moreover, it has been suggested that short-range connections are spatially isotropic (Ts’o et al., 1986; Das and Gilbert, 1999) (note that isotropy here refers to invariance with respect to direction on the cortical sheet, unlike the isotropy in the feature space we considered before). In contrast, our model does not distinguish between long-range and short-range connections, and in particular it does not include explicitly isotropic short range connections. We focused on the long-range type connections because isotropic connections alone, are hard to reconcile with the spontaneous emergence of OMs. This type of connectivity is invariant with respect to spatial rotation/ translation. Consequently, the spontaneous patterns it induces are also invariant with respect to rotation/translation. However, experimentally observed patterns of spontaneous activity are correlated with the OMs and do not exhibit this invariance (Kenet et al., 2003). It is also important to note that because of the dependence of the connectivity on the selectivity, our model does not exhibit a strong spatial anisotropy of the short-range connections. In the linear zones, neurons are surrounded by neurons with similar orientation preference, and therefore they exhibit roughly isotropic connections. Near pinwheels this is not true anymore; However there the connectivity depends only weakly on the orientation preference, and the global component (J0) dominates the connectivity, producing again a roughly isotropic connectivity. Nonetheless, an explicit incorporation of isotropic short range connections might be considered in a future work.

6.3. Spatial structure of the OMs

Two assumptions about the structure of the OMs were required for the formation of the ring attractor. The first assumption is that OMs can be well-approximated using the PM (Eq. (2)), that is, by considering at each location only the preferred orientation and selectivity. This assumption can be expressed by the condition that on average, a large fraction of the variance of the pixels’ tuning curves can be explained by its first Fourier component (Eq. (3)). We found that this condition is met in experimental data sets, and that individual OMs are highly correlated with their approximated versions. We note however, that the approximation of orientation tuning curves with cosine functions, as in Eq. (2), was criticized recently on the ground that it cannot describe the height and width of the tuning curves independently (Swindale et al., 2003). While deviations from cosine tuning curves can be functionally important, our analysis indicates that the PM provides a good first-order approximation for the OMs. The examples in Figs. 810 of individual experimental OMs compared with the output of the model, further support this claim.

The second assumption we made about the OMs is that the representation of orientation preference in the PM is isotropic, that is, that the distribution of preferred orientations is uniform and independent of the distribution of selectivities. For the experimental data set, we showed that orientation preference did not deviate strongly from the isotropy assumption. As discussed in Section 4, this result is consistent with some experimental studies in cats. Other studies, however, found a small bias of the distribution of preferred orientations towards the cardinal orientations. Interestingly, Kenet et al. (2003) reported a similar cardinal-orientations bias in the distribution of spontaneous OMs. It is thus conceivable that a small anisotropy in the distribution of preferred orientations indeed exists in cats’ V1, and that this anisotropy induces a significant anisotropy in spontaneous activity patterns. While our current model does not account for anisotropy, it provides a framework in which the effects of anisotropy in V1 could be addressed in future works.

6.4. Relation to the ring model

The model we presented here is strongly related to the ring model for feature selectivity (Ben-Yishai et al., 1995; Hansel and Sompolinsky, 1998). In fact, the ring model can be viewed as a special case of our model, in which all locations have the same selectivity. Therefore, these two models share some features that do not depend on the distribution of selectivities P(r), e.g., the general structure of the phase diagrams (Figs. 8(E) and 9(C)). Since in our model multiple selectivities are allowed, the attractor states can exhibit a much richer spatial structure and approximate the experimental OMs.

6.5. Lateral extent of the connections

For most of the work, the connectivity pattern that we considered (Eq. (6)), did not depend on the distance between the pre- and post- synaptic locations. In contrast, neuronal connections in the cortex have a limited lateral extent. Therefore, in Section 5, we suggested a modification of the connectivity rule in which the connections were spatially restricted. Using simulations, we showed that this version of the model has properties similar to the spatially unrestricted one, except for one important difference: the states generated during spontaneous activity are not necessarily OMs, but can be a mosaic of several OMs. Consequently, we predict that such states exist in spontaneous activity data. Preliminary analysis we of the data obtained in Kenet et al. (2003), indicates that mosaic states indeed emerge spontaneously.

It is important to note that the distinction between OM states and mosaic states is somewhat blurred. An OM state might in fact be a small part of a larger mosaic state, that would have been observed had a larger cortical area been considered. It is also possible, that what looks as an OM state is a composition of several OMs of similar orientations, that cannot be distinguished with current techniques. We can thus expect that if a large cortical region will be imaged during spontaneous activity, spontaneous states will correspond more closely to mosaic states, rather than to OM states. From a theoretical point of view, the prevalence and characteristics of mosaic states in spontaneous activity is an interesting question because mosaic states provide a signature of the effective strength and extent of the lateral connections in V1. Spontaneous states that correspond to OM states rather than to mosaic states indicate a strong lateral connectivity with large spatial extent. A further analysis, which is beyond the scope of this contribution, is required to fully characterize the behavior of the spatially restricted model, and its implications on spontaneous activity.

6.6. Dynamics

Our approach was based on identifying the attractor states of the intracortical network with the OMs. Because we used a static input and simple rate dynamics, attractor states always existed; the network always converged to a stationary activity profile. In contrast, spontaneous activity in anaesthetized cats is highly dynamic, continuously switching between states. This gap between the dynamics of spontaneous activity in cats and the dynamics of our model can be bridged by considering various mechanisms that can prevent the asymptotic convergence of the model to a fixed point. The dynamical switching between states can be induced by a dynamical external input. Specifically, Goldberg et al. (2004) suggested that spatio-temporally correlated noise coming from the LGN may drive the cortical dynamics. In the marginal phase, where the shape of the activity is dominated by recurrent connections, this type of afferent noise would cause the network to perform a random walk in the space of OMs. An analysis of experimental data reveals that signatures of its dynamics are different from those of a simple random walk along the ring attractor. Another possibility, which might be more compatible with experimental data, is that the system is in the linear phase. The observed cortical states in this case are the sum of the afferent input and a component in the shape of the OMs, induced by the recurrent connections (Goldberg et al., 2004).

The dynamical switching between states can be also induced by mechanisms intrinsic to V1. For example, firing rate adaptation or synaptic depression/facilitation can destabilize the attractor states on a slow time-scale. Including a term for firing rate adaptation in the dynamical equation (Eq. (4)), as in Hansel and Sompolinsky (1998), causes the network, in the marginal phase, to perform a limit-cycle in the space of OMs. This possibility seems too simplistic to account for the spontaneous cortical dynamics. However, a more complicated dynamical behavior can emerge if synaptic depression/facilitation is included. Such complicated behavior indeed emerges in the ring model with synaptic dynamics (Tsodyks, unpublished results), and we expect similar results for our model. It is also likely that spontaneous activity in V1 is driven by a combination of several of the mechanisms we mentioned above. Our model provides the framework in which the contribution of each of these mechanisms could be studied.

6.7. The stimulus

Throughout the paper, we assumed that the evoked OMs encode the orientation of the stimulus. This might not be the case. Basole et al. (2003) found that states highly similar to OMs, can also be evoked by texture stimuli containing short line segments. The OM to which the resulting state corresponded depended on both the orientation of the line segments and the direction in which they moved. The authors concluded that OM-like states encode the spatio-temporal properties of the stimulus, rather than its (spatial) orientation. The issue of which stimulus property is encoded by the OMs is largely avoided in our model because we did not consider in details how a particular visual stimulus is transformed into a pattern of LGN input to V1. For the types of input we considered, the LGN-V1 mapping is assumed to extract some information about features of the visual scene (e.g. orientation) and the intrinsic V1 processing acts to amplify and denoise the afferent signal. Input patterns of higher complexity may lead to a more intricate transformation by the recurrent V1 network.

In a broader context, our model can be considered as a general model for population encoding of a sensory or motor variable in the presence of a weak or noisy input. This variable can be some spatio-temporal characteristic of a sensory stimulus, but also direction of a planned movement, or direction of gaze. The variable encoded must however be periodic, otherwise the concept of a ring attractor is not relevant. In fact, the generality of the model is inherited from the ring model, which our model generalizes. Extensions of the ring model and other similar models supporting ring attractors have been considered in several systems such as spatial working-memory (Camperi and Wang, 1998; Compte et al., 2000), head direction system (Skaggs et al., 1995; Redish et al., 1996; Zhang, 1996; Xie et al., 2002; Boucheny et al., 2005), hippocampal place cells (Tsodyks and Sejnowski, 1995), general statistical inference (Pouget et al., 1998; Deneve et al., 1999) and TMS-induced perceptual suppression (Miyawaki and Okada, 2004). Some of these models build directly on the formulation of Ben-Yishai et al. (1995); others use different formulations but are similar in many aspects (Ermentrout, 1998). Our model adds to this class of models the possibility of variability in response amplitude, i.e. selectivity, of different neurons. As a result, the ring attractor is not associated with a simple spatial form of the activity pattern, like a single localized bump in the ring model, but rather includes the complex spatial structure of the OMs.

Regardless of the precise nature of the features represented by the OMs, the spatially restricted version of the model suggests how local features of the stimulus can be encoded when the stimulus properties, e.g. orientation, vary across the visual field. Since the coarse grain mapping of the visual field to the cortex is retinotopic, we can expect the activity patterns at a small cortical regions to represent the properties of the stimulus in the corresponding regions of the visual field. The states generated by the model, e.g. the mosaic states in Fig. 10, when evoked, achieve precisely this. We thus predict that states similar to the mosaic states will play important role in encoding complex stimuli whose features vary across the visual field.

1

Footnotes
1

For the theoretical analysis we let orientations take values over the interval [0, 2π) rather than the natural encoding over the interval [0, π). This encoding induces a 2π periodicity that simplifies the equations we present. Thus, φj represents a stimulus whose actual orientation was φj/2.

 

Acknowledgments

We thank A. Grinvald, T. Kenet, and A. Arieli for kindly providing us with the experimental data. Special thanks to D. Hansel for critical reading of the manuscript and inspiring suggestions. We thank H. Sompolinsky for fruitful discussions, and S. Preminger, A. Loebel, O. Shriki, M. Katkov, O. Barak, and M. Szwed for their comments on the manuscript. Supported by grants from the Israeli Science Foundation and Irving B. Harris Foundation.

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • Barak Blumenfeld
    • 1
  • Dmitri Bibitchkov
    • 1
  • Misha Tsodyks
    • 1
  1. 1.Department of NeurobiologyWeizmann Institute of ScienceRehovotIsrael

Personalised recommendations