Neural population representation hypothesis of visual flow and its illusory after effect in the brain: psychophysics, neurophysiology and computational approaches

Abstract

The neural representation of motion aftereffects induced by various visual flows (translational, rotational, motion-in-depth, and translational transparent flows) was studied under the hypothesis that the imbalances in discharge activities would occur in favor in the direction opposite to the adapting stimulation in the monkey MST cells (cells in the medial superior temporal area) which can discriminate the mode (i.e., translational, rotational, or motion-in-depth) of the given flow. In single-unit recording experiments conducted on anaesthetized monkeys, we found that the rate of spontaneous discharge and the sensitivity to a test stimulus moving in the preferred direction decreased after receiving an adapting stimulation moving in the preferred direction, whereas they increased after receiving an adapting stimulation moving in the null direction. To consistently explain the bidirectional perception of a transparent visual flow and its unidirectional motion aftereffect by the same hypothesis, we need to assume the existence of two subtypes of MST D cells which show directionally selective responses to a translational flow: component cells and integration cells. Our physiological investigation revealed that the MST D cells could be divided into two types: one responded to a transparent flow by two peaks at the instances when the direction of one of the component flow matched the preferred direction of the cell, and the other responded by a single peak at the instance when the direction of the integrated motion matched the preferred direction. In psychophysical experiments on human subjects, we found evidence for the existence of component and integration representations in the human brain. To explain the different motion perceptions, i.e., two transparent flows during presentation of the flows and a single flow in the opposite direction to the integrated flows after stopping the flow stimuli, we suggest that the pattern-discrimination system can select the motion representation that is consistent with the perception of the pattern from two motion representations. We discuss the computational aspects related to the integration of component motion fields.

Introduction

Aftereffects experienced after a long exposure to moving visual stimuli are called movement aftereffects (MAEs) (see reviews by Anstis et al. 1998; Taylor et al. 2000; Huk et al. 2001; Clifford et al. 2007; Mather et al. 2008). To explore the cause of MAEs, adaptation-induced changes in neural responses to moving stimuli have been investigated in the monkey MT (medial temporal) area, which is known as one of the visual-movement analyzing brain sites (Zeki 1974; Van Essen et al. 1981; Tanaka et al. 1986; Komatsu and Wurtz 1988), and several reports have indicated that the activities were reduced after the receptive field of the MT neurons was stimulated with prolonged moving stimuli (Petersen et al. 1985; van Wezel and Britten 2002; Kohn and Movshon 2003). For example, Petersen et al. (1985) reported that MT neurons changed their responsiveness after receiving prolonged moving stimuli such that the sensitivity decreased after the neurons received a stimulus moving in the neuron’s preferred direction and it increased after they received a stimulus moving in the opposite direction. They thought that the sensitivity change could be a cause of the MAE.

In this report, we investigate the cause of MAEs elicited by three modes of wide-field visual flows, i.e., translational, rotational, motion-in-depth flows. Special focus is put on transparent field motion (in which two sets of translational flows of different directions are superimposed) and its unexpected aftereffect. We chose to study the mechanism of flow perception including MAEs for four reasons: (1) The perception of a wide-field visual flow is important for controlling locomotion and posture in humans and monkeys, equally. Thus an investigation into the changes in the responses of neurons in the monkey brain would contribute to an understanding of MAEs in humans. (2) The size of the receptive field of MT neurons is too small to discriminate the mode of visual flows, and thus the mode-specific flow-caused-MAEs cannot be explained by the activity-change of the MT neurons. (3) Physiological studies on monkeys identified the medial superior temporal (MST) area as a brain site in which the visual flow is specifically analyzed (Tanaka et al. 1986; Saito et al. 1986; Komatsu and Wurtz 1988; Duffy and Wurtz 1991), and brain-imaging studies (Tootell et al. 1995; Snowden and Milne 1997) have found corresponding areas in the human brain. (4) There is as yet no convincing interpretation of the neural mechanisms underlying the perception of transparent motion and its MAE.

We hypothesized that the change in the responsiveness of the population of MST neurons is the cause of the visual flow-induced MAEs. We measured the response properties of neurons in the MST of macaque monkeys to the three visual flow modes, i.e. translational, rotational, and flow in-depth. We previously found three classes of neurons in the MST, i.e., D cells responding to a translational flow in a directionally selective manner, R cells responding selectively to a clockwise or a counterclockwise rotary flow, and E/C cells responding selectively to an approaching (E cells) or a receding flow (C cells) (Saito et al. 1986). We also tested the responses of MST neurons to transparent-translational flows. In interpreting the transparent motion perception and its interesting aftereffect consistently, we assumed the presence of two sets of cells, component cells and integration cells: the latter integrates two flows of different directions contained in the transparent motion stimuli. As assumed, we indeed found the two groups in the MST. Moreover, we conducted psychophysical experiments on human subjects to prove, even though indirectly, the existence of two subgroups of flow-responding neurons in the human brain. We propose a computational model to explain the neural mechanism of integrating the two flows.

Our hypothesis is as follows: when we see a right-ward translational field motion, for example, the D cells (preferred cells) which are selective to the right-ward motion discharge maximally, while the D cells (null cells) which are selective to the left-ward field motion show no response or somewhat suppressed activities, and the D cells which have the preferred direction in between the right-ward and the left-ward motions show activities in between those of the preferred cells and the null cells. This is shown schematically in Fig. 1a. Given such a population representation, we get a perception of the right-ward translational field motion because the population vector is directed right-ward. Over a course of a presentation of a prolonged visual flow, activeness of strongly activated cells will reduced as a cause of gain control (see Clifford et al. 2007; Mather et al. 2008). The magnitude of the reduction will be inverse proportional to the activated level of the cells. Thus, when the right-warded flow stops, the relative strength of the cells’ activity (as observed in the spontaneous activity) may change to give the population pattern shown in Fig. 1b. Although the discharge levels are low, the population vector matches the activity profile caused by the left-warded translational flow stimulus, and thus, we get a perception (MAE) of the left-ward visual flow notwithstanding the fact that the visual pattern is stationary.

Fig. 1

Schematic illustration of population representation in the plain of MST D cells. a Relative firing levels of MST D cells during the course of the stimulation by an adapting visual flow moving rightward. The arrow within each circle indicates a preferred direction of the cell. b Rerative firing levels of the same cells after stopping the adapting flow stimulus (spontaneous discharge levels in the presence of a stationary random-dot pattern)

We also tried to interpret the perception of a transparent visual flow as well as of its MAE using the same hypothesis. When we are presented with a random dot pattern consisting of two sets of dots moving coherently in different directions, we perceive two component movements as if there were two transparent fields of dots moving in their respective directions (Qian and Andersen 1994; Qian et al. 1994; Snowden and Verstraten 1999) (Fig. 2). This phenomenon, where one perceives two separate directions of motion, is known as bidirectional transparent motion. The aftereffect of a bidirectional transparent motion stimulus is not bidirectional but a unidirectional motion in the direction opposite to the vector average (integration) of the two component directions of motions in the adapting stimulus (Verstraten et al. 1994; Vidnyanszky et al. 2002; Alais et al. 2005). This interesting phenomenon provides a good experimental model to test whether the same hypothesis can interpret transparent motion perception and its MAE consistently. To answer the question, we propose that two groups of cells exist in the MST, one representing the component directions of field movements contained in the transparent flow, the other representing the integrated motion direction of the superimposed flow stimuli.

Fig. 2

Pictorial illustration of a bidirectional transparent visual flow. One coherent flow moving rightward (dark arrows) is superposed on another coherent flow moving toward the 10 o’clock direction (light arrows) so as to produce a bidirectional transparent flow

To test this idea, we studied the tuning curve of MST D cells, which showed direction-selective responses to translational flow stimuli, for bidirectional transparent motion stimuli with various combinations of directions. As expected, we found two subtypes of D cells. One is characterized by a bimodal tuning curve, such that the cell responds strongly when one of the two directions of the motion stimulus coincides with its preferred direction. Since each stimulus has two directions of motion, cells of this subtype are excited by two positions as the direction of the bidirectional stimulus changes systematically. We call a cell of this subtype a component D cell since it responds to either of the component directions of the movement. The other subtype has a unimodal tuning curve, responding most strongly when the combined (integrated) motion direction of the transparent motion stimulus matches its preferred direction. We call a cell of this subtype an integration D cell because it can represent the integrated direction of component movements.

Because one target of the present study is the aftereffects induced by various modes of visual flows, and because we conjecture that the aftereffect itself will be represented as a population code made of an activity profile among MST cells, we have measured the changes in spontaneous activities of MST cells (D cells, R cells, and E/C cells) after presentation of an adapting flow stimulus. We found that almost all MST cells showed clear changes in activity (as measured by the spontaneous activities or by the magnitude of the response to the preferred stimulus) when the adapting stimulus was removed: the activity of the cell decreased when the adapting flow stimulus was the cell’s preferred direction, and it increased when the adapting flow stimulus was the cell’s null direction. The change in sensitivity was specific to the mode of the adapting visual flow: a translational flow suppressed the D cells’ activities, a rotational flow suppressed the R cells’ activities, and a motion-in depth flow suppressed the E/C cells’ activities when the flow was in the cell’s preferred direction. This is strong supporting evidence for the population-representation hypothesis that the illusory motion aftereffect caused by the flow stimulus is generated by imbalanced spontaneous activities between flow-mode selective MST cells having mutually opposite preferred directions. The same phenomenon can be observed after presentation of a bidirectional motion pattern, suggesting that the integrated illusory aftereffect is caused by the adapted spontaneous activities of the integration D cells of the MST.

We also conducted psychophysical experiments on how human beings perceive movement when they are exposed to a bidirectional wide-field transparent flow stimulus and how such a perception depends on the duration of exposure. We found that when the exposure is very short, people perceive a unidirectional motion in the direction that is the vector sum of the component motions, but that when the exposure is longer, they perceive bidirectional motions. This provides evidence that both the component and the integrated motion representations exist in the human brain.

We finally propose a possible computational mechanism to produce two different subtypes of MST D cells, component cells and integration cells, by receiving local motion signals from the MT. It will be shown by using the neural field theory that additive integration of input signals produces component D cells whereas multiplicative one produces integration D cells.

Methods

Monkey preparation and experimental procedure

Experiments were performed on two Japanese monkeys (Macaca fuscata) weighing between 7 and 8.5 kg. They were anaesthetized and ventilated artificially. The animals were treated in accordance with regulations promulgated by the guidelines of the National Institutes of Health. Details are given in the “Appendix”. Extracellular single-unit recordings of activities of the MST cells were made by using glass-coated platinum-iridium electrodes. In penetrating the electrode tangentially in the extrastriate cortex of the monkeys, we encountered a group of cells that have a small receptive field and who respond selectively to the direction of a translational motion stimulus presented in the receptive field. We thus identified the region as the MT area. After passing 1–2 mm of a silent region (cell-free region of the superior temporal sulcus), we again encountered directionally selective cells having a very large receptive field. Thus, we could easily identify the MST area.

The recorded MST cells were classified on the basis of flow-specific response properties into four types: D cells (responding to translational flow in a directionally selective manner), Rf cells (responding to clockwise or to counterclockwise rotation in the front-parallel plane), and E or C cells (responding to either expansion or contraction of the flow pattern, i.e., flow in-depth) (Saito et al. 1986) using a hand-held cardboard (textured board) on which random-dots were printed. Rd cells (Saito et al. 1986) were not identified because we did not present a rotation-in-depth flow in the present study. Hereafter, we will call Rf cells R cells.

Measuring the effect of adaptation to flow stimuli on MST cells

The receptive field of the recorded cell was mapped on the screen using a textured board and a screen of a computer-controlled display was set so as to fit the center of the receptive field. All the stimuli (moving or stationary random dot patterns) used in the present experiments were generated by a computer workstation (IRIS Crimson, Silicon Graphics Co.). The stimulus, refreshed at a rate of 60 Hz, was displayed on a CRT screen set 50 cm from the eyes of the monkey. The display covered a visual area 34° high and 44° wide. The mode, direction, and speed of the test flow stimulus were set so as to activate the cell maximally. If the receptive field size exceeded the size of the display’s screen, the screen was positioned at the center of the receptive field. If the receptive field size was smaller than the screen size, a circular mask was set on the screen in order to confine the stimulus within the receptive field. Stimuli were given monocularly to the eye contralateral to the recording site.

The stimulus sequence used for analyzing neural correlates of MAEs was as follows. After giving an adaptation field consisting of 640 square dots (each 1.0° × 1.0°) distributed at random in the stimulus field for 30 s, a stationary pattern was presented for 1.0 s. As will be mentioned in relation to our psychophysical experiments, the sensation of illusory motion aftereffects was strongest during the first second after the adapting stimulus had stopped. The effects of the adapting stimulus were therefore evaluated by measuring the spontaneous firing rate during the 1.0–s period immediately after the cessation of the adapting stimuli. We tested whether not only spontaneous discharges but also the cell’s sensitivity to the preferred stimulus changed as a result of the adapting stimuli. In those experiments, a stationary pattern was given for 3 s and a test stimulus moving in the preferred direction was given for 1 s at the middle of the 3 s duration (see Fig. 3). Three types of adaptation fields, (1) a stationary field (SF), (2) motion in the best direction (MBD) field, and (3) motion in the opposite direction (MOD) field were presented sequentially, and the total set of sequential stimuli was repeated eight times to obtain an averaged peristimulus time-histogram (PSTH) for further data analysis.

Fig. 3

Representative responses of MST D-, R-, and E/C cells to the test stimulus (preferred stimulus) after adapting to a stationary field (SF), or a motion in the best direction (MBD) field, or a motion in the opposite direction (MOD) field. Each adapting field was given for 30 s. The bars under the response-histogram indicates the duration of the test stimulus

Psychophysical experiments on human subjects

The MAEs induced by a single visual flow are now so familiar that anyone can find numerous websites that will enable a personal computer screen to display a demonstration pattern. Hence we studied only the perceptions of the transparent motion and its MAE in this paper. As in the case of the physiological experiments, the stimuli (moving or stationary random dot patterns) were generated by a computer workstation (IRIS Crimson, Silicon Graphics Co.). The adaptation field consisted of 640 square dots (each 0.7° × 0.7°) distributed randomly in the stimulus field. The stimulus, refreshed at a rate of 60 Hz, was displayed on the CRT screen 50 cm from the eyes of the subjects. The screen covered visual angles of 50° high and 65° wide. University students in their twenties were requested to fixate binocularly on the center of the display during the presentation of stimuli and to report the direction and speed of the motion aftereffect orally. A bidirectional transparent visual flow was presented for 30–60 s and was followed immediately by a stationary test pattern. They were requested to indicate the perceived direction of the MAE by selecting one of 24 candidates set at 15° steps between 0° and 360°. Data were compiled in terms of the angular deviation from the direction opposite to the integrated motion direction. Perceived directions were grouped in 30° bins. The perceived directions of 15° and 30°, for example, were compiled as 30°. We also studied the effect of varying the stimulus duration on the transparent motion perception.

Results of physiological experiments

Movement aftereffects on the D-, R-, and E/C cells in the MST

We tested our hypothesis for the cells in the MST area by using single unit recordings. The middle and right columns of Fig. 3 respectively show typical examples of the responses of D, R, and E/C cells to the test stimulus (motion in the best direction, duration: 1 s) given 1 s after stopping the prolonged (30 s) adapting stimulus moving in the best direction (MBD) or in the opposite direction (MOD). As a control, the responses to the same test stimulus given after the presentation of a stationary field (SF) for 30 s are shown in the left column of Fig. 3. Compared with the control test responses, the responses to the test stimulus are significantly reduced after the MBD adaptation, whereas they are a little increased after the MOD adaptation. Figure 4 illustrates the magnitude of the adapting effect for 11 D cells, 8 R cells, 3 E/C cells. The MBD adaptation has a strong suppressive effect, whereas the MOD adaptation has facilitating effect though it is not strong.

Fig. 4

Changes in the responsiveness of MST D-, R-, and E/C cells to the test stimulus (preferred stimulus) after adapting to a motion in the best direction (MBD) field or a motion in the opposite direction (MOD) field. The changes were normalized by the control responses to the test stimulus obtained after adapting to a stationary field (SF). Data were pooled for 11 D cells, 8 R cells, and 3 E/C cells

In the cells that showed relatively high rates of spontaneous activities, the change in the spontaneous discharge rate caused by the adapting stimuli (MBD and MOD) was measured in addition to the change in the sensitivity to the preferred test stimulus. As expected, these changes showed the same tendency. The change in the spontaneous discharge rate was evaluated for 14 D cells, 6 R cells, and 5 E/C cells (Fig. 5). All of the tested cells saw a consistent decrease in the spontaneous discharge rate after the MBD adaptation and an increase in the spontaneous discharge rate after the MOD adaptation relative to the control level (i.e., the spontaneous discharge rate seen after a 30 s presentation of the stationary stimulus). The spontaneous discharge rate after the MBD adaptation was 3.76 ± 4.01 spikes/s (n = 25), whereas the rate after the MOD adaptation was 6.26 ± 8.16 spikes/s (n = 25). The spontaneous discharge rate after the MOD adaptation was much greater than the rate after the MBD adaptation. This indicates that after the adaptation, the level of spontaneous activity differs between preferred cells and null cells. A statistical test (paired t test) showed that this difference was significant at the 5% level. A regression analysis showed that the slope of the regression line was 0.57 for MBD adaptation and 1.56 for MOD adaptation. A software for statistically testing the difference between two regression lines (Hasegawa 1997) showed that the difference between the regression lines for MBD and MOD adaptation was also significant at the 5% level.

Fig. 5

Comparison of the spontaneous discharge rates of D-, R-, and E/C cells in MST during the first second after the end of the adapting stimulus. Abscissa: spontaneous discharge rate after presenting a SF (stationary field). Ordinate: spontaneous discharge rate after presenting MBD (motion in the best direction) (filled circle) or MOD (motion in the opposite direction) (open circle). The slope of the regression line for SF versus MBD stimulation was 0.57 and the slope of the regression line for SF versus MOD stimulation was 1.56. The statistical significance of the difference between the regression lines was tested after the statistical test for each regression line was carried out. Data were compiled from 14 D cells, 6 R cells, and 5 E/C cells

Motion aftereffect induced by bidirectional transparent motion: a scenario

To account for the perception of the bidirectional transparent motion itself and its illusory aftereffect by a single neural mechanism, we need to postulate the existence of two subtypes of D cells in the MST: component cells that respond maximally when one of the directions of the bidirectional motion matches the preferred direction of the cells, and integration cells that respond maximally when the direction of the integrated motion (the vector sum of the two component motion directions) matches the preferred direction of the cells. Given two such subtypes of D cells in the MST, the activity profile of MST cells during a bidirectional motion stimulation would be the one shown in the middle row of Fig. 6. As shown in the left half of that figure, the activity profile of the component cells would be bimodal when a fixed bidirectional motion pattern is given. In contrast, the activity profile of the integration cells would have a single peak as shown in the right half. The profile of the spontaneous firing levels of the component cells would be bimodal when the bidirectional transparent motion stops (bottom of the left half of Fig. 6), while at the same time, the profile of the spontaneous firing levels of the integration cells would show a single peak (bottom right half of Fig. 6). The perception of a bidirectional flow itself and its motion aftereffect can be explained as follows. During the presentation of a bidirectional transparent flow, we perceive bidirectional motions of two independently moving fields of dots. The bimodal activity distribution in the neural field of the component cells of the MST might be the source of this perception. Alternatively, after termination of the adapting flow stimulus, the unimodal spontaneous activity distribution in the neural field of the integration cells of the MST might play a major role in our perception of the unidirectional illusory motion. To find a neural substrate for this hypothesis, we analyzed the direction tuning of D cells to bidirectional transparent flows.

Fig. 6

Predicted profiles of the firing levels of MST D cells before, during, and after presentation of bidirectional transparent motion. Spontaneous discharge levels of the component cells and integration cells are well-balanced before exposition to the adapting stimulus (upper row). Component cells (left) show maximal responses when the preferred direction of the cell coincides with either of the two sets of the coherent flow in the transparent motion. Thus, two peaks appear in the profile of the firing level of the D cells (middle row of the left side). On the other hand, integration cells show the best responses when the preferred direction of the cell coincides with the direction of the integrated motion of the two sets of coherent visual flows in the bidirectional transparent flow, and this results in a single peak appearing in the response profile (middle row of the right side). When the bidirectional transparent motion stops, the spontaneous firing profile of the component cells exhibits two peaks (bottom row of left side). By contrast, the spontaneous firing profile of the integration cells has a single peak (bottom row of right side). The single arrow in each circle indicates the preferred direction of each cell. Double arrows shown in the middle of the left-half and right-half figures denote the motion directions of two sets of coherent visual flows in the bidirectional transparent motion

Tuning properties of two D cell subtypes to bidirectional transparent visual flows

We chose a fixed bidirectional transparent motion pattern in which the directions of constituent motion flows differed by 120° (Fig. 2). This test stimulus was rotated in 30° steps (maintaining 120° of angular difference), as shown in the inset of the response histograms given in Figs. 7 and 8. Each test sequence, consisting of 12 directions of motion, was repeated eight times to compile averaged peri-stimulus time histograms (PSTHs) for an analysis of the firing rate during the presentation of the test stimuli. We first calculated the raw mean firing rate during 1 s of the responses to the test stimulus and then subtracted from it the mean spontaneous firing rate for 1 s just before onset of the test stimulus. We called this value the response magnitude and used it to construct the direction tuning curves.

Fig. 7

Responses from a representative component cell in MST to bidirectional transparent motion. Twelve histograms (numbers 1–12) in panel a show the component cell’s responses to a bidirectional transparent motion. The stimulus period is shown by the bar under each histogram in panel a. Inset arrows in each histogram show the motion directions of two sets of coherent flows contained in the bidirectional transparent motion. Direction tuning of the cell for unidirectional visual flow is shown as a radar graph in b. Each response is normalized by the maximal response, and the arrows attached to each corner in the radar graph b show the direction of the unidirectional flow. The tuning property of the same cell for the bidirectional transparent motion is shown as a radar graph c in which each response magnitude is normalized by the maximal response. Arrows at each corner in the radar graph c show the directions of two sets of coherent flows in the transparent motion. The same profile of the responses elicited by the bidirectional flow is shown in panel d as a histogram normalized by the maximal response. Note that the cell responded best when the direction of either flow in the bidirectional transparent motion coincided with the best direction of the cell

Fig. 8

Responses from a representative integration cell in MST to bidirectional transparent motion. Descriptions are similar in those in Fig. 7. Note that the cell responded best when the integrated direction of the two sets of coherent flows in the bidirectional transparent motion coincided with the best direction of the cell

Two types of response profile were found, one having two peaks, the other having a single peak. Thus, we found two types of D cells in the MST: component cells and integration cells. Representative data from the component cells and integration cells are shown in Figs. 7 and 8. Before testing the tuning to the transparent flow stimulus, the cell’s preferred direction was determined by taking a direction tuning curve using a single flow rotated in 30° steps, as shown in the panel (b) of Figs. 7 and 8. After determining the preferred direction of the cell to single flow, we tested the tuning properties of cells to the transparent flow stimulus. As shown in the panels (c) and (d) of Fig. 7, the component cells manifested bimodal response profiles in which a peak in the response occurred at two positions when the preferred direction of the cell (11 o’clock, see the panel (b) of Fig. 7) coincided with either of the two constituent coherent flow directions. The integration cells, in contrast, showed unimodal response profiles in which the response reached its maximum when the preferred direction of the cell (6 o’clock as seen in the panel (b) of Fig. 8) coincided with the integrated direction of flows (panels (c) and (d) of Fig. 8). Of the 34 D cells analyzed, ten (29.4%) were component cells and 18 cells (52.9%) were integration cells. The other six cells (17.7%) did not respond clearly to a bidirectional transparent flow, presumably because their preferred responses were suppressed by the superposition of motion in a different direction.

Results of psychophysical experiments on human subjects

Confirmation of motion aftereffect induced by prolonged presentation of transparent motion

When a transparent motion flow is presented, an activity pattern with two peaks arises in the neural field of component D cells in the MST and we perceive motion in two directions. According to our hypothesis, the perceived motion aftereffect could also be bidirectional if the MST had only the component cell representation. Previous psychophysical studies, however, have found that the aftereffect induced by a bidirectional transparent motion is unidirectional (Mather 1980; Verstraten et al. 1994; van der Smagt et al. 1999). We confirmed these results in psychophysical experiments in which the angular difference between the two constituent motion directions was either 60°, 90°, 120°, or 150°. The speed of the adapting transparent motion was set at 5.0°/s or at 10.0°/s. The perceptual properties (direction and speed etc.) of the perceived motion aftereffect were reported orally. The results obtained from fifteen male subjects (ages 20–25) are summarized in Fig. 9. All subjects consistently reported that motion aftereffect appeared immediately after the adapting stimuli had stopped and that the sensation of the illusory motion was strongest within 1–2 s, decreased gradually, and disappeared within about 5 s. As shown in Fig. 9, the direction of the perceived illusory motion was opposite to the integrated direction of the adapting stimulus. The perceived illusory motion was sluggish irrespective of the speed of the adapting stimulus. No motion aftereffect was perceived when adapting stimuli in which the two constituent random dot patterns moved in opposite directions were presented.

Fig. 9

Motion aftereffect caused by bidirectional transparent motion. Human subjects were requested to indicate the perceived direction by choosing one of 24 candidates set in 15° steps between 0° and 360°. Data were compiled as the angular deviation from the moving direction, which was opposite to the integrated motion direction of the two sets of coherent visual flows in the bidirectional transparent flow. The perceived direction was compiled in every 30° bins. For example, the perceived directions of 15° and 30° were each compiled as 30°. Thus, the perceived direction of the illusory motion (motion aftereffect) is indicated by the angular deviation (−30°, 0°, +30°) from the motion direction which is opposite to the integrated motion of the two sets of coherent visual flows in the bidirectional transparent flow. The upper graph (a) and lower graph (b) were obtained from transparent motions with speeds of 5.0°/s and of 10.0°/s. In each graph, the abscissa indicates the angular difference between constituent coherent flows in the bidirectional transparent motion stimulus. The data were from 15 adult male subjects

Perception of briefly presented bidirectional transparent visual flows

We investigated the roles of the component and integration D cells in generating motion percepts by examining the perceived properties of bidirectional transparent visual flows presented briefly. The subjects were requested to indicate the perceived motion directions for bidirectional wide-field dot patterns moving in two orthogonal directions. The speed of each test stimulus was 5.0°/s. Figure 10 showed the results obtained when test patterns were displayed for 16.7 × n ms (n = 2, 3, 4, 5, 6, 7). When the stimulus duration was less than 50 ms, no subject perceived movement. When a test pattern was presented for 50 ms, subjects perceived unidirectional movement in 24 out of 50 trials (48%). The perceived direction was an integrated one between the two orthogonal directions of the bidirectional stimulus. Movement was not perceived in the other 26 trials (52%). When the stimulus duration was 66.7 ms, the subjects perceived unidirectional movement in 28 trials (56%), two independently moving patterns (component motion perception) in 13 trials (26%), and no movement in the other 9 trials (18%). When the stimulus duration was 83.4 ms, unidirectional movement was perceived in only 13 trials (26%) and bidirectional movement was perceived in 37 trials (74%). When the stimulus duration was more than 100 ms, subjects perceived bidirectional movement in all trials. These results clearly show the existence of dual representations of the translational flow, i.e., component and integration representations, in the human brain.

Fig. 10

Perceptual properties of human subjects for a brief presentation of a bidirectional transparent visual flow in which two coherently moving patterns move in orthogonal directions. The test pattern used in this experiment was similar to the one used for analyzing motion aftereffects induced by a bidirectional transparent flow (see Psychophysical experiments). The angular difference between the motion directions of the constituent coherent flows in the bidirectional test stimulus was set to be 90°. The subjects were requested to indicate the motion directions of the bidirectional wide-field visual motion. The speed of each test stimulus was 5.0°/s. The test pattern was displayed for 16.7 × n ms (n = 2, 3, 4, 5, 6, 7). Each histogram for a given duration of the bidirectional test stimulus (16.7 × n ms (n = 2, 3, 4, 5, 6, 7)) was compiled from data gathered from 50 trials performed by five adult male subjects

Computational model for integration of component motion directions

We have shown evidence for two representations of motion directions in the population field of MST D cells, one bimodal the other unimodal. Here, we present a computational model for generating such dual representations.

Let us assume that the motion direction is represented by an excitation pattern in a neural field (Amari 1976; Coombes 2005) where neurons are arranged in one dimension such that each position represents the direction of motion. Since the direction is cyclic from 0 to 2π, the underlying field is circular. We assume that, in the earlier area of local representation (MT), the two component directions will be calculated in separate neural fields. Let us represent them by two patterns p ₁ (z) and p ₂ (z), where z denotes the location of neurons in a neural field, and p _i (z) is the firing activity of the field at position z. Here, z corresponds to the motion direction because it is reported that direction-selective cells in the MT region are arranged in motion-direction columns (Albright et al. 1984). They are unimodal activity patterns. How are these two component distributions combined?

When the local component distributions are Gaussian, we have

$$ p_{1} (z) = \exp \left\{ { - \frac{1}{2}\left( {z - z_{1} } \right)^{2} } \right\}, $$

(1)

$$ p_{2} (z) = \exp \left\{ { - \frac{1}{2}\left( {z - z_{2} } \right)^{2} } \right\}. $$

(2)

where z ₁ and z ₂ correspond to the peak positions of the component directions. The two activity patterns are integrated to give a new activity pattern p(z) in the next neural field. The simplest combination of the two components is the arithmetic sum of p ₁ (z) and p ₂ (z), giving a new representation of the form

$$ p(z) = \exp \left\{ { - \frac{1}{2}\left( {z - z_{1} } \right)^{2} } \right\} + \exp \left\{ { - \frac{1}{2}\left( {z - z_{2} } \right)^{2} } \right\}. $$

(3)

When two peaks z ₁ and z ₂ are close, the distribution is unimodal and only one direction is perceived. When the two peaks are sufficiently separated, however, the distribution is bimodal and we perceive two directions of motion corresponding to the two peaks. This is the case when we perceive bidirectional transparent motion.

We next assume that the field composed of integration D cells has a different way of combination, one in which two local fields p ₁ (z) and p ₂ (z) are integrated by multiplication:

$$ p(z) = \exp \left\{ { - \frac{1}{2}\left( {z - z_{1} } \right)^{2} - \frac{1}{2}\left( {z - z_{2} } \right)^{2} } \right\}. $$

(4)

The integrated field is always unimodal with the peak position at the midway between the locations of the two component peaks. This corresponds to the direction that is the vector sum of the two component directions. We conjecture that this is the computational mechanism of the integration D cells.

In order to unify the above two operations, we use the α-representation, which is closely related to the Weber-Fecher law and Stevens’ law (Amari 2007). The α-representation assumes that a physical sensation s can be represented by its power s^q, q = (1 − α)², or log s.

We first remark that the logarithm of activation at position z,

$$ l(z) = \log p(z), $$

(5)

and the simple addition

$$ \tilde{l}(z) = l_{1} (z) + l_{2} (z) $$

(6)

give the logarithm of multiplicative integration. Since the Weber-Fechner law describes a logarithmic relation between the physical magnitude of a stimulus and the intensity of the sensation elicited by that stimulus, this could be the mechanism of integration. Stevens’ law, however, suggests the use of the α-representation for a real α such that the sensation is

$$ f_{\alpha } (s) = \frac{2}{1 - \alpha }s^{{\frac{1 - \alpha }{2}}} . $$

(7)

When α = − 1, f _α(s) = s, and α = 1, we take the limit α → 1, giving

$$ f_{1} (s) = \log s. $$

(8)

The α-representation of p(z) is

$$ p^{\alpha } (z) = f_{\alpha } [p(z)]. $$

(9)

The α-integration of two local activities p ₁(z) and p ₂(z) uses the α-representation to give

$$ \tilde{p}^{\alpha } (z) = p_{1}^{\alpha } (z) + p_{2}^{\alpha } (z). $$

(10)

This is a method of integration of stochastic evidence (Amari 2007), which has a theoretical basis in information geometry (Amari and Nagaoka 2000). When α is close to 1, the α-representation is positive and gives a result similar to the one obtained using the logarithmic (multiplicative) integration. A model field is capable of calculating the α-integration for any α. Another theoretical basis arises from the dynamics of excitations in a recurrently connected neural field (see, e.g., Amari 1976). This also works to shape a noisy excitation pattern. It is possible to generate an integrated unimodal pattern from bimodal components.

The theory predicts that α-representation of s is used in the brain (a different α is used in different parts). This prediction should be experimentally confirmed.

Discussion

Through psychophysical studies, we confirmed that a prolonged presentation of a wide-field visual flow (translation, rotation, expansion/contraction) induces a compelling MAE in the opposite direction, and that the MAE elicited by a bidirectional transparent motion pattern is unidirectional even if the perceived motion during the stimulus presentation is bidirectional. We hypothesize that all flow-induced aftereffects are caused by an imbalance in spontaneous activities between MST cells of mutually opposite directional selectivity, under the assumption that the activities of the cells are suppressed in proportion to the stimulated levels due to the given flow. In single-unit recording experiments, we found that the expected changes in the spontaneous activities of D, R, E, and C cells occur in a direction-selective manner. To give a comprehensive explanation for both the bidirectional perception of the transparent motion and the unidirectional MAE, we postulated that there are two subtypes of D cells (component cells and integration cells). In the present study, we indeed found two subtypes of D cells as assumed, by conducting single-unit recordings of the activity of MST cells in monkeys. Preliminary short reports on this discovery have also been presented (Hida et al. 1998; Ohno et al. 2002).

The next question is, then, why a single percept (a transparent motion percept during stimulation, and a single flow towards the direction opposite to the integrated motion direction after stopping the flow stimuli) comes from two different simultaneous population representations formed by the component cell group and the integration cell group. A possible inference is as follows: Each of the two flows moving coherently in different directions is grouped into a single pattern through a motion similarity, and we perceive the transparent motion. The outcome of the similarity binding mechanism suppresses the integrated motion representation because it is not compatible with the bidirectional perception. The psychophysical results shown in Fig. 10 suggest the presence of dual motion representations, namely the component representation and the integration representation, in our visual system. When a brief (50.0–66.7 ms) bidirectional transparent flow was presented, human subjects frequently perceived a unidirectional integrated flow rather than bidirectional flows (Fig. 10). This suggests that a certain period of time would be required to bind component patterns by recruiting a ‘motion similarity.’ So within a short period of time before the ‘similarity-binding’ occurs, the integration cells’ representation dominates the component cells’ representation, but after the occurrence of the similarity binding the integration representation would be suppressed because of an inconsistency with the pattern information. In contrast, the component representation is suppressed after transparent flow stimuli stops because distributed dots make a stationary pattern and cannot move in different directions. This results in unidirectional MAE.

In our psychophysical investigation, discernible MAEs did not arise when coherent component dots moved in opposite directions. After receiving oppositely directed flows, the motion representation has two peaks in the directions opposite to the given flow. Grunewald and Lankheet (1996) reported this phenomenon. They analyzed the perceptual properties of MAEs by using dynamic random patterns as a test stimulus, so their experiments could detect an imbalance elicited in motion signals. In contrast, MAEs could not be detected in our experiments, because the pattern presented after the flow-adaptation was stationary random dots containing no motion signal to receive directional bias from the imbalance.

The illusory motion perceived in MAEs seems sluggish. It has been widely reported that the human subjects perceive an equiluminous color-based visual flow to be slower than a luminance-based visual flow (Cavanagh et al. 1984; Cavanagh and Anstis 1991; Anstis 2001). Correspondingly, the response of MST cells to a color-based visual flow was smaller than their response to a luminance-based one (Ohno et al. 1996). Thus, the sluggishness of the movement perceived in the MAEs could be due to the low level of the activity of the MST cells that contribute to the MAEs.

Recent papers have postulated that the MAEs can not be attributed to a single cortical area but to multiple areas involved in visual motion processing (see the review by Mather et al. 2008). It is convincing that from the retina to the MT, the stimulated cells reduce their sensitivities due to a luminance-contrast based gain control. As shown by Kohn and Movshon (2003), cells in the MT reduced their responsiveness after exposure to drifting grating stimuli. They concluded that such a gain control would occur in the V1 since the effect depended on the stimulated position within the receptive field. This type of sensitivity control, however, is not selective to the functional differences used to perceive whether the given flow is translational, rotational, or motion-in depth. All the MST cells would equally receive the retinal position specific sensitivity control, irrespective of their directional selectivities and therefore induce no imbalance in the population representation of the MST.

Recent brain imaging studies on human subjects have revealed that areas corresponding to the MT and the MST exist in the human brain (see the reviews by Greenlee 2000; Mather et al. 2008). Furthermore, it has been suggested that the MAE receives an attentional modulation (cf., Mather et al. 2008) and that the anterior cortical areas are the sites responsible. However, flow-induced MAEs are almost passive and automatic and give a strong sensation of motion without the viewer having to pay special attention. We thus concluded that flow-induced MAEs can be explained by an imbalance in the population representation of MST cells with the selectivity to the flow’s mode and its directionality.

Here, let us discuss how the MST could serve as the center of the population representation for the flow discrimination recruiting the cells those have relatively broad tuning properties to the direction of motion (see the tuning curves in Figs. 7 and 8). In fact, many D cells responded to some extent to a rotary flow and/or motion-in-depth flow, if the center of such a flow is far from the center of the receptive field of the cell. As shown in Fig. 11, however, such a response result in many decentered D cells having different preferred directions. As a result, a plateau would emerge on the population representation plane of the D cells with no single peak at a specific direction. Only when the visual system receives a translational flow does a clear peak appear at the corresponding position in the D cell representation plane of the MST for the correct perception of the mode and direction of the incoming flow. Similarly, R cells and E/C cells responded to a translational flow weakly. However, for the same reason, an imbalance would not occur between cells having a mutually antagonistic directional selectivity, i.e., clockwise cells versus counterclockwise cells, and expansion cells versus contraction cells. Only when an incoming flow is a rotation or a motion-in-depth does a clear imbalance occur in the plane of the R cells representation or in the plane of E/C cells representation in the MST. Thus, reliable information on the mode and the directionality of the incoming flow could be detected as the population representation on the cell-plane of the MST.

Fig. 11

Schematic drawing of a situation in which D cells show a response to a rotary flow. The size and position of the receptive fields of five cells (1–5) in the visual field are illustrated by thick circles. Motion fields resulted by a rotary flow stimulation are shown as pale arrows. Because of the broadness of the directional tuning to the flow stimulation, D cells numbered “2” to “5” show responses to some extent to the given rotary flow since the center of the receptive field of the cells is far from the center of the rotary flow, and part of the rotary stimulation roughly matches the preferred direction (shown by a thick arrow in the receptive field) of these cells. Such responses will be, however, evenly distributed on the D cells of a different directional selectivity. Thus, there would be no significant peak in the plane of the population representation of the D cells. The cell numbered “1” would give no response whatever the directionality, because the center of the receptive field is close to the center of the rotation of the flow and an excitation elicited by the preferably directed moving components contained in the flow will be canceled out by an inhibition elicited by the oppositely directed moving components contained in the same flow

Finally, let us discuss how the local motion signals represented in the MT are integrated to give ‘component’ and ‘integrated’ representations in the MST. Movshon et al. (1985) reported two types of MT cells, namely the component cells and the pattern cells. Pattern cells respond to local integrated motion in the stimulus and are thought to integrate signals from component cells (Movshon et al. 1985; Rodman and Albright 1989). Kawakami and Okamoto (1996) proposed a neural network model in which neural activity consistent with pattern cells and component cells is produced through the pathways from the lateral geniculate nucleus to the MT via area V1. The model predicts that the component cells of the MT can be made from additive operations on the inputs from V1 cells having the same directionality at different retinal locations, while the pattern cells of the MT can be made by ‘and-operations’, identical to a multiplicative operations, performed on the inputs from the component cells of the MT. They also confirmed that the response properties of MT cells fit their model well (Okamoto et al. 1999). Given two types of MT cells, the simplest explanation is that the component cells and integration cells of the MST will receive inputs from different types of MT cells: namely, the component cells of the MST receive inputs from the component cells of the MT, and the integration cells of the MST receive inputs from pattern cells of the MT. But this is not the case. Our preliminary studies on the response behaviors to plaid stimuli [similar stimuli used by Movshon et al. (1985)] have shown that both the component cells and the integration cells of the MST responded, with no exception, to the stimuli with a single directional tuning just as the pattern cells of the MT did. This fact strongly suggests that the MST D cells receive inputs only from the pattern cells of the MT. The pattern ‘plaid’ is already integrated as a fixed single pattern in the MT through local integration. The local pattern information represented in the MT is thought to be globally integrated in the MST in order to discriminate single translational flows, transparent overlapping translational flows, rotational flows, flows in depth, etc. Our model considerations clearly showed that the component and integration representations could be produced by performing different computational operations, i.e., additive and multiplicative operations on the same input signals. These operations could possibly be realized in the neuronal connections of many brain areas including the MT and MST. Our prediction is shown in Fig. 12. In short, the output cells of the MT to the MST would be pattern cells that make a local integration of the motion signals (the first stage of motion integration). In the MST, the second stage of integration, a global integration, takes place to produce perceptual attributes of various visual flow modes through certain additive operations (single translational flow, rotary flow, flow in depth) for the directional signals distributed in the visual field (cf. Saito et al. 1986; Tanaka and Saito 1989; Tanaka et al. 1989). Through the second stage of integration, the integration D cells are made by multiplicative operations performed on the local directional signals, and induce an interesting unidirectional MAE after adaptation to the transparent flow stimulation.

Fig. 12

Possible connection diagram in the visual motion-processing pathway. The motion signal is processed from local- to global-information through additive and multiplicative operations to produce different types of cells necessary for perception of the visual flow modes

Appendix: Treatment of experimental monkeys

The physiological experiments on the monkeys (Macaca fuscata) as well as their care were conducted in accordance with the NIH Guide for the Care and Use of Laboratory Animals, the guidelines of Handbook for the Use of Animals in Neuroscience Research (Society for Neuroscience 1991), and the experiments were approved by the Council of Tamagawa University for Animal Experiments.

In each session of the single unit recording in the macaque MST area, the animal was paralyzed by a muscular injection of 10 mg/kg of ketamine hydrochloride and then gas anesthesia was administered by artificial ventilation with a 70:30 N₂O–O₂ gas mixture with 0.5 − 1.0% of sevoflurane. Throughout the recording session, gas anesthesia was maintained and the animal was immobilized with a muscle relaxant (pancuronium bromide: 3 μg/kg/h). The eyes were treated with a local application of 0.5% tropicamide and phenylephrine hydrochloride and focused at a CRT screen with contact lens.