Procedure for extracting temporal structure embedded within psychophysical data

Abstract

The idea that mental events unfold over time with an intrinsically paced regularity has a long history within experimental psychology, and it has gained traction from the actual measurement of brain rhythms evident in EEG signals recorded from the human brain and from direct recordings of action potentials and local field potentials within the nervous systems of nonhumans. The weak link in this idea, however, is the challenge of extracting signatures of this temporal structure from behavioral measures. Because there is nothing in the seamless stream of conscious awareness that belies rhythmic modulations in sensitivity or mental acuity, one must deploy inferential strategies for extracting evidence for the existence of temporal regularities in neural activity. We have devised a parametric procedure for analysis of temporal structure embedded in behaviorally measured data comprising durations. We confirm that this procedure, dubbed PATS, achieves comparable results to those obtained using spectral analysis, and that it outperforms conventional spectral analysis when analyzing human response time data containing just a few hundred data points per condition. PATS offers an efficient, sensitive means for bridging the gap between oscillations identified neurophysiologically and estimates of rhythmicity embedded within durations measured behaviorally.

Appendix 1 Strategies for revealing temporal structure in measures of perceptual performance

Evidence for periodicity has been observed in detection performance, as exemplified by Landau and Fries’ study of exogenous attention using time-tagged visual events (Landau & Fries, 2012). Illustrated schematically in Fig. 1C, a participant’s task in that study was to fixate a small central dot placed midway between two grating patches comprising oriented, drifting contours. At an unpredictable time following onset of this display, a brief, 33 ms contrast decrement—the test probe—could appear within a small region of one of the two gratings, and the participant judged in which grating—left or right—the decrement occurred, guessing if necessary. The magnitude of the contrast decrement was set for each individual to a value making the task difficult but not impossible. In addition to the probe, a brief, 33 ms flash comprising four white spots could occur at the border of one of the two gratings, i.e., either the grating receiving the test probe on that trial or the grating imaged in the opposite visual field—the location of the flash was irrelevant for judging in which grating the probe decrement occurred, but in virtue of its unpredictability and conspicuousness, the flash did serve to orient attention (but not fixation) to one of the two locations where there was a grating. The interval between the transient flash and the probe was varied unpredictably over trials in precise steps of 16.7 ms, ranging from values where the probe occurred prior to the attention-grabbing flash to values where the probe’s appearance occurred after the flash. It is important to keep in mind that the transient flash provided no useful information about where the probe would appear. Plots of probe detection accuracy as a function of the interval between flash and probe revealed that detection performance immediately after presentation of the flash fluctuated periodically, regardless of whether the probe appeared on the grating where the flash occurred or on the grating at the non-flashed location. Moreover, the fluctuations in performance for those two probe conditions were in anti-phase, implying that attention was switching rhythmically between the two locations. The dominant frequency of the inferred switching process was not the same in both hemifields, being approximately 4 Hz in the right visual field and closer to 8 Hz in the left visual field. Remarkably, their technique proved sufficiently sensitive to yield behavioral evidence for measurable intermittency in the form of periodic alternations in spatial attention. Moreover, using Fourier analysis, Landau and Fries were able to extract clear peaks in the frequency spectrogram at frequencies around 4 Hz.

Besides shifts in visual attention, voluntary planning of motor actions can impact visual detection performance in a rhythmic, modulatory way. For example, Benedetto and Morrone (2017) discovered that suprathreshold contrast discrimination performance measured with very brief probe targets varied periodically at approximately 3 Hz immediately before voluntary initiation of a saccadic eye movement and continued until about a second after completion of the saccade. In a follow-up study, Benedetto & Morrone (2019) replicated those results using a contrast detection task. They used a multi-variable generalized linear model to extract power spectra, allowing them to distinguish separate peaks in those spectra. One peak they associated with response bias and the other with sensitivity.

Judgments of perceptual simultaneity of discrete sensory events

In the studies described above, a visual event or the intention to execute an action presumably served to reset the phase of ongoing, putative neural oscillations, thereby time-locking that reset relative to subsequently presented, brief visual test probes. The study described next employs an even more direct procedure to accomplish time-locking based on ongoing, rhythmic EEG signals themselves. Varela et al. (1981) asked participants to view two briefly flashed, laterally separated LED targets and to report whether the two flashes (each 6 ms in duration) appeared to occur “simultaneously or successively.” The key to this experiment was that the presentation of the pair of flashes was triggered based on the phase of the alpha wave component of the EEG signal being recorded from scalp electrodes, with three different phase conditions being employed: (i) presentations triggered at the positive peak of the alpha cycle, (ii) presentations triggered at the negative trough of the cycle, and (iii) presentations triggered without regard to the alpha cycle. The onset asynchrony for the pair of stimuli was held constant at a fixed duration optimized for each individual. Shown schematically in Fig. 1D are the results which reveal a clear tendency for the two stimuli to be seen as simultaneous when presented at the negative trough of the alpha wave and successive when presented at the positive peak of the alpha wave. This and other, related conditions confirmed that the phase of the alpha wave at the time of stimulus presentation had a pronounced effect on the perceived simultaneity of visual events occurring closely in time. This time-locking effect was most pronounced when triggering was locked to alpha waves recorded over the occipital pole. It is relevant for our purposes to note that Varela et al. conclude that the relation between perceptual framing, as they call it, and neural oscillations is most likely to emerge in “an experimental design which deals exclusively with a perceptual phenomenon whose measure is not tied to the further states in processing which lead to the timing of a motor behavior (p. 683).”²

Extracting signatures of temporal structure in binocular rivalry and bistable perception

Binocular rivalry, a form of perceptual bistability (Blake & Logothetis, 2002; Leopold & Logothetis, 1999; Sterzer et al., 2009), is characterized by fluctuations in perception over time when the two eyes view dissimilar monocular stimuli (Breese, 1909). The durations of successive periods of perceptual dominance of each eye’s views are unpredictable and, collectively, are well described by a lognormal distribution (Brascamp et al., 2005). Time series analyses of the successive dominance durations measured during extended viewing of rivalry reveal no signs of periodicity or any other form of temporal dependency in the time series of alternating perception—there is no predicting exactly how long the current state of perception will last based on the time series of the preceding state durations (Fox & Herrmann, 1967; Lehky, 1995).

The same pattern of results was reported by Pressnitzer and Hupé (2006), who measured a form of auditory bistability. They had people listen for several minutes to a looping stream of two sequentially broadcast tones differing in frequency, with the instruction being to continuously track whether they were hearing two distinct sequences or a single, grouped pair of tones. Perception fluctuated over time, and the histogram of durations conformed to gamma distribution but showed no sequential relation over successive durations. Interestingly, there were significant individual differences in rate of switches, but those were not correlated with switches measured for a visual task involving tracking bistable perception.

A compelling hint for the involvement of periodic temporal structure in binocular rivalry was provided by an EEG study. Doesburg et al. (2005) recorded EEG signals from an array of scalp electrodes while people pressed buttons to track fluctuations in binocular rivalry between two sets of colored bars that were oriented dissimilarly and colored differently for the two eyes. By analyzing the epochs of EEG signals for individual periods of rivalry as denoted by button presses, Doesburg and colleagues were able to identify brief bursts of increased global phase synchrony in the gamma band, with these transient signals peaking in strength a few hundred milliseconds before the manual response denoting a state change, and they were most prominent among electrodes located over visual cortical areas and frontal electrodes. The authors surmised that this gamma synchrony, which was widely dispersed among electrodes, signified the existence of neural synchronization across multiple brain areas coincident with intrinsic changes in conscious awareness.³

Fries et al. (2002) recorded single-unit activity in alert animals whose optokinetic nystagmus eye movements (OKN) were used as a proxy for stimulus dominance during dichoptic viewing of vertical gratings moving in opposite directions in the two eyes. With this technique, the direction signified by the slow phases of the OKN epochs were taken as proxies for the successive, temporary dominance states of the two monocular gratings (Fox et al., 1975). Fries and colleagues successfully biased rivalry in favor of one eye or the other by manipulating the onset timing and the relative contrasts of the rival targets, and by collecting data from strabismic cats with a patently dominant eye. Their results showed that neurons recorded in the primary visual cortex exhibited marked gamma-frequency synchronization during times when OKN signified perceptual dominance of a given neuron’s preferred direction of motion, with this synchronization within the neuron’s responses dissolving when the OKN proxy signified suppression of that preferred direction of motion. Synchrony, in other words, was strongly correlated with alternations in rivalry dominance. Fries and colleagues speculated that “enhanced gamma-frequency synchronization of the selected responses enhances the impact of the responses on target neurons at higher processing levels and thereby leads to perceptual dominance” (p. 3753).

Reaction times to sensory events

One straightforward way to look for periodicities in perceptual data is to amass a large sample of response times (RTs) to a given stimulus event and examine the pattern of variability within the resulting RT distributions. A signature of periodicity would appear as peaks and troughs in those distributions, and the temporal structure associated with that periodicity could be extracted analytically. Deployment of this strategy is exemplified in Dehaene (1993), a study that entailed measuring choice RTs to auditory and to visual events. Without going into detail, the important points to mention are (i) Dehaene’s use of regularly sequenced, discrete trials where participants made one of two speeded responses indicating which one of two equally possible stimulus events occurred, and (ii) the presentation of 1600 repetitions of the same stimulus event to each of five observers, thus yielding a data set amenable to fine-grained analysis of temporal structure. Using both autocorrelation and Fourier analysis, Dehaene extracted from individual RT histograms evidence for multiple peaks indicating that “responses were emitted more frequently at regularly recurring time intervals following stimulus presentation” (p. 264). The period of the implied oscillations varied among participants and was related to task difficulty. Of relevance to the PATS procedure presented here, those peaks in the RT distributions were neither visually conspicuous nor statistically significant “when fewer than 1000 RTs were included in the distributions” (p. 267).