Shape specificity of neural persistence for the kinetic-depth effect matches perceptual adaptation but not sensory memory

When multistable displays—stimuli that are compatible with several comparably likely perceptual interpretations—are presented intermittently, the perceptual state at the stimulus onset shows a complex dependence on the duration of the preceding blank interval. Specifically, perception is maximally destabilized for interruptions that are approximately 500 ms long, but it is stabilized by the use of shorter or longer blank intervals. This nonmonotonic dependence of perceptual stability on the blank interval duration raises questions about a number of history effects that are involved and about their nature, including the underlying neural representations. One way to characterize history effects is by looking at their specificity to the change of display properties. Here we measured the shape specificity for perception of the kinetic-depth effect when interruptions were brief (50 ms). For this time interval, perception is thought to be stabilized by neural persistence, a lingering trace of the prior neural activity. We found that perceptual stability depended on the shapes of the objects presented both before and after the break, but not on the similarity between the objects. These results matched earlier reports of the shape specificity of neural adaptation (destabilizing aftereffect for blanks 200–800 ms long). However, our results were markedly different from the shape specificity of sensory memory of multistable perception (a stabilizing effect for blanks > 800–1,000 ms). We concluded that whereas neural persistence and adaptation both act on the same motion-selective neural representation, sensory memory depends on another, possibly partially overlapping, shape-selective neural ensemble.

The effect of the blank interval duration follows a nonmonotonic inverted U-shape curve (Kornmeier & Bach, 2004) with maximal destabilization occurring, typically, for interruptions that are 400-500 ms long. Both shorter and longer blank intervals lead to a more stable perception. Tentatively, the former stabilization occurs due to neural persistence, a lingering trace of the neural activity, effective for brief interruptions < 500 ms (Coltheart, 1980;Loftus & Irwin, 1998), and the latter due to sensory memory of multistable displays, a visual memory specific to multistable displays (de Jong, Knapen, & van Ee, 2012) that biases onset perception toward a recently dominant perceptual state (Leopold, Wilke, Maier, & Logothetis, 2002). However, the two stabilizing effects are sometimes bundled together under the common "priming" label, implying a single stabilizing memory mechanism (Brascamp, Knapen, Kanai, van Ee, & van den Berg, 2007;Jiang, Jiang, & Parasuraman, 2014). It has been also suggested that a single neural mechanism can be responsible for all these negative and positive history effects (Brascamp et al., 2007).
One way to understand the history effects at play is by examining whether they depend on and influence the same neural representations. This, in turn, can be deduced by measuring their stimulus specificity by systematically varying stimuli before and after the interruption and measuring how that changes the effect's strength. For example, this approach was used to characterize the feature selectivity of sensory memory for binocular rivalry (Pearson & Clifford, 2004). For kinetic-depth effect (KDE) displays, it was used to measure shape specificity-that is, how the strength of the effect is changed when a specific shape is used (e.g., a sphere or a cylinder), and whether the effect depends on the difference between the shapes of the objects presented before and after an interruption (e.g., a sphere followed by a cylinder vs. a sphere followed by a sphere). This effect was measured for perceptual adaptation (Pastukhov, Lissner, & Braun, 2014) and for sensory memory of multistable displays (Pastukhov, Füllekrug, & Braun, 2013). The former effect is modulated by the shape of the objects, and more specifically, its strength appears to be related to the volume of an object. However, the strength of adaptation does not depend on whether or not the two objects presented before and after the interruption are the same. Conversely, sensory memory is modulated by the similarity of two objects, and is strongest when the same object is presented both before and after the blank. When the two objects are not identical, the strength of sensory memory is proportional to their similarity, which can be defined as an overlap between the two shapes (see Figs. 6 and 7 in Pastukhov et al., 2013). However, sensory memory is not affected by the specific shape(s) of the objects, but only by their similarity. Accordingly, this indicates that the two memories influence different, even if potentially overlapping, neural representations.
In the present study, we used this paradigm to investigate the shape specificity of the history effect for short blank intervals (< 200 ms). As we noted above, it is thought to occur due to the lingering activation of the neural population after the stimulus offset (Coltheart, 1980;Loftus & Irwin, 1998). This neural persistence enables perception to "bridge" brief interruptions in presentation, integrating sequentially presented stimuli (Eriksen & Collins, 1967). An example of such temporal integration is binocular rivalry, produced by an interleaved dichoptic presentation (O'Shea & Crassini, 1984;van Boxtel, Alais, Erkelens, & van Ee, 2008).
The aim of our work was to characterize the shape specificity of neural persistence and to compare it to the shape specificity of the two other history effects, neural adaptation and sensory memory. We report that it matches the former but not the latter.

Method
Observers Ten observers (five females, five males), 19-42 years of age participated in this experiment. All procedures were in accordance with the national ethical standards on human experimentation and with the Declaration of Helsinki of 1975, as revised in 2008. The study was in full accordance with the ethical guidelines of the University of Bamberg and was approved by an umbrella evaluation for psychophysical testing of the university ethics committee (Ethikrat) on August 18, 2017. Informed consent was obtained from all observers prior to each experimental session. All participants had normal or corrected-to-normal vision and normal color vision, all tested by standard tests in situ, and were naïve to the purpose of the study. For their participation, observers received credits within the framework of a mandatory module of research participation in accordance with the standards of the University of Bamberg.

Apparatus
The stimuli were generated using custom code and the PsychoPy library (Peirce et al., 2019). They were displayed on an EIZO CG245W screen (www.eizoglobal.com/support/ db/products/model/CG245W), with the size of the visible area being 52.0 × 32.4 cm, operating at a resolution of 1,920 × 1,200 pixels and a refresh rate of 60 Hz. At a viewing distance of 65 cm, a single pixel subtended 0.023°of visual angle. Participants viewed the display through anaglyph redgreen glasses.
The experimental room was dimly lit. Chin and forehead rests were used to stabilize the viewing position. Observers responded using the arrow keys of a computer keyboard.

Stimuli
We employed an ordered set of four "band" shapes that had been used in prior work on the shape specificity of perceptual adaptation (Pastukhov et al., 2014) and sensory memory of multistable displays (Pastukhov et al., 2013); see Fig. 1A and the supplementary video. The advantage of this set is that each shape includes the shapes to the right of it (as depicted in Fig.  1A). For instance, the dual-band is a band plus an extra band, the quad-band is a dual-band plus two more bands, and so forth. These stimuli also allow for a crude but straightforward measure of shape difference, by assigning the indices 1-4 to, respectively, the sphere, quad-band, dual-band, and band, and computing the absolute difference between the indices. For example, the quad-band is more similar to the dual-band than to the band stimulus. Note, however, that different dissimilarity levels have different numbers of corresponding stimulus pairs. Specifically, for the dissimilarity of 1, the results were averaged over three pairs of stimuli; for the dissimilarity of 2, over two pairs; but for the dissimilarity of 3, the data come from only a single pair (the sphere vs. the band).
The KDE display stimuli consisted of 2,000 dots distributed randomly over the surface of the objects. All shapes were generated anew for each trial. All of the objects measured 4°in height and rotated at 0.25 Hz. The diameter of the individual dots represented 0.15°, and their luminance was 110 cd/m 2 .

Procedure
Each experimental session consisted of 11 experimental blocks. The first block was dedicated to training and was discarded from the analysis. Each block consisted of 48 trials, so that each ordered pair of objects was presented four times. Individual trials consisted of the prime period (1,500 ms), followed by a brief blank (50 ms) and a probe period (1,000 ms); see Fig. 1B. Following the presentation, the participants reported on the final direction of rotation of both objects.
The axis of rotation of the prime was initially tilted by 45°e ither clockwise or counterclockwise, the specific orientation was picked randomly on each trial. The axis of rotation remained stationary for 400 ms, before rotating to the opposite complementary orientation-respectively, counterclockwise or clockwise. The rotation of the axis took 500 ms. The orientation of the axis of rotation of the probe matched the final orientation of the prime. The purpose of this manipulation was to reduce the opposing influence of perceptual adaptation, which would otherwise mask the effect of neural persistence (Blake, Sobel, & Gilroy, 2003).
During the initial 400 ms (T bias ), the prime shape was biased toward a random direction of rotation via stereoscopic cues. The projections for the two eyes differed in their orientations around the vertical axis by from 0°(fully ambiguous) up to 1°(maximally disambiguated). The disambiguation strength was maximal at the onset and was linearly reduced to zero over the course of 400 ms. The purpose of this manipulation was to randomize the prime's direction of rotation. Otherwise, the direction of rotation of both the prime and the probe could be determined by a common perceptual Fig. 1 Displays and procedure. a The four objects used in the study, as viewed on the screen (top row) and from above (bottom row). b The schematic procedure, with time intervals not drawn to scale. See also the supplementary video. T delay , randomized onset delay at the beginning of each trial; T bias , time interval when stereo disambiguation was applied so as to bias the prime's direction of rotation; T rotation , time interval when the axis of rotation rotated by 90°to the complementary orientation, in this case from 45°counterclockwise to 45°clockwise; T stationary , time interval when the prime was fully ambiguous and rotated around its final axis of rotation; T probe , time interval for the probe stimulus history, making it hard to disentangle the influence of the prime from that of the common effects, such as sensory memory (Leopold et al., 2002).

Statistical analysis
Following Leopold et al. (2002), the influence of the perceptual state of the prime on that of the probe was quantified using the probability of survival (P survival ), which is the probability that both objects rotated in the same direction. Values above .5 indicate the stabilizing history effect (probe tends to rotate in the same direction as the prime). Conversely, values below .5 indicate a destabilizing history effect (probe tends to rotate in the opposite direction).
The analysis was performed in R (R Core Team, 2018), version 3.5.1. The effects of the prime and probe shapes and their interaction were assessed using linear mixed models (Bates, Mächler, Bolker, & Walker, 2015) with a logit link function. Bayes factors were computed using the BayesFactor package (Morey & Rouder, 2012). The coefficient of determination based on the likelihood-ratio test was computed using the MuMIn package (Barton, 2019).

Results
The results of the experiment are summarized in Fig. 2 and Table 1. In short, only the shape of the prime and the probe influenced the strength of the persistence aftereffect, but we observed no interaction between them, indicating that persistence did not depend on the similarity of the two shapes. We also assessed this hypothesis by computing the probability of survival as a function of the (dis)similarity of shapes, with a linear mixed model analysis showing no effect (Fig. 2B). for a linear-mixed model, with participants' identity as a random factor and shape similarity/difference as a fixed factor (R 2 is a coefficient of determination based on the likelihood-ratio test, and BF is the Bayes factor; see the Method section for details). c, d P surival for a given shape when it was acting as a prime (c) or a probe (d). Dots depict individual participants Moreover, a shape that was a good prime was also a good probe, and vice versa, since the probabilities of survival for the individual shapes used either as a prime (Fig. 2C) or a probe (Fig. 2D) were highly correlated (R = .97, t(38) = 23, p < .0001, BF = 3×10 18) . Finally, we compared the results of our experiment with those of two earlier studies of shape specificity, in perceptual adaptation (Pastukhov et al., 2014) and sensory memory (Pastukhov et al., 2013). To this end, we computed the average probability of survival for each pair of shapes (16 data points per study) and plotted it against the probability of survival for the same pair of shapes in this study. Our present results were a mirror image of perceptual adaptation (lower plot in Fig. 3). In other words, pairs of shapes that led to stronger persistence for short blanks (50 ms) also led to stronger adaptation aftereffects for longer blank intervals (~500 ms) or when neural persistence was curtailed via a mask to maximize the influence of adaptation (see Exp. 2 in Pastukhov et al., 2014). However, we found no correlation between neural persistence and sensory memory (upper plot in Fig. 3). Thus, persistence for short blank intervals (50 ms) did not predict persistence for long blank intervals (1,000+ ms).

Discussion
The aim of this study was to investigate the shape specificity of neural persistence for kinetic-depth effect displays and to compare it to the shape specificity of other history effects. We report that the shape of the prime and of the probe objects strongly modulated the probability that the same direction of motion was reported before and after the break. However, we found no interaction between the prime and the probe, with shapes' similarity having no effect on the probability of survival. When comparing the results of the present study with those for perceptual adaptation (Pastukhov et al., 2014) and sensory memory (Pastukhov et al., 2013), we found a strong negative correlation with the former but no correlation with the latter.
As we noted in the Results section, the observed probability of survival for individual pairs of shapes is an almost perfect mirror image of the observed probabilities from an earlier study on the shape specificity of perceptual adaptation; see Fig. 3 above and Fig. 2 in Pastukhov et al. (2014). In other words, shapes that led to stronger persistence for short blank intervals also produced stronger perceptual adaptation. In both cases, shapes that produced stronger history effects as primes were also more sensitive as probes. Also, in both cases, the effects were stronger for objects that had bigger "volume"; see Fig. 7 in Pastukhov et al. (2014). However, further studies will be required in order to clarify the relationship between shape geometry and the activation of underlying neural representations. Taken together, these results indicate that the same or heavily overlapping neural representations are involved in both history effects. This validates earlier experimental and theoretical work suggesting that these two opposing influences can be modeled using a single mechanism (Brascamp et al., 2007).    Conversely, shape specificity for neural persistence (a positive effect over short blank intervals < 200 ms) was markedly different from that of sensory memory of multistable displays (also a positive history effect, but for longer blanks > 1,000 ms). Prior evidence showed that their difference goes beyond shape specificity. For example, neural persistence is sensitive to masking (Irwin & Thomas, 2008), whereas the sensory memory of multistable perception is not (Maier, Wilke, Logothetis, & Leopold, 2003). This indicates that although the two effects may appear to be similar and are even sometimes bundled together under a common "priming" label (Brascamp et al., 2007;Jiang et al., 2014), they rely on different neural mechanisms and should be studied and modeled as two independent effects.
The dependence of sensory memory on an object's shape congruency makes it tempting to view sensory memory as a form of "higher-level" memory, as compared to neural persistence and perceptual adaptation. However, that may not be the case, because other properties of sensory memory indicate its low-level, sensory nature. For example, it is maintained independently for individual visual features rather than for an object as a whole (Pearson & Clifford, 2004). It is also specific to a retinotopic location of the original stimulus (Knapen, Brascamp, Adams, & Graf, 2009). These properties are clearly at odds with an idea of a high-level object-based memory and are rather similar to those of neural persistence (McRae, Butler, & Popiel, 1987;Pastukhov, Prasch, & Carbon, 2018) and perceptual adaptation (Clifford et al., 2007). Perhaps the shape specificity of sensory memory may indicate that it has a special role in perceptual inference (Pastukhov, 2016). However, our current lack of understanding of the neural mechanisms, and even the purpose, of sensory memory complicates the comparison with the other two history effects.
As we noted above, the identical shape specificities of neural persistence and perceptual adaptation indicate that the same neural populations both persist and adapt. Prior neurophysiology (Dodd, Krug, Cumming, & Parker, 2001) and imaging studies (Brouwer & van Ee, 2007) place them in the medial temporal region within the dorsal visual stream; see also Orban (2011) for a comprehensive review. In contrast, shape-congruent sensory memory may also involve neurons in the neighboring object-sensitive lateral occipital cortex (Ferber, Humphrey, & Vilis, 2003) or in regions of intraparietal sulcus (Vanduffel et al., 2002).
To conclude, we report that the strength of neural persistence depends only on the shape of the two objects, not on their similarity. This shape specificity is identical to that of perceptual adaptation but is different from that for sensory memory of multistable displays. Thus, for ambiguously rotating KDE displays, the same motion-sensitive neurons persist and adapt.
Open Practices Statement All data files and the analysis code are available under a Creative Commons Attribution 4.0 International Public License, at https://osf.io/cdtfr or at https://github.com/alexander-pastukhov/shape-specificity-ofneural-persistence.