Abstract
Transsaccadic integration (TSI) refers to the perceptual integration of visual information collected across separate gaze fixations. Current theories of TSI disagree on whether it relies solely on visual algorithms or also uses extra-retinal signals. We designed a task in which subjects had to rely on internal oculomotor signals to synthesize remembered stimulus features presented within separate fixations. Using a mouse-controlled pointer, subjects estimated the intersection point of two successively presented bars, in the dark, under two conditions: Saccade task (bars viewed in separate fixations) and Fixation task (bars viewed in one fixation). Small, but systematic biases were observed in both intersection tasks, including position-dependent vertical undershoots and order-dependent horizontal biases. However, the magnitude of these errors was statistically indistinguishable in the Saccade and Fixation tasks. Moreover, part of the errors in the Saccade task were dependent on saccade metrics, showing that egocentric oculomotor signals were used to fuse remembered location and orientation features across saccades. We hypothesize that these extra-retinal signals are normally used to reduce the computational load of calculating visual correspondence between fixations. We further hypothesize that TSI may be implemented within dynamically updated recurrent feedback loops that interconnect a common eye-centered map in occipital cortex with both the “dorsal” and “ventral” streams of visual analysis.
Similar content being viewed by others
References
Barry SR, Bloomberg JJ, Huebner WP (1997) The effects of visual context on manual localization of remembered targets. Neuroreport 8:469–473
Blohm G, Missal M, Lefevre P (2003) Interaction between smooth anticipation and saccades during ocular orientation in darkness. J Neurophysiol 89(3):1423–1433
Bock O (1986) Contribution of retinal versus extraretinal signals towards visual localization in goal-directed movements. Exp Brain Res 64:476–482
Bridgeman B, Mayer M (1983) Failure to integrate visual information from successive fixations. Bull Psychon Soc 21:285–286
Chen LL, Nakamura K (1998) Head-centred representation and spatial memory in rat posterior parietal cortex. Psychobiology 26:119–127
Colby CL, Duhamel JR, Goldberg ME (1996) Visual, presaccadic and cognitive activation of single neurons in monkey lateral intraparietal area. J Neurophysiol 76(5):2841–2852
Crawford JD, Medendorp WP, Marotta JJ (2004) Spatial transformations for eye–hand coordination. J Neurophysiol 92:10–19
Currie CB, McConkie GW, Carlson-Radvansky LA, Irwin DE (2000) The role of the saccade target object in the perception of a visually stable world. Percept Psychophys 62:673–683
Deubel H, Schneider WX, Bridgeman B (1996) Postsaccadic target blanking prevents saccadic suppression of image displacement. Vision Res 36:985–996
Deubel H, Bridgeman B, Schneider WX (1998) Immediate post-saccadic information mediates space constancy. Vision Res 38:3147–3159
DiMattia BV, Kesner RP (1988) Spatial cognitive maps: differential role of parietal cortex and hippocampal formation. Behav Neurosci 102(4):471–480
Duhamel J, Colby CL, Goldberg ME (1992) The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255:90–92
Germeys F, De Graef P, Verfaillie K (2002) Transsaccadic perception of saccade target and flanker objects. J Exp Psychol Hum Percept Perform 28:868–883
Hallett PE, Lightstone AD (1976) Saccadic eye movements towards stimuli triggered by prior saccades. Vision Res 16:99–106
Hayhoe M, Lachter J, Feldman J (1991) Integration of form across saccadic eye movements. Perception 20:393–402
Hayhoe MM, Shrivastava A, Mruczek R, Pelz JB (2003) Visual memory and motor planning in a natural task. J Vis 3(1):49–63
Haywood R (1986) Acquisition of a micro scale photographic survey using an autonomous submersible. IEEE Oceans 5:1423–1426
von Helmholtz H (1963) Hanbuch der Physiologischen Optik [Handbook of physiological optics]. In: Southall JPC (ed. 2nd Trans.) Helmholtz’s treatise on physiological optics, vol 3. Dover, New York (Original published 1866; English translation originally published 1925), pp 247–270
Henderson JM (1992) Identifying objects across saccades: effects of extrafoveal preview and flanker object context. J Exp Psychol Learn Mem Cogn 18(3):521–530
Henderson JM, Hollingworth A (1999) The role of fixation position in detecting scene changes across saccades. Psychol Sci 10:438–443
Henderson JM, Siefert ABC (1999) The influence of enantiomorphic transformation on transsaccadic object integration. J Exp Psychol Hum Percept Perform 25:243–255
Henriques DYP, Crawford JD (2000) Direction-dependent distortions of retinocentric space in the visuomotor transformation for pointing. Exp Brain Res 132:179–194
Henriques DYP, Klier EM, Smith MA, Lowy D, Crawford JD (1998) Gaze-centered remapping of remembered visual space in an open-loop pointing task. J Neurosci 18(4):1583–1594
Hess R, Field D (1999) Integration of contours: new insights. Trends Cogn Sci 3:480–486
Hubbard TL (1995) Cognitive representation of motion: evidence for friction and gravity analogues. J Exp Psychol Learn Mem Cogn 21:241–254
Irwin DE (1991) Information integration across saccadic eye movements. Cogn Psychol 23:420–456
Irwin DE (1992) Memory for position and identity across eye movements. J Exp Psychol Learn Mem Cogn 18:307–317
Irwin DE (1996) Integrating information across saccadic eye movements. Curr Dir Psychol Sci 5:94–100
Irwin DE, Andrews R (1996) Integration and accumulation of information across saccadic eye movements. In: Inui T, McClelland JL (eds) Attention and performance XVI: information integration in perception and communication. MIT Press, Cambridge, pp 125–155
Irwin DE, Yantis S, Jonides J (1983) Evidence against visual integration across saccadic eye movements. Percept Psychophys 34:49–57
Irwin DE, Brown J, Sun J (1988) Visual masking and visual integration across saccadic eye movements. J Exp Psychol Gen 117:276–287
Irwin DE, Zacks J, Brown J (1990) Visual memory and the perception of a stable visual environment. Percept Psychophys 47:35–46
Irwin DE, McConkie GW, Carlson-Radvansky L, Currie C (1994) A localist evaluation solution for visual stability across saccades. Behav Brain Sci 17:265–266
Kahneman D (1968) Method, findings, and theory in studies of visual masking. Psychol Bull 70:404–425
Kerzel D (2002) The locus of “memory displacement” is at least partially perceptual: effects of velocity, expectation, friction, memory averaging, and weight. Percept Psychophys 64:680–692
Kourtzi Z, Tolias AS, Altman CF, Augath M, Logothetis NK (2003) Integration of local features into global shapes: monkey and human fMRI studies. Neuron 37:333–346
Landman R, Spekreijse H, Lamme VAF (2003) Large capacity storage of integrated objects before change blindness. Vision Res 43:149–164
Lemay M, Bertram CP, Stelmach GE (2004) Pointing to an allocentric and egocentric remembered target. Motor Control 8(1): 16–32
MacKay DM (1973) Visual stability and voluntary eye movements. In: Jung R (ed) Handbook of sensory physiology, vol 8. Springer, Berlin Heidelberg New York, pp 307–331
Mateeff J, Gourevich A (1983) Peripheral vision and perceived visual direction. Biol Cybern 49:111–118
Matin E (1974) Saccadic suppression: a review and an analysis. Psychol Bull 81:899–917
Mays LE, Sparks DL (1980) Saccades are spatially, not retinocentrically, coded. Science 208(4448):1163–1165
McConkie GW, Currie C (1996) Visual stability across saccades while viewing complex pictures. J Exp Psychol Hum Percept Perform 22:563–581
McConkie GW, Rayner K (1976) Identifying the span of the effective stimulus in reading: literature review and theories in reading. In: Singer H, Ruddell RB (eds) Theoretical models and processes of reading. International Reading Association, Newark, pp 137–162
McConkie GW, Zola D (1979) Is visual information integrated across successive fixations in reading? Percept Psychophys 25:21–224
Melcher D, Morrone MC (2003) Spatiotopic temporal integration of visual motion across saccadic eye movements. Nat Neurosci 8:877–881
Medendorp WP, Goltz H, Vilis T, Crawford JD (2003) Gaze-centered updating of visual space in human parietal cortex. J Neurosci 23:6209–6214
Milner AD, Goodale MA (1995) The visual brain in action. Oxford University Press, London
Mishkin M, Ungerleider LG, Macko KA (1983) Object vision and spatial vision: two cortical pathways. Trends Neurosci 6(10):414–417
Moore T, Armstrong KM (2003) Selective gating of visual signals by microstimulation of frontal cortex. Nature 421:370–373
Musseler J, van der Heijden AHC, Mahmud SH, Deubel H, Ertsey S (1999) Relative mislocalization of briefly presented stimuli in the retinal periphery. Percept Psychophys 61: 1646–1661
Nakamura K, Colby CL (2000) Visual, saccade-related, and cognitive activation of single neurons in monkey extrastriate area V3A. J Neurophysiol 84(2):677–692
Nakamura K, Colby CL (2002) Updating of the visual representation in monkey striate and extrastriate cortex during saccades. Proc Natl Acad Sci 99:4026–4031
Niemeier M, Crawford JD, Tweed DB (2003) Optimal transsaccadic integration explains distorted spatial perception. Nature 422(6927):76–80
O’Regan JK, Levy-Schoen A (1983) Integrating visual information from successive fixations: does transsaccadic fusion exist? Vision Res 23:765–768
Palmer J, Ames CT (1992) Measuring the effect of multiple eye fixations on memory for visual attributes. Percept Psychophys 52:295–306
Pisella L, Mattingley JB (2004) The contribution of spatial remapping impairments to unilateral visual neglect. Neurosci Biobehav Rev 28:181–200
Pollatsek A, Rayner K, Collins W (1984) Integrating pictorial information across eye movements. J Exp Psychol Gen 113:426–442
Rayner K (1978) Eye movements in reading and information processing. Psychol Bull 85:618–660
Rayner K, Pollatsek A (1983) Is visual information integrated across saccades? Percept Psychophys 34:39–48
Robinson DA (1963) A method of measuring eye movement using a sclera search coil in a magnetic field. IEEE Trans Biomed Eng 10:137–145
Salthouse TA, Ellis CL, Diener DC, Somberg BL (1981) Stimulus processing during eye fixations. J Exp Psychol Hum Percept Perform 7(3):611–623
Schiller PH (1965) Monotopic and dichoptic visual masking by patterns and flashes. J Exp Psychol 69:193–199
Schoumans N, Koenderink JJ, Kappers AML (2000) Change in perceived spatial directions due to context. Percept Psychophys 63(3) 532–539
Schreiber K, Crawford JD, Fetter M, Tweed D (2001) The motor side of depth vision. Nature 410(6830):819–822
Simons DJ, Mitroff SR, Franconeri SL (2003) Scene perception: what we can learn from visual integration and change detection. In: Rhodes G, Peterson MA (eds) Perception of faces, objects, and scenes: analytic and holistic processes. Oxford University Press, London, pp 335–351
Sheth BR, Shimojo S (2001) Compression of space in visual memory. Vision Res 41:329–341
Stewart AL, Purcell DG (1974) Visual backward masking by a flash of light: a study of U-shaped detection functions. J Exp Psychol 103:553–566
Umeno MM, Goldberg ME (1997) Spatial processing in the monkey frontal eye field I. Predictive visual responses. J Neurophysiol 78:1373–1383
van der Heijden AHC, van der Geest JN, de Leeuw F, Krikke K, Muessler J (1999) Sources of position-perception error for small isolated targets. Psychol Res 62(1):20–35
Verfaillie K, De Graef P (2000) Transsaccadic memory for position and orientation of saccade source and target. J Exp Psychol Hum Percept Perform 26:1243–1259
Verfaillie K, De Troy A, Van Rensbergen J (1994) Transsaccadic integration of biological motion. J Exp Psychol Learn Mem Cogn 20:649–670
Verfaillie K, De Graef P, Germeys F, Gysen V, Van Eccelpoel C (2001) Selective transsaccadic coding of object and event-diagnostic information. Psychol Belg 41(1–2):89–114
Vindras P, Desmurget M, Prablanc C, Viviani P (1998) Pointing errors reflect biases in the perception of the initial hand position. J Neurophysiol 79:3290–3294
Walker MF, Fitzgibbon EJ, Goldberg ME (1995) Neurons in the monkey superior colliculus predict the visual result of impending saccadic eye movements. J Neurophysiol 73:1988–2003
Zheng Q, Chellappa R (1993) A computational vision approach to image registration. IEEE Trans Image Process 2(3):311–325
Acknowledgments
The authors thank Saihong Sun and Dr. Hongying Wang for technical assistance. This work was supported by grants from the Natural Sciences and Engineering Research Council of Canada and the Canadian Institutes of Health Research. J.D. Crawford holds a Canada Research Chair.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Prime, S.L., Niemeier, M. & Crawford, J. Transsaccadic integration of visual features in a line intersection task. Exp Brain Res 169, 532–548 (2006). https://doi.org/10.1007/s00221-005-0164-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-005-0164-1