Experimental Brain Research

, Volume 169, Issue 4, pp 532–548 | Cite as

Transsaccadic integration of visual features in a line intersection task

  • Steven L. Prime
  • Matthias Niemeier
  • J.D. Crawford
Research Article

Abstract

Transsaccadic integration (TSI) refers to the perceptual integration of visual information collected across separate gaze fixations. Current theories of TSI disagree on whether it relies solely on visual algorithms or also uses extra-retinal signals. We designed a task in which subjects had to rely on internal oculomotor signals to synthesize remembered stimulus features presented within separate fixations. Using a mouse-controlled pointer, subjects estimated the intersection point of two successively presented bars, in the dark, under two conditions: Saccade task (bars viewed in separate fixations) and Fixation task (bars viewed in one fixation). Small, but systematic biases were observed in both intersection tasks, including position-dependent vertical undershoots and order-dependent horizontal biases. However, the magnitude of these errors was statistically indistinguishable in the Saccade and Fixation tasks. Moreover, part of the errors in the Saccade task were dependent on saccade metrics, showing that egocentric oculomotor signals were used to fuse remembered location and orientation features across saccades. We hypothesize that these extra-retinal signals are normally used to reduce the computational load of calculating visual correspondence between fixations. We further hypothesize that TSI may be implemented within dynamically updated recurrent feedback loops that interconnect a common eye-centered map in occipital cortex with both the “dorsal” and “ventral” streams of visual analysis.

Keywords

Visual perception Saccades 

Notes

Acknowledgments

The authors thank Saihong Sun and Dr. Hongying Wang for technical assistance. This work was supported by grants from the Natural Sciences and Engineering Research Council of Canada and the Canadian Institutes of Health Research. J.D. Crawford holds a Canada Research Chair.

References

  1. Barry SR, Bloomberg JJ, Huebner WP (1997) The effects of visual context on manual localization of remembered targets. Neuroreport 8:469–473 PubMedCrossRefGoogle Scholar
  2. Blohm G, Missal M, Lefevre P (2003) Interaction between smooth anticipation and saccades during ocular orientation in darkness. J Neurophysiol 89(3):1423–1433PubMedCrossRefGoogle Scholar
  3. Bock O (1986) Contribution of retinal versus extraretinal signals towards visual localization in goal-directed movements. Exp Brain Res 64:476–482PubMedCrossRefGoogle Scholar
  4. Bridgeman B, Mayer M (1983) Failure to integrate visual information from successive fixations. Bull Psychon Soc 21:285–286Google Scholar
  5. Chen LL, Nakamura K (1998) Head-centred representation and spatial memory in rat posterior parietal cortex. Psychobiology 26:119–127Google Scholar
  6. Colby CL, Duhamel JR, Goldberg ME (1996) Visual, presaccadic and cognitive activation of single neurons in monkey lateral intraparietal area. J Neurophysiol 76(5):2841–2852PubMedGoogle Scholar
  7. Crawford JD, Medendorp WP, Marotta JJ (2004) Spatial transformations for eye–hand coordination. J Neurophysiol 92:10–19PubMedCrossRefGoogle Scholar
  8. Currie CB, McConkie GW, Carlson-Radvansky LA, Irwin DE (2000) The role of the saccade target object in the perception of a visually stable world. Percept Psychophys 62:673–683PubMedGoogle Scholar
  9. Deubel H, Schneider WX, Bridgeman B (1996) Postsaccadic target blanking prevents saccadic suppression of image displacement. Vision Res 36:985–996PubMedCrossRefGoogle Scholar
  10. Deubel H, Bridgeman B, Schneider WX (1998) Immediate post-saccadic information mediates space constancy. Vision Res 38:3147–3159PubMedCrossRefGoogle Scholar
  11. DiMattia BV, Kesner RP (1988) Spatial cognitive maps: differential role of parietal cortex and hippocampal formation. Behav Neurosci 102(4):471–480PubMedCrossRefGoogle Scholar
  12. Duhamel J, Colby CL, Goldberg ME (1992) The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255:90–92PubMedCrossRefGoogle Scholar
  13. Germeys F, De Graef P, Verfaillie K (2002) Transsaccadic perception of saccade target and flanker objects. J Exp Psychol Hum Percept Perform 28:868–883PubMedCrossRefGoogle Scholar
  14. Hallett PE, Lightstone AD (1976) Saccadic eye movements towards stimuli triggered by prior saccades. Vision Res 16:99–106PubMedCrossRefGoogle Scholar
  15. Hayhoe M, Lachter J, Feldman J (1991) Integration of form across saccadic eye movements. Perception 20:393–402PubMedCrossRefGoogle Scholar
  16. Hayhoe MM, Shrivastava A, Mruczek R, Pelz JB (2003) Visual memory and motor planning in a natural task. J Vis 3(1):49–63PubMedCrossRefGoogle Scholar
  17. Haywood R (1986) Acquisition of a micro scale photographic survey using an autonomous submersible. IEEE Oceans 5:1423–1426CrossRefGoogle Scholar
  18. von Helmholtz H (1963) Hanbuch der Physiologischen Optik [Handbook of physiological optics]. In: Southall JPC (ed. 2nd Trans.) Helmholtz’s treatise on physiological optics, vol 3. Dover, New York (Original published 1866; English translation originally published 1925), pp 247–270Google Scholar
  19. Henderson JM (1992) Identifying objects across saccades: effects of extrafoveal preview and flanker object context. J Exp Psychol Learn Mem Cogn 18(3):521–530PubMedCrossRefGoogle Scholar
  20. Henderson JM, Hollingworth A (1999) The role of fixation position in detecting scene changes across saccades. Psychol Sci 10:438–443CrossRefGoogle Scholar
  21. Henderson JM, Siefert ABC (1999) The influence of enantiomorphic transformation on transsaccadic object integration. J Exp Psychol Hum Percept Perform 25:243–255CrossRefGoogle Scholar
  22. Henriques DYP, Crawford JD (2000) Direction-dependent distortions of retinocentric space in the visuomotor transformation for pointing. Exp Brain Res 132:179–194PubMedCrossRefGoogle Scholar
  23. Henriques DYP, Klier EM, Smith MA, Lowy D, Crawford JD (1998) Gaze-centered remapping of remembered visual space in an open-loop pointing task. J Neurosci 18(4):1583–1594PubMedGoogle Scholar
  24. Hess R, Field D (1999) Integration of contours: new insights. Trends Cogn Sci 3:480–486PubMedCrossRefGoogle Scholar
  25. Hubbard TL (1995) Cognitive representation of motion: evidence for friction and gravity analogues. J Exp Psychol Learn Mem Cogn 21:241–254PubMedCrossRefGoogle Scholar
  26. Irwin DE (1991) Information integration across saccadic eye movements. Cogn Psychol 23:420–456CrossRefPubMedGoogle Scholar
  27. Irwin DE (1992) Memory for position and identity across eye movements. J Exp Psychol Learn Mem Cogn 18:307–317CrossRefGoogle Scholar
  28. Irwin DE (1996) Integrating information across saccadic eye movements. Curr Dir Psychol Sci 5:94–100CrossRefGoogle Scholar
  29. Irwin DE, Andrews R (1996) Integration and accumulation of information across saccadic eye movements. In: Inui T, McClelland JL (eds) Attention and performance XVI: information integration in perception and communication. MIT Press, Cambridge, pp 125–155Google Scholar
  30. Irwin DE, Yantis S, Jonides J (1983) Evidence against visual integration across saccadic eye movements. Percept Psychophys 34:49–57PubMedGoogle Scholar
  31. Irwin DE, Brown J, Sun J (1988) Visual masking and visual integration across saccadic eye movements. J Exp Psychol Gen 117:276–287PubMedCrossRefGoogle Scholar
  32. Irwin DE, Zacks J, Brown J (1990) Visual memory and the perception of a stable visual environment. Percept Psychophys 47:35–46PubMedGoogle Scholar
  33. Irwin DE, McConkie GW, Carlson-Radvansky L, Currie C (1994) A localist evaluation solution for visual stability across saccades. Behav Brain Sci 17:265–266CrossRefGoogle Scholar
  34. Kahneman D (1968) Method, findings, and theory in studies of visual masking. Psychol Bull 70:404–425PubMedCrossRefGoogle Scholar
  35. Kerzel D (2002) The locus of “memory displacement” is at least partially perceptual: effects of velocity, expectation, friction, memory averaging, and weight. Percept Psychophys 64:680–692PubMedGoogle Scholar
  36. Kourtzi Z, Tolias AS, Altman CF, Augath M, Logothetis NK (2003) Integration of local features into global shapes: monkey and human fMRI studies. Neuron 37:333–346PubMedCrossRefGoogle Scholar
  37. Landman R, Spekreijse H, Lamme VAF (2003) Large capacity storage of integrated objects before change blindness. Vision Res 43:149–164PubMedCrossRefGoogle Scholar
  38. Lemay M, Bertram CP, Stelmach GE (2004) Pointing to an allocentric and egocentric remembered target. Motor Control 8(1): 16–32PubMedGoogle Scholar
  39. MacKay DM (1973) Visual stability and voluntary eye movements. In: Jung R (ed) Handbook of sensory physiology, vol 8. Springer, Berlin Heidelberg New York, pp 307–331Google Scholar
  40. Mateeff J, Gourevich A (1983) Peripheral vision and perceived visual direction. Biol Cybern 49:111–118PubMedCrossRefGoogle Scholar
  41. Matin E (1974) Saccadic suppression: a review and an analysis. Psychol Bull 81:899–917PubMedCrossRefGoogle Scholar
  42. Mays LE, Sparks DL (1980) Saccades are spatially, not retinocentrically, coded. Science 208(4448):1163–1165PubMedCrossRefGoogle Scholar
  43. McConkie GW, Currie C (1996) Visual stability across saccades while viewing complex pictures. J Exp Psychol Hum Percept Perform 22:563–581PubMedCrossRefGoogle Scholar
  44. McConkie GW, Rayner K (1976) Identifying the span of the effective stimulus in reading: literature review and theories in reading. In: Singer H, Ruddell RB (eds) Theoretical models and processes of reading. International Reading Association, Newark, pp 137–162Google Scholar
  45. McConkie GW, Zola D (1979) Is visual information integrated across successive fixations in reading? Percept Psychophys 25:21–224Google Scholar
  46. Melcher D, Morrone MC (2003) Spatiotopic temporal integration of visual motion across saccadic eye movements. Nat Neurosci 8:877–881CrossRefGoogle Scholar
  47. Medendorp WP, Goltz H, Vilis T, Crawford JD (2003) Gaze-centered updating of visual space in human parietal cortex. J Neurosci 23:6209–6214PubMedGoogle Scholar
  48. Milner AD, Goodale MA (1995) The visual brain in action. Oxford University Press, LondonGoogle Scholar
  49. Mishkin M, Ungerleider LG, Macko KA (1983) Object vision and spatial vision: two cortical pathways. Trends Neurosci 6(10):414–417CrossRefGoogle Scholar
  50. Moore T, Armstrong KM (2003) Selective gating of visual signals by microstimulation of frontal cortex. Nature 421:370–373PubMedCrossRefGoogle Scholar
  51. Musseler J, van der Heijden AHC, Mahmud SH, Deubel H, Ertsey S (1999) Relative mislocalization of briefly presented stimuli in the retinal periphery. Percept Psychophys 61: 1646–1661PubMedGoogle Scholar
  52. Nakamura K, Colby CL (2000) Visual, saccade-related, and cognitive activation of single neurons in monkey extrastriate area V3A. J Neurophysiol 84(2):677–692PubMedGoogle Scholar
  53. Nakamura K, Colby CL (2002) Updating of the visual representation in monkey striate and extrastriate cortex during saccades. Proc Natl Acad Sci 99:4026–4031PubMedCrossRefGoogle Scholar
  54. Niemeier M, Crawford JD, Tweed DB (2003) Optimal transsaccadic integration explains distorted spatial perception. Nature 422(6927):76–80PubMedCrossRefGoogle Scholar
  55. O’Regan JK, Levy-Schoen A (1983) Integrating visual information from successive fixations: does transsaccadic fusion exist? Vision Res 23:765–768PubMedCrossRefGoogle Scholar
  56. Palmer J, Ames CT (1992) Measuring the effect of multiple eye fixations on memory for visual attributes. Percept Psychophys 52:295–306PubMedGoogle Scholar
  57. Pisella L, Mattingley JB (2004) The contribution of spatial remapping impairments to unilateral visual neglect. Neurosci Biobehav Rev 28:181–200PubMedCrossRefGoogle Scholar
  58. Pollatsek A, Rayner K, Collins W (1984) Integrating pictorial information across eye movements. J Exp Psychol Gen 113:426–442PubMedCrossRefGoogle Scholar
  59. Rayner K (1978) Eye movements in reading and information processing. Psychol Bull 85:618–660PubMedCrossRefGoogle Scholar
  60. Rayner K, Pollatsek A (1983) Is visual information integrated across saccades? Percept Psychophys 34:39–48PubMedGoogle Scholar
  61. Robinson DA (1963) A method of measuring eye movement using a sclera search coil in a magnetic field. IEEE Trans Biomed Eng 10:137–145PubMedGoogle Scholar
  62. Salthouse TA, Ellis CL, Diener DC, Somberg BL (1981) Stimulus processing during eye fixations. J Exp Psychol Hum Percept Perform 7(3):611–623CrossRefGoogle Scholar
  63. Schiller PH (1965) Monotopic and dichoptic visual masking by patterns and flashes. J Exp Psychol 69:193–199PubMedCrossRefGoogle Scholar
  64. Schoumans N, Koenderink JJ, Kappers AML (2000) Change in perceived spatial directions due to context. Percept Psychophys 63(3) 532–539Google Scholar
  65. Schreiber K, Crawford JD, Fetter M, Tweed D (2001) The motor side of depth vision. Nature 410(6830):819–822PubMedCrossRefGoogle Scholar
  66. Simons DJ, Mitroff SR, Franconeri SL (2003) Scene perception: what we can learn from visual integration and change detection. In: Rhodes G, Peterson MA (eds) Perception of faces, objects, and scenes: analytic and holistic processes. Oxford University Press, London, pp 335–351Google Scholar
  67. Sheth BR, Shimojo S (2001) Compression of space in visual memory. Vision Res 41:329–341PubMedCrossRefGoogle Scholar
  68. Stewart AL, Purcell DG (1974) Visual backward masking by a flash of light: a study of U-shaped detection functions. J Exp Psychol 103:553–566PubMedCrossRefGoogle Scholar
  69. Umeno MM, Goldberg ME (1997) Spatial processing in the monkey frontal eye field I. Predictive visual responses. J Neurophysiol 78:1373–1383PubMedGoogle Scholar
  70. van der Heijden AHC, van der Geest JN, de Leeuw F, Krikke K, Muessler J (1999) Sources of position-perception error for small isolated targets. Psychol Res 62(1):20–35PubMedCrossRefGoogle Scholar
  71. Verfaillie K, De Graef P (2000) Transsaccadic memory for position and orientation of saccade source and target. J Exp Psychol Hum Percept Perform 26:1243–1259PubMedCrossRefGoogle Scholar
  72. Verfaillie K, De Troy A, Van Rensbergen J (1994) Transsaccadic integration of biological motion. J Exp Psychol Learn Mem Cogn 20:649–670PubMedCrossRefGoogle Scholar
  73. Verfaillie K, De Graef P, Germeys F, Gysen V, Van Eccelpoel C (2001) Selective transsaccadic coding of object and event-diagnostic information. Psychol Belg 41(1–2):89–114Google Scholar
  74. Vindras P, Desmurget M, Prablanc C, Viviani P (1998) Pointing errors reflect biases in the perception of the initial hand position. J Neurophysiol 79:3290–3294PubMedGoogle Scholar
  75. Walker MF, Fitzgibbon EJ, Goldberg ME (1995) Neurons in the monkey superior colliculus predict the visual result of impending saccadic eye movements. J Neurophysiol 73:1988–2003PubMedGoogle Scholar
  76. Zheng Q, Chellappa R (1993) A computational vision approach to image registration. IEEE Trans Image Process 2(3):311–325PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2005

Authors and Affiliations

  • Steven L. Prime
    • 1
    • 2
    • 3
  • Matthias Niemeier
    • 1
    • 2
    • 4
  • J.D. Crawford
    • 1
    • 2
    • 3
    • 5
  1. 1.Centre for Vision ResearchYork UniversityTorontoCanada
  2. 2.CIHR Group for Action and PerceptionYork UniversityTorontoCanada
  3. 3.Department of PsychologyYork UniversityTorontoCanada
  4. 4.Department of Life SciencesUniversity of TorontoTorontoCanada
  5. 5.Department of Biology and Kinesiology and Health SciencesYork UniversityTorontoCanada

Personalised recommendations