Advertisement

Predictive eye movements in natural vision

Abstract

In the natural world, the brain must handle inherent delays in visual processing. This is a problem particularly during dynamic tasks. A possible solution to visuo-motor delays is prediction of a future state of the environment based on the current state and properties of the environment learned from experience. Prediction is well known to occur in both saccades and pursuit movements and is likely to depend on some kind of internal visual model as the basis for this prediction. However, most evidence comes from controlled laboratory studies using simple paradigms. In this study, we examine eye movements made in the context of demanding natural behavior, while playing squash. We show that prediction is a pervasive component of gaze behavior in this context. We show in addition that these predictive movements are extraordinarily precise and operate continuously in time across multiple trajectories and multiple movements. This suggests that prediction is based on complex dynamic visual models of the way that balls move, accumulated over extensive experience. Since eye, head, arm, and body movements all co-occur, it seems likely that a common internal model of predicted visual state is shared by different effectors to allow flexible coordination patterns. It is generally agreed that internal models are responsible for predicting future sensory state for control of body movements. The present work suggests that model-based prediction is likely to be a pervasive component in natural gaze control as well.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  1. Abernethy B (1990) Expertise, visual search, and information pick-up in squash. Perception 19:63–77

  2. Aivar P, Hayhoe M, Mruczek R (2005) Role of spatial memory in saccadic targeting in natural tasks. J Vis 5:177–193

  3. Assad JA, Maunsell JH (1995) Neuronal correlates of inferred motion in primate posterior parietal cortex. Nature 373(6514):518–521

  4. Babcock JS, Pelz JB (2004) Building a lightweight eyetracking headgear. In: ACM SIGCHI eye tracking research & applications symposium, pp 109–114

  5. Bahill AT, LaRitz T (1984) Why can’t batters keep their eye on the ball? Am Sci 72:249–253

  6. Bahill AT, Baldwin DG, Venkateswaran J (2005) Predicting a baseball’s path. Am Sci 93:218–225

  7. Barborica A, Ferrera VP (2003) Estimating invisible target speed from neuronal activity in monkey frontal eye field. Nat Neurosci 6:66–74

  8. Barnes G, Collins C (2008) Evidence for a link between the extra-retinal component of random-onset pursuit and the anticipatory pursuit of predictable object motion. J Neurophysiol 100:1135–1146

  9. Battaglia PW, Schrater P (2007) Humans trade off viewing time and movement duration to improve visuomotor accuracy in a fast reaching task. J Neurosci 27(26):6984–6994

  10. Battaglia PW, Schrater P, Kersten D (2005) Auxiliary object knowledge influences visually-guided interception behavior. In: Proceedings of 2nd symposium APGV, pp 145–152

  11. Becker W, Fuchs A (1985) Prediction in the oculomotor system: smooth pursuit during the transient disappearance of a visual target. Exp Brain Res 57:562–575

  12. Blakemore SJ, Frith CD, Wolpert DW (2001) The cerebellum is involved in predicting the sensory consequences of action. Neuroreport 12(9):1879–1884

  13. Brouwer A, Knill DC (2007) The role of memory in visually guided reaching. J Vis 7(5):6 (1–12)

  14. Chen-Harris H, Joiner WM, Ethier V, Zee DS, Shadmehr R (2008) Adaptive control of saccades via internal feedback. J Neurosci 28(11):2804–2813

  15. Churchland M, Chou I, Lisberger S (2003) Evidence for object permanence in the smooth pursuit movements of monkeys. J Neurophysiol 90:2205–2218

  16. Collins C, Barnes GR (1999) Independent control of head and gaze movements during head-free pursuit in humans. J Physiol 515.1:299–314

  17. Ferrera V, Barborica A (2010) Internally generated error signals in monkey frontal eye field during an inferred motion task. J Neurosci 30(35):11612–11623

  18. Findlay JM (1981) Spatial and temporal factors in the predictive generation of saccadic eye movements. Vision Res 21(3):347–354

  19. Flanagan J, Wing A (1997) The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads. J Neurosci 17:1519–1528

  20. Fukushima K, Akao T, Shichinohe N, Nitta T, Kurkin S, Fukushima J (2008) Predictive signals in the pursuit area of the monkey frontal eye fields. Prog Brain Res 171:433–440

  21. Fukushima K, Kasahara S, Akao T, Kurkin S, Fukushima J, Peterson B (2009) Eye-pursuit and reafferent head movement signals carried by pursuit neurons in the caudal part of the frontal eye fields during head-free pursuit. Cereb Cortex 19:263–275

  22. Hayhoe M (2009) Visual memory in motor planning and action. In: Brockmole J (ed) Memory for the visual world. Psychology Press, UK, pp 117–139

  23. Hayhoe M, Ballard D (2005) Eye movements in natural behavior. Trends Cogn Sci 9(4):188–193

  24. Hayhoe M, Mennie N, Sullivan B, Gorgos K (2005) The role of internal models and prediction in catching balls. In: Proceedings of AAAI fall symposium series

  25. Isotalo E, Lasker AG, Zee DS (2005) Cognitive influences on predictive saccadic tracking. Exp Brain Res 165:461–469

  26. Itti L, Koch C (2001) Computational modeling of visual attention. Nat Rev Neurosci 2:194–203

  27. Johansson RS (1996) Sensory control of dextrous manipulation in humans. In: Wing A, Haggard P, Flanagan J (eds) Hand and brain: the neurophysiology and psychology of hand movements. Academic Press, San Diego, pp 381–414

  28. Joiner W, Shelhamer M (2006) Pursuit and saccadic tracking exhibit a similar dependence on movement preparation time. Exp Brain Res 173:572–586

  29. Jovancevic-Misic J, Hayhoe M (2009) Adaptive gaze control in natural environments. J Neurosci 29(19):6234–6238

  30. Kawato M (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9(6):718–727

  31. Koerding K, Wolpert D (2004) Bayesian integration in sensorimotor learning. Nature 427:244–247

  32. Koerding K, Wolpert D (2006) Bayesian decision theory in sensorimotor control. Trends Cogn Sci 10:319–326

  33. Kowler E, Martins A, Pavel M (1984) The effect of expectations on slow oculomotor control- IV Anticipatory smooth eye movements depend on prior target motions. Vis Res 24:197–210

  34. Krauzlis R (2005) The control of voluntary eye movements: new perspectives. Neuroscientist 11(2):124–137

  35. Land M (2004) Eye movements in daily life. In: Chalupa L, Werner J (eds) The visual neurosciences, vol 2. MIT Press, Cambridge, pp 1357–1368

  36. Land MF, Furneaux S (1997) The knowledge base of the oculomotor system. Philos Trans R Soc Lond B Biol Sci 352:1231–1239

  37. Land MF, McLeod P (2000) From eye movements to actions: how batsmen hit the ball. Nat Neurosci 3:1340–1345

  38. Lopez-Moliner J, Field DT, Wann JP (2007) Interceptive timing: prior knowledge matters. J Vis 7:1–8

  39. Madelain L, Krauzlis RJ (2003) Effects of learning on smooth pursuit during transient disappearance of a visual target. J Neurophysiol 90:972–982

  40. Mulliken G, Andersen R (2009) Forward modles and state estimation in parietal cortex. In: Gazzaniga MS (ed) The cognitive neurosciences IV. MIT Press, Cambridge, pp 599–611

  41. Nyffeler T, Rivaud-Pechoux S, Wattiez N, Gaymard N (2008) Involvement of the supplementary eye field in oculomotor predictive behavior. J Cog Neurosci 20(9):1583–1594

  42. Pierrot-Deseilligny Ch, Müri A, Nyffeler B, Milea D (2005) The role of the human dorsolateral prefrontal cortex in ocular motor behavior. Ann NY Acad Sci 1039:239–251

  43. Saunders JA, Knill DC (2003) Humans use continuous visual feedback from the hand to control fast reaching movements. Exp Brain Res 152:341–352

  44. Schlicht E, Schrater P (2007) Reach-to-grasp trajectories adjust for uncertainty in the location of visual targets. Exp Brain Res 182:47–57

  45. Shadmehr R, Mussa-Ivaldi FA (1994) Adaptive representation of dynamics during learning of a motor task. J Neurosci 14:3208–3224

  46. Shelhamer M, Joiner WM (2003) Saccades exhibit abrupt transition between reactive and predictive, predictive saccade sequence have long-term correlations. J Neurophysiol 90:2763–2769

  47. Shichinohe N, Akao T, Kurkin S, Fukushima J, Kaneko C, Fukushima K (2009) Memory and decision making in the frontal cortex during visual motion processing for smooth pursuit eye movements. Neuron 62:717–732

  48. Simo LS, Krisky CM, Sweeney JA (2005) Functional neuroanatomy of anticipatory behavior: dissociation between sensory-driven and memory-driven systems. Cereb Cortex 15:1982–1991

  49. Sprague N, Ballard D, Robinson A (2007) Modeling embodied visual behaviors. ACM Trans Appl Percept 4(2):11

  50. Tabata H, Muira K, Kawano K (2008) Trial-by-trial updating of the gain in preparation for smooth pursuit eye movement based on past experience in humans. J Neurophysiol 99:747–758

  51. Tassinari H, Hudson TE, Landy MS (2006) Combining priors and noisy visual cues in a rapid pointing task. J Neurosci 26:10154–10163

  52. Tresilian JR, Houseman JH (2005) Systematic variation in performance of an interceptive action with changes in the temporal constraints. Q J Exp Psychol 58A:447–466

  53. Wolpert D (2007) Probabilistic models in human sensorimotor control. Hum Mov Sci 26:511–524

  54. Wolpert D, Miall C, Kawato M (1998) Internal models in the cerebellum. Trends Cogn Sci 2:338–347

  55. Xaio Q, Barborica A, Ferrera V (2007) Modulation of visual responses in macaque frontal eye field during covert tracking of invisible targets. Cereb Cortex 17(4):918–928

  56. Zago M, Bosco G, Maffei V, Iosa M, Ivanenko Y, Lacquaniti F (2004) Internal models of target motion: expected dynamics overrides measured kinematics in timing manual interceptions. J Neurophysiol 91:1620–1634

  57. Zago M, McIntyre J, Patrice Senot P, Lacquaniti F (2008) Internal models and prediction of visual gravitational motion. Vis Res 48:1532–1538

  58. Zago M, McIntyre J, Patrice Senot P, Lacquaniti F (2009) Visuo-motor coordination and internal models for object interception. Exp Brain Res 192(4):571–604

Download references

Acknowledgments

This work was supported by NIH Grant EY05729. The authors wish to acknowledge the assistance in data collection of Jason Droll and also players Eric Hernady and Mithun Mukherjee, as well as Martin Heath and members of the University of Rochester Squash Team.

Author information

Correspondence to Mary M. Hayhoe.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (AVI 263 kb)

Supplementary material 2 (AVI 137 kb)

Supplementary material 3 (AVI 219 kb)

Supplementary material 1 (AVI 263 kb)

Supplementary material 2 (AVI 137 kb)

Supplementary material 3 (AVI 219 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hayhoe, M.M., McKinney, T., Chajka, K. et al. Predictive eye movements in natural vision. Exp Brain Res 217, 125–136 (2012) doi:10.1007/s00221-011-2979-2

Download citation

Keywords

  • Saccadic eye movements
  • Prediction
  • Internal models
  • Squash
  • Gaze pursuit