Skip to main content
SpringerLink
Log in

A Computational Framework for Understanding Eye–Hand Coordination

  • Review Article
  • Published:
Journal of the Indian Institute of Science Aims and scope

Abstract

Although many studies have documented the robustness of eye–hand coordination, the computational mechanisms underlying such coordinated movements remain elusive. Here, we review the literature, highlighting the differences between mostly phenomenological studies, while emphasizing the need to develop a computational architecture which can explain eye–hand coordination across different tasks. We outline a recent computational approach which uses the accumulator model framework to elucidate the mechanisms involved in coordination of the two effectors. We suggest that, depending on the behavioral context, one of the two independent mechanisms can be flexibly used for the generation of eye and hand movements. When the context requires a tight coupling between the effectors, a common command is instantiated to drive both the effectors (common mode). Conversely, when the behavioral context demands flexibility, separate commands are sent to eye and hand effectors to initiate them flexibly (separate mode). We hypothesize that a higher order executive controller assesses behavioral context, allowing switching between the two modes. Such a computational architecture can provide a conceptual framework that can explain the observed heterogeneity in eye–hand coordination.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1:
Figure 2:
Figure 3:

Similar content being viewed by others

References

  1. von Hofsten C (1982) Eye–hand coordination in the newborn. Dev Psychol 18:450–461

    Article  Google Scholar 

  2. Land MF, Hayhoe M (2001) In what ways do eye movements contribute to everyday activities? Vis Res 41:3559–3565

    Article  Google Scholar 

  3. Prablanc C, Echallier JF, Komilis E, Jeannerod M (1979) Optimal response of eye and hand motor systems in pointing at a visual target. Biol Cybern 35:113–124

    Article  Google Scholar 

  4. Bekkering H, Adam JJ, Kingma H, Huson A, Whiting HTA (1994) Reaction time latencies of eye and hand movements in single- and dual-task conditions Harold. Exp Brain Res 97:471–476

    Article  Google Scholar 

  5. Bisley JW, Goldberg ME (2010) Attention, intention, and priority in the parietal lobe. Annu Rev Neurosci 33:1–21

    Article  Google Scholar 

  6. Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22:9475–9489

    Article  Google Scholar 

  7. Steenrod SC, Phillips MH, Goldberg ME (2013) The lateral intraparietal area codes the location of saccade targets and not the dimension of the saccades that will be made to acquire them. J Neurophysiol 109:2596–2605

    Article  Google Scholar 

  8. Cui H, Andersen RA (2007) Posterior parietal cortex encodes autonomously selected motor plans. Neuron 56:552–559

    Article  Google Scholar 

  9. Andersen RA, Cui H (2009) Intention, action planning, and decision making in parietal-frontal circuits. Neuron 63:568–583

    Article  Google Scholar 

  10. Andersen RA, Buneo CA (2002) Intentional maps in posterior parietal cortex. Annu Rev Neurosci 25:189–220

    Article  Google Scholar 

  11. Hanes DP, Schall JD (1996) Neural control of voluntary movement initiation. Science 274:427–430

    Article  Google Scholar 

  12. Thompson KG, Hanes DP, Bichot NP, Schall JD (1996) Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol 76:4040–4055

    Article  Google Scholar 

  13. Stuphorn V, Taylor TL, Schall JD (2000) Performance monitoring by the supplementary eye field. Nature 408:857–860

    Article  Google Scholar 

  14. Schall JD, Stuphorn V, Brown JW (2002) Monitoring and control of action by the frontal lobes. Neuron 36:309–322

    Article  Google Scholar 

  15. Song J-H, Rafal R, McPeek R (2010) Neural substrates of target selection for reaching movements in superior colliculus. J. Vis. 10:1082

    Article  Google Scholar 

  16. Cisek P, Kalaska JF (2005) Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45:801–814

    Article  Google Scholar 

  17. Rizzolatti G, Scandolara C, Matelli M, Gentilucci M (1981) Afferent properties of periarcuate neurons in macaque monkeys. I. Somatosensory responses. Behav Brain Res 2:125–146

    Article  Google Scholar 

  18. Rizzolatti G et al (1988) Functional organization of inferior area 6 in the macaque monkey. Exp Brain Res 71:491–507

    Article  Google Scholar 

  19. Munoz DP, Wurtz RH (1995) Saccade-related activity in monkey superior colliculus. II. Spread of activity during saccades. J Neurophysiol 73:2334–2348

    Article  Google Scholar 

  20. Munoz DP, Wurtz RH (1995) Saccade-related activity in monkey superior colliculus. I. Characteristics of burst and buildup cells. J Neurophysiol 73:2313–2333

    Article  Google Scholar 

  21. Fecteau JH, Munoz DP (2006) Salience, relevance, and firing: a priority map for target selection. Trends Cogn. Sci. 10:382–390

    Article  Google Scholar 

  22. Georgopoulos AP, Kalaska JF, Massey JT (1981) Spatial trajectories and reaction times of aimed movements: effects of practice, uncertainty, and change in target location. J. Neurophysiol 46:725–743

    Article  Google Scholar 

  23. Fromm C, Evarts EV (1982) Pyramidal tract neurons in somatosensory cortex: central and peripheral inputs during voluntary movement. Brain Res 238:186–191

    Article  Google Scholar 

  24. Biguer B, Prablanc C, Jeannerod M (1984) The contribution of coordinated eye and head movements in hand pointing accuracy. Exp Brain Res 55:462–469

    Article  Google Scholar 

  25. Sailer U, Eggert T, Ditterich J, Straube A (2000) Spatial and temporal aspects of eye–hand coordination across different tasks. Exp Brain Res 134:163–173

    Article  Google Scholar 

  26. Dean HL, Martí D, Tsui E, Rinzel J, Pesaran B (2011) Reaction time correlations during eye–hand coordination: behavior and modeling. J Neurosci 31:2399–2412

    Article  Google Scholar 

  27. Gielen CCAM, van den Heuvel PJM, van Gisbergen JAM (1984) Coordination of fast eye and arm movements in a tracking task. Exp Brain Res 56:154–161

    Article  Google Scholar 

  28. Herman R, Herman R, Maulucci R (1981) Visually triggered eye–arm movements in man. Exp Brain Res 42:392–398

    Google Scholar 

  29. Fisher B, Rogal L (1986) Eye–hand-coordination in man: a reaction time study. Biol Cybern 261:253–261

    Article  Google Scholar 

  30. Frens MA, Erkelens CJ (1991) Coordination of hand movements and saccades: evidence for a common and a separate pathway. Exp Brain Res 85:682–690

    Article  Google Scholar 

  31. Gopal A, Viswanathan P, Murthy A (2015) A common stochastic accumulator with effector-dependent noise can explain eye–hand coordination. J Neurophysiol. https://doi.org/10.1152/jn.00802.2014

    Google Scholar 

  32. Bizzi E, Kalil RE, Tagliasco V (1971) Eye–head coordination in monkeys: evidence for centrally patterned organization. Science 173:452–454

    Article  Google Scholar 

  33. Lunenburger L, Kutz DF, Hoffmann KP (2000) Influence of arm movements on saccades in humans. Eur J Neurosci 12:4107–4116

    Article  Google Scholar 

  34. Gribble PL, Everling S, Ford K, Mattar A (2002) Hand–eye coordination for rapid pointing movements: arm movement direction and distance are specified prior to saccade onset. Exp Brain Res 145:372–382

    Article  Google Scholar 

  35. Bekkering H, Pratt J, Abrams RA (1996) The gap effect for eye and hand movements. Percept Psychophys 58:628–635

    Article  Google Scholar 

  36. Armstrong IT, Judson M, Munoz DP, Johansson RS, Flanagan JR (2013) Waiting for a hand: saccadic reaction time increases in proportion to hand reaction time when reaching under a visuomotor reversal. Front Hum Neurosci 7:319

    Article  Google Scholar 

  37. Song J-H, McPeek RM (2009) Eye–hand coordination during target selection in a pop-out visual search. J Neurophysiol 102:2681–2692

    Article  Google Scholar 

  38. Fisk JD, Goodale MA (1985) The organization of eye and limb movements during unrestricted reaching to targets in contralateral and ipsilateral visual space*. Exp Brain Res 60:159–178

    Article  Google Scholar 

  39. Lawrence BM, Gardella AL (2009) Saccades and reaches, behaving differently. Exp Brain Res 195:413–418

    Article  Google Scholar 

  40. Thura D, Hadj-Bouziane F, Meunier M, Boussaoud D (2008) Hand position modulates saccadic activity in the frontal eye field. Behav Brain Res 186:148–153

    Article  Google Scholar 

  41. Thura D, Hadj-Bouziane F, Meunier M, Boussaoud D (2011) Hand modulation of visual, preparatory, and saccadic activity in the monkey frontal eye field. Cereb Cortex 21:853–864

    Article  Google Scholar 

  42. Mushiake H, Fujii N, Tanji J (1996) Visually guided saccade versus eye–hand reach: contrasting Neuronal activity in the cortical supplementary and frontal eye fields. J Neurophysiol 75:2187–2191

    Article  Google Scholar 

  43. Oristaglio J, Schneider DM, Balan PF, Gottlieb J (2006) Behavioral/systems/cognitive integration of visuospatial and effector information during symbolically cued limb movements in monkey lateral intraparietal area. J Neurosci 26:8310–8319

    Article  Google Scholar 

  44. Philipp R, Hoffmann K-P (2014) Arm movements induced by electrical microstimulation in the superior colliculus of the macaque monkey. J Neurosci 34:3350–3363

    Article  Google Scholar 

  45. Lünenburger L, Kleiser R, Stuphorn V, Miller LE, Hoffmann K-PP (2001) A possible role of the superior colliculus in eye–hand coordination. Prog Brain Res 134:109–125

    Article  Google Scholar 

  46. Pesaran B, Nelson MJ, Andersen RA (2006) Dorsal premotor neurons encode the relative position of the hand, eye, and goal during reach planning. Neuron 51:125–134

    Article  Google Scholar 

  47. Boussaoud D, Jouffrais C, Bremmer F (1998) Eye position effects on the neuronal activity of dorsal premotor cortex in the macaque monkey. J Neurophysiol 80:1132–1150

    Article  Google Scholar 

  48. Mushiake H, Tanatsugu Y, Tanji J (1997) Neuronal activity in the ventral part of premotor cortex during target-reach movement is modulated by direction of gaze. J Neurophysiol 78:567–571

    Article  Google Scholar 

  49. Batista AP, Buneo CA, Snyder LH, Andersen RA (1999) Reach plans in eye-centered coordinates. Science 285:257–260

    Article  Google Scholar 

  50. Cohen YE, Andersen RA (2000) Reaches to sounds encoded in an eye-centered reference frame. Neuron 27:647–652

    Article  Google Scholar 

  51. Song J-H, Rafal RD, McPeek RM (2011) Deficits in reach target selection during inactivation of the midbrain superior colliculus. Proc Natl Acad Sci 108:E1433–E1440

    Article  Google Scholar 

  52. Yttri EA, Liu Y, Snyder LH (2013) Lesions of cortical area LIP affect reach onset only when the reach is accompanied by a saccade, revealing an active eye–hand coordination circuit. Proc Natl Acad Sci USA 110:2371–2376

    Article  Google Scholar 

  53. Yttri EA, Wang C, Liu Y, Snyder LH (2014) The parietal reach region is limb specific and not involved in eye-hand coordination. J Neurophysiol 111:520–32

  54. Christopoulos V, Bonaiuto J, Kagan I, Andersen RA (2015) Inactivation of parietal reach region affects reaching but not saccade choices in internally guided decisions. J Neurosci 35:11719–11728

    Article  Google Scholar 

  55. Jackson SR, Newport R, Mort D, Husain M (2005) Where the eye looks, the hand follows: limb-dependent magnetic misreaching in optic ataxia. Curr Biol 15:42–46

    Google Scholar 

  56. Carey DP, Coleman RJ, Della Sala S (1997) Magnetic misreaching. Cortex 33:639–652

    Article  Google Scholar 

  57. Beurze SM, de Lange FP, Toni I, Medendorp WP (2009) Spatial and effector processing in the human parietofrontal network for reaches and saccades. J Neurophysiol 101:3053–3062

    Article  Google Scholar 

  58. Gallivan JP, McLean DA, Smith FW, Culham JC (2011) Decoding effector-dependent and effector-independent movement intentions from human parieto-frontal brain activity. J Neurosci 31:17149–17168

    Article  Google Scholar 

  59. Heed T, Beurze SM, Toni I, Röder B, Medendorp WP (2011) Functional rather than effector-specific organization of human posterior parietal cortex. J Neurosci 31:3066–3076

    Article  Google Scholar 

  60. Macaluso E, Frith CD, Driver J (2007) Delay activity and sensory-motor translation during planned eye or hand movements to visual or tactile targets. J Neurophysiol 98:3081–3094

    Article  Google Scholar 

  61. Jonikaitis D, Deubel H (2011) Independent allocation of attention to eye and hand targets in coordinated eye–hand movements. Psychol Sci A J Am Psychol Soc 22:339–347

    Article  Google Scholar 

  62. Binsted G, Chua R, Helsen W, Elliott D (2001) Eye–hand coordination in goal-directed aiming. Hum Mov Sci 20:563–585

    Article  Google Scholar 

  63. Aglioti S, DeSouza JF, Goodale MA (1995) Size-contrast illusions deceive the eye but not the hand. Curr Biol 5:679–685

    Article  Google Scholar 

  64. de Grave DDJ, Franz VH, Gegenfurtner KR (2006) The influence of the Brentano illusion on eye and hand movements. J Vis 6:727–738

    Article  Google Scholar 

  65. Gribble PL, Everling S, Ford K, Mattar A (2002) Hand–eye coordination for rapid pointing movements. Exp Brain Res 145:372–382

    Article  Google Scholar 

  66. Carpenter RHS, Williams MLL (1995) Neural computation of log likelihood in control of saccadic eye movements. Nature 377:59–62

    Article  Google Scholar 

  67. Ratcliff R (1978) A theory of memory retrieval. Psychol Rev 85:59–108

    Article  Google Scholar 

  68. Smith PL, Ratcliff R (2004) Psychology and neurobiology of simple decisions. Trends Neurosci 27:161–168

    Article  Google Scholar 

  69. Ratcliff R, Van Dongen HPA (2011) Diffusion model for one-choice reaction-time tasks and the cognitive effects of sleep deprivation. Proc Natl Acad Sci USA 108:11285–11290

    Article  Google Scholar 

  70. Ratcliff R (1980) A note on modeling accumulation of information when the rate of accumulation changes over time. J Math Psychol 84:178–184

    Article  Google Scholar 

  71. Carpenter RH (1981) Oculomotor procrastination. Eye Mov Cogn Vis Percept 237–246

  72. Wagenmakers E-J, Brown S (2007) On the linear relation between the mean and the standard deviation of a response time distribution. Psychol Rev 114:830–841

    Article  Google Scholar 

  73. Wagenmakers E-J, Grasman RPPP, Molenaar PCM (2005) On the relation between the mean and the variance of a diffusion model response time distribution. J Math Psychol 49:195–204

    Article  Google Scholar 

  74. Jana S, Gopal A, Murthy A (2017) Evidence of common and separate eye and hand accumulators underlying flexible eye–hand coordination. J Neurophysiol 117:348–364

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Sumitash Jana and Atul Gopal have contributed equally to this work.

Authors and Affiliations

  1. Centre for Neuroscience, Indian Institute of Science, Bangalore, 560012, Karnataka, India

    Sumitash Jana, Atul Gopal & Aditya Murthy

Authors
  1. Sumitash Jana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Atul Gopal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aditya Murthy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Sumitash Jana or Atul Gopal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jana, S., Gopal, A. & Murthy, A. A Computational Framework for Understanding Eye–Hand Coordination. J Indian Inst Sci 97, 543–554 (2017). https://doi.org/10.1007/s41745-017-0054-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41745-017-0054-0

Keywords

Search

Navigation