Experimental Brain Research

, Volume 236, Issue 12, pp 3131–3148 | Cite as

Effects of auditory feedback on movements with two-segment sequence and eye–hand coordination

  • Miya K. RandEmail author
Research Article


The present study investigated the effect of auditory feedback on planning and control of two-segment reaching movements and eye–hand coordination. In particular, it was examined whether additional auditory information indicating the progression of the initial reach (i.e., passing the midway and contacting the target) affects the performance of that reach and gaze shift to the second target at the transition between two segments. Young adults performed a rapid two-segment reaching task, in which both the first and second segments had two target sizes. One out of three auditory feedback conditions included the reach-progression information: a continuous tone was delivered at a consistent timing during the initial reach from the midway to the target contact. Conversely, the other two were control conditions: a continuous tone was delivered at a random timing in one condition or not delivered in the other. The results showed that the initial reach became more accurate with the auditory reach-progression cue compared to without any auditory cue. When that cue was available, movement time of the initial reach was decreased, which was accompanied by an increased peak velocity and a decreased time to peak velocity. These findings suggest that the auditory reach-progression feedback enhanced the preplanned control of the initial reach. Deceleration time of that reach was also decreased with auditory feedback, but it was observed regardless of whether the sound contained the reach-progression information. At the transition between the two segments, the onset latencies of both the gaze shift and reach to the second target became shorter with the auditory reach-progression cue, the effect of which was pronounced when the initial reach had a higher terminal accuracy constraint. This suggests that the reach-progression cue enhanced verification of the termination of initial reach, thereby facilitating the initiation of eye and hand movements to the second target. Taken together, the additional auditory information of reach-progression enhances the planning and control of multi-segment reaches and eye–hand coordination at the segment transition.


Auditory feedback Eye–hand coordination Gaze anchoring Multimodal sensorimotor control Reaching Sequential movement 



This research was supported by the German Research Foundation (DFG), Ra 2183/1–3. The author thanks Anika Beyer, Maleen Greine, and Franziska Schywalski for their support in data collection.

Compliance with ethical standards

Conflict of interest

The author declares that she has no conflict of interest.


  1. Abrahamse EL. Ruitenberg MFL, De Kleine E, Verwey WB (2013) Control of automated behaviour: insights from the discrete sequence production task. Front Hum Neurosci 7:1–16Google Scholar
  2. Adam JJ (1992) The effects of objectives and constraints on motor control strategy in reciprocal aiming movements. J Motor Behav 24:173–185Google Scholar
  3. Adam JJ, Bruggen DPW van der, Bekkering H (1993) The control of discrete and reciprocal target-aiming responses: evidence for the exploitation of mechanics. Hum Mov Sci 12:353–364Google Scholar
  4. Adam JJ, Paas FGWC, Eyssen ICJM, Slingerland H, Bekkering H, Drost M (1995) The control of two-element, reciprocal aiming movements: evidence for chunking. Hum Mov Sci 14:1–11Google Scholar
  5. Adam JJ, Nieuwenstein JH, Huys R, Paas FG, Kingma H, Willems P, Werry M (2000) Control of rapid targeted hand movements: the one-target advantage. J Exp Psychol Hum Percept Perform 26:295–312PubMedGoogle Scholar
  6. Adam JJ, Buetti S, Kerzel D (2012) Coordinated flexibility: how initial gaze position modulates eye-hand coordination and reaching. J Exp Psychol Hum Percept Perform 38:891–901PubMedGoogle Scholar
  7. Adamovich SV, Fluet GG, Mathai A, Qiu Q, Lewis J, Merians AS (2009) Design of a complex virtual reality simulation to train finger motion for persons with hemiparesis: a proof of concept study. J Neuroeng Rehabil 6:28PubMedPubMedCentralGoogle Scholar
  8. Alais D, Burr D (2004) No direction-specific bimodal facilitation for audiovisual motion detection. Brain Res Cogn Brain Res 19:185–194PubMedGoogle Scholar
  9. Bekkering H, Adam JJ, van den Aarssen A, Kingma H, Whiting HTA (1995) Interference between saccadic eye and goal-directed hand movements. Exp Brain Res 106:475–484PubMedGoogle Scholar
  10. Bernstein N (1967) The coordination and regulation of movements. Pergamon Press, LondonGoogle Scholar
  11. Bieńkiewicz MM, Young WR, Craig CM (2004) Balls to the wall: how acoustic information from a ball in motion guides interceptive movement in people with Parkinson’s disease. Neuroscience 275:508–518Google Scholar
  12. Bowman MC, Johansson RS, Flanagan JR (2009) Eye-hand coordination in a sequential target contact task. Exp Brain Res 195:273–283PubMedGoogle Scholar
  13. Bresciani JP, Dammeier F, Ernst MO (2006) Vision and touch are automatically integrated for the perception of sequences of events. J Vis 6:554–564PubMedGoogle Scholar
  14. Cheng K, Shettleworth SJ, Huttenlocher J, Rieser JJ (2007) Bayesian integration of spatial information. Psychol Bull 133:625–637PubMedGoogle Scholar
  15. Christina RW, Rose DJ (1985) Premotor and motor reaction time as a function of response complexity. Res Q Exerc Sport 56:306–315Google Scholar
  16. Colonius H, Diederich A (2017) Formal models and quantitative measures of multisensory integration: a selective overview. E J Neurosci. CrossRefGoogle Scholar
  17. Crawford JD, Medendorp WP, Marotta JJ (2004) Spatial transformations for eye-hand coordination. J Neurophysiol 92:10–19PubMedGoogle Scholar
  18. Debats NB, Heuer H (2018) Optimal integration of actions and their visual effects is based on both online and prior causality evidence. Sci Rep 8:9796PubMedPubMedCentralGoogle Scholar
  19. Debats NB, Ernst MO, Heuer H (2017a) Perceptual attraction in tool-use: evidence for a reliability-based weighting mechanism. J Neurophysiol 117:1569–1580PubMedPubMedCentralGoogle Scholar
  20. Debats NB, Ernst MO, Heuer H (2017b) Kinematic cross-correlation induces sensory integration across separate objects. E J Neurosci 46:2826–2834Google Scholar
  21. Driver J, Spence C (1998) Attention and the crossmodal construction of space. Trends Cogn Sci 2:254–262PubMedGoogle Scholar
  22. Elliott D (1988) The influence of visual target and limb information on manual aiming. Can J Psychol 42:57–68PubMedGoogle Scholar
  23. Elliott D, Hansen S, Grierson LE, Lyons J, Bennett SJ, Hayes SJ (2010) Goal-directed aiming: two components but multiple processes. Psychol Bull 136:1023–1044PubMedGoogle Scholar
  24. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433PubMedGoogle Scholar
  25. Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169PubMedGoogle Scholar
  26. Fischman MG, Reeve TG (1992) Slower movement times may not necessarily imply on-line programming. J Hum Mov Stud 22:131–144Google Scholar
  27. Fitts PM (1954) The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 47:381–391PubMedGoogle Scholar
  28. Flanagan JR, Johansson RS (2003) Action plans used in action observation. Nature 424:769–771PubMedGoogle Scholar
  29. Flanagan JR, Bowman MC, Johansson RS (2006) Control strategies in object manipulation tasks. Curr Opin Neurobiol 16:650–659PubMedGoogle Scholar
  30. Gepshtein S, Burge J, Ernst MO, Banks MS (2005) The combination of vision and touch depends on spatial proximity. J Vis 5:1013–1023PubMedPubMedCentralGoogle Scholar
  31. Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490PubMedGoogle Scholar
  32. Gielen CCAM, Van den Heuvel PJM, Van Gisbergen JAM (1984) Coordination of fast eye and arm movements in a tracking task. Exp Brain Res 56:156–161Google Scholar
  33. Harrar V, Harris LR (2008) The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity. Exp Brain Res 186:517–524PubMedGoogle Scholar
  34. Harrar V, Harris LR, Spence C (2017) Multisensory integration is independent of perceived simultaneity. Exp Brain Res 235:763–775PubMedGoogle Scholar
  35. Jeannerod M, Arbib MA, Rizzolatti G, Sakata H (1995) Grasping objects: the cortical mechanisms of visuomotor transformation. Trends Neurosci 18:314–320PubMedGoogle Scholar
  36. Johansson RS, Westling G, Bäckström A, Flanagan JR (2001) Eye-hand coordination in object manipulation. J Neurosci 21:6917–6932PubMedGoogle Scholar
  37. Kagerer FA, Contreras-Vidal JL (2009) Adaptation of sound localization induced by rotated visual feedback in reaching movements. Exp Brain Res 193:315–321PubMedGoogle Scholar
  38. Ketcham CJ, Seidler RD, Van Gemmert AWA, Stelmach GE (2002) Age-related kinematic differences as influenced by task difficulty, target size, and movement amplitude. J Gerontol: Psychol Sci 57B:P54–P64Google Scholar
  39. Khan MA, Mottram TM, Adam JJ, Buckolz E (2010) Sequential aiming with two limbs and the one-target advantage. J Mot Behav 42:325–330PubMedGoogle Scholar
  40. Lashley KS (1951) The problem of serial order in behavior. In: Jeffress LA (ed) Cerebral mechanisms in behavior. Wiley, New York, pp 112–136Google Scholar
  41. Marteniuk RG, MacKenzie CL, Jeannerod M, Athenes S, Dugas C (1987) Constraints on human arm movement trajectories. Can J Psychol 41:365–378PubMedGoogle Scholar
  42. Ma-Wyatt A, Stritzke M, Trommershäuser J (2010) Eye-hand coordination while pointing rapidly under risk. Exp Brain Res 203:131–145PubMedGoogle Scholar
  43. Meyer DE, Abrams RA, Kornblum S, Wright CE, Smith JEK (1988) Optimality in human motor performance: ideal control of rapid aimed movements. Psychol Rev 95:340–370PubMedGoogle Scholar
  44. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory–visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14:115–128PubMedGoogle Scholar
  45. Neggers SFW, Bekkering H (2000) Ocular gaze is anchored to the target of an ongoing pointing movement. J Neurophysiol 83:639–651PubMedGoogle Scholar
  46. Neggers SFW, Bekkering H (2001) Gaze anchoring to a pointing target is present during the entire pointing movement and is driven by a non-visual signal. J Neurophysiol 86:961–970PubMedGoogle Scholar
  47. Paillard J (1982) The contribution of peripheral and central vision to visually guided reaching. In: Ingle D, Goodale M, Mansfield R (eds) Analysis of visual behaviour. MIT Press, Cambridge, pp 367–385Google Scholar
  48. Pelz J, Hayhoe M, Loeber R (2001) The coordination of eye, head, and hand movements in a natural task. Exp Brain Res 139:266–277PubMedGoogle Scholar
  49. Posner MI, Nissen MJ, Klein RM (1976) Visual dominance: an information-processing account of its origins and significance. Psychol Rev 83:157–171PubMedGoogle Scholar
  50. Povel D-J, Collard R (1982) Structural factors in patterned finger tapping. Acta Psychol (Amst) 52:107–123Google Scholar
  51. Rand MK (2014) Segment interdependency and gaze anchoring during manual two–segment sequences. Exp Brain Res 232:2753–2765PubMedGoogle Scholar
  52. Rand MK, Shimansky YP (2013) Two-phase strategy of neural control for planar reaching movements. I. XY coordination variability and its relation to end-point variability. Exp Brain Res 225:55–73PubMedGoogle Scholar
  53. Rand MK, Stelmach GE (2000) Segment interdependency and difficulty in two-stroke sequences. Exp Brain Res 134:228–236PubMedGoogle Scholar
  54. Rand MK, Stelmach GE (2010) Effects of hand termination and accuracy constraint on eye-hand coordination during sequential two-segment movements. Exp Brain Res 207:197–211PubMedGoogle Scholar
  55. Rand MK, Stelmach GE (2011) Adaptation of gaze anchoring through practice in young and older adults. Neurosci Lett 492:47–51PubMedPubMedCentralGoogle Scholar
  56. Rand MK, Stelmach GE (2012) Effect of aging on coordinated eye and hand movements with two-segment sequence. Mot Control 16:447–465Google Scholar
  57. Rand MK, Alberts JL, Stelmach GE, Bloedel JR (1997) The influence of movement segment difficulty on movements with two-stroke sequence. Exp Brain Res 115:137–146PubMedGoogle Scholar
  58. Robertson JV, Hoellinger T, Lindberg P, Bensmail D, Hanneton S, Roby-Brami A (2009) Effect of auditory feedback differs according to hemiparesis: a comparative pilot study. J Neuroeng Rehabil 6:45PubMedPubMedCentralGoogle Scholar
  59. Rosati G, Oscari F, Spagnol S, Avanzini F, Masiero S (2012) Effect of task-related continuous auditory feedback during learning of tracking motion exercises. J Neuroeng Rehabil 9:79PubMedPubMedCentralGoogle Scholar
  60. Rosenbaum DA (1987) Successive approximations to a model of human motor programming. In: Bower G (ed) The psychology of learning and motivation. Academic Press, New York, pp 153–182Google Scholar
  61. Rosenbaum DA (1991) Human motor control. Academic Press, San DiegoGoogle Scholar
  62. Sanders JA, Knill DC (2004) Visual feedback control of hand movements. J Neurosci 24:3223–3234Google Scholar
  63. Schmidt RA, Zelaznik HN, Hawkins B, Frank JS, Quinn JT (1979) Motor output variability: a theory for the accuracy of rapid motor acts. Psychol Rev 86:415–451Google Scholar
  64. Sedda A, Monaco S, Bottini G, Goodale MA (2011) Integration of visual and auditory information for hand actions: preliminary evidence for the contribution of natural sounds to grasping. Exp Brain Res 209:365–374PubMedGoogle Scholar
  65. Shimansky YP (2007) Role of optimization in simple and learning based adaptation and its biologically plausible mechanisms. In: Williams TO (ed) Biological cybernetics research trends. Nova Science Publishers, Hauppauge, pp 95–164Google Scholar
  66. Shimansky YS, Rand MK (2013) Two-phase strategy of controlling motor coordination determined by task performance optimality. Biol Cybern 107:107–129PubMedGoogle Scholar
  67. Short MW, Fischman MG, Wang YT (1996) Cinematographical analysis of movement pathway constraints in rapid target-striking tasks. J Mot Behav 28:157–163PubMedGoogle Scholar
  68. Sidaway B, Sekiya H, Fairweather M (1995) Movement variability as a function accuracy demand in programmed serial aiming responses. J Mot Behav 27:67–76Google Scholar
  69. Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P (2015) Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Exp Brain Res 233:909–925PubMedGoogle Scholar
  70. Smeets JBJ, van den Dobbelsteen JJ, de Grave DDJ, van Beers MJ, Brenner E (2006) Sensory integration does not lead to sensory calibration. Proc Natl Acad Sci 103:18781–18786PubMedGoogle Scholar
  71. Soechting JF (1984) Effect of target size on spatial and temporal characteristics of a pointing movement in man. Exp Brain Res 54:121–132PubMedGoogle Scholar
  72. Spence C, Driver J (1997) On measuring selective attention to an expected sensory modality. Percept Psychophys 59:389–403PubMedGoogle Scholar
  73. Sternberg S, Knoll RL, Turock DL (1990) Hierarchical control in the execution of action sequences: test of two invariance properties. In: Jeannerod M (ed) Attention and performance XIII. Erlbaum, Hillsdale, pp 3–55Google Scholar
  74. Teasdale N, Bard C, Fleury M, Young D, Proteau L (1993) Determining movement onsets from temporal series. J Mot Behav 25:97–106PubMedGoogle Scholar
  75. Todorov E (2004) Optimality principles in sensorimotor control. Nat Neurosci 7:907–915PubMedPubMedCentralGoogle Scholar
  76. van Beers RJ, Sittig AC, Denier van der Gon JJ (1999) Integration of proprioceptive and visual position-information: An experimentally supported model. J Neurophysiol 81:1355–1364PubMedGoogle Scholar
  77. van Beers RJ, Wolpert DM, Haggard P (2002) When feeling is more important than seeing in sensorimotor adaptation. Curr Biol 12:834–837PubMedGoogle Scholar
  78. Van der Burg E, Olivers CN, Bronkhorst AW, Theeuwes J (2008) Pip and pop: nonspatial auditory signals improve spatial visual search. J Exp Psychol Hum Percept Perform 34:1053–1065PubMedGoogle Scholar
  79. Verwey WB (1999) Evidence for a multistage model of practice in a sequential movement task. J Exp Psychol Hum Percept Perform 25:1693–1708Google Scholar
  80. Verwey WB, Shea HC, Wright DL (2015) A cognitive framework for explaining serial processing and sequence execution strategies. Psychon Bull Rev 22:54–77PubMedGoogle Scholar
  81. Vindras P, Viviani P (2005) Planning short pointing sequences. Exp Brain Res 160:141–153PubMedGoogle Scholar
  82. Weiss P, Stelmach GE, Hefter H (1997) Programming of a movement sequence in Parkinson’s disease. Brain 120:91–102PubMedGoogle Scholar
  83. Wolpert DM, Flanagan JR (2001) Motor prediction. Curr Biol 11:R729–R732PubMedGoogle Scholar
  84. Woodworth RS (1899) The accuracy of voluntary movement. Psychol Rev 3(Suppl. 2):1–114Google Scholar
  85. Zahariev MA, MacKenzie CL (2007) Grasping at ‘thin air’: multimodal contact cues for reaching and grasping. Exp Brain Res 180:69–84PubMedGoogle Scholar
  86. Zahariev MA, Mackenzie CL (2008) Auditory contact cues improve performance when grasping augmented and virtual objects with a tool. Exp Brain Res 186:619–627PubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Leibniz Research Centre for Working Environment and Human Factors (IfADo)DortmundGermany

Personalised recommendations