Abstract
The arm movement control system often relies on visual feedback to drive motor adaptation and to help specify desired trajectories. Here we studied whether kinematic errors that were indicated with auditory feedback could be used to control reaching in a way comparable with when vision was available. We randomized twenty healthy adult subjects to receive either visual or auditory feedback of their movement trajectory error with respect to a line as they performed timed reaching movements while holding a robotic joystick. We delivered auditory feedback using spatialized pink noise, the loudness and location of which reflected kinematic error. After a baseline period, we unexpectedly perturbed the reaching trajectories using a perpendicular viscous force field applied by the joystick. Subjects adapted to the force field as well with auditory feedback as they did with visual feedback and exhibited comparable after effects when the force field was removed. When we changed the reference trajectory to be a trapezoid instead of a line, subjects shifted their trajectories by about the same amount with either auditory or visual feedback of error. These results indicate that arm motor networks can readily incorporate auditory feedback to alter internal models and desired trajectories, a finding with implications for the organization of the arm motor control adaptation system as well as sensory substitution and motor training technologies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Auvray M, Hanneton S, O’Regan JK (2007) Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘the voice’. Percept 36(3):416–430
Bach-y-Rita P, Kercel SW (2003) Sensory substitution and the human-machine interface. Trends Cogn Sci 7(12):541–546
Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadden L (1969) Vision substitution by tactile image projection. Trans Pac Coast Otoophthalmol Soc Annu Meet 221(5184):83–91
Brown D, Macpherson T, Ward J (2011) Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device. Percept 40(9):1120–1135
Burge J, Ernst MO, Banks MS (2008) The statistical determinants of adaptation rate in human reaching. J Vis 8(4):1–19
Collignon O, Champoux F, Voss P, Lepore F (2011) Sensory rehabilitation in the plastic brain. Prog Brain Res 191:211–231
Criscimagna-Hemminger SE, Bastian AJ, Shadmehr R (2010) Size of error affects cerebellar contributions to motor learning. J Neurophysiol 103(4):2275–2284
Fu KMG, Johnston TA, Shah AS, Arnold L, Smiley J, Hackett TA, Garraghty PE, Schroeder CE (2003) Auditory cortical neurons respond to somatosensory stimulation. J Neurosci 23(20):7510–7515
Ghahramani Z, Wolpert DM, Jordan MI (1997) Computational models for sensorimotor integration. In: Morasso PG, Sanguineti V (eds) Self-organization, computational maps and motor control. Elsevier, Amsterdam, pp 117–147
Hyvarinen J, Shelepin Y (1979) Distribution of visual and somatic functions in the parietal associative area 7 of the monkey. Brain Res 169(3):561–564
Kagerer F, Contreras-Vidal J (2009) Adaptation of sound localization induced by rotated visual feedback in reaching movements. Exp Brain Res 193:315–321
Klassen J, Tong C, Flanagan JR (2005) Learning and recall of incremental kinematic and dynamic sensorimotor transformations. Exp Brain Res 164(2):250–259
Molier BI, Prange GB, Buurke JH (2011) The role of visual feedback in conventional therapy and future research. In: Proceedings of ICVR2011, Zurich, Switzerland
Orban de Xivry JJ, Criscimagna-Hemminger SE, Shadmehr R (2011) Contributions of the motor cortex to adaptive control of reaching depend on the perturbation schedule. Cereb Cortex 21(7):1475–1484
Pouget A, Deneve S, Duhamel J (2002) A computational perspective on the neural basis of multisensory spatial representations. Nat Rev Neurosci 3(9):741–747
Proulx MJ, Stoerig P, Ludowig E, Knoll I (2008) Seeing ‘where’ through the ears: effects of learning-by-doing and long-term sensory deprivation on localization based on image-to-sound substitution. PLoS ONE 3(3):e1840
Rath M, Rocchesso D (2005) Continuous sonic feedback from a rolling ball. IEEE Multimedia 12(2):60–69
Reich L, Maidenbaum S, Amedi A (2012) The brain as a flexible task machine: implications for visual rehabilitation using noninvasive vs. invasive approaches. Curr Opin Neurol 25(1):86–95
Reuschel J, Rosler F, Henriques DYP, Fiehler K (2011) Testing the limits of optimal integration of visual and proprioceptive information of path trajectory. Exp Brain Res 209(4):619–630
Robertson JVG, Hoellinger T, Lindberg P, Bensmail D, Hanneton S, Roby-Brami A (2009) Effect of auditory feedback differs according to side of hemiparesis: a comparative pilot study. J NeuroEng Rehabil 6:45–56
Scheidt RA, Reinkensmeyer DJ, Conditt MA, Rymer WZ, Mussa-Ivaldi FA (2000) Persistence of motor adaptation during constrained, multi-joint, arm movements. J Neurophysiol 84(2):853–862
Scheidt RA, Conditt MA, Secco EL, Mussa-Ivaldi FA (2005) Interaction of visual and proprioceptive feedback during adaptation of human reaching movements. J Neurophysiol 93(6):3200–3213
Schroeder CE, Smiley J, Fu KG, Mcginnis T (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50(1–2):5–17
Secoli R, Rosati G, Reinkensmeyer DJ (2009) Using sound feedback to counteract visual distractor during robot-assisted movement training. In: Proceedings of HAVE2009, Lecco, Italy
Secoli R, Milot MH, Rosati G, Reinkensmeyer DJ (2011) Effect of visual distraction and auditory feedback on patient effort during robot-assisted movement training after stroke. J NeuroEng Rehabil 8(1):21–30
Shadmehr R, Mussa-Ivaldi FA (1994) Adaptive representation of dynamics during learning of a motor task. J Neurosci 14(5 pt 2):3208–3224
Takahashi CD, Nemet D, Rose-Gottron CM, Larson JK, Cooper DM, Reinkensmeyer DJ (2006) Effect of muscle fatigue on internal model formation and retention during reaching with the arm. J Appl Physiol 100(2):695–706
Taylor JA, Thoroughman KA (2007) Divided attention impairs human motor adaptation but not feedback control. J Neurophysiol 98(1):317–326
Thielman G (2010) Rehabilitation of reaching poststroke: a randomized pilot investigation of tactile versus auditory feedback for trunk control. J Neurol Phys Ther 34(3):138–144
Thoroughman KA, Shadmehr R (2000) Learning of action through adaptive combination of motor primitives. Stroke 407:742–747
Wolpert DM, Ghahramani Z, Jordan M (1995) Are arm trajectories planned in kinematic or dynamic coordinates? An adaptation study. Exp Brain Res 103(3):460–470
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Oscari, F., Secoli, R., Avanzini, F. et al. Substituting auditory for visual feedback to adapt to altered dynamic and kinematic environments during reaching. Exp Brain Res 221, 33–41 (2012). https://doi.org/10.1007/s00221-012-3144-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-012-3144-2