Experimental Brain Research

, Volume 235, Issue 3, pp 691–701

Investigating three types of continuous auditory feedback in visuo-manual tracking

  • Éric O. Boyer
  • Frédéric Bevilacqua
  • Patrick Susini
  • Sylvain Hanneton
Research Article

Abstract

The use of continuous auditory feedback for motor control and learning is still understudied and deserves more attention regarding fundamental mechanisms and applications. This paper presents the results of three experiments studying the contribution of task-, error-, and user-related sonification to visuo-manual tracking and assessing its benefits on sensorimotor learning. First results show that sonification can help decreasing the tracking error, as well as increasing the energy in participant’s movement. In the second experiment, when alternating feedback presence, the user-related sonification did not show feedback dependency effects, contrary to the error and task-related feedback. In the third experiment, a reduced exposure of 50% diminished the positive effect of sonification on performance, whereas the increase of the average energy with sound was still significant. In a retention test performed on the next day without auditory feedback, movement energy was still superior for the groups previously trained with the feedback. Although performance was not affected by sound, a learning effect was measurable in both sessions and the user-related group improved its performance also in the retention test. These results confirm that a continuous auditory feedback can be beneficial for movement training and also show an interesting effect of sonification on movement energy. User-related sonification can prevent feedback dependency and increase retention. Consequently, sonification of the user’s own motion appears as a promising solution to support movement learning with interactive feedback.

Keywords

Tracking Auditory feedback Sensorimotor learning Sound Interaction 

Supplementary material

Supplementary material 1 (mp4 912 KB)

References

  1. Avanzini F, De Götzen A, Spagnol S, Rodà A (2009) Integrating auditory feedback in motor rehabilitation systems. In: Proceedings of international conference on multimodal interfaces for skills transfer (SKILLS09)Google Scholar
  2. Bidet-Caulet A, Voisin J, Bertrand O, Fonlupt P (2005) Listening to a walking human activates the temporal biological motion area. Neuroimage 28:132–139CrossRefPubMedGoogle Scholar
  3. Boyer EO, Pyanet Q, Hanneton S, Bevilacqua F (2014) Learning movement kinematics with a targeted sound. In: Aramaki M, Derrien O, Kronland-Martinet R, Ystad S (eds) Sound, music & motion, lecture notes in computer science, vol 8905. Springer, New York, pp 218–233Google Scholar
  4. Boyer EO, Vandervoorde L, Bevilacqua F, Hanneton S (2015) Touching sounds: audio virtual surfaces. In: 2015 IEEE 2nd VR workshop on sonic interactions for virtual environments (SIVE). IEEE, pp 1–5Google Scholar
  5. Buchanan JJ, Wang C (2012) Overcoming the guidance effect in motor skill learning: feedback all the time can be beneficial. Exp Brain Res 219(2):305–320CrossRefPubMedGoogle Scholar
  6. Craik K (1947) Theory of the human operator in control systems. I. The operator as an engineering system. Br J Psychol 38(2):56–61Google Scholar
  7. Darainy M, Vahdat S, Ostry DJ (2013) Perceptual learning in sensorimotor adaptation. J Neurophysiol 110(9):2152–2162CrossRefPubMedPubMedCentralGoogle Scholar
  8. Dozza M, Chiari L, Horak FB (2005) Audio-biofeedback improves balance in patients with bilateral vestibular loss. Arch Phys Med Rehabil 86:1401–1403CrossRefPubMedGoogle Scholar
  9. Effenberg AO, Fehse U, Schmitz G, Krueger B, Mechling H (2016) Movement sonification: effects on motor learning beyond rhythmic adjustments. Front Neurosci 10(219)Google Scholar
  10. Effenberg AO (2004) Using sonification to enhance perception and reproduction accuracy of human movement patterns. In: Proceedings of the international workshop on interactive sonification, BielefeldGoogle Scholar
  11. Françoise J, Schnell N, Borghesi R, Bevilacqua F (2014) Probabilistic models for designing motion and sound relationships. In: Proceedings of the 2014 international conference on new interfaces for musical expression, pp 287–292Google Scholar
  12. Frassinetti F, Bolognini N, Làdavas E (2002) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332–343CrossRefPubMedGoogle Scholar
  13. Gehring WJ, Fencsik DE (2001) Functions of the medial frontal cortex in the processing of conflict and errors. J Neurosci 21(23):9430–9437PubMedGoogle Scholar
  14. Giard M, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490CrossRefPubMedGoogle Scholar
  15. Hanneton S, Berthoz A, Droulez J, Slotine J (1997) Does the brain use sliding variables for the control of movements? Biol Cybern 77(6):381–393CrossRefPubMedGoogle Scholar
  16. Hartveld A, Hegarty J (1996) Augmented feedback and physiotherapy practice. Physiotherapy 82(8):480–490CrossRefGoogle Scholar
  17. Huang C-T, Hwang I-S (2012) Eye-hand synergy and intermittent behaviors during target-directed tracking with visual and non-visual information. PLoS ONE 7(12):e51417CrossRefPubMedPubMedCentralGoogle Scholar
  18. Kagerer FA, Contreras-Vidal JL (2009) Adaptation of sound localization induced by rotated visual feedback in reaching movements. Exp Brain Res 193(2):315–321CrossRefPubMedGoogle Scholar
  19. Krakauer JW, Mazzoni P (2011) Human sensorimotor learning: adaptation, skill, and beyond. Curr Opin Neurobiol 21(4):636–644CrossRefPubMedGoogle Scholar
  20. McRuer D (1980) Human dynamics in man-machine systems. Automatica 16:237–253CrossRefGoogle Scholar
  21. Miall RC, Weir DJ, Stein JF (1993) Intermittency in human manual tracking tasks. J Motor Behav 25(1):53–63CrossRefGoogle Scholar
  22. Miall RC, Wolpert DM (1996) Forward models for physiological motor control. Neural Netw 9(8):1265–1279CrossRefPubMedGoogle Scholar
  23. Neilson P, Neilson M, O’Dwyer N (1988) Internal models and intermittency: a theoretical account of human tracking behavior. Biol Cybern 58:101–112CrossRefPubMedGoogle Scholar
  24. Orban de Xivry J-J, Lefèvre P (2007) Saccades and pursuit: two outcomes of a single sensorimotor process. J Physiol 548(1):11–23CrossRefGoogle Scholar
  25. Rath M, Schleicher R (2008) On the relevance of auditory feedback for quality of control in a balancing task. Acta Acust United Acust 94:12–20CrossRefGoogle Scholar
  26. Robertson JVG, Hoellinger T, Lindberg P, Bensmail D, Hanneton S, Roby-Brami A (2009) Effect of auditory feedback differs according to side of hemiparesis: a comparative pilot study. J Neuroeng Rehabil 6:45CrossRefPubMedPubMedCentralGoogle Scholar
  27. Ronsse R, Puttemans V, Coxon JP, Goble DJ, Wagemans J, Wenderoth N, Swinnen SP (2011) Motor learning with augmented feedback: modality-dependent behavioral and neural consequences. Cereb Cortex (New York, NY 1991) 21(6):1283–1294CrossRefGoogle Scholar
  28. Rosati G, Oscari F, Spagnol S, Avanzini F, Masiero S (2012) Effect of task-related continuous auditory feedback during learning of tracking motion exercises. J Neuroeng Rehabil 9(1):79CrossRefPubMedPubMedCentralGoogle Scholar
  29. Rosenkranz K, Rothwell JC (2012) Modulation of proprioceptive integration in the motor cortex shapes human motor learning. J Neurosci 32(26):9000–9006CrossRefPubMedGoogle Scholar
  30. Salmoni AW, Schmidt RA, Walter CB (1984) Knowledge of results and motor learning : a review and critical reappraisal. Psychol Bull 95(3):355–386CrossRefPubMedGoogle Scholar
  31. Schmitz G, Effenberg AO (2012) Perceptual effects of auditory information about own and other movements. In: Proceedings of the 18th international conference on auditory display, pp 89–94Google Scholar
  32. Seitz AR, Kim R, Shams L (2006) Sound facilitates visual learning. Curr Biol CB 16(14):1422–1427CrossRefPubMedGoogle Scholar
  33. Shams L, Seitz AR (2008) Benefits of multisensory learning. Trends Cogn Sci 12(11):411–417CrossRefPubMedGoogle Scholar
  34. Sigrist R, Rauter G, Riener R, Wolf P (2013) Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon Bull Rev 20(1):21–53CrossRefPubMedGoogle Scholar
  35. Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P (2015) Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Exp Brain Res 233(3):909–925CrossRefPubMedGoogle Scholar
  36. Tourville J, Reilly K, Guenther F (2008) Neural mechanisms underlying auditory feedback control of speech. Neuroimage 39(3):1429–1443CrossRefPubMedGoogle Scholar
  37. van Vliet PM, Wulf G (2006) Extrinsic feedback for motor learning after stroke: what is the evidence? Disabil Rehabil 28(13–14):831–840CrossRefPubMedGoogle Scholar
  38. Vogt K, Pirro D, Kobenz I, Höldrich R, Eckel G (2009) Physiosonic—movement sonification as auditory feedback. In: 6th international symposium, CMMR/ICAD 2009, Copenhagen, pp 1–7Google Scholar
  39. Vroomen J, de Gelder B (2000) Sound enhances visual perception: cross-modal effects of auditory organization on vision. J Exp Psychol Hum Percept Perform 26:1583–1590CrossRefPubMedGoogle Scholar
  40. Winstein CJ, Schmidt RA (1990) Reduced frequency of knowledge of results enhances motor skill learning. J Exp Psychol Learn Mem Cogn 16(4):677–691CrossRefGoogle Scholar
  41. Wolpert DM, Ghahramani Z (2000) Computational principles of movement neuroscience. Nat Neurosci 3(Suppl. November):1212–1217CrossRefPubMedGoogle Scholar
  42. Wong JD, Da Kistemaker, Chin A, Gribble PL (2012) Can proprioceptive training improve motor learning? J Neurophysiol 108(12):3313–3321CrossRefPubMedPubMedCentralGoogle Scholar
  43. Zhou Y, Liu Y, Lu H, Wu S, Zhang M (2016) Neuronal representation of saccadic error in macaque posterior parietal cortex (PPC). eLife 5:e10912PubMedPubMedCentralGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Éric O. Boyer
    • 1
    • 2
  • Frédéric Bevilacqua
    • 1
  • Patrick Susini
    • 1
  • Sylvain Hanneton
    • 2
  1. 1.Ircam - STMS CNRS UPMCParisFrance
  2. 2.Laboratoire de Psychologie de la Perception UMR CNRS 8242Université Paris DescartesParisFrance

Personalised recommendations