Advertisement

Investigating three types of continuous auditory feedback in visuo-manual tracking

Abstract

The use of continuous auditory feedback for motor control and learning is still understudied and deserves more attention regarding fundamental mechanisms and applications. This paper presents the results of three experiments studying the contribution of task-, error-, and user-related sonification to visuo-manual tracking and assessing its benefits on sensorimotor learning. First results show that sonification can help decreasing the tracking error, as well as increasing the energy in participant’s movement. In the second experiment, when alternating feedback presence, the user-related sonification did not show feedback dependency effects, contrary to the error and task-related feedback. In the third experiment, a reduced exposure of 50% diminished the positive effect of sonification on performance, whereas the increase of the average energy with sound was still significant. In a retention test performed on the next day without auditory feedback, movement energy was still superior for the groups previously trained with the feedback. Although performance was not affected by sound, a learning effect was measurable in both sessions and the user-related group improved its performance also in the retention test. These results confirm that a continuous auditory feedback can be beneficial for movement training and also show an interesting effect of sonification on movement energy. User-related sonification can prevent feedback dependency and increase retention. Consequently, sonification of the user’s own motion appears as a promising solution to support movement learning with interactive feedback.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Avanzini F, De Götzen A, Spagnol S, Rodà A (2009) Integrating auditory feedback in motor rehabilitation systems. In: Proceedings of international conference on multimodal interfaces for skills transfer (SKILLS09)

  2. Bidet-Caulet A, Voisin J, Bertrand O, Fonlupt P (2005) Listening to a walking human activates the temporal biological motion area. Neuroimage 28:132–139

  3. Boyer EO, Pyanet Q, Hanneton S, Bevilacqua F (2014) Learning movement kinematics with a targeted sound. In: Aramaki M, Derrien O, Kronland-Martinet R, Ystad S (eds) Sound, music & motion, lecture notes in computer science, vol 8905. Springer, New York, pp 218–233

  4. Boyer EO, Vandervoorde L, Bevilacqua F, Hanneton S (2015) Touching sounds: audio virtual surfaces. In: 2015 IEEE 2nd VR workshop on sonic interactions for virtual environments (SIVE). IEEE, pp 1–5

  5. Buchanan JJ, Wang C (2012) Overcoming the guidance effect in motor skill learning: feedback all the time can be beneficial. Exp Brain Res 219(2):305–320

  6. Craik K (1947) Theory of the human operator in control systems. I. The operator as an engineering system. Br J Psychol 38(2):56–61

  7. Darainy M, Vahdat S, Ostry DJ (2013) Perceptual learning in sensorimotor adaptation. J Neurophysiol 110(9):2152–2162

  8. Dozza M, Chiari L, Horak FB (2005) Audio-biofeedback improves balance in patients with bilateral vestibular loss. Arch Phys Med Rehabil 86:1401–1403

  9. Effenberg AO, Fehse U, Schmitz G, Krueger B, Mechling H (2016) Movement sonification: effects on motor learning beyond rhythmic adjustments. Front Neurosci 10(219)

  10. Effenberg AO (2004) Using sonification to enhance perception and reproduction accuracy of human movement patterns. In: Proceedings of the international workshop on interactive sonification, Bielefeld

  11. Françoise J, Schnell N, Borghesi R, Bevilacqua F (2014) Probabilistic models for designing motion and sound relationships. In: Proceedings of the 2014 international conference on new interfaces for musical expression, pp 287–292

  12. Frassinetti F, Bolognini N, Làdavas E (2002) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332–343

  13. Gehring WJ, Fencsik DE (2001) Functions of the medial frontal cortex in the processing of conflict and errors. J Neurosci 21(23):9430–9437

  14. Giard M, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490

  15. Hanneton S, Berthoz A, Droulez J, Slotine J (1997) Does the brain use sliding variables for the control of movements? Biol Cybern 77(6):381–393

  16. Hartveld A, Hegarty J (1996) Augmented feedback and physiotherapy practice. Physiotherapy 82(8):480–490

  17. Huang C-T, Hwang I-S (2012) Eye-hand synergy and intermittent behaviors during target-directed tracking with visual and non-visual information. PLoS ONE 7(12):e51417

  18. Kagerer FA, Contreras-Vidal JL (2009) Adaptation of sound localization induced by rotated visual feedback in reaching movements. Exp Brain Res 193(2):315–321

  19. Krakauer JW, Mazzoni P (2011) Human sensorimotor learning: adaptation, skill, and beyond. Curr Opin Neurobiol 21(4):636–644

  20. McRuer D (1980) Human dynamics in man-machine systems. Automatica 16:237–253

  21. Miall RC, Weir DJ, Stein JF (1993) Intermittency in human manual tracking tasks. J Motor Behav 25(1):53–63

  22. Miall RC, Wolpert DM (1996) Forward models for physiological motor control. Neural Netw 9(8):1265–1279

  23. Neilson P, Neilson M, O’Dwyer N (1988) Internal models and intermittency: a theoretical account of human tracking behavior. Biol Cybern 58:101–112

  24. Orban de Xivry J-J, Lefèvre P (2007) Saccades and pursuit: two outcomes of a single sensorimotor process. J Physiol 548(1):11–23

  25. Rath M, Schleicher R (2008) On the relevance of auditory feedback for quality of control in a balancing task. Acta Acust United Acust 94:12–20

  26. Robertson JVG, Hoellinger T, Lindberg P, Bensmail D, Hanneton S, Roby-Brami A (2009) Effect of auditory feedback differs according to side of hemiparesis: a comparative pilot study. J Neuroeng Rehabil 6:45

  27. Ronsse R, Puttemans V, Coxon JP, Goble DJ, Wagemans J, Wenderoth N, Swinnen SP (2011) Motor learning with augmented feedback: modality-dependent behavioral and neural consequences. Cereb Cortex (New York, NY 1991) 21(6):1283–1294

  28. Rosati G, Oscari F, Spagnol S, Avanzini F, Masiero S (2012) Effect of task-related continuous auditory feedback during learning of tracking motion exercises. J Neuroeng Rehabil 9(1):79

  29. Rosenkranz K, Rothwell JC (2012) Modulation of proprioceptive integration in the motor cortex shapes human motor learning. J Neurosci 32(26):9000–9006

  30. Salmoni AW, Schmidt RA, Walter CB (1984) Knowledge of results and motor learning : a review and critical reappraisal. Psychol Bull 95(3):355–386

  31. Schmitz G, Effenberg AO (2012) Perceptual effects of auditory information about own and other movements. In: Proceedings of the 18th international conference on auditory display, pp 89–94

  32. Seitz AR, Kim R, Shams L (2006) Sound facilitates visual learning. Curr Biol CB 16(14):1422–1427

  33. Shams L, Seitz AR (2008) Benefits of multisensory learning. Trends Cogn Sci 12(11):411–417

  34. Sigrist R, Rauter G, Riener R, Wolf P (2013) Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon Bull Rev 20(1):21–53

  35. Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P (2015) Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Exp Brain Res 233(3):909–925

  36. Tourville J, Reilly K, Guenther F (2008) Neural mechanisms underlying auditory feedback control of speech. Neuroimage 39(3):1429–1443

  37. van Vliet PM, Wulf G (2006) Extrinsic feedback for motor learning after stroke: what is the evidence? Disabil Rehabil 28(13–14):831–840

  38. Vogt K, Pirro D, Kobenz I, Höldrich R, Eckel G (2009) Physiosonic—movement sonification as auditory feedback. In: 6th international symposium, CMMR/ICAD 2009, Copenhagen, pp 1–7

  39. Vroomen J, de Gelder B (2000) Sound enhances visual perception: cross-modal effects of auditory organization on vision. J Exp Psychol Hum Percept Perform 26:1583–1590

  40. Winstein CJ, Schmidt RA (1990) Reduced frequency of knowledge of results enhances motor skill learning. J Exp Psychol Learn Mem Cogn 16(4):677–691

  41. Wolpert DM, Ghahramani Z (2000) Computational principles of movement neuroscience. Nat Neurosci 3(Suppl. November):1212–1217

  42. Wong JD, Da Kistemaker, Chin A, Gribble PL (2012) Can proprioceptive training improve motor learning? J Neurophysiol 108(12):3313–3321

  43. Zhou Y, Liu Y, Lu H, Wu S, Zhang M (2016) Neuronal representation of saccadic error in macaque posterior parietal cortex (PPC). eLife 5:e10912

Download references

Acknowledgements

This work has been funded by ANR French National Research Agency, under the ANR-Blanc program 2011 (LEGOS project ANR-11-BS02-012) and additional support from Cap Digital.

Author information

Correspondence to Éric O. Boyer.

Ethics declarations

Ethical standards

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 912 KB)

Supplementary material 1 (mp4 912 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Boyer, É.O., Bevilacqua, F., Susini, P. et al. Investigating three types of continuous auditory feedback in visuo-manual tracking. Exp Brain Res 235, 691–701 (2017). https://doi.org/10.1007/s00221-016-4827-x

Download citation

Keywords

  • Tracking
  • Auditory feedback
  • Sensorimotor learning
  • Sound
  • Interaction