Skip to main content

Advertisement

SpringerLink
Log in

Investigating three types of continuous auditory feedback in visuo-manual tracking

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

The use of continuous auditory feedback for motor control and learning is still understudied and deserves more attention regarding fundamental mechanisms and applications. This paper presents the results of three experiments studying the contribution of task-, error-, and user-related sonification to visuo-manual tracking and assessing its benefits on sensorimotor learning. First results show that sonification can help decreasing the tracking error, as well as increasing the energy in participant’s movement. In the second experiment, when alternating feedback presence, the user-related sonification did not show feedback dependency effects, contrary to the error and task-related feedback. In the third experiment, a reduced exposure of 50% diminished the positive effect of sonification on performance, whereas the increase of the average energy with sound was still significant. In a retention test performed on the next day without auditory feedback, movement energy was still superior for the groups previously trained with the feedback. Although performance was not affected by sound, a learning effect was measurable in both sessions and the user-related group improved its performance also in the retention test. These results confirm that a continuous auditory feedback can be beneficial for movement training and also show an interesting effect of sonification on movement energy. User-related sonification can prevent feedback dependency and increase retention. Consequently, sonification of the user’s own motion appears as a promising solution to support movement learning with interactive feedback.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Avanzini F, De Götzen A, Spagnol S, Rodà A (2009) Integrating auditory feedback in motor rehabilitation systems. In: Proceedings of international conference on multimodal interfaces for skills transfer (SKILLS09)

  • Bidet-Caulet A, Voisin J, Bertrand O, Fonlupt P (2005) Listening to a walking human activates the temporal biological motion area. Neuroimage 28:132–139

    Article  PubMed  Google Scholar 

  • Boyer EO, Pyanet Q, Hanneton S, Bevilacqua F (2014) Learning movement kinematics with a targeted sound. In: Aramaki M, Derrien O, Kronland-Martinet R, Ystad S (eds) Sound, music & motion, lecture notes in computer science, vol 8905. Springer, New York, pp 218–233

  • Boyer EO, Vandervoorde L, Bevilacqua F, Hanneton S (2015) Touching sounds: audio virtual surfaces. In: 2015 IEEE 2nd VR workshop on sonic interactions for virtual environments (SIVE). IEEE, pp 1–5

  • Buchanan JJ, Wang C (2012) Overcoming the guidance effect in motor skill learning: feedback all the time can be beneficial. Exp Brain Res 219(2):305–320

    Article  PubMed  Google Scholar 

  • Craik K (1947) Theory of the human operator in control systems. I. The operator as an engineering system. Br J Psychol 38(2):56–61

    CAS  Google Scholar 

  • Darainy M, Vahdat S, Ostry DJ (2013) Perceptual learning in sensorimotor adaptation. J Neurophysiol 110(9):2152–2162

    Article  PubMed  PubMed Central  Google Scholar 

  • Dozza M, Chiari L, Horak FB (2005) Audio-biofeedback improves balance in patients with bilateral vestibular loss. Arch Phys Med Rehabil 86:1401–1403

    Article  PubMed  Google Scholar 

  • Effenberg AO, Fehse U, Schmitz G, Krueger B, Mechling H (2016) Movement sonification: effects on motor learning beyond rhythmic adjustments. Front Neurosci 10(219)

  • Effenberg AO (2004) Using sonification to enhance perception and reproduction accuracy of human movement patterns. In: Proceedings of the international workshop on interactive sonification, Bielefeld

  • Françoise J, Schnell N, Borghesi R, Bevilacqua F (2014) Probabilistic models for designing motion and sound relationships. In: Proceedings of the 2014 international conference on new interfaces for musical expression, pp 287–292

  • Frassinetti F, Bolognini N, Làdavas E (2002) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332–343

    Article  PubMed  Google Scholar 

  • Gehring WJ, Fencsik DE (2001) Functions of the medial frontal cortex in the processing of conflict and errors. J Neurosci 21(23):9430–9437

    CAS  PubMed  Google Scholar 

  • Giard M, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490

    Article  CAS  PubMed  Google Scholar 

  • Hanneton S, Berthoz A, Droulez J, Slotine J (1997) Does the brain use sliding variables for the control of movements? Biol Cybern 77(6):381–393

    Article  CAS  PubMed  Google Scholar 

  • Hartveld A, Hegarty J (1996) Augmented feedback and physiotherapy practice. Physiotherapy 82(8):480–490

    Article  Google Scholar 

  • Huang C-T, Hwang I-S (2012) Eye-hand synergy and intermittent behaviors during target-directed tracking with visual and non-visual information. PLoS ONE 7(12):e51417

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Kagerer FA, Contreras-Vidal JL (2009) Adaptation of sound localization induced by rotated visual feedback in reaching movements. Exp Brain Res 193(2):315–321

    Article  PubMed  Google Scholar 

  • Krakauer JW, Mazzoni P (2011) Human sensorimotor learning: adaptation, skill, and beyond. Curr Opin Neurobiol 21(4):636–644

    Article  CAS  PubMed  Google Scholar 

  • McRuer D (1980) Human dynamics in man-machine systems. Automatica 16:237–253

    Article  Google Scholar 

  • Miall RC, Weir DJ, Stein JF (1993) Intermittency in human manual tracking tasks. J Motor Behav 25(1):53–63

    Article  CAS  Google Scholar 

  • Miall RC, Wolpert DM (1996) Forward models for physiological motor control. Neural Netw 9(8):1265–1279

    Article  PubMed  Google Scholar 

  • Neilson P, Neilson M, O’Dwyer N (1988) Internal models and intermittency: a theoretical account of human tracking behavior. Biol Cybern 58:101–112

    Article  CAS  PubMed  Google Scholar 

  • Orban de Xivry J-J, Lefèvre P (2007) Saccades and pursuit: two outcomes of a single sensorimotor process. J Physiol 548(1):11–23

    Article  Google Scholar 

  • Rath M, Schleicher R (2008) On the relevance of auditory feedback for quality of control in a balancing task. Acta Acust United Acust 94:12–20

    Article  Google Scholar 

  • Robertson JVG, Hoellinger T, Lindberg P, Bensmail D, Hanneton S, Roby-Brami A (2009) Effect of auditory feedback differs according to side of hemiparesis: a comparative pilot study. J Neuroeng Rehabil 6:45

    Article  PubMed  PubMed Central  Google Scholar 

  • Ronsse R, Puttemans V, Coxon JP, Goble DJ, Wagemans J, Wenderoth N, Swinnen SP (2011) Motor learning with augmented feedback: modality-dependent behavioral and neural consequences. Cereb Cortex (New York, NY 1991) 21(6):1283–1294

    Article  Google Scholar 

  • Rosati G, Oscari F, Spagnol S, Avanzini F, Masiero S (2012) Effect of task-related continuous auditory feedback during learning of tracking motion exercises. J Neuroeng Rehabil 9(1):79

    Article  PubMed  PubMed Central  Google Scholar 

  • Rosenkranz K, Rothwell JC (2012) Modulation of proprioceptive integration in the motor cortex shapes human motor learning. J Neurosci 32(26):9000–9006

    Article  CAS  PubMed  Google Scholar 

  • Salmoni AW, Schmidt RA, Walter CB (1984) Knowledge of results and motor learning : a review and critical reappraisal. Psychol Bull 95(3):355–386

    Article  CAS  PubMed  Google Scholar 

  • Schmitz G, Effenberg AO (2012) Perceptual effects of auditory information about own and other movements. In: Proceedings of the 18th international conference on auditory display, pp 89–94

  • Seitz AR, Kim R, Shams L (2006) Sound facilitates visual learning. Curr Biol CB 16(14):1422–1427

    Article  CAS  PubMed  Google Scholar 

  • Shams L, Seitz AR (2008) Benefits of multisensory learning. Trends Cogn Sci 12(11):411–417

    Article  PubMed  Google Scholar 

  • Sigrist R, Rauter G, Riener R, Wolf P (2013) Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon Bull Rev 20(1):21–53

    Article  PubMed  Google Scholar 

  • Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P (2015) Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Exp Brain Res 233(3):909–925

    Article  PubMed  Google Scholar 

  • Tourville J, Reilly K, Guenther F (2008) Neural mechanisms underlying auditory feedback control of speech. Neuroimage 39(3):1429–1443

    Article  PubMed  Google Scholar 

  • van Vliet PM, Wulf G (2006) Extrinsic feedback for motor learning after stroke: what is the evidence? Disabil Rehabil 28(13–14):831–840

    Article  PubMed  Google Scholar 

  • Vogt K, Pirro D, Kobenz I, Höldrich R, Eckel G (2009) Physiosonic—movement sonification as auditory feedback. In: 6th international symposium, CMMR/ICAD 2009, Copenhagen, pp 1–7

  • Vroomen J, de Gelder B (2000) Sound enhances visual perception: cross-modal effects of auditory organization on vision. J Exp Psychol Hum Percept Perform 26:1583–1590

    Article  CAS  PubMed  Google Scholar 

  • Winstein CJ, Schmidt RA (1990) Reduced frequency of knowledge of results enhances motor skill learning. J Exp Psychol Learn Mem Cogn 16(4):677–691

    Article  Google Scholar 

  • Wolpert DM, Ghahramani Z (2000) Computational principles of movement neuroscience. Nat Neurosci 3(Suppl. November):1212–1217

    Article  CAS  PubMed  Google Scholar 

  • Wong JD, Da Kistemaker, Chin A, Gribble PL (2012) Can proprioceptive training improve motor learning? J Neurophysiol 108(12):3313–3321

    Article  PubMed  PubMed Central  Google Scholar 

  • Zhou Y, Liu Y, Lu H, Wu S, Zhang M (2016) Neuronal representation of saccadic error in macaque posterior parietal cortex (PPC). eLife 5:e10912

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This work has been funded by ANR French National Research Agency, under the ANR-Blanc program 2011 (LEGOS project ANR-11-BS02-012) and additional support from Cap Digital.

Author information

Authors and Affiliations

  1. Ircam - STMS CNRS UPMC, 1 place Igor Stravinsky, 75004, Paris, France

    Éric O. Boyer, Frédéric Bevilacqua & Patrick Susini

  2. Laboratoire de Psychologie de la Perception UMR CNRS 8242, Université Paris Descartes, 45 rue des Saints Pères, 75006, Paris, France

    Éric O. Boyer & Sylvain Hanneton

Authors
  1. Éric O. Boyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frédéric Bevilacqua
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick Susini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sylvain Hanneton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Éric O. Boyer.

Ethics declarations

Ethical standards

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 912 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boyer, É.O., Bevilacqua, F., Susini, P. et al. Investigating three types of continuous auditory feedback in visuo-manual tracking. Exp Brain Res 235, 691–701 (2017). https://doi.org/10.1007/s00221-016-4827-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-016-4827-x

Keywords

Search

Navigation