Advertisement

Experimental Brain Research

, Volume 236, Issue 12, pp 3191–3201 | Cite as

Adaptation to visual feedback delays on touchscreens with hand vision

  • Elie Cattan
  • Pascal Perrier
  • François Bérard
  • Silvain Gerber
  • Amélie Rochet-Capellan
Research Article
  • 90 Downloads

Abstract

Direct touch finger interaction on a smartphone or a tablet is now ubiquitous. However, the latency inherent in digital computation produces an average feedback delay of ~ 75 ms between the action of the hand and its visible effect on digital content. This delay has been shown to affect users’ performance, but it is unclear whether users adapt to this delay and whether it influences skill learning. Previous work studied adaptation to feedback delays but only for longer delays, with hidden hand or indirect devices. This paper addresses adaptation to touchscreen delay in two empirical studies involving the tracking of a target moving along an elliptical path. Participants were trained for the task either at the minimal delay the system allows (~ 9 ms) or at a longer delay equivalent to commercialized touch devices latencies (75 ms). After 10 training sessions over a minimum of 2 weeks (Experiment 1), participants adapt to the delay. They also display long-term retention 7 weeks after the last training session. This adaptation generalizes to a similar tracking path (e.g., infinity symbol). We also observed generalization of learning from the longer delay to the minimal-delay condition (Experiment 2). The delay thus does not prevent the learning of tracking skill, which suggests that delay adaptation and tracking skill could be two separate components of learning.

Keywords

Motor adaptation Motor learning Visual feedback delay Tracking Direct touch interaction 

Notes

Acknowledgements

This work has been partially supported by the LabEx PERSYVAL-Lab (ANR-11-LABX-0025-01). This work is based on an earlier work: “Does Practice Make Perfect? Learning to Deal with Latency in Direct-Touch Interaction”, in CHI 2017© ACM,  https://doi.org/10.1145/3025453.3025585. The authors also thank Linda Sherwood for her careful proofreading of the manuscript.

References

  1. Bérard F, Blanch R (2013) Two touch system latency estimators: high accuracy and low overhead. In: Proceedings of the 2013 ACM international conference on interactive tabletops and surfaces, ACM, New York, NY, USA, pp 241–250Google Scholar
  2. Bérard F, Rochet-Capellan A (2015) The transfer of learning as HCI similarity: towards an objective assessment of the sensory-motor basis of naturalness. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, ACM, New York, NY, USA, pp 1315–1324Google Scholar
  3. Botzer L, Karniel A (2013) Feedback and feedforward adaptation to visuomotor delay during reaching and slicing movements. Eur J Neurosci 38(1):2108–2123CrossRefGoogle Scholar
  4. Cámara C, de la Malla C, López-Moliner J, Brenner E (2018) Eye movements in interception with delayed visual feedback. Exp Brain Res 236(7):1837–1847CrossRefGoogle Scholar
  5. Cattan E, Rochet-Capellan A, Perrier P, Bérard F (2015) Reducing latency with a continuous prediction: effects on users’ performance in direct-touch target acquisitions. In: Proceedings of the 2015 international conference on interactive tabletops & surfaces, ACM, New York, NY, USA, pp 205–214Google Scholar
  6. Cattan E, Rochet-Capellan A, Perrier P, Bérard F (2017) Does practice make perfect? In: Proceedings of the 2017 CHI conference on human factors in computing systems (pp 5619–5629), ACM, New York, NY, USAGoogle Scholar
  7. Criscimagna-Hemminger SE, Shadmehr R (2008) Consolidation patterns of human motor memory. J Neurosci 28(39):9610–9618CrossRefGoogle Scholar
  8. Cunningham DW, Billock VA, Tsou BH (2001a) Sensorimotor adaptation to violations of temporal contiguity. Psychol Sci 12(6):532–535CrossRefGoogle Scholar
  9. Cunningham DW, Chatziastros A, Von der Heyde M, Bülthoff HH (2001b) Driving in the future: temporal visuomotor adaptation and generalization. J Vis 1(2):88–98CrossRefGoogle Scholar
  10. de la Malla C, López-Moliner J, Brenner E (2014) Dealing with delays does not transfer across sensorimotor tasks. J Vis 14(12):8, 1–17CrossRefGoogle Scholar
  11. Foulkes AJMC, Miall RC (2000) Adaptation to visual feedback delays in a human manual tracking task. Exp Brain Res 131(1):101–110CrossRefGoogle Scholar
  12. Hothorn T, Bretz F, Westfall P (2008) Simultaneous inference in general parametric models. Biom J 50(3):346–363CrossRefGoogle Scholar
  13. Jota R, Ng A, Dietz P, Wigdor D (2013) How fast is fast enough?: a study of the effects of latency in direct-touch pointing tasks. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, New York, NY, USA, pp 2291–2300Google Scholar
  14. Kennedy JS, Buehner MJ, Rushton SK (2009) Adaptation to sensory-motor temporal misalignment: instrumental or perceptual learning? Q J Exp Psychol 62(3):453–469CrossRefGoogle Scholar
  15. Kitago T, Ryan SL, Mazzoni P, Krakauer JW, Haith AM (2013) Unlearning versus savings in visuomotor adaptation: comparing effects of washout, passage of time, and removal of errors on motor memory. Front Hum Neurosci 7:307CrossRefGoogle Scholar
  16. Krakauer JW, Mazzoni P (2011) Human sensorimotor learning: adaptation, skill, and beyond. Curr Opin Neurobiol 21(4):636–644CrossRefGoogle Scholar
  17. Leib R, Karniel A, Mussa-Ivaldi FA (2017) The mechanical representation of temporal delays. Sci Rep 7(1):7669CrossRefGoogle Scholar
  18. Miall RC, Jackson JK (2006) Adaptation to visual feedback delays in manual tracking: evidence against the Smith Predictor model of human visually guided action. Exp Brain Res 172(1):77–84CrossRefGoogle Scholar
  19. Mundry R, Nunn CL (2009) Stepwise model fitting and statistical inference: turning noise into signal pollution. Am Nat 173(1):119–123CrossRefGoogle Scholar
  20. Ng A, Lepinski J, Wigdor D, Sanders S, Dietz P (2012) Designing for low-latency direct-touch input. In: Proceedings of the 25th annual ACM symposium on user interface software and technology, ACM, New York, NY, USA, pp 453–464Google Scholar
  21. Norman DA (2010) The transmedia design challenge: technology that is pleasurable and satisfying. Interactions 17(1):12–15CrossRefGoogle Scholar
  22. Pinheiro J, Bates D, DebRoy S, Sarkar D (2007) Linear and nonlinear mixed effects models. R package version. 3: 57. R Foundation for Statistical Computing, ViennaGoogle Scholar
  23. R Development Core Team (2016) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna (ISBN 3-900051-07-0) Google Scholar
  24. Rohde M, Ernst MO (2016) Time, agency, and sensory feedback delays during action. Curr Opin Behav Sci 8:193–199CrossRefGoogle Scholar
  25. Rohde M, van Dam LC, Ernst MO (2014) Predictability is necessary for closed-loop visual feedback delay adaptation. J Vis 14(3):4, 1–23CrossRefGoogle Scholar
  26. Sarlegna FR, Baud-Bovy G, Danion F (2010) Delayed visual feedback affects both manual tracking and grip force control when transporting a handheld object. J Neurophysiol 104(2):641–653CrossRefGoogle Scholar
  27. Shadmehr R, Smith MA, Krakauer JW (2010) Error correction, sensory prediction, and adaptation in motor control. Annu Rev Neurosci 33:89–108CrossRefGoogle Scholar
  28. Smith MA, Ghazizadeh A, Shadmehr R (2006) Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol, 4(6):e179CrossRefGoogle Scholar
  29. Stetson C, Cui X, Montague PR, Eagleman DM (2006) Motor-sensory recalibration leads to an illusory reversal of action and sensation. Neuron 51(5):651–659CrossRefGoogle Scholar
  30. Sugano Y, Keetels M, Vroomen J (2012) The build-up and transfer of sensorimotor temporal recalibration measured via a synchronization task. Front Psychol 3:246CrossRefGoogle Scholar
  31. Telgen S, Parvin D, Diedrichsen J (2014) Mirror reversal and visual rotation are learned and consolidated via separate mechanisms: recalibrating or learning de novo? J Neurosci 34(41):13768–13779CrossRefGoogle Scholar
  32. Turnham EJA, Braun DA, Wolpert DM (2011) Inferring visuomotor priors for sensorimotor learning. PLoS Comput Biol 7(3):e1001112CrossRefGoogle Scholar
  33. Viviani P, Terzuolo C (1982) Trajectory determines movement dynamics. Neuroscience 7(2):431–437CrossRefGoogle Scholar
  34. Viviani P, Campadelli P, Mounoud P (1987) Visuo-manual pursuit tracking of human two-dimensional movements. J Exp Psychol Hum Percept Perform 13(1):62–78CrossRefGoogle Scholar
  35. Von Helmholtz H (1867) Handbuch der physiologischen Optik. In: Gustav Karsten (ed) Allgemeine Encyklopädie der Physik, vol 9. Leopold Voss, LeipzigGoogle Scholar
  36. Wei K, Yan X, Kong G, Yin C, Zhang F, Wang Q, Kording KP (2014) Computer use changes generalization of movement learning. Curr Biol 24(1):82–85CrossRefGoogle Scholar
  37. Wigdor D, Wixon D (2011) Brave NUI world: designing natural user interfaces for touch and gesture. Morgan Kaufmann Publishers, BurlingtonGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Univ. Grenoble Alpes, CNRS, Grenoble-INP, LIGGrenobleFrance
  2. 2.Univ. Grenoble Alpes, CNRS, Grenoble-INP, Gipsa-LabGrenobleFrance

Personalised recommendations