MediaSync pp 211-228 | Cite as

Perceiving, Interacting and Playing with Multimedia Delays

  • Ragnhild Eg
  • Kjetil Raaen


Just like interactions with the physical world, humans prefer to interact with computers without noticeable delay. However, in the digital world, the time between cause and effect can far exceed what we are accustomed to in real life. System processes, network transmission and rendering all add to the delay between an input action and an output response. While we expect and accept a computer system to process a request within a period of time, the duration of this period is dependent on the nature of the task. Highly interactive tasks, such as control systems, computer games, design software and even word processing have stringent temporal requirements, where any sustained delay can be detrimental to performance. The research interest for the human capability to operate with delay has grown over the past decades, and it has grown in at least three separate fields. In this chapter, we review relevant work from cognitive psychology, human-computer interaction and multimedia research.


Human-computer interaction Motor-visual interaction Delay 


  1. 1.
    Beigbeder, T., Coughlan, R., Lusher, C., Plunkett, J., Agu, E., Claypool, M.: The effects of loss and latency on user performance in unreal tournament 2003. In: Proceedings of 3rd Workshop on Network and system support for games (NetGames), pp. 144–151 (2004)Google Scholar
  2. 2.
    Chen, K.T., Lei, C.L.: Are all games equally cloud-gaming-friendly? an electromyographic approach. In: 11th Annual Workshop on Network and Systems Support for Games (NetGames), pp. 1–6 (2012)Google Scholar
  3. 3.
    Chen, K.T, Huang, P., Wang, G.S., Huang, C.Y., Lei, C.L.: On the sensitivity of online game playing time to network Qos. Proceedings of IEEE INFOCOM (2006)Google Scholar
  4. 4.
    Claypool, M.: The effect of latency on user performance in real-time strategy games. Comput. Netw. 49(1), 52–70 (2005)CrossRefGoogle Scholar
  5. 5.
    Claypool, M., Claypool, K.: Latency and player interaction in online games. Commun. ACM 49(11), 40–45 (2006)CrossRefGoogle Scholar
  6. 6.
    Claypool, M., Eg, R., Raaen, K.: Modeling user performance for moving target selection with a delayed mouse. In: Amsaleg, L.,Gudmundsson, G., Gurrin, C., Jonsson, B., Satoh, S. (eds.). Lecture Notes in Computer Science, pp. 226–237. Springer (2017)Google Scholar
  7. 7.
    Cunningham, D.W., Billock, V.A., Tsou, B.H.: Sensorimotor adaptation to violations of temporal contiguity. Psychol. Sci. 12(6), 532–535 (2001)CrossRefGoogle Scholar
  8. 8.
    Davis, S., Nesbitt, K., Nalivaiko, E.: A systematic review of cybersickness. In: Proceedings of the Conference on Interactive Entertainment (2014)Google Scholar
  9. 9.
    Dick, M., Wellnitz, O., Wolf, L.: Analysis of factors affecting players’ performance and perception in multiplayer games. In: Proceedings of 4th Workshop on Network and System Support for Games (NetGames), pp. 1–7 (2005)Google Scholar
  10. 10.
    Eagleman, D.M., Holcombe, A.O.: Causation and the perception of time. Trends Cogn. Sci. 6(8), 323–325 (2002)CrossRefGoogle Scholar
  11. 11.
    Ebert, J.P., Wegner, D.M.: Time warp: authorship shapes the perceived timing of actions and events. Conscious. Cogn. 19(1), 481–489 (2010)CrossRefGoogle Scholar
  12. 12.
    Eg, R., Raaen, K.: How to demonstrate delay? let’s play! In: Proceedings of the 7th International Conference on Multimedia Systems, MMSys ’16, pp. 1–28 (2016)Google Scholar
  13. 13.
    Fritsch, T., Ritter, H., Schiller, J.: The effect of latency and network limitations on MMORPGs: a field study of everquest2. In: Proceedings of 4th workshop on Network and system support for games (NetGames) (2005)Google Scholar
  14. 14.
    Fujisaki, W., Nishida, S.: Audio-tactile superiority over visuo-tactile and audio-visual combinations in the temporal resolution of synchrony perception. Exp. Brain Res. 198(2–3), 245–259 (2009)CrossRefGoogle Scholar
  15. 15.
    Haggard, P., Chambon, V.: Sense of agency. Curr. Biol. 22(10), R390–R392 (2012)CrossRefGoogle Scholar
  16. 16.
    Heron, J., Hanson, J.V.M., Whitaker, D.: Effect before cause: supramodal recalibration of sensorimotor timing. PLoS One 4(11), e7681 (2009)CrossRefGoogle Scholar
  17. 17.
    International Telecommunication Union (ITU-T) (2014) Methods for the subjective assessment of small impairments in audio systems bs series. Recommendation BS1116-2 2Google Scholar
  18. 18.
    Jota, R., Ng, A., Dietz, P., Wigdor, D.: How fast is fast enough? a study of the effects of latency in direct-touch pointing tasks. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2291–2300 (2013)Google Scholar
  19. 19.
    Libet, B.: Unconscious cerebral initiative and the role of conscious will in voluntary action. Behav. Brain Sci. 8(04), 529–539 (1985)CrossRefGoogle Scholar
  20. 20.
    Limerick, H., Coyle, D., Moore, J.W.: The experience of agency in human-computer interactions: a review. Front. Hum. Neurosci. 8(3), 643 (2014)Google Scholar
  21. 21.
    Metcalfe, J., Eich, T.S., Castel, A.D.: Metacognition of agency across the lifespan. Cognition 116(2), 267–282 (2010)CrossRefGoogle Scholar
  22. 22.
    Moore, J.W., Obhi, S.S.: Intentional binding and the sense agency: a review. Conscious. Cogn. 21, 546–561 (2012)CrossRefGoogle Scholar
  23. 23.
    Moore, J.W., Lagnado, D., Deal, D.C., Haggard, P.: Feelings of control: contingency determines experience of action. Cognition 110(2), 279–283 (2009)CrossRefGoogle Scholar
  24. 24.
    Nichols, J., Claypool, M.: The effects of latency on online madden NFL football. In: Proceedings of the 14th International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), pp 146–151 (2004)Google Scholar
  25. 25.
    Noel, J., Wallace, M., Orchard-Mills, E., Alais, D., van der Burg, E.: True and perceived synchrony are preferentially associated with particular sensory pairings. Sci. Rep. 5 (2015)Google Scholar
  26. 26.
    Occelli, V., Spence, C., Zampini, M.: Audiotactile interactions in temporal perception. Psychon. Bull. Rev. 18(3), 429–454 (2011)CrossRefGoogle Scholar
  27. 27.
    Pantel, L., Wolf, L.C.: On the impact of delay on real-time multiplayer games. In: Proceedings of the 12th International Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV ’02, pp. 23–29 (2002)Google Scholar
  28. 28.
    Quax, P., Monsieurs, P., Lamotte, W., De Vleeschauwer, D., Degrande, N.: Objective and subjective evaluation of the influence of small amounts of delay and jitter on a recent first person shooter game. In: Proceedings of 3rd Workshop on Network and System Support For Games (NetGames), pp. 152–156 (2004)Google Scholar
  29. 29.
    Raaen, K., Eg, R.: Instantaneous human-computer interactions : button causes and screen effects. Int. Conf. Hum. Comput. Interact. 2015, 1–12 (2015)Google Scholar
  30. 30.
    Raaen, K., Eg, R., Griwodz, C.: Can gamers detect cloud delay? In: Proceedings of the 13th Annual Workshop on Network and Systems Support for Games (NetGames), vol. 200 (2014)Google Scholar
  31. 31.
    Rohde, M., Ernst, M.O.: To lead and to lag-forward and backward recalibration of perceived visuo-motor simultaneity. Front. Psychol. 3, 1–8 (2013)CrossRefGoogle Scholar
  32. 32.
    Rohde, M., Scheller, M., Ernst, M.O.: Effects can precede their cause in the sense of agency. Neuropsychologia 65, 191–196 (2014)CrossRefGoogle Scholar
  33. 33.
    Seow, S.C.: Designing and Engineering Time. Addison-Wesley (2008)Google Scholar
  34. 34.
    Sheldon, N., Girard, E., Borg, S., Claypool, M., Agu, E.: The effect of latency on user performance in warcraft iii. In: Proceedings of the 2nd Workshop on Network and System Support for Games (NetGames), pp. 3–14 (2003)Google Scholar
  35. 35.
    Sheridan T, Ferrell W (1963) Remote manipulative control with transmission delay. IEEE Trans. Hum. Factors Electro. HFE 4(1), 25–29Google Scholar
  36. 36.
    Spence, C.: Crossmodal correspondences: a tutorial review. Atten. Percept. Psychophys. 73(4), 971–995 (2011)CrossRefGoogle Scholar
  37. 37.
    Spence, C., Baddeley, R., Zampini, M., James, R., Shore, D.I.: Multisensory temporal order judgments: when two locations are better than one. Percept. Psychophys. 65(2), 318–328 (2003)CrossRefGoogle Scholar
  38. 38.
    Stetson, C., Cui, X., Montague, P.R., Eagleman, D.M.: Motor-sensory recalibration leads to an illusory reversal of action and sensation. Neuron 51(5), 651–659 (2006)CrossRefGoogle Scholar
  39. 39.
    Stuckel, D., Gutwin, C.: The effects of local lag on tightly-coupled interaction in distributed groupware. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW), pp. 447–456 (2008)Google Scholar
  40. 40.
    Sugano, Y., Keetels, M., Vroomen, J.: Adaptation to motor-visual and motor-auditory temporal lags transfer across modalities. Exp. Brain Res. 201(3), 393–399 (2010)CrossRefGoogle Scholar
  41. 41.
    Weber, F., Haering, C., Thomaschke, R.: Improving the human computer dialogue with increased temporal predictability. Hum. Factors J. Hum. Factors Ergonomics 55, 881–892 (2013)CrossRefGoogle Scholar
  42. 42.
    Welch, R.B., Warren, D.H.: Immediate perceptual response to intersensory discrepancy. Psychol. Bull. 88(3), 638–667 (1980)CrossRefGoogle Scholar
  43. 43.
    Winter, R., Harrar, V., Gozdzik, M., Harris, L.R.: The relative timing of active and passive touch. Brain Res. 1242, 54–58 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Westerdals – Oslo School of Art, Communication and TechnologyOsloNorway

Personalised recommendations