Abstract
One popular application of augmented reality (AR) is the real-time guidance and training in which the AR user receives useful information by a remote expert. For relatively fast-paced tasks, presentation of such guidance in a way that the recipient can make immediate recognition and quick understanding can be an especially challenging problem. In this paper, we present an AR-based tele-coaching system applied to the game of tennis, called the AR coach, and explore for interface design guidelines through a user study. We have evaluated the player’s performance for instruction understanding when the coaching instruction was presented in four different modalities: (1) Visual—visual only, (2) Sound—aural only/mono, (3) 3D Sound—aural only/3D and (4) Multimodal—both visual and aural/mono. Results from the experiment suggested that, among the three, the visual-only augmentation was the most effective and least distracting for the given pace of information transfer (e.g., under every 3 s). We attribute such a result to the characteristic of the visual modality to encode and present a lot of information at once and the human’s limited capability in handling and fusing multimodal information at a relatively fast rate.
Similar content being viewed by others
References
Avery B, Sandor C, Thomas BH (2012) Improving spatial perception for augmented reality x-ray vision. In: Proceedings of the IEEE conference on virtual reality. IEEE, pp 79–82
Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B (2001) Recent advances in augmented reality. Comput Graph Appl IEEE 21(6):34–47
Bolt RA (1980) Put-that-there, voice and gesture at the graphics interface. Comput Graph 14(3):262–270
Borst J, Taatgen N, Rijn H (2010) The problem state: a cognitive bottleneck in multitasking. J Exp Psychol Learn Mem Cogn 36(2):363–382
Byrne M, Anderson J (2001) Serial modules in parallel: the psychological refractory period and perfect time sharing. Psychol Rev 108(4):847–869
Damos D (1991) Dual-task methodology: some common problems. In: Damos D (ed) Multiple-task performance. CRC Press, Boca Raton, pp 101–119
Fuchs H, Livingston MA, Raskar R, State A, Crawford JR, Rademacher R, Drake SH, Meyer AA (1998) Augmented reality visualization for laparoscopic surgery. In: Proceedings of the MICCAI, pp 934–943
Grasso MA, Ebert DS, Finin TW (1998) The integrality of speech in multimodal interfaces. ACM Trans CHI 5(4):303–325
Grohn M, Lokki T, Savioja L, Takala T (2001) Some aspects of role of audio in immersive visualization. Proc SPIE 4302:13–22
Hauptmann A, McAvinney P (1993) Gestures with speech for graphic manipulation. Int J Man Mach Stud 38(2):231–249
Hecht D, Reiner M, Halevy G (2006) Multimodal virtual environments: response times, attention, and presence. Presence Teleoper Virtual Environ 15(5):515–523
Jaimes A, Sebe N (2005) Multimodal human computer interaction: a survey. In: IEEE international workshop on human computer interaction
Kajastila R, Holsti L, Hämäläinen P (2016) The augmented climbing wall: high-exertion proximity interaction on a wall-sized interactive surface. In: Proceedings of the SIGCHI, pp 758–769
Lecuyer A, Mobuchon P, Megard C, Perret J, Andriot C, Colinot JP (2003) HOMERE: a multimodal system for visually impaired people to explore virtual environments. In: Proceedings of IEEE international conference on virtual reality. IEEE, pp 251–257
Meera MB, Denise AS, Robert MG (1989) Earcons and icons: their structure and common design principles. Hum Comput Interact 4(1):11–44
Narzt W, Pomberger G, Ferscha A, Kolb D, Müller R, Wieghardt J, Hörtner H, Lindinger C (2006) Augmented reality navigation systems. J Univers Access Inf Soc 4(3):177–187
Navab N, Traub J, Sielhorst T, Feuerstein M, Bichlmeier C (2007) Action-and workflow-driven augmented reality for computer-aided medical procedures. Comput Graph Appl IEEE 27(5):10–14
Oviatt S (2002) Multimodal interfaces. In: Jacko J, Sears A (eds) Handbook of human–computer interaction. Lawrence Erlbaum, Mahwah
Richard P, Burdea G, Gomez D, Coiffet P (1994) A comparison of haptic, visual and auditive force feedback for deformable virtual objects. In: Proceedings of ICAT. pp 49–62
Rubinstein J, Meyer D, Evans J (2001) Executive control of cognitive processes in task switching. J Exp Psychol Hum Percept Perform 27(4):763–797
Sallnas E, Grohn K, Sjostrom C (2000) Supporting presence in collaborative environment by haptic force feedback. ACM Trans CHI 7(4):461–476
Salvucci D, Tattgen N, Borst J (2009) Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption. In: Proceedings of the SIGCHI, pp 1819–1828
Sanna A, Manuri F (2016) A survey on applications of augmented reality. Adv Comput Sci Int J 5(1):18–27
Schwald B, Laval BD (2003) An augmented system for training and assistance to maintenance in the industrial context. In: Proceedings of WSCG 2003
Strayer D (2007) Multitasking in the automobile. In: Kramer A, Wiegmann D, Kirlik A (eds) Applied attention. Oxford University Press, Oxford, pp 121–133
Tatzgern M, Kalkofen D, Schmalstieg D (2013) Dynamic compact visualizations for augmented reality. In: Proceedings of the IEEE conference on virtual reality 2013. IEEE, pp 3–6
Tonnis M, Klinker G, Plavsic M (2009) Survey and classification of head-up display presentation principles. In: Proceedings of the international ergonomics association
Walker BN, Lindsay J, Nance A, Nakano Y, Palladino DK, Dingler T, Jeon M (2013) Spearcons (speech-based earcons) improve navigation performance in advanced auditory menus. Hum Factor 55(1):157–182
Ware C (2012) Information visualization: perception for design, 3rd edn. Morgan Kaufmann, Burlington
Zheng XS, Foucault C, da Silva PM, Dasari S, Yang T, Goose S (2015) Eye-wearable technology for machine maintenance: effects of display position and hands-free operation. In: Proceedings of the SIGCHI, pp 2125–2134
Acknowledgements
This work was supported in part by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIP) (No. R0190-16-2011, Development of Vulnerability Discovery Technologies for IoT Software Security), and also in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government(MSIP) (No.2011-0030079).
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Kim, Y., Hong, S. & Kim, G.J. Augmented reality-based remote coaching for fast-paced physical task. Virtual Reality 22, 25–36 (2018). https://doi.org/10.1007/s10055-017-0315-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-017-0315-2