The Effect of Onomatopoeia to Enhancing User Experience in Virtual Reality

  • Jiwon Oh
  • Gerard J. KimEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11574)


Onomatopoeia refers to a word that phonetically imitates, resembles, or suggests the sound (or motion that accompanies sound) that it describes. It is often used in comics, games, and videos, along with the actual sound in caption. This usage is a way to emphasize, exaggerate, dramatize and draw attention the situation. In this paper we explore if the use of onomatopoeia, associated with sound feedback, could also bring about similar effects and improve the user experience in immersive virtual reality. We present an experiment comparing the user’s subjective experiences and attentive performance in two virtual worlds, each configured in two test conditions: (1) sound feedback with no onomatopoeia and (2) sound feedback with it. Our experiment has found that the moderate and strategic use of onomatopoeia can indeed help direct user attention, offer object affordance and thereby enhance user experience and even the sense of presence and immersion.


Onomatopoeia Virtual reality Sound visualization User experience 



This work was partially supported by the Global Frontier R&D Program on <Human-centered Interaction for Coexistence> funded by the National Research Foundation of Korea grant funded by the Korean Government (MEST) (NRF-2015M3A6A3076490), and by the Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (No. 2017-0-00179, HD Haptic Technology for Hyper Reality Contents).


  1. 1.
    VR Tennis Online. Accessed 15 Aug 2018
  2. 2.
    Dumas, B., Lalanne, D., Oviatt, S.: Multimodal interfaces: a survey of principles, models and frameworks. In: Lalanne, D., Kohlas, J. (eds.) Human Machine Interaction. LNCS, vol. 5440, pp. 3–26. Springer, Heidelberg (2009). Scholar
  3. 3.
    Fukusato, T., Morishima, S.: Automatic depiction of onomatopoeia in animation considering physical phenomena. In: Proceedings of the Seventh International Conference on Motion in Games, pp. 161–169. ACM (2014)Google Scholar
  4. 4.
    Guynes, S.A.: Four-color sound: a Peircean semiotics of comic book onomatopoeia. Public J. Semiot. 6(1), 58–72 (2014)Google Scholar
  5. 5.
    Heeter, C.: Being there: the subjective experience of presence. Presence Teleoperators Virtual Environ. 1(2), 262–271 (1992)CrossRefGoogle Scholar
  6. 6.
    Jung, S.-Y.: A Study on the feature and the function of TV visual character: focusing on the real variety shows. Korean J. Journal. Commun. Stud. 53(6), 153–176 (2009)Google Scholar
  7. 7.
    Lee, J., Kim, G.J.: Visual information for inducing interactivity and presence. In: Proceedings of the Korean Society for Emotion and Sensibility Conference. The Korean Society for Emotion and SensibilityGoogle Scholar
  8. 8.
    Masuch, M., Röber, N.: Game graphics beyond realism: then, now and tomorrow. Level UP: Digital Games Research Conference. DIGRA, Faculty of Arts, University of Utrecht (2004)Google Scholar
  9. 9.
    Nienhaus, M., Dollner, J.: Depicting dynamics using principles of visual art and narrations. IEEE Comput. Graph. Appl. 25(3), 40–51 (2005)CrossRefGoogle Scholar
  10. 10.
    Norman, D.: The design of everyday things: Revised and expanded edition. Constellation (2013)Google Scholar
  11. 11.
    Norman, D.A.: The psychology of everyday things (The design of everyday things) (1988)Google Scholar
  12. 12.
    Shimoda, W., Yanai, K.: A visual analysis on recognizability and discriminability of onomatopoeia words with DCNN features. In: 2015 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEEGoogle Scholar
  13. 13.
    Slater, M., Usoh, M.: Presence in immersive virtual environments. In: IEEE Virtual Reality Annual International Symposium, pp. 90–96. IEEE (1993)Google Scholar
  14. 14.
    Slater, M., Usoh, M.: Representations systems, perceptual position, and presence in immersive virtual environments. Presence Teleoperators Virtual Environ. 2(3), 221–233 (1993)CrossRefGoogle Scholar
  15. 15.
    Wang, F., Nagano, H., Kashino, K., Igarashi, T.: Visualizing video sounds with sound word animation to enrich user experience. IEEE Trans. Multimed. 19(2), 418–429 (2017)CrossRefGoogle Scholar
  16. 16.
    Wikipedia. Onomatopoeia — Wikipedia, The Free Encyclopedia (2018). Accessed 15 Aug 2018
  17. 17.
    Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: a presence questionnaire. Presence 7(3), 225–240 (1998)CrossRefGoogle Scholar
  18. 18.
    Yamamoto, T., Matsubara, M., Saito, H.: Visualization of environmental sounds using onomatopoeia and effective fonts. J. Soc. Art. Sci. 11(1), 1–11 (2012)Google Scholar
  19. 19.
    Jick, J.Y., Seok, L.H.: The expression techniques of cartoon style in the subtitles of TV entertainment programs. J. Digit. Des. 15(3), 779–788 (2015)Google Scholar
  20. 20.
    Yuan, Y., Steed, A.: Is the rubber hand illusion induced by immersive virtual reality? In: 2010 IEEE Virtual Reality Conference (VR), pp. 95–102. IEEE (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Digital Experience LaboratoryKorea UniversitySeoulKorea

Personalised recommendations