Skip to main content
SpringerLink
Log in

Interaction with AI-Controlled Characters in AR Worlds

  • Chapter
  • First Online:
Springer Handbook of Augmented Reality

Part of the book series: Springer Handbooks ((SHB))

Abstract

With the release of commercial head worn augmented reality (AR) devices, the AR community has grown exponentially. These AR devices come with robust computer vision algorithms to perform adequate object tracking and spatial localization of the user as well as objects in the AR environment. They allow for user input in various ways to interact with the scene. In addition, these devices finally became lightweight and comfortable enough to be used for real jobs in real professional environments. A new era has begun that allows the development of AR applications on a new level. It is no longer necessary to be an expert in computer vision or computer graphics to create meaningful solutions for application ideas that mostly have been proposed already many years ago. The usage of top-level game engines enables app developers to integrate well-designed and textured models, animations, physical effects, and illumination into their solutions, even without programming knowledge. In the past, the big majority of AR applications has focused on placing additional rigid or animated objects into the AR environment. Alternatively, tracked real objects have been superimposed by virtual supplementary information such as virtual structures inside a real object. However, another type of augmented content has almost been neglected so far. This chapter addresses the integration of and interaction with virtual characters in AR environments. It provides an overview of research that has been published by the AR community in the past and classifies different types of characters. In addition, we introduce the reader to Game AI, an R&D subdiscipline of game engineering that focusses on controlling non-player characters (NPCs) in video games. Furthermore, this chapter discusses peculiarities of the interaction with NPCs in AR environments and takes a closer look at the psychological subjects namely perception, communication, and behavioral psychology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 309.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 399.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sutherland, I.E.: The ultimate display. In: Proceedings of the IFIP Congress, pp. 506–508 (1965)

    Google Scholar 

  2. Millington, I., Funge, J.: Artificial Intelligence for Games. CRC Press, Boca Raton (2009)

    Book  Google Scholar 

  3. Rabin, S.: Game AI Pro 3: Collected Wisdom of Game AI Professionals. CRC Press, Boca Raton (2017)

    Book  Google Scholar 

  4. Rabin, S.: Game AI Pro: Collected Wisdom of Game AI Professionals. CRC Press, Boca Raton (2014)

    Google Scholar 

  5. Rabin, S.: Game AI Pro 2: Collected Wisdom of Game AI Professionals. CRC Press, Boca Raton (2015)

    Book  Google Scholar 

  6. Bates, J.: The role of emotion in believable agents. Commun. ACM. 37(7), 122–125 (1994). https://doi.org/10.1145/176789.176803

    Article  Google Scholar 

  7. Oh, C.S., Bailenson, J.N., Welch, G.F.: A systematic review of social presence: definition, antecedents, and implications. Front. Robot. AI. 5, 483 (2018). https://doi.org/10.3389/frobt.2018.00114

    Article  Google Scholar 

  8. Kim, K., Bruder, G., Welch, G.: Blowing in the wind: increasing copresence with a virtual human via airflow influence in augmented reality. In: ICAT-EGVE 2018 – International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (2018)

    Google Scholar 

  9. Kim, K., Schubert, R., Hochreiter, J., Bruder, G., Welch, G.: Blowing in the wind: increasing social presence with a virtual human via environmental airflow interaction in mixed reality. Comput. Graph. 83, 23–32 (2019). https://doi.org/10.1016/j.cag.2019.06.006

    Article  Google Scholar 

  10. Hartholt, A., Mozgai, S., Fast, E., Liewer, M., Reilly, A., Whitcup, W., Rizzo, A.: Virtual humans in augmented reality. In: Proceedings of the 7th International Conference on Human-Agent Interaction – HAI '19, pp. 205–207. Kyoto (2019)

    Google Scholar 

  11. Anabuki, M., Kakuta, H., Yamamoto, H., Tamura, H.: Welbo. In: CHI ‘00 Extended Abstracts on Human Factors in Computing Systems – CHI ’00, p. 10. The Hague (2000)

    Google Scholar 

  12. Barakonyi, I., Schmalstieg, D.: Augmented reality agents in the development pipeline of computer entertainment. In: Kishino, F., Kitamura, Y., Kato, H., Nagata, N. (eds.) Lecture Notes in Computer Science, Entertainment Computing – ICEC 2005, pp. 345–356. Springer Berlin Heidelberg, Berlin/Heidelberg (2005)

    Google Scholar 

  13. Kim, K., Boelling, L., Haesler, S., Bailenson, J., Bruder, G., Welch, G.F.: Does a digital assistant need a body? The influence of visual embodiment and social behavior on the perception of intelligent virtual agents in AR. In: 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 105–114. Munich, Oct 2018 – Oct 2018

    Google Scholar 

  14. Norouzi, N., Kim, K., Hochreiter, J., Lee, M., Daher, S., Bruder, G., Welch, G.: A systematic survey of 15 years of user studies published in the intelligent virtual agents conference. In: Proceedings of the 18th International Conference on Intelligent Virtual Agents – IVA ‘18, pp. 17–22. Sydney (2018)

    Google Scholar 

  15. Maes, P., Darrell, T., Blumberg, B., Pentland, A.: The ALIVE system: full-body interaction with autonomous agents. In: Proceedings Computer Animation ‘95, pp. 11–18. Geneva, Apr 1995

    Google Scholar 

  16. Cassell, J., Ananny, M., Basu, A., Bickmore, T., Chong, P., Mellis, D., Ryokai, K., Smith, J., Vilhjalmsson, H., Yan, H.: Shared reality: physical collaboration with a virtual peer. In: CHI ‘00 Extended Abstracts on Human Factors in Computing Systems – CHI ‘00, pp. 259–260. The Hague (2000)

    Google Scholar 

  17. Cheok, A.D., Weihua, W., Yang, X., Prince, S., Wan, F.S., Billinghurst, M., Kato, H.: Interactive theatre experience in embodied + wearable mixed reality space. In: Proceedings of the International Symposium on Mixed and Augmented Reality, pp. 59–317. Darmstadt, Sept 2002

    Google Scholar 

  18. Balcisoy, S., Kallmann, M., Torre, R., Fua, P., Thalmann, D.: Interaction techniques with virtual humans in mixed environments. In: 5th IEEE EMBS International Summer School on Biomedical Imaging, p. 6. Berder Island, June 2002

    Google Scholar 

  19. Cavazza, M., Charles, F., Mead, S.J., Martin, O., Marichal, X., Nandi, A.: Multimodal acting in mixed reality interactive storytelling. IEEE Multimedia. 11(3), 30–39 (2004). https://doi.org/10.1109/MMUL.2004.11

    Article  Google Scholar 

  20. Magnenat-Thalmann, N., Papagiannakis, G.: Virtual worlds and augmented reality in cultural heritage applications. In: Recording, Modeling and Visualization of Cultural Heritage, pp. 419–430 (2005)

    Google Scholar 

  21. Wagner, D., Billinghurst, M., Schmalstieg, D.: How real should virtual characters be?. In: Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology – ACE ‘06, p. 57. Hollywood (2006)

    Google Scholar 

  22. Miller, T., Weising, G.: Method for an Augmented Reality Character to Maintain and Exhibit Awareness of an Observer. US2018185757 (A1), US US201815906448 20180227, 5 July 2018

    Google Scholar 

  23. Tomi, A.B., Rambli, D.R.A.: An interactive mobile augmented reality magical playbook: learning number with the thirsty crow. Procedia Comp. Sci. 25, 123–130 (2013). https://doi.org/10.1016/j.procs.2013.11.015

    Article  Google Scholar 

  24. Arroyo-Palacios, J., Marks, R.: Believable virtual characters for mixed reality. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 121–123. Nantes, Oct 2017 – Oct 2017

    Google Scholar 

  25. Ishii, H., Lee, N., Natkin, S., Tsushima, K. (eds.): Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology – ACE ‘06. ACM Press, New York (2006)

    Google Scholar 

  26. Kim, K., Maloney, D., Bruder, G., Bailenson, J.N., Welch, G.F.: The effects of virtual human's spatial and behavioral coherence with physical objects on social presence in AR. Comp. Anim. Virtual Worlds. 28(3–4), e1771 (2017). https://doi.org/10.1002/cav.1771

    Article  Google Scholar 

  27. Miller, M.R., Jun, H., Herrera, F., Yu Villa, J., Welch, G., Bailenson, J.N.: Social interaction in augmented reality. PLoS One. 14(5), e0216290 (2019). https://doi.org/10.1371/journal.pone.0216290

    Article  Google Scholar 

  28. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77(12), 1321--1329 (1994)

    Google Scholar 

  29. Stapleton, C., Hughes, C., Moshell, M., Micikevicius, P., Altman, M.: Applying mixed reality to entertainment. Computer. 35(12), 122–124 (2002). https://doi.org/10.1109/MC.2002.1106186

    Article  Google Scholar 

  30. Stapleton, C.B., Hughes, C.E.: Making memories of a lifetime. In: Emerging Technologies of Augmented Reality: Interfaces and Design. IGI Global, S. 329--351 (2007). https://books.google.de/books/about/Emerging_Technologies_of_Augmented_Reali.html?id=ItLhfvNGTssC&redir_esc=y

  31. Azuma, R.T.: A survey of augmented reality. Presence Teleop. Virt. 6(4), 355–385 (1997). https://doi.org/10.1162/pres.1997.6.4.355

    Article  Google Scholar 

  32. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Comput. Grap. Appl. 21(6), 34–47 (2001). https://doi.org/10.1109/38.963459

    Article  Google Scholar 

  33. Bichlmeier, C.: Immersive, interactive and contextual in-situ visualization for medical applications. Dissertation, Chair for Computer Aided Medical Procedures & Augmented Reality, Technische Universität München, München (2010). Available: http://mediatum.ub.tum.de/node?id=977862

  34. Agusanto, K., Li, L., Chuangui, Z., Sing, N.W.: Photorealistic rendering for augmented reality using environment illumination. In: The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings, pp. 208–216. Tokyo, Oct 2003

    Google Scholar 

  35. Scherer, L.: Lebenssimulation und Aufbauspiel in einer Augmented Reality Umgebung. Bachelor’s thesis, Faculty of Computer Science, Kempten University of Applied Sciences, Germany, Kempten (2019). Available: http://games.hs-kempten.de/lebenssimulation-und-aufbauspiel-in-einer-augmented-reality-umgebung/

  36. Röhner, J., Schütz, A.: Psychologie der Kommunikation. Springer Fachmedien Wiesbaden, Wiesbaden (2016)

    Book  Google Scholar 

  37. Cialdini, R.B.: Influence: Science and Practice, 5th edn. Pearson Education, Boston (2009)

    Google Scholar 

  38. Petty, R.E., Cacioppo, J.T. (eds.): Communication and Persuasion. Springer New York, New York (1986)

    Google Scholar 

  39. Pacchierotti, C., Sinclair, S., Solazzi, M., Frisoli, A., Hayward, V., Prattichizzo, D.: Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. IEEE Trans. Haptics. 10(4), 580–600 (2017). https://doi.org/10.1109/TOH.2017.2689006

    Article  Google Scholar 

  40. Furukawa, T., Niwa, D., Hirano, Y., Hanamitsu, N., Kamiyama, Y., Nii, H., Krekoukiotis, C., Minamizawa, K., Noda, A., Yamada, J., Kitamura, K.: Designing a full-body customizable haptic interface using two-dimensional signal transmission. In: ACM SIGGRAPH 2019 Posters on – SIGGRAPH '19, pp. 1–2. Los Angeles (2019)

    Google Scholar 

  41. Hall, E.T.: The Hidden Dimension. Anchor Books, New York (1990). Available: http://www.loc.gov/catdir/description/random043/90034870.html

    Google Scholar 

  42. Hargie, O., Dickson, D.: Skilled Interpersonal Communication: Research, Theory, and Practice, 4th edn. Routledge, London (2004)

    Book  Google Scholar 

  43. Blascovich, J., Bailenson, J.: Infinite Reality: Avatars, Eternal Life, New Worlds, and the Dawn of the Virtual Revolution, 1st edn. William Morrow, New York (2011)

    Google Scholar 

  44. Slater, M., Sadagic, A., Usoh, M., Schroeder, R.: Small-group behavior in a virtual and real environment: a comparative study. Presence Teleop. Virt. 9(1), 37–51 (2000). https://doi.org/10.1162/105474600566600

    Article  Google Scholar 

  45. Bailenson, J.N., Blascovich, J., Beall, A.C., Loomis, J.M.: Equilibrium theory revisited: mutual gaze and personal space in virtual environments. Presence Teleop. Virt. 10(6), 583–598 (2001). https://doi.org/10.1162/105474601753272844

    Article  Google Scholar 

  46. Bonsch, A., Radke, S., Overath, H., Asche, L.M., Wendt, J., Vierjahn, T., Habel, U., Kuhlen, T.W.: Social VR: how personal space is affected by virtual agents’ emotions. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 199–206. Reutlingen, Mar 2018 – Mar 2018

    Google Scholar 

  47. Reynolds, C.W.: Steering behaviors for autonomous characters. In Game developers conference (Vol. 1999, pp. 763--782) (1999)

    Google Scholar 

  48. Eisele, M.: Techniken zur Interaktion mit autonom gesteuerten Agenten NPCs in Augmented Reality Spielen. Bachelor’s thesis, Faculty of Computer Science, Kempten University of Applied Sciences, Kempten, 16 Mar 2017. Available: http://games.hs-kempten.de/techniken-zur-interaktion-mit-autonom-gesteuerten-agenten-npcs-in-augmented-reality-spielen/

Download references

Author information

Authors and Affiliations

  1. Fakultät Informatik, Studiengänge Game Engineering (BA) und Game Engineering & Visual Computing (MA), Hochschule für angewandte Wissenschaften Kempten, Kempten, Germany

    Christoph Bichlmeier

Authors
  1. Christoph Bichlmeier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoph Bichlmeier .

Editor information

Editors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore

    Andrew Yeh Ching Nee

  2. Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore

    Soh Khim Ong

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Bichlmeier, C. (2023). Interaction with AI-Controlled Characters in AR Worlds. In: Nee, A.Y.C., Ong, S.K. (eds) Springer Handbook of Augmented Reality. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-030-67822-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-67822-7_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-67821-0

  • Online ISBN: 978-3-030-67822-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation