Skip to main content
SpringerLink
Log in

Why, when and how to use augmented reality agents (AuRAs)

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Over the last number of years, multiple research projects have begun to create augmented reality (AR) applications that use augmented reality agents, or AuRAs, as their principle interaction and development paradigm. This paper aims to address this new and distinct field of AuRAs by asking three questions: why should AuRAs be researched, when are they a useful paradigm, and how can they be developed? The first question explores the motivation behind applying AuRAs to AR. Specifically, it investigates whether AuRAs are purely an interaction paradigm, or whether they can also serve as a development paradigm, by outlining in which circumstances it is appropriate for a project to use AuRAs and where their addition would only add unnecessary complexity. A navigational experiment, performed in simulated AR, explores the second question of when AuRAs can be a useful concept in AR applications. Results from this experiment suggest that an embodied virtual character allows for faster navigation along a shorter route than directional arrows or marking the target with an AR “bubble”. An exploration of the limitations of the simulated AR environment illuminates how faithfully the experiment recreated the environmental challenges that AuRAs can help to address. Finally, the question of how to develop such applications is addressed through the introduction of the agent factory augmented reality toolkit that allows the rapid prototyping of such applications. Results from a usability study on the toolkit are also presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. While similar systems have been deployed using robots as shopping guides (Black et al. 2009; Gharpure and Kulyukin 2008), they are quite common use cases for AR as well [e.g. the aforementioned ShopNavi (Nagao 1998) or the more recent PromoPad (Zhu and Owen 2008)].

  2. While it is an undecided issue whether presence scores can be compared between different experiments (Usoh et al. 2000), we believe they can nonetheless serve as a useful indicator of the level of immersion experienced by users.

  3. The supermarket used in the experiment was based on blueprints of award-winning designs to prevent any bias in its construction (Pegler 1990).

  4. The experiment software failed to record distance for some of the participants, which is why subject numbers and degrees of freedom are different for some of the distance statistics.

  5. http://www.oculusvr.com/.

  6. http://www.virtuix.com/.

  7. SUMI is a well-established commercial system that is referenced by the ISO 9241-11 standard (ISO 1998) as a method for testing user satisfaction.

  8. This is essentially the same objective given to participants that completed the user evaluation reported in the next section.

  9. A predicate is the overarching term for a statement of logic that an agent can process, be that a belief, plan or commitment rule.

References

  • Adams D (1990) Hyperland. BBC, London

    Google Scholar 

  • Ahrndt S, Fähndrich J, Lützenberger M, Rieger A, Albayrak S (2012) An agent-based augmented reality demonstrator in the domestic energy domain. In: PAAMS ’12, pp 225–228

  • Bellifemine F, Poggi A, Rimassa G (2001) Developing multi-agent systems with a FIPA-compliant agent framework. Softw Pract Exp 31(2):103–128

    Google Scholar 

  • Black D, Clemmensen N, Skov M (2009) Supporting the supermarket shopping experience through a context-aware shopping trolley. In: Proceedings of the 21st annual conference of the Australian computer–human interaction special interest group: design: open 24/7. ACM, pp 33–40

  • Bratman ME (1987) Intentions, plans, and practical reason. Harvard University Press, Cambridge

    Google Scholar 

  • Brooks RA (1991) Intelligence without representation. Artif Intell 47(1–3):139–159

    Article  Google Scholar 

  • Campbell A, Collier R, Dragone M, Görgü L, Holz T, O’GradyM, O’Hare GMP, Sassu A, Stafford J (2012) Facilitating ubiquitous interaction using intelligent agents. In: Human–computer interaction: the agency perspective, pp 303–326

  • Charles F, Cavazza M, Mead S, Martin O, Nandi A, Marichal X (2004) Compelling experiences in mixed reality interactive storytelling. In: Proceedings of the 2004 ACM SIGCHI international conference on advances in computer entertainment technology. ACM, Singapore, pp 32–40

  • Cruz-Neira C, Sandin DJ, DeFanti TA, Kenyon RV, Hart JC (1992) The CAVE: audio visual experience automatic virtual environment. Commun ACM 35(6):64–72

    Article  Google Scholar 

  • Dautenhahn K (1999) Embodiment and interaction in socially intelligent life-like agents. In: Computation for metaphors, analogy, and agents, pp 102–141

  • Dehn D, Van Mulken S (2000) The impact of animated interface agents: a review of empirical research. Int J Hum Comput Stud 52(1):1–22

    Article  Google Scholar 

  • Denby B, Campbell A, Carr H, O’Hare G (2009) The LAIR: lightweight affordable immersion room. Presence Teleoper Virtual Environ 18(5), 409–411

    Google Scholar 

  • Dragone M, Holz T, O’Hare GMP (2007) Using mixed reality agents as social interfaces for robots. In: Proceedings of the 16th IEEE international symposium on robot and human interactive communication (RO-MAN 2007). IEEE, pp 1161–1166

  • Drascic D, Milgram P (1996) Perceptual issues in augmented reality. Stereosc Disp Virtual Real Syst III 2653(1):123–134

    Article  Google Scholar 

  • Duffy B, O’Hare G, Bradley J, Martin A, Schön B (2005) Future reasoning machines: mind and body. Kybernetes 34(9/10):1404–1420

    Article  Google Scholar 

  • Ferber J (1999) Multi-agent systems: an introduction to distributed artificial intelligence, vol 33. Addison-Wesley, Reading

  • Foner LN (1993) What’s an agent anyway? A sociological case study. FTP report. MIT Media Lab, Cambridge, MA

  • Geiger C, Reimann C, Sticklein J, Paelke V (2002) JARToolKit: a java binding for ARToolKit. In: The first IEEE international augmented reality toolkit workshop (ART 02). IEEE

  • Geisser S, Greenhouse SW (1958) An extension of box’s results on the use of the f distribution in multivariate analysis. Ann Math Stat 29(3):885–891

    Article  MATH  MathSciNet  Google Scholar 

  • Gharpure C, Kulyukin V (2008) Robot-assisted shopping for the blind: issues in spatial cognition and product selection. Intell Serv Robot 1(3):237–251

    Article  Google Scholar 

  • Google (2009) Layar: augmented reality browser. http://www.google.com/mobile/goggles/, accessed Aug 2010

  • Görgü L, Campbell A, McCusker K, Dragone M, O’Grady M, O’Connor N, O’Hare G (2010) FreeGaming: mobile, collaborative, adaptive and augmented ExerGaming. In: Proceedings of the 8th international conference on advances in mobile computing and multimedia. ACM, pp 173–179

  • Haugeland J (1989) Artificial intelligence: the very idea. MIT Press, Cambridge

    Google Scholar 

  • Hedberg SR (1998) Is AI going mainstream at last? A look inside Microsoft research. IEEE Intell Syst 13(2):21–25

    Article  Google Scholar 

  • Holz T, Campbell AG, O’Hare GMP, Stafford JW, Martin A, Dragone M (2011) MiRA—mixed reality agents. Int J Hum Comput Stud 69:251–268

    Article  Google Scholar 

  • Horvitz E, Breese J, Heckerman D, Hovel D, Rommelse K (1998) The Lumiere Project: Bayesian user modeling for inferring the goals and needs of software users. In: Proceedings of the 14th conference on uncertainty in artificial intelligence, pp 256–265

  • ISO 9241-11 (1998) Ergonomic requirements for office work with visual display terminals (VDTs)—part 11: guidance on usability. International Organization for Standardization, Geneva, Switzerland

  • Iso M (2007) Denno coil. NHK Educational, Geneva, Switzerland

  • Ju W, Nickell S, Eng K, Nass C (2005) Influence of colearner agent behavior on learner performance and attitudes. In: Extended abstracts on human factors in computing systems (CHI ’05). ACM, New York, pp 1509–1512

  • Jurdak R, Ruzzelli A, O’Hare G (2010) Radio sleep mode optimization in wireless sensor networks. IEEE Trans Mobile Comput 9(7):955–968

    Article  Google Scholar 

  • Kanellopoulos Y, Antonellis P, Antoniou D, Makris C, Theodoridis E, Tjortjis C, Tsirakis N (2010) Code quality evaluation methodology using the ISO/IEC 9126 standard. Int J Softw Eng Appl 1(3):17–36

    Google Scholar 

  • Kato H, Billinghurst M (1999) Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: Proceedings of the 2nd international workshop on augmented reality

  • Katz B, Dramas F, Parseihian G, Gutierrez O, Kammoun S, Brilhault A, Brunet L, Gallay M, Oriola B, Auvray M et al (2012) Navig: guidance system for the visually impaired using virtual augmented reality. Technol Disabil 24(2):163–178

    Google Scholar 

  • Kirakowski J, Corbett M (1993) SUMI: the software usability measurement inventory. Br J Educ Technol 24(3):210–212

    Article  Google Scholar 

  • Koay K, Syrdal D, Walters M, Dautenhahn K (2009) A user study on visualization of agent migration between two companion robots. In: proceedings of 13th International Conference on Human-Computer Interaction (HCII 2009), San Diego, CA, USA

  • Kretschmer U, Coors V, Spierling U, Grasbon D, Schneider K, Rojas I, Malaka R (2001) Meeting the spirit of history. In: Proceedings of the conference on virtual reality, archeology and cultural heritage (VAST ’01), pp 141–152

  • Kruijff E, Swan JE, Feiner S (2010) Perceptual issues in augmented reality revisited. In: Proceedings of 9th IEEE International Symposium on mixed and augmented reality (ISMAR 2010). IEEE Computer Society, Seoul, Korea, pp 3–12

  • Lancelle M, Settgast V, Fellner D (2009) Definitely affordable virtual environment. In: Proceedings of IEEE virtual reality. IEEE, p 1

  • Lee C, Bonebrake S, Hollerer T, Bowman DA (2009) A replication study testing the validity of AR simulation in VR for controlled experiments. In: Proceedings of IEEE/ACM international symposium on mixed and augmented reality (ISMAR 2009). IEEE Computer Society, pp 203–204

  • Lee C, Bonebrake S, Höllerer T, Bowman DA (2010) The role of latency in the validity of AR simulation. In: IEEE virtual reality conference (VR 2010), pp 11–18

  • Lee Y, Choi J, Kim S, Lee S, Jang S (2011) Social augmented reality for sensor visualization in ubiquitous virtual reality. In: Shumaker R (ed) Virtual and mixed reality: new trends, vol 6773. Lecture Notes in Computer Science. Springer, Berlin, pp 69–75

  • Lieberman H (1997) Autonomous interface agents. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ’97). ACM, New York, pp 67–74

  • Lingley AR, Ali M, Liao Y, Mirjalili R, Klonner M, Sopanen M, Suihkonen S, Shen T, Otis BP, Lipsanen H et al (2011) A single-pixel wireless contact lens display. J Micromech Microeng 21(12):125014

    Article  Google Scholar 

  • Littman M, Cassandra A, Pack Kaelbling L (1995) Learning policies for partially observable environments: scaling up. In: International conference on machine learning workshop—then conference, Citeseer, pp 362–370

  • Maes P, Darrell T, Blumberg B, Pentland A (1995) The ALIVE system: full-body interaction with autonomous agents. In: Proceedings of the computer animation

  • Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst (special issue on networked reality), E77-D(12), 1321–1329

  • Nagao K (1998) Agent augmented reality: agents integrate the real world with cyberspace. In: Ishida T (ed) Community computing: collaboration over global information networks. Wiley, New York

  • Obaid M, Niewiadomski R, Pelachaud C (2011) Perception of spatial relations and of coexistence with virtual agents. In: Intelligent virtual agents. Springer, Berlin, pp 363–369

  • O’Hare G, O’Grady M (2003) Gulliver’s genie: a multi-agent system for ubiquitous and intelligent content delivery. Comput Commun 26(11):1177–1187

    Article  Google Scholar 

  • O’Hare GMP, Collier R, Conlon J, Abbas S (1998) Agent factory: an environment for constructing and visualising agent communities. In: Proceedings of the ninth Irish conference on artificial intelligence and cognitive science (AICS 98), pp 249–261, Dublin, Ireland

  • O’Hare GM, Campbell AG, Stafford JW, Aiken R (2005a) NeXuS: behavioural realism in mixed reality scenarios through virtual sensing. In: Proceedings of the eighteenth international conference on computer animation and social agents (CASA 2005), Hong Kong

  • O’Hare GMP, Campbell AG, Stafford JW (2005b) NeXuS: delivering behavioural realism through intentional agents. In: Proceedings of the 3rd international conference on active media technology (AMT 2005). IEEE Press, Takamatsu, Japan, pp 481–486

  • O’Hare G, O’Grady M, Muldoon C, Bradley J (2006) Embedded agents: a paradigm for mobile services. Int J Web Grid Serv 2(4):379–405

    Article  Google Scholar 

  • Pegler MM (1990) Market, supermarket and hypermarket design. Visual Reference, New York

    Google Scholar 

  • Pustka D, Huber M, Waechter C, Echtler F, Keitler P, Newman J, Schmalstieg D, Klinker G (2011) Automatic configuration of pervasive sensor networks for augmented reality. IEEE Pervasive Comput 10(3):68–79

    Article  Google Scholar 

  • Ragan E, Wilkes C, Bowman DA, Hollerer T (2009) Simulation of augmented reality systems in purely virtual environments. In: Proceedings of the 2009 IEEE virtual reality conference (VR ’09). IEEE Computer Society, Washington, DC, USA, pp 287–288

  • Rao AS, Georgeff MP (1995) BDI agents: from theory to practice. In: Proceedings of the first international conference on multi-agent systems (ICMAS 95), pp 312–319

  • Robert D, Breazeal C (2012) Blended reality characters. In: Proceedings of the seventh annual ACM/IEEE international conference on human–robot interaction. ACM, pp 359–366

  • Russell SJ, Norvig P (2003) Artificial intelligence: a modern approach, 2nd edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  • Saeedi E, Kim S, Parviz BA (2008) Self-assembled crystalline semiconductor optoelectronics on glass and plastic. J Micromech Microeng 18(7):075019

    Google Scholar 

  • Sas C, O’Hare GMP (2003) Presence equation: an investigation into cognitive factors underlying presence. Presence 12(5):523–537

    Article  Google Scholar 

  • Schmalstieg D (2005) Augmented Reality techniques in games. In: Proceedings of IEEE/ACM international symposium on mixed and augmented reality (ISMAR 2005), pp 176–177

  • Shneiderman B, Plaisant C (2010) Designing the user interface: strategies for effective human–computer interaction, 5th edn. Addison Wesley, Reading

  • Shoham Y (1993) Agent-oriented programming. Artif Intell 60(1):51–92

    Article  MathSciNet  Google Scholar 

  • Slater M, Steed A (2002) Meeting people virtually: experiments in shared virtual environments. The social life of avatars: presence and interaction in shared virtual environments, pp 146–171

  • Slater M, Usoh M, Steed A (1994) Depth of presence in virtual environments. Presence Teleoper Virtual Environ 3:130–144

    Google Scholar 

  • Sutherland IE (1965) The ultimate display. In: Proceedings of the IFIP congress, vol 2. International Federation for Information Processing, Arlington, VA

  • Syrdal D, Koay K, Walters M, Dautenhahn K (2009) The boy-robot should bark!-children’s impressions of agent migration into diverse embodiments. In: Proceedings of the new frontiers in human–robot interaction, a symposium at the AISB2009 convention

  • Tan DS, Czerwinski MP, Robertson GG (2006) Large displays enhance optical flow cues and narrow the gender gap in 3-D virtual navigation. Hum Fact 48(2):318–333

    Article  Google Scholar 

  • Thrun S (2005) Probabilistic robotics, vol 1. MIT press, Cambridge

  • Usoh M, Catena E, Arman S, Slater M (2000) Using presence questionnaires in reality. Presence Teleoper Virtual Environ 9(5):497–503

    Article  Google Scholar 

  • Wagner D, Reitmayr G, Mulloni A, Drummond T, Schmalstieg D (2010) Real-time detection and tracking for augmented reality on mobile phones. IEEE Trans Vis Comput Graph 16(3):355–368

    Article  Google Scholar 

  • Weiser M (1992) Does ubiquitous computing need interface agents? In: MIT media lab symposium on user interface agents

  • Wooldridge M, Jennings NR (1994) Intelligent agents: theory and practice. Knowl Eng Rev 10(2):115–152

    Article  Google Scholar 

  • Zhu W, Owen C (2008) Design of the promopad: an automated augmented-reality shopping assistant. J Organ End User Comput (JOEUC) 20(3):41–56

    Article  Google Scholar 

  • Ziemke T (2003) What’s that thing called embodiment. In: Proceedings of the 25th annual meeting of the cognitive science society. Lawrence Erlbaum, Mahwah, pp 1305–1310

Download references

Acknowledgments

This work is supported by Science Foundation Ireland under Grant No. 07/CE/I1147. The authors would like to thank the reviewers whose suggestions and constructive feedback throughout the review process led to a greatly improved paper.

Author information

Authors and Affiliations

  1. CLARITY: Centre for Sensor Web Technology, University College Dublin, Belfield, Dublin 4, Ireland

    Abraham G. Campbell, Thomas Holz & G. M. P. O’Hare

  2. School of Computer Science and Informatics, University College Dublin, Belfield, Dublin 4, Ireland

    John W. Stafford

Authors
  1. Abraham G. Campbell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John W. Stafford
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Holz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. G. M. P. O’Hare
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abraham G. Campbell.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Campbell, A.G., Stafford, J.W., Holz, T. et al. Why, when and how to use augmented reality agents (AuRAs). Virtual Reality 18, 139–159 (2014). https://doi.org/10.1007/s10055-013-0239-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-013-0239-4

Keywords

Search

Navigation