Abstract
Perception and attention mechanisms are of great importance for entities situated within complex dynamic environments. With roles extending greatly beyond passive information services about the external environment, such mechanisms actively prioritise, augment and expedite information to ensure that the potentially relevant is made available so appropriate action can take place. Here, we describe the rationale behind endowing artificial entities, or virtual agents, with real-time perception and attention systems. We cover the fundamentals of designing and building such systems. Once equipped, the resulting agents can achieve a more substantial connection with their environment for the purposes of reacting, planning, decision making and, ultimately, behaving.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Blumberg B (1997) Go with the flow: synthetic vision for autonomous animated creatures. In: Lewis Johnson W, Hayes-Roth B. (ed) Proceedings of the first international conference on autonomous agents (Agents’97). ACM Press, New York, NY, pp 538–539
Bordeux C, Boulic R, Thalmann D (1999) An efficient and flexible perception pipeline for autonomous agents. Proceedings of Eurographics ’99, Milano, Italy, pp 23–30
Bradski GR (1998) Computer vision face tracking for use in a perceptual user interface. Intel Tech J Q2:705–740
Bradski GR, Kaehler A (2008) Learning OpenCV: computer vision with the openCV library. O’Reilly, Cambridge, MA
Camurri A, Mazzarino B, Volpe G (2004) Analysis of expressive gesture: the eyesweb expressive gesture processing library. In: Camurri A, Volpe G. (eds) Gesture-based communication in human-computer interaction. LNAI vol 2915, Springer, Berlin, pp 460–467
Camurri A, Coletta P, Varni G, Ghisio S (2007) Developing multimodal interactive systems with EyesWeb XMI. In: Proceedings of the 2007 conference on new interfaces for musical expression, June 6–10, New York, NY, pp 305–308
Castellano G Movement expressivity analysis in affective computers: from recognition to expression of emotion. Ph.D. Dissertation, Faculty of Engineering, University of Genova, Italy, February 2008
Castellano G, Villalba SD, Camurri A (2007) Recognising human emotions from body movement and gesture dynamics. In: Paiva A, Prada R, Picard RW. (eds) affective computing and intelligent interaction, second international conference, ACII 2007, Lisbon, Portugal, September 12–14, 2007, Proceedings, vol 4738 of LNCS. Springer, Berlin, pp 71–82
Castellano G, Mortillaro M, Camurri A, Volpe G, Scherer K (2008) Automated analysis of body movement in emotionally expressive piano performances. Music Percept 26(2):103–119
Chopra-Khullar S, Badler NI (1999) Where to look? automating attending behaviors of virtual human characters. In: AGENTS ’99: Proceedings of the third annual conference on autonomous agents. ACM Press, New York, NY
Colburn A, Cohen M, Drucker S (2000) The role of eye gaze in avatar mediated conversational interfaces, Tech. Report MSR-TR-2000-81, Microsoft Research
Conde T, Thalmann D (2006) An integrated perception for autonomous virtual agents: active and predictive perception. Comput Animation amp Virtual Worlds 17(3–4):457–68
Courty N, Marchand E (2003) Visual perception based on salient features. In: IEEE international conference on intelligent robots and systems, IROS’03, vol 2, Las Vegas, Nevada, October 2003, pp 1024–1029
Delgado-Mata C, Aylett R (2001) Communicating emotion in virtual environments through artificial scents. In: IVA ’01: Proceedings of the third international workshop on intelligent virtual agents. Springer, London
Eriksen CW, Yeh YY (1985) Allocation of attention in the visual field. J Exp Psychol Hum Percept Perform 11(5):583–597
Garau M, Slater M, Bee S, Sasse MA (2001) The impact of eye gaze on communication using humanoid avatars. In: CHI ’01: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM Press, New York, NY, pp 309–316
Gillies M (2001) Practical behavioural animation based on vision and attention. PhD dissertation, University of Cambridge Computer Laboratory
Gu E, Wang J, Badler NI (2005) Generating sequence of eye fixations using decision-theoretic attention model. IEEE computer society conference on computer vision and pattern recognition (CVPRW’05) – Workshops, p 92
Herrero P, de Antonio A (2003) Introducing human-like hearing perception in intelligent virtual agents. In: Proceedings of the second international joint conference on autonomous agents and multiagent systems (AAMAS), July 14–18, Melbourne, Australia
Hill R (2000) Perceptual attention in virtual humans: towards realistic and believable gaze behaviours, AAAI Technical Report FS-00-03, pp 46–52
Isla D, Burke R, Downie M, Blumberg B (2001) A layered brain architecture for synthetic creatures. In: Nebel B. (ed) Proceedings of the seventeenth international joint conference on artificial intelligence, IJCAI 2001, Seattle, Washington, USA, August 4–10, 2001, Morgan Kaufmann, 2001, pp 1051–1058
Itti L, Koch C, Niebur E (1998 November) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259
Itti L, Dhavale N, Pighin F (2003) Realistic avatar eye and head animation using a neurobiological model of visual attention. In: Proceedings of the SPIE 48th annual international symposium on optical science and technology, San Diego, USA, August 3–8, pp 64–78
Kahneman D, Henrik A (1981) Perceptual organization, chapter perceptual organization and attention. Erlbaum, Hillsdale, NJ
Kim Y, Van Velsen M, Hill R (2005) Modeling dynamic perceptual attention in complex virtual environments. In: Panayiotopoulos T, Gratch J, Aylett R, Ballin D, Olivier P, Rist T. (eds) Intelligent virtual agents, 5th international working conference, IVA 2005, Kos, Greece, September 12–14, 2005, Proceedings. Lecture notes in computer science, vol 3661. Springer, London, pp 266–277
Koch C, Ullman S (1985) Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiol 4:219–227
Kranstedt A, Lücking A, Pfeiffer T, Rieser H, Wachsmuth I (2006) Deixis: how to determine demonstrated objects using a pointing cone. In: Gibet S, Courty N, Kamp JF (eds) Gesture in human-computer interaction and simulation, vol 3881. Springer, New York, NY, pp 300–301
Kuffner J, Latombe J-C (1999 May) Fast synthetic vision, memory, and learning models for virtual humans. In: Proceedings of computer animation ’99. IEEE Computer Society, Washington, DC
Laptev I, Lindeberg T (2003) Space-time interest points. In: Proceedings of ICCV03, Nice, France, October 13–16, pp 432–443
Lee SP, Badler JB, Badler NI (2002) Eyes alive. In: Proceedings of the 29th annual conference on computer graphics and interactive techniques (SIGGRAPH ’02). ACM Press, New York, NY, pp 637–644
Leonard T (2003) Building an AI sensory system – examining the design of thief - the dark project. In: Proceedings of game developers conference 2003, CMP Game Media Group San Francisco, CA
Lozano M, Lucia R, Barber F, Grimaldo F, Soares AL, Fornes A (2003) An efficient synthetic vision system for 3d multi-character systems. In: Rist T, Aylett R, Ballin D, Rickel J (eds) Intelligent agents, 4th international workshop, IVA 2003, Kloster Irsee, Germany, September 15–17, 2003, Proceedings, vol 2792 of Lecture notes in computer science. Springer, New York, NY, pp 356–357
Morency L-P, Darrell T (2004) From conversational tooltips to grounded discourse: head pose tracking in interactive dialog systems. In: Proceedings of the 6th international conference on multimodal interfaces, State College, PA, October 2004, pp 32–37
Neisser U (1967) Cognitive psychology. Appleton-Century-Crofts New York, NY
Noser H, Renault O, Thalmann D, Thalmann NM (1995) Navigation for digital actors based on synthetic vision, memory and learning. Comput Graph 19(1):7–19
Oikonomopoulos A, Patras I, Pantic M (2006 June) Spatiotemporal salient points for visual recognition of human actions. IEEE Trans Syst Man, Cybern B Cybern 36(3):710–719
Ouerhani N, von Wartburg R, Hugli H (2004) Empirical validation of the saliency-based model of visual attention. Electron Lett Comput Vis Image Anal 3(1):13–24
Pantic M, Bartlett MS (2007) Machine analysis of facial expressions. In: Delac K, Grgic M (eds) Face recognition. I-Tech Education and Publishing, Vienna, Austria, pp 377–416
Patras I, Pantic M (2004) Particle filtering with factorized likelihoods for tracking facial features. In: Proceedings of the IEEE international conference on face and gesture recognition. Seoul, Korea, May 17–19, pp 97–102
Peters C (2006) Evaluating perception of interaction initiation in virtual environments using humanoid agents. In: Proceedings of the 17th European conference on artificial intelligence, Riva Del Garda, Italy, August 2006, pp 46–50
Peters C, O’Sullivan C (2002) Synthetic vision and memory for autonomous virtual humans. Comput Graph Forum 21(4):743–743
Peters C, O’Sullivan C (2003) Bottom-up visual attention for virtual human animation. In: CASA ’03: proceedings of the 16th international conference on computer animation and social agents (CASA 2003), IEEE Computer Society, Washington, DC, p 111
Peters RJ, Itti L (2008) Applying computational tools to predict gaze direction in interactive visual environments. ACM Trans Appl Percept 5(2):8
Picot A, Bailly G, Elisei F, Raidt S (2007) Scrutinizing natural scenes: controlling the gaze of an embodied conversational agent. In: Pelachaud C, Martin J-C, André E, Chollet G, Karpouzis K, Pelé D (eds) Intelligent virtual agents 2007, Proceedings, LNCS, vol 4722. Springer, Berlin, pp 272–282
Poggi I, Pelachaud C, de Rosis F (2000) Eye communication in a conversational 3d synthetic agent. AI Commun 13(3):169–182
Posner MI (1980) Orienting of attention. Quart J Exp Psychol 32:3–25
Privitera H, Stark LW (2000) Algorithms for defining visual regions-of-interest: Comparison with eye fixations. IEEE Trans Pattern Anal Mach Intell (PAMI) 22(9):970–982
Rabiner LR (1990) A tutorial on hidden Markov models and selected applications in speech recognition. In: Waibel A, Lee K-F (eds) Readings in speech recognition. Morgan Kaufmann Publishers Inc., San Francisco, CA, pp 267–296
Raidt S, Bailly G, Elisei F (2005) Basic components of a face-to-face interaction with a conversational agent: mutual attention and deixis. Proceedings of the 2005 joint conference on smart objects and ambient intelligence: innovative context-aware services: usages and technologies, Grenoble, France, October 12–14, pp 247–252
Rapantzikos K, Avrithis Y, Kollias S (2005) Handling uncertainty in video analysis with spatiotemporal visual attention. In: Proceedings of IEEE international conference on fuzzy systems, Reno, Nevada, May 2005
Rapantzikos K, Tsapatsoulis N, Avrithis Y, Kollias S (June 2007) A bottom-up spatiotemporal visual attention model for video analysis. IET Image process 1(2):237–248
Rehm M, André E (2005) Where do they look?: gaze behaviors of multiple users interacting with an embodied conversational agent, vol 3661, pp 241–252
Reynolds CW (2000) Interaction with groups of autonomous characters. In: Proceedings of game developers conference 2000. CMP Game Media Group, San Francisco, CA, pp 449–460
Scherer K, Schorr A, Johnstone T (February 2001) Appraisal processes in emotion: theory, methods, research (series in affective science). Oxford University Press, Bethesda, MD
Terzopoulos D, Rabie TF (1997) Animat vision: active vision in artificial animals. Videre J Comput Vis Res 1(1):2–19
Thórisson KR (1997) Gandalf: an embodied humanoid capable of real-time multimodal dialogue with people. In: AGENTS ’97: Proceedings of the first international conference on autonomous agents. ACM Press, New York, NY, pp 536–537
Treisman AM, Gelade G (1980) A feature integration theory of attention. Cogn Psychol 12:97–136
Tu X, Terzopoulos D (1994) Artificial fishes: physics, locomotion, perception, behavior. In: SIGGRAPH ’94: Proceedings of the 21st annual conference on computer graphics and interactive techniques. ACM Press, New York, NY, pp 43–50
van Es I, Heylen D, van Dijk B, Nijholt A (2002) Making agents gaze naturally – does it work? Proceedings of the working conference on advanced visual interfaces, AVI ’02, Trento, Italy, pp 357–358, ACM, New York, NY
Vertegaal R, Ding Y (2002) Explaining effects of eye gaze on mediated group conversations: amount or synchronization? In: Churchill EF, McCarthy J, Neuwirth C, Rodden T (eds), CSCW ’02: proceedings of the 2002 ACM conference on computer supported cooperative work. ACM Press, New York, NY, pp 41–48
Viola P, Jones MJ (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, vol 1. Kauai, Hawaii, pp 511–518, December 8–14, IEEE Computer Society, Los Alamitos, CA
Vosinakis S, Panayiotopoulos T (2003) Programmable agent perception in intelligent virtual environments. In: Rist T, Aylett R, Ballin D, Rickel J (eds) Intelligent agents, 4th international workshop, IVA 2003, Kloster Irsee, Germany, September 15–17, 2003, proceedings, vol 2792 of Lecture notes in computer science. Springer, London, UK
Yarbus A (1967) Eye movements and vision, chapter Eye movements during perception of complex objects. Plenum Press, New York, NY
Zeng Z, Pantic M, Roisman G, Huang T (2009 January) A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Peters, C. et al. (2011). Fundamentals of Agent Perception and Attention Modelling. In: Cowie, R., Pelachaud, C., Petta, P. (eds) Emotion-Oriented Systems. Cognitive Technologies. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15184-2_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-15184-2_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15183-5
Online ISBN: 978-3-642-15184-2
eBook Packages: Computer ScienceComputer Science (R0)