Abstract
We present here a system for controlling the eye gaze of a virtual embodied conversational agent able to perceive the physical environment in which it interacts. This system is inspired by known components of human visual attention system and reproduces its limitations in terms of visual acuity, sensitivity to movement, limitations of short-memory and object pursuit. The aim of this coupling between animation and visual scene analysis is to provide sense of presence and mutual attention to human interlocutors. After a brief introduction to this research project and a focused state of the art, we detail the components of our system and confront simulation results to eye gaze data collected from viewers observing the same natural scenes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alami, R., Clodic, A., Montreuil, V., Sisbot, E.A., Chatila, R.: Toward human-aware robot task planning. In: AAAI Spring Symposium. To boldly go where no human-robot team has gone before (2006)
Bailly, G., Elisei, F., Badin, P., Savariaux, C.: Degrees of freedom of facial movements in face-to-face conversational speech. In: International Workshop on Multimodal Corpora, Genoa - Italy (2006)
Bailly, G., Elisei, F., Raidt, S., Casari, A., Picot, A.: Embodied conversational agents: computing and rendering realistic gaze patterns. In: Pacific Rim Conference on Multimedia Processing, Hangzhou (2006)
Bilvi, M., Pelachaud, C.: Communicative and statistical eye gaze predictions. In: AAMAS. International conference on Autonomous Agents and Multi-Agent Systems, Melbourne, Australia (2003)
Breazeal, C.: Designing Sociable Robots. MIT Press, Cambridge (2002)
Brooks, R.A., Breazeal, C., Marjanovic, M., Scassellati, B., Williamson, M.: The Cog Project: Building a Humanoid Robot. In: Nehaniv, C.L. (ed.) Computation for Metaphors, Analogy, and Agents. LNCS (LNAI), vol. 1562, pp. 52–87. Springer, Heidelberg (1999)
Casari, A., Elisei, F., Bailly, G., Raidt, S.: Contrôle du regard et des mouvements des paupières d’une tête parlante virtuelle. in Workshop sur les Agents Conversationnels Animés, Toulouse - France (2006)
Chopra-Khullar, S., Badler, N.I.: Where to look? Automating attending behaviors of virtual human characters. In: Annual Conference on Autonomous Agents, New York (1999)
Clodic, A., Fleury, S., Alami, R., Chatila, R., Bailly, G., Brèthes, L., Cottret, M., Danès, P., Dollat, X., Elisei, F., Ferrané, I., Herrb, M.: Rackham: an interactive robot-guide. In: IEEE International Workshop on Robots and Human Interactive Communications, Hatfield, UK (2006)
Courty, N.: Animation référencée vision : de la tâche au comportement. In IRISA 2002, INSA: Rennes, p. 198 (2002)
Godijn, R., Theeuwes, J.: The relationship between exognenous and endogenous saccades and attention. In: Hyönä, J., Radach, R., Deubel, H. (eds.) The mind’s eye: cognitive and applied aspects of eye movement research, pp. 3–26. North-Holland, Amsterdam (2003)
Gu, E., Badler, N.I.: Visual attention and eye gaze during multipartite conversations with distractions. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 193–204. Springer, Heidelberg (2006)
Itti, L., Dhavale, N., Pighin, F.: Realistic avatar eye and head animation using a neurobiological model of visual attention. In: SPIE 48th Annual International Symposium on Optical Science and Technology, San Diego, CA (2003)
Jeannerod, M.: The cognitive neuroscience of action, p. 236. Blackwell, Oxford, UK (1997)
Jiang, X., Binkert, M., Achermann, B., Bunke, H.: Detection of glasses in facial images. In: Jiang, X. (ed.) Asian Conference on Computer Vision, Hong Kong - China (1998)
Lee, S.P., Badler, J.B., Badler, N.: Eyes alive. ACM Transaction on Graphics 21(3), 637–644 (2002)
Lienhart, R., Maydt, J.: An extended set of haar-like features for rapid object detection. In: IEEE International Conference on Image Processing, Rochester - NY, IEEE Computer Society Press, Los Alamitos (2002)
Matsusaka, Y., Tojo, T., Kobayashi, T.: Conversation robot participating in group conversation. IEICE Transaction of Information and System 1, 26–36 (2003)
Mishkin, M., Ungerleider, L.G., Macko, K.A.: Object vision and spatial vision: two cortical pathways. Trends in Neuroscience 6, 414–417 (1983)
Navalpakkam, V., Itti, L.: Modeling the influence of task on attention. Vision Research 45(2), 205–231 (2005)
Perrett, D., Rolls, E., Caan, W.: Visual neurones responsive to faces in the monkey temporal cortex. Exp Brain Research 47, 329–342 (1982)
Peters, C., O’Sullivan, C.: Bottom-up visual attention for virtual human animation. In: Computer Animation and Social Agents, Rutgers University, New York (2003)
Raidt, S., Bailly, G., Elisei, F.: Does a virtual talking face generate proper multimodal cues to draw user’s attention towards interest points? In: Raidt, S. (ed.) Language Ressources and Evaluation Conference (LREC), Genova, Italy (2006)
Simons, D.J., Chabris, C.F.: Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception 28(9), 1059–1074 (1999)
Sun, Y.: Hierarchical object-based visual attention for machine vision, in Institute of Perception, Action and Behaviour. School of Informatics. University of Edinburgh, Edinburgh, p. 169 (2003)
Vatikiotis-Bateson, E., Eigsti, I.-M., Yano, S., Munhall, K.G.: Eye movement of perceivers during audiovisual speech perception. Perception & Psychophysics 60, 926–940 (1998)
Vergilino-Perez, D., Collins, T., Dore-Mazars, K.: Decision and metrics of refixations in reading isolated words. Vision research 44(17), 2009–2017 (2004)
Yarbus, A.L.: Eye movements during perception of complex objects. In: Riggs, L.A. (ed.) Eye Movements and Vision, pp. 171–196. Plenum Press, New York (1967)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Picot, A., Bailly, G., Elisei, F., Raidt, S. (2007). Scrutinizing Natural Scenes: Controlling the Gaze of an Embodied Conversational Agent. In: Pelachaud, C., Martin, JC., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds) Intelligent Virtual Agents. IVA 2007. Lecture Notes in Computer Science(), vol 4722. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74997-4_25
Download citation
DOI: https://doi.org/10.1007/978-3-540-74997-4_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74996-7
Online ISBN: 978-3-540-74997-4
eBook Packages: Computer ScienceComputer Science (R0)