Despite advances in enhancing the expressivity of embodied conversational agents (ECAs), communication with ECAs remains an unnatural experience due to an often encountered asymmetry in communication channels, in which user input modalities tend to be less sophisticated due to the limitation of input processing. This is becoming a limiting factor as virtual actors become an essential element of computer games, intelligent tutoring systems and new media such as Interactive Storytelling. Furthermore, user-agent communication should be properly situated in the context of the application at hand rather than isolated as a laboratory experiment. In addition, interaction with ECAs implicitly creates a social environment which requires mechanisms for recognizing the user’s affective, attentive and social cues. Nevertheless, research on the integration of speech, vision and other non-verbal sensors into an ECA’s conversational model is still rare compared to the enormous amount of work that has been done on the planning and rendering of ECA behaviors.

This special issue presents four articles that propose different ways to improve the interaction capabilities of ECAs. The first two papers focus on dialogue management for ECA systems. The paper by Gebhard and colleagues describes an authoring tool to create interactive performances with ECAs in a rapid prototyping style. The approach supports not only the definition of dialogue content and narrative structure, but also accounts for multiple character modeling. Crook and colleagues present a variety of context-sensitive strategies for handling user barge-ins in an ECA system. The approach is based on a classification of interruptions and resumption strategies that was informed by empirical data collected for human-human dialogue. The next two papers investigate how an agent’s interaction behavior may be adapted to individual characteristics of the agent, such as its personality or cultural background. The paper by Bevacqua and colleagues presents a computational model that generates appropriate backchannel signals for a listing agent according to the user’s visual and acoustic behavior and the agent’s assumed personality traits. The paper by Kistler and colleagues combines an approach to the generation of culture-specific ECA behaviors with full-body avatar control based on the Kinect sensor.

The guest editors wish to thank the reviewers for their insightful comments and help in preparing this special issue.