Abstract
An E-Partner is a photo-realistic conversation agent, which has a talking head that not only look photo-realistic but also can have a conversation with the user about a given topic. The conversation is multimedia-enriched in that the E-Partner presents relevant multimedia materials throughout the conversation. To address the challenges presented by the complex conversation domain and task, and to achieve adaptive behaviors, we have derived a novel dialogue manager design consisting of five parts: a domain model, a dialogue model, a discourse model, a task model, and a user model. We also extended existing facial animation techniques to create photo-realistic talking heads that facilitate conversational interactions. Some practical issues like how to handle the uncertainty from speech level are also discussed.
This work was performed while the authors were visiting Microsoft Research China.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Black, M.J., Yacoob, Y: Tracking and recognizing rigid and non-rigid facial motion using local parametric models of image motion. In Proceedings of IEEE Intl. Conf. Computer Vision, Cambridge, MA, 374–381, 1995.
Breglar, C., Covell, M., Slaney, M.: Video rewrite: Driving visual speech with audio. In Proceedings of SIGGRAPH’97, 353–360, July 1997.
Chu-Carroll, J.: Form-based reasoning for mixed-initiative dialogue management in information-query systems. In Proceedings of Eurospeech’99, 1519–1522, 1999.
Cossatto, E., Graf, H. P.: Photo-realistic talking-heads from image samples. IEEE Trans. on Multimedia, 2(3), September 2000.
Ferguson, G., Allen, J.: TRIPS: An Intelligent Integrated Problem-Solving Assistant. In Proceedings of the Fifteenth National Conference on Artificial Intelligence(AAAI-98), Madison, WI, 567–573, July 1998.
Flycht-Eriksson, A.: A survey of knowledge sources in dialogue systems. In Proceedings of IJCAI-99 Workshop on Knowledge and Reasoning in Practical Dialogue Systems, Stockholm, 1999.
Pandzic, I., Ostermann, J., Millen, D.: User evaluation: synthetic talking faces for interactive services. The Visual Computer, 15:330–340, 1999.
Schodl, A., Szeliski, R.: Video textures. In Proceedings of SIGGRAPH’99, 1999.
Wang, Y.: A robust parser for spoken language understanding. In Proceedings of Eurospeech’99, 1999.
Weizenbaum, J.: ELIZA-a computer program for the study of natural language communication between man and machine. C. ACM, 9:36–43, 1966.
Zue, V. et al.: JUPITER: A telephone-based conversational interface for weather information. IEEE Transactions on Speech and Audio Processing, 8(1), January 2000.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, B., Hu, C., Cai, Q., Guo, B., Shum, H. (2001). E-Partner: A Photo-Realistic Conversation Agent. In: Shum, HY., Liao, M., Chang, SF. (eds) Advances in Multimedia Information Processing — PCM 2001. PCM 2001. Lecture Notes in Computer Science, vol 2195. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45453-5_34
Download citation
DOI: https://doi.org/10.1007/3-540-45453-5_34
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42680-6
Online ISBN: 978-3-540-45453-3
eBook Packages: Springer Book Archive