Abstract
This paper presents a system for human-robot communication situated in an ambient assisted living scenario, where the robot performs an order-and-serve-procedure. The interaction is based on different modalities that extract information from the auditory and the visual channel in order to obtain an intuitive and natural dialog. The required interaction dialog structure is represented in first-order logic, which allows to split a complex task into simpler subtasks. The different communication modalities are utilized to conclude these subtasks by determining information about the human interaction partner. The system works in real-time and robust and utilizes emotional feedback to enrich the communication process.
All authors contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Brščić, D., Eggers, M., Rohrmüller, F., Kourakos, O., Sosnowski, S., Althoff, D., Lawitzky, M., Mörtl, A., Rambow, M., Koropouli, V., Medina Hernández, J.R., Zang, X., Wang, W., Wollherr, D., Kühnlenz, K., Mayer, C., Kruse, T., Kirsch, A., Blume, J., Bannat, A., Rehrl, T., Wallhoff, F., Lorenz, T., Basili, P., Lenz, C., Röder, T., Panin, G., Maier, W., Hirche, S., Buss, M., Beetz, M., Radig, B., Schubö, A., Glasauer, S., Knoll, A., Steinbach, E.: Multi Joint Action in CoTeSys - setup and challenges. Technical Report CoTeSys-TR-10-01, CoTeSys Cluster of Excelence: Technische Universität München & Ludwig-Maximilians-Universität München, Munich, Germany (June 2010)
Raymond, C., Riccardi, G.: Generative and discriminative algorithms for spoken language understanding. In: Proceedings of the Interspeech Conference, Antwerp, Belgium (2007)
Sharma, R., Pavlovic, V.I., Huang, T.S.: Toward multimodal human-computer interface. Proceedings of the IEEE 86, 853–869 (1998)
Oviatt, S.: Multimodal interfaces. In: The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pp. 286–304 (2003)
Stiefelhagen, R., Ekenel, H., Fügen, C., Gieselmann, P., Holzapfel, H., Kraft, F., Nickel, K., Voit, M., Waibel, A.: Enabling multimodal human-robot interaction for the karlsruhe humanoid robot. IEEE Transactions on Robotics 23, 840–851 (2007)
Fransen, B., Morariu, V., Martinson, E., Blisard, S., Marge, M., Thomas, S., Schultz, A., Perzanowski, D.: Using vision, acoustics, and natural language for disambiguation. In: HRI 2007: Proceeding of the ACM/IEEE International Conference on Human-Robot Interaction, pp. 73–80. ACM Press, New York (2007)
Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77 (1989)
Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1424–1445 (2000)
Cootes, T.F., Taylor, C.J.: Active shape models – smart snakes. In: Proceedings of the 3rd British Machine Vision Conference, pp. 266–275. Springer, Heidelberg (1992)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998)
Ahlberg, J.: Candide-3 – an updated parameterized face. Technical Report LiTH-ISY-R-2326, Linköping University, Sweden (2001)
Blanz, V., Scherbaum, K., Seidel, H.P.: Fitting a morphable model to 3d scans of faces. In: Proceedings of International Conference on Computer Vision (2007)
Turk, M., Pentland, A.: Face Recognition using Eigenfaces. In: Conference on Computer Vision and Pattern Recognition, pp. 586–591 (1991)
Rigoll, G., Kosmala, A., Rottland, J., Neukirchen, C.: A Comparison Between Continuous and Discrete Density Hidden Markov Models for Cursive Handwriting Recognition. In: International Conference on Pattern Recognition (ICPR), Vienna, Austria, August 1996, vol. 2, pp. 205–209 (1996)
Neukirchen, C., Rigoll, G.: Advanced Training Methods and New Network Topologies for Hybrid MMI-Connectionist/HMM Speech Recognition Systems. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Munich, Germany, April 1997, pp. 3257–3260 (1997)
Goebl, M., Färber, G.: A real-time-capable hard- and software architecture for joint image and knowledge processing in cognitive automobiles. In: Intelligent Vehicles Symposium, pp. 737–740 (June 2007)
Viola, P., Jones, M.J.: Robust real-time face detection. International Journal of Computer Vision (2004)
Gast, J., Bannat, A., Rehrl, T., Rigoll, G., Wallhoff, F., Mayer, C., Radig, B.: Did I Get It Right: Head Gestures Analysis for Human-Machine Interactions. In: Human-Computer Interaction. Novel Interaction Methods and Techniques, pp. 170–177.
Altarriba, J., Basnight, D.M., Canary, T.M.: Emotion representation and perception across cultures. Online Readings in Psychology and Culture (2003)
Ekman, P., Friesen, W.V.: Facial Action Coding Consulting. Psychologist Press, San Diego (1977)
e-learning-Kurs ”‘about facs’” (2003), http://www.uni-saarland.de/fak5/orga/Kurs/home.htm
Mayer, K.K.C., Sosnowski, S., Radig, B.: Towards robotic facial mimicry: system development and evaluation. In: 19th IEEE International Symposium in Robot and Human Interactive Communication, Speccial Session on Cognition for Interactive Robots (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bannat, A. et al. (2010). A Multimodal Human-Robot-Dialog Applying Emotional Feedbacks. In: Ge, S.S., Li, H., Cabibihan, JJ., Tan, Y.K. (eds) Social Robotics. ICSR 2010. Lecture Notes in Computer Science(), vol 6414. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17248-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-17248-9_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17247-2
Online ISBN: 978-3-642-17248-9
eBook Packages: Computer ScienceComputer Science (R0)