Advertisement

A Multimodal Human-Robot-Dialog Applying Emotional Feedbacks

  • Alexander Bannat
  • Jürgen Blume
  • Jürgen T. Geiger
  • Tobias Rehrl
  • Frank Wallhoff
  • Christoph Mayer
  • Bernd Radig
  • Stefan Sosnowski
  • Kolja Kühnlenz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6414)

Abstract

This paper presents a system for human-robot communication situated in an ambient assisted living scenario, where the robot performs an order-and-serve-procedure. The interaction is based on different modalities that extract information from the auditory and the visual channel in order to obtain an intuitive and natural dialog. The required interaction dialog structure is represented in first-order logic, which allows to split a complex task into simpler subtasks. The different communication modalities are utilized to conclude these subtasks by determining information about the human interaction partner. The system works in real-time and robust and utilizes emotional feedback to enrich the communication process.

Keywords

Facial Expression Ambient Assisted Living Dialog Manager Head Gesture Robotic Head 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Brščić, D., Eggers, M., Rohrmüller, F., Kourakos, O., Sosnowski, S., Althoff, D., Lawitzky, M., Mörtl, A., Rambow, M., Koropouli, V., Medina Hernández, J.R., Zang, X., Wang, W., Wollherr, D., Kühnlenz, K., Mayer, C., Kruse, T., Kirsch, A., Blume, J., Bannat, A., Rehrl, T., Wallhoff, F., Lorenz, T., Basili, P., Lenz, C., Röder, T., Panin, G., Maier, W., Hirche, S., Buss, M., Beetz, M., Radig, B., Schubö, A., Glasauer, S., Knoll, A., Steinbach, E.: Multi Joint Action in CoTeSys - setup and challenges. Technical Report CoTeSys-TR-10-01, CoTeSys Cluster of Excelence: Technische Universität München & Ludwig-Maximilians-Universität München, Munich, Germany (June 2010)Google Scholar
  2. 2.
    Raymond, C., Riccardi, G.: Generative and discriminative algorithms for spoken language understanding. In: Proceedings of the Interspeech Conference, Antwerp, Belgium (2007)Google Scholar
  3. 3.
    Sharma, R., Pavlovic, V.I., Huang, T.S.: Toward multimodal human-computer interface. Proceedings of the IEEE 86, 853–869 (1998)CrossRefGoogle Scholar
  4. 4.
    Oviatt, S.: Multimodal interfaces. In: The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pp. 286–304 (2003)Google Scholar
  5. 5.
    Stiefelhagen, R., Ekenel, H., Fügen, C., Gieselmann, P., Holzapfel, H., Kraft, F., Nickel, K., Voit, M., Waibel, A.: Enabling multimodal human-robot interaction for the karlsruhe humanoid robot. IEEE Transactions on Robotics 23, 840–851 (2007)CrossRefGoogle Scholar
  6. 6.
    Fransen, B., Morariu, V., Martinson, E., Blisard, S., Marge, M., Thomas, S., Schultz, A., Perzanowski, D.: Using vision, acoustics, and natural language for disambiguation. In: HRI 2007: Proceeding of the ACM/IEEE International Conference on Human-Robot Interaction, pp. 73–80. ACM Press, New York (2007)CrossRefGoogle Scholar
  7. 7.
    Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77 (1989)Google Scholar
  8. 8.
    Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1424–1445 (2000)CrossRefGoogle Scholar
  9. 9.
    Cootes, T.F., Taylor, C.J.: Active shape models – smart snakes. In: Proceedings of the 3rd British Machine Vision Conference, pp. 266–275. Springer, Heidelberg (1992)Google Scholar
  10. 10.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998)Google Scholar
  11. 11.
    Ahlberg, J.: Candide-3 – an updated parameterized face. Technical Report LiTH-ISY-R-2326, Linköping University, Sweden (2001)Google Scholar
  12. 12.
    Blanz, V., Scherbaum, K., Seidel, H.P.: Fitting a morphable model to 3d scans of faces. In: Proceedings of International Conference on Computer Vision (2007)Google Scholar
  13. 13.
    Turk, M., Pentland, A.: Face Recognition using Eigenfaces. In: Conference on Computer Vision and Pattern Recognition, pp. 586–591 (1991)Google Scholar
  14. 14.
    Rigoll, G., Kosmala, A., Rottland, J., Neukirchen, C.: A Comparison Between Continuous and Discrete Density Hidden Markov Models for Cursive Handwriting Recognition. In: International Conference on Pattern Recognition (ICPR), Vienna, Austria, August 1996, vol. 2, pp. 205–209 (1996)Google Scholar
  15. 15.
    Neukirchen, C., Rigoll, G.: Advanced Training Methods and New Network Topologies for Hybrid MMI-Connectionist/HMM Speech Recognition Systems. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Munich, Germany, April 1997, pp. 3257–3260 (1997)Google Scholar
  16. 16.
    Goebl, M., Färber, G.: A real-time-capable hard- and software architecture for joint image and knowledge processing in cognitive automobiles. In: Intelligent Vehicles Symposium, pp. 737–740 (June 2007)Google Scholar
  17. 17.
    Viola, P., Jones, M.J.: Robust real-time face detection. International Journal of Computer Vision (2004)Google Scholar
  18. 18.
    Gast, J., Bannat, A., Rehrl, T., Rigoll, G., Wallhoff, F., Mayer, C., Radig, B.: Did I Get It Right: Head Gestures Analysis for Human-Machine Interactions. In: Human-Computer Interaction. Novel Interaction Methods and Techniques, pp. 170–177.Google Scholar
  19. 19.
    Altarriba, J., Basnight, D.M., Canary, T.M.: Emotion representation and perception across cultures. Online Readings in Psychology and Culture (2003)Google Scholar
  20. 20.
    Ekman, P., Friesen, W.V.: Facial Action Coding Consulting. Psychologist Press, San Diego (1977)Google Scholar
  21. 21.
    e-learning-Kurs ”‘about facs’” (2003), http://www.uni-saarland.de/fak5/orga/Kurs/home.htm
  22. 22.
    Mayer, K.K.C., Sosnowski, S., Radig, B.: Towards robotic facial mimicry: system development and evaluation. In: 19th IEEE International Symposium in Robot and Human Interactive Communication, Speccial Session on Cognition for Interactive Robots (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Alexander Bannat
    • 1
  • Jürgen Blume
    • 1
  • Jürgen T. Geiger
    • 1
  • Tobias Rehrl
    • 1
  • Frank Wallhoff
    • 1
    • 4
  • Christoph Mayer
    • 2
  • Bernd Radig
    • 2
  • Stefan Sosnowski
    • 3
  • Kolja Kühnlenz
    • 3
  1. 1.Human-Machine Communication, Department of Electrical Engineering and Information TechnologiesTechnische Universität MünchenMunichGermany
  2. 2.Image Understanding and Knowledge-Based Systems, Department of InformaticsTechnische Universität MünchenMunichGermany
  3. 3.Institute of Automatic Control Engineering, Department of Electrical Engineering and Information TechnologiesTechnische Universität MünchenMunichGermany
  4. 4.Jade University of Applied SciencesOldenburgGermany

Personalised recommendations