Abstract
Developing a robot system that can interact directly with a human instructor in a natural way requires not only highly-skilled sensorimotor coordination and action planning on the part of the robot, but also the ability to understand and communicate with a human being in many modalities. A typical application of such a system is interactive assembly for construction tasks. A human communicator sharing a common view of the work area with the robot system instructs the latter by speaking to it in the same way that he would communicate with a human partner.
Chapter PDF
References
Erlhagen, W., Mukovskiy, A., Bicho, E., Panin, G., Kiss, C., Knoll, A., van Schie, H., Bekkering, H.: Goal-directed imitation for robots: a bio-inspired approach to action understanding and skill learning. Robotics and Autonomous Systems 54(5), 353–360 (2006)
Foster, M.E., By, T., Rickert, M., Knoll, A.: Human-robot dialogue for joint construction tasks. In: Proceedings of the International Conference on Multimodal Interfaces, pp. 68–71. ACM Press, New York (2006)
van Breemen, A.J.N., Yan, X., Meerbeek, B.: iCat: An animated user-interface robot with personality. In: Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 143–144. ACM Press, New York (2005)
Knoll, A., Hildebrandt, B., Zhang, J.: Instructing cooperating assembly robots through situated dialogues in natural language. In: Proceedings of the IEEE International Conference on Robotics and Automation, IEEE Computer Society Press, Los Alamitos (1997)
Zhang, J., Knoll, A.: A two-arm situated artificial communicator for human-robot cooperative assembly. In: Proceedings of the IEEE International Workshop on Human Robot Communication, pp. 292–299 (2001)
Knoll, A.: basic system for multimodal robot instruction. In: Kühnlein, P., Rieser, H., Zeevat, H. (eds.) Perspectives on Dialogue in the New Millennium. Pragmatics and Beyond New Series, vol. 114, John Benjamins, Amsterdam (2003)
Knoll, A.: Distributed contract networks of sensor agents with adaptive reconfiguration: Modelling, simulation, implementation. Journal of the Franklin Institute 338(6), 669–705 (2001)
Breazeal, C., Brooks, A., Gray, J., Hoffman, G., Kidd, C., Lee, H., Lieberman, J., Lockerd, A., Chilongo, D.: Tutelage and collaboration for humanoid robots. International Journal of Humanoid Robotics 1(2), 315–348 (2004)
Sidner, C.L., Dzikovska, M.: first experiment in engagement for human-robot interaction in hosting activities. In: Bernsen, N., Dybkjær, L., van Kuppevelt, J. (eds.) Advances in Natural Multimodal Dialogue Systems, Springer, Heidelberg (2005)
Fong, T.W., Kunz, C., Hiatt, L., Bugajska, M.: The human-robot interaction operating system. In: Proceedings of the International Conference on Human-Robot Interaction, ACM (2006)
Burghart, C., Mikut, R., Stiefelhagen, R., Asfour, T., Holzapfel, H., Steinhaus, P., Dillmann, R.: cognitive architecture for humanoid robot: first approach. In: Proceedings of the IEEE-RAS International Conference on Humanoid Robots, pp. 357–362 (2005)
Henning, M.: new approach to object-oriented middleware. IEEE Internet Computing 8(1), 66–75 (2004)
Nuance Communications, Inc.: Dragon NaturallySpeaking, http://www.nuance.com/
Panin, G., Ladikos, A., Knoll, A.: An efficient and robust real-time contour tracking system. In: Proceedings of IEEE International Conference on Computer Vision Systems, vol. 44, IEEE Computer Society, Los Alamitos (2006)
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: Proceedings of the International Conference on Computer Graphics and Interactive Techniques, pp. 75–84. ACM Press, New York (1998)
Moddemeijer, R.: On estimation of entropy and mutual information of continuous distributions. Signal Processing 16(3), 233–246 (1989)
Steedman, M.: The Syntactic Process. MIT Press, Cambridge, MA (2000)
White, M.: Efficient realization of coordinate structures in combinatory categorial grammar. Research on Language and Computation 4(1), 39–75 (2006)
Morrow, J.D., Khosla, P.K.: Manipulation task primitives for composing robot skills. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3354–3359 (1997)
Zhang, J., von Collani, Y., Knoll, A.: Interactive assembly by a two-arm robot agent. Journal of Robotics and Autonomous Systems 29(1), 91–100 (1999)
Thomas, U., Finkemeyer, B., Kröger, T., Wahl, F.: Error-tolerant execution of complex robot tasks based on skill primitives. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3069–3075 (2003)
Milighetti, G., Kuntze, H.-B.: Multi-sensor controlled skills for humanoid robots. In: Proceedings of the IFAC International Symposium on Robot Control (2006)
Jilka, M., Syrdal, A.K.: The AT&T German text-to-speech system: Realistic linguistic description. In: Proceedings of the International Conference on Spoken Language Processing, pp. 113–116 (2002)
Sidner, C.L., Lee, C., Kidd, C.D., Lesh, N., Rich, C.: Explorations in engagement for humans and robots. Artificial Intelligence 166(1-2), 140–164 (2005)
White, M., Foster, M.E., Oberlander, J., Brown, A.: Using facial feedback to enhance turn-taking in multimodal dialogue system. In: Proceedings of the International Conference on Human-Computer Interaction (2005)
Traum, D., Larsson, S.: The information state approach to dialogue management. In: Smith, R.W., van Kuppevelt, J. (eds.) Current and New Directions in Discourse and Dialogue, Kluwer Academic Publishers, Dordrecht (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rickert, M., Foster, M.E., Giuliani, M., By, T., Panin, G., Knoll, A. (2007). Integrating Language, Vision and Action for Human Robot Dialog Systems. In: Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Ambient Interaction. UAHCI 2007. Lecture Notes in Computer Science, vol 4555. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73281-5_108
Download citation
DOI: https://doi.org/10.1007/978-3-540-73281-5_108
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73280-8
Online ISBN: 978-3-540-73281-5
eBook Packages: Computer ScienceComputer Science (R0)