Dialogue Model Specification and Interpretation for Intelligent Multimodal HCI

  • Luis A. Pineda
  • Ivan V. Meza
  • Lisset Salinas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6433)


A methodology and programming environment for the specification and interpretation of dialogue models for grounded multimodal interaction is presented. This conceptual framework permits the declarative specification of complex interactive systems with multimodal input and output, including speech, computer vision and motor behavior. We first introduce the present notion of dialogue model with its motivation on the structure of conversation. Then, the specification and interpretation of dialogue models is presented and discussed. We also present a cognitive architecture for the construction of intelligent Human-Computer Interaction (HCI) applications within this conceptual framework. The paper concludes with references to working systems, demos and work in progress built within the present framework.


Dialogue Models Task Structure Cognitive Architecture HCI 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aguilar, W., Pineda, L.A.: Integrating Graph-Based Vision Perception to Spoken Conversation in Human-Robot Interaction. In: Cabestany, J., et al. (eds.) IWANN 2009. LNCS, vol. 5517, pp. 789–796. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Allen, J., Core, M.: Draft of DAMSL: Dialog Act Markup in Several Layers Annotation Scheme. Department of Computer Science, Rochester University (1997)Google Scholar
  3. 3.
    Avilés, H., Meza, I., Aguilar, W., Pineda, L.A.: Integrating Pointing Gestures into Spanish-Spoken Dialog System for Conversational Service Robots. In: Proceedings of ICAART 2010, Valencia, España, pp. 585–588 (DVD) (2010)Google Scholar
  4. 4.
    Avilés, H., Alvarado-González, M., Venegas, E., Rascón, C., Meza, I., Pineda, L.A.: Development of a Tour-Guide Robot Using Dialogue Models and a Cognitive Architecture. In: Kuri-Morales, A., Simari, G. (eds.) IBERAMIA 2010. LNCS (LNAI), vol. 6433, pp. 512–521. Springer, Heidelberg (2010)Google Scholar
  5. 5.
    Clark, H., Schaefer, E.: Contributing to Discourse. Cognitive Science 13, 259–294Google Scholar
  6. 6.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004) (1989)CrossRefGoogle Scholar
  7. 7.
    Mann, W.C., Thompson, S.: Rhetorical Structure Theory: Towards a functional theory of text organization. Text 8(3), 243–281 (1988)CrossRefGoogle Scholar
  8. 8.
    Meza, I., Pérez-Pavón, P., Salinas, L., Avilés, H., Pineda, L.A.: A Multimodal Dialogue System for Playing the Game “Guess the Card”. Procesamiento del Lenguaje Natural (44), 131–138 (2010)Google Scholar
  9. 9.
    Meza, I., Salinas, L., Venegas, E., Castellanos, H., Chavarria, A., Pineda, L.A.: Specification and Evaluation of a Spanish Conversational System Using Dialogue Models. In: Kuri-Morales, A., Simari, G. (eds.) IBERAMIA 2010. LNCS (LNAI), vol. 6433, pp. 346–355. Springer, Heidelberg (2010)Google Scholar
  10. 10.
    Nakauchi, Y., Naphattalung, P., Takahashi, T., Matsubara, T., Kashiwagi, E.: Proposal and Evaluation of Nat. Lang. Human-Robot Interface System based on Conversation Theory. In: Proceedings of the 2003 IEEE Int. Conf. on Robotics and Automation, Taipei, Taiwan, September 14-19, pp. 380–385 (2003)Google Scholar
  11. 11.
    Pineda, L.A.: Conservation principles and action schemes in the synthesis of geometric concepts. Artificial Intelligence 171(4), 197–238 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Pineda, L.A., Estrada, V., Coria, S., Allen, J.: The Obligations and Common Ground Structure of Practical Dialogues, Inteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial 11(36), 9–17 (2007)Google Scholar
  13. 13.
    Pineda, L.A.: Specification and Interpretation of Multimodal Dialogue Models for Human-Robot Interaction. In: Sidorov, G. (ed.) Artificial Intelligence for Humans: Service Robots and Social Modeling, SMIA, México, pp. 33–50 (2008)Google Scholar
  14. 14.
    Rascón, C., Avilés, H., Pineda, L.A.: Robotic Orientation towards Speaker in Human-Robot Interaction. In: Kuri-Morales, A., Simari, G. (eds.) IBERAMIA 2010. LNCS (LNAI), vol. 6433, pp. 10–19. Springer, Heidelberg (2010)Google Scholar
  15. 15.
    Tulving, E.: Memory systems: episodic and semantic memory. In: Tulving, E., Donaldson, W. (eds.) Organization of Memory, pp. 381–403. Academic Press, New York (1972)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Luis A. Pineda
    • 1
  • Ivan V. Meza
    • 1
  • Lisset Salinas
    • 1
  1. 1.Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas (IIMAS)Universidad Nacional Autónoma de México (UNAM)Mexico

Personalised recommendations