Integrating Planning and Dialogue in a Lifestyle Agent

  • Cameron Smith
  • Marc Cavazza
  • Daniel Charlton
  • Li Zhang
  • Markku Turunen
  • Jaakko Hakulinen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5208)


In this paper, we describe an Embodied Conversational Agent advising users to promote a healthier lifestyle. This embodied agent provides advice on everyday user activities, in order to promote a healthy lifestyle. It operates by generating user activity models (similar to decompositional task models), using a Hierarchical Task Network (HTN) planner. These activity models are refined through various cycles of planning and dialogue, during which the agent suggests possible activities to the user, and the user expresses her preferences in return. A first prototype has been fully implemented (as a spoken dialogue system) and tested with 20 subjects. Early results show a high level of task completion despite the word error rate, and further potential for improvement.


Multimodal interaction with intelligent virtual agents Embodied Cognitive Modelling Conversational and non-verbal behavior 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allen, J., Chambers, N., Ferguson, G., Galescu, L., Jung, H., Swift, M., Taysom, W.: PLOW: A Collaborative Task Learning Agent. In: AAAI 2007 Proceedings, Vancouver, British Columbia, pp. 1514–1519 (2007)Google Scholar
  2. 2.
    Allen, J., Ferguson, G., Blaylock, N., Byron, D., Chambers, N., Dzikovska, M., Galescu, L., Swift, M.: Chester: Towards a Personal Medication Advisor. Journal of Biomedical Informatics 39(5), 500–513 (2006)CrossRefGoogle Scholar
  3. 3.
    Allen, J., Ferguson, G., Swift, M., Stent, A., Stoness, S., Galescu, L., Chambers, N., Campana, E., Aist, G.: Two Diverse Systems Built using Generic Components for Spoken Dialogue (Recent Progress on TRIPS). In: ACL 2005 Proceedings of the Interactive Poster and Demonstration Sessions, pp. 85–88. Ann Arbor Michigan (2005)Google Scholar
  4. 4.
    Bickmore, T.W., Sidner, C.L.: Towards Plan-Based Health Behavior Change Counseling Systems. In: Proceedings of AAAI Spring Symposium on Argumentation for Consumers of Healthcare, Stanford, CA, pp. 14–18 (2006)Google Scholar
  5. 5.
    Bohus, D., Rudnicky, A.I.: RavenClaw: Dialog Management using Hierarchical Task Decomposition and an Expectation Agenda. In: Proceedings of Eurospeech 2003, pp. 597–600 (2003)Google Scholar
  6. 6.
    De Rosis, F., Novielli, N., Carofiglio, V., Cavalluzzi, A., De Carolis, B.: User modeling and adaptation in health promotion dialogs with an animated character. Journal of Biomedical Informatics 39(5), 514–531 (2006)CrossRefGoogle Scholar
  7. 7.
    Frokjaer, E., Hertzum, M., Hornbaek, K.: Measuring usability: Are effectiveness, efficiency, and satisfaction really correlated. In: Proceedings of CHI 2000 Conference, pp. 345–352. ACM Press, New York (2000)Google Scholar
  8. 8.
    Ghallab, M., Nau, D., Traverso, P.: Automated Planning: Theory and Practice. Morgan Kaufmann, San Francisco (2004)MATHGoogle Scholar
  9. 9.
    Gruenstein, A., Cavedon, L.: Using an activity model to address issues in task-oriented dialogue interaction over extended periods. In: AAAI Spring Symposium on Interaction between Humans and Autonomous Systems over Extended Operation, March 2004, AAAI Press, Menlo Park (2004)Google Scholar
  10. 10.
    Kim, H.-S., Gratch, J.: A planner-independent collaborative planning assistant. In: AAMAS 2004 Proceedings, pp. 764–771 (2004)Google Scholar
  11. 11.
    Leuski, A., Kennedy, B., Patel, R., Traum, D.: Asking Questions to Limited Domain Virtual Characters: How Good Does Speech Recognition have to be? In: 25th Army Science Conference, Orlando, Florida (2006)Google Scholar
  12. 12.
    Nau, D.S., Smith, S.J.J., Erol, K.: Control Strategies in HTN Planning: Theory versus Practice. In: AAAI 1998/IAAI 1998 Proceedings, pp. 1127–1133 (1998)Google Scholar
  13. 13.
    Paek, T.: Empirical methods for evaluating dialog systems. In: ACL 2001 Workshop on Evaluation Methodologies for Language and Dialogue Systems 2001, pp. 3–10 (2001)Google Scholar
  14. 14.
    Rich, C., Sidner, C.L.: COLLAGEN: A Collaboration Manager for Software Interface Agents. User Modeling and User-Adapted Interaction 8(3-4) (1998)Google Scholar
  15. 15.
    Traum, D., Rickel, J., Gratch, J., Marsella, S.: Negotiation over Tasks in Hybrid Human-Agent Teams for Simulation-Based Training. In: AAMAS 2003 Proceedings, Melbourne, Australia, July 2003, pp. 441–448 (2003)Google Scholar
  16. 16.
    Traum, D.R., Robinson, S., Stephan, J.: Evaluation of multi-party virtual reality dialogue interaction. In: LREC 2004 Proceedings, pp. 1699–1702 (2004)Google Scholar
  17. 17.
    Turunen, M., Salonen, E.-P., Hartikainen, M., Hakulinen, J.: Robust and Adaptive Architecture for Multilingual Spoken Dialogue Systems. In: Proceedings of ICSLP 2004 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Cameron Smith
    • 1
  • Marc Cavazza
    • 1
  • Daniel Charlton
    • 1
  • Li Zhang
    • 1
  • Markku Turunen
    • 2
  • Jaakko Hakulinen
    • 2
  1. 1.School of ComputingUniversity of TeessideMiddlesbroughUnited Kingdom
  2. 2.Department of Computer Sciences, Speech-based and Pervasive Interaction GroupUniversity of TampereFinland

Personalised recommendations