Separating Representation, Reasoning, and Implementation for Interaction Management: Lessons from Automated Planning

  • Mary Ellen Foster
  • Ronald P. A. Petrick
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 427)


Numerous toolkits are available for developing speech-based dialogue systems. We survey a range of currently available toolkits, highlighting the different facilities provided by each. Most of these toolkits include not only a method for representing states and actions, but also a mechanism for reasoning about and selecting the actions, often combined with a technical framework designed to simplify the task of creating end-to-end systems. This near-universal tight coupling of representation, reasoning, and implementation in a single toolkit makes it difficult both to compare different approaches to dialogue system design, as well as to analyse the properties of individual techniques. We contrast this situation with the state of the art in a related research area—automated planning—where a set of common representations have been defined and are widely used to enable direct comparison of different reasoning approaches. We argue that adopting a similar separation would greatly benefit the dialogue research community.


Interaction management Automated planning Representation and reasoning Systems integration 



This research has been partially funded by the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant no. 270435 (JAMES, and grant no. 610917 (STAMINA,, and by the European Union’s Horizon 2020 research and innovation programme under grant no. 688147 (MuMMER,


  1. 1.
    Bui, T.H.: Multimodal dialogue management—state of the art. Technical Report 06–01, University of Twente (UT), Enschede, The Netherlands (2006)Google Scholar
  2. 2.
    Asher, N., Lascarides, A.: Logics of Conversation. Cambridge University Press (2003)Google Scholar
  3. 3.
    Jokinen, K., McTear, M.: Spoken Dialogue Systems. Morgan & Claypool (2009)Google Scholar
  4. 4.
    McTear, M., Callejas, Z., Griol, D.: The Conversational Interface. Springer International Publishing (2016)Google Scholar
  5. 5.
    Peltason, J., Wrede, B.: The curious robot as a case-study for comparing dialog systems. AI Mag. 32(4), 85–99 (2011)Google Scholar
  6. 6.
    Olaso, J.M., Milhorat, P., Himmelsbach, J., Boudy, J., Chollet, G., Schlögl, S., Torres, M.I.: A multi-lingual evaluation of the vAssist spoken dialog system: comparing Disco and RavenClaw. In: Jokinen, K. & Wilcock, G. (Eds.) Dialogues with Social Robots, Springer pp.221–238 (this volume) (2016)Google Scholar
  7. 7.
    Coles, A., Coles, A., García Olaya, A., Jiménez, S., Linares López, C., Sanner, S., Yoon, S.: A survey of the seventh international planning competition. AI Mag. 33(1), 83–88 (2012)Google Scholar
  8. 8.
    Gat, E.: Three-layered architectures. In: AI-Based Mobile Robots: Case Studies of Successful Robot Systems. MIT Press (1998)Google Scholar
  9. 9.
    Dumas, B., Lalanne, D., Oviatt, S.: Multimodal interfaces: a survey of principles, models and frameworks. In: Human Machine Interaction, Lecture Notes in Computer Science, vol. 5440, pp. 3–26 (2009)Google Scholar
  10. 10.
    Atrey, P.K., Hossain, M.A., El Saddik, A., Kankanhalli, M.S.: Multimodal fusion for multimedia analysis: a survey. Multimedia Syst. 16(6), 345–379 (2010)CrossRefGoogle Scholar
  11. 11.
    Foster, M.E.: State of the art review: multimodal fission. Deliverable 6.1, COMIC project (2002)Google Scholar
  12. 12.
    Larsson, S., Traum, D.R.: Information state and dialogue management in the TRINDI dialogue move engine toolkit. Nat. Lang. Eng. 6(3&4), 323–340 (2000)CrossRefGoogle Scholar
  13. 13.
    Bos, J., Klein, E., Lemon, O., Oka, T.: DIPPER: Description and formalisation of an information-state update dialogue system architecture. In: Proceedings of SIGdial, pp. 115–124 (2003)Google Scholar
  14. 14.
    Martin, D.L., Cheyer, A.J., Moran, D.B.: The open agent architecture: a framework for building distributed software systems. Appl. Artif. Intell. 13(1–2), 91–128 (1999)CrossRefGoogle Scholar
  15. 15.
    Johnston, M., Bangalore, S., Vasireddy, G., Stent, A., Ehlen, P., Walker, M., Whittaker, S., Maloor, P.: MATCH: an architecture for multimodal dialogue systems. In: Proceedings of ACL, pp. 376–383, Philadelphia, Pennsylvania, USA (2002)Google Scholar
  16. 16.
    ter Maat, M., Heylen, D.: Flipper: an information state component for spoken dialogue systems. In: Intelligent Virtual Agents. Lecture Notes in Computer Science, vol. 6895, pp. 470–472. Springer, Berlin (2011)Google Scholar
  17. 17.
    Janarthanam, S., Hastie, H., Deshmukh, A., Aylett, R., Foster, M.E.: A reusable interaction management module: use case for empathic robotic tutoring. In: Proceedings of goDIAL, Gothenburg, Sweden (2015)Google Scholar
  18. 18.
    Bohus, D., Rudnicky, A.I.: The RavenClaw dialog management framework: architecture and systems. Comput. Speech Lang. 23(3), 332–361 (2009)CrossRefGoogle Scholar
  19. 19.
    Rich, C., Sidner, C.L.: COLLAGEN: a collaboration manager for software interface agents. User Model. User-Adap. Inter. 8(3–4), 315–350 (1998)CrossRefGoogle Scholar
  20. 20.
    Rich, C., Sidner, C.L.: Using collaborative discourse theory to partially automate dialogue tree authoring. In: Intelligent Virtual Agents, Lecture Notes in Computer Science, vol. 7502, pp. 327–340 (2012)Google Scholar
  21. 21.
    Lison, P.: A hybrid approach to dialogue management based on probabilistic rules. Comput. Speech Lang. (2015)Google Scholar
  22. 22.
    Skantze, G., Al Moubayed, S.: IrisTK: a statechart-based toolkit for multi-party face-to-face interaction. In: Proceedings of ICMI, pp. 69–76 (2012)Google Scholar
  23. 23.
    Al Moubayed, S., Beskow, J., Skantze, G., Granström, B.: Furhat: a back-projected human-like robot head for multiparty human-machine interaction. In: Cognitive Behavioural Systems, Lecture Notes in Computer Science, vol. 7403, pp. 114–130 (2012)Google Scholar
  24. 24.
    Harel, D.:Statecharts: a visual formalism for complex systems. Sci. Comput. Program. 8(3), 231–274 (1987). ISSN 0167-6423.
  25. 25.
    Baumann, T., Schlangen, D.: The InproTK 2012 release. In: Proceedings of the NAACL-HLT Workshop on Future directions and needs in the Spoken Dialog Community: Tools and Data, pp. 29–32 (2012).
  26. 26.
    Kennington, C., Kousidis, S., Schlangen, D.: InproTKs: a toolkit for incremental situated processing. In: Proceedings of SIGdial, pp. 84–88 (2014)Google Scholar
  27. 27.
    Wienke, J., Wrede, S.: A middleware for collaborative research in experimental robotics. In: Proceedings of the 2011 IEEE/SICE International Symposium on System Integration, pp. 1183–1190 (2011)Google Scholar
  28. 28.
    Kousidis, S., Kennington, C., Schlangen, D.: Investigating speaker gaze and pointing behaviour in human-computer interaction with the collection. In: Proceedings of SIGDIAL, pp. 319–323, Metz, France (2013)Google Scholar
  29. 29.
    Peltason, J., Wrede, B.: Pamini: a framework for assembling mixed-initiative human-robot interaction from generic interaction patterns. In: Proceedings of SIGdial, pp. 229–232 (2010)Google Scholar
  30. 30.
    Wrede, S., Hanheide, M., Bauckhage, C., Sagerer, G.: An active memory as a model for information fusion. In: Proceedings of the 7th International Conference on Information Fusion, pp. 198–205 (2004)Google Scholar
  31. 31.
    Henning, M.: A new approach to object-oriented middleware. IEEE Internet Comput. 8(1), 66–75 (2004)CrossRefGoogle Scholar
  32. 32.
    Keizer, S., Foster, M.E., Lemon, O., Gaschler, A., Giuliani, M.: Training and evaluation of an MDP model for social multi-user human-robot interaction. In: Proceedings of SIGdial (2013)Google Scholar
  33. 33.
    Ghallab, M., Nau, D., Traverso, P.: Automated Planning: Theory and Practice. Morgan Kaufmann (2004)Google Scholar
  34. 34.
    McDermott, D., Ghallab, M., Howe, A., Knoblock, C., Ram, A., Veloso, M., Weld, D., Wilkins, D.: PDDL—The Planning Domain Definition Language (Version 1.2). Technical Report CVC TR-98-003/DCS TR-1165, Yale Center for Computational Vision and Control (1998)Google Scholar
  35. 35.
    Younes, H.L.S., Littman, M.L.: PPDDL1.0: an extension to PDDL for expressing planning domains with probabilistic effects. Technical Report CMU-CS-04-162, Carnegie Mellon University (2004)Google Scholar
  36. 36.
    Sanner, S.: Relational dynamic influence diagram language (RDDL): language description. (2010)
  37. 37.
  38. 38.
    Rintanen, J.: Complexity of planning with partial observability. In: Proceedings of ICAPS, pp. 345–354 (2004)Google Scholar
  39. 39.
    Palacios, H., Geffner, H.: Compiling uncertainty away in conformant planning problems with bounded width. J. Artif. Intell. Res. 35, 623–675 (2009)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Albore, A., Palacios, H., Geffner, H.: A translation-based approach to contingent planning. In: Proceedings of IJCAI, pp. 1623–1628 (2009)Google Scholar
  41. 41.
    Petrick, R.P.A., Foster, M.E.: Planning for social interaction in a robot bartender domain. In: Proceedings of ICAPS 2013 (2013)Google Scholar
  42. 42.
    Petrick, R.P.A., Bacchus, F.: A knowledge-based approach to planning with incomplete information and sensing. In: Proceedings of AIPS, pp. 212–221 (2002)Google Scholar
  43. 43.
    Petrick, R.P.A., Bacchus, F.: Extending the knowledge-based approach to planning with incomplete information and sensing. In: Proceedings of ICAPS, pp. 2–11 (2004)Google Scholar
  44. 44.
    Sharma, V.: Automated Planning for Natural Language Robot Dialogue. M.Sc. Project, University of Edinburgh, Edinburgh (2012)Google Scholar
  45. 45.
    Petrick, R.P.A., Gaschler, A.: Extending knowledge-level planning with sensing for robot task planning. In: Proceedings of PlanSIG (2014)Google Scholar
  46. 46.
    Perrault, C.R., Allen, J.F.: A plan-based analysis of indirect speech acts. Am. J. Comput. Linguist. 6(3–4), 167–182 (1980)Google Scholar
  47. 47.
    Appelt, D.: Planning English Sentences. Cambridge University Press, Cambridge (1985)CrossRefzbMATHGoogle Scholar
  48. 48.
    Hovy, E.: Generating Natural Language Under Pragmatic Constraints. Lawrence Erlbaum Associates, Hillsdale (1988)Google Scholar
  49. 49.
    Cohen, P., Levesque, H.: Rational interaction as the basis for communication. In: Cohen, P., Morgan, J., Pollack, M. (eds.) Intentions in Communication, pp. 221–255. MIT Press, Cambridge (1990)Google Scholar
  50. 50.
    Young, R.M., Moore, J.D.: DPOCL: a principled approach to discourse planning. In: Proceedings of INLG, pp. 13–20, Kennebunkport, Maine, USA (1994)Google Scholar
  51. 51.
    Koller, A., Stone, M.: Sentence generation as planning. In: Proceedings of ACL, pp. 336–343, Prague, Czech Republic (2007)Google Scholar
  52. 52.
    Benotti, L.: Accommodation through tacit sensing. In: Proceedings of LONDIAL, pp. 75–82, London, UK (2008)Google Scholar
  53. 53.
    Brenner, M., Kruijff-Korbayová, I.: A continual multiagent planning approach to situated dialogue. In: Proceedings of LONDIAL, pp. 67–74 (2008)Google Scholar
  54. 54.
    Koller, A., Petrick, R.P.A.: Experiences with planning for natural language generation. Comput. Intell. 27(1), 23–40 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Mackaness, W., Boye, J., Clark, S., Fredriksson, M., Geffner, H., Lemon, O., Minnock, M., Webber, B.: The SpaceBook project: pedestrian exploration of the city using dialogue based interaction over smartphones. In: Proceedings of the 8th Symposium on Location-Based Services, Vienna, Austria (2011)Google Scholar
  56. 56.
    Henderson, M., Thomson, B., Williams, J.D.: The second dialog state tracking challenge. In: Proceedings of SIGdial, pp. 263–272, Philadelphia, PA, USA (2014)Google Scholar
  57. 57.
    Cashmore, M., Fox, M., Long, D., Magazzeni, D., Ridder, B., Carrera, A., Palomeras, N., Hurtos, N., Carreras, M.: ROSPlan: planning in the robot operating system. In: Proceedings of ICAPS (2015)Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2017

Authors and Affiliations

  1. 1.School of Computing ScienceUniversity of GlasgowGlasgowUK
  2. 2.Department of Computer ScienceHeriot-Watt UniversityEdinburghUK

Personalised recommendations