Starting to Cook a Coaching Dialogue System in the Olympus framework

Conference paper


This paper presents the COOKCOACH, a system that guides the user while cooking a recipe during a spoken dialogue coaching session. We describe our experience creating the first version of our coaching dialogue system that helps the user cook a selected dish.We use the CMU dialogue systems framework OLYMPUS. The main challenge we faced was the change of paradigm: instead of the system being driven by the user, the user is guided by the system. The result is a system capable of dictating generic tasks to the user. The methodology allows us to create different systems in diverse domains for similar guidance tasks that support the user while s/he performs some procedural task. The use of ontologies to facilitate the creation of new systems is briefly explored.


Coaching Ontologies Cooking Knowledge Acquisition Interface Design Human-Human Dialogue Corpora 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



Joana Paulo Pardal is supported by a PhD fellowship from the Portuguese National Science Foundation, Fundao para a Cilncia e Tecnologia, FCT (SFRH/BD/30791/ 2006). This work was partially funded by FCT (INESC-ID multi-annual funding) through the PIDDAC Program funds.

We gratefully thank James F. Allen and Mary D. Swift for their insightful input and comments.

The OntoChef ontology was joint work with Fernando Batista, Ricardo Ribeiro and Paula Vaz.

The prototype of the Cook Tutor was joint work with Filipe Martins, Luís Franqueira, Pedro Arez and Luísa Coheur.

The authors would also like to thank Dan Bohus and Matthew Marge for the support provided in the use of Olympus/RavenClaw and about LARRI and IPA implementations.

Finally, none of this work would have been possible without the help of family and friends that supported our work throughout the experiences, and, most importantly, volunteered to be recorded for the Cooking Corpus.


  1. 1.
    Dan Bohus, Antoine Raux, Thomas K. Harris, Maxine Eskenazi, and Alexander I. Rudnicky, “Olympus: an open-source framework for conversational spoken language interface research,” in NAACL-HLT ’07: Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies, 2007.Google Scholar
  2. 2.
    Dan Bohus and Alexander I. Rudnicky, “The RavenClaw dialog management framework: Architecture and systems,” Computer Speech and Language, vol. 23, no. 3, 2009.Google Scholar
  3. 3.
    Alan Black and Maxine Eskenazi, “The spoken dialogue challenge,” in Proceedings of SIGDIAL. 2009, ACL.Google Scholar
  4. 4.
    Dan Bohus and Alexander I. Rudnicky, “LARRI: A language-based maintenance and repair assistant,” in Spoken Multimodal Human-Computer Dialogue in Mobile Environments, vol. 28 of Text, Speech and Language Technology, pp. 203–218. Springer Netherlands, 2005.Google Scholar
  5. 5.
    Gregory Aist, John Dowding, Beth Ann Hockey, Manny Rayner, James Hieronymus, Dan Bohus, Bradley Boven, Nate Blaylock, Ellen Campana, Susana Early, Genevieve Gorrell, and Steven Phan, “Talking through procedures: an Intelligent Space Station procedure assistant,” in Proceedings of EACL, 2003.Google Scholar
  6. 6.
    Manny Rayner, Beth Ann Hockey, James Hieronymus, John Dowding, Gregory Aist, and Susana Early, “An intelligent procedure assistant built using REGULUS 2 and ALTERF,” in Proceedings of ACL, 2003.Google Scholar
  7. 7.
    Robert E. Kraut, Susan R. Fussell, and Jane Siegel, “Visual information as a conversational resource in collaborative physical tasks,” Human-computer interaction, vol. 18, no. 1, 2003.Google Scholar
  8. 8.
    Ricardo Ribeiro, Fernando Batista, Joana Paulo Pardal, Nuno J. Mamede, and H. Sofia Pinto, “Cooking an ontology,” in AIMSA. 2006, vol. 4183 of LNCS, Springer.Google Scholar
  9. 9.
    Telmo Ramos Machado, “Extracç ão de informaç ão – introduç ão automática de receitas de acordo com ontologia,” M.S. thesis, IST, Technical University of Lisbon, 2007.Google Scholar
  10. 10.
    Dan Tasse and Noah A. Smith, “SOUR CREAM: Toward semantic processing of recipes,” Tech. Rep. CMU-LTI-08-005, Carnegie Mellon University, 2008.Google Scholar
  11. 11.
    Brian K. Reid, “USENET CookBook: An experiment in electronic publication,” Tech. Rep. WRL-87-7, Digital Equipment Corporation Western Research Lab (now HP Labs), 1987.Google Scholar
  12. 12.
    Filipe M. Martins, Ana Cristina Mendes, Márcio Viveiros, Joana Paulo Pardal, Pedro Arez, Nuno J. Mamede, and Joao Paulo Neto, “Reengineering a domainindependent framework for spoken dialogue systems,” in Software Engineering, Testing, and Quality Assurance for Natural Language Processing. 2008, SETQA-NLP, ACL.Google Scholar
  13. 13.
    Filipe M. Martins, Joana Paulo Pardal, Luís Franqueira, Pedro Arez, and Nuno J. Mamede, “Starting to cook a tutoring dialogue system,” in Proc. SLT, 2008.Google Scholar
  14. 14.
    James Allen, George Ferguson, Mary Swift, Amanda Stent, Scott Stoness, Lucian Galescu, Nathan Chambers, Ellen Campana, and Gregory Aist, “Two diverse systems built using generic components for spoken dialogue (recent progress on TRIPS),” in Proc. ACL Interactive Poster and Demos Sessions, 2005.Google Scholar
  15. 15.
    David Milward and Martin Beveridge, “Ontology-based dialogue systems,” in 3rd Workshop on Knowledge and Reasoning in Practical Dialogue Systems – IJCAI, 2003.Google Scholar
  16. 16.
    Annika Flycht-Eriksson, Design and Use of Ontologies in Informationproviding Dialogue Systems, Ph.D. thesis, School of Engineering at Link¨oping University, 2004.Google Scholar
  17. 17.
    Joana Paulo Pardal, “Dynamic use of ontologies in dialogue systems,” in Proc. NAACL-HLT 2007 Doctoral Consortium, 2007.Google Scholar
  18. 18.
    Joana Paulo Pardal, “Position paper on spoken dialogue systems,” in Young Researchers Roundtable on Spoken Dialogue Systems, 2009, YRRoSDS.Google Scholar
  19. 19.
    Joana Paulo Pardal, “Position paper on spoken dialogue systems,” in Young Researchers Roundtable on Spoken Dialogue Systems, 2008, YRRoSDS.Google Scholar
  20. 20.
    Michael Schneider, “The Semantic Cookbook: Sharing cooking experiences in the Smart Kitchen,” in IET, 2007.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Spoken Language Systems LaboratoryL2F – INESC-ID IST / Technical University of LisbonLisboaPortugal

Personalised recommendations