Applied Intelligence

, Volume 37, Issue 4, pp 558–568 | Cite as

An approach to conversational agent design using semantic sentence similarity

  • Karen O’Shea


This paper presents a novel framework for constructing a Semantic-Based Conversational Agent (SCAF). Traditional conversational agents (CA) interpret scripts using structural patterns of sentences, which require the script writer to consider every possible permutation that a user may send as input to the CA. This is a time-consuming process, which takes no consideration of semantic content, working solely with the structural form of the sentence. Furthermore, this has proven to be a high maintenance task that can produce some unforeseen consequences when modifying or introducing new patterns into a script. This invariably results in the script writer reassessing the entire script to prevent such occurrences. Different script writers possess differing levels of skill and as such this can prove to be an exasperating task. The proposed SCAF interprets scripts consisting of natural language sentences by means of a semantic sentence similarity measure. User input is measured semantically against the natural language sentences of the context in order to respond with an appropriate output. Such scripting is effortless and alleviates the burden of the traditional pattern-scripted methodologies. Evaluation of the framework has highlighted its potential and shown improvements on traditional CAs.


Conversational agents Scripting methodologies Semantic similarity Sentence similarity measures 


  1. 1.
    Michie D (2001) Return of the imitation game. Electron Trans Artif Intell 6(2):203–221 Google Scholar
  2. 2.
    Massaro DW, Cohen MM, Beskow WJ, Cole RA (1998) Developing and evaluating conversational agents. University of California, Santa Cruz Google Scholar
  3. 3.
    Cassell J (2000) Embodied conversational agents. MIT Press, Cambridge Google Scholar
  4. 4.
    Sanders GA, Scholtz J (2000) Measurement and evaluation of embodied conversational agents. In: Cassell J, Sullivan J, Prevost S, Churchill E (eds) Embodied conversational agents. MIT Press, Cambridge Google Scholar
  5. 5.
    Isbister K, Doyle P (2004) The blind men and the elephant revisited: evaluating interdisciplinary embodied conversational agents. In: From brows to trust, Ruttkay Z, Pelachaud C (eds) Kluwer Academic, Netherlands, pp 3–26, Chap 1 Google Scholar
  6. 6.
    Malatesta L, Raouzalou A, Karpouzis K, Kollias K (2009) Towards modeling embodied conversational agent character profiles using appraisal theory predictions in expression synthesis. Int J Appl Intell 30(1):58–64 CrossRefGoogle Scholar
  7. 7.
    Weizenbaum J (1966) ELIZA—a computer program for the study of natural language communication between man and machine. Commun ACM 9:36–45 CrossRefGoogle Scholar
  8. 8.
    Colby K (1975) Artificial paranoia: a computer simulation of paranoid process. Pergamon Press, New York. Cited by Mauldin ML (1994). Chatterbots, tinymuds, and the turing test: entering the Loebner prize competition. Carnegie Mellon University, Pittsburgh Google Scholar
  9. 9.
    Carpenter R (2006) Jabberwacky. Accessed 14.04.11
  10. 10.
    Chatbots (2009) Accessed 13.03.11
  11. 11.
    ejTalk (2009) Accessed 13.03.11
  12. 12.
    Chatbots (2010) Accessed 13.03.11
  13. 13.
    Wallace R (2003) The elements of AIML style. ALICE Artificial Intelligence Foundations, Inc. Cited by Schmaker RP, Chen H (2008). Interaction analysis of the ALICE chatterbot: a two-study investigation of dialog and domain questioning. University of Arizona Google Scholar
  14. 14.
    Convagent (2001) InfoBot scripter’s manual. Accessed 14.03.11
  15. 15.
    Landauer TK, Foltz PW, Laham D (1998) Introduction to latent semantic analysis. Discourse Process 25(2–3):259–284 CrossRefGoogle Scholar
  16. 16.
    Li Y, McLean D, Bandar Z, O’Shea JD, Crockett K (2006) Sentence similarity based on semantic nets and corpus statistics. IEEE Trans Knowl Data Eng 18(8):1138–1149 CrossRefGoogle Scholar
  17. 17.
    Sammut C (2001) Managing context in a conversational agent. Electron Trans Artif Intell, Sect B, 5, 189–202. Google Scholar
  18. 18.
    Li Y, Bandar ZA, McLean D (2003) An approach for measuring semantic similarity between words using multiple information sources. IEEE Trans Knowl Data Eng 15(4):871–881 CrossRefGoogle Scholar
  19. 19.
    Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38(11), 39–41 CrossRefGoogle Scholar
  20. 20.
  21. 21.
    Walker A, Litman DJ, Kamm CA, Abella A (1997) PARADISE: a framework for evaluating spoken dialogue agents. In: Proc of the 35th annual meeting of the assoc for computational linguistics. Morgan Kaufmann, San Francisco. Cited by Hjalmarsson A (2002). Evaluating adapt, a multi-modal conversational, dialogue system using PARADISE. Master’s thesis, Royal Institute of Technology, Stockholm Google Scholar
  22. 22.
    Ruttkay Z, Dormann C, Noot H (2004) Embodied conversational agents on a common ground: a framework for design and evaluation. In: From brows to trust, Ruttkay Z, Pelachaud C (eds) Kluwer Academic, Norwell, pp 27–66, Chap 2 Google Scholar
  23. 23.
    Neale JM, Liebert RM (1986) Science and behaviour an introduction to methods of research. Prentice Hall, New York. Cited by Christoph N (2004) Empirical evaluation methodology for embodied conversational agents: on conducting evaluation studies. Chap 3. In: From brows to trust, Ruttkay Z, Pelachaud C (eds) Kluwer Academic, pp 67–99, 2004 Google Scholar
  24. 24.
    Faul F, Erdfelder E, Lang A-G, Buchner A (2007) G* Power: a flexible statistical power analysis program for the social sciences. Behav Res Methods 39(2):175–191 CrossRefGoogle Scholar
  25. 25.
    On B-W, Lee I (2011) Meta similarity. Int J Appl Intell 35(3):359–374 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.School of ComputingMathematics and Digital TechnologiesManchesterUK

Personalised recommendations