Assuring Chatbot Relevance at Syntactic Level

  • Boris Galitsky


In this chapter we implement relevance mechanism based on similarity of parse trees for a number of chatbot components including search. We extend the mechanism of logical generalization towards syntactic parse trees and attempt to detect weak semantic signals from them. Generalization of syntactic parse tree as a syntactic similarity measure is defined as the set of maximum common sub-trees and performed at a level of paragraphs, sentences, phrases and individual words. We analyze semantic features of such similarity measure and compare it with semantics of traditional anti-unification of terms. Nearest neighbor machine learning is then applied to relate a sentence to a semantic class.

Using syntactic parse tree-based similarity measure instead of bag-of-words and keyword frequency approaches, we expect to detect a weak semantic signal otherwise unobservable. The proposed approach is evaluated in four distinct domains where a lack of semantic information makes classification of sentences rather difficult. We describe a toolkit which is a part of Apache Software Foundation project OpenNLP.chatbot, designed to aid search engineers and chatbot designers in tasks requiring text relevance assessment.


  1. Allen JF (1987) Natural language understanding. Benjamin Cummings, Menlo ParkzbMATHGoogle Scholar
  2. Abney S (1991) Parsing by chunks. In: Principle-based parsing. Kluwer Academic Publishers, pp 257–278Google Scholar
  3. Aleman-Meza B. Halaschek C, Arpinar I Sheth A (2003) A Context-Aware Semantic Association Ranking. In: Proceedings of first int’l workshop Semantic Web and Databases (SWDB ‘03), pp. 33-50.Google Scholar
  4. Amiridze N, Kutsia T (2018) Anti-unification and natural language processing fifth workshop on natural language and computer science, NLCS’18, EasyChair Preprint no. 203Google Scholar
  5. Banko M, Cafarella J, Soderland S, Broadhead M, Etzioni O (2007) Open information extraction from the web. In: Proceedings of the twentieth international joint conference on artificial intelligence. AAAI Press, Hyderabad, pp 2670–2676Google Scholar
  6. Bar-Haim R, Dagan I, Greental I, Shnarch E (2005) Semantic inference at the lexical-syntactic level AAAI-05.Google Scholar
  7. Bunke H (2003) Graph-based tools for data mining and machine learning. Lect Notes Comput Sci 2734/2003:7–19CrossRefGoogle Scholar
  8. Cardie C, Mooney RJ (1999) Machine learning and natural language, Mach Learn 1(5)Google Scholar
  9. Carreras X, Marquez L (2004) Introduction to the CoNLL-2004 shared task: semantic role labeling. In: Proceedings of the eighth conference on computational natural language learning. ACL, Boston, pp 89–97Google Scholar
  10. Chakrabarti D, Faloutsos C (2006) Graph mining: laws, generators, and algorithms. ACM Comput Surv 38(1)Google Scholar
  11. Collins M, Duffy N (2002) New ranking algorithms for parsing and tagging: kernels over discrete structures, and the voted perceptron. In: ACL02Google Scholar
  12. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297zbMATHGoogle Scholar
  13. Pighin D, Moschitti A (2009) Reverse engineering of tree kernel feature spaces. In: Proceedings of the 2009 conference on empirical methods in natural language processing. Association for Computational Linguistics, Singapore, pp 111–120Google Scholar
  14. de Salvo Braz R, Girju R, Punyakanok V, Roth D, Sammons M (2005) An inference model for semantic entailment in natural language, Proc AAAI-05Google Scholar
  15. Ding L, Finin T, Joshi A, Pan R, Cost RS, Peng Y, Reddivari P, Doshi V, Sachs J (2004) Swoogle: a search and metadata engine for the semantic web. In: Proceeding of the 13th ACM International Conference on Information and Knowledge Management (CIKM’04), pp 652–659Google Scholar
  16. Ducheyne S (2008) J.S. Mill’s canons of induction: from true causes to provisional ones. History and Philosophy of Logic 29(4):361–376CrossRefGoogle Scholar
  17. Durme BV, Huang Y, Kupsc A, Nyberg E (2003) Towards light semantic processing for question answering. HLT Workshop on Text MeaningGoogle Scholar
  18. Dzikovska M., Swift M, Allen J, William de Beaumont W (2005) Generic parsing for multi-domain semantic interpretation. International Workshop on Parsing Technologies (Iwpt05), Vancouver BC.Google Scholar
  19. Fukunaga K (1990) Introduction to statistical pattern recognition, 2nd edn. Academic Press Professional, Inc., San DiegozbMATHGoogle Scholar
  20. Galitsky B, Josep Lluis de la Rosa, Gabor Dobrocsi (2011a) Building integrated opinion delivery environment. FLAIRS-24, West Palm Beach FL May 2011Google Scholar
  21. Galitsky B, Dobrocsi G, de la Rosa JL, Kuznetsov SO (2011b) Using generalization of syntactic parse trees for taxonomy capture on the web. ICCS:104–117Google Scholar
  22. Galitsky BA, G Dobrocsi, JL De La Rosa, SO Kuznetsov (2010) From generalization of syntactic parse trees to conceptual graphs. International Conference on Conceptual Structures, 185-190.Google Scholar
  23. Galitsky B (2003) Natural language question answering system: technique of semantic headers. Advanced Knowledge International, AustraliaGoogle Scholar
  24. Galitsky B, Kuznetsov SO (2008) Learning communicative actions of conflicting human agents. J Exp Theor Artif Intell 20(4):277–317CrossRefGoogle Scholar
  25. Galitsky B, D Usikov (2008) Programming spatial algorithms in natural language. AAAI Workshop Technical Report WS-08-11.–Palo Alto, pp 16–24Google Scholar
  26. Galitsky B, González MP, Chesñevar CI (2009) A novel approach for classifying customer complaints through graphs similarities in argumentative dialogue. Decision Support Systems 46(3):717–729CrossRefGoogle Scholar
  27. Galitsky B, De La Rosa JL, Dobrocsi G (2012) Inferring the semantic properties of sentences by mining syntactic parse trees. Data Knowl Eng 81:21–45CrossRefGoogle Scholar
  28. Galitsky B, Kuznetsov SO, Usikov D (2013) Parse thicket representation for multi-sentence search. In: International conference on conceptual structures, pp 153–172Google Scholar
  29. Galitsky B, Ilvovsky DI, Kuznetsov SO (2014) Extending tree kernels towards paragraphs. Int J Comput Linguist Appl 5(1):105–116Google Scholar
  30. Galitsky B, Botros S (2015) Searching for associated events in log data. US Patent 9,171,037Google Scholar
  31. Galitsky B (2017a) Improving relevance in a content pipeline via syntactic generalization. Eng Appl Artif Intell 58:1–26CrossRefGoogle Scholar
  32. Galitsky B (2017b) Matching parse thickets for open domain question answering. Data Knowl Eng 107:24–50CrossRefGoogle Scholar
  33. Ganter B, Kuznetsov S (2001) Pattern Structures and Their Projections, Proceedings of the 9th International Conference on Conceptual Structures, ICCS’01, ed. G. Stumme and H. Delugach, Lecture Notes in Artificial Intelligence, 2120, 129–142.Google Scholar
  34. Garey MR, Johnson DS (1979) Computers and intractability: a guide to the theory of NP-completeness. Freeman, San FranciscozbMATHGoogle Scholar
  35. Gildea D (2003) Loosely tree-based alignment for machine translation. In: Proceedings of the 41th annual conference of the Association for Computational Linguistics (ACL-03), Sapporo, pp 80–87Google Scholar
  36. Iosif E, Potamianos A (2009) Unsupervised semantic similarity computation between terms using web documents. IEEE Trans Knowl Data Eng 13Google Scholar
  37. Kapoor S, Ramesh H (1995) Algorithms for Enumerating All Spanning Trees of Undirected and Weighted Graphs. SIAM J Comput 24:247–265MathSciNetCrossRefGoogle Scholar
  38. Kok S, Domingos P (2008) Extracting semantic networks from text via relational clustering. In: Proceedings of the nineteenth European conference on machine learning. Springer, Antwerp, Belgium, pp 624–639Google Scholar
  39. Kuznetsov SO, Samokhin, MV (2005) Learning closed sets of labeled graphs for chemical applications. In: Inductive Logic Programming pp 190–208Google Scholar
  40. Lamberti F, Sanna A, Demartini C (2009) A Relation-Based Page Rank Algorithm for Semantic Web Search Engines. IEEE Trans Knowl Data Eng 21(1):123–136CrossRefGoogle Scholar
  41. Lin D, Pantel P (2001) DIRT: discovery of inference rules from text. In: Proceedings of ACM SIGKDD conference on knowledge discovery and data mining 2001, 323–328Google Scholar
  42. Makhalova T, Ilvovsky DI, Galitsky BA (2015) Pattern Structures for News Clustering. FCA4AI@ IJCAI, 35-42Google Scholar
  43. Mill JS (1843) A system of logic, racionative and inductive, LondonGoogle Scholar
  44. Moldovan D, Clark C, Harabagiu S, Maiorano S (2003) Cogex: a logic prover for question answering. In: Proceedings of HLTNAACL 2003Google Scholar
  45. Moreda P, Navarro B, Palomar M (2007) Corpus-based semantic role approach in information retrieval. Data Knowl Eng 61:467–483CrossRefGoogle Scholar
  46. Moschitti A (2008) Kernel Methods, Syntax and Semantics for Relational Text Categorization. In: Proceeding of ACM 17th Conference on Information and Knowledge Management (CIKM). Napa Valley, California.Google Scholar
  47. Moschitti A, Pighin D, Basili R (2006). Semantic role labeling via tree kernel joint inference. In Proceedings of the 10th conference on computational natural language learning, New York, USAGoogle Scholar
  48. Plotkin GD (1970) A note on inductive generalization. In: Meltzer B, Michie D (eds) Machine Intelligence, vol 5. Elsevier North-Holland, New York, pp 153–163Google Scholar
  49. Poon H, Domingos P (2008) Joint unsupervised coreference resolution with Markov logic. In: Proceedings of the conference on empirical methods in natural language processing (EMNLP’08). Association for Computational Linguistics, Stroudsburg, pp 650–659CrossRefGoogle Scholar
  50. Ravichandran D, Hovy E (2002) Learning surface text patterns for a Question Answering system. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002), Philadelphia, PAGoogle Scholar
  51. Robinson JA (1965) A machine-oriented logic based on the resolution principle. J Assoc Comput Mach 12:23–41MathSciNetCrossRefGoogle Scholar
  52. Romano L, Kouylekov M, Szpektor I, Dagan I, Lavelli A (2006) Investigating a generic paraphrase-based approach for relation extraction. In: Proceedings of EACL, 409–416Google Scholar
  53. Stevenson M, Greenwood MA (2005) A semantic approach to IE pattern induction. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005), Ann ArborGoogle Scholar
  54. Strok F, Galitsky B, Ilvovsky D, Kuznetsov S (2014) Pattern structure projections for learning discourse structures. International Conference on Artificial Intelligence: methodology, Systems, and Applications. Springer, Cham, pp 254–260Google Scholar
  55. Strzalkowski T, Carballo JP, Karlgren J, Tapanainen AHP, Jarvinen T (1999) Natural language information retrieval: TREC-8 report. In: Text Retrieval conferenceGoogle Scholar
  56. Suykens JAK, Horvath G, Basu S, Micchelli C, Vandewalle J (Eds.) (2003) Advances in learning theory: methods, models and applications, vol. 190 NATO-ASI series III: computer and systems sciences, IOS PressGoogle Scholar
  57. Thompson C, Mooney R, Tang L (1997) Learning to parse NL database queries into logical form. In: Workshop on automata induction, grammatical inference and language acquisitionGoogle Scholar
  58. Voorhees EM (2004) Overview of the TREC 2001 Question Answering track. In: TRECGoogle Scholar
  59. Zanzotto FM, Moschitti A (2006) Automatic learning of textual entailments with cross-pair similarities. In: Proceedings of the Joint 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics (COLING-ACL), Sydney, Australia.Google Scholar
  60. Zhang M, Zhou GD, Aw A (2008) Exploring syntactic structured features over parse trees for relation extraction using kernel methods. Inf Process Manage Int J 44(2):687–701CrossRefGoogle Scholar
  61. Zhao Y, Shen X, Senuma H, Aizawa A (2018) A comprehensive study: sentence compression with linguistic knowledge-enhanced gated neural network. Data Knowl Eng V117:307–318CrossRefGoogle Scholar
  62. Zettlemoyer LS, Collins M (2005) Learning to map sentences to logical form: structured classification with probabilistic categorial grammars. In: Bacchus F, Jaakkola T (eds) Proceedings of the twenty-first conference on uncertainty in artificial intelligence (UAI’05). AUAI Press, Arlington, pp 658–666Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Boris Galitsky
    • 1
  1. 1.Oracle (United States)San JoseUSA

Personalised recommendations