Active Learning Based Relation Classification for Knowledge Graph Construction from Conversation Data

Abstract

Creation of a Knowledge Graph (KG) from text, and its usages in solving several Natural Language Processing (NLP) problems are emerging research areas. Creating KG from text is a challenging problem which requires several NLP modules working together in unison. This task becomes even more challenging when constructing knowledge graph from a conversational data, as user and agent stated facts in conversations are often not grounded and can change with dialogue turns. In this paper, we explore KG construction from conversation data in travel and taxi booking domains. We use a fixed ontology for each of the conversation domain, and extract the relation triples from the conversation. Using active learning technique we build a state-of-the-art BERT based relation classifier which uses minimal data, but still performs accurate classification of the extracted relation triples. We further design heuristics for constructing KG that uses the BERT based relation classifier and Semantic Role Labelling (SRL) for handling negations in extracted relationship triples. Through our experiments we show that using our active learning trained classifier and heuristic based method, KG can be built with good correctness and completeness scores for domain specific conversational datasets. To the best of our knowledge this is the very first attempt at creating a KG from the conversational data that could be efficiently augmented in a dialogue agent to tackle the issue of data sparseness and improve the quality of generated response.