Knowledge Representation Learning Based on GAN and Pre-training

Abstract

Knowledge representation learning can realize the representation of entity and relation semantic information by projecting the triple entity or relation in knowledge graph into vector space, and can effectively calculate entity, relation and their complex semantic association. However, most of the existing models only use the structure information of triples, ignoring the semantic rich entity description. In addition, knowledge representation learning methods usually use negative sampling method. Through random sampling, the negative samples generated by this method are easy to be recognized by the model, and with the iteration of training, the contribution to the improvement of the model performance becomes smaller and smaller. Therefore, we propose a Knowledge Representation learning method based on Generative adversarial network and Pre-training language model (KRGP). Specifically, it uses the idea of generative adversarial network, semantic matching model and translation distance model to generate high-quality negative samples. We combine descriptions of head and tail entities and relations corresponding to negative sample triples into text sequences, and then use next sentence prediciton task of BERT to transform text sequences into a binary classification problem. Experimental results show that this method can improve the performance of link prediction and triple classification on multiple data sets.