Supervised Knowledge Aggregation for Knowledge Graph Completion

Abstract

We explore data-driven rule aggregation based on latent feature representations in the context of knowledge graph completion. For a given query and a collection of rules obtained by a symbolic rule learning system, we propose end-to-end trainable aggregation functions for combining the rules into a confidence score answering the query. Despite using latent feature representations for rules, the proposed models remain fully interpretable in terms of the underlying symbolic approach. While our models improve the base learner constantly and achieve competitive results on various benchmark knowledge graphs, we outperform current state-of-the-art with respect to a biomedical knowledge graph by a significant margin. We argue that our approach is in particular well suited for link prediction tasks dealing with a large multi-relational knowledge graph with several million triples, while the queries of interest focus on only one specific target relation.

A Experimental Details

1.1 A.1 Model Input

On the highest abstraction level, our models take as input a list of rules and output a real-valued score. More precisely, for a query $q=(s,p,?)$ (same in head direction) we collect the $top\_n$ answer candidates $c_i$ proposed by AnyBURL, that is, the candidates that were generated by at least one rule. For each of these candidates the respective list of rules defines the model input. Then, the descriptions of the main text apply. Finally, we obtain a vector of scores and likewise a ranking in regard to all candidate answers $c_i$. At test time, this ranking can directly be used for the evaluation. At training time, we distinguish the true answer/candidate $c^{*}$ and the remaining candidates $c'$ which we filter with the training set, i.e., we exclude a $c'\ne c^{*}$ if a triple $(s,p,c')$ exists in train. For a ranking loss we can now calculate the query loss of q as explained in Sect. 5.2. For some arbitrary loss function such as cross-entropy, $c^*$ defines the true candidate and the remaining candidates $c'$ define the reference candidates or pseudo negative candidates.

1.2 A.2 Hyperparameters

For all the experiments, we use a max top-n = 100, the Adagrad optimizer, and a batch-size of 256. Training is performed by using early stopping based on the validation set. LibKGE based configuration files for the experiments are provided in the supplementary materials.

Sparse Aggregator. The hyperparameters that we are concerned with are dropout on the latent features, the latent dimension d, and the learning rate lr. For Hetionet we set d = 10, dropout = 0.15 and lr = 0.9. On Fb15k-237 we set d = 40, dropout = 0.4, and lr = 0.02. For WNRR we set d = 50, dropout = 0.4 and lr = 0.03. For Codex-m we set d = 40, dropout = 0.4 and lr = 0.02. For all the experiments we use a value of 5 for lambda when training on the mean-rank loss.

Dense Aggregator. The dense aggregator follows in its architecture the PyTorch BERT encoder with the modification as explained in Sect. 5.3. We use 4 heads and 4 layers throughout all the experiments. The feed-forward dimensionality within the encoder is 256. For Hetionet we use d = 20, dropout = 0.15 and lr = 0.01. For the remaining datasets we use d = 56, dropout = 0.15, and lr = 0.005. We set the maximal number of rules per input list to 50 for the dense aggregator for all the experiments.

1.3 A.3 Rule Sets

The base data for our experiments are the rules learned with AnyBURL. These are processed in a pre-processing pipeline to generate the inputs for the aggregators as explained above. For all the datasets we exclude AC2 rules and rules with an empty body. This leaves the AnyBURL performance mostly unchanged but we report newest AnyBURL results reported by the authors.

For WNRR rules are mined for 3600 s and we set the maximum length for cyclical rules equal to 5 as suggested in the AnyBURL documentation. All the learned rules are processed for training the models on this dataset. For the remaining datasets the default AnyBURL parameters are used. Here, we prune the learned rulesets slightly and only process rules that had at least 5 (10) true predictions for sparse (dense). On Hetionet rules are learned for 1000 s for dense and sparse. On Fb15k-237 rules are learned for 3600 (500) s for sparse (dense). Finally, on Codex-M rules are learend for 1000 (500) s for sparse (dense).