Continual Few-Shot Relation Extraction with Prompt-Based Contrastive Learning

Abstract

Continual relation extraction (CRE) aims to continually learn new relations while maintaining knowledge of previous relations in the data streams. Recently, continual few-shot relation extraction (CFRE) is introduced, in which only the first step of training has enough data, while the later steps have only a handful of samples, since emerging relations are often difficult to obtain enough training data quickly in realistic scenarios. Some previous work has proved that storing samples from previous tasks can alleviate the problem of catastrophic forgetting in continual learning. However, such approaches rely heavily on memory size and fail to fully exploit the knowledge associated with memorized samples in the pre-trained language model (PLM). To solve these problems, we propose a novel Prompt-based Contrastive Learning method, namely PCLfor CFRE with two optimizations. Firstly, to make better use of the knowledge in the limited number of training samples, we propose to obtain better representations of the relations by a prompt-based encoder with the assistance of a well-designed template. A mix-up data augmentation strategy is harnessed to increase the robustness of the model and prevent few-shot tasks from over-fitting. Secondly, to alleviate forgetting previous knowledge, in addition to replaying samples in the memory, our method maintains the stability of instance embeddings and retains the knowledge of the learned relations by an instance-level distillation loss. With extensive experiments on two benchmarks, we demonstrate that our model significantly outperforms state-of-the-art baselines and increases the generalization and robustness of the CFRE model on the imbalanced dataset.