Explain to Not Forget: Defending Against Catastrophic Forgetting with XAI

Abstract

The ability to continuously process and retain new information like we do naturally as humans is a feat that is highly sought after when training neural networks. Unfortunately, the traditional optimization algorithms often require large amounts of data available during training time and updates w.r.t. new data are difficult after the training process has been completed. In fact, when new data or tasks arise, previous progress may be lost as neural networks are prone to catastrophic forgetting. Catastrophic forgetting describes the phenomenon when a neural network completely forgets previous knowledge when given new information. We propose a novel training algorithm called Relevance-based Neural Freezing in which we leverage Layer-wise Relevance Propagation in order to retain the information a neural network has already learned in previous tasks when training on new data. The method is evaluated on a range of benchmark datasets as well as more complex data. Our method not only successfully retains the knowledge of old tasks within the neural networks but does so more resource-efficiently than other state-of-the-art solutions.

S. Ede and S. Baghdadlian—Contributed equally.

A Appendix

1.1 A.1 MNIST-Split

The model architecture was taken from [40], which compares multiple methods for mitigating catastrophic forgetting. It consists of two hidden layers with 400 neurons each and ReLU activations. As for most experiments except the real-world dataset, the pruning threshold is set to 2%, meaning that the accuracy can drop by up to 2% before the pruning procedure is halted. We use the Adam optimizer with a learning rate of 0.001, $\beta _1 = 0.9$ and $\beta _2=0.999$ with a batch size of 128.

1.2 A.2 MNIST-Permuted

For this experiment, the architecture from [34] was adapted by increasing the number of hidden layer units to 1000 per layer to match the increased complexity of the task. Additionally, the learning rate was decreased to 0.0001 and the model was trained for ten instead of four epochs per task.

1.3 A.3 CIFAR10 and CIFAR100

In this experiment, we adopted architecture and experimental setup from [46].

1.4 A.4 ImageNet Split

Here, we replicate the conditions from [43] but establish our baseline after ten instead of 70 epochs, which we also use when applying RNF.

1.5 A.5 Adience

As is state-of-the art for this dataset [9], we normalize to zero mean and unit standard deviation during training and apply data augmentation for the training data by randomly cropping to 224 $\times$ 224 as well as horizontal flipping. For testing, each sample is cropped five times to 224 $\times$ 224 (four corner crops and one center crop), where each crop is additionally mirrored horizontally. The ground truth is then compared to the mean of the Softmax activations of the ten samples. Only the center crops are used in the reference data. As the dataset is strongly imbalanced, we additionally employ a resampling strategy during training that undersamples classes with a high number of samples and oversamples classes with a low number of samples by computing the class probabilities and then sampling from a multinomial distribution.

In this experiment, we employ a VGG-16 network architecture [38] that has been pretrained on ImageNet (from the PyTorch [29] model zoo), as well as an Adam optimizer and L2 regularization.

• Split: We used a learning rate of 0.0001, L2 regularization with $\lambda = 0.01$ and a batch size of 32.

• Entire Dataset: The model was trained with a learning rate of 0.00001 and $\lambda = 0.001$.