Machine Unlearning with Affine Hyperplane Shifting and Maintaining for Image Classification

Abstract

Machine unlearning enables a well-trained model to forget certain samples in the training set while keeping its performance on the remaining samples. Existing algorithms based on decision boundary require generating adversarial samples in input space in order to obtain the nearest but incorrect class labels. However, due to the nonlinearity of the decision boundary in input space, multiple iterations are needed to generate the adversarial samples, and the generated adversarial samples are affected by the bound of noise in adversarial attack, which greatly limits the speed and efficiency of unlearning. In this paper, a machine unlearning method with affine hyperplane shifting and maintaining is proposed for image classification, in which the nearest but incorrect class labels are directly obtained with the distance from the point to the hyperplane without generating adversarial samples for boundary shifting. Moreover, knowledge distillation is leveraged for boundary maintenance. Specifically, the output of the original model is decoupled into remaining class logits and forgetting class logits, and the remaining class logits is utilized to guide the unlearn model to avoid catastrophic forgetting. Our experimental results on CIFAR-10 and VGGFace2 have demonstrated that the proposed method is very close to the retrained model in terms of classification accuracy and privacy guarantee, and is about 4 times faster than Boundary Shrink.