Abstract
The rise of the phenomenon of the “right to be forgotten” has prompted research on machine unlearning, which grants data owners the right to actively withdraw data that has been used for model training and requires the elimination of the contribution of that data to the model. A simple method to achieve this is to use the remaining data to retrain the model, but this is not acceptable for other data owners who continue to participate in training. Existing machine unlearning methods have been found to be ineffective in quickly removing knowledge from deep learning models. This paper proposes using a stochastic network as a teacher to expedite the mitigation of the influence caused by forgotten data on the model. We performed experiments on three datasets, and the findings demonstrate that our approach can efficiently mitigate the influence of target data on the model within a single epoch. This allows for a one-time erasure and reconstruction of the model, and the reconstructed model achieves the same performance as the retrained model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aldaghri, N., Mahdavifar, H., Beirami, A.: Coded machine unlearning. IEEE Access 9, 88137–88150 (2021)
Baumhauer, T., Schöttle, P., Zeppelzauer, M.: Machine unlearning: linear filtration for logit-based classifiers. Mach. Learn. 111(9), 3203–3226 (2022)
Biggio, B., Nelson, B., Laskov, P.: Poisoning attacks against support vector machines. arXiv:1206.6389 (2012)
Bourtoule, L., et al.: Machine unlearning. In: 2021 IEEE Symposium on Security and Privacy, pp. 141–159. IEEE (2021)
Brophy, J., Lowd, D.: Machine unlearning for random forests. In: International Conference on Machine Learning, pp. 1092–1104. PMLR (2021)
Cao, Z., Wang, J., Si, S., Huang, Z., Xiao, J.: Machine unlearning method based on projection residual. In: 9th IEEE International Conference on Data Science and Advanced Analytics, pp. 1–8. IEEE (2022)
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., Zhang, Y.: When machine unlearning jeopardizes privacy. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 896–911 (2021)
Cho, J.H., Hariharan, B.: On the efficacy of knowledge distillation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4794–4802 (2019)
Gou, J., Yu, B., Maybank, S.J., Tao, D.: Knowledge distillation: a survey. Int. J. Comput. Vis. 129(6), 1789–1819 (2021)
Gupta, V., Jung, C., Neel, S., Roth, A., Sharifi-Malvajerdi, S., Waites, C.: Adaptive machine unlearning. Adv. Neural. Inf. Process. Syst. 34, 16319–16330 (2021)
Harding, E.L., Vanto, J.J., Clark, R., Hannah Ji, L., Ainsworth, S.C.: Understanding the scope and impact of the California consumer privacy act of 2018. J. Data Protect. Priv. 2(3), 234–253 (2019)
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv:1503.02531 (2015)
Kim, J., Woo, S.S.: Efficient two-stage model retraining for machine unlearning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4361–4369 (2022)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
LeCun, Y.: The mnist database of handwritten digits. http://yann.lecun.com/exdb/mnist/ (1998)
Liu, G., Ma, X., Yang, Y., Wang, C., Liu, J.: FedEraser: enabling efficient client-level data removal from federated learning models. In: 2021 IEEE/ACM 29th International Symposium on Quality of Service, pp. 1–10. IEEE (2021)
Marchant, N.G., Rubinstein, B.I., Alfeld, S.: Hard to forget: poisoning attacks on certified machine unlearning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 7691–7700 (2022)
Nguyen, T.T., Huynh, T.T., Nguyen, P.L., Liew, A.W.C., Yin, H., Nguyen, Q.V.H.: A survey of machine unlearning. arXiv:2209.02299 (2022)
Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: FitNets: hints for thin deep nets. arXiv:1412.6550 (2014)
Rong, D., Ye, S., Zhao, R., Yuen, H.N., Chen, J., He, Q.: FedRecAttack: model poisoning attack to federated recommendation. In: 38th IEEE International Conference on Data Engineering, pp. 2643–2655. IEEE (2022)
Sekhari, A., Acharya, J., Kamath, G., Suresh, A.T.: Remember what you want to forget: algorithms for machine unlearning. Adv. Neural. Inf. Process. Syst. 34, 18075–18086 (2021)
Voigt, P., von dem Bussche, A.: The EU General Data Protection Regulation (GDPR). Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57959-7
Wang, J., Guo, S., Xie, X., Qi, H.: Federated unlearning via class-discriminative pruning. In: Proceedings of the ACM Web Conference 2022, pp. 622–632 (2022)
Wu, C., Zhu, S., Mitra, P.: Federated unlearning with knowledge distillation. arXiv:2201.09441 (2022)
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747 (2017)
Yang, C., Wu, Q., Li, H., Chen, Y.: Generative poisoning attack method against neural networks. arXiv:1703.01340 (2017)
Zagoruyko, S., Komodakis, N.: Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In: 5th International Conference on Learning Representations (2017)
Zhang, Q.S., Zhu, S.C.: Visual interpretability for deep learning: a survey. Front. Inf. Technol. Electron. Eng. 19(1), 27–39 (2018)
This paper is supported by the Key Research and Development Program of Guangdong Province under grant No.2021B0101400003.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, X., Wang, J., Cheng, N., Sun, Y., Zhang, C., Xiao, J. (2023). Machine Unlearning Methodology Based on Stochastic Teacher Network. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14180. Springer, Cham. https://doi.org/10.1007/978-3-031-46677-9_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-46677-9_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46676-2
Online ISBN: 978-3-031-46677-9
eBook Packages: Computer ScienceComputer Science (R0)