Abstract
Deep neural networks (DNNs) become increasingly popular. However, the vulnerability of DNNs can lead to a performance decrease when they cannot correctly predict the given samples. We propose a repair method for DNN-based classifiers to solve this problem, such that the accuracy can be improved by modifying the parameters of a DNN. First, we transform the DNN repair problem into a linear programming model, by encoding the constraints and the objective in linear programming. Second, to reduce the scale of the LP model, we repair the DNN by considering the parameters in the last layer. Third, to enhance the accuracy on the previously wrongly predicted samples without sacrificing the accuracy on the previously correctly predicted samples, we adopt these two types of samples in the optimization process. The evaluation on two popular datasets shows that our method outperforms the state-of-the-art methods and improves the accuracies by \(25.4\%\) points in the adversarial attacking scenario and \(67.6\%\) points in the backdooring attacking scenario. Meanwhile, our method can avoid obvious accuracy decreasing on standard test sets, which is at most 0.5%. The extensive experimentation demonstrates that the proposed method is effective and efficient in repairing DNN based classifiers.
Keywords
Supported by the Key Research Program of Frontier Sciences, CAS under grant No. QYZDJSSW-JSC036 and the National Natural Science Foundation of China (NSFC) under grant number 62132020.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Dong, G., Sun, J., Wang, J., Wang, X., Dai, T.: Towards repairing neural networks correctly. arXiv preprint arXiv:2012.01872 (2020)
Goldberger, B., Katz, G., Adi, Y., Keshet, J.: Minimal modifications of deep neural networks using verification. In: LPAR, vol. 2020, p. 23rd (2020)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014)
Gu, T., Liu, K., Dolan-Gavitt, B., Garg, S.: BadNets: evaluating backdooring attacks on deep neural networks. IEEE Access 7, 47230–47244 (2019)
Gupta, R., Pal, S., Kanade, A., Shevade, S.: DeepFix: fixing common c language errors by deep learning. In: 31st AAAI Conference on Artificial Intelligence (2017)
Gurobi Optimization, LLC: Gurobi optimizer reference manual (2021). https://www.gurobi.com
Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., Song, D.: Natural adversarial examples. In: Proceedings of the IEEE Computer Society Conference on CVPR, pp. 15262–15271 (2021)
Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: alexnet-level accuracy with 50x fewer parameters and \(<\)0.5 mb model size. arXiv preprint arXiv:1602.07360 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)
Ma, S., Liu, Y., Lee, W.C., Zhang, X., Grama, A.: Mode: automated neural network model debugging via state differential analysis and input selection. In: Proceedings of the 2018 26th ACM Joint Meeting ESEC/FSE, pp. 175–186 (2018)
Mu, N., Gilmer, J.: MNIST-C: a robustness benchmark for computer vision. arXiv preprint arXiv:1906.02337 (2019)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference CVPR, pp. 815–823 (2015)
Singh, G., Gehr, T., Püschel, M., Vechev, M.: An abstract domain for certifying neural networks. Proc. ACM Program. Lang. 3(POPL), 1–30 (2019)
Sotoudeh, M., Thakur, A.V.: Provable repair of deep neural networks. In: Proceedings of the 42nd ACM SIGPLAN International Conference PLDI, pp. 588–603 (2021)
Usman, M., Gopinath, D., Sun, Y., Noller, Y., Pasareanu, C.: NNrepair: constraint-based repair of neural network classifiers. arXiv preprint arXiv:2103.12535 (2021)
Wei, Y., et al.: Automated fixing of programs with contracts. In: Proceedings of the 19th ISSTA, pp. 61–72 (2010)
Yuan, Z., Lu, Y., Wang, Z., Xue, Y.: Droid-sec: deep learning in android malware detection. In: Proceedings of the 2014 ACM conference SIGCOMM, pp. 371–372 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, S., Yan, J., Yan, R. (2022). Layer-Specific Repair of Neural Network Classifiers. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13529. Springer, Cham. https://doi.org/10.1007/978-3-031-15919-0_46
Download citation
DOI: https://doi.org/10.1007/978-3-031-15919-0_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15918-3
Online ISBN: 978-3-031-15919-0
eBook Packages: Computer ScienceComputer Science (R0)