Abstract
Deep neural networks (DNNs) are increasingly applied in safety-critical domains, such as self-driving cars, unmanned aircraft, and medical diagnosis. It is of fundamental importance to certify the safety of these DNNs, i.e. that they comply with a formal safety specification. While safety certification tools exactly answer this question, they are of no help in debugging unsafe DNNs, requiring the developer to iteratively verify and modify the DNN until safety is eventually achieved. Hence, a repair technique needs to be developed that can produce a safe DNN automatically. To address this need, we present SpecRepair, a tool that efficiently eliminates counter-examples from a DNN and produces a provably safe DNN without harming its classification accuracy. SpecRepair combines specification-based counter-example search and resumes training of the DNN, penalizing counter-examples and certifying the resulting DNN. We evaluate SpecRepair’s effectiveness on the ACAS Xu benchmark, a DNN-based controller for unmanned aircraft, and two image classification benchmarks. The results show that SpecRepair is more successful in producing safe DNNs than comparable methods, has a shorter runtime, and produces safe DNNs while preserving their classification accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bauer-Marquart, F., Boetius, D., Leue, S., Schilling, C.: SpecRepair: counter-example guided safety repair of deep neural networks - supplementary material (2022)
Chen, J., Jordan, M.I., Wainwright, M.J.: HopSkipJumpAttack: a query-efficient decision-based attack. In: IEEE Symposium on Security and Privacy, pp. 1277–1294. IEEE (2020). https://doi.org/10.1109/SP40000.2020.00045
Cheng, C.: Provably-robust runtime monitoring of neuron activation patterns. In: DATE, pp. 1310–1313. IEEE (2021). https://doi.org/10.23919/DATE51398.2021.9473957
Djavanshir, G.R., Chen, X., Yang, W.: A review of artificial intelligence’s neural networks (deep learning) applications in medical diagnosis and prediction. IT Prof. 23(3), 58–62 (2021)
Dong, G., Sun, J., Wang, X., Wang, X., Dai, T.: Towards repairing neural networks correctly. In: QRS, pp. 714–725. IEEE (2021)
Endres, S.C., Sandrock, C., Focke, W.W.: A simplicial homology algorithm for Lipschitz optimisation. J. Global Optim. 72(2), 181–217 (2018). https://doi.org/10.1007/s10898-018-0645-y
Fischer, M., Balunovic, M., Drachsler-Cohen, D., Gehr, T., Zhang, C., Vechev, M.T.: DL2: training and querying neural networks with logic. In: ICML, Proceedings of Machine Learning Research, vol. 97, pp. 1931–1941. PMLR (2019)
Goldberger, B., Katz, G., Adi, Y., Keshet, J.: Minimal modifications of deep neural networks using verification. In: LPAR, EPiC Series in Computing, vol. 73, pp. 260–278. EasyChair (2020)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: ICLR (Poster) (2015)
Goodfellow, I.J., Bengio, Y., Courville, A.C.: Deep learning. In: Adaptive Computation and Machine Learning. MIT Press (2016). ISBN 978-0-262-03561-3. http://www.deeplearningbook.org/
Goodfellow, I.J., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)
Henzinger, T.A., Lukina, A., Schilling, C.: Outside the box: abstraction-based monitoring of neural networks. In: ECAI, FAIA, vol. 325, pp. 2433–2440. IOS Press (2020). https://doi.org/10.3233/FAIA200375
Huang, X., Kwiatkowska, M., Wang, S., Wu, M.: Safety verification of deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 3–29. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_1
Huang, X., et al.: A survey of safety and trustworthiness of deep neural networks: verification, testing, adversarial attack and defence, and interpretability. Comput. Sci. Rev. 37, 100270 (2020)
Julian, K.D., Lopez, J., Brush, J.S., Owen, M.P., Kochenderfer, M.J.: Policy compression for aircraft collision avoidance systems. In: 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC), pp. 1–10. IEEE (2016)
Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 97–117. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_5
Katz, G., et al.: The Marabou framework for verification and analysis of deep neural networks. In: Dillig, I., Tasiran, S. (eds.) CAV 2019. LNCS, vol. 11561, pp. 443–452. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-25540-4_26
Kochenderfer, M.J., Wheeler, T.A.: Algorithms for Optimization. MIT Press, Cambridge (2019)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Lamport, L.: Proving the correctness of multiprocess programs. IEEE Trans. Software Eng. 3(2), 125–143 (1977)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010). http://yann.lecun.com/exdb/mnist/
LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
Liu, C., Arnon, T., Lazarus, C., Strong, C.A., Barrett, C.W., Kochenderfer, M.J.: Algorithms for verifying deep neural networks. Found. Trends Optim. 4(3–4), 244–404 (2021)
Lukina, A., Schilling, C., Henzinger, T.A.: Into the unknown: active monitoring of neural networks. In: Feng, L., Fisman, D. (eds.) RV 2021. LNCS, vol. 12974, pp. 42–61. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88494-9_3
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: ICLR (Poster) (2018). OpenReview.net
Mirman, M., Gehr, T., Vechev, M.: Differentiable abstract interpretation for provably robust neural networks. In: International Conference on Machine Learning (ICML) (2018). https://www.icml.cc/Conferences/2018/Schedule?showEvent=2477
Moon, S., An, G., Song, H.O.: Parsimonious black-box adversarial attacks via efficient combinatorial optimization. In: ICML, PMLR, vol. 97, pp. 4636–4645 (2019). http://proceedings.mlr.press/v97/moon19a.html
Onishi, T., Motoyoshi, T., Suga, Y., Mori, H., Ogata, T.: End-to-end learning method for self-driving cars with trajectory recovery using a path-following function. In: IJCNN, pp. 1–8. IEEE (2019)
Singh, G., Gehr, T., Püschel, M., Vechev, M.T.: An abstract domain for certifying neural networks. In: POPL, vol. 3, pp. 41:1–41:30 (2019). https://doi.org/10.1145/3290354
Smith, A.E., Coit, D.W., Baeck, T., Fogel, D., Michalewicz, Z.: Penalty functions. In: Handbook of Evolutionary Computation, vol. 97, no. (1), C5 (1997)
Sotoudeh, M., Thakur, A.V.: Provable repair of deep neural networks. In: PLDI, pp. 588–603. ACM (2021)
Szegedy, C., et al.: Intriguing properties of neural networks. In: ICLR (2014). http://arxiv.org/abs/1312.6199
Usman, M., Gopinath, D., Sun, Y., Noller, Y., Păsăreanu, C.S.: NNrepair: constraint-based repair of neural network classifiers. In: Silva, A., Leino, K.R.M. (eds.) CAV 2021. LNCS, vol. 12759, pp. 3–25. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-81685-8_1
Acknowledgments
This research was partly supported by DIREC - Digital Research Centre Denmark and the Villum Investigator Grant S4OS.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bauer-Marquart, F., Boetius, D., Leue, S., Schilling, C. (2022). SpecRepair: Counter-Example Guided Safety Repair of Deep Neural Networks. In: Legunsen, O., Rosu, G. (eds) Model Checking Software. SPIN 2022. Lecture Notes in Computer Science, vol 13255. Springer, Cham. https://doi.org/10.1007/978-3-031-15077-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-15077-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15076-0
Online ISBN: 978-3-031-15077-7
eBook Packages: Computer ScienceComputer Science (R0)