Abstract
Neural networks have many applications in safety and mission critical systems. As industrial standards in various safety-critical domains require developers of critical systems to provide safety assurance, tools and techniques must be developed that enable effective creation of safety evidence for AI systems. In this position paper, we propose the use of rules extracted from neural networks as artefacts for safety evidence. We discuss the rationale behind the use of rules and illustrate it using the MNIST dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ericson, C.A.: Concise Encyclopedia of System Safety: Definition of Terms and Concepts. Wiley, New York (2011)
Ministry of Defence (MoD) UK: Defence Stanandard 00–56 Issue 4: Safety Management Requirements for Defence Systems (2007)
de la Vara, J.L., et al.: Towards a model-based evolutionary chain of evidence for compliance with safety standards. In: Ortmeier, F., Daniel, P. (eds.) SAFECOMP 2012. LNCS, vol. 7613, pp. 64–78. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33675-1_6
de la Vara, J., Nair, S., Walawege, R.P.: On the use of artefacts as safety evidence: a conceptual model. Technical report (2013)
Kornecki, A., Janusz, Z.: Certification of software for real-time safety critical systems: state of the art. Innov. Syst. Softw. Eng. 5, 149–161 (2009). https://doi.org/10.1007/s11334-009-0088-1
Craven, M.W.: Extracting comprehensible models from trained neural networks. Ph.D dissertation, University of Wisconsin-Madison (1996)
Bologna, G.: A model for single and multiple knowledge based networks. Artif. Intell. Med. 28, 141–163 (2003)
Croce, F., Andriushchenko, M., Hein, M.: Provable robustness of ReLU networks via maximization of linear regions. In: AISTATS (2019)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: ICLR (2015)
Craven, M., Shavlik, J.: Using sampling and queries to extract rules from trained neural networks. In Machine Learning: Proceedings of the 11th International Conference, San Francisco, CA (1994)
KumarSethi, K., Kumar Mishra, D., Mishra, B.: Extended taxonomy of rule extraction techniques and assessment of KDRuleEx. Int. J. Comput. Appl. 50(21), 25–31 (2012)
Ozbakır, L., Baykasoglu, A., Kulluk, S.: A soft computing-based approach for integrated training and rule extraction from artificial neural networks: DIFACONN-miner. Appl. Soft Comput. 10, 304–317 (2010)
Sato, M., Tsukimoto, H.: Rule extraction from neural networks via decision tree induction. In: International Joint Conference On Neural Network (2001)
Zilke, J.: Extracting rules from deep neural networks. M.S. thesis, Computer Science Department, Technische Universitaet Darmstadt (2015)
Setiono, R., Leow, W.K.: FERNN: an algorithm for fast extraction of rules from neural networks. Appl. Intell. 12, 12–25 (2000). https://doi.org/10.1023/A:1008307919726
Andrews, R., et al.: An evaluation and comparison of techniques for extracting and refining rules from artificial neural networks. QUT NRC, February 1996
Fu, L.M.: Rule generation from neural networks. IEEE Trans. Syst. Man Cybern. 28(8), 1114–1124 (1994)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. 10, 1–19 (2019)
Wong, E., Kolter, J.Z.: Provable defenses against adversarial examples via the convex outer adversarial polytope. In: ICML (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Beyene, T.A., Sahu, A. (2020). Rule-Based Safety Evidence for Neural Networks. In: Casimiro, A., Ortmeier, F., Schoitsch, E., Bitsch, F., Ferreira, P. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2020 Workshops. SAFECOMP 2020. Lecture Notes in Computer Science(), vol 12235. Springer, Cham. https://doi.org/10.1007/978-3-030-55583-2_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-55583-2_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-55582-5
Online ISBN: 978-3-030-55583-2
eBook Packages: Computer ScienceComputer Science (R0)