An Abstraction-Refinement Approach to Verification of Artificial Neural Networks

  • Luca Pulina
  • Armando Tacchella
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6174)


A key problem in the adoption of artificial neural networks in safety-related applications is that misbehaviors can be hardly ruled out with traditional analytical or probabilistic techniques. In this paper we focus on specific networks known as Multi-Layer Perceptrons (MLPs), and we propose a solution to verify their safety using abstractions to Boolean combinations of linear arithmetic constraints. We show that our abstractions are consistent, i.e., whenever the abstract MLP is declared to be safe, the same holds for the concrete one. Spurious counterexamples, on the other hand, trigger refinements and can be leveraged to automate the correction of misbehaviors. We describe an implementation of our approach based on the HySAT solver, detailing the abstraction-refinement process and the automated correction strategy. Finally, we present experimental results confirming the feasibility of our approach on a realistic case study.


Root Mean Square Error Hide Layer Input Vector Generalization Error Abstract Domain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Zhang, G.P.: Neural networks for classification: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 30(4), 451–462 (2000)CrossRefGoogle Scholar
  2. 2.
    Smith, D.J., Simpson, K.G.L.: Functional Safety – A Straightforward Guide to applying IEC 61505 and Related Standards, 2nd edn. Elsevier, Amsterdam (2004)Google Scholar
  3. 3.
    Kurd, Z., Kelly, T., Austin, J.: Developing artificial neural networks for safety critical systems. Neural Computing & Applications 16(1), 11–19 (2007)Google Scholar
  4. 4.
    Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural networks 2(5), 359–366 (1989)CrossRefGoogle Scholar
  5. 5.
    Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: 4th ACM SIGACT-SIGPLAN symposium on Principles of programming languages, pp. 238–252 (1977)Google Scholar
  6. 6.
    Franzle, M., Herde, C., Teige, T., Ratschan, S., Schubert, T.: Efficient solving of large non-linear arithmetic constraint systems with complex boolean structure. Journal on Satisfiability, Boolean Modeling and Computation 1, 209–236 (2007)Google Scholar
  7. 7.
    Clarke, E., Grumberg, O., Jha, S., Lu, Y., Veith, H.: Counterexample-guided abstraction refinement for symbolic model checking. Journal of the ACM (JACM) 50(5), 794 (2003)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Gordon, D.F.: Asimovian adaptive agents. Journal of Artificial Intelligence Research 13(1), 95–153 (2000)zbMATHMathSciNetGoogle Scholar
  9. 9.
    Pulina, L., Tacchella, A.: NEVER: A tool for Neural Network Verification (2010),
  10. 10.
    Igel, C., Glasmachers, T., Heidrich-Meisner, V.: Shark. Journal of Machine Learning Research 9, 993–996 (2008)Google Scholar
  11. 11.
    Haykin, S.: Neural networks: a comprehensive foundation. Prentice Hall, Englewood Cliffs (2008)Google Scholar
  12. 12.
    Gordeau, R.: Roboop – a robotics object oriented package in C++ (2005),
  13. 13.
    Igel, C., Husken, M.: Empirical evaluation of the improved Rprop learning algorithms. Neurocomputing 50(1), 105–124 (2003)zbMATHCrossRefGoogle Scholar
  14. 14.
    Schumann, J., Gupta, P., Nelson, S.: On verification & validation of neural network based controllers. In: Proc. of International Conf. on Engineering Applications of Neural Networks, EANN’03 (2003)Google Scholar
  15. 15.
    Witten, I.H., Frank, E.: Data Mining, 2nd edn. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  16. 16.
    Pappas, G., Kress-Gazit, H. (eds.): ICRA Workshop on Formal Methods in Robotics and Automation (2009)Google Scholar
  17. 17.
    Solar-Lezama, A., Jones, C.G., Bodik, R.: Sketching concurrent data structures. In: 2008 ACM SIGPLAN conference on Programming language design and implementation, pp. 136–148. ACM, New York (2008)CrossRefGoogle Scholar
  18. 18.
    Vechev, M., Yahav, E., Yorsh, G.G.: Abstraction-guided synthesis of synchronization. In: 37th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pp. 327–338. ACM, New York (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Luca Pulina
    • 1
  • Armando Tacchella
    • 1
  1. 1.DISTUniversità di GenovaGenovaItaly

Personalised recommendations