Skip to main content
SpringerLink
Log in

NeVer: a tool for artificial neural networks verification

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

The adoption of Artificial Neural Networks (ANNs) in safety-related applications is often avoided because it is difficult to rule out possible misbehaviors with traditional analytical or probabilistic techniques. In this paper we present NeVer, our tool for checking safety of ANNs. NeVer encodes the problem of verifying safety of ANNs into the problem of satisfying corresponding Boolean combinations of linear arithmetic constraints. We describe the main verification algorithm and the structure of NeVer. We present also empirical results confirming the effectiveness of NeVer on realistic case studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhang, G.P.: Neural networks for classification: a survey. IEEE Trans. Syst. Man Cybern., Part C Appl. Rev. 30(4), 451–462 (2000)

    Article  Google Scholar 

  2. Smith, D.J., Simpson, K.G.L.: Functional Safety – A Straightforward Guide to Applying IEC 61505 and Related Standards (2nd edn.). Elsevier (2004)

  3. Schumann, J., Gupta, P., Nelson, S.: On verification & validation of neural network based controllers. In: Proc. of International Conf. on Engineering Applications of Neural Networks (EANN’03) (2003)

  4. Kurd, Z., Kelly, T., Austin, J.: Developing artificial neural networks for safety critical systems. Neural Comput. Appl. 16(1), 11–19 (2007)

    Google Scholar 

  5. Clarke, E.M., Emerson, E.A., Sistla, A.P.: Automatic verification of finite-state concurrent systems using temporal logic specifications. ACM Trans. Program. Lang. Syst. (TOPLAS) 8(2), 263 (1986)

    Article  Google Scholar 

  6. Queille, J., Sifakis, J.: Specification and verification of concurrent systems in CESAR. In: International Symposium on Programming, pp. 337–351. Springer (1982)

  7. Schubert, T.: High level formal verification of next-generation microprocessors. In: Proceedings of the 40th annual Design Automation Conference. ACM (2003)

  8. Ball, T., Cook, B., Levin, V., Rajamani, S.K.: SLAM and static driver verifier: Technology transfer of formal methods inside Microsoft. In: Integrated Formal Methods, pp. 1–20. Springer (2004)

  9. Armando, A., Carbone, R., Compagna, L.: LTL model checking for security protocols. In: 20th IEEE Computer Security Foundations Symposium, pp. 385–396 (2007)

  10. Alur, R., Henzinger, T.A., Ho, P.: Automatic symbolic verification of embedded systems. In: IEEE Real-Time Systems Symposium, pp. 2–11 (1993)

  11. Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. Springer (1999)

  12. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw 2(5), 359–366 (1989)

    Article  Google Scholar 

  13. Pulina, L., Tacchella, A.: An abstraction-refinement approach to verification of artificial neural networks. In: 22nd International Conference on Computer Aided Verification (CAV 2010). Lecture Notes in Computer Science, vol. 6174, pp. 243–257. Springer (2010)

  14. Solar-Lezama, A., Jones, C.G., Bodik, R.: Sketching concurrent data structures. In: 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 136–148. ACM (2008)

  15. Vechev, M., Yahav, E., Yorsh, G.G.: Abstraction-guided synthesis of synchronization. In: 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 327–338. ACM (2010)

  16. Igel, C., Glasmachers, T., Heidrich-Meisner, V.: Shark. J. Mach. Learn. Res. 9, 993–996 (2008)

    MATH  Google Scholar 

  17. Franzle, M., Herde, C., Teige, T., Ratschan, S., Schubert, T.: Efficient solving of large non-linear arithmetic constraint systems with complex boolean structure. JSAT, Boolean Modeling and Computation 1, 209–236 (2007)

    Google Scholar 

  18. Yeh, I.C.: Modeling of strength of high-performance concrete using artificial neural networks. Cem. Concr. Res. 28(12), 1797–1808 (1998)

    Article  Google Scholar 

  19. Haykin, S.: Neural Networks: a Comprehensive Foundation. Prentice Hall (2008)

  20. Mackworth, A.K.: Consistency in networks of relations. Artif. Intell. 8(1), 99–118 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  21. Van Hentenryck, P.: Numerica: a modeling language for global optimization. In: Fifteenth International Joint Conference on Artificial Intelligence (IJCAI), pp. 1642–1650 (1997)

  22. Rossi, F., Van Beek, P., Walsh, T.: Handbook of Constraint Programming. Elsevier Science Ltd (2006)

  23. Barichard, V., Hao, J.K.: A population and interval constraint propagation algorithm. In: Evolutionary Multi-Criterion Optimization, Second International Conference (EMO 2003), pp. 88–101. Springer (2003)

  24. Marques-Silva, J., Lynce, I., Malik, S.: Conflict-driven Clause Learning SAT Solvers. Handbook of Satisfiability, pp. 131–153. IOS Press, Amsterdam (2009)

    Google Scholar 

  25. Barrett, C., Sebastiani, R., Seshia, S.A., Tinelli, C.: Satisfiability Modulo Theories. Handbook of Satisfiability, pp. 825–885. IOS Press, Amsterdam (2009)

    Google Scholar 

  26. Jermann, C., Sam-Haroud, D., Trombettoni, G. (eds.): CP Workshop on Interval Analysis, Constraint Propagation, Applications (IntCP 2009) (2009)

  27. Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, pp. 238–252 (1977)

  28. Clarke, E., Grumberg, O., Jha, S., Lu, Y., Veith, H.: Counterexample-guided abstraction refinement for symbolic model checking. J. ACM (JACM) 50(5), 794 (2003)

    Article  MathSciNet  Google Scholar 

  29. Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: Yale: rapid prototyping for complex data mining tasks. In: 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’06), pp. 935–940. ACM, New York (2006)

    Chapter  Google Scholar 

  30. Gordeau, R.: Roboop – a robotics object oriented package in C++. http://www.cours.polymtl.ca/roboop (2005)

  31. Rabunal, J.R., Dorrado, J.: Artificial Neural Networks in Real-life Applications. Idea Group Pub (2006)

  32. Witten, I.H., Frank, E.: Data Mining (2nd edn.). Morgan Kaufmann (2005)

  33. Gordon, D.F.: Asimovian adaptive agents. J. Artif. Intell. Res. 13(1), 95–153 (2000)

    MATH  Google Scholar 

  34. Pappas, G., Kress-Gazit, H. (eds.): ICRA Workshop on Formal Methods in Robotics and Automation (2009)

Download references

Author information

Authors and Affiliations

  1. DIST, Università di Genova, Viale Causa, 13, 16145, Genova, Italy

    Luca Pulina & Armando Tacchella

Authors
  1. Luca Pulina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Armando Tacchella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca Pulina.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pulina, L., Tacchella, A. NeVer: a tool for artificial neural networks verification. Ann Math Artif Intell 62, 403–425 (2011). https://doi.org/10.1007/s10472-011-9243-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-011-9243-0

Keywords

Mathematics Subject Classifications (2010)

Search

Navigation