Reachable Set Estimation and Verification for Neural Network Models of Nonlinear Dynamic Systems

  • Weiming Xiang
  • Diego Manzanas Lopez
  • Patrick Musau
  • Taylor T. JohnsonEmail author
Part of the Unmanned System Technologies book series (UST)


Neural networks have been widely used to solve complex real-world problems. Due to the complex, nonlinear, non-convex nature of neural networks, formal safety and robustness guarantees for the behaviors of neural network systems are crucial for their applications in safety-critical systems. In this paper, the reachable set estimation and safety verification problems for Nonlinear Autoregressive-Moving Average (NARMA) models in the forms of neural networks are addressed. The neural networks involved in the model are a class of feed-forward neural networks called Multi-Layer Perceptrons (MLPs). By partitioning the input set of an MLP into a finite number of cells, a layer-by-layer computation algorithm is developed for reachable set estimation of each individual cell. The union of estimated reachable sets of all cells forms an over-approximation of the reachable set of the MLP. Furthermore, an iterative reachable set estimation algorithm based on reachable set estimation for MLPs is developed for NARMA models. The safety verification can be performed by checking the existence of non-empty intersections between unsafe regions and the estimated reachable set. Several numerical examples are provided to illustrate the approach.



The material presented in this paper is based upon work supported by the National Science Foundation (NSF) under grant numbers CNS 1464311, CNS 1713253, SHF 1527398, and SHF 1736323, the Air Force Office of Scientific Research (AFOSR) through contract numbers FA9550-15-1-0258, FA9550-16-1-0246, and FA9550-18-1-0122, the Defense Advanced Research Projects Agency (DARPA) through contract number FA8750-18-C-0089, and the Office of Naval Research through contract number N00014-18-1-2184. The US government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation thereon. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of AFOSR, DARPA, NSF, or ONR.


  1. 1.
    J.I. Baig, A. Mahmood, Robust control design of a magnetic levitation system, in 2016 19th International Multi-Topic Conference (INMIC) (2016), pp. 1–5Google Scholar
  2. 2.
    S. Bak, P. Sridhar Duggirala, HyLAA: a tool for computing simulation-equivalent reachability for linear systems, in Proceedings of the 20th International Conference on Hybrid Systems: Computation and Control (ACM, New York, 2017), pp. 173–178Google Scholar
  3. 3.
    S. Bak, P. Sridhar Duggirala, Rigorous simulation-based analysis of linear hybrid systems, in International Conference on Tools and Algorithms for the Construction and Analysis of Systems (Springer, Berlin, 2017), pp. 555–572Google Scholar
  4. 4.
    M.H. Beale, M.T. Hagan, H.B. Demuth, Neural network toolbox user’s guide, in R2016a. The MathWorks, Inc., Natick (2012).
  5. 5.
    P.J. Berkelman, R.L. Hollis, Lorentz magnetic levitation for haptic interaction: device design, performance, and integration with physical simulations. Int. J. Robot. Res. 19(7), 644–667 (2000)CrossRefGoogle Scholar
  6. 6.
    M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L.D. Jackel, M. Monfort, U. Muller, J. Zhang, et al. End to end learning for self-driving cars (2016). Arxiv preprint arXiv:1604.07316Google Scholar
  7. 7.
    O. De Jesus, A. Pukrittayakamee, M.T. Hagan, A comparison of neural network control algorithms, in International Joint Conference on Neural Networks, 2001. Proceedings. IJCNN ’01., vol. 1 (2001), pp. 521–526Google Scholar
  8. 8.
    R.J. Duffin, Free suspension and earnshaw’s theorem. Arch. Ration. Mech. Anal. 14(1), 261–263 (1963)MathSciNetCrossRefGoogle Scholar
  9. 9.
    P.S. Duggirala, S. Mitra, M. Viswanathan, M. Potok, C2E2: a verification tool for stateflow models, in International Conference on Tools and Algorithms for the Construction and Analysis of Systems (Springer, Berlin, 2015), pp. 68–82CrossRefGoogle Scholar
  10. 10.
    A. El Hajjaji, M. Ouladsine, Modeling and nonlinear control of magnetic levitation systems. IEEE Trans. Ind. Electron. 48(4), 831–838 (2001)CrossRefGoogle Scholar
  11. 11.
    C. Fan, B. Qi, S. Mitra, M. Viswanathan, P. Sridhar Duggirala, Automatic reachability analysis for nonlinear hybrid models with C2E2, in International Conference on Computer Aided Verification (Springer, Berlin, 2016), pp. 531–538Google Scholar
  12. 12.
    S.S. Ge, C.C. Hang, T. Zhang, Adaptive neural network control of nonlinear systems by state and output feedback. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 29(6), 818–828 (1999)CrossRefGoogle Scholar
  13. 13.
    K. Hornik, M. Stinchcombe, H. White, Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)CrossRefGoogle Scholar
  14. 14.
    X. Huang, M. Kwiatkowska, S. Wang, M. Wu, Safety verification of deep neural networks (2016). Arxiv preprint arXiv:1610.06940Google Scholar
  15. 15.
    K. Jetal Hunt, D. Sbarbaro, R. Żbikowski, P.J Gawthrop, Neural networks for control systems: a survey. Automatica 28(6), 1083–1112 (1992)MathSciNetCrossRefGoogle Scholar
  16. 16.
    J. Kaloust, C. Ham, J. Siehling, E. Jongekryg, Q. Han, Nonlinear robust control design for levitation and propulsion of a maglev system. IEE Proc. Control Theory Appl. 151(4), 460–464 (2004)CrossRefGoogle Scholar
  17. 17.
    G. Katz, C. Barrett, D. Dill, K. Julian, M. Kochenderfer, Reluplex: an efficient SMT solver for verifying deep neural networks (2017). Arxiv preprint arXiv:1702.01135Google Scholar
  18. 18.
    C.H. Kim, J. Lim, J.M. Lee, H.S. Han, D.Y. Park, Levitation control design of super-speed maglev trains, in 2014 World Automation Congress (WAC) (2014), pp. 729–734Google Scholar
  19. 19.
    S. Lawrence, C. Lee Giles, Ah. Chung Tsoi, A.D Back, Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Netw. 8(1), 98–113 (1997)CrossRefGoogle Scholar
  20. 20.
    X.-D. Li, J.K.L. Ho, T.W.S. Chow, Approximation of dynamical time-variant systems by continuous-time recurrent neural networks. IEEE Trans. Circuits Syst. Express Briefs 52(10), 656–660 (2005)CrossRefGoogle Scholar
  21. 21.
    L.S.H. Ngia, J. Sjoberg, Efficient training of neural nets for nonlinear adaptive filtering using a recursive levenberg-marquardt algorithm. IEEE Trans. Signal Process. 48(7), 1915–1927 (2000)CrossRefGoogle Scholar
  22. 22.
    M. Ono, S. Koga, H. Ohtsuki, Japan’s superconducting maglev train. IEEE Instrum. Meas. Mag. 5(1), 9–15 (2002)CrossRefGoogle Scholar
  23. 23.
    L. Pulina, A. Tacchella, An abstraction-refinement approach to verification of artificial neural networks, in International Conference on Computer Aided Verification (Springer, Berlin, 2010), pp. 243–257Google Scholar
  24. 24.
    L. Pulina, A. Tacchella, Challenging SMT solvers to verify neural networks. AI Commun. 25(2), 117–135 (2012)MathSciNetzbMATHGoogle Scholar
  25. 25.
    D.M. Rote, Y. Cai, Review of dynamic stability of repulsive-force maglev suspension systems. IEEE Trans. Mag. 38(2), 1383–1390 (2002)CrossRefGoogle Scholar
  26. 26.
    J. Schmidhuber, Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)CrossRefGoogle Scholar
  27. 27.
    D. Silver, A. Huang, C.J Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)CrossRefGoogle Scholar
  28. 28.
    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, R. Fergus, Intriguing properties of neural networks (2013). Arxiv preprint arXiv:1312.6199Google Scholar
  29. 29.
    R. Uswarman, A.I. Cahyadi, O. Wahyunggoro, Control of a magnetic levitation system using feedback linearization, in 2013 International Conference on Computer, Control, Informatics and Its Applications (IC3INA) (2013), pp. 95–98Google Scholar
  30. 30.
    M. Viet Thuan, H. Manh Tran, H. Trinh, Reachable sets bounding for generalized neural networks with interval time-varying delay and bounded disturbances. Neural Comput. Appl. 29(10), 783–794 (2018)CrossRefGoogle Scholar
  31. 31.
    R.J. Wai, J.D. Lee, Robust levitation control for linear maglev rail system using fuzzy neural network. IEEE Trans. Control Syst. Technol. 17(1), 4–14 (2009)CrossRefGoogle Scholar
  32. 32.
    W. Xiang, On equivalence of two stability criteria for continuous-time switched systems with dwell time constraint. Automatica 54, 36–40 (2015)MathSciNetCrossRefGoogle Scholar
  33. 33.
    W. Xiang, Necessary and sufficient condition for stability of switched uncertain linear systems under dwell-time constraint. IEEE Trans. Automatic Control 61(11), 3619–3624 (2016)MathSciNetCrossRefGoogle Scholar
  34. 34.
    W. Xiang, Parameter-memorized Lyapunov functions for discrete-time systems with time-varying parametric uncertainties. Automatica 87, 450–454 (2018)MathSciNetCrossRefGoogle Scholar
  35. 35.
    W. Xiang, J. Xiao, Stabilization of switched continuous-time systems with all modes unstable via dwell time switching. Automatica 50(3), 940–945 (2014)MathSciNetCrossRefGoogle Scholar
  36. 36.
    W. Xiang, J. Lam, J. Shen, Stability analysis and \(\mathcal {L}_1\)-gain characterization for switched positive systems under dwell-time constraint. Automatica 85, 1–8 (2017)CrossRefGoogle Scholar
  37. 37.
    W. Xiang, H.-D. Tran, T.T. Johnson, Robust exponential stability and disturbance attenuation for discrete-time switched systems under arbitrary switching. IEEE Trans. Autom. Control (2017). MathSciNetCrossRefGoogle Scholar
  38. 38.
    W. Xiang, H.-D. Tran, T.T. Johnson, On reachable set estimation for discrete-time switched linear systems under arbitrary switching, in American Control Conference (ACC), 2017 (IEEE, New York, 2017), pp. 4534–4539Google Scholar
  39. 39.
    W. Xiang, H.-D. Tran, T.T. Johnson, Output reachable set estimation and verification for multi-layer neural networks (2017). Arxiv preprint arXiv:1708.03322Google Scholar
  40. 40.
    W. Xiang, H.-D. Tran, T.T. Johnson, Output reachable set estimation for switched linear systems and its application in safety verification. IEEE Trans. Autom. Control 62(10), 5380–5387 (2017)MathSciNetCrossRefGoogle Scholar
  41. 41.
    W. Xiang, H.-D. Tran, T.T. Johnson, Reachable set computation and safety verification for neural networks with ReLU activations (2017). Arxiv preprint arXiv: 1712.08163Google Scholar
  42. 42.
    Z. Xu, H. Su, P. Shi, R. Lu, Z.-G. Wu, Reachable set estimation for Markovian jump neural networks with time-varying delays. IEEE Trans. Cybern. 47(10), 3208–3217 (2017)CrossRefGoogle Scholar
  43. 43.
    L. Zhang, W. Xiang, Mode-identifying time estimation and switching-delay tolerant control for switched systems: an elementary time unit approach. Automatica 64, 174–181 (2016)MathSciNetCrossRefGoogle Scholar
  44. 44.
    L. Zhang, Y. Zhu, W.X. Zheng, Synchronization and state estimation of a class of hierarchical hybrid neural networks with time-varying delays. IEEE Trans. Neural Netw. Learn. Syst. 27(2), 459–470 (2016)MathSciNetCrossRefGoogle Scholar
  45. 45.
    L. Zhang, Y. Zhu, W.X. Zheng, State estimation of discrete-time switched neural networks with multiple communication channels. IEEE Trans. Cybern. 47(4), 1028–1040 (2017)CrossRefGoogle Scholar
  46. 46.
    S.T. Zhao, X.W. Gao, Neural network adaptive state feedback control of a magnetic levitation system, in The 26th Chinese Control and Decision Conference (2014 CCDC) (2014), pp. 1602–1605Google Scholar
  47. 47.
    Z. Zuo, Z. Wang, Y. Chen, Y. Wang, A non-ellipsoidal reachable set estimation for uncertain neural networks with time-varying delay. Commun. Nonlinear Sci. Numer. Simul. 19(4), 1097–1106 (2014)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Weiming Xiang
    • 1
  • Diego Manzanas Lopez
    • 1
  • Patrick Musau
    • 1
  • Taylor T. Johnson
    • 1
    Email author
  1. 1.Department of Electrical Engineering and Computer ScienceVanderbilt UniversityNashvilleUSA

Personalised recommendations