Neural networks can learn complex, non-convex functions, and it is challenging to guarantee their correct behavior in safety-critical contexts. Many approaches exist to find failures in networks (e.g., adversarial examples), but these cannot guarantee the absence of failures. Verification algorithms address this need and provide formal guarantees about a neural network by answering “yes or no” questions. For example, they can answer whether a violation exists within certain bounds. However, individual “yes or no" questions cannot answer qualitative questions such as “what is the largest error within these bounds”; the answers to these lie in the domain of optimization. Therefore, we propose strategies to extend existing verifiers to perform optimization and find: (i) the most extreme failure in a given input region and (ii) the minimum input perturbation required to cause a failure. A naive approach using a bisection search with an off-the-shelf verifier results in many expensive and overlapping calls to the verifier. Instead, we propose an approach that tightly integrates the optimization process into the verification procedure, achieving better runtime performance than the naive approach. We evaluate our approach implemented as an extension of Marabou, a state-of-the-art neural network verifier, and compare its performance with the bisection approach and MIPVerify, an optimization-based verifier. We observe complementary performance between our extension of Marabou and MIPVerify.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price includes VAT (USA)
Tax calculation will be finalised during checkout.
Availability of data and materials
The networks and datasets used for testing can be found at https://github.com/castrong/NeuralOptimization.jl.
The benchmarking framework is available at https://github.com/castrong/NeuralOptimization.jl. The Marabou verifier can be found at https://github.com/NeuralNetworkVerification/Marabou with the optimization extension at https://github.com/castrong/Marabou/tree/opt_branch_7_30. A wrapper to run MIPVerify on these benchmarks is located at https://github.com/castrong/MIPVerifyWrapper and the original implementation is located at https://github.com/vtjeng/MIPVerify.jl. Implementations of adversarial attacks can be found at https://github.com/jaypmorgan/Adversarial.jl.
Solving each LP takes polynomial time, but note that k may be exponentially large compared to the input representation. This exponential growth in the number of output sets is a challenge for both reachability verifiers and optimizers.
Marabou also later integrated the DeepPoly analysis from Singh et al. (2019b), which can derive tighter bounds then the symbolic bound tightening technique. This experimental evaluation was conducted before DeepPoly was made available in Marabou.
Bak, S., Liu, C., & Johnson, T. T. (2021). VNN21. https://sites.google.com/view/vnn2021. Accessed 17 July 2021
Balunovic, M., Baader, M., Singh, G., Gehr, T., & Vechev, M. (2019). Certifying geometric robustness of neural networks. In Advances in neural information processing systems (NIPS) (pp. 15313–15323).
Bjorner, N., Phan, A. D., & Fleckenstein, L. (2015). \(\nu\)z-an optimizing SMT solver. In International conference on tools and algorithms for the construction and analysis of systems (TACAS) (pp. 194–199). Springer.
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., & Zieba, K. (2016). End to end learning for self-driving cars. Technical Report. arXiv:1604.07316
Boopathy, A., Weng, T. W., Chen, P. Y., Liu, S., & Daniel, L. (2019). Cnn-cert: An efficient framework for certifying robustness of convolutional neural networks. AAAI Conference on Artificial Intelligence, 33, 3240–3247.
Botoeva, E., Kouvaros, P., Kronqvist, J., Lomuscio, A., & Misener, R. (2020). Efficient verification of relu-based neural networks via dependency analysis. In AAAI conference on artificial intelligence (AAAI).
Bunel, R., Lu, J., Turkaslan, I., Kohli, P., Torr, P., & Mudigonda, P. (2020). Branch and bound for piecewise linear neural network verification. Journal of Machine Learning Research, 21(2020), 1–39.
Carlini, N., & Wagner, D. (2017). Towards evaluating the robustness of neural networks. In 2017 IEEE symposium on security and privacy (SP) (pp. 39–57). IEEE.
Carlini, N., Katz, G., Barrett, C., & Dill, D. L. (2017). Provably minimally-distorted adversarial examples. arXiv preprint arXiv:170910207
Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A., & Mukhopadhyay, D. (2021). A survey on adversarial attacks and defences. CAAI Transactions on Intelligence Technology, 6(1), 25–45.
Cheng, C. H., Nührenberg, G., & Ruess, H. (2017). Maximum resilience of artificial neural networks. In International symposium on automated technology for verification and analysis (pp. 251–268). Springer.
Fischetti, M., & Jo, J. (2018). Deep neural networks and mixed integer linear optimization. Constraints, 23(3), 296–309. https://doi.org/10.1007/s10601-018-9285-6.
Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, E., Chaudhuri, S., & Vechev, M. (2018). AI2: Safety and robustness certification of neural networks with abstract interpretation. In IEEE symposium on security and privacy (S&P).
Goodfellow, I., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. In International conference on learning representations. arxiv.org/abs/1412.6572
Gurobi Optimization L (2020). Gurobi optimizer reference manual. http://www.gurobi.com
Huang, X., Kwiatkowska, M., Wang, S., & Wu, M. (2017). Safety verification of deep neural networks. In International conference on computer-aided verification (pp. 3–29).
Hunt, K. J., Sbarbaro, D., Zbikowski, R., & Gawthrop, P. J. (1992). Neural networks for control systems—a survey. Automatica, 28(6), 1083–1112.
Johnson, T. T., & Liu, C. (2020). Vnn20. https://sites.google.com/view/vnn20/. Accessed 17 July 2021.
Julian, K., Lopez, J., Brush, J., Owen, M., & Kochenderfer, M. (2016). Policy compression for aircraft collision avoidance systems. In Digital avionics systems conf. (DASC) (pp. 1–10).
Julian, K. D., Lee, R., & Kochenderfer, M. J. (2020). Validation of image-based neural network controllers through adaptive stress testing. In 2020 IEEE 23rd international conference on intelligent transportation systems (ITSC) (pp. 1–7).
Katz, G., Barrett, C., Dill, D. L., Julian, K., & Kochenderfer, M. J. (2017). Reluplex: An efficient smt solver for verifying deep neural networks. In International conference on computer-aided verification (pp. 97–117). Springer.
Katz, G., Huang, D. A., Ibelingm, D., Julian, K., Lazarus, C., Lim, R., Shah, P., Thakoor, S., Wu, H., & Zeljić, A., et al. (2019). The marabou framework for verification and analysis of deep neural networks. In International conference on computer-aided verification (pp. 443–452). Springer.
Kochenderfer, M. J., & Wheeler, T. A. (2019). Algorithms for optimization. London: MIT Press.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (NIPS) (pp. 1097–1105).
Le, Q. V. (2013). Building high-level features using large scale unsupervised learning. In IEEE international conference on acoustics, speech and signal processing(pp. 8595–8598).
Liu, C., Arnon, T., Lazarus, C., Strong, C., Barrett, C., & Kochenderfer, M. J. (2021). Algorithms for verifying deep neural networks. Foundations and Trends® in Optimization,4(3–4), 244–404. https://doi.org/10.1561/2400000035
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017). A survey of deep neural network architectures and their applications. Neurocomputing, 234, 11–26.
Lomuscio, A., & Maganti, L. (2017). An approach to reachability analysis for feed-forward relu neural networks. arXiv preprint arXiv:170607351
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In International conference on learning representations. https://openreview.net/forum?id=rJzIBfZAb
Makhorin, A. (2004). GLPK (Gnu linear programming kit), version 4.42. http://www.gnu.org/software/glpk
Müller, C., Singh, G., Püschel, M., & Vechev, M.T. (2020). Neural network robustness verification on gpus. CoRR arxiv.org/abs/2007.10868
Otter, D. W., Medina, J. R., & Kalita, J. K. (2020). A survey of the usages of deep learning for natural language processing. IEEE Transactions on Neural Networks and Learning Systems, 1–21.
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?” explaining the predictions of any classifier. In ACM SIGKDD International conference on knowledge discovery and data mining (pp. 1135–1144).
Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85–117.
Sebastiani, R., & Trentin, P. (2015). Pushing the envelope of optimization modulo theories with linear-arithmetic cost functions. In International conference on tools and algorithms for the construction and analysis of systems (TACAS) (pp. 335–349). Springer.
Sebastiani, R., & Trentin, P. (2020). Optimathsat: A tool for optimization modulo theories. Journal of Automated Reasoning, 64(3), 423–460.
Singh, G., Gehr, T., Mirman, M., Püschel, M., & Vechev, M. (2018a). Fast and effective robustness certification. In Advances in neural information processing systems (NIPS) (pp. 10802–10813).
Singh, G., Gehr, T., Püschel, M., & Vechev, M. (2018b). Boosting robustness certification of neural networks. In International conference on learning representations.
Singh, G., Ganvir, R., Püschel, M., & Vechev, M. (2019a). Beyond the single neuron convex barrier for neural network certification. In Advances in neural information processing systems (NIPS) (pp. 15098–15109).
Singh, G., Gehr, T., Püschel, M., & Vechev, M. (2019b). An abstract domain for certifying neural networks. In Proceedings of the ACM on programming languages 3(POPL) (pp. 1–30).
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2014). Intriguing properties of neural networks. In International conference on learning representations. arxiv.org/abs/1312.6199
Tjeng, V., Xiao, K. Y., & Tedrake, R. (2019). Evaluating robustness of neural networks with mixed integer programming. In International conference on learning representations. https://openreview.net/forum?id=HyGIdiRqtm
Tran, H. D., Lopez, D. M., Musau, P., Yang, X., Nguyen, L. V., Xiang, W., & Johnson, T. T. (2019). Star-based reachability analysis of deep neural networks. In International symposium on formal methods (pp. 670–686). Springer.
Tran, H. D., Bak, S., Xiang, W., & Johnson, T. T. (2020a). Verification of deep convolutional neural networks using imagestars. In International conference on computer aided verification (pp. 18–42). Springer.
Tran, H. D., Yang, X., Lopez, D. M., Musau, P., Nguyen, L., Xiang, W., et al. (2020). Nnv: The neural network verification tool for deep neural networks and learning-enabled cyber-physical systems. Computer Aided Verification, 12224, 3–17.
Vincent, J. A., & Schwager, M. (2021). Reachable polyhedral marching (rpm): A safety verification algorithm for robotic systems with deep neural network components. In IEEE international conference on robotics and automation (ICRA).
Wang, S., Pei, K., Whitehouse, J., Yang, J., & Jana, S. (2018a). Efficient formal safety analysis of neural networks. In Advances in neural information processing systems (NIPS) (pp. 6367–6377).
Wang, S., Pei, K., Whitehouse, J., Yang, J., & Jana, S. (2018b). Formal security analysis of neural networks using symbolic intervals. In \(USENIX\)security symposium (pp. 1599–1614).
Weng, T. W., Zhang, H., Chen, H., Song, Z., Hsieh, C. J., Boning, D., Dhillon, I. S., & Daniel, L. (2018). Towards fast computation of certified robustness for relu networks. In International conference on machine learning (ICML).
Wolsey, L. A. (1998). Integer programming (Vol. 52). London: Wiley.
Wu, H., Ozdemir, A., Zeljić, A., Julian, K., Irfan, A., Gopinath, D., Fouladi, S., Katz, G., Pasareanu, C., & Barrett, C. (2020). Parallelization techniques for verifying neural networks. In Formal methods in computer aided design (FMCAD).
Xiang, W., Tran, H. D., Rosenfeld, J. A., & Johnson, T. T. (2018). Reachable set estimation and safety verification for piecewise linear systems with neural network controllers. In Annual American control conference (pp. 1574–1579). https://doi.org/10.23919/ACC.2018.8431048
Yuan, X., He, P., Zhu, Q., & Li, X. (2019). Adversarial examples: Attacks and defenses for deep learning. IEEE Transactions on Neural Networks and Learning Systems, 30(9), 2805–2824.
Zhang, H., Weng, T. W., Chen, P. Y., Hsieh, C. J., & Daniel, L. (2018). Efficient neural network robustness certification with general activation functions. In Advances in neural information processing systems (NeurIPS).
Zhu, C., Byrd, R. H., Lu, P., & Nocedal, J. (1997). Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software (TOMS), 23(4), 550–560.
We would like to acknowledge support from Tomer Arnon, Christopher Lazarus, Changliu Liu, Ahmed Irfan, Chelsea Sidrane, Jayesh Gupta, Alex Usvyatsov, Rianna Jitosho, Eric Luxenberg, and Katherine Strong.
Funding in direct support of this work: DARPA under contract FA8750-18-C-0099.
Conflict of interest
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Editors: Daniel Fremont, Alessio Lomuscio, Dragos Margineantu, Cheng Soon-Ong.
About this article
Cite this article
Strong, C.A., Wu, H., Zeljić, A. et al. Global optimization of objective functions represented by ReLU networks. Mach Learn (2021). https://doi.org/10.1007/s10994-021-06050-2
- Neural network verification
- Adversarial examples