Verifying Neural Networks by Approximating Convex Hulls

Abstract

The increasing prevalence of neural networks necessitates their verification in order to ensure security. Verifying neural networks is a challenge due to the use of non-linear activation functions. This work concentrates on approximating the convex hull of activation functions. An approach is proposed to construct a convex polytope to over-approximate the ReLU hull (the convex hull of the ReLU function) when considering multi-variables. The key idea is to construct new faces based on the known faces and vertices by uniqueness of the ReLU hull. Our approach has been incorporated into the state-of-the-art PRIMA framework, which takes into account multi-neuron constraints. The experimental evaluation demonstrates that our method is more efficient and precise than existing ReLU hull exact/approximate approaches, and it makes a significant contribution to the verification of neural networks. Our concept can be applied to other non-linear functions in neural networks, and this could be explored further in future research.