Testing Techniques

Abstract

Verification techniques, as discussed in Chaps. 10 and 13, are to ascertain—with mathematical proof—whether a property holds on a mathematical model. The soundness and completeness required by the mathematical proof result in the scalability problem that verification algorithms can only work with either small models (e.g., the MILP-based method as in Sect. 11.2) or limited number of input dimensions (e.g., the reachability analysis as in Sect. 11.3). In practice, when working with real-world systems where the machine learning models are large in nature, other techniques have to be considered for the certification purpose. Similar to traditional software testing against software verification, neural network testing provides a certification methodology with a balance between completeness and efficiency. In established industries, e.g., avionics and automotive, the needs for software testing have been settled in various standards such as DO-178C and MISRA. However, due to the lack of logical structures and system specification, it is less straightforward on how to extend such standards to work with systems with neural network components. In the following, we discuss some existing neural network testing techniques. The readers are referred to the survey (Huang et al., Comput. Sci. Rev. 37:100270, 2020) for more discussion.