Is There a Bug in the Program? Structural Program Testing

Abstract:

As Black-Box (BB) testing (see Chap. 4), Structural (White-Box) program testing is an experiment with the program, in which the actual program results are compared with the expected ones, prescribed by the testing “oracle,” ideally the program specification. However, in structural testing, tests are synthesized on the basis of the code itself, rather than its specification. A structural testing strategy is defined in terms of Required Elements, i.e, program statements, or combinations thereof, that are to be exercised. The best known coverage criteria are expressed in terms of simple properties of the flow of control: These are statement and branch coverage. Less known, but potentially more powerful are data flow coverage criteria: Definition-Use Chains (Data Dependency in Chap. 7) and its generalization U(se)-Context. All four strategies are supported by STAD. However, due to the difficulties of test synthesis structural testing as a stand-alone method is currently out of the question. It can only be used as a measure of completeness of BB-testing. The main weakness of structural testing is the lack of sound theoretical foundations. To remedy this situation, an attempt has been made to (1) define formally the notions of program faults and errors in terms of the program verification schema and (2) formulate testing strategies in terms of program dependencies. Also, it has been suggested that besides the standard “main” objective of program testing, the detection of the fault, testing also offers a measure of fault localization and, for a passing test suite, the degree of confidence that the program is indeed correct.