Wit4Java: A Violation-Witness Validator for Java Veriﬁers (Competition Contribution)

. We describe and evaluate a violation-witness validator for Java ver-iﬁers called Wit4Java. It takes a Java program with a safety property and the respective violation-witness output by a Java veriﬁer to generate a new Java program whose execution deterministically violates the property. We extract the value of the program variables from the counterexample represented by the violation-witness and feed this information back into the original program. In addition, we have two implementations for instantiating source programs by injecting counterexamples. Experimental results show that Wit4Java can correctly validate the violation-witnesses produced by JBMC and GDart in a few seconds.


Overview
Witness validation is the process of checking whether the same results can be reproduced independently according to the given program, specification, verification result, and the generated witness, improving the trust level of the software verifiers [2].
Here, we describe and evaluate a new violation-witness validator for Java programs called Wit4Java. We take an approach similar to Rocha et al. [5] and Beyer et al. [1] for C programs and apply it to Java programs. As a result, we implement Wit4Java as a Python script that creates a new Java program or a unit test case using Mockito with the program variable values extracted from the counterexample. As input, Wit4Java uses the violation-witness in the GraphML format to extract the value of the non-deterministic variables in Java programs. Lastly, Wit4Java runs the new created program using the Java Virtual Machine (JVM) to check the assert statements.
There are some validators for C programs in the literature [6,12]. For example, NitWit is an interpretation-based witness validator that can execute each statement stepby-step without compiling the entire program [12]. The concept of MetaVal is to generate a new program based on the input and then use any checker to check for specifications [6]. CPA-witness2test and FShell-witness2test are execution-based validators for C programs that can process the witness in GraphML format and generate a test harness that drives the program to the specification violation [1]. Rocha et al. focus on the counterexample produced by ESBMC [4] while CPA-witness2test and FShell-witness2test can process GraphML files. However, witness validation for SV-COMP's Java track [7] * Jury Member is still at an early stage. GWIT is another validator that uses assumptions to prune the search space for dynamic symbolic execution, limiting the analysis to paths where a given assumption holds [10,11].

Validation Approach
The architecture of Wit4Java is illustrated in Fig. 1. First, Wit4Java takes the Java program and the witness as input. Then, it uses the Python package NetworkX to read the graph content of the witness and extracts the counterexample values of the variables corresponding to the source program from the violation-witness and saves them. After that, it generates new programs that contain the witness's assumptions. Finally, the validation process is performed by the JVM (using the -ea option) to check whether the execution of the generated program exhibits the detected assertion failure.
There are two implementations (Wit4Java 1.0 and Wit4Java 2.0) to extract and use counterexamples. The first version is to save them as tuples (linenum, counterexample). Then it reads the source program and replaces the variables of the program statements with counterexamples if the line number and variable in the program match the tuple, thus generating a new created Java program. In comparison, the second version records the data types and values of the counterexamples and saves them sequentially into two lists. Moreover, only the assumptions made in the witness for the non-deterministic variables (as determined by Verifier.nondet) are recorded. Then, it builds a unit test case and employs the Mockito framework to mock the Verifier.nondet calls in the source program to make them return deterministic counterexample values from the lists. This makes the execution of the source program follow the path described in the witness and eventually reach the violated property.    [3,8] produces an empty witness, or the witness does not contain a counterexample for a non-deterministic value. Besides, the validation for strings is not supported yet, which occurs in almost half of witnesses because JBMC does not yet output counterexample values for strings. Thus we were not able to test it. Generally, there are not enough witnesses of high quality for testing the witness validator yet because JBMC sometimes correctly terminates without producing a witness in SV-COMP. The witness support in the Java verifiers requires further development work so that they are able to produce complete violation witnesses whenever they terminate with verdict false.

Tool Setup and Configuration
The competition submission is based on Wit4Java version 1.0 (naive version). 1 For the competition [9], Wit4Java is called by executing the script wit4java.py. It reads .java source files and corresponding witnesses in the given benchmark directories.
The answer would be false if the assertion failure was found. As an example, we can validate the witness by executing the following command: ./wit4java.py -witness <path-to-sv-witnesses>/witness.graphml <path-tosv-benchmarks>/java/jbmc-regression/return2 where witness.graphml indicates the witness to be validated, and return2 indicates the benchmark name. The Benchexec tool info module is called wit4java.py and the benchmark definition file wit4java-validate-violation-witnesses.xml. NetworkX should be installed separately in the SV-COMP machines. If a validation task does not find a property violation, it will return unknown.

Software Project and Contributors
Tong Wu maintains Wit4Java. It is publicly available under a BSD-style license. The source code is available at https://github.com/Anthonysdu/wit4java, and instructions for running the tool are given in the README file.