Abstract
JKind is an open-source industrial model checker developed by Rockwell Collins and the University of Minnesota. JKind uses multiple parallel engines to prove or falsify safety properties of infinite state models. It is portable, easy to install, performance competitive with other state-of-the-art model checkers, and has features designed to improve the results presented to users: inductive validity cores for proofs and counterexample smoothing for test-case generation. It serves as the back-end for various industrial applications.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
1 Introduction
JKind is an open-sourceFootnote 1 industrial infinite-state inductive model checker for safety properties. Models and properties in JKind are specified in Lustre [17], a synchronous data-flow language, using the theories of linear real and integer arithmetic. JKind uses SMT-solvers to prove and falsify multiple properties in parallel. A distinguishing characteristic of JKind is its focus on the usability of results. For a proven property, JKind provides traceability between the property and individual model elements. For a falsified property, JKind provides options for simplifying the counterexample in order to highlight the root cause of the failure. In industrial applications, we have found these additional usability aspects to be at least as important as the primary results. Another important characteristic of JKind is that is it designed to be integrated directly into user-facing applications. Written in Java, JKind runs on all major platforms and is easily compiled into other Java applications. JKind bundles the Java-based SMTInterpol solver and has no external dependencies. However, it can optionally call Z3, Yices 1, Yices 2, CVC4, and MathSAT if they are available.
2 Functionality and Main Features
JKind is structured as several parallel engines that coordinate to prove properties, mimicking the design of PKind and Kind 2 [8, 21]. Some engines are directly responsible for proving properties, others aid that effort by generating invariants, and still others are reserved for post-processing of proof or counterexample results. Each engine can be enabled or disabled separately based on the user’s needs. The architecture of JKind allows any engine to broadcast information to the other engines (for example, lemmas, proofs, counterexamples) allowing straightforward integration of new functionality.
The solving engines in JKind are show in Fig. 1. The Bounded Model Checking (BMC) engine performs a standard iterative unrolling of the transition relation to find counterexamples and to serve as the base case of k-induction. The BMC engine guarantees that any counterexample it finds is minimal in length. The k -Induction engine performs the inductive step of k-induction, possibly using invariants generated by other engines. The Invariant Generation engine uses a template-based invariant generation technique [22] using its own k-induction loop. The Property Directed Reachability (PDR) engine performs property directed reachability [11] using the implicit abstraction technique [9]. Unlike BMC and k-induction, each property is handled separately by a different PDR sub-engine. Finally, the Advice engine produces invariants based on previous runs of JKind as described in the next section.
Invariant sharing between the solvers (shown in Fig. 1) is an important part of the architecture. In our internal benchmarking, we have found that implicit abstraction PDR performs best when operating over a single property at a time and without use of lemmas generated by other approaches. On the other hand, the invariants generated by PDR and template lemma generation often allow k-induction, which operates on all properties in parallel, to substantially reduce the verification time required for models with large numbers of properties.
2.1 Post Processing and Re-verification
A significant part of the research and development effort for JKind has focused on post-processing results for presentation and repeated verification of models under development.
Inductive Validity Cores (IVC). For a proven property, an inductive validity core is a subset of Lustre equations from the input model for which the property still holds [13, 14]. Inductive validity cores can be used for traceability from property to model elements and determining coverage of the model by a set of properties [15]. This facility can be used to automatically generate traceability and adequacy information (such as traceability matrices [12] important to the certification of safety-critical avionics systems [26]). The IVC engine uses a heuristic algorithm to efficiently produce minimal or nearly minimal cores. In a recent experiment over a superset of the benchmark models described in the experiment in Sect. 3, we found that our heuristic IVC computation added 31% overhead to model checking time, and yielded cores approximately 8% larger than the guaranteed minimal core computed by a very expensive “brute force” algorithm. As a side-effect, the IVC algorithm also minimizes the set of invariants used to prove a property and emits this reduced set to other engines (notably the Advice engine, described below).
Smoothing. To aid in counterexample understanding and in creating structural coverage tests that can be more easily explained, JKind provides an optional post-processing step to minimize the number of changes to input variables—smoothing the counterexample. For example, applied to 129 test cases generated for a production avionics flight control state machine, smoothing increased runtime by 40% and removed 4 unnecessary input changes per test case on average. The smoothing engine uses a MaxSat query over the original BMC-style unrolling of the transition relation combined with weighted assertions that each input variable does not change on each step. The MaxSat query tries to satisfy all of these weighted assertions, but will break them if needed. This has the effect of trying to hold all inputs constant while still falsifying the original property and only allowing inputs to change when needed. This engine is only available with SMT-solvers that support MaxSat such as Yices 1 and Z3.
Advice. The advice engine saves and re-uses the invariants that were used by JKind to prove the properties of a model. Prior to analysis, JKind performs model slicing and flattening to generate a flat transition-relation model. Internally, invariants are stored as a set of proven formulas (in the Lustre syntax) over the variables in the flattened model. An advice file is simply the emitted set of these invariant formulas. When a model is loaded, the formulas are loaded into memory. Formulas that are no longer syntactically or type correct are discarded, and the remaining set of formulas are submitted as an initial set of possible invariants to be proved via k-induction. If they are proved, they are passed along to other engines; if falsified, they are discarded. Names constructed between multiple runs of JKind are stable, so if a model is unchanged, it can be usually be re-proved quickly using the invariants and k-induction. If the model is slightly changed, it is often the case that most of the invariants can be re-proved, leading to reduced verification times.
If the IVC engine is also enabled, then advice emits a (close to) minimal set of lemmas used for proof; this often leads to faster re-verification (but more expensive initial verification), and can be useful for examining which of the generated lemmas are useful for proofs.
3 Experimental Evaluation
We evaluated the performance of JKind against Kind 2 [8], Zustre [20], Generalized PDR in Z3 [19], and IC3 in nuXmv [9]. We used the default options for each tool (using check_invar_ic3 for nuXmv). Our benchmark suite comes from [9] and contains 688 models over the theory of linear integer arithmeticFootnote 2. All experiments were performed on a 64-bit Ubuntu 17.10 Linux machine with a 12-core Intel Xeon CPU E5-1650 v3 @ 3.50 GHz, with 32 GB of RAM and a time limit of 60 s per model.
Performance comparisons are show in Fig. 2. The key describes the number of benchmarks solved for each tool, and the graph shows the aggregate time required for solving, ordered by time required per-problem, ordered independently for each tool. JKind was able to verify or falsify the most properties, although Z3 was often the fastest tool. Many of the benchmarks in this set are quickly evaluated: Z3 solves the first 400 benchmarks in just over 12 s. Due to JKind’s use of Java, the JVM/JKind startup time for an empty model is approximately 0.35s, which leads to poor performance on small modelsFootnote 3. As always, such benchmarks should be taken with a large grain of salt. In [8], a different set of benchmarks slightly favored Kind 2, and in [9], nuXmv was the most capable tool. We believe that all the solvers are relatively competitive.
4 Integration and Applications
JKind is the back-end for a variety of user-facing applications. In this section, we briefly highlight a few of these applications and how they employ the features discussed previously.
-
(1)
The Specification and Analysis of Requirements (SpeAR) tool is an open-source tool for prototyping and analysis of requirements [12]. Starting from a set of formalized requirements, SpeAR uses JKind to determine whether or not the requirements meet certain properties. It uses IVCs to create a traceability matrix between requirements and properties, highlighting unused requirements, over-constrained properties, and other common problems. SpeAR also uses JKind with smoothing for test-case generation using the Unique First Cause criteria [28].
-
(2)
The Assume Guarantee Reasoning Environment (AGREE) tool is an open-source compositional verification tool that proves properties of hierarchically-composed models in the Architectural Analysis and Design Language (AADL) language [3, 10, 23]. AGREE makes use of multiple JKind features including smoothing to present clear counterexamples, IVC to show requirements traceability, and counterexample generation to check the consistency of an AADL component’s contract. AGREE also uses JKind for test-case generation from component contracts.
-
(3)
The Static IMPerative AnaLyzer (SIMPAL) tool is an open-source tool for compositional reasoning over software [27]. SIMPAL is based on Limp, a Lustre-like imperative language with extensions for control flow elements, global variables, and a syntax for specifying preconditions, postconditions, and global variable interactions of preexisting components. SIMPAL translates Limp programs to an equivalent Lustre representation which is passed to JKind to perform assume-guarantee reasoning, reachability, and viability analyses.
-
(4)
JKind is also used by two proprietary tools used by product areas within Rockwell Collins. The first is a Mode Transition Table verification tool used for the complex state machines which manage flight modes of an aircraft. JKind is used to check properties and generate tests for mode and transition coverage from Lustre models generated from the state machines. IVCs are used to establish traceability, i.e. which transitions are covered by which properties. The second is a Crew Alerting System MC/DC test-case generation tool for a proprietary domain-specific language used for messages and alerts to airplane pilots. Smoothing is very important in this context as test cases need to be run on the actual hardware where timing is not precisely controllable. Thus, test cases with a minimum of changes to the inputs are ideal.
5 Related Work
JKind is one of a number of similar infinite-state inductive model checkers including Kind 2 [8], nuXmv [9], Z3 with generalized PDR [19], and Zustre [20]. They operate over a transition relation described either as a Lustre program (Kind 2, JKind, and Zustre), an extension of the SMV language (nuXmv), or as a set of Horn clauses (Z3). Each tool uses a portfolio-based solver approach, with nuXmv, JKind, and Kind 2 all supporting both k-induction and a variant of PDR/IC3. nuXmv also supports guided reachability and k-liveness. Other tools such as ESBMC-DepthK [25], VVT [4] CPAchecker, [5], CPROVER [7] use similar techniques for reasoning about C programs.
We believe that the JKind IVC support is similar to proof-core support provided by commercial hardware model checkers: Cadence Jasper Gold and Synopsys VC Formal [1, 2, 18]. The proof-core provided by these tools is used for internal coverage analysis measurements performed by the tools. Unfortunately, the algorithms used in the commercial tool support are undocumented and performance comparisons are prohibited by the tool licenses, so it is not possible to compare performance on this aspect.
Previous work has been done on improving the quality of counterexamples along various dimensions similar to the JKind notion of smoothing, e.g. [16, 24]. Our work is distinguished by its focus on minimizing the number of deltas in the input values. This metric has been driven by user needs and by our own experiences with test-case generation.
There are several tools that support reuse or exchange of verification results, similar to our advice feature. Recently, there has been progress on standardized formats [6] of exchange between analysis tools. Our current advice format is optimized for use and performance with our particular tool and designed for re-verification rather than exchange of partial verification information. However, supporting a standardized format for exchanging verification information would be a useful feature for future use.
6 Conclusion
JKind is similar to a number of other solvers that each solve infinite state sequential analysis problems. Nevertheless, it has some important features that distinguish it. First, a focus on quality of feedback to users for both valid properties (using IVCs) and invalid properties (using smoothing). Second, it is supported across all major platforms and is straightforward to port due to its implementation in Java. Third, it is small, modular, and well-architected, allowing straightforward extension with new engines. Fourth, it is open-source with a liberal distribution license (BSD), so it can be adapted for various purposes, as demonstrated by the number of tools that have incorporated it.
Notes
- 1.
- 2.
https://es.fbk.eu/people/griggio/papers/tacas14-ic3ia.tar.bz2. Note that we removed 263 duplicate benchmarks from the original set.
- 3.
Without startup time, the curve for JKind is close to the curve for Zustre.
References
Cadence JasperGold Formal Verification Platform. https://www.cadence.com/
Synopsys VC Formal Platform. https://www.synopsys.com
Backes, J., Cofer, D., Miller, S., Whalen, M.W.: Requirements analysis of a quad-redundant flight control system. In: Havelund, K., Holzmann, G., Joshi, R. (eds.) NFM 2015. LNCS, vol. 9058, pp. 82–96. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17524-9_7
Beyer, D., Dangl, M.: SMT-based software model checking: an experimental comparison of four algorithms. In: Blazy, S., Chechik, M. (eds.) VSTTE 2016. LNCS, vol. 9971, pp. 181–198. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48869-1_14
Beyer, D., Dangl, M., Wendler, P.: Boosting k-Induction with continuously-refined invariants. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 622–640. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21690-4_42
Beyer, D., et al.: Correctness witnesses: exchanging verification results between verifiers. In: FSE, pp. 326–337 (2016)
Brain, M., Joshi, S., Kroening, D., Schrammel, P.: Safety verification and refutation by k-invariants and k-induction. In: Blazy, S., Jensen, T. (eds.) SAS 2015. LNCS, vol. 9291, pp. 145–161. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-48288-9_9
Champion, A., Mebsout, A., Sticksel, C., Tinelli, C.: The Kind 2 model checker. In: Chaudhuri, S., Farzan, A. (eds.) CAV 2016. LNCS, vol. 9780, pp. 510–517. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41540-6_29
Cimatti, A., Griggio, A., Mover, S., Tonetta, S.: IC3 modulo theories via implicit predicate abstraction. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 46–61. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54862-8_4
Cofer, D., Gacek, A., Miller, S., Whalen, M.W., LaValley, B., Sha, L.: Compositional verification of architectural models. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 126–140. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28891-3_13
Eén, N., et al.: Efficient implementation of property directed reachability. In: FMCAD (2011)
Fifarek, A.W., Wagner, L.G., Hoffman, J.A., Rodes, B.D., Aiello, M.A., Davis, J.A.: SpeAR v2.0: formalized past LTL specification and analysis of requirements. In: Barrett, C., Davies, M., Kahsai, T. (eds.) NFM 2017. LNCS, vol. 10227, pp. 420–426. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57288-8_30
Ghassabani, E., et al.: Efficient generation of all minimal inductive validity cores. In: FMCAD (2017)
Ghassabani, E., et al.: Efficient generation of inductive validity cores for safety properties. In: FSE (2016)
Ghassabani, E., et al.: Proof-based coverage metrics for formal verification. In: ASE (2017)
Groce, A., Kroening, D.: Making the most of BMC counterexamples. Proc. BMC 2004 ENTCS 119, 67–81 (2005)
Halbwachs, N., et al.: The synchronous dataflow programming language Lustre. IEEE (1991)
Hanna, Z., et al.: Formal verification coverage metrics for circuit design properties (2015). https://www.google.com/patents/US20150135150
Hoder, K., Bjørner, N.: Generalized property directed reachability. In: Cimatti, A., Sebastiani, R. (eds.) SAT 2012. LNCS, vol. 7317, pp. 157–171. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31612-8_13
Kahsai, T., Bourbouh, H.: Zustre product home page. https://github.com/coco-team/zustre
Kahsai, T., Tinelli, C.: PKIND: a parallel k-induction based model checker. In: PDMC (2011)
Kahsai, T., Garoche, P.-L., Tinelli, C., Whalen, M.: Incremental verification with mode variable invariants in state machines. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 388–402. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28891-3_35
Murugesan, A., et al.: Compositional verification of a medical device system. In: HILT (2013)
Ravi, K., Somenzi, F.: Minimal assignments for bounded model checking. In: Jensen, K., Podelski, A. (eds.) TACAS 2004. LNCS, vol. 2988, pp. 31–45. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24730-2_3
Rocha, H., et al.: Model checking embedded C software using k-induction and invariants. In: SBESC (2015)
RTCA DO-178C: Software Considerations in Airborne Systems and Equipment Certification (2011)
Wagner, L., et al.: SIMPAL: a compositional reasoning framework for imperative programs. In: SPIN (2017)
Whalen, M.W., et al.: Coverage metrics for requirements-based testing. In: ISSTA (2006)
Acknowledgments
The work presented here was sponsored by DARPA as part of the HACMS program under contract FA8750-12-9-0179.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
<SimplePara><Emphasis Type="Bold">Open Access</Emphasis>This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.</SimplePara><SimplePara>The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.</SimplePara>
Copyright information
© 2018 The Author(s)
About this paper
Cite this paper
Gacek, A., Backes, J., Whalen, M., Wagner, L., Ghassabani, E. (2018). The JKind Model Checker. In: Chockler, H., Weissenbacher, G. (eds) Computer Aided Verification. CAV 2018. Lecture Notes in Computer Science(), vol 10982. Springer, Cham. https://doi.org/10.1007/978-3-319-96142-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-96142-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-96141-5
Online ISBN: 978-3-319-96142-2
eBook Packages: Computer ScienceComputer Science (R0)