Abstract
In formal verification, we verify that a system is correct with respect to a specification. Even when the system is proven to be correct, there is still a question of how complete the specification is, and whether it really covers all the behaviors of the system. In this paper we study coverage metrics for model checking.Coverage metrics are based on modifications we apply to the system in order to check which parts of it were actually relevant for the verification process to succeed. We introduce two principles that we believe should be part of any coverage metric for model checking: a distinction between state-based and logicbased coverage, and a distinction between the system and its environment. We suggest several coverage metrics that apply these principles, and we describe two algorithms for finding the uncovered parts of the system under these definitions. The first algorithm is a symbolic implementation of a naive algorithm that model checks many variants of the original system. The second algorithm improves the naive algorithm by exploiting overlaps in the variants. We also suggest a few helpful outputs to the user, once the uncovered parts are found.
Supported in part by NSF grant CCR-9700061, NSF grant CCR-9988322, and by a grant from the Intel Corporation.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
D. Beaty and R. Bryant. Formally verifying a microprocessor using a simulation methodology. In Proc. 31st DAC, pp. 596–602. IEEE Computer Society, 1994.
I. Beer, S. Ben-David, C. Eisner, and Y. Rodeh. Efficient detection of vacuity in ACTL formulas. In Proc. 9th CAV, LNCS 1254, pp. 279–290, 1997.
M.C. Browne, E.M. Clarke, and O. Grumberg. Characterizing finite Kripke structures in propositional temporal logic. Theoretical Computer Science, 59:115–131, 1988.
J.P. Bergmann and M.A. Horowitz. Improving coverage analysis and test generation for large designs. In IEEE Int. Conf. for CAD, pp. 580–584, 1999.
E.M. Clarke and E.A. Emerson. Design and synthesis of synchronization skeletons using branching time temporal logic. In Proc. Workshop on Logic of Programs, LNCS 131, pp. 52–71. Springer-Verlag, 1981.
E.M. Clarke, O. Grumberg, K.L. McMillan, and X. Zhao. Efficient generation of counterexamples and witnesses in symbolic model checking. In Proc. 32nd DAC, pp. 427–432. IEEE Computer Society, 1995.
E.M. Clarke, O. Grumberg, and D. Peled. Model Checking. MIT Press, 1999.
K.-T. Cheng and A. Krishnakumar. Automatic functional test generation using the extended finite state machine model. In Proc. 30th DAC, pp. 86–91, 1993.
S. Devadas, A. Ghosh, and K. Keutzer. An observability-based code coverage metric for functional simulation. In Proc. CAD, pp. 418–425, 1996.
E.A. Emerson. Temporal andmodal logic. Handbook of TheoreticalComputer Science, pp. 997–1072, 1990.
F. Fallah, P. Ashar, and S. Devadas. Simulation vector generation from hdl descriptions for observability enhanced-statement coverage. In Proc. 36th DAC, pp. 666–671, 1999.
F. Fallah, S. Devadas, and K. Keutzer. OCCOM: Efficient Computation of Observability-Based Code Coverage Metrics for Functional Simulation. In Proc. 35th DAC, pp. 152–157, 1998.
R.C. Ho and M.A. Horowitz. Validation coverage analysis for complex digital designs. In Proc. CAD, pp. 146–151, 1996.
Y. Hoskote, T. Kam, P.-H Ho, and X. Zhao. Coverage estimation for symbolic model checking. In Proc. 36th DAC, pp. 300–305, 1999.
Y. Hoskote, D. Moundanos, and J. Abraham. Automatic extraction of the control flow machine and application to evaluating coverage of verification vectors. In Proc. ICDD, pp. 532–537, 1995.
C.A.R. Hoare. Communicating Sequential Processes. Prentice-Hall, 1985.
D. Harel and A. Pnueli. On the development of reactive systems. NATO Advanced Summer Institutes, volume F-13, pp. 477–498. Springer-Verlag, 1985.
G.D. Hachtel and F. Somenzi. Logic Synthesis and Verification Algorithms. Kluwer Academic Publishers, MA, 1996.
R. Ho, C. Yang, M. Horowitz, and D. Dill. Architecture validation for processors. In Proc. of the 22nd Annual Symp. on Comp. Arch., pp. 404–413, 1995.
S. Katz, D. Geist, and O. Grumberg. “Have I written enough properties?” a method of comparison between specification and implementation. In 10th CHARME, LNCS 1703, pp. 280–297. Springer-Verlag, 1999.
M. Kantrowitz and L. Noack. I’m done simulating: Now what? verification coverage analysis and correctness checking of the dec chip 21164 alpha microprocessor. In Proc. of DAC, pp. 325–330, 1996.
R.P. Kurshan. FormalCheck User’s Manual. Cadence Design, Inc., 1998.
O. Kupferman and M.Y. Vardi. Vacuity detection in temporal model checking. In 10th CHARME, LNCS 1703. pp. 82–96, Springer-Verlag, 1999.
O. Kupferman, M.Y. Vardi, and P. Wolper. An automata-theoretic approach to branching-time model checking. Journal of the ACM, 47(2):312–360, 2000.
O. Lichtenstein and A. Pnueli. Checking that finite state concurrent programs satisfy their linear specification. In Proc. 12th POPL, pp. 97–107, 1985.
D. Moumdanos, J.A. Abraham, and Y.V. Hoskote. Abstraction techniques for validation coverage analysis and test generation. IEEE Trans. on Computers, 1998.
Z. Manna and A. Pnueli. The Temporal Logic of Reactive and Concurrent Systems: Specification. Springer-Verlag, 1992.
R. G. Nigmatulin. The Complexity of Boolean Functions. Nauka, Main Editorial Board for Phys. and Math. Lit., Moscow, 1990.
J.P. Queille and J. Sifakis. Specification and verification of concurrent systems in Cesar. In Proc. 5th Int. Symp. on Progr., LNCS 137, pp. 337–351. Springer-Verlag, 1981.
I. Wegener. The Complexity of Boolean Functions. JohnWiley & Sons, 1987.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chockler, H., Kupferman, O., Vardi, M.Y. (2001). Coverage Metrics for Temporal Logic Model Checking. In: Margaria, T., Yi, W. (eds) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2001. Lecture Notes in Computer Science, vol 2031. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45319-9_36
Download citation
DOI: https://doi.org/10.1007/3-540-45319-9_36
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41865-8
Online ISBN: 978-3-540-45319-2
eBook Packages: Springer Book Archive