Abstract
When engineers design a system, there is always a question about how exhaustive the system has been examined to be correct. Coverage estimation provides an answer to this question in testing. A model checker verifies a design exhaustively, and proves the satisfaction of property specifications. However, people have noticed that design errors exist even after model checking is done, which goes to show that the question “How complete is the model checking once done?” is still left relatively unaddressed by model checkers, except for some state-based coverage metrics and the coverage estimator for symbolic simulation in RED . As a more complete solution, we propose several structural mutation models and coverage metrics to cover different design aspects in a state graph and to estimate the completeness of model checking, respectively. Once a system state graph satisfies a given set of property specifications, we estimate the coverage of completeness for the set of properties by applying some mutations to the state graph and checking if the given set of properties is sensitive to the mutation. Our experiences on five application examples demonstrate how the proposed coverage estimation methodology helps verification engineers to find the uncovered hole.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abramovici, M.: Dos and don’ts in computing fault coverage. In: Proceedings of the International Test Conference (ITC 1993), p. 594 (October 1993)
Alur, R., Dill, D.L.: A theory of timed automata. Theoretical Computer Science 126, 183–235 (1994)
Beer, I., Ben-David, S., Eisner, C., Rodeh, Y.: Efficient detection of vacuity in ACTL formulas. In: Grumberg, O. (ed.) CAV 1997. LNCS, vol. 1254, pp. 279–290. Springer, Heidelberg (1997)
Chockler, H., Kupferman, O.: Coverage of implementations by simulating specifications. In: Proceedings of the IFIP International Conference on Theoretical Computer Science (TCS 2002), August 2002, pp. 409–421. Kluwer, Dordrecht (2002)
Chockler, H., Kupferman, O., Kurshan, R.P., Vardi, M.Y.: A practical approach to coverage in model checking. In: Berry, G., Comon, H., Finkel, A. (eds.) CAV 2001. LNCS, vol. 2102, pp. 66–78. Springer, Heidelberg (2001)
Chockler, H., Kupferman, O., Vardi, M.Y.: Coverage metrics for temporal logic model checking. In: Margaria, T., Yi, W. (eds.) TACAS 2001. LNCS, vol. 2031, pp. 528–542. Springer, Heidelberg (2001)
Chockler, H., Kupferman, O., Vardi, M.Y.: Coverage metrics for formal verification. In: Geist, D., Tronci, E. (eds.) CHARME 2003. LNCS, vol. 2860, pp. 111–125. Springer, Heidelberg (2003)
Henzinger, T.A., Nicollin, X., Sifakis, J., Yovine, S.: Symbolic model checking for real-time systems. In: Proceedings of the IEEE International Conference on Logics in Computer Science (LICS 1992), June 1992, pp. 394–406 (1992)
Hoskote, Y., Kam, T., Ho, P.-H., Zhao, X.: Coverage estimation for symbolic model checking. In: Proceedings of the Design Automation Conference, June 1999, pp. 300–305 (1999)
Jayakumar, N., Purandare, M., Somenzi, F.: Dos and don’ts of CTL state coverage estimation. In: Proceedings of the Design Automation Conference (DAC 2003), June 2003, pp. 292–295. ACM Press, New York (2003)
Katz, S., Grumberg, O., Geist, D.: Have I written enough properties? – A method of comparison between specification and implementation. In: Pierre, L., Kropf, T. (eds.) CHARME 1999. LNCS, vol. 1703, pp. 280–297. Springer, Heidelberg (1999)
Kim, V.-K., Chen, T., Tegetho, M.: Fault coverage estimation for early stage of VLSI design. In: Proceedings of Ninth Great Lakes Symposium on VLSI (GLSVLSI 1999), March 1999, pp. 105–108 (1999)
Liu, C.-N., Jou, J.-Y.: Efficient coverage analysis metric for HDL design validation. In: Proceedings of IEEE International Conference on Computers and Digital Techniques, January 2001, pp. 1–6 (2001)
Mao, W.: Improving gate level fault coverage by RTL fault grading. In: Proceedings of IEEE International Test Conference 1996, Test and Design Validity, October 1996, pp. 150–159. IEEE Computer Society, Los Alamitos (1996)
Wang, F., Hsiung, P.-A.: Efficient and user-friendly verification. IEEE Transactions on Computers 51(1), 61–83 (2002)
Wang, F., Hwang, G.-D., Yu, F.: Numerical coverage estimation for the symbolic simulation of real-time systems. In: CPM 2003. LNCS, vol. 2676, pp. 160–176. Springer, Heidelberg (2003)
Woodward, M.R.: Mutation testing – an evolving technique. In: Proceedings of IEE Colloquium on Software Testing for Critical Systems, pp. 3/1–3/6 (June 1990)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, TC., Hsiung, PA. (2004). Mutation Coverage Estimation for Model Checking. In: Wang, F. (eds) Automated Technology for Verification and Analysis. ATVA 2004. Lecture Notes in Computer Science, vol 3299. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30476-0_29
Download citation
DOI: https://doi.org/10.1007/978-3-540-30476-0_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23610-8
Online ISBN: 978-3-540-30476-0
eBook Packages: Springer Book Archive