# Model Checking of Safety Properties

- 574 Downloads
- 149 Citations

## Abstract

Of special interest in formal verification are safety properties, which assert that the system always stays within some allowed region. Proof rules for the verification of safety properties have been developed in the proof-based approach to verification, making verification of safety properties simpler than verification of general properties. In this paper we consider model checking of safety properties. A computation that violates a general linear property reaches a bad cycle, which witnesses the violation of the property. Accordingly, current methods and tools for model checking of linear properties are based on a search for bad cycles. A symbolic implementation of such a search involves the calculation of a nested fixed-point expression over the system's state space, and is often infeasible. Every computation that violates a safety property has a finite prefix along which the property is violated. We use this fact in order to base model checking of safety properties on a search for finite bad prefixes. Such a search can be performed using a simple forward or backward symbolic reachability check. A naive methodology that is based on such a search involves a construction of an automaton (or a tableau) that is doubly exponential in the property. We present an analysis of safety properties that enables us to prevent the doubly-exponential blow up and to use the same automaton used for model checking of general properties, replacing the search for bad cycles by a search for bad prefixes.

## Preview

Unable to display preview. Download preview PDF.

## References

- 1.Y. Abarbanel, I. Beer, L. Gluhovsky, S. Keidar, and Y. Wolfstal, “FoCs—Automatic generation of simulation checkers from formal specifications,” in
*Computer Aided Verification, Proc. 12th Int. Conference*, Lecture Notes in Computer Science, Vol. 1855, Springer-Verlag, 2000, pp. 538–542.Google Scholar - 2.B. Alpern and F.B. Schneider, “Defining liveness,”
*Information Processing Letters*, Vol. 21, pp. 181–185, 1985.Google Scholar - 3.B. Alpern and F.B. Schneider, “Recognizing safety and liveness,”
*Distributed Computing*,Vol. 2, pp. 117–126, 1987.Google Scholar - 4.A. Biere, A. Cimatti, E.M. Clarke, M. Fujita, and Y. Zhu, “Symbolic model checking using SAT procedures instead of BDDs,” in
*Proc. 36th Design Automaion Conference*, IEEE Computer Society, 1999, pp. 317–320.Google Scholar - 5.J.R. Burch, E.M. Clarke, K.L. McMillan, D.L. Dill, and L.J. Hwang, “Symbolic model checking: 1020 states and beyond,”
*Information and Computation*, Vol. 98, No. 2, pp. 142–170, 1992.Google Scholar - 6.A. Boujjani, J.-C. Fernandez, S. Graf, C. Rodriguez, and J. Sifakis, “Safety for branching time,” in
*Proc. 18th International Colloquium on Automata Languages and Programming*, LNCS, Springer-Verlag, 1991, pp. 76–92.Google Scholar - 7.R.S. Boyer and J.S. Moore, “Proof-checking, theorem-proving and program verification,” Technical Report 35, Institute for Computing Science and Computer Applications, University of Texas at Austin, 1983.Google Scholar
- 8.E.M. Clarke and E.A. Emerson, “Design and synthesis of synchronization skeletons using branching time temporal logic,” in
*Proc. Workshop on Logic of Programs*, Lecture Notes in Computer Science, Vol. 131, Springer-Verlag, 1981, pp. 52–71.Google Scholar - 9.E.M. Clarke, E.A. Emerson, and A.P. Sistla, “Automatic verification of finite-state concurrent systems using temporal logic specifications,”
*ACM Transactions on Programming Languages and Systems*, Vol. 8, No. 2, pp. 244–263, 1986.Google Scholar - 10.W. Canfield, E.A. Emerson, and A. Saha, “Checking formal specifications under simulation,” in
*Proc. International Conference on Computer Design*, 1997, pp. 455–460.Google Scholar - 11.A.K. Chandra, D.C. Kozen, and L.J. Stockmeyer, “Alternation,”
*Journal of the Association for Computing Machinery*, Vol. 28, No. 1, pp. 114–133, 1981.Google Scholar - 12.C. Courcoubetis, M.Y. Vardi, P Wolper, and M. Yannakakis, “Memory efficient algorithms for the verification of temporal properties,”
*Formal Methods in System Design*, Vol. 1, pp. 275–288, 1992.Google Scholar - 13.E.A. Emerson, “Alternative semantics for temporal logics,”
*Theoretical Computer Science*, Vol. 26, pp. 121–130, 1983.Google Scholar - 14.E.A. Emerson, “Temporal and modal logic,” in
*Handbook of Theoretical Computer Science*, 1990, pp. 997–1072.Google Scholar - 15.N. Francez,
*Program Verification*. International Computer Science, Addison-Weflay, 1992.Google Scholar - 16.R. Gerth, D. Peled, M.Y. Vardi, and P. Wolper, “Simple on-the-fly automatic verification of linear temporal logic,” in P. Dembiski and M. Sredniawa (Eds),
*Protocol Specification, Testing, and Verification*, Chapman & Hall, 1995, pp. 3–18.Google Scholar - 17.P. Godefroid and P. Wolper, “Using partial orders for the efficient verification of deadlock freedom and safety properties,” in
*Proc. 3rd Conference on Computer Aided Verification*, Lecture Notes in Computer Science, Vol. 575, Springer-Verlag, 1991, pp. 332–342.Google Scholar - 18.R.H. Hardin, R.P. Kurshan, S.K. Shukla, and M.Y. Vardi, “A new heuristic for bad cycle detection using BDDs,” in
*Computer Aided Verification, Proc. 9th Int. Conference*, Lecture Notes in Computer Science, Vol. 1254, Springer-Verlag, 1997, pp. 268–278.Google Scholar - 19.H. Iwashita and T. Nakata, “Forward model checking techniques oriented to buggy designs,” in
*Proc. IEEE/ACM International Conference on Computer Aided Design*, 1997, pp. 400–404.Google Scholar - 20.N. Klarlund, “Mona & Fido: The logic-automaton connection in practice,” in
*Computer Science Logic, CSL '97*, Lecture Notes in Computer Science, 1998.Google Scholar - 21.O. Kupferman and M.Y. Vardi, “Weak alternating automata are not that weak,” in
*Proc. 5th Israeli Symposium on Theory of Computing and Systems*, IEEE Computer Society Press, 1997, pp. 147–158.Google Scholar - 22.O. Kupferman and M.Y. Vardi, “Freedom, weakness, and determinism: From linear-time to branching-time,” in
*Proc. 13th IEEE Symposium on Logic in Computer Science*, 1998, pp. 81–92.Google Scholar - 23.O. Kupferman, M.Y. Vardi, and P. Wolper, “An automata-theoretic approach to branching-time model checking,”
*Journal of the ACM*, Vol. 47, No. 2, 2000.Google Scholar - 24.L. Lamport, “Logical foundation,” in
*Distributed systems—Methods and Tools for Specification*, Lecture Notes in Computer Science, Vol. 190, Springer-Verlag, 1985.Google Scholar - 25.O. Lichtenstein and A. Pnueli, “Checking that finite state concurrent programs satisfy their linear specification,” in
*Proc. 12th ACM Symposium on Principles of Programming Languages*, New Orleans, 1985, pp. 97–107.Google Scholar - 26.Y. Luo, T. Wongsonegoro, and A. Aziz, “Hybrid techniques for fast functional simulation,” in
*Proc. 35th Design Automation Conference*, IEEE Computer Society, 1998.Google Scholar - 27.Z. Manna, A. Anuchitanukul, N. Bjorner, A. Browne, E. Chang, M. Colon, L. De Alfaro, H. Devarajan, H. Sipma, and T. Uribe, “STeP: The Stanford temporal prover,” Technical Report STAN-CS-TR–94–1518, Dept. of Computer Science, Stanford University, 1994.Google Scholar
- 28.K.L. McMillan, “Using unfolding to avoid the state explosion problem in the verification of asynchronous circuits,” in
*Proc. 4th Conference on Computer Aided Verification*, Lecture Notes in Computer Science, Vol. 663, Springer-Verlag, 1992, pp. 164–174.Google Scholar - 29.A.R. Meyer and M.J. Fischer, “Economy of description by automata, grammars, and formal systems,” in
*Proc. 12th IEEE Symp. on Switching and Automata Theory*, 1971, pp. 188–191.Google Scholar - 30.S. Miyano and T. Hayashi, “Alternating finite automata on ω-words,”
*Theoretical Computer Science*, Vol. 32, pp. 321–330, 1984.Google Scholar - 31.Z. Manna and A. Pnueli,
*The Temporal Logic of Reactive and Concurrent Systems: Specification*, Springer-Verlag, Berlin, 1992.Google Scholar - 32.Z. Manna and A. Pnueli,
*The Temporal Logic of Reactive and Concurrent Systems: Safety*, Springer-Verlag, New York, 1995.Google Scholar - 33.S. Melzer and S. Roemer, “Deadlock checking using net unfoldings,” in
*Computer Aided Verification, Proc. 9th Int. Conference*, Lecture Notes in Computer Science, Vol. 1254, Springer-Verlag, 1997, pp. 364–375.Google Scholar - 34.A.R. Meyer and L.J. Stockmeyer, “The equivalence problem for regular expressions with squaring requires exponential time,” in
*Proc. 13th IEEE Symp. on Switching and Automata Theory*, 1972, pp. 125–129.Google Scholar - 35.S. Owicki and L. Lamport, “Proving liveness properties of concurrent programs,”
*ACM Transactions on Programming Languages and Systems*, Vol. 4, No. 3, pp. 455–495, 1982.Google Scholar - 36.J.P. Queille and J. Sifakis, “Specification and verification of concurrent systems in Cesar,” in
*Proc. 5th International Symp. on Programming*, Lecture Notes in Computer Science, Vol. 137, Springer-Verlag, 1981, pp. 337–351.Google Scholar - 37.K. Ravi and F. Somenzi, “High-density reachability analysis,” in
*Proc. Int'l Conf. on Computer-Aided Design*, San Jose, 1995, pp. 154–158.Google Scholar - 38.S. Safra, “On the complexity of ω-automata,” in
*Proc. 29th IEEE Symposium on Foundations of Computer Science*, White Plains, October 1988, pp. 319–327.Google Scholar - 39.A.P. Sistla and E.M. Clarke, “The complexity of propositional linear temporal logic,”
*Journal ACM*, Vol. 32, pp. 733–749, 1985.Google Scholar - 40.A.P. Sistla, “Satefy, liveness and fairness in temporal logic,”
*Formal Aspects of Computing*,Vol. 6, pp. 495–511, 1994.Google Scholar - 41.R.E. Shankar, S. Owre, and J.M. Rushby, “The PVS proof checker: A reference manual (beta release),” Technical Report, Computer Science laboratory, SRI International, Menlo Park, California, 1993.Google Scholar
- 42.H.J. Touati, R.K. Brayton, and R. Kurshan, “Testing language containment for
*?*-automata using BDD's,”*Information and Computation*, Vol. 118, No. 1, pp. 101–109, 1995.Google Scholar - 43.A. Valmari, “On-the-fly verification with stubborn sets,” in
*Proc. 5nd Conference on Computer Aided Verifi-cation*, Lecture Notes in Computer Science, Vol. 697, Springer-Verlag, 1993.Google Scholar - 44.M.Y.6Vardi, “An automata-theoretic approach to linear temporal logic,” in F. Moller and G. Birtwistle (Eds.),
*Logics for Concurrency: Structure versus Automata*, Lecture Notes in Computer Science, Vol. 1043, Springer-Verlag, Berlin, 1996, pp. 238–266.Google Scholar - 45.M.Y. Vardi and P. Wolper, “An automata-theoretic approach to automatic program verification,” in
*Proc. First Symposium on Logic in Computer Science*, Cambridge, 1986, pp. 332–344.Google Scholar - 46.M.Y. Vardi and P. Wolper, “Automata-theoretic techniques for modal logics of programs,”
*Journal of Computer and System Science*, Vol. 32, No. 2, pp. 182–221, 1986.Google Scholar - 47.M.Y. Vardi and P. Wolper, “Reasoning about infinite computations,”
*Information and Computation*, Vol. 115, No. 1, pp. 1–37, 1994.Google Scholar - 48.P. Wolper, “Synthesis of Communicating Processes from Temporal Logic Specifications,” Ph.D. Thesis, Stanford University, 1982.Google Scholar
- 49.J. Yuan, J. Shen, J. Abraham, and A. Aziz, “On combining formal and informal verification,” in
*Computer Aided Verification, Proc. 9th Int. Conference*, Lecture Notes in Computer Science, Vol. 1254, Springer-Verlag, 1997, pp. 376–387.Google Scholar