Skip to main content
SpringerLink
Log in

Property-Driven Benchmark Generation

  • Conference paper
Model Checking Software (SPIN 2013)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7976))

Included in the following conference series:

Abstract

We present a systematic approach to the automatic generation of platform-independent benchmarks of tailored complexity for evaluating verification tools for reactive systems. Key to this approach is a tool chain that essentially transforms a set of automatically generated LTL properties into source code for various formats, platforms, and competition scenarios via a sequence of property-preserving steps. These steps go through dedicated representations in terms of Büchi Automata, Mealy machines, Decision Diagram Models, Code Models, and finally the source code of the chosen scenario. The required transformations comprise LTL synthesis, model checking, property-oriented expansion, path condition extraction, theorem proving, SAT solving, and code motion. This combination allows us to address different communities via a growing set of programming languages, tailored sets of programming constructs, different notions of observation, and the full variety of LTL properties – ranging from mere reachability over general safety properties to arbitrary liveness properties. The paper illustrates the whole tool chain along accompanying examples, emphasizes the current state of development, and sketches the envisioned potential and impact of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Gastin, P., Oddoux, D.: Fast LTL to Büchi Automata Translation. In: Berry, G., Comon, H., Finkel, A. (eds.) CAV 2001. LNCS, vol. 2102, pp. 53–65. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  2. Giannakopoulou, D., Lerda, F.: From States to Transitions: Improving Translation of LTL Formulae to Büchi Automata. In: Peled, D.A., Vardi, M.Y. (eds.) FORTE 2002. LNCS, vol. 2529, pp. 308–326. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  3. Clarke, E., Grumberg, O., Peled, D.: Model Checking. MIT Press (2001)

    Google Scholar 

  4. Steffen, B.: Unifying models. In: Reischuk, R., Morvan, M. (eds.) STACS 1997. LNCS, vol. 1200, pp. 1–20. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  5. Snelting, G., Robschnik, T., Krinke, J.: Efficient path conditions in dependence graphs for software safety analysis. ACM Transactions on Software Engineering and Methodology (TOSEM) 15(4), 410–457 (2006)

    Article  Google Scholar 

  6. Robinson, A., Voronkov, A. (eds.): Handbook of Automated Reasoning, vol. I & II. Elsevier (2001)

    Google Scholar 

  7. Biere, A., Heule, M.J.H., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press (February 2009)

    Google Scholar 

  8. Morel, E., Renvoise, C.: Global optimization by suppression of partial redundancies. Comm. ACM 22(2), 96–103 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  9. Steffen, B., Knoop, J.: Finite Constants: Characterizations of a New Decidable Set of Constants. In: Kreczmar, A., Mirkowska, G. (eds.) MFCS 1989. LNCS, vol. 379, pp. 481–491. Springer, Heidelberg (1989)

    Chapter  Google Scholar 

  10. Rosen, B.K., Wegman, M.N., Zadeck, F.K.: Global Value Numbers and Redundant Computations. In: Conference Record of the Fifteenth Annual ACM Symposium on Principles of Programming Languages. ACM Press (1988)

    Google Scholar 

  11. Knoop, J., Rüthing, O., Steffen, B.: Lazy code motion. In: Proc. of the ACM SIGPLAN 1992 Conference on Programming Language Design and Implementation (PLDI), pp. 224–234. ACM (1992)

    Google Scholar 

  12. Knoop, J., Rüthing, O., Steffen, B.: Lazy Strength Reduction. Journal of Programming Languages 1, 71–91 (1993)

    Google Scholar 

  13. Briggs, P., Cooper, K.D.: Effective partial redundancy elimination. In: Proc. ACM SIGPLAN Conf. Prog. Lang. Design and Impl. (PLDI 1994), pp. 159–170 (1994)

    Google Scholar 

  14. Howar, F., Isberner, M., Merten, M., Steffen, B., Beyer, D.: The RERS Grey-Box Challenge 2012: Analysis of Event-Condition-Action Systems. In: Margaria, T., Steffen, B. (eds.) ISoLA 2012, Part I. LNCS, vol. 7609, pp. 608–614. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  15. Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: Proc. of the 1999 Int. Conf. on Software Engineering, pp. 411–420. IEEE (1999)

    Google Scholar 

  16. Steffen, B.: Property-oriented expansion. In: Cousot, R., Schmidt, D.A. (eds.) SAS 1996. LNCS, vol. 1145, pp. 22–41. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  17. Hayes-Roth, F.: Rule-Based Systems. Commun. ACM 28(9), 921–932 (1985)

    Article  Google Scholar 

  18. McCarthy, D.R., Dayal, U.: The Architecture of An Active Data Base Management System. In: Proceedings of the 1989 ACM SIGMOD International Conference on Management of Data, pp. 215–224. ACM Press (1989)

    Google Scholar 

  19. Almeida, E.E., Luntz, J.E., Tilbury, D.M.: Event-Condition-Action Systems for Reconfigurable Logic Control. IEEE T. Automation Science and Engineering 4(2), 167–181 (2007)

    Article  Google Scholar 

  20. Apt, K.R., Olderog, E.R.: Verification of Sequential and Concurrent Programs. Texts and Monographs in Computer Science. Springer (1991)

    Google Scholar 

  21. Knoop, J., Rüthing, O., Steffen, B.: Partial Dead Code Elimination. In: Proceedings of the ACM SIGPLAN 1994 Conference on Programming Language Design and Implementation (PLDI), pp. 147–158. ACM (1994)

    Google Scholar 

  22. Knoop, J., Rüthing, O., Steffen, B.: Expansion-Based Removal of Semantic Partial Redundancies. In: Jähnichen, S. (ed.) CC 1999. LNCS, vol. 1575, pp. 91–107. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  23. Blom, S., van de Pol, J., Weber, M.: LTSmin: Distributed and symbolic reachability. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 354–359. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  24. Mealy, G.H.: A Method for Synthesizing Sequential Circuits. Bell System Technical Journal 34(5), 1045–1079 (1955)

    MathSciNet  Google Scholar 

  25. Milner, R.: Communication and concurrency. PHI Series in computer science. Prentice-Hall, Inc., Upper Saddle River (1989)

    MATH  Google Scholar 

  26. Kaynar, D.K., Lynch, N.A., Segala, R., Vaandrager, F.W.: Timed I/O Automata: A Mathematical Framework for Modeling and Analyzing Real-Time Systems. In: Proceedings of the 24th IEEE Real-Time Systems Symposium (RTSS 2003), pp. 166–177. IEEE Computer Society (2003)

    Google Scholar 

  27. Floyd, R.W.: Assigning meaning to programs. In: Proc. of Symposium on Applied Mathematics. Mathematical aspects of computer science, vol. 19, pp. 19–32. American Mathematical Society (1967)

    Google Scholar 

  28. Hoare, C.A.R.: An axiomatic basis for computer programming. Communications of the ACM 12(10), 576–580 (1969)

    Article  MATH  Google Scholar 

  29. Steffen, B., Rüthing, O.: Quality Engineering: Leveraging Heterogeneous Information - (Invited Talk). In: Jhala, R., Schmidt, D. (eds.) VMCAI 2011. LNCS, vol. 6538, pp. 23–37. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  30. Dhamdhere, D.M.: A new algorithm for composite hoisting and strength reduction optimisation (+ Corrigendum). Int. J. Comp. Math. 27, 1–14 (1989)

    Article  Google Scholar 

  31. Knoop, J., Rüthing, O., Steffen, B.: Optimal Code Motion: Theory and Practice. ACM Trans. Program. Lang. Syst. 16(4), 1117–1155 (1994)

    Article  Google Scholar 

  32. Knoop, J., Rüthing, O., Steffen, B.: The Power of Assignment Motion. In: Proceedings of the ACM SIGPLAN 1995 Conference on Programming Language Design and Implementation (PLDI). ACM (1995)

    Google Scholar 

  33. Rüthing, O., Knoop, J., Steffen, B.: Sparse Code Motion. In: Proceedings of the 27th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 2000), pp. 170–183. ACM (2000)

    Google Scholar 

  34. Steffen, B., Knoop, J., Rüthing, O.: The Value Flow Graph: A Program Representation for Optimal Program Transformations. In: Jones, N.D. (ed.) ESOP 1990. LNCS, vol. 432, pp. 389–405. Springer, Heidelberg (1990)

    Chapter  Google Scholar 

  35. Steffen, B., Knoop, J., Rüthing, O.: Efficient Code Motion and an Adaption to Strength Reduction. In: Abramsky, S. (ed.) TAPSOFT 1991. LNCS, vol. 494, pp. 394–415. Springer, Heidelberg (1991)

    Google Scholar 

  36. Holzmann, G.J.: The SPIN Model Checker - Primer and Reference Manual. Addison-Wesley (2004)

    Google Scholar 

  37. Steffen, B.: Characteristic Formulae. In: Ronchi Della Rocca, S., Ausiello, G., Dezani-Ciancaglini, M. (eds.) ICALP 1989. LNCS, vol. 372, pp. 723–732. Springer, Heidelberg (1989)

    Chapter  Google Scholar 

  38. Steffen, B., Claßen, A., Klein, M., Knoop, J., Margaria, T.: The Fixpoint-Analysis Machine. In: Lee, I., Smolka, S.A. (eds.) CONCUR 1995. LNCS, vol. 962, pp. 72–87. Springer, Heidelberg (1995)

    Chapter  Google Scholar 

  39. Burkart, O., Steffen, B.: Model Checking the Full Modal Mu-Calculus for Infinite Sequential Processes. In: Degano, P., Gorrieri, R., Marchetti-Spaccamela, A. (eds.) ICALP 1997. LNCS, vol. 1256, pp. 419–429. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  40. Steffen, B., Margaria, T., Nagel, R., Jörges, S., Kubczak, C.: Model-Driven Development with the jABC. In: Bin, E., Ziv, A., Ur, S. (eds.) HVC 2006. LNCS, vol. 4383, pp. 92–108. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chair for Programming Systems, Dortmund University of Technology, Dortmund, D-44227, Germany

    Bernhard Steffen, Malte Isberner, Stefan Naujokat & Maren Geske

  2. Carnegie Mellon University, Moffett Field, CA, USA

    Malte Isberner

  3. Chair Service and Software Engineering, Universität Potsdam, Potsdam, D-14482, Germany

    Tiziana Margaria

Authors
  1. Bernhard Steffen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Malte Isberner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Naujokat
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tiziana Margaria
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maren Geske
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Engineering, Vienna, University of Technology, Austria

    Ezio Bartocci

  2. Department of Computer Science, Stony Brook University, 11794-4400, Stony Brook, NY, USA

    C. R. Ramakrishnan

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Steffen, B., Isberner, M., Naujokat, S., Margaria, T., Geske, M. (2013). Property-Driven Benchmark Generation. In: Bartocci, E., Ramakrishnan, C.R. (eds) Model Checking Software. SPIN 2013. Lecture Notes in Computer Science, vol 7976. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39176-7_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39176-7_21

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39175-0

  • Online ISBN: 978-3-642-39176-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation