Tailored generation of concurrent benchmarks

  • Bernhard Steffen
  • Falk Howar
  • Malte Isberner
  • Stefan NaujokatEmail author
  • Tiziana Margaria


In this paper, we show how to extend our approach to property-driven benchmark generation (PDBG) to support concurrency in the benchmarks: we systematically produce multi-process PROMELA code of known and tailored complexity that can then serve as benchmark for the evaluation of analysis and verification tools for concurrent software systems. Key to this extension is the way in which we control the corresponding property profiles, i.e., the statements about the validity of a predefined set of LTL properties, along the PDBG tool chain. It is based on the idea to introduce intricate structural elements like synchronization and interference at a local and therefore controllable level, only concerning a few parallel components, and subsequently obfuscating the locality of these structures at the global level via property-preserving transformations of both the system components and the properties. This leads to a scalable approach which we illustrate along some concrete examples.


Benchmark generation Concurrency Interference Synchronization LTL synthesis Model checking Property-oriented expansion Path condition extraction Theorem proving SAT solving Code motion 


  1. 1.
    Abadi, M., Lamport, L.: Conjoining specifications. ACM Trans. Program. Lang. Syst. 17(3), 507–535 (1995)CrossRefGoogle Scholar
  2. 2.
    Biere, A., Heule, M.J.H., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)Google Scholar
  3. 3.
    Briggs, P., Cooper, K.D.: Effective partial redundancy elimination. In: Proceedings of the ACM SIGPLAN’93 Conference on Programming Language Design and Implementation (PLDI’94), pp. 159–170 (1994)Google Scholar
  4. 4.
    Burkart, O., Steffen, B.: Model checking the full modal mu-calculus for infinite sequential processes. In: Degano, P., Gorrieri, R., Marchetti-Spaccamela, A. (eds.) Automata. Languages and Programming. Lecture Notes in Computer Science, vol. 1256, pp. 419–429. Springer, Berlin (1997)Google Scholar
  5. 5.
    Clarke, E.M., Grumberg, O., Peled, D.: Model Checking. MIT Press, Cambridge (2001)Google Scholar
  6. 6.
    Elrad, T., Francez, N.: Decomposition of distributed programs into communication-closed layers. Sci. Comput. Program. 2, 155–173 (1982)CrossRefzbMATHGoogle Scholar
  7. 7.
    Gastin, P., Oddoux, D.: Fast LTL to Büchi automata translation. In: Gérard, B., Hubert, C., Alain, F. (eds.) Proceedings of CAV’01. LNCS, vol 2102, pp. 53–65, Paris, France. Springer, Berlin (2001)Google Scholar
  8. 8.
    Giannakopoulou, D., Lerda, F.: From states to transitions: improving translation of LTL formulae to Büchi automata. In: Proceedings of FORTE’02, pp. 308–326, London, UK. Springer, Berlin (2002)Google Scholar
  9. 9.
    Hoare, C.A.R.: Communicating Sequential Processes. Prentice-Hall Inc., Upper Saddle River (1985)zbMATHGoogle Scholar
  10. 10.
    Holzmann, G.J.: Holzmann. The SPIN Model Checker—Primer and Reference Manual. Addison-Wesley, Reading (2004)Google Scholar
  11. 11.
    Howar, F., Isberner, M., Merten, M., Steffen, B., Beyer, D.: The RERS grey-box challenge 2012: analysis of event-condition-action systems. In: Margaria, T., Steffen, B. (eds.) Proceedings of ISOLA’12. LNCS, vol. 7609, pp. 608–614. Springer, Berlin (2012)Google Scholar
  12. 12.
    Knoop, J., Rüthing, O., Steffen, B.: Lazy code motion. In Proceedings of the ACM SIGPLAN’92 Conference on Programming Language Design and Implementation (PLDI), pp. 224–234. ACM (1992)Google Scholar
  13. 13.
    Knoop, J., Rüthing, O., Steffen, B.: Lazy strength reduction. J. Program. Lang. 1, 71–91 (1993)Google Scholar
  14. 14.
    Kupferman, O., Vardi, M.Y.: Vacuity detection in temporal model checking. In: Pierre, L., Kropf, T. (eds.) Correct Hardware Design and Verification Methods. LNCS, vol. 1703, pp. 82–98. Springer, Berlin (1999)Google Scholar
  15. 15.
    Larsen, K.G.: Modal Specifications. In: Automatic Verification Methods for Finite State Systems, pp. 232–246 (1989)Google Scholar
  16. 16.
    Milner, R.: Communication and Concurrency. PHI Series in Computer Science. Prentice-Hall Inc, Upper Saddle River (1989)Google Scholar
  17. 17.
    Morel, E., Renvoise, C.: Global optimization by suppression of partial redundancies. Commun. ACM 22(2), 96–103 (1979)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Naujokat, S., Lybecait, M., Steffen, B., Kopetzki, D., Margaria, T.: Full generation of domain-specific graphical modeling tools: a meta\(^2\)modeling approach (2014 under submission) Google Scholar
  19. 19.
    Naujokat, S., Traonouez, L-M., Isberner, M., Steffen, B., Axel, L.: Domain-specific code generator modeling: a case study for multi-faceted concurrent systems. In: Proceedings of the 6th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation (ISoLA 2014) (2014 to appear)Google Scholar
  20. 20.
    Robinson, A., Voronkov, A. (eds.): Handbook of Automated Reasoning, vol. I & II. Elsevier, Amsterdam (2001)Google Scholar
  21. 21.
    Rosen, B.K., Wegman, M.N., Zadeck, F.K.: Global value numbers and redundant computations. In: Conference Record of the Fifteenth Annual ACM Symposium on Principles of Programming Languages. ACM Press, New York (1988)Google Scholar
  22. 22.
    Snelting, G., Robschnik, T., Krinke, J.: Efficient path conditions in dependence graphs for software safety analysis. ACM Trans. Softw. Eng. Methodol (TOSEM) 15(4), 410–457 (2006)CrossRefGoogle Scholar
  23. 23.
    Steffen, B.: Characteristic formulae. In: Ausiello, G., Dezani-Ciancaglini, M., Rocca, S.R. (eds.) Automata, Languages and Programming. Lecture Notes in Computer Science, vol. 372, pp. 723–732. Springer, Berlin (1989)CrossRefGoogle Scholar
  24. 24.
    Steffen, B.: Unifying models. In: Reischuk, R., Morvan, M. (eds.) STACS 97. Lecture Notes in Computer Science, vol. 1200, pp. 1–20. Springer, Berlin (1997)Google Scholar
  25. 25.
    Steffen, B., Claßen, A., Klein, M., Knoop, J., Margaria, T.: The fixpoint-analysis machine. In: Lee, I., Smolka, S.A. (eds.) CONCUR ’95: Concurrency Theory. Lecture Notes in Computer Science, vol. 962, pp. 72–87. Springer, Berlin (1995)Google Scholar
  26. 26.
    Steffen, B., Isberner, M., Naujokat, S., Margaria, T., Geske, M.: Property-driven benchmark generation. In: International SPIN Symposium on Model Checking of Software (SPIN2013). LNCS, vol. 7976, pp. 341–357. Springer, Berlin (2013)Google Scholar
  27. 27.
    Steffen, B., Isberner, M., Naujokat, S., Margaria, T., Geske, M.: Property-driven benchmark generation: synthesizing programs of realistic structure. Softw. Tools Technol. Transf. doi: 10.1007/s10009-014-0336-z (2014)
  28. 28.
    Steffen, B., Knoop, J.: Finite constants: characterizations of a new decidable set of constants. In: Kreczmar, A., Mirkowska, G. (eds.) Mathematical Foundations of Computer Science (MFCS’89). LNCS, vol. 379, pages 481–491. Springer, Berlin (1989)Google Scholar
  29. 29.
    Steffen, B., Margaria, T., Nagel, R., Jörges, S., Kubczak, C.: Model-driven development with the jABC. In: Bin, E., Ziv, A., Ur, S. (eds.) Haifa Verification Conference. LNCS, vol. 4383, pp. 92–108. Springer, Berlin (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Bernhard Steffen
    • 1
  • Falk Howar
    • 2
  • Malte Isberner
    • 1
  • Stefan Naujokat
    • 1
    Email author
  • Tiziana Margaria
    • 3
  1. 1.Chair for Programming SystemsTechnische Universität DortmundDortmundGermany
  2. 2.Carnegie Mellon UniversityMoffett FieldUSA
  3. 3.Chair Service and Software EngineeringUniversität PotsdamPotsdamGermany

Personalised recommendations