The first reactive synthesis competition (SYNTCOMP 2014)

Swen Jacobs; Roderick Bloem; Romain Brenguier; Rüdiger Ehlers; Timotheus Hell; Robert Könighofer; Guillermo A. Pérez; Jean-François Raskin; Leonid Ryzhyk; Ocan Sankur; Martina Seidl; Leander Tentrup; Adam Walker

doi:10.1007/s10009-016-0416-3

International Journal on Software Tools for Technology Transfer

June 2017, Volume 19, Issue 3, pp 367–390

The first reactive synthesis competition (SYNTCOMP 2014)

Authors
Authors and affiliations

Swen JacobsEmail author
Roderick Bloem
Romain Brenguier
Rüdiger Ehlers
Timotheus Hell
Robert Könighofer
Guillermo A. Pérez
Jean-François Raskin
Leonid Ryzhyk
Ocan Sankur
Martina Seidl
Leander Tentrup
Adam Walker

Regular Paper

First Online:: 09 April 2016

5 Citations
2 Shares
170 Downloads

Abstract

We introduce the reactive synthesis competition (SYNTCOMP), a long-term effort intended to stimulate and guide advances in the design and application of synthesis procedures for reactive systems. The first iteration of SYNTCOMP is based on the controller synthesis problem for finite-state systems and safety specifications. We provide an overview of this problem and existing approaches to solve it, and report on the design and results of the first SYNTCOMP. This includes the definition of the benchmark format, the collection of benchmarks, the rules of the competition, and the five synthesis tools that participated. We present and analyze the results of the competition and draw conclusions on the state of the art. Finally, we give an outlook on future directions of SYNTCOMP.

Keywords

Synthesis Reactive systems Competition Experimental evaluation Benchmarks Safety games

References

1.
Alur, R., Bodík, R., Dallal, E., Fisman, D., Garg, P., Juniwal, G., Kress-Gazit, H., Madhusudan, P., Martin, M.M.K., Raghothaman, M., Saha, S., Seshia, S.A., Singh, R., Solar-Lezama, A., Torlak, E., Udupa, A.: Syntax-guided synthesis. In: Dependable Software Systems Engineering, NATO Science for Peace and Security Series, D: Information and Communication Security, vol. 40, pp. 1–25. IOS Press (2015)Google Scholar
2.
Alur, R., Madhusudan, P., Nam, W.: Symbolic computational techniques for solving games. STTT 7(2), 118–128 (2005)CrossRefMATHGoogle Scholar
3.
Aziz, A., Tasiran, S., Brayton, R.K.: BDD variable ordering for interacting finite state machines. In: DAC, pp. 283–288 (1994)Google Scholar
4.
Balint, A., Diepold, D., Gall, D., Gerber, S., Kapler, G., Retz, R.: EDACC - an advanced platform for the experiment design, administration and analysis of empirical algorithms. In: LION 5. Selected Papers, LNCS, vol. 6683, pp. 586–599. Springer, (2011)Google Scholar
5.
Barrett, C.W., de Moura, L.M., Stump, A.: Design and results of the first satisfiability modulo theories competition (SMT-COMP 2005). J. Autom. Reason. 35(4), 373–390 (2005)CrossRefMATHGoogle Scholar
6.
Beyer, D.: Competition on software verification - (SV-COMP). In: TACAS, LNCS, vol. 7214, pp. 504–524. Springer (2012)Google Scholar
7.
Beyer, D., Löwe, S., Wendler, P.: Benchmarking and resource measurement. In: SPIN 2015, LNCS, vol. 9232, pp. 160–178. Springer (2015)Google Scholar
8.
Bloem, R., Cimatti, A., Greimel, K., Hofferek, G., Könighofer, R., Roveri, M., Schuppan, V., Seeber, R.: RATSY - A new requirements analysis tool with synthesis. In: CAV, LNCS, vol. 6174, pp. 425–429. Springer (2010)Google Scholar
9.
Bloem, R., Egly, U., Klampfl, P., Könighofer, R., Lonsing, F.: SAT-based methods for circuit synthesis. In: FMCAD’14, pp. 31–34. IEEE (2014)Google Scholar
10.
Bloem, R., Galler, S.J., Jobstmann, B., Piterman, N., Pnueli, A., Weiglhofer, M.: Automatic hardware synthesis from specifications: a case study. In: DATE, pp. 1188–1193. ACM (2007)Google Scholar
11.
Bloem, R., Galler, S.J., Jobstmann, B., Piterman, N., Pnueli, A., Weiglhofer, M.: Specify, compile, run: hardware from PSL. Electr. Notes Theor. Comput. Sci. 190(4), 3–16 (2007)CrossRefGoogle Scholar
12.
Bloem, R., Jobstmann, B., Piterman, N., Pnueli, A., Sa’ar, Y.: Synthesis of reactive(1) designs. J. Comput. Syst. Sci. 78(3), 911–938 (2012)MathSciNetCrossRefMATHGoogle Scholar
13.
Bloem, R., Könighofer, R., Seidl, M.: SAT-based synthesis methods for safety specs. In: VMCAI, LNCS, vol. 8318, pp. 1–20. Springer (2014)Google Scholar
14.
Bohy, A., Bruyère, V., Filiot, E., Jin, N., Raskin, J.-F.: Acacia+, a tool for LTL synthesis. In: CAV, LNCS, vol. 7358, pp. 652–657. Springer (2012)Google Scholar
15.
Bradley, A.R.: SAT-based model checking without unrolling. In: VMCAI, LNCS, vol. 6538, pp. 70–87. Springer (2011)Google Scholar
16.
Brayton, R.K., Hachtel, G.D., Sangiovanni-Vincentelli, A.L., Somenzi, F., Aziz, A., Cheng, S., Edwards, S.A., Khatri, S.P., Kukimoto, Y., Pardo, A., Qadeer, S., Ranjan, R.K., Sarwary, S., Shiple, T.R., Swamy, G., Villa, T.: VIS: a system for verification and synthesis. In: CAV, LNCS, vol. 1102, pp. 428–432. Springer (1996)Google Scholar
17.
Brayton, R.K., Mishchenko, A.: ABC: an academic industrial-strength verification tool. In: CAV, LNCS, vol. 6174, pp. 24–40. Springer (2010)Google Scholar
18.
Brenguier, R., Pérez, G.A., Raskin, J.-F., Sankur, O.: AbsSynthe: abstract synthesis from succinct safety specifications. In: SYNT, EPTCS, vol. 157, pp. 100–116. Open Publishing Association (2014)Google Scholar
19.
Bryant, R.E.: Graph-based algorithms for boolean function manipulation. IEEE Trans. Comput. 35(8), 677–691 (1986)CrossRefMATHGoogle Scholar
20.
Büchi, J., Landweber, L.: Solving sequential conditions by finite-state strategies. Trans. Am. Math. Soc. 138, 295–311 (1969)CrossRefMATHGoogle Scholar
21.
Burch, J.R., Clarke, E.M., Long, D.E.: Symbolic model checking with partitioned transistion relations. In: VLSI, pp. 49–58 (1991)Google Scholar
22.
Chiang, T., Jiang. J.R.: Property-directed synthesis of reactive systems from safety specifications. In: ICCAD, pp. 794–801. ACM (2015)Google Scholar
23.
Church, A.: Logic, arithmetic and automata. In: Proceedings of the International Congress of Mathematicians, pp. 23–35 (1962)Google Scholar
24.
Coudert, O., Madre, J.C.: A unified framework for the formal verification of sequential circuits. In: ICCAD, pp. 126–129 (1990)Google Scholar
25.
Cousot, P., Cousot, R.: Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: POPL, pp. 238–252. ACM (1977)Google Scholar
26.
de Alfaro, L., Roy, P.: Solving games via three-valued abstraction refinement. In: CONCUR, LNCS, vol. 4703, pp. 74–89. Springer (2007)Google Scholar
27.
Ehlers, R.: Experimental aspects of synthesis. In: iWIGP, EPTCS, vol. 50, pp. 1–16 (2011)Google Scholar
28.
Ehlers, R.: Unbeast: symbolic bounded synthesis. In: TACAS, LNCS, vol. 6605, pp. 272–275. Springer (2011)Google Scholar
29.
Ehlers, R.: Symbolic bounded synthesis. Formal Methods Syst. Des. 40(2), 232–262 (2012)CrossRefMATHGoogle Scholar
30.
Ehlers, R., Könighofer, R., Hofferek, G.: Symbolically synthesizing small circuits. In: FMCAD’12, pp. 91–100. IEEE (2012)Google Scholar
31.
Emerson, E.A., Clarke, E.M.: Using branching time temporal logic to synthesize synchronization skeletons. Sci. Comput. Program. 2(3), 241–266 (1982)CrossRefMATHGoogle Scholar
32.
Filiot, E., Jin, N., Raskin, J.: Exploiting structure in LTL synthesis. STTT 15(5–6), 541–561 (2013)CrossRefGoogle Scholar
33.
Filiot, E., Jin, N., Raskin, J.-F.: Antichains and compositional algorithms for LTL synthesis. Formal Methods Syst. Des. 39(3), 261–296 (2011)CrossRefMATHGoogle Scholar
34.
Finkbeiner, B., Jacobs, S.: Lazy synthesis. In: VMCAI, LNCS, vol. 7148, pp. 219–234. Springer (2012)Google Scholar
35.
Finkbeiner, B., Schewe, S.: Bounded synthesis. STTT 15(5–6), 519–539 (2013)CrossRefMATHGoogle Scholar
36.
Graf, S., Saïdi, H.: Construction of abstract state graphs with PVS. In: CAV, LNCS, vol. 1254, pp. 72–83. Springer (1997)Google Scholar
37.
Henzinger, T.A., Jhala, R., Majumdar, R.: Counterexample-guided control. In: ICALP, LNCS, vol. 2719, pp. 886–902, Springer (2003)Google Scholar
38.
Hong, Y., Beerel, P.A., Burch, J.R., McMillan, K.L.: Sibling-substitution-based BDD minimization using don’t cares. IEEE Trans. CAD of Integr. Circuits Syst. 19(1), 44–55 (2000)CrossRefGoogle Scholar
39.
Jacobs, S.: Extended AIGER format for synthesis. CoRR (2014). arXiv:1405.5793. Accessed Feb 2016
40.
Jacobs, S., Bloem, R., Brenguier, R., Ehlers, R., Hell, T., Könighofer, R., Pérez, G.A., Raskin, J., Ryzhyk, L., Sankur, O., Seidl, M., Tentrup, L., Walker, A.: The first reactive synthesis competition (SYNTCOMP 2014). CoRR (2015). arXiv:1506.08726. Accessed Feb 2016
41.
Jacobs, S., Bloem, R., Brenguier, R., Könighofer, R., Pérez, G.A., Raskin, J.-F., Ryzhyk, L., Sankur, O., Seidl, M., Tentrup, L., Walker, A.: The second reactive synthesis competition (SYNTCOMP 2015). In: SYNT, EPTCS, vol. 202, pp. 27–57. Open Publishing Association (2016)Google Scholar
42.
Jacobs, S., Klein, F.: A high-level LTL synthesis format: TLSF v1.0. CoRR (2016). arXiv:1601.05228. Accessed Feb 2016
43.
Järvisalo, M., Berre, D.L., Roussel, O., Simon, L.: The international SAT solver competitions. AI Mag 33(1), 89–94 (2012)Google Scholar
44.
Jobstmann, B., Bloem, R.: Optimizations for LTL synthesis. In: FMCAD, pp. 117–124. IEEE Computer Society (2006)Google Scholar
45.
Jobstmann, B., Galler, S.J., Weiglhofer, M., Bloem, R.: Anzu: A tool for property synthesis. In: CAV, LNCS, vol. 4590, pp. 258–262. Springer (2006)Google Scholar
46.
Kupferman, O., Vardi, M.Y.: Safraless decision procedures. In: FOCS, pp. 531–542. IEEE Computer Society (2005)Google Scholar
47.
Kurshan, R.P.: Automata-theoretic verification of coordinating processes. In: Analysis and Optimization of Systems: Discrete Event Systems, pp. 16–28. Springer (1994)Google Scholar
48.
Lecoutre, C., Roussel, O., van Dongen, M.R.C.: Promoting robust black-box solvers through competitions. Constraints 15(3), 317–326 (2010)CrossRefMATHGoogle Scholar
49.
Mishchenko, A., Chatterjee, S., Brayton, R.K.: Dag-aware AIG rewriting a fresh look at combinational logic synthesis. In: DAC, pp. 532–535. ACM (2006)Google Scholar
50.
Mishchenko, A., Chatterjee, S., Jiang, R., Brayton, R.: FRAIGs: A unifying representation for logic synthesis and verification. Technical report, EECS Department, U. C. Berkeley (2005)Google Scholar
51.
Morgenstern, A., Gesell, M., Schneider, K.: Solving games using incremental induction. In: IFM’13, LNCS 7940, pp. 177–191. Springer (2013)Google Scholar
52.
Niemetz, A., Preiner, M., Lonsing, F., Seidl, M., Biere, A.: Resolution-based certificate extraction for QBF. In: SAT’12, LNCS 7317, pp. 430–435. Springer (2012)Google Scholar
53.
Pnueli, A., Rosner, R.: On the synthesis of a reactive module. In: POPL, pp. 179–190. ACM Press (1989)Google Scholar
54.
Rabin, M.O.: Decidability of second-order theories and automata on infinite trees. Trans. Am. Math. Soc. 141, 1–35 (1969)MATHGoogle Scholar
55.
Ranjan, R.K., Aziz, A., Brayton, R.K., Plessier, B., Pixley, C.: Efficient bdd algorithms for fsm synthesis and verification. In: International Workshop on Logic Synthesis (1995)Google Scholar
56.
Roussel, O.: Controlling a solver execution with the runsolver tool. JSAT 7(4), 139–144 (2011)MathSciNetMATHGoogle Scholar
57.
Rudell, R.: Dynamic variable ordering for ordered binary decision diagrams. In: ICCAD, pp. 42–47. IEEE Computer Society (1993)Google Scholar
58.
Seidl, M., Könighofer, R.: Partial witnesses from preprocessed quantified boolean formulas. In: DATE’14, pp. 1–6. IEEE (2014)Google Scholar
59.
Sohail, S., Somenzi, F.: Safety first: a two-stage algorithm for the synthesis of reactive systems. STTT 15(5–6), 433–454 (2013)CrossRefGoogle Scholar
60.
Somenzi, F.: Binary decision diagrams. In: Calculational System Design, vol. 173, pp. 303. IOS Press (1999)Google Scholar
61.
Sutcliffe, G., Suttner, C.B.: Evaluating general purpose automated theorem proving systems. Artif. Intell. 131(1–2), 39–54 (2001)MathSciNetCrossRefMATHGoogle Scholar
62.
Sutcliffe, G., Suttner, C.B.: The state of CASC. AI Commun. 19(1), 35–48 (2006)MathSciNetMATHGoogle Scholar
63.
Thomas, W.: On the synthesis of strategies in infinite games. In: STACS, pp. 1–13 (1995)Google Scholar
64.
van Dijk, T., Laarman, A., van de Pol, J.: Multi-core BDD operations for symbolic reachability. Electron. Notes Theor. Comput. Sci. 296, 127–143 (2013)CrossRefGoogle Scholar
65.
van Dijk, T., van de Pol, J.: Sylvan: Multi-core decision diagrams. In: TACAS 2015, LNCS, vol. 9035, pp. 677–691 Springer (2015)Google Scholar

Copyright information

Authors and Affiliations

Swen Jacobs
- 1
- 2
Email author
Roderick Bloem
- 1
Romain Brenguier
- 3
Rüdiger Ehlers
- 4
- 5
Timotheus Hell
- 1
Robert Könighofer
- 1
Guillermo A. Pérez
- 3
Jean-François Raskin
- 3
Leonid Ryzhyk
- 6
- 7
Ocan Sankur
- 3
- 8
Martina Seidl
- 9
Leander Tentrup
- 2
Adam Walker
- 6

1.Graz University of TechnologyGrazAustria
2.Saarland UniversitySaarbrückenGermany
3.Université Libre de BruxellesBrusselsBelgium
4.University of BremenBremenGermany
5.DFKI GmbHBremenGermany
6.NICTASydneyAustralia
7.Carnegie Mellon UniversityPittsburghUSA
8.CNRS, IRISARennesFrance
9.Johannes-Kepler-University LinzLinzAustria

About this article

CrossMark

Cite this article as:: Jacobs, S., Bloem, R., Brenguier, R. et al. Int J Softw Tools Technol Transfer (2017) 19: 367. https://doi.org/10.1007/s10009-016-0416-3

The first reactive synthesis competition (SYNTCOMP 2014)

Abstract

Keywords

Copyright information

Authors and Affiliations

Personalised recommendations

Export citation

Cookies