Benchmarks for Parity Games

  • Jeroen J. A. KeirenEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9392)


We propose a benchmark suite for parity games that includes the benchmarks that have been used in the literature, and make it available online. We give an overview of the parity games, including a description of how they have been generated. We also describe structural properties of parity games, and using these properties we show that our benchmarks are representative. With this work we provide a starting point for further experimentation with parity games.


Model Check Equivalence Check Quotient Graph Model Check Problem Cache Coherence Protocol 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adler, I.: Directed tree-width examples. Journal of Combinatorial Theory, Series B 97(5), 718–725 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Albert, M.H., Grossman, J.P., Nowakowski, R.J., Wolfe, D.: An introduction to clobber. Integers 5(2) (2005)Google Scholar
  3. 3.
    Arnborg, S., Corneil, D.G., Proskurowski, A.: Complexity of finding embeddings in a k-tree. SIAM Journal on Algebraic Discrete Methods 8(2), 277–284 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bartlett, K.A., Scantlebury, R.A., Wilkinson, P.T.: A note on reliable full-duplex transmission over half-duplex links. Communications of the ACM 12(5), 260–261 (1969)CrossRefGoogle Scholar
  5. 5.
    Beck, A., Bleicher, M.N., Crowe, D.W.: Excursions into Mathematics: The Millennium Edition. CRC Press (2000)Google Scholar
  6. 6.
    Beffara, E., Vorobyov, S.G.: Adapting Gurvich-Karzanov-Khachiyan’s algorithm for parity games. Technical report, Uppsala University, Sweden, Uppsala (2001)Google Scholar
  7. 7.
    Berwanger, D., Dawar, A., Hunter, P.W., Kreutzer, S., Obdržálek, J.: The DAG-width of directed graphs. Journal of Combinatorial Theory, Series B 102(4), 900–923 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Berwanger, D., Grädel, E.: Fixed-point logics and solitaire games. Theory of Computing Systems 37(6), 675–694 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Berwanger, D., Grädel, E.: Entanglement – A measure for the complexity of directed graphs with applications to logic and games. In: Baader, F., Voronkov, A. (eds.) LPAR 2004. LNCS (LNAI), vol. 3452, pp. 209–223. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  10. 10.
    Berwanger, D., Grädel, E., Kaiser, L., Rabinovich, R.: Entanglement and the complexity of directed graphs. Theoretical Computer Science 463, 2–25 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Bodlaender, H.L.: Treewidth: Algorithmic techniques and results. In: Privara, I., Ružička, P. (eds.) MFCS 1997. LNCS, vol. 1295, pp. 19–36. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  12. 12.
    Bodlaender, H.L., Koster, A.M.C.A.: Treewidth computations I. upper bounds. Information and Computation 208(3), 259–275 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Bodlaender, H.L., Koster, A.M.C.A.: Treewidth computations II. lower bounds. Information and Computation 209(7), 1103–1119 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Bradfield, J.C., Stirling, C.: Modal logics and mu-calculi: an introduction. In: Handbook of Process Algebra, pp. 293–330. Elsevier (2000)Google Scholar
  15. 15.
    Cerf, V., Kahn, R.E.: A protocol for packet network intercommunication. IEEE Transactions on Communications 22(5), 637–648 (1974)CrossRefGoogle Scholar
  16. 16.
    Chatterjee, K., Henzinger, T.A., Jobstmann, B., Radhakrishna, A.: GIST: A solver for probabilistic games. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 665–669. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Chen, T., Ploeger, S.C.W., van de Pol, J.C., Willemse, T.A.C.: Equivalence checking for infinite systems using parameterized Boolean equation systems. In: Caires, L., Vasconcelos, V.T. (eds.) CONCUR 2007. LNCS, vol. 4703, pp. 120–135. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  18. 18.
    Cleaveland, R., Klein, M., Steffen, B.: Faster model checking for the modal mu-calculus. In: Probst, D.K., von Bochmann, G. (eds.) CAV 1992. LNCS, vol. 663, pp. 410–422. Springer, Heidelberg (1993)CrossRefGoogle Scholar
  19. 19.
    Courcelle, B., Olariu, S.: Upper bounds to the clique width of graphs. Discrete Applied Mathematics 101(1-3), 77–114 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Cranen, S., Groote, J.F., Keiren, J.J.A., Stappers, F.P.M., de Vink, E.P., Wesselink, J.W., Willemse, T.A.C.: An overview of the mCRL2 toolset and its recent advances. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013 (ETAPS 2013). LNCS, vol. 7795, pp. 199–213. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  21. 21.
    Cranen, S., Keiren, J.J.A., Willemse, T.A.C.: Stuttering mostly speeds up solving parity games. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 207–221. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  22. 22.
    Cranen, S., Keiren, J.J.A., Willemse, T.A.C.: A cure for stuttering parity games. In: Roychoudhury, A., D’Souza, M. (eds.) ICTAC 2012. LNCS, vol. 7521, pp. 198–212. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  23. 23.
    Di Stasio, A., Murano, A., Prignano, V., Sorrentino, L.: Solving parity games in Scala. In: Lanese, I., Madelaine, E. (eds.) FACS 2014. LNCS, vol. 8997, pp. 145–161. Springer, Heidelberg (2015)Google Scholar
  24. 24.
    Emerson, E.A., Jutla, C.S.: Tree automata, mu-calculus and determinacy. In: SFCS 1991: Proceedings of the 32nd Annual Symposium on Foundations of Computer Science, pp. 368–377. IEEE Computer Society (1991)Google Scholar
  25. 25.
    Emerson, E.A., Lei, C.L.L.: Efficient model checking in fragments of the propositional mu-calculus. In: Proceedings of LICS 1986, pp. 267–278. IEEE Computer Society (1986)Google Scholar
  26. 26.
    Friedmann, O.: A super-polynomial lower bound for the parity game strategy improvement algorithm as we know it. In: 2009 24th Annual IEEE Symposium on Logic In Computer Science, vol. 7, pp. 145–156 (2009)Google Scholar
  27. 27.
    Friedmann, O.: The Stevens-Stirling-algorithm for solving parity games locally requires exponential time. International Journal of Foundations of Computer Science 21(03), 277–287 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Friedmann, O.: An exponential lower bound for the latest deterministic strategy iteration algorithms. Logical Methods in Computer Science 7, 1–42 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Friedmann, O.: Recursive algorithm for parity games requires exponential time. RAIRO - Theoretical Informatics and Applications 45(4), 449–457 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Friedmann, O., Lange, M.: Solving parity games in practice. In: Liu, Z., Ravn, A.P. (eds.) ATVA 2009. LNCS, vol. 5799, pp. 182–196. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  31. 31.
    Friedmann, O., Lange, M.: The PGSolver collection of parity game solvers. Technical report, Institut für Informatik, Ludwig-Maximilians-Universität München, Germany (2010)Google Scholar
  32. 32.
    Friedmann, O., Lange, M.: A solver for modal fixpoint logics. In: Electronic Notes in Theoretical Computer Science, vol. 262, pp. 99–111. Elsevier (2010)Google Scholar
  33. 33.
    Friedmann, O., Latte, M., Lange, M.: A decision procedure for CTL* based on tableaux and automata. In: Giesl, J., Hähnle, R. (eds.) IJCAR 2010. LNCS, vol. 6173, pp. 331–345. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  34. 34.
    Gardner, M.: Mathematical games: Cram, crosscram and quadraphage: New games having elusive winning strategies. Scientific American 230, 106–108 (1974)CrossRefGoogle Scholar
  35. 35.
    Gazda, M.W., Willemse, T.A.C.: Zielonka’s recursive algorithm: dull, weak and solitaire games and tighter bounds. In: Proceedings GandALF 2013. EPTCS, vol. 119, pp. 7–20 (2013)Google Scholar
  36. 36.
    Gogate, V., Dechter, R.: A complete anytime algorithm for treewidth. In: Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, UAI 2004, pp. 201–208. AUAI Press (2004)Google Scholar
  37. 37.
    Groote, J.F., Pang, J., Wouters, A.G.G.: Analysis of a distributed system for lifting trucks. The Journal of Logic and Algebraic Programming 55(1-2), 21–56 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Groote, J.F., van de Pol, J.: A bounded retransmission protocol for large data packets. In: Nivat, M., Wirsing, M. (eds.) AMAST 1996. LNCS, vol. 1101, pp. 536–550. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  39. 39.
    Hesselink, W.H.: Invariants for the construction of a handshake register. Information Processing Letters 68, 173–177 (1998)CrossRefzbMATHGoogle Scholar
  40. 40.
    Hunter, P.W., Kreutzer, S.: Digraph measures: Kelly decompositions, games, and orderings. Theoretical Computer Science 399(3), 206–219 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Johnson, T., Robertson, N., Seymour, P.D., Thomas, R.: Directed tree-width. Journal of Combinatorial Theory, Series B 82(1), 138–154 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Jurdziński, M.: Deciding the winner in parity games is in UP ∩ co-UP. Information Processing Letters 68(3), 119–124 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Jurdziński, M.: Small progress measures for solving parity games. In: Reichel, H., Tison, S. (eds.) STACS 2000. LNCS, vol. 1770, pp. 290–301. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  44. 44.
    Jurdziński, M., Paterson, M., Zwick, U.: A deterministic subexponential algorithm for solving parity games. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm, SODA 2006, pp. 117–123 (2006)Google Scholar
  45. 45.
    Keiren, J.J.A.: Advanced Reduction Techniques for Model Checking. PhD thesis, Eindhoven University of Technology (2013)Google Scholar
  46. 46.
    Keiren, J.J.A.: Benchmarks for parity games. CoRR, abs/1407.3121 (2014)Google Scholar
  47. 47.
    Keiren, J.J.A., Willemse, T.A.C.: Bisimulation minimisations for Boolean equation systems. In: Namjoshi, K., Zeller, A., Ziv, A. (eds.) HVC 2009. LNCS, vol. 6405, pp. 102–116. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  48. 48.
    Koymans, C.P.J., Mulder, J.C.: A modular approach to protocol verification using process algebra. In: Applications of Process Algebra. Cambridge Tracts in Theoretical Computer Science, vol. 17, pp. 261–306 (1990)Google Scholar
  49. 49.
    Lange, M.: Solving parity games by a reduction to SAT. In: Proc. of the Workshop on Games in Design and Verification, GDV 2005 (2005)Google Scholar
  50. 50.
    Larsen, K.G.: Efficient local correctness checking. In: Probst, D.K., von Bochmann, G. (eds.) CAV 1992. LNCS, vol. 663, pp. 30–43. Springer, Heidelberg (1993)CrossRefGoogle Scholar
  51. 51.
    Luttik, S.P.: Description and formal specification of the link layer of P1394. In: Workshop on Applied Formal Methods in System Design, pp. 43–56 (1997)Google Scholar
  52. 52.
    Maarup, T.: Hex - everything you always wanted to know about hex but were afraid to ask. Master’s thesis (2005)Google Scholar
  53. 53.
    Mader, A.: Verification of Modal Properties Using Boolean Equation Systems. PhD thesis, Technische Universität München (1997)Google Scholar
  54. 54.
    Mateescu, R.: A generic on-the-fly solver for alternation-free Boolean equation systems. In: Garavel, H., Hatcliff, J. (eds.) TACAS 2003. LNCS, vol. 2619, pp. 81–96. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  55. 55.
    McNaughton, R.: Infinite games played on finite graphs. Annals of Pure and Applied Logic 65(2), 149–184 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  56. 56.
    Nowakowski, R., Winkler, P.: Vertex-to-vertex pursuit in a graph. Discrete Mathematics 43(2-3), 235–239 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Obdržálek, J.: Fast mu-calculus model checking when tree-width is bounded. In: Hunt Jr., W.A., Somenzi, F. (eds.) CAV 2003. LNCS, vol. 2725, pp. 80–92. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  58. 58.
    Obdržálek, J.: Algorithmic Analysis of Parity Games. PhD thesis, Laboritory for Foundations of Computer Science, School of Informatics, University of Edinburgh (2006)Google Scholar
  59. 59.
    Pang, J., Fokkink, W.J., Hofman, R., Veldema, R.: Model checking a cache coherence protocol of a Java DSM implementation. The Journal of Logic and Algebraic Programming 71(1), 1–43 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  60. 60.
    Pelánek, R.: Typical structural properties of state spaces. In: Graf, S., Mounier, L. (eds.) SPIN 2004. LNCS, vol. 2989, pp. 5–22. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  61. 61.
    Pelánek, R.: Web portal for benchmarking explicit model checkers. Technical Report FIMU-RS-2006-03, Faculty of Informatics Masaryk University Brno (2006)Google Scholar
  62. 62.
    Pelánek, R.: BEEM: benchmarks for explicit model checkers. In: Bošnački, D., Edelkamp, S. (eds.) SPIN 2007. LNCS, vol. 4595, pp. 263–267. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  63. 63.
    Quilliot, A.: Jeux et pointes fixes sur les graphes. PhD thesis, Université de Paris VI (1978)Google Scholar
  64. 64.
    Robertson, N., Seymour, P.D.: Graph minors. II. algorithmic aspects of tree-width. Journal of Algorithms 7(3), 309–322 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  65. 65.
    Rose, B.: Othello: A Minute to Learn... A Lifetime to Master (2005)Google Scholar
  66. 66.
    Safra, S.: On the complexity of omega-automata. In: 29th Annual Symposium on Foundations of Computer Science, pp. 319–327. IEEE (1988)Google Scholar
  67. 67.
    Schewe, S.: Solving parity games in big steps. In: Arvind, V., Prasad, S. (eds.) FSTTCS 2007. LNCS, vol. 4855, pp. 449–460. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  68. 68.
    Schewe, S.: An optimal strategy improvement algorithm for solving parity and payoff games. In: Kaminski, M., Martini, S. (eds.) CSL 2008. LNCS, vol. 5213, pp. 369–384. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  69. 69.
    Schewe, S.: Synthesis of Distributed Systems. Phd thesis, Universität des Saarlandes (2008)Google Scholar
  70. 70.
    Siek, J.G., Lee, L.Q., Lumsdaine, A.: The Boost Graph Library: User Guide and Reference Manual. Addison-Wesley (2002)Google Scholar
  71. 71.
    Sighireanu, M., Mateescu, R.: Verification of the link layer protocol of the IEEE-1394 serial bus (FireWire): An experiment with e-Lotos. STTT 2(1), 68–88 (1998)CrossRefzbMATHGoogle Scholar
  72. 72.
    Stevens, P., Stirling, C.: Practical model checking using games. In: Steffen, B. (ed.) TACAS 1998. LNCS, vol. 1384, pp. 85–101. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  73. 73.
    Stirling, C.: Bisimulation, modal logic and model checking games. Logic Journal of IGPL 7(1), 103–124 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  74. 74.
    Tsay, Y.K., Chen, Y.F., Tsai, M.H., Chan, W.C., Luo, C.J.: GOAL extended: Towards a research tool for omega automata and temporal logic. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 346–350. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  75. 75.
    van de Pol, J.C., Weber, M.: A multi-core solver for parity games. Electronic Notes in Theoretical Computer Science 220(2), 19–34 (2008)CrossRefzbMATHGoogle Scholar
  76. 76.
    Veldema, R., Hofman, R.F.H., Bhoedjang, R.A.F., Jacobs, C.J.H., Bal, H.E.: Source-level global optimizations for fine-grain distributed shared memory systems. ACM SIGPLAN Notices 36(7), 83–92 (2001)CrossRefGoogle Scholar
  77. 77.
    Vergauwen, B., Lewi, J.: Efficient local correctness checking for single and alternating Boolean equation systems. In: Shamir, E., Abiteboul, S. (eds.) ICALP 1994. LNCS, vol. 820, pp. 304–315. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  78. 78.
    Zielonka, W.: Infinite games on finitely coloured graphs with applications to automata on infinite trees. Theoretical Computer Science 200(1-2), 135–183 (1998)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  1. 1.Faculty of Management, Science & TechnologyOpen University of the NetherlandsHeerlenThe Netherlands
  2. 2.Theoretical Computer ScienceVU University AmsterdamAmsterdamThe Netherlands

Personalised recommendations