Automatic Evaluation of Context-Free Grammars (System Description)

  • Carles Creus
  • Guillem Godoy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8560)


We implement an online judge for context-free grammars. Our system contains a list of problems describing formal languages, and asking for grammars generating them. A submitted proposal grammar receives a verdict of acceptance or rejection depending on whether the judge determines that it is equivalent to the reference solution grammar provided by the problem setter. Since equivalence of context-free grammars is an undecidable problem, we consider a maximum length ℓ and only test equivalence of the generated languages up to words of length ℓ. This length restriction is very often sufficient for the well-meant submissions. Since this restricted problem is still NP-complete, we design and implement methods based on hashing, SAT, and automata that perform well in practice.


grammars equivalence hashing SAT automata 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agarwal, A.: edX (2012),
  2. 2.
    Alexandrova, S., Balandin, A., Compeau, P., Kladov, A., Rayko, M., Sosa, E., Vyahhi, N., Dvorkin, M.: Rosalind (2012),
  3. 3.
    Axelsson, R., Heljanko, K., Lange, M.: Analyzing context-free grammars using an incremental SAT solver. In: Aceto, L., Damgård, I., Goldberg, L.A., Halldórsson, M.M., Ingólfsdóttir, A., Walukiewicz, I. (eds.) ICALP 2008, Part II. LNCS, vol. 5126, pp. 410–422. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  4. 4.
    Biere, A.: PicoSAT essentials. Journal on Satisfiability, Boolean Modeling and Computation 4(2-4), 75–97 (2008)zbMATHGoogle Scholar
  5. 5.
    Eén, N., Sörensson, N.: An extensible SAT-solver. In: Giunchiglia, E., Tacchella, A. (eds.) SAT 2003. LNCS, vol. 2919, pp. 502–518. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Ganesh, V., Kieżun, A., Artzi, S., Guo, P.J., Hooimeijer, P., Ernst, M.: HAMPI: A string solver for testing, analysis and vulnerability detection. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 1–19. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  7. 7.
    García, C., Revilla, M.A.: UVa online judge (1997),
  8. 8.
    Hooker, J.N.: Solving the incremental satisfiability problem. Journal of Logic Programming 15(1&2), 177–186 (1993)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Hopcroft, J.E., Motwani, R., Ullman, J.D.: Introduction to Automata Theory, Languages, and Computation, 3rd edn. Addison-Wesley (2006)Google Scholar
  10. 10.
    Khan, S.: Khan Academy (2006),
  11. 11.
    Knuth, D.E.: The Art of Computer Programming, vol. III: Sorting and Searching. Addison-Wesley (1973)Google Scholar
  12. 12.
    Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., Malik, S.: Chaff: Engineering an efficient SAT solver. In: Annual ACM IEEE Design Automation Conference, pp. 530–535. ACM (2001)Google Scholar
  13. 13.
    Ng, A., Koller, D.: Coursera (2012),
  14. 14.
    Petit, J., Giménez, O., Roura, S.: an educational programming judge. In: ACM Special Interest Group on Computer Science Education, pp. 445–450 (2012)Google Scholar
  15. 15.
    Thrun, S., Stavens, D., Sokolsky, M.: Udacity (2012),
  16. 16.
    Whittemore, J., Kim, J., Sakallah, K.A.: SATIRE: A new incremental satisfiability engine. In: Design Automation Conference, pp. 542–545 (2001)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Carles Creus
    • 1
  • Guillem Godoy
    • 1
  1. 1.Department of SoftwareUniversitat Politècnica de CatalunyaBarcelonaSpain

Personalised recommendations