DCGs + Memoing = Packrat Parsing but Is It Worth It?

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4902)


Packrat parsing is a newly popular technique for efficiently implementing recursive descent parsers. Packrat parsing avoids the potential exponential costs of recursive descent parsing with backtracking by ensuring that each production rule in the grammar is tested at most once against each position in the input stream. This paper argues that (a) packrat parsers can be trivially implemented using a combination of definite clause grammar rules and memoing, and that (b) packrat parsing may actually be significantly less efficient than plain recursive descent with backtracking, but (c) memoing the recognizers of just one or two nonterminals, selected in accordance with Amdahl’s law, can sometimes yield speedups. We present experimental evidence to support these claims.


Mercury parsing packrat recursive descent DCG memoing tabling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amdahl, G.: Validity of the single processor approach to achieving large scale computing capabilities. In: Proceedings of the AFIPS Spring Join Computer Conference, Atlantic City, New Jersey, pp. 483–485 (1967)Google Scholar
  2. 2.
    Ford, B.: Packrat parsing: Simple, powerful, lazy, linear time. In: Proceedings of the Seventh ACM SIGPLAN International Conference on Functional Programming, Pittsburgh, Pennsylvania, pp. 36–47 (2002)Google Scholar
  3. 3.
    Grimm, R.: Practical packrat parsing. New York University Technical Report, Dept. of Computer Science, TR2004-854 (2004)Google Scholar
  4. 4.
    Grimm, R.: Better extensibility through modular syntax. In: Proceedings of the ACM SIGPLAN 2006 Conference on Programming Language Design and Implementation, Ottawa, Canada, pp. 38–51 (2006)Google Scholar
  5. 5.
    Henderson, F., Somogyi, Z.: Compiling Mercury to high-level C code. In: Horspool, N. (ed.) Proceedings of the 2002 International Conference on Compiler Construction. LNCS, Springer, Heidelberg (2002)Google Scholar
  6. 6.
    Perreira, F., Warren, D.: Definite clause grammars for language analysis — a survey of the formalism and a comparison with augmented transition networks. Artificial Intelligence 13, 231–278 (1980)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Redziejowski, R.: Parsing expression grammar as a primitive recursive-descent parser with backtracking. Fundamenta Informaticae 79(1-4), 513–524 (2007)zbMATHGoogle Scholar
  8. 8.
    Somogyi, Z., Henderson, F., Conway, T.: The execution algorithm of Mercury: An efficient purely declarative logic programming language. Journal of Logic Programming 29(1-3), 17–64 (1996)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  1. 1.NICTA and Department of Computer Science and Software EngineeringThe University of MelbourneParkville VictoriaAustralia

Personalised recommendations