A New Method for Dependent Parsing

  • Trevor Jim
  • Yitzhak Mandelbaum
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6602)


Dependent grammars extend context-free grammars by allowing semantic values to be bound to variables and used to constrain parsing. Dependent grammars can cleanly specify common features that cannot be handled by context-free grammars, such as length fields in data formats and significant indentation in programming languages. Few parser generators support dependent parsing, however. To address this shortcoming, we have developed a new method for implementing dependent parsers by extending existing parsing algorithms. Our method proposes a point-free language of dependent grammars, which we believe closely corresponds to existing context-free parsing algorithms, and gives a novel transformation from conventional dependent grammars to point-free ones.

To validate our technique, we have specified the semantics of both source and target dependent grammar languages, and proven our transformation sound and complete with respect to those semantics. Furthermore, we have empirically validated the suitability of our point-free language by adapting four parsing engines to support it: an Earley parsing engine; a GLR parsing engine; memoizing, arrow-style parser combinators; and PEG parser combinators.


Target Language Semantic Action Empty String Lambda Calculus Attribute Grammar 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Baars, A., Swierstra, S.D., Viera, M.: Typed transformations of typed grammars: The left corner transform. In: Proceedings of the Ninth Workshop on Language Descriptions, Tools, and Applications, LDTA 2009, pp. 8–33 (March 2009)Google Scholar
  2. 2.
    Conway, M.E.: Design of a separable transition-diagram compiler. Commun. ACM 6(7), 396–408 (1963)CrossRefzbMATHGoogle Scholar
  3. 3.
    Danvy, O., Filinski, A.: Representing Control: A study of the CPS transformation. Mathematical Structures in Computer Science 2(4), 361–391 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Fisher, K., Gruber, R.: PADS: A domain specific language for processing ad hoc data. In: PLDI 2005: Programming Language Design and Implementation, pp. 295–304. ACM Press, New York (2005)CrossRefGoogle Scholar
  5. 5.
    Ford, B.: Parsing expression grammars: a recognition-based syntactic foundation. In: POPL 2004: ACM Symposium on Principles of Programming Languages, pp. 111–122. ACM Press, New York (2004)Google Scholar
  6. 6.
    Ganz, S.E., Friedman, D.P., Wand, M.: Trampolined style. In: ICFP 1999: Proceedings of the Fourth ACM SIGPLAN International Conference on Functional Programming, pp. 18–27. ACM, New York (1999)CrossRefGoogle Scholar
  7. 7.
    HTML5 (including next generation additions still in development) Draft Standard October 6 (2010),
  8. 8.
    Hutton, G., Meijer, E.: Monadic parsing in Haskell. Journal of Functional Programming 8(4), 437–444 (1998)CrossRefzbMATHGoogle Scholar
  9. 9.
    Jim, T., Mandelbaum, Y.: A new method for dependent parsing. Technical Report TD:100334, AT&T Labs—Research (2011)Google Scholar
  10. 10.
    Jim, T., Mandelbaum, Y., Walker, D.: Semantics and algorithms for data-dependent grammars. In: POPL 2010: Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 417–430. ACM, New York (2010)Google Scholar
  11. 11.
    Johnson, M.: Memoization in top-down parsing. Computational Linguistics 21(3), 405–417 (1995)Google Scholar
  12. 12.
    Leijen, D., Meijer, E.: Parsec: Direct style monadic parser combinators for the real world. Technical Report UU-CS-2001-27, Department of Computer Science, Universiteit Utrecht (2001)Google Scholar
  13. 13.
    Onzon, E.: Dypgen: self-extensible parsers and lexers for OCaml,
  14. 14.
    Paterson, R.: A new notation for arrows. In: ICFP 2001: Proceedings of the Sixth ACM SIGPLAN International Conference on Functional Programming, pp. 229–240. ACM, New York (2001)CrossRefGoogle Scholar
  15. 15.
    Pereira, F.C.N., Warren, D.H.D.: Definite clause grammars for language analysis. Artificial Intelligence 13, 231–278 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Pierce, B.C.: Types and Programming Languages. MIT Press, Cambridge (2002)zbMATHGoogle Scholar
  17. 17.
    Scott, E.: SPPF-style parsing from Earley recognisers. In: Proceedings of the Seventh Workshop on Language Descriptions, Tools, and Applications (LDTA 2007). Electronic Notes in Theoretical Computer Science, vol. 203, pp. 53–67. Elsevier, Amsterdam (2008)Google Scholar
  18. 18.
    Scott, E., Johnstone, A.: Right nulled GLR parsers. ACM Transactions on Programming Languages and Systems 28(4), 577–618 (2006)CrossRefzbMATHGoogle Scholar
  19. 19.
    Swierstra, S.D., Azero Alcocer, P.R.: Fast, error correcting parser combinatiors: A short tutorial. In: Bartosek, M., Tel, G., Pavelka, J. (eds.) SOFSEM 1999. LNCS, vol. 1725, pp. 112–131. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  20. 20.
    Warren, S.K.: The coroutine model of attribute grammar evaluation. PhD thesis, Rice University, Houston, TX, USA (1976)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Trevor Jim
    • 1
  • Yitzhak Mandelbaum
    • 1
  1. 1.AT&T Labs–ResearchUSA

Personalised recommendations