Parse Table Composition

Separate Compilation and Binary Extensibility of Grammars
  • Martin Bravenboer
  • Eelco Visser
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5452)

Abstract

Module systems, separate compilation, deployment of binary components, and dynamic linking have enjoyed wide acceptance in programming languages and systems. In contrast, the syntax of languages is usually defined in a non-modular way, cannot be compiled separately, cannot easily be combined with the syntax of other languages, and cannot be deployed as a component for later composition. Grammar formalisms that do support modules use whole program compilation.

Current extensible compilers focus on source-level extensibility, which requires users to compile the compiler with a specific configuration of extensions. A compound parser needs to be generated for every combination of extensions. The generation of parse tables is expensive, which is a particular problem when the composition configuration is not fixed to enable users to choose language extensions.

In this paper we introduce an algorithm for parse table composition to support separate compilation of grammars to parse table components. Parse table components can be composed (linked) efficiently at runtime, i.e. just before parsing. While the worst-case time complexity of parse table composition is exponential (like the complexity of parse table generation itself), for realistic language combination scenarios involving grammars for real languages, our parse table composition algorithm is an order of magnitude faster than computation of the parse table for the combined grammars.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Van Wyk, E., Krishnan, L., Bodin, D., Schwerdfeger, A.: Attribute grammar-based language extensions for java. In: Ernst, E. (ed.) ECOOP 2007. LNCS, vol. 4609, pp. 575–599. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Van Wyk, E., Bodin, D., Gao, J., Krishnan, L.: Silver: an extensible attribute grammar system. In: Proc. of the Seventh Workshop on Language Descriptions, Tools and Applications (LDTA 2007). ENTCS, vol. 203, pp. 103–116. Elsevier, Amsterdam (2008)Google Scholar
  3. 3.
    Ekman, T., Hedin, G.: The JastAdd extensible Java compiler. In: OOPSLA 2007: Proc. of the 22nd ACM SIGPLAN Conf. on Object-Oriented Programming, Systems, Languages, and Applications, pp. 1–18. ACM, New York (2007)Google Scholar
  4. 4.
    Nystrom, N., Clarkson, M.R., Myers, A.C.: Polyglot: An extensible compiler framework for java. In: Hedin, G. (ed.) CC 2003. LNCS, vol. 2622, pp. 138–152. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  5. 5.
    Bravenboer, M., Dolstra, E., Visser, E.: Preventing injection attacks with syntax embedding – a host and guest language independent approach. In: GPCE 2007: Proc. of the 6th Intl. Conf. on Generative Programming and Component Engineering, pp. 3–12. ACM, New York (2007)Google Scholar
  6. 6.
    Lhoták, O., Hendren, L.: Jedd: A BDD-based relational extension of Java. In: Proc. of the ACM SIGPLAN 2004 Conf. on Programming Language Design and Implementation (2004)Google Scholar
  7. 7.
    Millstein, T.: Practical predicate dispatch. In: OOPSLA 2004: Proc. of Conf. on Object Oriented Programming, Systems, Languages, and Applications, pp. 345–364. ACM, New York (2004)Google Scholar
  8. 8.
    Arnoldus, B.J., Bijpost, J.W., van den Brand, M.G.J.: Repleo: A syntax-safe template engine. In: GPCE 2007: Proc. of the 6th Intl. Conf. on Generative Programming and Component Engineering, pp. 25–32. ACM, New York (2007)Google Scholar
  9. 9.
    Batory, D., Lofaso, B., Smaragdakis, Y.: JTS: tools for implementing domain-specific languages. In: Proc. Fifth Intl. Conf. on Software Reuse (ICSR 1998), pp. 143–153. IEEE Computer Society Press, Los Alamitos (1998)Google Scholar
  10. 10.
    Visser, E.: Meta-programming with concrete object syntax. In: Batory, D., Consel, C., Taha, W. (eds.) GPCE 2002. LNCS, vol. 2487, pp. 299–315. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  11. 11.
    ASF+SDF Meta-Environment website, http://www.meta-environment.org
  12. 12.
    van Wyk, E., Bodin, D., Huntington, P.: Adding syntax and static analysis to libraries via extensible compilers and language extensions. In: Proc. of Library-Centric Software Design (LCSD 2006), pp. 35–44 (2006)Google Scholar
  13. 13.
    Nystrom, N., Qi, X., Myers, A.C.: J&: nested intersection for scalable software composition. In: OOPSLA 2006: Proc. of Conf. on Object Oriented Programming Systems, Languages, and Applications, pp. 21–36. ACM, New York (2006)Google Scholar
  14. 14.
    Odersky, M., Zenger, M.: Scalable component abstractions. In: OOPSLA 2005: Proc. of Conf. on Object Oriented Programming, Systems, Languages, and Applications, pp. 41–57. ACM, New York (2005)Google Scholar
  15. 15.
    Onzon, E.: Dypgen: Self-extensible parsers for ocaml (2007), http://dypgen.free.fr
  16. 16.
    Salomon, D.J., Cormack, G.V.: Scannerless NSLR(1) parsing of programming languages. In: PLDI 1989: Proc. of the ACM SIGPLAN 1989 Conf. on Programming Language Design and Implementation, pp. 170–178. ACM, New York (1989)CrossRefGoogle Scholar
  17. 17.
    Tomita, M.: Efficient Parsing for Natural Languages. A Fast Algorithm for Practical Systems. Kluwer Academic Publishers, Dordrecht (1985)Google Scholar
  18. 18.
    Rekers, J.: Parser Generation for Interactive Environments. PhD thesis, University of Amsterdam (1992)Google Scholar
  19. 19.
    Visser, E.: Syntax Definition for Language Prototyping. PhD thesis, University of Amsterdam (September 1997)Google Scholar
  20. 20.
    Horspool, R.N.: Incremental generation of LR parsers. Computer Languages 15(4), 205–223 (1990)CrossRefGoogle Scholar
  21. 21.
    Heering, J., Klint, P., Rekers, J.: Incremental generation of parsers. IEEE Transactions on Software Engineering 16(12), 1344–1351 (1990)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Cardelli, L., Matthes, F., Abadi, M.: Extensible syntax with lexical scoping. SRC Research Report 121, Digital Systems Research Center, Palo Alto, California (February 1994)Google Scholar
  23. 23.
    Knuth, D.E.: On the translation of languages from left to right. Information and Control 8(6), 607–639 (1965)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Aho, A.V., Sethi, R., Ullman, J.: Compilers: Principles, techniques, and tools. Addison Wesley, Reading (1986)MATHGoogle Scholar
  25. 25.
    Grune, D., Jacobs, C.J.H.: Parsing Techniques - A Practical Guide. Ellis Horwood, Upper Saddle River (1990)MATHGoogle Scholar
  26. 26.
    Johnstone, A., Scott, E.: Generalised reduction modified LR parsing for domain specific language prototyping. In: 35th Annual Hawaii Intl. Conf. on System Sciences (HICSS 2002), Washington, DC, USA, p. 282. IEEE Computer Society Press, Los Alamitos (2002)Google Scholar
  27. 27.
    Hopcroft, J.E., Motwani, R., Ullman, J.D.: Introduction to Automata Theory, Languages, and Computation, 3rd edn. Addison-Wesley, Boston (2006)MATHGoogle Scholar
  28. 28.
    Leslie, T.: Efficient approaches to subset construction. Master’s thesis, University of Waterloo, Waterloo, Ontario, Canada (1995)Google Scholar
  29. 29.
    van Noord, G.: Treatment of epsilon moves in subset construction. Computational Linguistics 26(1), 61–76 (2000)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Seidel, R., Aragon, C.R.: Randomized search trees. Algorithmica 16(4/5), 464–497 (1996)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Bravenboer, M.: Exercises in Free Syntax. Syntax Definition, Parsing, and Assimilation of Language Conglomerates. PhD thesis, Utrecht University, The Netherlands (January 2008)Google Scholar
  32. 32.
    Bravenboer, M., Tanter, E., Visser, E.: Declarative, formal, and extensible syntax definition for AspectJ – A case for scannerless generalized-LR parsing. In: OOPSLA 2006: Proc. of the 21st ACM SIGPLAN Conf. on Object-Oriented Programming Systems, Languages, and Applications, pp. 209–228. ACM, New York (2006)Google Scholar
  33. 33.
    Grimm, R.: Better extensibility through modular syntax. In: PLDI 2006: Proc. of Conf. on Programming Language Design and Implementation, pp. 38–51. ACM, New York (2006)Google Scholar
  34. 34.
    van Wyk, E., Schwerdfeger, A.: Context-aware scanning for parsing extensible languages. In: GPCE 2007: Proc. of the 6th Intl. Conf. on Generative Programming and Component Engineering, pp. 63–72. ACM, New York (2007)Google Scholar
  35. 35.
    Earley, J.: An efficient context-free parsing algorithm. Communications of the ACM 13(2), 94–102 (1970)CrossRefMATHGoogle Scholar
  36. 36.
    Tratt, L.: Domain specific language implementation via compile-time meta-programming. ACM Transactions on Programming Languages and Systems (TOPLAS) 30(6), 1–40 (2008)CrossRefGoogle Scholar
  37. 37.
    Kolbly, D.M.: Extensible Language Implementation. PhD thesis, University of Texas at Austin (December 2002)Google Scholar
  38. 38.
    Baker, J., Hsieh, W.: Maya: multiple-dispatch syntax extension in Java. In: PLDI 2002: Proc. of the ACM SIGPLAN 2002 Conf. on Programming Language Design and Implementation, pp. 270–281. ACM, New York (2002)CrossRefGoogle Scholar
  39. 39.
    de Rauglaudre, D.: Camlp4 Reference Manual (September 2003)Google Scholar
  40. 40.
    Cervelle, J., Forax, R., Roussel, G.: Separate compilation of grammars with Tatoo. In: Proc. of the Intl. Multiconf. on Computer Science and Information Technology, pp. 1093–1101 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Martin Bravenboer
    • 1
  • Eelco Visser
    • 2
  1. 1.University of OregonUSA
  2. 2.Delft University of TechnologyThe Netherlands

Personalised recommendations