Lessons learned from developing mbeddr: a case study in language engineering with MPS

  • Markus Voelter
  • Bernd Kolb
  • Tamás Szabó
  • Daniel Ratiu
  • Arie van Deursen
Regular Paper

Abstract

Language workbenches are touted as a promising technology to engineer languages for use in a wide range of domains, from programming to science to business. However, not many real-world case studies exist that evaluate the suitability of language workbench technology for this task. This paper contains such a case study. In particular, we evaluate the development of mbeddr, a collection of integrated languages and language extensions built with the Jetbrains MPS language workbench. mbeddr consists of 81 languages, with their IDE support, 34 of them C extensions. The mbeddr languages use a wide variety of notations—textual, tabular, symbolic and graphical—and the C extensions are modular; new extensions can be added without changing the existing implementation of C. mbeddr’s development has spanned 10 person-years so far, and the tool is used in practice and continues to be developed. This makes mbeddr a meaningful case study of non-trivial size and complexity. The evaluation is centered around five research questions: language modularity, notational freedom and projectional editing, mechanisms for managing complexity, performance and scalability issues and the consequences for the development process. We draw generally positive conclusions; language engineering with MPS is ready for real-world use. However, we also identify a number of areas for improvement in the state of the art in language engineering in general, and in MPS in particular.

Keywords

Language engineering Language extension Language workbenches Domain-specific language Case study Languages Experimentation 

References

  1. 1.
    Code Orchestra IDE. http://codeorchestra.com/ide
  2. 2.
  3. 3.
  4. 4.
    Jetbrains Meta Programming System. http://www.Jetbrains.com/mps
  5. 5.
    Jetbrains MPS Documentation. https://www.jetbrains.com/mps/documentation
  6. 6.
  7. 7.
    Aho, A.V., Johnson, S.C., Ullman, J.D.: Deterministic parsing of ambiguous grammars. Commun. ACM 18(8), 441–452 (1975)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Backes, M., Hriţcu, C., Tarrach, T.: Automatically verifying typing constraints for a data processing language. In: Certified Programs and Proofs. Springer (2011)Google Scholar
  9. 9.
    Barroca, B., Lúcio, L., Amaral, V., Félix, R., Sousa, V.: DSLtrans: a turing incomplete transformation language. In: Software Language Engineering. Springer, pp. 296–305 (2011)Google Scholar
  10. 10.
    Basten, B., van den Bos, J., Hills, M., Klint, P., Lankamp, A., Lisser, B., van der Ploeg, A., van der Storm, T., Vinju, J.: Modular language implementation in Rascal—experience report. Sci. Comput. Program. 114, 7–19 (2015)CrossRefGoogle Scholar
  11. 11.
    Bettini, L.: Implementing Domain-Specific Languages with Xtext and Xtend. Packt Publishing Ltd, Birmingham (2013)Google Scholar
  12. 12.
    Bierman, G.M., Gordon, A.D., Hriţcu, C., Langworthy, D.: Semantic subtyping with an SMT solver. In: ACM Sigplan Notices, vol. 45. ACM, pp. 105–116 (2010)Google Scholar
  13. 13.
    Broy, M., Kirstan, S., Krcmar, H., Schätz, B.: What is the benefit of a model-based design of embedded software systems in the car industry? In: Rech, J., Bunse, C. (eds.) Emerging Technologies for the Evolution and Maintenance of Software Models. IGI Global, Hershey (2011)Google Scholar
  14. 14.
    Campagne, F.: The MPS Language Workbench, vol. 1. Fabien Campagne, New York (2014)Google Scholar
  15. 15.
    Chiş, A., Gîrba, T., Nierstrasz, O.: The moldable debugger: a framework for developing domain-specific debuggers. In: International Conference on Software Language Engineering. Springer, pp. 102–121 (2014)Google Scholar
  16. 16.
    Christerson,M., Kolk, H.: Domain Expert DSLs, 2009. talk at QCon London 2009. http://www.infoq.com/presentations/DSL-Magnus-Christerson-Henk-Kolk
  17. 17.
    Cimatti, A., Clarke, E., Giunchiglia, E., Giunchiglia, F., Pistore, M., Roveri, M., Sebastiani, R., Tacchella, A.: NUSMV 2: an opensource tool for symbolic model checking. In: Computer Aided Verification. Springer, pp. 359–364 (2002)Google Scholar
  18. 18.
    Clark, T.: A declarative approach to heterogeneous multi-mode modelling languages. In: Proceedings of the GEMOC Workshop (2014)Google Scholar
  19. 19.
    Clarke, E., Kroening, D., Lerda, F.: A tool for checking ANSI-C programs. In: Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science, vol. 2988. Springer (2004)Google Scholar
  20. 20.
    Conrad, M., Sandmann, G., Munier, P.: Software tool qualification according to ISO 26262. Technical report, SAE (2011)Google Scholar
  21. 21.
    Czarnecki, K., Antkiewicz, M.: Mapping features to models: a template approach based on superimposed variants. In: Generative Programming and Component Engineering. Springer, pp. 422–437 (2005)Google Scholar
  22. 22.
    Czarnecki, K., Eisenecker, U.W.: Generative Programming: Methods, Tools, and Applications. ACM Press/Addison-Wesley, New York (2000)Google Scholar
  23. 23.
    De Moura, L., Bjørner, N.: Z3: An efficient SMT solver. In: Tools and Algorithms for the Construction and Analysis of Systems. Springer, pp. 337–340 (2008)Google Scholar
  24. 24.
    Dmitriev, S.: Language oriented programming: the next programming paradigm. JetBrains onBoard 1(2), 1–13 (2004)Google Scholar
  25. 25.
    Dutertre, B., De Moura, L.: The Yices SMT solver. Tool paper at http://yices.csl.sri.com/tool-paper.pdf, 2(2) (2006)
  26. 26.
    Dybå, T., Sjøberg, D.I., Cruzes, D.S.: What works for whom, where, when, and why? On the role of context in empirical software engineering. In: Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (2012)Google Scholar
  27. 27.
    Earley, J.: Ambiguity and precedence in syntax description. Acta Inform. 4(2), 183–192 (1975)CrossRefMATHGoogle Scholar
  28. 28.
    Ellison, C., Rosu, G.: An executable formal semantics of C with applications. In: ACM SIGPLAN Notices, vol. 47. ACM, pp. 533–544 (2012)Google Scholar
  29. 29.
    Erdweg, S., Kats, L.C., Rendel, T., Kästner, C., Ostermann, K., Visser, E.: Growing a language environment with editor libraries. In: Proceedings of OOPSLA 2011, vol. 47. ACM, pp. 167–176 (2011)Google Scholar
  30. 30.
    Erdweg, S., Giarrusso, P.G., Rendel, T.: Language composition untangled. In: Proceedings of LDTA (2012)Google Scholar
  31. 31.
    Erdweg, S., Storm, T., Völter, M., et al.: The state of the art in language workbenches. In: Erwig, M., Paige, R., Wyk, E. (eds.) Software Language Engineering. LNCS, vol. 8225. Springer (2013)Google Scholar
  32. 32.
    Erdweg, S., van der Storm, T., Dai, Y.: Capture-avoiding and hygienic program transformations. In: European Conference on Object-Oriented Programming. Springer, pp. 489–514 (2014)Google Scholar
  33. 33.
    Fowler, M.: Language workbenches: killer-app for DSLs? ThoughtWorks. http://www.martinfowler.com/articles/languageWorkbench.html (2005)
  34. 34.
    Fowler, M., Beck, K.: Refactoring: Improving the design of existing code. Addison-Wesley Professional, Boston (1999)Google Scholar
  35. 35.
    Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Pearson Education, London (1994)MATHGoogle Scholar
  36. 36.
    Gray, J., Karsai, G.: An examination of DSLs for concisely representing model traversals and transformations. In: Proceedings of HICSS (2003)Google Scholar
  37. 37.
    Hathhorn, C., Ellison, C., Roşu, G.: Defining the undefinedness of c. In: ACM SIGPLAN Notices, vol. 50. ACM, pp. 336–345 (2015)Google Scholar
  38. 38.
    Heering, J., Hendriks, P.R.H., Klint, P., Rekers, J.: The syntax definition formalism SDF—reference manual. SIGPLAN 24(11), 43–75 (1989)CrossRefGoogle Scholar
  39. 39.
    Hermans, F., Pinzger, M., Van Deursen, A.: Domain-specific languages in practice: a user study on the success factors. In: International Conference on Model Driven Engineering Languages and Systems. Springer (2009)Google Scholar
  40. 40.
    Hillebrand, J., Reichenpfader, P., Mandic, I., Siegl, H., Peer, C.: Establishing confidence in the usage of software tools in context of ISO 26262. In: Computer Safety, Reliability, and Security. Springer, pp. 257–269 (2011)Google Scholar
  41. 41.
    Holzmann, G.J.: The model checker SPIN. IEEE Trans. Softw. Eng. 5, 279–295 (1997)CrossRefGoogle Scholar
  42. 42.
    Jimenez, M., Rosique, F., Sanchez, P., Alvarez, B., Iborra, A.: Habitation: a domain-specific language for home automation. IEEE Softw. 26(4), 30–38 (2009)CrossRefGoogle Scholar
  43. 43.
    Jones, C., Bonsignour, O.: The Economics of Software Quality. Addison-Wesley Professional, Boston (2011)Google Scholar
  44. 44.
    Kats, L.C.L., Visser, E.: The Spoofax language workbench: rules for declarative specification of languages and IDEs. In: OOPSLA. ACM (2010)Google Scholar
  45. 45.
    Kieburtz, R.B., McKinney, L., Bell, J.M., Hook, J., Kotov, A., Lewis, J., Oliva, D.P., Sheard, T., Smith, I., Walton, L.: A software engineering experiment in software component generation. In: Proceedings ICSE (1996)Google Scholar
  46. 46.
    Klint, P.: A meta-environment for generating programming environments. ACM Trans. Softw. Eng. Methodol. 2(2), 176–201 (1993)CrossRefGoogle Scholar
  47. 47.
    Klint, P., Van Rozen, R.: Micro-machinations. In: Software Language Engineering. Springer, pp. 36–55 (2013)Google Scholar
  48. 48.
    Klint, P., van der Storm, T., Vinju, J.: EASY meta-programming with Rascal. In: GTTSE III. LNCS, vol. 6491. Springer (2011)Google Scholar
  49. 49.
    Kosar, T., Martı, P.E., Barrientos, P.A., Mernik, M., et al.: A preliminary study on various implementation approaches of domain-specific language. Inf. Softw. Technol. 50(5), 390–405 (2008)CrossRefGoogle Scholar
  50. 50.
    Kosar, T., Oliveira, N., Mernik, M., Pereira, V.J.M., Črepinšek, M., Da, C.D., Henriques, R.P.: Comparing general-purpose and domain-specific languages: an empirical study. Comput. Sci. Inf. Syst. 7(2), 247–264 (2010)CrossRefGoogle Scholar
  51. 51.
    Krebbers, R.: The C Standard Formalized in Coq. PhD thesis, Radboud University Nijmegen (2015)Google Scholar
  52. 52.
    Kuhn, A., Murphy, G., Thompson, C.: An exploratory study of forces and frictions affecting large-scale model-driven development. In: France, R., Kazmeier, J., Breu, R., Atkinson, C. (eds.) Model Driven Engineering Languages and Systems. LNCS, vol. 7590. Springer (2012)Google Scholar
  53. 53.
    Le Berre, D., Parrain, A.: The Sat4j library, release 2.2. J. Satisf. Boolean Model. Comput. 7, 59–64 (2010)Google Scholar
  54. 54.
    Liebel, G., Marko, N., Tichy, M., Leitner, A., Hansson, J.: Assessing the state-of-practice of model-based engineering in the embedded systems domain. In: Proceedings of the MODELS (2014)Google Scholar
  55. 55.
    Liggesmeyer, P., Trapp, M.: Trends in embedded software engineering. IEEE Softw. 26, 19–25 (2009)CrossRefGoogle Scholar
  56. 56.
    Lussenburg, V., Van Der Storm, T., Vinju, J., Warmer, J.: Mod4j: a qualitative case study of model-driven software development. In: Model Driven Engineering Languages and Systems. Springer, pp. 346–360 (2010)Google Scholar
  57. 57.
    Medina-Mora, R., Feiler, P.H.: An incremental programming environment. IEEE Trans. Softw. Eng. 7(5), 472–482 (1981)CrossRefGoogle Scholar
  58. 58.
    Molotnikov, Z., Völter, M., Ratiu, D.: Automated domain-specific C verification with mbeddr. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering. ACM (2014)Google Scholar
  59. 59.
    Neron, P., Tolmach, A.P., Visser, E., Wachsmuth, G.: A theory of name resolution. In: Vitek, J., (ed.), 24th European Symposium on Programming, ESOP 2015. Lecture Notes in Computer Science, vol. 9032. Springer, pp. 205–231 (2015) ISBN 978-3-662-46668-1. doi:10.1007/978-3-662-46669-8_9
  60. 60.
    Nichols, B., Buttlar, D., Farrell, J.: Pthreads Programming: A POSIX Standard for Better Multiprocessing. O’Reilly, Sebastopol (1996)Google Scholar
  61. 61.
    Notkin, D.: The GANDALF project. J. Syst. Softw. 5(2), 91–106 (1985)CrossRefGoogle Scholar
  62. 62.
    Odersky, M., Spoon, L., Venners, B.: Programming in Scala. Artima, Mountain View (2008)Google Scholar
  63. 63.
    Pavletic, D., Haßlbauer, K.: Interactive debugging for extensible languages in multi-stage transformation environments. In: 2nd International Workshop on Executable Modeling at MODELs 2016 (2016)Google Scholar
  64. 64.
    Pavletic, D., Voelter, M., Raza, S.A., Kolb, B., Kehrer, T.: Extensible debugger framework for extensible languages. In: Reliable Software Technologies–Ada-Europe 2015. Springer, pp. 33–49 (2015)Google Scholar
  65. 65.
    Porter, S.W.: Design of a Syntax Directed Editor for PSDL. Master’s thesis, Naval Postgraduate School, Monterey, CA, USA (1988)Google Scholar
  66. 66.
    Ratiu, D., Voelter, M.: Automated testing of DSL implementations. In: 11th IEEE/ACM International Workshop on Automation of Software Test (AST 2016) (2016)Google Scholar
  67. 67.
    Ratiu, D., Voelter, M., Molotnikov, Z., Schaetz, B.: Implementing modular domain specific languages and analyses. In: Proceedings of the Workshop on Model-Driven Engineering, Verification and Validation. ACM (2012)Google Scholar
  68. 68.
    Ratiu, D., Voelter, M., Schaetz, B., Kolb, B.: Language engineering as enabler for incrementally defined formal analyses. In: Proceedings of the Workshop on Formal Methods in Software Engineering: Rigorous and Agile Approaches (FORMSERA’2012) (2012)Google Scholar
  69. 69.
    Ratiu, D., Voelter, M., Kolb, B., Schaetz, B.: Using language engineering to lift languages and analyses at the domain level. In: Proceedings the 5th NASA Formal Methods Symposium (NFM’13) (2013)Google Scholar
  70. 70.
    Ratiu, D., Zeller, M., Kilian, L.: Safety.Lab: Model-based domain specific tooling for safety argumentation. In: Proceedings of the 3rd International Workshop on Assurance Cases for Software-intensive Systems (2015)Google Scholar
  71. 71.
    Reps, T.W., Teitelbaum, T.: The Synthesizer Generator. In: First ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments. ACM (1984)Google Scholar
  72. 72.
    Rosu, G., Serbănută, T.F.: An overview of the k semantic framework. J. Log. Algebraic Program. 79(6), 397–434 (2010)MathSciNetCrossRefMATHGoogle Scholar
  73. 73.
    Runeson, P., Host, M., Rainer, A., Regnell, B.: Case Study Research in Software Engineering: Guidelines and Examples. Wiley, New York (2012)CrossRefGoogle Scholar
  74. 74.
    Selim, G.M., Lúcio, L., Cordy, J.R., Dingel, J., Oakes, B.J.: Specification and verification of graph-based model transformation properties. In: Graph Transformation. Springer, pp. 113–129 (2014)Google Scholar
  75. 75.
    Simi, M., Campagne, F.: Composable languages for bioinformatics: the NYoSh experiment. PeerJ PrePrints, 1:e112v2 (2013)Google Scholar
  76. 76.
    Simonyi, C.: The death of computer languages, the birth of intentional programming. In: NATO Science Committee Conference (1995)Google Scholar
  77. 77.
    Simonyi, C., Christerson, M., Clifford, S.: Intentional software. SIGPLAN Not. 41(10), 451–464 (2006)CrossRefGoogle Scholar
  78. 78.
    Steinberg, D., Budinsky, F., Merks, E., Paternostro, M.: EMF: Eclipse Modeling Framework. Pearson Education, London (2008)Google Scholar
  79. 79.
    Sullivan, K.J., Griswold, W.G., Cai, Y., Hallen, B.: The structure and value of modularity in software design. In: ACM SIGSOFT Software Engineering Notes, vol. 26. ACM, pp. 99–108 (2001)Google Scholar
  80. 80.
    Teitelbaum, T., Reps, T.: The Cornell program synthesizer: a syntax-directed programming environment. Commun. ACM 24(9), 563–573 (1981)CrossRefGoogle Scholar
  81. 81.
    Thomas, D., Hunt, A.: Mock objects. IEEE Softw. 19(3), 22–24 (2002)CrossRefGoogle Scholar
  82. 82.
    Tolvanen, J.-P., Kelly, S.: MetaEdit+: defining and using integrated domain-specific modeling languages. In: OOPSLA 2009, OOPSLA ’09. ACM (2009)Google Scholar
  83. 83.
    Tomita, M.: Generalized LR Parsing. Springer, New York (2012)Google Scholar
  84. 84.
    Tratt, L.: Domain specific language implementation via compile-time meta-programming. TOPLAS 30(6), 1–40 (2008)CrossRefGoogle Scholar
  85. 85.
    Ujhelyi, Z., Bergmann, G., Hegedüs, Á., Horváth, Á., Izsó, B., Ráth, I., Szatmári, Z., Varró, D.: EMF-IncQuery: an integrated development environment for live model queries. Sci. Comput. Program. 98, 80–99 (2015)CrossRefGoogle Scholar
  86. 86.
    van Antwerpen, H., Neron, P., Tolmach, A.P., Visser, E., Wachsmuth, G.: A constraint language for static semantic analysis based on scope graphs. In: Erwig, M., Rompf, T. (eds.) 2016 Workshop on Partial Evaluation and Program Manipulation, PEPM 2016. ACM, pp. 49–60 (2016). doi:10.1145/2847538.2847543
  87. 87.
    van den Bos, J., van der Storm, T.: Bringing domain-specific languages to digital forensics. In: Proceedings of the 33rd International Conference on Software Engineering. ACM, pp. 671–680 (2011)Google Scholar
  88. 88.
    van Deursen, A.: Domain-specific languages versus object-oriented frameworks: A financial engineering case study. In: Smalltalk and Java in Industry and Academia, STJA’97, pp. 35–39 (1997)Google Scholar
  89. 89.
    van Deursen, A., Klint, P.: Little languages: little maintenance? J. Softw. Maint. 10(2), 75–92 (1998)CrossRefGoogle Scholar
  90. 90.
    Van Deursen, A., Klint, P., Tip, F.: Origin tracing. J. Symb. Comput. 15(5), 523–545 (1993)CrossRefMATHGoogle Scholar
  91. 91.
    van Rest, O., Wachsmuth, G., Steel, J.R., Süß, J.G., Visser, E.: Robust real-time synchronization between textual and graphical editors. In: Theory and Practice of Model Transformations. Springer, pp. 92–107 (2013)Google Scholar
  92. 92.
    Vergu, V., Neron, P., Visser, E.: DynSem: A DSL for dynamic semantics specification. In: LIPIcs-Leibniz International Proceedings in Informatics, vol. 36. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2015)Google Scholar
  93. 93.
    Visser, E.: WebDSL: a case study in domain-specific language engineering. In: Generative and Transformational Techniques in Software Engineering II, International Summer School, GTTSE 2007, LNCS (2007)Google Scholar
  94. 94.
    Visser, E.: Program transformation with Stratego/XT. In: Domain-Specific Program Generation. Springer, pp. 216–238 (2004)Google Scholar
  95. 95.
    Voelter, M.: Embedded software development with projectional language workbenches. In: MODELS 2010, Lecture Notes in Computer Science. Springer (2010)Google Scholar
  96. 96.
    M. Voelter. Language and IDE development, modularization and composition with MPS. In: GTTSE 2011, LNCS. Springer (2011)Google Scholar
  97. 97.
    Voelter, M.: Integrating prose as first-class citizens with models and code. In: 7th International Workshop on Multi-Paradigm Modeling MPM 2013 (2013)Google Scholar
  98. 98.
    Voelter, M.: Generic Tools, Specific Languages. Delft University of Technology, Delft (2014)Google Scholar
  99. 99.
    Voelter, M., Lisson, S.: Supporting diverse notations in MPS’ projectional editor. In: Proceedings of the GEMOC Workshop (2014)Google Scholar
  100. 100.
    Voelter, M., Ratiu, D., Schaetz, B., Kolb, B.: mbeddr: an extensible C-based programming language and IDE for embedded systems. In: Proceedings of the 3rd Annual Conference on Systems, Programming, and Applications: Software for Humanity. ACM (2012)Google Scholar
  101. 101.
    Voelter, M., Benz, S., Dietrich, C., Engelmann, B., Helander, M., Kats, L., Visser, E., Wachsmuth, G.: DSL Engineering. dslbook.org (2013)Google Scholar
  102. 102.
    Voelter, M., Ratiu, D., Kolb, B., Schaetz, B.: mbeddr: instantiating a language workbench in the embedded software domain. Autom. Softw. Eng. 20(3), 339–390 (2013)CrossRefGoogle Scholar
  103. 103.
    Voelter, M., Ratiu, D., Tomassetti, F.: Requirements as first-class citizens: integrating requirements closely with implementation artifacts. In: ACESMB@MoDELS (2013)Google Scholar
  104. 104.
    Voelter, M., Siegmund, J., Berger, T., Kolb, B.: Towards user-friendly projectional editors. In: 7th International Conference on Software Language Engineering (SLE) (2014)Google Scholar
  105. 105.
    Voelter, M., Kolb, B., Warmer, J.: Projecting a modular future. IEEE Softw. 32(5), 46–52 (2015)CrossRefGoogle Scholar
  106. 106.
    Voelter, M., van Deursen, A., Kolb, B., Eberle, S.: Using C language extensions for developing embedded software: a case study. In: OOPSLA 2015 (2015)Google Scholar
  107. 107.
    Voelter, M., Szabo, T., Lisson, S., Kolb, B., Erdweg, S., Berger, T.: Efficient development of consistent projectional editors using grammar cells. In: Proceedings of the 9th Conference on Software Language Engineering (SLE) (2016)Google Scholar
  108. 108.
    Winkler, S., Pilgrim, J.: A survey of traceability in requirements engineering and model-driven development. Softw. Syst. Model. 9, 529–565 (2010)CrossRefGoogle Scholar
  109. 109.
    Wu, H., Gray, J., Roychoudhury, S., Mernik, M.: Weaving a debugging aspect into domain-specific language grammars. In: Proceedings of the 2005 ACM Symposium on Applied Computing. ACM, pp. 1370–1374 (2005)Google Scholar
  110. 110.
    Yin, R.K.: Case Study Research: Design and Methods. Sage, Thousand Oaks (2014)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Markus Voelter
    • 1
  • Bernd Kolb
    • 2
  • Tamás Szabó
    • 2
    • 3
  • Daniel Ratiu
    • 4
  • Arie van Deursen
    • 3
  1. 1.independent/itemisStuttgartGermany
  2. 2.itemis AGStuttgartGermany
  3. 3.Delft University of TechnologyDelftThe Netherlands
  4. 4.Siemens AGMunichGermany

Personalised recommendations