Skip to main content
SpringerLink
Log in

A benchmark of incremental model transformation tools based on an industrial case study with AADL

  • Regular Paper
  • Published:
Software and Systems Modeling Aims and scope Submit manuscript

Abstract

Incremental model transformation (IMT) tools have been proposed to improve performances of model transformations by updating only the parts of a model that need to be changed when another model on which it depends has been changed. Yet, the question is how these tools are suitable for modeling large and complex systems with rich modeling languages as used in industry. In this paper, we report the results of a benchmark of the most mature IMT tools. Particularly, we benchmark MoTE, eMoflon, VIATRA and YAMTL to evaluate their usability, maintainability and runtime performances. Our benchmark is based on a model transformation of an industrial case study using the standard architecture description language AADL. We propose a reusable evaluation framework, available for tool developers and users. Besides the capability to process large models, our benchmark also assesses IMT tool performances according to the different kinds of complex structures that typically exist in models of rich languages, as well as the complexity of the transformation specifications. Our results show the promising potential of some tools to specify sophisticated specifications and transform large models with good performance, but their use still requires the help of their developers, especially to solve serious problems with their runtime performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Notes

  1. https://osate.org/.

  2. A contest for users and developers of model transformation tools (e.g., TTC 2020 http://www.transformation-tool-contest.eu/2020/).

  3. https://www.hpi.uni-potsdam.de/giese/public/mdelab/mdelab-projects/mote-a-tgg-based-model-transformation-engine/.

  4. While the MoTE source repository is private, all its sources are released and automatically installed with the tool plugins.

  5. MoTE2 TGG Model Transformation Engine 1.0.0.

  6. https://emoflon.org/.

  7. https://github.com/eMoflon.

  8. https://paper.dropbox.com/doc/Meta-Modelling-with-eMoflonCore-zyOqELGZ0X9jL85TAs7pf.

  9. eMoflon::IBeX (Democles) 1.0.0.

  10. https://github.com/viatra.

  11. https://www.eclipse.org/viatra/documentation/tutorial.html.

  12. https://www.eclipse.org/xtend/.

  13. VIATRA Transformation Runtime 2.3.1.

  14. https://yamtl.github.io/.

  15. https://github.com/yamtl/examples.

  16. YAMTL 0.0.10.

  17. https://github.com/eMoflon/emoflon-ibex-examples.

  18. https://paper.dropbox.com/doc/Meta-Modelling-with-eMoflonCore-zyOqELGZ0X9jL85TAs7pf.

  19. https://gitlab.telecom-paris.fr/mbe-tools/aadl-mt-bench.

  20. ATL Transformation Language SDK (4.2.0) with default EMF VM

  21. This model provided by an industrial partner cannot be disclosed for confidentiality reasons.

References

  1. Actifsource. http://www.actifsource.com/. Accessed: 2021-04-10

  2. Mdworkbench. https://www.sodiuswillert.com/en/products/modeldev-tools/mdworkbench. Accessed: 2021-04-10

  3. Mote. https://www.hpi.uni-potsdam.de/giese/public/mdelab/mdelab-projects/mote-a-tgg-based-model-transformation-engine/. Accessed: 2021-04-10

  4. Qvtd. https://projects.eclipse.org/projects/modeling.mmt.qvtd. Accessed: 2021-04-10

  5. Viatra cps benchmark. https://github.com/viatra/viatra-cps-benchmark. Accessed: 2021-05-10

  6. Anjorin, A., Buchmann, T., Westfechtel, B., Diskin, Z., Ko, H-.S., Eramo, R., Hinkel, G., Samimi-Dehkordi, L., Zündorf, A.: Benchmarking bidirectional transformations: theory, implementation, application, and assessment. Software and Systems Modeling, pp. 1–45 (2019)

  7. Anjorin, A., Lauder, M., Patzina, S., Schürr, A.: Emoflon: leveraging emf and professional case tools. In: GI-Jahrestagung, p. 281 (2011)

  8. Arendt, T., Biermann, E., Jurack, S., Krause, C., Taentzer, G.: Henshin: advanced concepts and tools for in-place emf model transformations. In: International Conference on Model Driven Engineering Languages and Systems, pp. 121–135. Springer (2010)

  9. Baki, I., Sahraoui, H.: Multi-step learning and adaptive search for learning complex model transformations from examples. ACM Trans. Softw. Eng. Methodol. (TOSEM) 25(3), 1–37 (2016)

    Article  Google Scholar 

  10. Barisic, A., Amaral, V., Goulao, M., Aguiar, A.: Introducing usability concerns early in the DSL Development Cycle: FlowSL experience report. In: MD2P2@ MoDELS, pp. 8–17. Citeseer (2014)

  11. Bergmann, G.: Incremental model queries in model-driven design (2013)

  12. Bergmann, G., Dávid, I., Hegedüs, Á., Horváth, Á., Ráth, I., Ujhelyi, Z., Varró, D.: Viatra 3: a reactive model transformation platform. In: International Conference on Theory and Practice of Model Transformations, pp. 101–110. Springer (2015)

  13. Bergmann, G., Horváth, Á., Ráth, I., Varró, D., Balogh, A., Balogh, Z., Ökrös, A.: Incremental evaluation of model queries over EMF models. In: International conference on model driven engineering languages and systems, pp. 76–90. Springer (2010)

  14. Biehl, M.: Literature study on model transformations. Royal Institute of Technology, Tech. Rep. ISRN/KTH/MMK, 291 (2010)

  15. Bodeveix, J.-P., Filali, M., Garnacho, M., Spadotti, R., Yang, Z.: Towards a verified transformation from AADL to the formal component-based language FIACRE. Sci. Comput. Program. 106, 30–53 (2015)

    Article  Google Scholar 

  16. Borde, E., Rahmoun, S., Cadoret, F., Pautet, L., Singhoff, F., Dissaux, P.: Architecture models refinement for fine grain timing analysis of embedded systems. In: 2014 25nd IEEE International Symposium On Rapid System Prototyping, pp. 44–50. IEEE (2014)

  17. Boronat, A.: Expressive and efficient model transformation with an internal dsl of xtend. In: Proceedings of the 21th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, pp. 78–88 (2018)

  18. Boronat, A.: Incremental execution of rule-based model transformation. Int. J. Softw. Tools Technol. Transf, pp. 1–23 (2020)

  19. Boydston, A., Feiler, P., Vestal, S., Lewis, B.: Architecture centric virtual integration process (ACVIP): a key component of the DoD digital engineering strategy. Technical report, Software Engineering Institute, (2019)

  20. Brun, C., Pierantonio, A.: Model differences in the eclipse modeling framework. UPGRADE Eur. J. Inf. Profess. 9(2), 29–34 (2008)

    Google Scholar 

  21. Bucchiarone, A., Cabot, J., Paige, R..F., Pierantonio, A.: Grand challenges in model-driven engineering: an analysis of the state of the research. Softw. Syst. Model. 19(1), 5–13 (2020)

    Article  Google Scholar 

  22. Cicchetti, A., D. Di Ruscio, Eramo, R., Pierantonio, A.: Jtl: a bidirectional and change propagating transformation language. In: International Conference on Software Language Engineering, pp. 183–202. Springer (2010)

  23. Czarnecki, K., Helsen, S.: Feature-based survey of model transformation approaches. IBM Syst. J. 45(3), 621–645 (2006)

    Article  Google Scholar 

  24. Debruyne, V., Simonot-Lion, F., Trinquet, Y.: East-adl—an architecture description language. In: IFIP World Computer Congress, TC 2, pp. 181–195. Springer (2004)

  25. Fischer, T., Niere, J., Torunski, L., Zündorf, A.: Story diagrams: a new graph rewrite language based on the unified modeling language and java. In: International Workshop on Theory and Application of Graph Transformations, pp. 296–309. Springer (1998)

  26. Forgy, C.L.: Rete: a fast algorithm for the many pattern/many object pattern match problem. In: Readings in Artificial Intelligence and Databases, pp. 547–559. Elsevier (1982)

  27. Greenyer, J., Rieke, J.: Applying advanced tgg concepts for a complex transformation of sequence diagram specifications to timed game automata. In: International Symposium on Applications of Graph Transformations with Industrial Relevance, pp. 222–237. Springer (2011)

  28. Hause, M., et al.: The sysml modelling language. In: Fifteenth European Systems Engineering Conference, vol. 9, pp. 1–12 (2006)

  29. Hidaka, S., Tisi, M., Cabot, J., Hu, Z.: Feature-based classification of bidirectional transformation approaches. Softw. Syst. Model. 15(3), 907–928 (2016)

    Article  Google Scholar 

  30. Hinkel, G., Goldschmidt, T., Burger, E., Reussner, R.: Using internal domain-specific languages to inherit tool support and modularity for model transformations. Softw. Syst. Model. 18(1), 129–155 (2019)

    Article  Google Scholar 

  31. Jakumeit, E., Buchwald, S., Kroll, M.: Grgen. net. Int. J. Softw. Tools Technol. Transf. 12(3), 263–271 (2010)

    Article  Google Scholar 

  32. Jakumeit, E., Buchwald, S., Wagelaar, D., Dan, L., Á. Hegedüs, M. Herrmannsdörfer, Horn, T., Kalnina, E., Krause, C., Lano, K., et al. (2014) A survey and comparison of transformation tools based on the transformation tool contest. Science of computer programming, 85:41–99

  33. Jouault, F., Allilaire, F., Bézivin, J., Kurtev, I.: ATL: a model transformation tool. Sci. Comput. Program. 72(1–2), 31–39 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  34. Kahani, N., Bagherzadeh, M., J. Cordy, R., Dingel, J., Varró, D.: Survey and classification of model transformation tools. Softw. Syst. Model. 1–37 (2018)

  35. Klassen, L., Wagner, R.: Emorf-a tool for model transformations. Electron. Commun. EASST, 54 (2012)

  36. Kleppe, A.G., Warmer, J., Warmer, J.B., Bast, W.: MDA Explained: The Model Driven Architecture: Practice and Promise. Addison-Wesley Professional, Boston (2003)

    MATH  Google Scholar 

  37. Kolovos, D.S., Paige, R.F., Polack, F.A.: The epsilon transformation language. In: International Conference on Theory and Practice of Model Transformations, pp. 46–60. Springer (2008)

  38. Lano, K., Kolahdouz-Rahimi, S.: Constraint-based specification of model transformations. J. Syst. Softw. 86(2), 412–436 (2013)

    Article  Google Scholar 

  39. Lano, K., Kolahdouz-Rahimi, S., Yassipour-Tehrani, S., Sharbaf, M.: A survey of model transformation design pattern usage. In: International Conference on Theory and Practice of Model Transformations, pp. 108–118. Springer (2017)

  40. Le Calvar, T., Jouault, F., Chhel, F., Clavreul, M.: Efficient atl incremental transformations. J. Object Technol. 18(3), 2 (2019)

    Article  Google Scholar 

  41. Leblebici, E., Anjorin, A., Schürr, A.: Developing emoflon with emoflon. In: Theory and Practice of Model Transformations, pp. 138–145. Springer, Cham (2014)

  42. Leblebici, E., Anjorin, A., Schürr, A., Hildebrandt, S., Rieke, J., Greenyer, J.: A comparison of incremental triple graph grammar tools. Electron. Commun. EASST 67 (2014)

  43. Malavolta, I., Lago, P., Muccini, H., Pelliccione, P., Tang, A.: What industry needs from architectural languages: a survey. IEEE Trans. Softw. Eng. 39(6), 869–891 (2012)

    Article  Google Scholar 

  44. Martínez, S., Tisi, M., Douence, R.: Reactive model transformation with atl. Sci. Comput. Program. 136, 1–16 (2017)

    Article  Google Scholar 

  45. Mkaouar, H., Zalila, B., Hugues, J., Jmaiel, M.: A Formal Approach to AADL Model-Based Software Engineering. Int. J. Softw. Tools Technol. Transf. 1–29 (2019)

  46. Nickel, U., Niere, J., Zündorf, A.: The Fujaba environment. In: Proceedings of the 22nd International Conference on Software Engineering, pp. 742–745 (2000)

  47. Nielsen, J.: Usability Engineering. Morgan Kaufmann, Burlington (1994)

    MATH  Google Scholar 

  48. Paz, F., Pow-Sang, J.A.: Current trends in usability evaluation methods: a systematic review. In: 2014 7th International Conference on Advanced Software Engineering and Its Applications, pp. 11–15. IEEE (2014)

  49. SAE. AS5506C: SAE Architecture Analysis and Design Language (AADL) AADL V2.2. (2017)

  50. Schalles, C., Creagh, J., Rebstock, M.: Usability of modelling languages for model interpretation: an empirical research report. In: 10th International Conference on Wirtschaftsinformatik, pp. 787–796. Zurich, Switzerland (2011)

  51. Schürr, A.: Specification of graph translators with triple graph grammars. In: International Workshop on Graph-Theoretic Concepts in Computer Science, pp. 151–163. Springer (1994)

  52. Steinberg, D., Budinsky, F., Merks, E., Paternostro, M.: EMF: Eclipse Modeling Framework. Pearson Education, London (2008)

    Google Scholar 

  53. Stevens, P.: A landscape of bidirectional model transformations. In: International Summer School on Generative and Transformational Techniques in Software Engineering, pp. 408–424. Springer (2007)

  54. Strüber, D., Kehrer, T., Arendt, T., Pietsch, C., Reuling, D.: Scalability of model transformations: position paper and benchmark set. In: BigMDE@STAF, pp. 21–30 (2016)

  55. Szárnyas, G., Izsó, B., Ráth, I., Varró, D.: The train benchmark: cross-technology performance evaluation of continuous model queries. Softw. Syst. Model. 17(4), 1365–1393 (2018)

    Article  Google Scholar 

  56. Varró, D., Bergmann, G., Hegedüs, Á., Horváth, Á., Ráth, I., Ujhelyi, Z.: Road to a reactive and incremental model transformation platform: three generations of the viatra framework. Softw. Syst. Model. 15(3), 609–629 (2016)

    Article  Google Scholar 

  57. Wagelaar, D., Tisi, M., Cabot, J., Jouault, F.: Towards a general composition semantics for rule-based model transformation. In: International Conference on Model Driven Engineering Languages and Systems, pp. 623–637. Springer (2011)

  58. Weidmann, N., Anjorin, A., Robrecht, P., Varró, G.: Incremental (unidirectional) model transformation with emoflon:: Ibex. In: International Conference on Graph Transformation, pp. 131–140. Springer (2019)

Download references

Acknowledgements

We would like to thank the eMoflon, VIATRA, ATL/ATOL and YAMTL developers (Anthony Anjorin, Lars Fritsche, Zoltan Ujhelyi, Gábor Bergmann, Frédéric Jouault and Artur Boronat) for their help with their tools.

Author information

Authors and Affiliations

  1. LTCI, Telecom Paris, Institut Polytechnique de Paris, Palaiseau, France

    Hana Mkaouar, Dominique Blouin & Etienne Borde

Authors
  1. Hana Mkaouar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dominique Blouin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Etienne Borde
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Communicated by Daniel Varro.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mkaouar, H., Blouin, D. & Borde, E. A benchmark of incremental model transformation tools based on an industrial case study with AADL. Softw Syst Model 22, 175–201 (2023). https://doi.org/10.1007/s10270-022-00989-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-022-00989-z

Keywords

Search

Navigation