Software & Systems Modeling

, Volume 18, Issue 2, pp 1305–1344 | Cite as

End-to-end model-transformation comprehension through fine-grained traceability information

  • Victor GuanaEmail author
  • Eleni Stroulia
Regular Paper


The construction and maintenance of model-to-model and model-to-text transformations pose numerous challenges to novice and expert developers. A key challenge involves tracing dependency relationships between artifacts of a transformation ecosystem. This is required to assess the impact of metamodel evolution, to determine metamodel coverage, and to debug complex transformation expressions. This paper presents an empirical study that investigates the performance of developers reflecting on the execution semantics of model-to-model and model-to-text transformations. We measured the accuracy and efficiency of 25 developers completing a variety of traceability-driven tasks in two model-based code generators. We compared the performance of developers using ChainTracker, a traceability analysis environment developed by our team, and that of developers using Eclipse Modeling. We present statistically significant evidence that ChainTracker improves the performance of developers reflecting on the execution semantics of transformation ecosystems. We discuss how developers supported by off-the-shelf development environments are unable to effectively identify dependency relationships in nontrivial model-transformation chains.


Model-transformation comprehension Transformation comprehension Traceability analysis Software maintenance Development environments 



This work was supported by The Killam Trust, NSERC (the Discovery and the IRC program), the GRAND NCE and IBM Canada. The development of PhyDSL was supported by the AGE-WELL NCE. We would like to thank Kelsey Gaboriau and Vina Nguyen for their support during the development of ChainTracker. We would also like to thank Kelly Garces for her generous help during the execution of this study.


  1. 1.
    Czarnecki, K.: Generative programming: methods, techniques, and applications tutorial abstract. In: Software Reuse: Methods, Techniques, and Tools, pp. 477–503 (2002)Google Scholar
  2. 2.
    Hemel, Z., Kats, L.C., Visser, E.: Code generation by model transformation, pp. 183–198 (2008)Google Scholar
  3. 3.
    Vara, J.M., Vela, B., Cavero, J.M., Marcos, E.: Model transformation for object-relational database development. In: Proceedings of the 2007 ACM Symposium on Applied Computing. ACM, pp. 1012–1019 (2007)Google Scholar
  4. 4.
    Vara, J.M., Marcos, E.: A framework for model-driven development of information systems: technical decisions and lessons learned. J. Syst. Softw. 85(10), 2368–2384 (2012)CrossRefGoogle Scholar
  5. 5.
    Csertán, G., Huszerl, G., Majzik, I., Pap, Z., Pataricza, A., Varró, D.: VIATRA-visual automated transformations for formal verification and validation of UML models. In: Automated Software Engineering, 2002. Proceedings. ASE 2002. 17th IEEE International Conference on. IEEE, pp. 267–270 (2002)Google Scholar
  6. 6.
    Cariou, E., Ballagny, C., Feugas, A., Barbier, F.: Contracts for model execution verification. In: European Conference on Modelling Foundations and Applications, pp. 3–18. Springer, Berlin (2011)Google Scholar
  7. 7.
    Selim, G.M., Wang, S., Cordy, J.R., Dingel, J.: Model transformations for migrating legacy deployment models in the automotive industry. Softw. Syst. Model. 14(1), 365–381 (2015)CrossRefGoogle Scholar
  8. 8.
    Kavimandan, A., Gokhale, A.: Applying model transformations to optimizing real-time QOS configurations in DRE systems. In: International Conference on the Quality of Software Architectures, pp. 18–35. Springer, Berlin (2009)Google Scholar
  9. 9.
    France, R., Rumpe, B.: Model-driven development of complex software: a research roadmap. In: 2007 Future of Software Engineering. IEEE Computer Society, pp. 37–54 (2007)Google Scholar
  10. 10.
    Hutchinson, J., Whittle, J., Rouncefield, M., Kristoffersen, S.: Empirical assessment of MDE in industry. In: Proceedings of the 33rd International Conference on Software Engineering, ACM, pp. 471–480 (2011)Google Scholar
  11. 11.
    Di Ruscio, D., Iovino, L., Pierantonio, A.: Evolutionary togetherness: how to manage coupled evolution in metamodeling ecosystems. In: International Conference on Graph Transformation, pp. 20–37. Springer, Berlin (2012)Google Scholar
  12. 12.
    Walderhaug, S., Stav, E., Johansen, U., Olsen, G.K.: Traceability in model-driven software development. In: Designing Software-Intensive Systems: Methods and Principle, pp. 133–159 (2008)Google Scholar
  13. 13.
    Von Knethen, A., Grund, M.: Quatrace: a tool environment for (semi-) automatic impact analysis based on traces. In: Software Maintenance, 2003. ICSM 2003. Proceedings of International Conference on. IEEE, pp. 246–255 (2003)Google Scholar
  14. 14.
    Winkler, S., Pilgrim, J.: A survey of traceability in requirements engineering and model-driven development. Softw. Syst. Model. 9(4), 529–565 (2010)CrossRefGoogle Scholar
  15. 15.
    Amar, B., Leblanc, H., Coulette, B.: A traceability engine dedicated to model transformation for software engineering. In: ECMDA Traceability Workshop (ECMDA-TW), pp. 7–16 (2008)Google Scholar
  16. 16.
    Correa, A., Werner, C.: Applying refactoring techniques to UML/OCL models. In:<< UML>> 2004-The Unified Modeling Language. Modelling Languages and Applications. Springer, Berlin, pp. 173–187 (2004)Google Scholar
  17. 17.
    Ermel, C., Ehrig, H., Ehrig, K.: Refactoring of model transformations. In: Electronic Communications of the EASST, vol. 18 (2009)Google Scholar
  18. 18.
    Wang, J., Kim, S.-K., Carrington, D.: Verifying metamodel coverage of model transformations. In: Software Engineering Conference, 2006, Australian. IEEE, p. 10 (2006)Google Scholar
  19. 19.
    Wang, J., Kim, S., Carrington, D.: Automatic generation of test models for model transformations. In: 19th Australian Conference on Software Engineering (ASWEC 2008). IEEE, pp. 432–440 (2008)Google Scholar
  20. 20.
    Finot, O., Mottu, J.-M., Sunyé, G., Degueule, T.: Using meta-model coverage to qualify test oracles. In: Analysis of Model Transformations, pp. 1613–0073 (2013)Google Scholar
  21. 21.
    Burgueno, L.: Testing M2M/M2T/T2M transformations. In: Model Driven Engineering Languages and Systems (MODELS), 2015 ACM/IEEE 18th International Conference on (Student Competition) (2015)Google Scholar
  22. 22.
    Van Amstel, M.F., Van Den Brand, M.G.: Model transformation analysis: staying ahead of the maintenance nightmare, pp. 108–122 (2011)Google Scholar
  23. 23.
    Kleppe, A.: First european workshop on composition of model transformations-cmt 2006. In: Technical Report TR-CTIT-06-34 (2006)Google Scholar
  24. 24.
    Kemerer, C.F.: Now the learning curve affects case tool adoption. IEEE Softw. 9(3), 23–28 (1992)CrossRefGoogle Scholar
  25. 25.
    Hardgrave, B.C., Davis, F.D., Riemenschneider, C.K.: Investigating determinants of software developers’ intentions to follow methodologies. J. Manag. Inf. Syst. 20(1), 123–151 (2003)CrossRefGoogle Scholar
  26. 26.
    Whittle, J., Hutchinson, J., Rouncefield, M., Burden, H., Heldal, R.: Industrial adoption of model-driven engineering: are the tools really the problem? In: Model-Driven Engineering Languages and Systems, pp. 1–17. Springer, Berlin (2013)Google Scholar
  27. 27.
    Guana, V., Gaboriau, K., Stroulia, E.: Chaintracker: towards a comprehensive tool for building code-generation environments. In: Proceedings of the 2014 International Conference on Software Maintenance and Evolution (ICSME). IEEE Press (2014)Google Scholar
  28. 28.
    Guana, V., Stroulia, E.: Chaintracker, a model-transformation trace analysis tool for code-generation environments. In: Theory and Practice of Model Transformations, pp. 146–153. Springer, Berlin (2014)Google Scholar
  29. 29.
    Guana, V., Stroulia, E.: Reflecting on model-based code generators using traceability information. In: ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS) (2015)Google Scholar
  30. 30.
    Warmer, J., Kleppe, A.: The Object Constraint Language: Getting Your Models Ready for MDA. Addison-Wesley Professional, Reading (2003)Google Scholar
  31. 31.
    Jouault, F., Kurtev, I.: Transforming models with ATL. In: Satellite Events at the MoDELS 2005 Conference, pp. 128–138. Springer, BerlinGoogle Scholar
  32. 32.
    Musset, J., Juliot, É., Lacrampe, S., Piers, W., Brun, C., Goubet, L., Lussaud, Y., Allilaire, F.: Acceleo User Guide (2006)Google Scholar
  33. 33.
    Bragança, A., Machado, R.J.: Transformation patterns for multi-staged model driven software development. In: Software Product Line Conference, 2008. SPLC’08, 12th International. IEEE, pp. 329–338 (2008)Google Scholar
  34. 34.
    Czarnecki, K., Helsen, S.: Classification of model transformation approaches. In: Proceedings of the 2nd OOPSLA Workshop on Generative Techniques in the Context of the Model Driven Architecture, vol. 45, no. 3. Citeseer, pp. 1–17 (2003)Google Scholar
  35. 35.
    Van Deursen, A., Visser, E., Warmer, J.: Model-driven software evolution: a research agenda. In: Proceedings 1st International Workshop on Model-Driven Software Evolution, pp. 41–49 (2007)Google Scholar
  36. 36.
    Bennett, K., Rajlich, V.: Software maintenance and evolution: a roadmap. In: Proceedings of the Conference on the Future of Software Engineering. ACM, pp. 73–87 (2000)Google Scholar
  37. 37.
    Di Ruscio, D., Iovino, L., Pierantonio, A.: A methodological approach for the coupled evolution of metamodels and atl transformations. In: International Conference on Theory and Practice of Model Transformations, pp. 60–75. Springer, Berlin (2013)Google Scholar
  38. 38.
    Di Rocco, J., Di Ruscio, D., Iovino, L., Pierantonio, A.: Dealing with the coupled evolution of metamodels and model-to-text transformations. In: Alfonso Pierantonio (co-chair) Universita degli Studi dellâĂŹAquila (Italy) Bernhard Schätz (co-chair) fortiss GmbH (Germany), p. 22 (2014)Google Scholar
  39. 39.
    Group, O.M.: A proposal for an MDA foundation model. Needham ormsc/05-04-01 ed (2005)Google Scholar
  40. 40.
    Aizenbud-Reshef, N., Nolan, B.T., Rubin, J., Shaham-Gafni, Y.: Model traceability. IBM Syst. J. 45(3), 515–526 (2006)CrossRefGoogle Scholar
  41. 41.
    Galvao, I., Goknil, A.: Survey of traceability approaches in model-driven engineering. In: Enterprise Distributed Object Computing Conference, 2007. EDOC 2007. 11th IEEE International. IEEE, pp. 313–313 (2007)Google Scholar
  42. 42.
    Santiago, I., Jiménez, A., Vara, J.M., De Castro, V., Bollati, V.A., Marcos, E.: Model-driven engineering as a new landscape for traceability management: a systematic literature review. Inf. Softw. Technol. 54(12), 1340–1356 (2012)CrossRefGoogle Scholar
  43. 43.
    Falleri, J., Huchard, M., Nebut, C. et al.: Towards a traceability framework for model transformations in kermeta (2006)Google Scholar
  44. 44.
    Jouault, F.: Loosely coupled traceability for atl. In: Proceedings of the European Conference on Model Driven Architecture (ECMDA) Workshop on Traceability, Nuremberg, Germany, vol. 91. Citeseer (2005)Google Scholar
  45. 45.
    von Pilgrim, J., Vanhooff, B., Schulz-Gerlach, I., Berbers, Y.: Constructing and visualizing transformation chains. In: Model Driven Architecture–Foundations and Applications. Springer, Berlin (2008)Google Scholar
  46. 46.
    Matragkas, N.D., Kolovos, D.S., Paige, R.F., Zolotas, A.: A traceability-driven approach to model transformation testing. In: AMT@MoDELS (2013)Google Scholar
  47. 47.
    Santiago, I., Vara, J.M., de Castro, M.V., Marcos, E.: Towards the effective use of traceability in model-driven engineering projects. In: Conceptual Modeling, pp. 429–437. Springer, Berlin (2013)Google Scholar
  48. 48.
    Di Rocco, J., Di Ruscio, D., Iovino, L., Pierantonio, A.: Traceability visualization in metamodel change impact detection. In: Proceedings of the Second Workshop on Graphical Modeling Language Development. ACM, pp. 51–62 (2013)Google Scholar
  49. 49.
    Di Ruscio, D., Iovino, L., Pierantonio, A.: Managing the coupled evolution of metamodels and textual concrete syntax specifications. In: Software Engineering and Advanced Applications (SEAA), 2013 39th EUROMICRO Conference on. IEEE, pp. 114–121 (2013)Google Scholar
  50. 50.
    van Amstel, M., Serebrenik, A., van den Brand, M.: Visualizing traceability in model transformation compositions. In: Pre-Proceedings of the First Workshop on Composition and Evolution of Model Transformations (2011)Google Scholar
  51. 51.
    Oldevik, J., Neple, T.: Traceability in model to text transformations. In: 2nd ECMDA Traceability Workshop (ECMDA-TW). Citeseer, pp. 17–26 (2006)Google Scholar
  52. 52.
    García, J., Azanza, M., Irastorza, A., Díaz, O.: Testing MOFscript transformations with HandyMOF. In: Theory and Practice of Model Transformations, pp. 42–56. Springer, Berlin (2014)Google Scholar
  53. 53.
    Olsen, G.K., Oldevik, J.: Scenarios of traceability in model to text transformations. In: European Conference on Model Driven Architecture-Foundations and Applications, pp. 144–156. Springer, Berlin (2007)Google Scholar
  54. 54.
    Santiago, I., Vara, J.M., de Castro, V., Marcos, E.: Reducing the level of complexity of working with model transformations. In: International Conference on Evaluation of Novel Approaches to Software Engineering, pp. 1–17. Springer, Berlin (2014)Google Scholar
  55. 55.
    Wieringa, R.: An introduction to requirements traceability (1995)Google Scholar
  56. 56.
    Almeida, J., Van Eck, P., Iacob, M.: Requirements traceability and transformation conformance in model-driven development. In: Enterprise Distributed Object Computing Conference, 2006. EDOC’06. 10th IEEE International. IEEE, pp. 355–366 (2006)Google Scholar
  57. 57.
    Pinheiro, F.A.: Requirements traceability. In: Perspectives on Software Requirements. Springer, Berlin, pp. 91–113 (2004)Google Scholar
  58. 58.
    Duan, C., Cleland-Huang, J.: Visualization and analysis in automated trace retrieval. In: 2006 First International Workshop on Requirements Engineering Visualization (REV’06-RE’06 Workshop). IEEE, p. 5 (2006)Google Scholar
  59. 59.
    Card, D.N.: Designing software for producibility. J. Syst. Softw. 17(3), 219–225 (1992)CrossRefGoogle Scholar
  60. 60.
    Cleland-Huang, J.: Toward improved traceability of non-functional requirements. In: Proceedings of the 3rd International Workshop on Traceability in Emerging Forms of Software Engineering. ACM, pp. 14–19 (2005)Google Scholar
  61. 61.
    Acceleo traceability: Eclipse plug-in, Accessed 14 Sept 2016
  62. 62.
    Atlas transformation language (atl) user guide, Accessed 14 Sept 2016
  63. 63.
    Cleland-Huang, J., Gotel, O.C., Huffman Hayes, J., Mäder, P., Zisman, A.: Software traceability: trends and future directions. In: Proceedings of the on Future of Software Engineering. ACM, pp. 55–69 (2014)Google Scholar
  64. 64.
    Cuadrado, J., Molina, J., Tortosa, M.: Rubytl: a practical, extensible transformation language. In: Model Driven Architecture–Foundations and Applications, pp. 158–172. Springer, Berlin (2006)Google Scholar
  65. 65.
    Kolovos, D., Paige, R., Polack, F.: The epsilon transformation language. In: Theory and Practice of Model Transformations, pp. 46–60 (2008)Google Scholar
  66. 66.
    T. project (IRISA), The Metamodeling Language Kermeta. (2006)
  67. 67.
    Ellson, J., Gansner, E., Koutsofios, L., North, S.C., Woodhull, G.: GraphvizâĂŤopen source graph drawing tools. In: Graph Drawing, pp. 483–484. Springer, Berlin (2002)Google Scholar
  68. 68.
    van Amstel, M.F., van den Brand, M.G., Serebrenik, A.: Traceability visualization in model transformations with tracevis. In: International Conference on Theory and Practice of Model Transformations, pp. 152–159. Springer, Berlin (2012)Google Scholar
  69. 69.
    Von Pilgrim, J., Duske, K., McIntosh, P.: Eclipse GEF3D: bringing 3D to existing 2D editors. Information Visualization 8(2), 107–119 (2009)CrossRefGoogle Scholar
  70. 70.
    Kolovos, D.S., Rose, L.M., Matragkas, N., Paige, R.F., Guerra, E., Cuadrado, J.S., De Lara, J., Ráth, I., Varró, D., Tisi, M. et al.: A research roadmap towards achieving scalability in model driven engineering. In: Proceedings of the Workshop on Scalability in Model Driven Engineering. ACM, p. 2 (2013)Google Scholar
  71. 71.
    Wettel, R., Lanza, M.: Program comprehension through software habitability. In: 15th IEEE International Conference on Program Comprehension (ICPC’07). IEEE, pp. 231–240 (2007)Google Scholar
  72. 72.
    Störrle, H.: On the impact of size to the understanding of UML diagrams. Softw. Syst. Model., pp. 1–20 (2016)Google Scholar
  73. 73.
    van Ravensteijn, W.: Visual traceability across dynamic ordered hierarchies (2011)Google Scholar
  74. 74.
    Holten, D.: Hierarchical edge bundles: visualization of adjacency relations in hierarchical data. IEEE Trans. Vis. Comput. Gr. 12(5), 741–748 (2006)CrossRefGoogle Scholar
  75. 75.
    Clements, P., Garlan, D., Bass, L., Stafford, J., Nord, R., Ivers, J., Little, R.: Documenting Software Architectures: Views and Beyond. Pearson Education (2002)Google Scholar
  76. 76.
    Soni, D., Nord, R. L., Hofmeister, C.: Software architecture in industrial applications. In: Software Engineering, 1995. ICSE 1995. 17th International Conference on. IEEE, pp. 196–196 (1995)Google Scholar
  77. 77.
    Kleppe, A.: First european workshop on composition of model transformations-cmt 2006 (2006)Google Scholar
  78. 78.
    Wegman, E.J.: Hyperdimensional data analysis using parallel coordinates. J. Am. Stat. Assoc. 85(411), 664–675 (1990)CrossRefGoogle Scholar
  79. 79.
    Cariou, E., Marvie, R., Seinturier, L., Duchien, L.: Ocl for the specification of model transformation contracts. In: OCL and Model Driven Engineering, UML 2004 Conference Workshop, vol. 12, pp. 69–83 (2004)Google Scholar
  80. 80.
    Inselberg, A., Dimsdale, B.: Parallel coordinates: a tool for visualizing multi-dimensional geometry. In: Proceedings of the 1st conference on Visualization’90. IEEE Computer Society Press, pp. 361–378 (1990)Google Scholar
  81. 81.
    Myers, B.A., Ko, A.J., LaToza, T.D., Yoon, Y.: Programmers are users too: Human-centered methods for improving programming tools. Computer 49(7), 44–52 (2016)CrossRefGoogle Scholar
  82. 82.
    Guana, V., Stroulia, E.: Phydsl: a code-generation environment for 2d physics-based games. In: 2014 IEEE Games, Entertainment, and Media Conference (IEEE GEM) (2014)Google Scholar
  83. 83.
    Guana, V., Stroulia, E., Nguyen, V.: Building a game engine: a tale of modern model-driven engineeringGoogle Scholar
  84. 84.
    Tong, T., Guana, V., Jovanovic, A., Tran, F., Mozafari, G., Chignell, M., Stroulia, E.: Rapid deployment and evaluation of mobile serious games: a cognitive assessment case study. Procedia Comput. Sci. 69, 96–103 (2015)CrossRefGoogle Scholar
  85. 85.
    Ricca, F., Di Penta, M., Torchiano, M., Tonella, P., Ceccato, M., Visaggio, C.A.: Are fit tables really talking? In: 2008 ACM/IEEE 30th International Conference on Software Engineering. IEEE, pp. 361–370 (2008)Google Scholar
  86. 86.
    Ricca, F., Leotta, M., Reggio, G., Tiso, A., Guerrini, G., Torchiano, M.: Using unimod for maintenance tasks: an experimental assessment in the context of model driven development. In: Proceedings of the 4th International Workshop on Modeling in Software Engineering. IEEE Press, pp. 77–83 (2012)Google Scholar
  87. 87.
    Guana, V., Stroulia, E.: How do developers solve software-engineering tasks on model-based code generators? an empirical study design. In: First International Workshop on Human Factors in Modeling (HuFaMo 2015). CEUR-WS, pp. 33–38 (2015)Google Scholar
  88. 88.
    Burkhardt, J.-M., Détienne, F., Wiedenbeck, S.: Object-oriented program comprehension: effect of expertise, task and phase. Empir. Softw. Eng. 7(2), 115–156 (2002)CrossRefzbMATHGoogle Scholar
  89. 89.
    Hermans, F., Aivaloglou, E.: Do code smells hamper novice programming? a controlled experiment on scratch programs. In: Program Comprehension (ICPC), 2016 IEEE 24th International Conference on. IEEE, pp. 1–10 (2016)Google Scholar
  90. 90.
    Gravino, C., Risi, M., Scanniello, G., Tortora, G.: Do professional developers benefit from design pattern documentation? A replication in the context of source code comprehension. In: International Conference on Model Driven Engineering Languages and Systems, pp. 185–201. Springer, Berlin (2012)Google Scholar
  91. 91.
    OMG: MOF model to text transformation language (mofm2t), 1.0 (2008)Google Scholar
  92. 92.
    OMG: Meta object facility (mof) 2.0 query/view/transformation (qvt) (2015)Google Scholar
  93. 93.
    Di Ruscio, D., Kolovos, D., Matragkas, N.: Scalability in model driven engineering: Bigmde’13 workshop summary. In: Proceedings of the Workshop on Scalability in Model Driven Engineering, ACM, p. 1 (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.Department of Computing ScienceUniversity of AlbertaEdmontonCanada

Personalised recommendations