Software & Systems Modeling

, Volume 14, Issue 2, pp 623–644 | Cite as

Specification-driven model transformation testing

  • Esther GuerraEmail author
  • Mathias Soeken
Special Section Paper


Testing model transformations poses several challenges, among them the automatic generation of appropriate input test models and the specification of oracle functions. Most approaches for the generation of input models ensure a certain coverage of the source meta-model or the transformation implementation code, whereas oracle functions are frequently defined using query or graph languages. However, these two tasks are usually performed independently regardless of their common purpose, and sometimes, there is a gap between the properties exhibited by the generated input models and those considered by the transformations. Recently, we proposed a formal specification language for the declarative formulation of transformation properties (by means of invariants, pre-, and postconditions) from which we generated partial oracle functions used for transformation testing. Here, we extend the usage of our specification language for the automated generation of input test models by SAT solving. The testing process becomes more intentional because the generated models ensure a certain coverage of the transformation requirements. Moreover, we use the same specification to consistently derive both the input test models and the oracle functions. A set of experiments is presented, aimed at measuring the efficacy of our technique.


Model transformation Model transformation specification Model transformation testing Model finding  Test oracle 



We thank the referees for their useful comments. This work has been sponsored by the Spanish Ministry of Science and Innovation with project “Go-Lite” (TIN2011-24139), by the R&D program of the Community of Madrid with project “e-Madrid” (S2009/TIC-1650), and by the German Research Foundation (DFG) within the Reinhart Koselleck project (DR 287/23-1).


  1. 1.
    Balogh, A., Bergmann, G., Csertán, G., Gönczy, L., Horváth, Á., Majzik, I., Pataricza, A., Polgár, B., Ráth, I., Varró, D., Varró, G.: Workflow-driven tool integration using model transformations. In: Graph Transformations and Model-Driven Engineering, vol. 5765 of LNCS, pp. 224–248. Springer, Berlin (2010)Google Scholar
  2. 2.
    Baudry, B., Ghosh, S., Fleurey, F., France, R.B., Traon, Y.L., Mottu, J.-M.: Barriers to systematic model transformation testing. CACM 53(6), 139–143 (2010)CrossRefGoogle Scholar
  3. 3.
    Boronat, A., Carsí, J.A., Ramos, I.: Algebraic specification of a model transformation engine. In: FASE’06, vol. 3922 of LNCS, pp. 262–277. Springer, Berlin (2006)Google Scholar
  4. 4.
    Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on Java predicates. In: ISSTA’02, pp. 123–133 (2002)Google Scholar
  5. 5.
  6. 6.
    Budd, T.A.: Mutation analysis: ideas, examples, problems and prospects. In: Proceedings of Summer School on Computer Program Testing, pp. 129–148 (1981)Google Scholar
  7. 7.
    Cabot, J., Clarisó, R., Guerra, E., de Lara, J.: Verification and validation of declarative model-to-model transformations through invariants. J. Syst. Softw. 83(2), 283–302 (2010)CrossRefGoogle Scholar
  8. 8.
    Cabot, J., Clarisó, R., Riera, D.: UMLtoCSP: a tool for the formal verification of UML/OCL models using constraint programming. In: ASE’07, pp. 547–548 (2007)Google Scholar
  9. 9.
    Cariou, E., Marvie, R., Seinturier, L., Duchien, L.: OCL for the specification of model transformation contracts. In: ECEASST, vol. 12, pp. 69–83 (2004)Google Scholar
  10. 10.
    de Moura, L.M., Bjørner, N.: Z3: an efficient SMT solver. In: TACAS’08, vol. 4963 of LNCS, pp. 337–340. Springer, Berlin (2008)Google Scholar
  11. 11.
    Ehrig, H., Ehrig, K., Prange, U., Taentzer, G.: Fundamentals of Algebraic Graph Transformation. Springer, Berlin (2006)zbMATHGoogle Scholar
  12. 12.
    Ehrig, H., Prange, U.: Formal analysis of model transformations based on triple graph rules with kernels. In: ICGT’08, vol. 5214 of LNCS, pp. 178–193. Springer, Berlin (2008)Google Scholar
  13. 13.
    Fiorentini, C., Momigliano, A., Ornaghi, M., Poernomo, I.: A constructive approach to testing model transformations. In: ICMT’10, vol. 6142 of LNCS, pp. 77–92. Springer, Berlin (2010)Google Scholar
  14. 14.
    Fleurey, F., Baudry, B., Muller, P.-A., Traon, Y.: Qualifying input test data for model transformations. Softw. Syst. Model. 8, 185–203 (2009)CrossRefGoogle Scholar
  15. 15.
    García-Domínguez, A., Kolovos, D., Rose, L., Paige, R., Medina-Bulo, I.: Eunit: A unit testing framework for model management tasks. In: MoDELS’11, vol. 6981 of LNCS, pp. 395–409. Springer, Berlin (2011)Google Scholar
  16. 16.
    Ghamarian, A.H., de Mol, M., Rensink, A., Zambon, E., Zimakova, M.: Modelling and analysis using GROOVE. STTT 14(1), 15–40 (2012)CrossRefGoogle Scholar
  17. 17.
    Giner, P., Pelechano, V.: Test-driven development of model transformations. In: MoDELS’09, vol. 5795 of LNCS, pp. 748–752. Springer, Berlin (2009)Google Scholar
  18. 18.
    Godefroid, P., de Halleux, J., Nori, A.V., Rajamani, S.K., Schulte, W., Tillmann, N., Levin, M.Y.: Automating software testing using program analysis. IEEE Softw. 25(5), 30–37 (2008)CrossRefGoogle Scholar
  19. 19.
    Gogolla, M., Büttner, F., Richters, M.: USE: A UML-based specification environment for validating UML and OCL. Sci. Comput. Program. 69(1–3), 27–34 (2007)CrossRefzbMATHGoogle Scholar
  20. 20.
    Gogolla, M., Vallecillo, A.: Tractable model transformation testing. In: ECMFA’11, vol. 6698 of LNCS, pp. 221–235. Springer, Berlin (2011)Google Scholar
  21. 21.
    González, C.A., Cabot, J.: ATL-Test: a white-box test generation approach for ATL transformations. In: MoDELS’12, vol. 7590 of LNCS, pp. 449–464. Springer, Berlin (2012)Google Scholar
  22. 22.
    Guerra, E.: Specification-driven test generation for model transformations. In: ICMT’12, vol. 7307 of LNCS, pp. 40–55. Springer, Berlin (2012)Google Scholar
  23. 23.
    Guerra, E., de Lara, J.: Colouring: execution, debug and analysis of QVT-relations transformations through coloured Petri nets. Softw. Syst. Model. (2013). doi: 10.1007/s10270-012-0292-6
  24. 24.
    Guerra, E., de Lara, J., Kolovos, D., Paige, R., dos Santos, O.: Engineering model transformations with transML. Softw. Syst. Model. 12(3), 555–577 (2013)CrossRefGoogle Scholar
  25. 25.
    Guerra, E., de Lara, J., Kolovos, D.S., Paige, R.F.: A visual specification language for model-to-model transformations. In: IEEE Computer Society on VL/HCC’10, pp. 119–126 (2010)Google Scholar
  26. 26.
    Guerra, E., de Lara, J., Wimmer, M., Kappel, G., Kusel, A., Retschitzegger, W., Schönböck, J., Schwinger, W.: Automated verification of model transformations based on visual contracts. Autom. Softw. Eng. 12(1), 5–46 (2013)CrossRefGoogle Scholar
  27. 27.
    Harm, J., Lämmel, R.: Two-dimensional approximation coverage. Informatica, 24(3), 355–369 (2000)Google Scholar
  28. 28.
    Hierons, R.M.: Testing from a Z specification. J. Softw. Test. Verif. Reliab. 7(1), 19–33 (1997)CrossRefGoogle Scholar
  29. 29.
    Jackson, D.: Software Abstractions. Logic, Language, and Analysis. MIT Press, Cambridge, MA (2006)Google Scholar
  30. 30.
    Jouault, F., Allilaire, F., Bézivin, J., Kurtev, I.: ATL: a model transformation tool. Sci. Comput. Program. 72(1–2), 31–39 (2008)CrossRefzbMATHGoogle Scholar
  31. 31.
    Kolovos, D.S., Paige, R.F., Polack, F.: The epsilon transformation language. In: ICMT’08, vol. 5063 of LNCS, pp. 46–60. Springer, Berlin (2008)Google Scholar
  32. 32.
    Küster, J.M.: Definition and validation of model transformations. Softw. Syst. Model. 5(3), 233–259 (2006)CrossRefGoogle Scholar
  33. 33.
    Küster, J.M., Abd-El-Razik, M.: Validation of model transformations—first experiences using a white box approach. In: MoDELS Workshops, vol. 4364 of LNCS, pp. 193–204. Springer, Berlin (2007)Google Scholar
  34. 34.
    Lämmel, R., Schulte, W.: Controllable combinatorial coverage in grammar-based testing. In: TestCom’06, pp. 19–38 (2006)Google Scholar
  35. 35.
    Lin, Y., Zhang, J., Gray, J.: A framework for testing model transformations. In: Model-driven Software Development—Research and Practice in Software Engineering. Springer, Berlin (2005)Google Scholar
  36. 36.
    Mottu, J., Baudry, B., Traon, Y.: Mutation analysis testing for model transformations. In: ECMDA-FA’06, vol. 4066 of LNCS, pp. 376–390. Springer, Berlin (2006)Google Scholar
  37. 37.
    Mottu, J., Baudry, B., Traon, Y.: Reusable MDA components: a testing-for-trust approach. In: MoDELS’06, vol. 4199 of LNCS, pp. 589–603. Springer, Berlin (2006)Google Scholar
  38. 38.
    Offutt, A.J., Liu, S.: Generating test data from SOFL specifications. J. Syst. Softw. 49(1), 49–62 (1999)CrossRefGoogle Scholar
  39. 39.
    Oster, S., Zorcic, I., Markert, F., Lochau, M.: MoSo-PoLiTe: tool support for pairwise and model-based software product line testing. In: VaMoS’11, ACM International Conference Proceedings Series, pp. 79–82. ACM (2011)Google Scholar
  40. 40.
    Perrouin, G., Oster, S., Sen, S., Klein, J., Baudry, B., Traon, Y.: Pairwise testing for software product lines: comparison of two approaches. Softw. Qual. J. 20(3–4), 605–643 (2012)CrossRefGoogle Scholar
  41. 41.
    Quillan, J.A.M., Power, J.F.: White-box coverage criteria for model transformations. In: 1st International Workshop on Model Transformation with ATL (2009)Google Scholar
  42. 42.
    Sen, S., Baudry, B., Mottu, J.: Automatic model generation strategies for model transformation testing. In: ICMT’09, vol. 5563 of LNCS, pp. 148–164. Springer, Berlin (2009)Google Scholar
  43. 43.
    Sen, S., Mottu, J.-M., Tisi, M., Cabot, J.: Using models of partial knowledge to test model transformations. In: ICMT’12, vol. 7307 of LNCS, pp. 24–39. Springer, Berlin (2012)Google Scholar
  44. 44.
    Sherwood, G.B., Martirosyan, S.S., Colbourn, C.: Covering arrays of higher strength from permutation vectors. J. Comb. Des. 14(3), 202–213 (2005)CrossRefMathSciNetGoogle Scholar
  45. 45.
    Soeken, M., Wille, R., Kuhlmann, M., Gogolla, M., Drechsler, R.: Verifying UML/OCL models using boolean satisfiability. In: IEEE on DATE’10, pp. 1341–1344 (2010)Google Scholar
  46. 46.
    Spivey, J.M.: An introduction to Z and formal specifications. Softw. Eng. J. 4(1), 40–50 (1989)CrossRefGoogle Scholar
  47. 47.
    Tillmann, N., Schulte, W.: Unit tests reloaded: parameterized unit testing with symbolic execution. IEEE Softw. 23(4), 38–47 (2006)CrossRefGoogle Scholar
  48. 48.
    Traon, Y.L., Baudry, B., Jézéquel, J.-M.: Design by contract to improve software vigilance. IEEE Trans. Softw. Eng. 32(8), 571–586 (2006)CrossRefGoogle Scholar
  49. 49.
    Troya, J., Vallecillo, A.: A rewriting logic semantics for ATL. J. Object Technol. 10(5), 1–29 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1. Computer Science DepartmentUniversidad Autónoma de MadridMadridSpain
  2. 2.University of BremenBremenGermany

Personalised recommendations