# Specification-driven model transformation testing

- 484 Downloads
- 14 Citations

## Abstract

Testing model transformations poses several challenges, among them the automatic generation of appropriate input test models and the specification of oracle functions. Most approaches for the generation of input models ensure a certain coverage of the source meta-model or the transformation implementation code, whereas oracle functions are frequently defined using query or graph languages. However, these two tasks are usually performed independently regardless of their common purpose, and sometimes, there is a gap between the properties exhibited by the generated input models and those considered by the transformations. Recently, we proposed a formal specification language for the declarative formulation of transformation properties (by means of invariants, pre-, and postconditions) from which we generated partial oracle functions used for transformation testing. Here, we extend the usage of our specification language for the automated generation of input test models by SAT solving. The testing process becomes more *intentional* because the generated models ensure a certain coverage of the transformation requirements. Moreover, we use the same specification to consistently derive both the input test models and the oracle functions. A set of experiments is presented, aimed at measuring the efficacy of our technique.

## Keywords

Model transformation Model transformation specification Model transformation testing Model finding Test oracle## Notes

### Acknowledgments

We thank the referees for their useful comments. This work has been sponsored by the Spanish Ministry of Science and Innovation with project “Go-Lite” (TIN2011-24139), by the R&D program of the Community of Madrid with project “e-Madrid” (S2009/TIC-1650), and by the German Research Foundation (DFG) within the Reinhart Koselleck project (DR 287/23-1).

## References

- 1.Balogh, A., Bergmann, G., Csertán, G., Gönczy, L., Horváth, Á., Majzik, I., Pataricza, A., Polgár, B., Ráth, I., Varró, D., Varró, G.: Workflow-driven tool integration using model transformations. In: Graph Transformations and Model-Driven Engineering, vol. 5765 of LNCS, pp. 224–248. Springer, Berlin (2010)Google Scholar
- 2.Baudry, B., Ghosh, S., Fleurey, F., France, R.B., Traon, Y.L., Mottu, J.-M.: Barriers to systematic model transformation testing. CACM
**53**(6), 139–143 (2010)CrossRefGoogle Scholar - 3.Boronat, A., Carsí, J.A., Ramos, I.: Algebraic specification of a model transformation engine. In: FASE’06, vol. 3922 of LNCS, pp. 262–277. Springer, Berlin (2006)Google Scholar
- 4.Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on Java predicates. In: ISSTA’02, pp. 123–133 (2002)Google Scholar
- 5.BPMN. http://www.bpmn.org/
- 6.Budd, T.A.: Mutation analysis: ideas, examples, problems and prospects. In: Proceedings of Summer School on Computer Program Testing, pp. 129–148 (1981)Google Scholar
- 7.Cabot, J., Clarisó, R., Guerra, E., de Lara, J.: Verification and validation of declarative model-to-model transformations through invariants. J. Syst. Softw.
**83**(2), 283–302 (2010)CrossRefGoogle Scholar - 8.Cabot, J., Clarisó, R., Riera, D.: UMLtoCSP: a tool for the formal verification of UML/OCL models using constraint programming. In: ASE’07, pp. 547–548 (2007)Google Scholar
- 9.Cariou, E., Marvie, R., Seinturier, L., Duchien, L.: OCL for the specification of model transformation contracts. In: ECEASST, vol. 12, pp. 69–83 (2004)Google Scholar
- 10.de Moura, L.M., Bjørner, N.: Z3: an efficient SMT solver. In: TACAS’08, vol. 4963 of LNCS, pp. 337–340. Springer, Berlin (2008)Google Scholar
- 11.Ehrig, H., Ehrig, K., Prange, U., Taentzer, G.: Fundamentals of Algebraic Graph Transformation. Springer, Berlin (2006)zbMATHGoogle Scholar
- 12.Ehrig, H., Prange, U.: Formal analysis of model transformations based on triple graph rules with kernels. In: ICGT’08, vol. 5214 of LNCS, pp. 178–193. Springer, Berlin (2008)Google Scholar
- 13.Fiorentini, C., Momigliano, A., Ornaghi, M., Poernomo, I.: A constructive approach to testing model transformations. In: ICMT’10, vol. 6142 of LNCS, pp. 77–92. Springer, Berlin (2010)Google Scholar
- 14.Fleurey, F., Baudry, B., Muller, P.-A., Traon, Y.: Qualifying input test data for model transformations. Softw. Syst. Model.
**8**, 185–203 (2009)CrossRefGoogle Scholar - 15.García-Domínguez, A., Kolovos, D., Rose, L., Paige, R., Medina-Bulo, I.: Eunit: A unit testing framework for model management tasks. In: MoDELS’11, vol. 6981 of LNCS, pp. 395–409. Springer, Berlin (2011)Google Scholar
- 16.Ghamarian, A.H., de Mol, M., Rensink, A., Zambon, E., Zimakova, M.: Modelling and analysis using GROOVE. STTT
**14**(1), 15–40 (2012)CrossRefGoogle Scholar - 17.Giner, P., Pelechano, V.: Test-driven development of model transformations. In: MoDELS’09, vol. 5795 of LNCS, pp. 748–752. Springer, Berlin (2009)Google Scholar
- 18.Godefroid, P., de Halleux, J., Nori, A.V., Rajamani, S.K., Schulte, W., Tillmann, N., Levin, M.Y.: Automating software testing using program analysis. IEEE Softw.
**25**(5), 30–37 (2008)CrossRefGoogle Scholar - 19.Gogolla, M., Büttner, F., Richters, M.: USE: A UML-based specification environment for validating UML and OCL. Sci. Comput. Program.
**69**(1–3), 27–34 (2007)CrossRefzbMATHGoogle Scholar - 20.Gogolla, M., Vallecillo, A.: Tractable model transformation testing. In: ECMFA’11, vol. 6698 of LNCS, pp. 221–235. Springer, Berlin (2011)Google Scholar
- 21.González, C.A., Cabot, J.: ATL-Test: a white-box test generation approach for ATL transformations. In: MoDELS’12, vol. 7590 of LNCS, pp. 449–464. Springer, Berlin (2012)Google Scholar
- 22.Guerra, E.: Specification-driven test generation for model transformations. In: ICMT’12, vol. 7307 of LNCS, pp. 40–55. Springer, Berlin (2012)Google Scholar
- 23.Guerra, E., de Lara, J.: Colouring: execution, debug and analysis of QVT-relations transformations through coloured Petri nets. Softw. Syst. Model. (2013). doi: 10.1007/s10270-012-0292-6
- 24.Guerra, E., de Lara, J., Kolovos, D., Paige, R., dos Santos, O.: Engineering model transformations with transML. Softw. Syst. Model.
**12**(3), 555–577 (2013)CrossRefGoogle Scholar - 25.Guerra, E., de Lara, J., Kolovos, D.S., Paige, R.F.: A visual specification language for model-to-model transformations. In: IEEE Computer Society on VL/HCC’10, pp. 119–126 (2010)Google Scholar
- 26.Guerra, E., de Lara, J., Wimmer, M., Kappel, G., Kusel, A., Retschitzegger, W., Schönböck, J., Schwinger, W.: Automated verification of model transformations based on visual contracts. Autom. Softw. Eng.
**12**(1), 5–46 (2013)CrossRefGoogle Scholar - 27.Harm, J., Lämmel, R.: Two-dimensional approximation coverage. Informatica,
**24**(3), 355–369 (2000)Google Scholar - 28.Hierons, R.M.: Testing from a Z specification. J. Softw. Test. Verif. Reliab.
**7**(1), 19–33 (1997)CrossRefGoogle Scholar - 29.Jackson, D.: Software Abstractions. Logic, Language, and Analysis. MIT Press, Cambridge, MA (2006)Google Scholar
- 30.Jouault, F., Allilaire, F., Bézivin, J., Kurtev, I.: ATL: a model transformation tool. Sci. Comput. Program.
**72**(1–2), 31–39 (2008)CrossRefzbMATHGoogle Scholar - 31.Kolovos, D.S., Paige, R.F., Polack, F.: The epsilon transformation language. In: ICMT’08, vol. 5063 of LNCS, pp. 46–60. Springer, Berlin (2008)Google Scholar
- 32.Küster, J.M.: Definition and validation of model transformations. Softw. Syst. Model.
**5**(3), 233–259 (2006)CrossRefGoogle Scholar - 33.Küster, J.M., Abd-El-Razik, M.: Validation of model transformations—first experiences using a white box approach. In: MoDELS Workshops, vol. 4364 of LNCS, pp. 193–204. Springer, Berlin (2007)Google Scholar
- 34.Lämmel, R., Schulte, W.: Controllable combinatorial coverage in grammar-based testing. In: TestCom’06, pp. 19–38 (2006)Google Scholar
- 35.Lin, Y., Zhang, J., Gray, J.: A framework for testing model transformations. In: Model-driven Software Development—Research and Practice in Software Engineering. Springer, Berlin (2005)Google Scholar
- 36.Mottu, J., Baudry, B., Traon, Y.: Mutation analysis testing for model transformations. In: ECMDA-FA’06, vol. 4066 of LNCS, pp. 376–390. Springer, Berlin (2006)Google Scholar
- 37.Mottu, J., Baudry, B., Traon, Y.: Reusable MDA components: a testing-for-trust approach. In: MoDELS’06, vol. 4199 of LNCS, pp. 589–603. Springer, Berlin (2006)Google Scholar
- 38.Offutt, A.J., Liu, S.: Generating test data from SOFL specifications. J. Syst. Softw.
**49**(1), 49–62 (1999)CrossRefGoogle Scholar - 39.Oster, S., Zorcic, I., Markert, F., Lochau, M.: MoSo-PoLiTe: tool support for pairwise and model-based software product line testing. In: VaMoS’11, ACM International Conference Proceedings Series, pp. 79–82. ACM (2011)Google Scholar
- 40.Perrouin, G., Oster, S., Sen, S., Klein, J., Baudry, B., Traon, Y.: Pairwise testing for software product lines: comparison of two approaches. Softw. Qual. J.
**20**(3–4), 605–643 (2012)CrossRefGoogle Scholar - 41.Quillan, J.A.M., Power, J.F.: White-box coverage criteria for model transformations. In: 1st International Workshop on Model Transformation with ATL (2009)Google Scholar
- 42.Sen, S., Baudry, B., Mottu, J.: Automatic model generation strategies for model transformation testing. In: ICMT’09, vol. 5563 of LNCS, pp. 148–164. Springer, Berlin (2009)Google Scholar
- 43.Sen, S., Mottu, J.-M., Tisi, M., Cabot, J.: Using models of partial knowledge to test model transformations. In: ICMT’12, vol. 7307 of LNCS, pp. 24–39. Springer, Berlin (2012)Google Scholar
- 44.Sherwood, G.B., Martirosyan, S.S., Colbourn, C.: Covering arrays of higher strength from permutation vectors. J. Comb. Des.
**14**(3), 202–213 (2005)CrossRefMathSciNetGoogle Scholar - 45.Soeken, M., Wille, R., Kuhlmann, M., Gogolla, M., Drechsler, R.: Verifying UML/OCL models using boolean satisfiability. In: IEEE on DATE’10, pp. 1341–1344 (2010)Google Scholar
- 46.Spivey, J.M.: An introduction to Z and formal specifications. Softw. Eng. J.
**4**(1), 40–50 (1989)CrossRefGoogle Scholar - 47.Tillmann, N., Schulte, W.: Unit tests reloaded: parameterized unit testing with symbolic execution. IEEE Softw.
**23**(4), 38–47 (2006)CrossRefGoogle Scholar - 48.Traon, Y.L., Baudry, B., Jézéquel, J.-M.: Design by contract to improve software vigilance. IEEE Trans. Softw. Eng.
**32**(8), 571–586 (2006)CrossRefGoogle Scholar - 49.Troya, J., Vallecillo, A.: A rewriting logic semantics for ATL. J. Object Technol.
**10**(5), 1–29 (2011)Google Scholar