A Model-Level Mutation Tool to Support the Assessment of the Test Case Quality

  • Maria Fernanda GrandaEmail author
  • Nelly Condori-Fernández
  • Tanja E. J. Vos
  • Oscar Pastor
Conference paper
Part of the Lecture Notes in Information Systems and Organisation book series (LNISO, volume 22)


Although mutation testing is a well-known technique for assessing the quality of tests, there is not a lot of support available for model-level mutation analysis. It is also considered to be expensive due to: (i) the large number of mutants generated; (ii) the time-consuming activity of determining equivalent mutants; and (iii) the mutant execution time. It should also be remembered that real software artefacts of appropriate size including real faults are hard to find and prepare appropriately. In this paper we propose a mutation tool to generate valid First Order Mutants (FOM) for Conceptual Schemas (CS) based on UML Class Diagrams and evaluate its effectiveness and efficiency in generating valid and non-equivalent mutants. Our main findings were: (1) FOM mutation operators can be automated to avoiding non-valid mutants (49.1%). (2) Fewer equivalent mutants were generated (7.2%) and 74.3% were reduced by analysing the CS static structure in six subject CSs.


Mutation tool Model-level mutation Class diagram mutants Test cases quality 



This work has been developed with the financial support by SENESCYT of the Republic of Ecuador, SHIP (SMEs and HEIs in Innovation Partnerships, ref: EACEA/A2/UHB/CL 554187), PERTEST (TIN2013-46928-C3-1-R), European Commission (CaaS project) and Generalitat Valenciana (PROMETEOII/2014/039).


  1. 1.
    Jia, Y., Harman, M.: Higher order mutation testing. Inf. Softw. Technol. 51, 1379–1393 (2009)CrossRefGoogle Scholar
  2. 2.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. Softw. Eng. IEEE Trans. 37, 1–31 (2011)CrossRefGoogle Scholar
  3. 3.
    Andrews, J.H., Briand, L.C., Labiche, Y.: Is mutation an appropriate tool for testing experiments? Proc. ICSE 2005, 402–411 (2005)Google Scholar
  4. 4.
    Vincenzi, A.M.R., Simão, A.S., Delamaro, M.E., Maldonado, J.C.: Muta-Pro: towards the definition of a mutation testing process. J. Braz. Comput. Soc. 12, 49–61 (2006)CrossRefGoogle Scholar
  5. 5.
    Granda, M.F., Condori-Fernandez, N., Vos, T.E.J., Pastor, Ó.: Mutation operators for UML Class Diagrams. In: CAiSE 2016 (2016)Google Scholar
  6. 6.
    Object Management Group: Unified Modeling Language (UML) (2015)Google Scholar
  7. 7.
    Object Management Group: Action Language for Foundational UML (ALF) (2013)Google Scholar
  8. 8.
    Object Management Group: Semantics of a Foundational Subset for Executable UML Models (fUML) (2012)Google Scholar
  9. 9.
    Sauro, J.: Estimating productivity: composite operators for Keystroke Level Modeling. In: Human-Computer Interaction. New Trends, pp. 1–10 (2009)Google Scholar
  10. 10.
    Card, S.K., Moran, T.P., Newel, A.: The keystroke-level model for user performance time with interactive systems. Commun. ACM 23, 396–410 (1980)CrossRefGoogle Scholar
  11. 11.
    Haunold, P., Kuhn, W.: A keystroke level analysis of a graphics application: manual map digitizing. In: CHI ’94, pp. 337–343 (1994)Google Scholar
  12. 12.
    Teo, L., John, B.E.: Comparisons of keystroke-level model predictions to observed data. In: CHI ’06, pp. 1421–1426 (2006)Google Scholar
  13. 13.
    Kieras, D.: Using the keystroke-level model to estimate execution times (2001)Google Scholar
  14. 14.
    Granda, M.F., Condori-fernández, N., Vos, T.E.J., Pastor, O.: What do we know about the Defect Types detected in Conceptual Models ? In: IEEE 9th International Conference on Research Challenges in Information Science (RCIS), pp. 96–107. IEEE, Athens (2015)Google Scholar
  15. 15.
    Tort, A., Olivé, A.: Case Study: Conceptual Modeling of Basic Sudoku.
  16. 16.
    van Solingen, R., Berghout, E.: The Goal/Question/Metric Method—A Practical Guide for Quality Improvement of Software Development. McGraw-Hill (1999)Google Scholar
  17. 17.
    España, S., González, A., Pastor, Ó., Ruiz, M.: Technical Report Communication Analysis and the OO-Method: Manual Derivation of the Conceptual Model the SuperStationery Co. Lab Demo, Valencia (2011)Google Scholar
  18. 18.
    España, S., González, A., Pastor, Ó., Ruiz, M.: Integration of Communication Analysis and the OO-Method: Rules for the manual derivation of the Conceptual Model, Valencia (2011)Google Scholar
  19. 19.
    Holleis, P., Otto, F., Hussmann, H., Schmidt, A.: Keystroke-level model for advanced mobile phone interaction. In: CHI ’07 Proceedings of the SIGCHI Conference on Human factors in Computing Systems 1505–1514 (2007)Google Scholar
  20. 20.
    Granda, M.F., Condori-Fernandez, N., Vos, T.E.J., Pastor, O.: Towards the automated generation of abstract test cases from requirements models. In: 1st International Workshop on Requirements Engineering and Testing. pp. 39–46. IEEE, Karlskrona (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Maria Fernanda Granda
    • 1
    • 3
    Email author
  • Nelly Condori-Fernández
    • 2
  • Tanja E. J. Vos
    • 3
  • Oscar Pastor
    • 3
  1. 1.University of CuencaCuencaEcuador
  2. 2.Vrije Universiteit van AmsterdamAmsterdamThe Netherlands
  3. 3.Universitat Politècnica de ValènciaValenciaSpain

Personalised recommendations