Quality Evaluation of Object-Oriented and Standard Mutation Operators Applied to C# Programs

  • Anna Derezińska
  • Marcin Rudnik
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7304)


Mutation testing is a kind of fault injection approach that can be used to generate tests or to assess the quality of test sets. For object-oriented languages, like C#, both object-oriented and standard (traditional) mutation operators should be applied. The methods that can contribute to reducing the number of applied operators and lowering the costs of mutation testing were experimentally investigated. We extended the CREAM mutation tool to support selective testing, sampling and clustering of mutants, and combining code coverage with mutation testing. We propose an approach to quality evaluation and present experimental results of mutation operators applied to C# programs.


mutation testing object-oriented mutation operators C# 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jia, Y., Harman, M.: An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37(5), 649–678 (2011)CrossRefGoogle Scholar
  2. 2.
    Chevalley, P.: Applying Mutation Analysis for Object-Oriented Programs Using a Reactive Approach. In: Proc. of the 8th Asia-Pacific Software Engineering Conference, ASPEC, pp. 267–270 (2001)Google Scholar
  3. 3.
    Kim, S., Clark, J., McDermid J.A.: Class Mutation: Mutation Testing for Object-Oriented Programs. In: Conference on Object-Oriented Software Systems, Erfurt, Germany (2000) Google Scholar
  4. 4.
    Ma, Y.-S., Kwon, Y.-R., Offutt, J.: Inter-class Mutation Operators for Java. In: Proc. of International Symposium on Software Reliability Engineering, ISSRE 2002. IEEE Computer Soc. (2002)Google Scholar
  5. 5.
    Derezińska, A.: Advanced Mutation Operators Applicable in C# Programs. In: Sacha, K. (ed.) Software Engineering Techniques: Design for Quality. IFIP, vol. 227, pp. 283–288. Springer, Boston (2006)CrossRefGoogle Scholar
  6. 6.
    Derezińska, A.: Quality Assessment of Mutation Operators Dedicated for C# Programs. In: Proc. of the 6th International Conference on Quality Software, QSIC 2006, pp. 227–234. IEEE Soc. Press (2006) Google Scholar
  7. 7.
    Derezińska, A., Szustek, A.: Tool-supported Mutation Approach for Verification of C# Programs. In: Zamojski, W., et al. (eds.) Proc. of International Conference on Dependability of Computer Systems, DepCoS-RELCOMEX 2008, pp. 261–268. IEEE Comp. Soc. (2008)Google Scholar
  8. 8.
    Derezińska, A., Szustek, A.: Object-Oriented Testing Capabilities and Performance Evaluation of the C# Mutation System, In: Szmuc, T., Szpyrka, M., Zendulka, J. (eds.) CEE-SET 2009. LNCS, vol. 7054, pp. 229–242 (2012) Google Scholar
  9. 9.
    Derezińska, A., Sarba, K.: Distributed Environment Integrating Tools for Software Testing. In: Elleithy, K. (ed.) Advanced Techniques in Computing Sciences and Software Engineering, pp. 545–550. Springer, Dordrecht (2009)Google Scholar
  10. 10.
    Derezińska, A., Kowalski, K.: Object-oriented Mutation Applied in Common Intermediate Language Programs Originated from C#. In: Proc. of 4th International Conference Software Testing Verification and Validation Workshops, 6th Workshop on Mutation Analysis, pp. 342–350. IEEE Comp. Soc. (2011)Google Scholar
  11. 11.
    Derezińska, A.: Classification of Operators of C# Language. In: Borzemski, L., et al. (eds.) Information Systems Architecture and Technology, New Developments in Web-Age Information Systems, pp. 261–271. Wrocław University of Technology (2010)Google Scholar
  12. 12.
    Offut, J., Rothermel, G., Zapf, C.: An Experimental Evaluation of Selective Mutation. In: Proc. of 15th International Conference on Software Engineering, pp. 100–107. IEEE Comp. Soc. Press (1993)Google Scholar
  13. 13.
    Barbosa, E.F., Maldonado, J.C., Vincenzi, A.M.R.: Toward the Determination of Sufficient Mutant Operators for C. Journal Software, Testing, Verification, and Reliability 11, 113–136 (2001)CrossRefGoogle Scholar
  14. 14.
    Namin, S., Andrews, J.H.: On Sufficiency of Mutants. In: Proc. of 29th International Conference on Software Engineering, ICSE 2007 (2007) Google Scholar
  15. 15.
    Zhang, L., Hou, S.-S., Hu, J.-J., Xie, T., Mei, H.: Is Operator-Based Mutant Selection Superior to Random Mutant Selection? In: Proc. of the 32nd International Conference on Software Engineering, ICSE 2010, pp. 435–444 (2010)Google Scholar
  16. 16.
    Ma, Y.-S., Kwon, Y.-R., Kim, S.-W.: Statistical Investigation on Class Mutation Operators. ETRI Journal 31(2), 140–150 (2009)CrossRefGoogle Scholar
  17. 17.
    Hu, J., Li, N., Offutt, J.: An Analysis of OO Mutation Operators. In: Proc. of 4th International Conference Software Testing Verification and Validation Workshops, 6th Workshop on Mutation Analysis, pp. 334–341. IEEE Comp. Soc. (2011) Google Scholar
  18. 18.
    Kaminski, G., Praphamontripong, U., Ammann, P., Offutt, J.: A Logic Mutation Approach to Selective Mutation for Programs and Queries. Information and Software Technology, 1137–1152 (2011)Google Scholar
  19. 19.
    Usaola, M.P., Mateo, P.R.: Mutation Testing Cost Reduction Techniques: a Survey. IEEE Software 27(3), 80–86 (2010)CrossRefGoogle Scholar
  20. 20.
    Mathur, A.P., Wong, W.E.: Reducing the Cost of Mutation Testing: An Empirical Study. Journal of Systems and Software 31, 185–196 (1995)CrossRefGoogle Scholar
  21. 21.
    Hussain, S.: Mutation Clustering. Ms. Th., King’s College London, Strand, London (2008) Google Scholar
  22. 22.
    Ji, C., Chen, Z.Y., Xu, B.W., Zhao, Z.: A Novel Method of Mutation Clustering Based on Domain Analysis. In: Proc. of 21st International Conference on Software Engineering & Knowledge Engineering, SEKE 2009, pp.422–425 (2009) Google Scholar
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
    Derezińska, A., Rudnik, M.: Empirical Evaluation of Cost Reduction Techniques of Mutation Testing for C# Programs, Warsaw Univ. of Tech., Inst. of Computer Science Res. Rap. 1/2012 (2012) Google Scholar
  28. 28.
  29. 29.
    Lee, H.-J., Ma, Y.-S., Kwon, Y.-R.: Empirical Evaluation of Orthogonality of Class Mutation Operators. In: Proc. of 11th Asia-Pacific Software Engineering Conference. IEEE Comp. Soc. (2004) Google Scholar
  30. 30.
    Ma, Y.-S., Harrold, M.J., Kwon, Y.-R.: Evaluation of Mutation Testing for Object-Oriented Programs. In: Proc. of 28th International Conference on Software Engineering, pp 869–872. IEEE Comp. Soc. Press (2006) Google Scholar
  31. 31.
    Ma, Y.-S., Offutt, J., Kwon, Y.-R.: MuJava: an Automated Class Mutation System. Software Testing, Verification and Reliability 15(2) (June 2005) Google Scholar
  32. 32.
    Smith, B.H., Williams, L.: A Empirical Evaluation of the MuJava Mutation Operators. In: Proc. 3rd International Workshop on Mutation Analysis Mutation 2007 at TAIC.Part 2007, Cumberland Lodge, Windsor UK, pp. 193–202 (September 2007)Google Scholar
  33. 33.
  34. 34.
  35. 35.
    Segura, S., Hierons, R.M., Benavides, D., Ruiz-Cortes, A.: Mutation Testing on an Object-oriented Framework: an Experience Report. Information and Software Technology, 1124–1136 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Anna Derezińska
    • 1
  • Marcin Rudnik
    • 1
  1. 1.Institute of Computer ScienceWarsaw University of TechnologyWarsawPoland

Personalised recommendations