Abstract
The main purpose of mutation testing approach is to check a test suite quality in terms of the adequacy to killing programs with inserted programming faults. We present advances in the C# mutation testing system that supports object-oriented mutation operators. The system enhancements related to functional requirements (mutation operators, avoiding generation of invalid and partially of equivalent mutants) and non-functional ones (speed-up using a new parser and reflection, space reduction storing mutant updates). Mutation testing of six widely used open source programs is discussed. The quality of the tests supplied with these programs was experimentally determined. Performance measures were evaluated to assess system enhancements (2-4 faster mutants creation, 10-100 times disk space reduction, tradeoff of time overhead for storing mutants of different size in a local or remote repository).
This work was supported by the Polish Ministry of Science and Higher Education under grant 4297/B/T02/2007/33.
Chapter PDF
Similar content being viewed by others
References
Voas, J.M., McGraw, G.: Software Fault Injection, Inoculating Programs Against Errors. John Wiley & Sons Inc. (1998)
Andrews, J.H., Briand, C., Labiche, Y., Namin, A.S.: Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria. IEEE Transactions on Software Engineering 342(8), 608–624 (2006)
Offut, J.: A Mutation Carol Past, Present and Future. In: Proc. of the 4th International Workshop on Mutation Analysis, Mutation 2009, Denver, Colorado (2009)
Chevalley, P.: Applying Mutation Analysis for Object-oriented Programs Using a Reflective Approach. In: Proc. of the 8th Asia-Pacific Software Engineering Conference, ASPEC, pp. 267–270 (2001)
Chevalley, P., Thevenod-Fosse, P.: A Mutation Analysis Tool for Java Programs. Journal on Software Tools for Techn. Transfer (STTT) 5(1), 90–103 (2003)
Kim, S., Clark, J., McDermid, J.A.: Class Mutation: Mutation Testing for Object-oriented Programs. In: Proc. of Conference on Object-Oriented Software Systems, Erfurt, Germany (2000)
Ma, Y.-S., Kwon, Y.-R., Offutt, J.: Inter-class Mutation Operators for Java. In: Proc. of International Symposium on Software Reliability Engineering, ISSRE 2002. IEEE Computer Soc. (2002)
Derezińska, A.: Advanced Mutation Operators Applicable in C# programs. In: Sacha, K. (ed.) Software Engineering Techniques: Design for Quality. IFIP, vol. 227, pp. 283–288. Springer, Boston (2006)
Derezińska, A.: Quality Assessment of Mutation Operators Dedicated for C# Programs. In: Proc. of the 6th International Conference on Quality Software, QSIC 2006, pp. 227–234. IEEE Computer Soc. Press, California (2006)
Derezińska, A., Szustek, A.: CREAM - a System for Object-oriented Mutation of C# Programs. In: Szczepański, S., Kłosowski, M., Felendzer, Z. (eds.) Annals Gdansk University of Technology Faculty of ETI, No 5, Information Technologies, vol. 13, pp. 389–406. Gdańsk (2007)
Derezińska, A., Szustek, A.: Tool-supported Mutation Approach for Verification of C# programs. In: Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T. (eds.) Proc. of International Conference on Dependability of Computer Systems, DepCoS-RELCOMEX 2008, pp. 261–268. IEEE Comp. Soc., USA (2008)
Derezińska, A., Sarba, K.: Distributed Environment Integrating Tools for Software Testing. In: Elleithy, K. (ed.) Advanced Techniques in Computing Sciences and Software Engineering, pp. 545–550. Springer, Heidelberg (2010)
Subversion svn, http://subversion.tigris.org
Delmaro, M., Maldonado, J.: Proteum - a Tool for the Assessment of Test Adequacy for C Programs. In: Proc. of Conference on Performability in Computing Sys., PCS 1996, pp. 79–95 (1996)
Namin, S., Andrews, J.H.: On Sufficiency of Mutants. In: Proc. of the 29th International Conference on Software Engineering, ICSE 2007 (2007)
Jia, Y., Harman, M.: MiLu: a Customizable, Runtime-Optimized Higher Order Mutation Testing Tool for the Full C Language. TAIC-Part (2008)
Irvine, S.A., et al.: Jumble Java Byte Code to Measure the Effectiveness of Unit Tests. In: Mutation 2007 at TAIC.Part 2007, 3th Inter. Workshop on Mutation Analysis, pp. 169–175. Cumberland Lodge, Windsor UK (2007)
Kim, S.-W., Harrold, M.J., Kwon, Y.-R.: MuGamma: Mutation Analysis of Deployed Software to Increase Confidence and Assist Evolution. In: Proc. of the 2nd Workshop on Mutation Analysis, Mutation 2006, Releigh, North Carolina (November 2006)
Madeyski, L.: On the Effects of Pair Programming on Thoroughness and Fault-finding Effectiveness of Unit Tests. In: Münch, J., Abrahamsson, P. (eds.) PROFES 2007. LNCS, vol. 4589, pp. 207–221. Springer, Heidelberg (2007)
Grün, B.J.M., Schuler, D., Zeller, A.: The Impact of Equivalent Mutants. In: Proc. of the 4th International Workshop on Mutation Analysis, Mutation 2009, Denver, Colorado (2009)
Moore, I.: Jester a JUnit Test Tester. eXtreme Programming and Flexible Process in Software Engineering – XP 2000 (2000)
Do, H., Rothermel, G.: A Controlled Experiment Assessing Test Case Prioritization Techniques via Mutation Faults. In: Proc. of the 21st IEEE International Confe on Software Maintenance, ICSM 2005, pp. 411–420. IEEE Comp. Soc. (2005)
Ma, Y.-S., Offutt, J., Kwon, Y.-R.: MuJava: an Automated Class Mutation System, Software Testing, Verification and Reliability 15(2) (2005)
Nester, http://nester.sourceforge.net/
Lee, H.-J., Ma, Y.-S., Kwon, Y.-R.: Empirical Evaluation of Orthogonality of Class Mutation Operators. In: 11th Asia-Pacific Software Engineering Conference. IEEE Computer Society (2004)
Smith, B.H., Williams, L.: A Empirical Evaluation of the MuJava Mutation Operators. In: Mutation 2007 at TAIC.Part 2007, 3th International Workshop on Mutation Analysis, pp. 193–202. Cumberland Lodge, Windsor UK (2007)
Baudry, B., Fleurey, F., Jezequel, J.-M., Traon, Y.L.: From Genetic to Bacteriological Algorithms for Mutation-based Testing. Software Testing, Verification and Reliability 15(2), 73–96 (2005)
Kowalski, K.: Implementing Object Mutations into Intermediate Code for C# programs, Bach. Thesis, Inst. of Comp. Science, Warsaw Univ. of Technology (2008) (in polish)
Offut, J., Rothermel, G., Zapf, C.: An Experimental Evaluation of Selective Mutation. In: Proc. of the 15th International Conference on Software Engineering, pp. 100–107 (1993)
NRefactory, http//codeconverter.sharpdeveloper.net/Convert.aspx
NUnit, http://www.nunit.org
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 IFIP International Federation for Information Processing
About this paper
Cite this paper
Derezińska, A., Szustek, A. (2012). Object-Oriented Testing Capabilities and Performance Evaluation of the C# Mutation System. In: Szmuc, T., Szpyrka, M., Zendulka, J. (eds) Advances in Software Engineering Techniques. CEE-SET 2009. Lecture Notes in Computer Science, vol 7054. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28038-2_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-28038-2_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28037-5
Online ISBN: 978-3-642-28038-2
eBook Packages: Computer ScienceComputer Science (R0)