Abstract
Mutation testing is a software engineering methodology where code mutation is used to assess the quality of a testing technique. Mutation testing is carried out by injecting errors in the code and measuring the ability of a testing tool to detect these errors. However, it is a time-consuming process, as tests need to be run on many variants of the code, called mutants. Each mutant represents a version of the code under test, with an injected error. In this paper, we propose HadoopMutator; a cloud-based mutation testing framework that reuses the MapReduce programming model in order to speed up the generation and testing of mutants. We show, through experimentation, that we can significantly enhance the performance of automated mutation testing and provide a scalable solution that is applicable for large-scale software projects. Based on two use cases, we show that the performance can be enhanced 10 folds, on average, using our proposed framework. By treating source code as data, our work paves the way for new reuse opportunities of the novel data-centric frameworks.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Buyya, R., et al.: Cloud computing: Principles and paradigms. John Wiley & Sons (2010)
Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM, 51(1), 107–113 (2008)
DeMillo, R.A., et al.: An extended overview of the Mothra software testing environment. Presented at the Proceedings of the Second Workshop on Software Testing, Verification, and Analysis (1988)
Derezińska, A.: Advanced Mutation Operators Applicable in C# Programs. In: Sacha, K. (ed.) Software Engineering Techniques: Design for Quality. IFIP, vol. 227, pp. 283–288. Springer, Boston (2007)
Derezinska, A., Szustek, A.: Tool-Supported Advanced Mutation Approach for Verification of C# Programs. In: 3rd International Conference on Dependability of Computer Systems (2008)
Fraser, G., Zeller, A.: Mutation-Driven Generation of Unit Tests and Oracles. IEEE Transactions on Software Engineering 38(2), 278–292 (2012)
Gospodnetic, O., Hatcher, E.: Lucene. Manning (2005)
Khare, R., et al.: Nutch: A flexible and scalable open-source web search engine. Oregon State University 1, 32 (2004)
Marynowski, J.E., et al.: Testing MapReduce-Based Systems. arXiv preprint arXiv:1209.6580 (2012)
Mateo, P.R., Usaola, M.P.: Bacterio: Java Mutation Testing Tool: A Framework to Evaluate Quality of Tests Cases. In: 28th IEEE International Conference on Software Maintenance (ICSM). IEEE (2012)
Mateo, P.R., Usaola, M.P.: Parallel Mutation Testing. Software Testing, Verification and Reliability 23(4), 315–350 (2013)
Matyas, S., et al.: Continuous Integration: Improving Software Quality and Reducing Risk. Addison-Wesley (2007)
Ma, Y.-S., et al.: MuJava: An Automated Class Mutation System. Software Testing, Verification and Reliability 15(2), 97–133 (2005)
Miller, M.: Cloud Computing: Web-Based Applications that Change the Way you Work and Collaborate Online. Que Publishing (2008)
Miner, D., Shook, A.: MapReduce Design Patterns: Building Effective Algorithms and Analytics for Hadoop and Other Systems. O’Reilly Media, Inc. (2012)
Nica, S., et al.: Is Mutation Testing Scalable for Real-World Software Projects?. In: The 3rd International Conference on Advances in System Testing and Validation Lifecycle, VALID 2011, pp. 40–45 (2011)
Parveen, T., et al.: Towards a Distributed Execution Framework for JUnit Test Cases. In: IEEE International Conference on Software Maintenance (ICSM 2009), pp. 425–428. IEEE (2009)
Saleh, I., et al.: Formal Methods for Data-Centric Web Services: From Model to Implementation. In: Proceedings of the IEEE International Conference on Web Services (ICWS 2013), pp. 332–339 (2013)
Saleh, I.: Formal Specification and Verification of Data-Centric Web Services. Virginia Polytechnic Institute and State University (2012)
Schuler, D., et al.: Efficient Mutation Testing by Checking Invariant Violations. In: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, pp. 69–80. ACM (2009)
Schuler, D., Zeller, A.: Javalanche: Efficient Mutation Testing for Java. In: Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, pp. 297–298. ACM (2009)
Shvachko, K., et al.: The Hadoop Distributed File System. In: 26th IEEE Symposium on Mass Storage Systems and Technologies (MSST 2010), pp. 1–10. IEEE (2010)
Tilley, S., Parveen, T.: HadoopUnit: Test Execution in the Cloud. In: Software Testing in the Cloud, 37–53. Springer (2012)
Tukey, J.W.: Exploratory Data Analysis. Addison Wesley (1977)
Voas, J.M., McGraw, G.: Software Fault Injection: Inoculating Programs Against Errors. John Wiley & Sons (1998)
White, T.: Hadoop: The definitive guide: The definitive guide. O’Reilly Media, Inc. (2009)
Jia, Y., Harman, M.: An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37(5), 649–678 (2011)
Apache Wicket, http://wicket.apache.org
CruiseControl, http://cruisecontrol.sourceforge.net/
Hadoop Word Count Problem, http://wiki.apache.org/hadoop/WordCount
Jenkins, http://jenkins-ci.org/
Pitest Mutation Operators, http://Pitest.org/quickstart/mutators/
Pitest Mutation Testing Framework, http://pitest.org/
The Apache Software Foundation, http://www.apache.org/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Saleh, I., Nagi, K. (2014). HadoopMutator: A Cloud-Based Mutation Testing Framework. In: Schaefer, I., Stamelos, I. (eds) Software Reuse for Dynamic Systems in the Cloud and Beyond. ICSR 2015. Lecture Notes in Computer Science, vol 8919. Springer, Cham. https://doi.org/10.1007/978-3-319-14130-5_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-14130-5_13
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14129-9
Online ISBN: 978-3-319-14130-5
eBook Packages: Computer ScienceComputer Science (R0)