HadoopMutator: A Cloud-Based Mutation Testing Framework

  • Iman Saleh
  • Khaled Nagi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8919)


Mutation testing is a software engineering methodology where code mutation is used to assess the quality of a testing technique. Mutation testing is carried out by injecting errors in the code and measuring the ability of a testing tool to detect these errors. However, it is a time-consuming process, as tests need to be run on many variants of the code, called mutants. Each mutant represents a version of the code under test, with an injected error. In this paper, we propose HadoopMutator; a cloud-based mutation testing framework that reuses the MapReduce programming model in order to speed up the generation and testing of mutants. We show, through experimentation, that we can significantly enhance the performance of automated mutation testing and provide a scalable solution that is applicable for large-scale software projects. Based on two use cases, we show that the performance can be enhanced 10 folds, on average, using our proposed framework. By treating source code as data, our work paves the way for new reuse opportunities of the novel data-centric frameworks.


Hadoop MapReduce Mutation Testing Experimentation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Buyya, R., et al.: Cloud computing: Principles and paradigms. John Wiley & Sons (2010)Google Scholar
  2. 2.
    Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM, 51(1), 107–113 (2008)Google Scholar
  3. 3.
    DeMillo, R.A., et al.: An extended overview of the Mothra software testing environment. Presented at the Proceedings of the Second Workshop on Software Testing, Verification, and Analysis (1988)Google Scholar
  4. 4.
    Derezińska, A.: Advanced Mutation Operators Applicable in C# Programs. In: Sacha, K. (ed.) Software Engineering Techniques: Design for Quality. IFIP, vol. 227, pp. 283–288. Springer, Boston (2007)CrossRefGoogle Scholar
  5. 5.
    Derezinska, A., Szustek, A.: Tool-Supported Advanced Mutation Approach for Verification of C# Programs. In: 3rd International Conference on Dependability of Computer Systems (2008)Google Scholar
  6. 6.
    Fraser, G., Zeller, A.: Mutation-Driven Generation of Unit Tests and Oracles. IEEE Transactions on Software Engineering 38(2), 278–292 (2012)CrossRefGoogle Scholar
  7. 7.
    Gospodnetic, O., Hatcher, E.: Lucene. Manning (2005)Google Scholar
  8. 8.
    Khare, R., et al.: Nutch: A flexible and scalable open-source web search engine. Oregon State University 1, 32 (2004)Google Scholar
  9. 9.
    Marynowski, J.E., et al.: Testing MapReduce-Based Systems. arXiv preprint arXiv:1209.6580 (2012)Google Scholar
  10. 10.
    Mateo, P.R., Usaola, M.P.: Bacterio: Java Mutation Testing Tool: A Framework to Evaluate Quality of Tests Cases. In: 28th IEEE International Conference on Software Maintenance (ICSM). IEEE (2012)Google Scholar
  11. 11.
    Mateo, P.R., Usaola, M.P.: Parallel Mutation Testing. Software Testing, Verification and Reliability 23(4), 315–350 (2013)CrossRefGoogle Scholar
  12. 12.
    Matyas, S., et al.: Continuous Integration: Improving Software Quality and Reducing Risk. Addison-Wesley (2007)Google Scholar
  13. 13.
    Ma, Y.-S., et al.: MuJava: An Automated Class Mutation System. Software Testing, Verification and Reliability 15(2), 97–133 (2005)CrossRefGoogle Scholar
  14. 14.
    Miller, M.: Cloud Computing: Web-Based Applications that Change the Way you Work and Collaborate Online. Que Publishing (2008)Google Scholar
  15. 15.
    Miner, D., Shook, A.: MapReduce Design Patterns: Building Effective Algorithms and Analytics for Hadoop and Other Systems. O’Reilly Media, Inc. (2012)Google Scholar
  16. 16.
    Nica, S., et al.: Is Mutation Testing Scalable for Real-World Software Projects?. In: The 3rd International Conference on Advances in System Testing and Validation Lifecycle, VALID 2011, pp. 40–45 (2011)Google Scholar
  17. 17.
    Parveen, T., et al.: Towards a Distributed Execution Framework for JUnit Test Cases. In: IEEE International Conference on Software Maintenance (ICSM 2009), pp. 425–428. IEEE (2009)Google Scholar
  18. 18.
    Saleh, I., et al.: Formal Methods for Data-Centric Web Services: From Model to Implementation. In: Proceedings of the IEEE International Conference on Web Services (ICWS 2013), pp. 332–339 (2013)Google Scholar
  19. 19.
    Saleh, I.: Formal Specification and Verification of Data-Centric Web Services. Virginia Polytechnic Institute and State University (2012)Google Scholar
  20. 20.
    Schuler, D., et al.: Efficient Mutation Testing by Checking Invariant Violations. In: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, pp. 69–80. ACM (2009)Google Scholar
  21. 21.
    Schuler, D., Zeller, A.: Javalanche: Efficient Mutation Testing for Java. In: Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, pp. 297–298. ACM (2009)Google Scholar
  22. 22.
    Shvachko, K., et al.: The Hadoop Distributed File System. In: 26th IEEE Symposium on Mass Storage Systems and Technologies (MSST 2010), pp. 1–10. IEEE (2010)Google Scholar
  23. 23.
    Tilley, S., Parveen, T.: HadoopUnit: Test Execution in the Cloud. In: Software Testing in the Cloud, 37–53. Springer (2012)Google Scholar
  24. 24.
    Tukey, J.W.: Exploratory Data Analysis. Addison Wesley (1977)Google Scholar
  25. 25.
    Voas, J.M., McGraw, G.: Software Fault Injection: Inoculating Programs Against Errors. John Wiley & Sons (1998)Google Scholar
  26. 26.
    White, T.: Hadoop: The definitive guide: The definitive guide. O’Reilly Media, Inc. (2009)Google Scholar
  27. 27.
    Jia, Y., Harman, M.: An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37(5), 649–678 (2011)CrossRefGoogle Scholar
  28. 28.
    Apache Wicket,
  29. 29.
  30. 30.
  31. 31.
    Hadoop Word Count Problem,
  32. 32.
  33. 33.
    Pitest Mutation Operators,
  34. 34.
    Pitest Mutation Testing Framework,
  35. 35.
    The Apache Software Foundation,

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Iman Saleh
    • 1
  • Khaled Nagi
    • 2
  1. 1.Graduate SchoolUniversity of MiamiCoral GablesUSA
  2. 2.Dept. of Computer and Systems Engineering, Faculty of EngineeringAlexandria UniversityEgypt

Personalised recommendations