Machine Learning Approach in Mutation Testing

  • Joanna Strug
  • Barbara Strug
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7641)


This paper deals with an approach based on the similarity of mutants. This similarity is used to reduce the number of mutants to be executed. In order to calculate such a similarity among mutants their structure is used. Each mutant is converted into a hierarchical graph, which represents the program’s flow, variables and conditions. On the basis of this graph form a special graph kernel is defined to calculate similarity among programs. It is then used to predict whether a given test would detect a mutant or not. The prediction is carried out with the help of a classification algorithm. This approach should help to lower the number of mutants which have to be executed. An experimental validation of this approach is also presented in this paper. An example of a program used in experiments is described and the results obtained, especially classification errors, are presented.


mutation testing machine learning graph distance classification test evaluation 


  1. 1.
    Acree, A.T.: On Mutation, PhD Thesis, Georgia Institute of Technology, Atlanta, Georgia (1980)Google Scholar
  2. 2.
    Agrawal, R., Imielinski, T., Swami, A.: Mining association rules between sets of items in large databases. In: Proc. 1993 ACM-SIGMOD Int. Conf. Management of Data (SIGMOD 1993), Washington, DC, pp. 207–216 (1993)Google Scholar
  3. 3.
    Borgwardt, K.M., Kriegel, H.P.: Shortest-path kernels on graphs. In: ICDM 2005, pp. 74–81 (2005)Google Scholar
  4. 4.
    Bunke, H., Riesen, K.: Improving vector space embedding of graphs through feature selection algorithms. Pattern Recognition 44(9), 1928–1940 (2011)CrossRefGoogle Scholar
  5. 5.
    Bunke, H., Riesen, K.: Recent advances in graph-based pattern recognition with applications in document analysis. Pattern Recognition 44(5), 1057–1067 (2011)zbMATHCrossRefGoogle Scholar
  6. 6.
    Budd, T.A.: Mutation Analysis of Program Test Data. PhD Thesis. Yale University, New Haven, Connecticut (1980)Google Scholar
  7. 7.
    Chein, M., Mugnier, M.L., Simonet, G.: Nested Graphs: A Graph-based Knowledge Representation Model with FOL Semantics. In: Proceedings of the 6th International Conference ”Principles of Knowledge Representation and Reasoning” (KR 1998), Trento, Italy, pp. 524–534. Morgan Kaufmann Publishers (June 1998)Google Scholar
  8. 8.
    Ji, C., Chen, Z., Xu, B., Zhao, Z.: A Novel Method of Mutation Clustering Based on Domain Analysis. In: Proceedings of the 21st International Conference on Software Engineering and Knowledge Engineering (SEKE 2009), July 1-3. Knowledge Systems Institute Graduate School, Boston (2009)Google Scholar
  9. 9.
    Chevalley, P.: Applying Mutation Analysis for Object-oriented Programs Using a Reflective Approach. In: Proceedings of the 8th Asia- Pacific Software Engineering Conference (APSEC 2001), Macau, China, December 4-7, p. 267 (2001)Google Scholar
  10. 10.
    Chevalley, P., Th’evenod-Fosse, P.: A Mutation Analysis Tool for Java Programs. International Journal on Software Tools for Technology Transfer 5(1), 90–103 (2002)CrossRefGoogle Scholar
  11. 11.
    Collins, M., Duffy, N.: New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron. In: Proceedings of ACL 2002 (2002)Google Scholar
  12. 12.
    DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on Test Data Selection: Help for the Practicing Programmer. Computer 11(4), 34–41 (1978)CrossRefGoogle Scholar
  13. 13.
    Gartner, T.: A survey of kernels for structured data. SIGKDD Explorations 5(1), 49–58 (2003)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Gartner, T.: Kernels for structured data. Series in Machine Perception and Artificial Intelligence. World Scientific (2009)Google Scholar
  15. 15.
    Han, J., Pei, J., Yin, Y., Mao, R.: Mining Frequent Patterns without Candidate Generation: A Frequent-pattern Tree Approach. Data Mining and Knowledge Discovery: An International Journal 8(1), 53–87 (2004)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Haussler, D.: Convolutional kernels on discrete structures. Technical Report UCSC-CRL-99-10, Computer Science Department, UC Santa Cruz (1999)Google Scholar
  17. 17.
    Hussain, S.: Mutation Clustering, Masters Thesis. King’s College London, Strand, London (2008)Google Scholar
  18. 18.
    Inokuchi, A., Washio, T., Motoda, H.: An Apriori-Based Algorithm for Mining Frequent Substructures from Graph Data. In: Zighed, D.A., Komorowski, J., Żytkow, J.M. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 13–23. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  19. 19.
    Jia, Y., Harman, M.: An Analysis and Survey of the Development of Mutation Testing. IEEE Trans. Software Eng., 649–678 (2011)Google Scholar
  20. 20.
    Kashima, H., Tsuda, K., Inokuchi, A.: Marginalized Kernels Between Labeled Graphs. In: ICML 2003, pp. 321–328 (2003)Google Scholar
  21. 21.
    Kim, S., Clark, J.A., McDermid, J.A.: Assessing Test Set Adequacy for Object Oriented Programs Using Class Mutation. In: Proceedings of the 3rd Symposium on Software Technology (SoST 1999), Buenos Aires, Argentina, September 8-9 (1999)Google Scholar
  22. 22.
    Kim, S., Clark, J.A., McDermid, J.A.: The Rigorous Generation of Java Mutation Operators Using HAZOP. In: Proceedings of the 12th International Cofference Software and Systems Engineering and their Applications (ICSSEA 1999), Paris, France, November 29-December 1 (1999)Google Scholar
  23. 23.
    Kim, S., Clark, J.A., McDermid, J.A.: Class Mutation: Mutation Testing for Object-oriented Programs. In: Proceedings of the Net. Object Days Conference on Object-Oriented Software Systems (2000)Google Scholar
  24. 24.
    Kim, S., Clark, J.A., McDermid, J.A.: Investigating the effectiveness of object-oriented testing strategies using the mutation method. In: Proceedings of the 1st Workshop on Mutation Analysis (MUTATION 2000), Published in Book Form, as Mutation Testing for the New Century, San Jose, California, October 6-7, pp. 207–225 (2001)Google Scholar
  25. 25.
    King, K.N., Offutt, A.J.: A Fortran Language System for Mutation- Based Software Testing. Software: Practice and Experience 21(7), 685–718 (1991)CrossRefGoogle Scholar
  26. 26.
    Liwicki, M., Bunke, H., Pittman, J.A., Knerr, S.: Combining diverse systems for handwritten text line recognition. Mach. Vis. Appl. 22(1), 39–51 (2011)CrossRefGoogle Scholar
  27. 27.
    Liwicki, M., Schlapbach, A., Bunke, H.: Automatic gender detection using on-line and off-line information. Pattern Anal. Appl. 14(1), 87–92 (2011)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Preller, A., Mugnier, M.-L., Chein, M.: Logic for Nested Graphs. Computational Intelligence 14(3), 335–357 (1998)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Ma, Y., Offutt, J., Kwon, Y.R.: MuJava: a mutation system for java. In: ICSE, pp. 827–830 (2006)Google Scholar
  30. 30.
    Mathur, A.P.: Performance, Effectiveness, and Reliability Issues in Software Testing. In: Proceedings of the 5th International Computer Software and Applications Conference (COMPSAC 1979), Tokyo, Japan, September 11-13, pp. 604–605 (1991)Google Scholar
  31. 31.
    Mathur, A.P., Wong, W.E.: An Empirical Comparison of Mutation and Data Flow Based Test Adequacy Criteria, Purdue University, West Lafayette, Indiana, Technique Report (1993)Google Scholar
  32. 32.
    Offutt, A.J., Rothermel, G., Zapf, C.: An Experimental Evaluation of Selective Mutation. In: Proceedings of the 15th International Conference on Software Engineering (ICSE 1993), pp. 100–107. IEEE Computer Society Press, Baltimore (1993)CrossRefGoogle Scholar
  33. 33.
    Richiardi, J., Van De Ville, D., Riesen, K., Bunke, H.: Vector Space Embedding of Undirected Graphs with Fixed-cardinality Vertex Sequences for Classification. In: ICPR 2010, pp. 902–905 (2010)Google Scholar
  34. 34.
    Riesen, K., Bunke, H.: Cluster Ensembles Based on Vector Space Embeddings of Graphs. In: Benediktsson, J.A., Kittler, J., Roli, F. (eds.) MCS 2009. LNCS, vol. 5519, pp. 211–221. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  35. 35.
    Riesen, K., Bunke, H.: Dissimilarity Based Vector Space Embedding of Graphs Using Prototype Reduction Schemes. In: Perner, P. (ed.) MLDM 2009. LNCS, vol. 5632, pp. 617–631. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  36. 36.
    Riesen, K., Bunke, H.: Reducing the dimensionality of dissimilarity space embedding graph kernels. Eng. Appl. of AI 22(1), 48–56 (2009)CrossRefGoogle Scholar
  37. 37.
    Rozenberg, G.: Handbook of Graph Grammars and Computing by Graph. Transformations. Fundations, vol. 1. World Scientific, London (1997)zbMATHCrossRefGoogle Scholar
  38. 38.
    Rozenberg, G.: Handbook of Graph Grammars and Computing by Graph. Transformations. Applications, Languages and Tools, vol. 2. World Scientific, London (1999)Google Scholar
  39. 39.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. 1-462 (2004)Google Scholar
  40. 40.
    Schölkopf, B., Smola, A.J.: A Short Introduction to Learning with Kernels. In: Mendelson, S., Smola, A.J. (eds.) Advanced Lectures on Machine Learning. LNCS (LNAI), vol. 2600, pp. 41–64. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  41. 41.
    Schölkopf, B., Smola, A.J.: Learning with kernels. MIT Press, Cambridge (2002)Google Scholar
  42. 42.
    Strug, B.: Using Kernels on Hierarchical Graphs in Automatic Classification of Designs. In: Jiang, X., Ferrer, M., Torsello, A. (eds.) GbRPR 2011. LNCS, vol. 6658, pp. 335–344. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  43. 43.
    Tsuda, K., Kin, T., Asai, K.: Marginalized kernels for biological sequences. Bioinformatics 18, 268–275Google Scholar
  44. 44.
    Vishwanathan, S.V.N., Borgwardt, K.M., Schraudolph, N.N.: Fast Computation of Graph Kernels. In: NIPS 2006, pp. 1449–1456 (2006)Google Scholar
  45. 45.
    Wong, W.E.: On Mutation and Data Flow. PhD Thesis, Purdue University, West Lafayette, Indiana (1993)Google Scholar
  46. 46.
    Yan, X., Yu, P.S., Han, J.: Substructure Similarity Search in Graph Databases. In: Proc. of 2005 Int. Conf. on Management of Data, SIGMOD 2005 (2005)Google Scholar
  47. 47.
    Yan, X., Yu, P.S., Han, J.: Graph Indexing: A Frequent Structure-based Approach. In: Proc. of 2004 Int. Conf. on Management of Data, SIGMOD 2004 (2004)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2012

Authors and Affiliations

  • Joanna Strug
    • 1
  • Barbara Strug
    • 2
  1. 1.Faculty of Electrical and Computer EngineeringCracow University of TechnologyKrakowPoland
  2. 2.Department of Physics, Astronomy and Applied Computer ScienceJagiellonian UniversityKrakowPoland

Personalised recommendations