An Investigation into the Use of Mutation Analysis for Automated Program Repair

  • Christopher Steven TimperleyEmail author
  • Susan Stepney
  • Claire Le Goues
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10452)


Research in Search-Based Automated Program Repair has demonstrated promising results, but has nevertheless been largely confined to small, single-edit patches using a limited set of mutation operators. Tackling a broader spectrum of bugs will require multiple edits and a larger set of operators, leading to a combinatorial explosion of the search space. This motivates the need for more efficient search techniques. We propose to use the test case results of candidate patches to localise suitable fix locations. We analysed the test suite results of single-edit patches, generated from a random walk across 28 bugs in 6 programs. Based on the findings of this analysis, we propose a number of mutation-based fault localisation techniques, which we subsequently evaluate by measuring how accurately they locate the statements at which the search was able to generate a solution. After demonstrating that these techniques fail to result in a significant improvement, we discuss why this may be the case, despite the successes of mutation-based fault localisation in previous studies.


Automated program repair Mutation analysis Fault localisation 



This research was partially funded by AFRL (#FA8750-15-2-0075) and DARPA (#FA8750-16-2-0042), and an EPSRC DTG; the authors are grateful for their support. Any opinions, findings, or recommendations expressed are those of the authors and do not necessarily reflect those of the US Government. The authors additionally wish to thank the anonymous reviewers, whose comments were especially insightful and constructive.


  1. 1.
    Cambridge University Study States Software Bugs Cost Economy $312 Billion Per Year. Accessed Apr 2017
  2. 2.
    Arcuri, A.: Evolutionary repair of faulty software. Appl. Soft Comput. 11(4), 3494–3514 (2011)CrossRefGoogle Scholar
  3. 3.
    Do, H., Elbaum, S., Rothermel, G.: Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empirical Software Eng. 10(4), 405–435 (2005)CrossRefGoogle Scholar
  4. 4.
    Fast, E., Le Goues, C., Forrest, S., Weimer, W.: Designing better fitness functions for automated program repair. In: Genetic and Evolutionary Computation Conference, GECCO 2010, pp. 965–972 (2010)Google Scholar
  5. 5.
    Ke, Y., Stolee, K.T., Le Goues, C., Brun, Y.: Repairing programs with semantic code search. In: Automated Software Engineering, ASE 2015, pp. 295–306 (2015)Google Scholar
  6. 6.
    Kim, D., Nam, J., Song, J., Kim, S.: Automatic patch generation learned from human-written patches. In: International Conference on Software Engineering, ICSE 2013, pp. 802–811 (2013)Google Scholar
  7. 7.
    Le, X.B.D., Lo, D., Goues, C.L.: History driven program repair. In: International Conference on Software Analysis, Evolution, and Reengineering, SANER 2016, vol. 1, pp. 213–224 (2016)Google Scholar
  8. 8.
    Le Goues, C., Dewey-Vogt, M., Forrest, S., Weimer, W.: A systematic study of automated program repair: fixing 55 out of 105 bugs for $8 each. In: International Conference on Software Engineering, ICSE 2012, pp. 3–13 (2012)Google Scholar
  9. 9.
    Le Goues, C., Weimer, W., Forrest, S.: Representations and operators for improving evolutionary software repair. In: Genetic and Evolutionary Computation Conference, GECCO 2012, pp. 959–966 (2012)Google Scholar
  10. 10.
    Long, F., Rinard, M.: Staged program repair with condition synthesis. In: Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, pp. 166–178 (2015)Google Scholar
  11. 11.
    Long, F., Rinard, M.: Automatic patch generation by learning correct code. In: Principles of Programming Languages, POPL 2016, pp. 298–312 (2016)Google Scholar
  12. 12.
    Mechtaev, S., Yi, J., Roychoudhury, A.: Angelix: scalable multiline program patch synthesis via symbolic analysis. In: International Conference on Software Engineering, ICSE 2016, pp. 691–701 (2016)Google Scholar
  13. 13.
    Monperrus, M.: A critical review of automatic patch generation learned from human-written patches: essay on the problem statement and the evaluation of automatic software repair. In: International Conference on Software Engineering, ICSE 2014, pp. 234–242 (2014)Google Scholar
  14. 14.
    Moon, S., Kim, Y., Kim, M., Yoo, S.: Ask the mutants: mutating faulty programs for fault localization. In: International Conference on Software Testing, Verification and Validation, ICST 2014, pp. 153–162 (2014)Google Scholar
  15. 15.
    Moon, S., Kim, Y., Kim, M., Yoo, S.: Hybrid-MUSE: mutating faulty programs for precise fault localization. Technical report, KAIST (2014)Google Scholar
  16. 16.
    Papadakis, M., Le Traon, Y.: Metallaxis-FL: mutation-based fault localization. Softw. Test. Verification Reliab. 25(5–7), 605–628 (2015)CrossRefGoogle Scholar
  17. 17.
    Qi, Y., Mao, X., Lei, Y.: Efficient automated program repair through fault-recorded testing prioritization. In: International Conference on Software Maintenance, pp. 180–189 (2013)Google Scholar
  18. 18.
    Qi, Y., Mao, X., Lei, Y., Dai, Z., Wang, C.: The strength of random search on automated program repair. In: International Conference on Software Engineering, ICSE 2014, pp. 254–265 (2014)Google Scholar
  19. 19.
    Qi, Y., Mao, X., Lei, Y., Wang, C.: Using automated program repair for evaluating the effectiveness of fault localization techniques. In: International Symposium on Software Testing and Analysis, ISSTA 2013, pp. 191–201 (2013)Google Scholar
  20. 20.
    Schulte, E., Fry, Z.P., Fast, E., Weimer, W., Forrest, S.: Software mutational robustness. Genet. Program Evolvable Mach. 15(3), 281–312 (2013)CrossRefGoogle Scholar
  21. 21.
    Soto, M., Thung, F., Wong, C.P., Le Goues, C., Lo, D.: A deeper look into bug fixes: patterns, replacements, deletions, and additions. In: International Conference on Mining Software Repositories, MSR 2016, pp. 512–515 (2016)Google Scholar
  22. 22.
    Vargha, A., Delaney, H.D.: A critique and improvement of the CL common language effect size statistics of McGraw and Wong. J. Educ. Behav. Stat. 25(2), 101–132 (2000)Google Scholar
  23. 23.
    Weimer, W., Fry, Z.P., Forrest, S.: Leveraging program equivalence for adaptive program repair: Models and first results. In: International Conference on Automated Software Engineering, ASE 2013, pp. 356–366 (2013)Google Scholar
  24. 24.
    Weiser, M.: Program slicing. In: International Conference on Software Engineering, ICSE 1981, pp. 439–449 (1981)Google Scholar
  25. 25.
    Yoo, S.: Evolving human competitive spectra-based fault localisation techniques. In: Fraser, G., Teixeira de Souza, J. (eds.) SSBSE 2012. LNCS, vol. 7515, pp. 244–258. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33119-0_18 CrossRefGoogle Scholar
  26. 26.
    Zeller, A., Hildebrandt, R.: Simplifying and isolating failure-inducing input. IEEE Trans. Software Eng. 28(2), 183–200 (2002)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Christopher Steven Timperley
    • 1
    Email author
  • Susan Stepney
    • 2
  • Claire Le Goues
    • 1
  1. 1.Carnegie Mellon UniversityPittsburghUSA
  2. 2.University of YorkYorkUK

Personalised recommendations