Exploring Fitness and Edit Distance of Mutated Python Programs

  • Saemundur O. Haraldsson
  • John R. Woodward
  • Alexander E. I. Brownlee
  • David Cairns
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10196)

Abstract

Genetic Improvement (GI) is the process of using computational search techniques to improve existing software e.g. in terms of execution time, power consumption or correctness. As in most heuristic search algorithms, the search is guided by fitness with GI searching the space of program variants of the original software. The relationship between the program space and fitness is seldom simple and often quite difficult to analyse. This paper makes a preliminary analysis of GI’s fitness distance measure on program repair with three small Python programs. Each program undergoes incremental mutations while the change in fitness as measured by proportion of tests passed is monitored. We conclude that the fitnesses of these programs often does not change with single mutations and we also confirm the inherent discreteness of bug fixing fitness functions. Although our findings cannot be assumed to be general for other software they provide us with interesting directions for further investigation.

Keywords

Search Based Software Engineering Genetic Improvement Genetic Programming Automatic programming Software repair 

References

  1. 1.
    Ackling, T., Alexander, B., Grunert, I.: Evolving patches for software repair. In: 13th Annual Conference on Genetic and Evolutionary Computation, GECCO 2011, pp. 1427–1434. ACM, Dublin, July 2011Google Scholar
  2. 2.
    Arcuri, A.: On the automation of fixing software bugs. In: Companion of the 30th International Conference on Software Engineering, ICSE Companion 2008, pp. 1003–1006. ACM, New York (2008)Google Scholar
  3. 3.
    Arcuri, A., Yao, X.: A novel co-evolutionary approach to automatic software bug fixing. In: 2008 IEEE World Congress on Computational Intelligence, pp. 162–168. IEEE Computational Intelligence Society (2008)Google Scholar
  4. 4.
    Bradbury, J.S., Jalbert, K.: Automatic repair of concurrency bugs. In: Proceedings of the 2nd International Symposium on Search Based Software Engineering, p. 2 (2010)Google Scholar
  5. 5.
    Bruce, B.R.: Energy optimisation via genetic improvement a SBSE technique for a new era in software development. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO Companion 2015, pp. 819–820. ACM, Madrid, July 2015Google Scholar
  6. 6.
    Bruce, B.R., Petke, J., Harman, M.: Reducing energy consumption using genetic improvement. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO 2015, pp. 1327–1334. ACM, Madrid, July 2015Google Scholar
  7. 7.
    Burles, N., Bowles, E., Brownlee, A.E.I., Kocsis, Z.A., Swan, J., Veerapen, N.: Object-oriented genetic improvement for improved energy consumption in Google Guava. In: Barros, M., Labiche, Y. (eds.) SSBSE 2015. LNCS, vol. 9275, pp. 255–261. Springer, Cham (2015). doi:10.1007/978-3-319-22183-0_20 CrossRefGoogle Scholar
  8. 8.
    Burles, N., Swan, J., Brownlee, A.E.I., Kocsis, Z.A., Veerapen, N.: Embedded dynamic improvement. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO Companion 2015, pp. 831–832. ACM, Madrid, July 2015Google Scholar
  9. 9.
    Cody-Kenny, B., Barrett, S.: The emergence of useful bias in self-focusing genetic programming for software optimisation. In: Ruhe, G., Zhang, Y. (eds.) SSBSE 2013. LNCS, vol. 8084, pp. 306–311. Springer, Heidelberg (2013). doi:10.1007/978-3-642-39742-4_29 CrossRefGoogle Scholar
  10. 10.
    Cody-kenny, B., Galván-lópez, E., Barrett, S.: locoGP: improving performance by genetic programming java source code. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO Companion 2015, pp. 811–818. ACM, Madrid, July 2015Google Scholar
  11. 11.
    Forrest, S., Nguyen, T., Weimer, W., Goues, C.L.: A genetic programming approach to automated software repair. In: Genetic and Evolutionary Computation Conference, pp. 947–954 (2009)Google Scholar
  12. 12.
    Haraldsson, S.O., Woodward, J.R.: Automated design of algorithms and genetic improvement: contrast and commonalities. In: Proceedings of the 2014 Conference Companion on Genetic and Evolutionary Computation Companion, GECCO Comp 2014, pp. 1373–1380. ACM, Vancouver, July 2014Google Scholar
  13. 13.
    Haraldsson, S.O., Woodward, J.R.: Genetic improvement of energy usage is only as reliable as the measurements are accurate. In: Proceedings of the 2015 Conference Companion on Genetic and Evolutionary Computation Companion, pp. 831–832. ACM, Madrid (2015)Google Scholar
  14. 14.
    Harman, M., Jia, Y., Langdon, W.B.: Babel Pidgin: SBSE can grow and graft entirely new functionality into a real world system. In: Goues, C., Yoo, S. (eds.) SSBSE 2014. LNCS, vol. 8636, pp. 247–252. Springer, Heidelberg (2014). doi:10.1007/978-3-319-09940-8_20 Google Scholar
  15. 15.
    Kocsis, Z.A., Neumann, G., Swan, J., Epitropakis, M.G., Brownlee, A.E.I., Haraldsson, S.O., Bowles, E.: Repairing and optimizing hadoop hashCode implementations. In: Goues, C., Yoo, S. (eds.) SSBSE 2014. LNCS, vol. 8636, pp. 259–264. Springer, Heidelberg (2014). doi:10.1007/978-3-319-09940-8_22 Google Scholar
  16. 16.
    Krawiec, K., Swan, J.: Pattern-guided genetic programming. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, GECCO 2013, pp. 949–956. ACM, Amsterdam (2013)Google Scholar
  17. 17.
    Langdon, W.B.: Genetic improvement of programs. In: 18th International Conference on Soft Computing, MENDEL 2012 (2012)Google Scholar
  18. 18.
    Langdon, W.B.: Improved CUDA 3D medical image registration. In: UK Many-Core Developer Conference 2014, UKMAC 2014, December 2014Google Scholar
  19. 19.
    Langdon, W.B.: Performance of genetic programming optimised Bowtie2 on genome comparison and analytic testing (GCAT) benchmarks. BioData Min. 8(1), 1 (2015)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Langdon, W.B., Harman, M.: Genetically improved CUDA C++ software. In: Nicolau, M., Krawiec, K., Heywood, M.I., Castelli, M., García-Sánchez, P., Merelo, J.J., Rivas Santos, V.M., Sim, K. (eds.) EuroGP 2014. LNCS, vol. 8599, pp. 87–99. Springer, Heidelberg (2014). doi:10.1007/978-3-662-44303-3_8 Google Scholar
  21. 21.
    Langdon, W., Harman, M.: Genetically improved CUDA kernels for StereoCamera. Technical report, UCL Department of Computer Science, London, UK (2014)Google Scholar
  22. 22.
    Langdon, W.B., Harman, M.: Grow and Graft a better CUDA pknotsRG for RNA pseudoknot free energy calculation. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO Companion 2015, pp. 805–810. ACM, Madrid, July 2015Google Scholar
  23. 23.
    Langdon, W.B., Lam, B.Y.H., Petke, J., Harman, M.: Improving CUDA DNA analysis software with genetic programming. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO 2015, pp. 1063–1070. ACM, Madrid, July 2015Google Scholar
  24. 24.
    Le Goues, C., Dewey-Vogt, M., Forrest, S., Weimer, W.: A systematic study of automated program repair: fixing 55 out of 105 bugs for $8 each. In: 2012 34th International Conference on Software Engineering (ICSE). pp. 3–13. IEEE, Zurich, June 2012Google Scholar
  25. 25.
    Le Goues, C., Forrest, S., Weimer, W.: Current challenges in automatic software repair. Softw. Qual. J. 21(3), 421–443 (2013)CrossRefGoogle Scholar
  26. 26.
    Le Goues, C., Nguyen, T., Forrest, S., Weimer, W.: GenProg: a generic method for automatic software repair. IEEE Trans. Softw. Eng. 38(1), 54–72 (2012)CrossRefGoogle Scholar
  27. 27.
    Marginean, A., Barr, E.T., Harman, M., Jia, Y.: Automated transplantation of call graph and layout features into Kate. In: Barros, M., Labiche, Y. (eds.) SSBSE 2015. LNCS, vol. 9275, pp. 262–268. Springer, Heidelberg (2015). doi:10.1007/978-3-319-22183-0_21 CrossRefGoogle Scholar
  28. 28.
    Nguyen, H.D.T., Qi, D., Roychoudhury, A., Chandra, S.: SemFix: program repair via semantic analysis. In: Proceedings - International Conference on Software Engineering, pp. 772–781 (2013)Google Scholar
  29. 29.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetMATHGoogle Scholar
  30. 30.
    Petke, J., Harman, M., Langdon, W.B., Weimer, W.: Using genetic improvement and code transplants to specialise a C++ program to a problem class. In: Nicolau, M., Krawiec, K., Heywood, M.I., Castelli, M., García-Sánchez, P., Merelo, J.J., Rivas Santos, V.M., Sim, K. (eds.) EuroGP 2014. LNCS, vol. 8599, pp. 137–149. Springer, Heidelberg (2014). doi:10.1007/978-3-662-44303-3_12 Google Scholar
  31. 31.
    Qi, Z., Long, F., Achour, S., Rinard, M.: An analysis of patch plausibility and correctness for generate-and-validate patch generation systems. In: Xie, T. (ed.) ISSTA 2015, pp. 24–36. ACM, Baltimore (2015)CrossRefGoogle Scholar
  32. 32.
    Risco-Martín, J.L., Colmenar, J.M., Hidalgo, J.I., Lanchares, J., Díaz, J.: A methodology to automatically optimize dynamic memory managers applying grammatical evolution. J. Syst. Softw. 91, 109–123 (2014)CrossRefGoogle Scholar
  33. 33.
    Ryan, C., Collins, J.J., Neill, M.O.: Grammatical evolution: evolving programs for an arbitrary language. In: Banzhaf, W., Poli, R., Schoenauer, M., Fogarty, T.C. (eds.) EuroGP 1998. LNCS, vol. 1391, pp. 83–96. Springer, Heidelberg (1998). doi:10.1007/BFb0055930 CrossRefGoogle Scholar
  34. 34.
    Smith, E.K., Barr, E.T., Le Goues, C., Brun, Y.: Is the cure worse than the disease? Overfitting in automated program repair. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, pp. 532–543. ACM, Bergamo (2015)Google Scholar
  35. 35.
    Swan, J., Epitropakis, M.G., Woodward, J.R.: Gen-O-Fix: an embeddable framework for dynamic adaptive genetic improvement programming. Technical report CSM-195, Department of Computing Science and Mathematics University of Stirling, Stirling, UK (2014)Google Scholar
  36. 36.
    Tomassini, M., Vanneschi, L., Collard, P., Clergue, M.: A study of fitness distance correlation as a difficulty measure in genetic programming. Evol. Comput. 13(2), 213–239 (2005)CrossRefMATHGoogle Scholar
  37. 37.
    Weimer, W., Forrest, S., Le Goues, C., Nguyen, T.: Automatic program repair with evolutionary computation. Commun. ACM 53(5), 109 (2010)CrossRefGoogle Scholar
  38. 38.
    Weimer, W., Fry, Z.P., Forrest, S.: Leveraging program equivalence for adaptive program repair: models and first results. In: 28th IEEE/ACM International Conference on Automated Software Engineering, Palo Alto, USA, pp. 356–366, November 2013Google Scholar
  39. 39.
    Weimer, W., Nguyen, T., Le Goues, C., Forrest, S.: Automatically finding patches using genetic programming. In: Proceedings of the 31st International Conference on Software Engineering, pp. 364–374. IEEE, Vancouver (2009)Google Scholar
  40. 40.
    White, D.R.: An unsystematic review of genetic improvement. In: 45th CREST Open Workshop on Genetic Improvement, London (2016)Google Scholar
  41. 41.
    White, D.R., Arcuri, A., Clark, J.A.: Evolutionary improvement of programs. IEEE Trans. Evol. Comput. 15(4), 515–538 (2011)CrossRefGoogle Scholar
  42. 42.
    White, D.R., Clark, J., Jacob, J., Poulding, S.M.: Searching for resource-efficient programs. In: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, GECCO 2008, vol. 1, p. 1775 (2008)Google Scholar
  43. 43.
    White, D.R.: Genetic programming for low-resource systems, December 2009Google Scholar
  44. 44.
    Wu, F., Weimer, W., Harman, M., Jia, Y., Krinke, J.: Deep parameter optimisation. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO 2015, pp. 1375–1382. ACM, Madrid, July 2015Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Saemundur O. Haraldsson
    • 1
  • John R. Woodward
    • 1
  • Alexander E. I. Brownlee
    • 1
  • David Cairns
    • 1
  1. 1.University of StirlingStirlingScotland

Personalised recommendations