Abstract
Automated program repair has drawn significant attention in recent years for its potential to alleviate the heavy burden of debugging activities. Fault localization, which generally provides a rank list of suspicious statements, is necessary for automated repair tools to identify the fault. With such rank list, existing repair tools have two statement selecting strategies for statement modification: suspiciousness-first algorithm (SFA) based on the suspiciousness of statements and rank-first algorithm (RFA) relying on the rank of statements. However, despite the extensive application of SFA and RFA in repair tools and different selecting methods between both strategies, little is known about the performance of the two strategies in automated program repair.
In this paper we conduct an empirical study to compare the effectiveness of SFA and RFA in automated program repair. We implement SFA and RFA as well as 6 popular spectrum-based fault localization techniques into 4 automated program repair tools, and then use these tools to perform repair experiments on 44 real-world bugs from Defects4J to compare SFA against RFA in automated program repair. The results suggest that: (1) RFA performs statistically significantly better than SFA in 64.76% cases according to the number of candidate patches generated before a valid patch is found (NCP), (2) SFA achieves better performance in parallel repair and patch diversity, (3) The performance of SFA can be improved by increasing the suspiciousness accuracy of fault localization techniques. These observations provide directions for future research on the usage of statement selecting strategies and a new avenue for improving the effectiveness of fault localization techniques in automated program repair.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Gazzola, L., Micucci, D., Mariani, L.: Automatic software repair: a survey. In: ICSE (2018)
Qi, Y., et al.: Using automated program repair for evaluating the effectiveness of fault localization techniques. In: ISSTA (2013)
Wong, W.E., et al.: A survey on software fault localization. IEEE Trans. Softw. Eng. (TSE) 42(8), 707–740 (2016)
Assiri, F.Y., Bieman, J.M.: Fault localization for automated program repair: effectiveness, performance, repair correctness. Softw. Qual. J. (SQJ) 25(1), 171–199 (2017)
Monperrus, M.: Automatic software repair: a bibliography. ACM Comput. Surv. (CSUR) 51(1), 17 (2018)
Chen, L., Pei, Y., Furia, C.A.: Contract-based program repair without the contracts. In: ASE (2017)
Saha, R.K., et al.: ELIXIR: effective object oriented program repair. In: ASE (2017)
Mechtaev, S., Yi, J., Roychoudhury, A.: Angelix: scalable multiline program patch synthesis via symbolic analysis. In: ICSE (2016)
Le, X.-B.D., et al.: JFIX: semantics-based repair of Java programs via symbolic PathFinder. In: ISSTA (2017)
Martinez, M., Monperrus, M.: Astor: a program repair library for Java. In: ISSTA (2016)
Xiong, Y., et al.: Precise condition synthesis for program repair. In: ICSE (2017)
Xuan, J., et al.: Nopol: automatic repair of conditional statement bugs in Java programs. IEEE Trans. Softw. Eng. (TSE) 43(1), 34–55 (2017)
Pearson, S., et al.: Evaluating and improving fault localization. In: ICSE (2017)
Yang, D., Qi, Y., Mao, X.: An empirical study on the usage of fault localization in automated program repair. In: 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME NIER Track). IEEE (2017)
Kim, D., et al.: Automatic patch generation learned from human-written patches. In: ICSE (2013)
Le Goues, C., et al.: Genprog: a generic method for automatic software repair. TSE 38(1), 54–72 (2012)
Qi, Y., et al.: The strength of random search on automated program repair. In: ICSE (2014)
Le, X.B.D., Lo, D., Le Goues, C.: History driven program repair. In: SANER (2016)
Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: ISSTA (2014)
Keller, F., et al.: A critical evaluation of spectrum-based fault localization techniques on a large-scale software system. In: 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS). IEEE (2017)
Xie, X., et al.: A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. TOSEM 22(4), 31 (2013)
Parnin, C., Orso, A.: Are automated debugging techniques actually helping programmers? In: ISSTA (2011)
Kochhar, P.S., et al.: Practitioners’ expectations on automated fault localization. In: ISSTA (2016)
Moon, S., et al.: Ask the mutants: mutating faulty programs for fault localization. In: ICST (2014)
Xiong, Y., et al.: Identifying patch correctness in test-based program repair. In: ICSE (2018)
Ye, H., Martinez, M., Monperrus, M.: A comprehensive study of automatic program repair on the QuixBugs benchmark. arXiv preprint arXiv:1805.03454 (2018)
Martinez, M., et al.: Automatic repair of real bugs in Java: a large-scale experiment on the Defects4J dataset. Empir. Softw. Eng. 22(4), 1936–1964 (2017)
Tanikado, A., et al.: New strategies for selecting reuse candidates on automated program repair. In: COMPSAC (2017)
Xin, Q., Reiss, S.P.: Leveraging syntax-related code for automated program repair. In: ASE (2017)
Qi, Z., et al.: An analysis of patch plausibility and correctness for generate-and-validate patch generation systems. In: ISSTA (2015)
Durieux, T., et al.: The patches of the Nopol automatic repair system on the bugs of Defects4J version 1.1. 0. Dissertation, Université Lille 1-Sciences et Technologies (2017)
Arcuri, A., Briand, L.: A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: ICSE (2011)
Xin, Q., Reiss, S.P.: Identifying test-suite-overfitted patches through test case generation. In: ISSTA (2017)
Tao, Y., et al.: Automatically generated patches as debugging aids: a human study. In: FSE (2014)
Mechtaev, S., et al.: Semantic program repair using a reference implementation. In: ICSE (2018)
Jia Y., et al.: Finding and fixing software bugs automatically with SapFix and Sapienz. https://code.fb.com/developer-tools/finding-and-fixing-software-bugs-automatically-with-sapfix-and-sapienz/ (2018)
Wong, W.E., et al.: The DStar method for effective software fault localization. IEEE Trans. Reliab. 63, 290–308 (2014)
Wen, M., et al.: Context-aware patch generation for better automated program repair. In: ICSE (2018)
Just, R., et al.: Comparing developer-provided to user-provided tests for fault localization and automated program repair. In: ISSTA (2018)
Jiang, J., et al.: Shaping program repair space with existing patches and similar code. In: ISSTA (2018)
Yang, D.: SFA-RFA. https://github.com/DehengYang/sfa-rfa (2018)
Le, X.-B.D., et al.: S3: syntax-and semantic-guided repair synthesis via programming by examples. In: FSE (2017)
Acknowledgements
This research was partially supported by National Natural Science Foundation of China (61502015, 61672529, 61379054, 61602504).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, D., Qi, Y., Mao, X. (2018). Evaluating the Strategies of Statement Selection in Automated Program Repair. In: Bu, L., Xiong, Y. (eds) Software Analysis, Testing, and Evolution. SATE 2018. Lecture Notes in Computer Science(), vol 11293. Springer, Cham. https://doi.org/10.1007/978-3-030-04272-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-04272-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04271-4
Online ISBN: 978-3-030-04272-1
eBook Packages: Computer ScienceComputer Science (R0)