Automatic software refactoring: a systematic literature review

Abstract

Refactoring a software artifact is an embedded task in the maintenance phase of the software life cycle. To reduce the time and effort required for this task, researchers proposed methods to automate the software refactoring process at the design and code levels. In this paper, we conducted a systematic literature review of papers that suggest, propose, or implement an automated refactoring process. Using different phases, setting several quality measures, and snowballing, only 41 papers passed to the last stage to be analyzed and reviewed. We observe an increase in the number of papers that propose automatic refactoring. The results show that while most of the papers discuss code refactoring, only a few recent papers are focused on model refactoring. Search-based refactoring is gaining more popularity, and several researchers have used it to perform refactoring in a quick and efficient manner.

Appendices

Appendix A. Final selected papers

The final selected papers are as follows:

[1] R. Mahouachi, M. Kessentini, and K. Ghedira, A new design defects classification: marrying detection and correction, presented at the Proceedings of the 15th international conference on Fundamental Approaches to Software Engineering, Tallinn, Estonia, 2012.

[2] A.-R. Han, D.-H. Bae, and S. Cha, An efficient approach to identify multiple and independent Move Method refactoring candidates, Information and Software Technology, vol. 59, pp. 53-66, 2015.

[3] G. Bavota, A. De Lucia, A. Marcus, and R. Oliveto, Automating extract class refactoring: an improved method and its evaluation, Empirical Software Engineering, vol. 19, no. 6, pp. 1617-1664, Dec 2014.

[4] S. Demeyer, S. Ducasse, and O. Nierstrasz, Finding refactorings via change metrics, in 15th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, 2000, vol. 35, no. 10, pp. 166-177.

[5] D. Romano, S. Raemaekers, and M. Pinzger, Refactoring Fat Interfaces Using a Genetic Algorithm, presented at the Proceedings of the 2014 IEEE International Conference on Software Maintenance and Evolution, 2014.

[6] K. Praditwong, M. Harman, and Y. Xin, Software Module Clustering as a Multi-Objective Search Problem, IEEE Transactions on Software Engineering, vol. 37, no. 2, pp. 264-282, 2011.

[7] R. Dijkman, B. Gfeller, J. Küster, and H. Völzer, Identifying refactoring opportunities in process model repositories, Information and Software Technology, vol. 53, no. 9, pp. 937-948, 2011.

[8] A. Ouni, M. Kessentini, H. Sahraoui, K. Inoue, and M. S. Hamdi, Improving multi-objective code-smells correction using development history, Journal of Systems and Software, vol. 105, pp. 18-39, 2015.

[9] R. Tairas and J. Gray, Increasing clone maintenance support by unifying clone detection and refactoring activities, Information and Software Technology, vol. 54, no. 12, pp. 1297-1307, 2012.

[10] Z. Ujhelyi et al., Performance comparison of query-based techniques for anti-pattern detection, Information and Software Technology, vol. 65, pp. 147-165, 2015.

[11] B. F. d. Santos Neto, M. Ribeiro, V. T. d. Silva, C. Braga, C. J. P. d. Lucena, and E. d. B. Costa, AutoRefactoring: A platform to build refactoring agents, Expert Systems with Applications, vol. 42, no. 3, pp. 1652-1664, 2015.

[12] G. Ganea, I. Verebi, and R. Marinescu, Continuous quality assessment with inCode, Science of Computer Programming, vol. 134, pp. 19-36, 2017.

[13] M. Fokaefs, N. Tsantalis, E. Stroulia, and A. Chatzigeorgiou, Identification and application of Extract Class refactorings in object-oriented systems, Journal of Systems and Software, vol. 85, no. 10, pp. 2241-2260, 2012.

[14] N. Tsantalis and A. Chatzigeorgiou, Identification of refactoring opportunities introducing polymorphism, Journal of Systems and Software, vol. 83, no. 3, pp. 391-404, 2010.

[15] G. Bavota, A. De Lucia, and R. Oliveto, Identifying Extract Class refactoring opportunities using structural and semantic cohesion measures, Journal of Systems and Software, vol. 84, no. 3, pp. 397-414, 2011.

[16] M. W. Mkaouer, M. Kessentini, S. Bechikh, K. Deb, and M. Ó. Cinnéide, High dimensional search-based software engineering: finding tradeoffs among 15 objectives for automating software refactoring using NSGA-III, presented at the Proceedings of the 2014 conference on Genetic and evolutionary computation, Vancouver, BC, Canada, 2014.

[17] A. Ouni, M. Kessentini, H. Sahraoui, and M. Boukadoum, Maintainability defects detection and correction: a multi-objective approach, Automated Software Engg., vol. 20, no. 1, pp. 47-79, 2013.

[18] M. W. Mkaouer, M. Kessentini, S. Bechikh, K. Deb, and M. Ó. Cinnéide, Recommendation system for software refactoring using innovization and interactive dynamic optimization, presented at the Proceedings of the 29th ACM/IEEE international conference on Automated software engineering, Vasteras, Sweden, 2014.

[19] W. Wang and M. W. Godfrey, Recommending Clones for Refactoring Using Design, Context, and History, presented at the Proceedings of the 2014 IEEE International Conference on Software Maintenance and Evolution, 2014.

[20] W.-F. Pan, B. Jiang, and B. Li, Refactoring Software Packages via Community Detection in Complex Software Networks, (in English), International Journal of Automation and Computing, vol. 10, no. 2, pp. 157-166, 2013.

[21] M. Kessentini, H. Sahraoui, M. Boukadoum, and M. Wimmer, Search-based design defects detection by example, in Proceedings of the 14th international conference on Fundamental approaches to software engineering: part of the joint European conferences on theory and practice of software, Germany, 2011, pp. 401-415.

[22] P. Langer, M. Kessentini, A. b. Fadhel, and M. Wimmer, Search-based detection of high-level model changes, presented at the Proceedings of the 2012 IEEE International Conference on Software Maintenance (ICSM), 2012.

[23] M. W. Mkaouer, M. Kessentini, S. Bechikh, and M. O. Cinneide, A Robust Multi-objective Approach for Software Refactoring under Uncertainty, in Search-Based Software Engineering, vol. 8636, C. LeGoues and S. Yoo, Eds. (Lecture Notes in Computer Science, 2014, pp. 168-183.

[24] M. O'Keeffe and M. Cinnéide, Automated Design Improvement by Example, in New Trends in Software Methodologies, Tools and Techniques, vol. 161, 2007, pp. 315-329.

[25] D. Dig, C. Comertoglu, D. Marinov, and R. Johnson, Automated detection of refactorings in evolving components, in Proceedings of the 20th European conference on Object-Oriented Programming, 2006, vol. 4067, pp. 404-428.

[26] G. Bavota, M. Gethers, R. Oliveto, D. Poshyvanyk, and A. De Lucia, Improving Software Modularization via Automated Analysis of Latent Topics and Dependencies, Acm Transactions on Software Engineering and Methodology, vol. 23, no. 1, 2014.

[27]G. Bavota, R. Oliveto, M. Gethers, D. Poshyvanyk, and A. De Lucia, Methodbook: Recommending Move Method Refactorings via Relational Topic Models, IEEE Transactions on Software Engineering, vol. 40, no. 7, pp. 671-694, 2014.

[28] M. Kessentini, H. Sahraoui, M. Boukadoum, and O. Ben Omar, Search-based model transformation by example, Software and Systems Modeling, vol. 11, no. 2, pp. 209-226, 2012.

[29] G. Czibula and I. G. Czibula, Unsupervised restructuring of object-oriented software systems using self-organizing feature maps, International Journal of Innovative Computing Information and Control, vol. 8, no. 3A, pp. 1689-1704, 2012.

[30] I. G. Czibula and G. Czibula, Clustering Based Automatic Refactorings Identification, presented at the Proceedings of the 2008 10th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, 2008.

[31] H. Liu, Z. Niu, Z. Ma, and W. Shao, Identification of generalization refactoring opportunities, (in English), Automated Software Engineering, vol. 20, no. 1, pp. 81-110, 2013.

[32] A. Ghannem, G. El Boussaidi, and M. Kessentini, On the use of design defect examples to detect model refactoring opportunities, (in English), Software Quality Journal, pp. 1-19, 2015.

[33] L. Zhao and J. Hayes, Rank-based refactoring decision support: two studies, (in English), Innovations in Systems and Software Engineering, vol. 7, no. 3, pp. 171-189, 2011.

[34] T. Mens, G. Taentzer, and O. Runge Analysing refactoring dependencies using graph transformation, Software & Systems Modeling, vol. 6, no. 3, pp. 269-285, 2007.

[35] P. Mayer and A. Schroeder, Automated Multi-Language Artifact Binding and Rename Refactoring between Java and DSLs Used by Java Frameworks, in Proceedings of the 28th European Conference on Object-Oriented Programming, 2014, vol. 8586, pp. 437-462.

[36] M. S. Zanetti, C. J. Tessone, I. Scholtes, and F. Schweitzer, Automated software remodularization based on move refactoring: a complex systems approach, in Proceedings of the 13th international conference on Modularity, Lugano, Switzerland, 2014, pp. 73-84.

[37] W. Kessentini, M. Kessentini, H. Sahraoui, S. Bechikh, and A. Ouni, A Cooperative Parallel Search-Based Software Engineering Approach for Code-Smells Detection, IEEE Transactions on Software Engineering, vol. 40, no. 9, pp. 841-861, 2014.

[38] A. Ouni, M. Kessentini, S. Bechikh, and H. Sahraoui, Prioritizing code-smells correction tasks using chemical reaction optimization, Software Quality Journal, vol. 23, no. 2, pp. 323-361, 2015.

[39] N. Tsantalis and A. Chatzigeorgiou, Identification of Move Method Refactoring Opportunities, IEEE Transactions on Software Engineering, vol. 35, no. 3, pp. 347-367, 2009.

[40] N. Tsantalis and A. Chatzigeorgiou, Identification of extract method refactoring opportunities for the decomposition of methods, Journal of Systems and Software, vol. 84, no. 10, pp. 1757-1782, 2011.

[41] P. Weissgerber and S. Diehl, Identifying Refactorings from Source-Code Changes, in 21st IEEE/ACM International Conference on Automated Software Engineering (ASE'06), 2006, pp. 231-240.

Appendix B. Data extraction

Paper no.	Behavior preservation?	Consistency-check?	Systems	Refactorings	Metrics	Validation	Research methodology
1	Checked: manually by eighteen graduate students	–	GanttProject, Xcerces, ArgoUML, Quick UML, LOG4j, AZUREUS	Exhaustive lists of refactorings	Recall and precision	Metric-based	Empirical
2	Checked: by checking the preconditions	Checked without mentioning how between the code and the design model	jEdit, jGit, Columba	Move method	Maintainability, coupling, and cohesion	Metric-based	Empirical
3	–	–	ArgoUML, JEdit, Eclipse, JFreeChart, GanttProject, JHotDraw, Xerces, HSQLDB	Extract class	Cohesion, coupling	Metric and developer judgment	Empirical
4	NA	NA	The Visual Works, The HotDraw Framework, The Refactoring Browser	–	False positive and interesting false positive	Metric-based	Empirical
5	–	–	42318 JAVA APIs	–	NA	–	Empirical
6	–	–	17 Module dependency graphs	–	Cohesion and coupling	Metric-based	Empirical
7	NA	NA	IBM IV Service Application Architecture Model, The SAP Referencing Model	–	Recall, precision and F score	Metric-based	Empirical
8	Checked: by maximizing different semantic measures: (1) vocabulary-based similarity (cosine similarity between words: name of code elements); and (2) dependency-based similarity (call-graph, coupling and cohesion).	Checked: using cross-project context similarity for the code before and after refactoring.	Xcerces-J, JFreeChart, GanttProject, JHotDraw, ArtOfIllusion	Move method, move field, pull up method, pull up field, push down method, push down field, inline class, move class, extract class, extract interface	Qmood quality metrics	Quality metrics and metric-based	Empirical
9	Checked: automated using an extension of Eclipse (CeDAR). which uses preconditions checking	–	ApacheAnt, Columba, JEdit, JFreeChart, EMF, Hibernate, Jakarta JMETER, Squirrerl-SQL	Extract method	NA	Metric-based	Empirical
10	NA	NA	ArgoUML, CloudStack, Ecliplse, Firinika, GWT, Hibernate, Jackrabbit, Java DjVu, javax.usb, JFreechart, JML, JTransforms, Makumba, OpenEJB, Physhun, proteinShader, Qwicap guess, Robocode, sdedit, Stendhal, Struts2, Superversion, SVNKit, Tomcat, WebWork, Weka, Xalan, Xins	–	Cyclamate complexity	Metric-based	Empirical
11	Checked: using SafeRefactor tool though executing a set of unit tests before and after refactoring	–	LOG4j, jEdit, Xcerces, SweetH3D, HSQLDP	–	Reusability, flexibility, extendibility, effectiveness and quality evolution	Quality metrics and metric-based	Empirical
12	–	–	JHotDraw, MAVEN, ArgoUML, VUZE, Eclipse JDT	–	Severity and time performance	Quality metrics	Controlled experiment
13	Checked: by checking a set of preconditions	–	JHotDraw	Extract class	Recall and precision	Quality metrics	Empirical
14	Checked: by checking a set of preconditions	–	Violet, IHM, Nuteh, FreeCol, JMOL, JFreeChart	Replace conditional with polymorphism, replace type code with state	Recall, precision and accuracy	Metric-based	Empirical
15	–	–	ArgoUML, JHotDraw, Eclipse	Extract class refactoring	Recall, precision and F score	Metric-based	Empirical
16	–	–	ArgoUML, Xcerces, Ant Apache	–	–	Metric-based	Empirical
17	–	–	GanttProject, Quick UML, AZUREUS, LOG4J, ArgoUML, Xcerces	–	Recall and precision	Metric-based	Empirical
18	–	–	Xcerces-J, JHotDraw, JFreeChart, GanttProject, JDI-Ford	–	Recall, precision and F measure	Quality-metrics	Empirical
19	NA	NA	ArgoUML, Apache Ant, Lucene	–	Recall and precision	Metric-based	Empirical
20	–	–	Trama, Font4MySQL	Package refactoring	Recall and precision	Metric-based	Empirical
21	NA	NA	GanttProject, Xcerces	–	Recall and precision	Metric-based	Empirical
22	NA	NA	GMF Map, GMF Graph, GMF GEN	Exhaustive list of refactorings	Severity, importance and correctness	Quality metrics	Empirical
23	Checked: by checking a set of preconditions	–	Xcerces-J, JFreeChart, GanttProject, Apache Ant, RHINO, JHotDraw	Move method, move field, push down method, push down field, pull up method, pull up field, inline class, extract class, extract method, move class, extract interface.	Not applicable	Developer judgment	Survey
24	Checked: by checking a set of static preconditions	–	Beaver, Speck-Check, Mamo	Push down method, push down field, pull up method, pull up field, extract hierarchy, collapse hierarchy, increase field security, decrease field security, replace inheritance with delegation, replace delegation with inheritance, increase method security, decrease method security, make superclass abstract, make superclass concrete	Dissimilarity	Metric-based	Empirical
25	NA	NA	Eclipse, Strut, JHotDraw	Rename package, rename class, rename method, pull up method, push down method, move method, change method signature	Precision and recall	Quality metrics	Empirical
26	–	–	ETour, Exvantage, GanttProject, GEST, jEdit, JHotDraw, jVLT, SESA, SMOS	Move class	Coupling	Quality metrics	Empirical and questionnaire
27	Checked: by checking a set of preconditions	–	Agile Planner, Exvantage, GESA, jEdit, JFreeChart, SMOS	Move method	Cohesion and coupling	Quality metrics	Survey
28	Checked: fitness functions	Checked: fitness functions between UML class diagram and Rational Schema	12 Source Models	Model transformation	Precision	Metric-based	Empirical
29	–	–	JHotDraw, DICOM, HL7PACS	Move method, move attribute, inline class, extract class.	Not applicable	Developer judgment	Empirical
30	–	–	JHotDraw, DICOM, HL7	Move method, move attribute, inline class, extract class	None	Developer judgment	Empirical
31	NA	NA	GEF, Java Source, JFace, Thout Reader, AutoMed	–	Precision and recall	Quality metrics	Empirical
32	NA	NA	GanttProject, Log4J, ArgoUML, Xcerces	–	Precision and recall	Metric-based	Empirical
33	NA	NA	Tomcat, postgraduate project	–	Recall and precision	Quality metric and developer judgment	Empirical and survey
34	Checked: using Augur2 tool which generates Petri graph.	Checked: manual check using graph transformation between UML class diagram and Graphs.	LAN Simulator	Pull up variable, pull up method, move method, move variable, create superclass, encapsulate variables, add parameter, remove parameter, rename class, rename variable, rename method	NA	Implicit dependency	Empirical
35	–	–	Plasma, TuduLists, Itracker, PicketLink, Brix, GiddooCMS, JTrace	Rename refactoring	Na	Manual inspection	Empirical
36	Checked but did not specify how.	–	39	Move refactoring	Precision and recall	Metric-based	Empirical
37	–	–	JFreeChart, Gantt, Apache Ant, Nuteh, Log4j, Lucene, Rhino, Xerces	–	Recall and precision	Metric-based	Empirical
38	NA	NA	Xerces-J, JFreeChart, GanttProject, ArtOfIllusion, JHotDraw	Move method, move field, pull up method, pull up field, push down method, push down field, inline, class, extract class, extract interface, extract superclass, extract subclass	Correction ratio	Metric-based	Empirical
39	Checked: by checking a set of preconditions	–	JEdit 3.0, JFreeChart 0.9.6	Move method	Coupling and cohesion	Metric-based	Empirical
40	Checked: by checking a set of preconditions	–	JFreeChart	Extract method	Precision and recall	Metric-based	Empirical
41	NA	NA	jakarta-tomcat, jEdit	Move class, move interface, move field, move method, and rename class, rename method, hide method, unhide method, add parameter, and remove parameter	Precision and recall	Metric-based	Empirical
	– Not mentioned, N/A not applicable (the approach does not propose correction)

Further data extractions of the final set of 38 papers

Paper no.	Publisher	Year	Semi-full	Design code	Diagram, language or paradigm	Method	Detection/correction	Technique/tool
1	FASE	2012	F	Code	Java	Search-based (genetic programming)	Detection/ correction	Technique
2	Information and Software Technology	2015	F	Code	Java	Maximal independent set	Correction	Technique
3	Empirical Software Engineering	2014	F	Code	Java	Similarity matrix, chain of method	Detection / correction	Technique
4	OOPSLA	2000	F	Code	OO	Change metrics	Detection	Technique
5	International Conference on Software Maintenance and Evolution	2004	F	Code	Java	Search-based (random, MOSA, GA)	Detection / correction	Technique
6	IEEE Transaction on Software Engineering	2011	F	Graphs	Module dependency graph	Search-based (GA, HC, ECA, and MCA) and clustering	Correction	Technique
7	Information and Software Technology	2011	F	Model	Process model	Similarity	Detection	Technique
8	The Journal of Systems and Software	2015	F	Code	Java	Search-based (NSGA-II)	Detection /correction	Technique
9	Information and Software Technology	2012	F	Code	Java	Clone detection	Detection /correction	Tool
10	Information and Software Technology	2015	S	Model	Java	Query	Detection	Tool
11	Expert systems with applications	2015	F	Code	Java	Agent-based	Detection/ correction	Technique
12	Science of Computer Programming	2015	F	Code	Java	Plugins	Detection/ correction	Tool
13	The Journal of Systems and Software	2012	F	Code	Java	Clustering	Detection/ correction	Tool
14	The Journal of Systems and Software	2010	S	Code	Java	Polymorphism	Detection/ correction	Tool
15	The Journal of Systems and Software	2011	F	Code	Java	Graph algorithm	Detection/ correction	Technique
16	GECCO 14	2010	F	Code	Java	Search-based (NSGA-III, IBEA, MOE/D, NSGA-II)	Detection/ correction	Technique
17	Automated Software Engineering Journal	2013	F	Code	Java	Search-based (GP, NSGA-II)	Detection/ correction	Technique
18	ACM/IEEE International Conference on Automated Software Engineering	2014	S	Code	Java	Search-based (NSGA-II)	Detection/ correction	Tool
19	IEEE International Conference on Software Maintenance and Evolution	2014	F	Code	Java	Cloning	Detection	Technique
20	International Journal of Automation and Computing	2013	F	Package	Java	Community detection	Detection/ correction	Technique
21	FASE	2011	F	Code	Java	Search-based (harmony search, PSO, SA)	Detection	Technique
22	International Conference on Software Maintenance	2012	F	Model	Graphical Modeling Framework	Search-based (GA)	Detection	Technique
23	Symposium on Search-based Software Engineering	2014	F	Code	Java	Search-based (NSGA-II)	Detection/ correction	Technique
24	Frontiers in Artificial Intelligence and Applications	2007	F	Code	Java and spec benchmark	Search-based (undefined)	Correction	Technique
25	ECOOP – Object-Oriented Programming	2006	F	Code	Java	Similarity-based	Detection	Plug-ins
26	ACM Transaction on Embedded Computing Systems	2014	F	Code	Java	Probabilistic topic modeling	Detection/ correction	Tool
27	IEEE Transaction on Software Engineering	2014	F	Code	Java	Probabilistic topic modeling	Detection/ correction	Tool
28	Software System Modeling	2012	F	Model	Class diagram	Search-based (particle swarm and simulated annealing), model transformation	Detection/ correction	Technique
29	International Journal of Innovative Computing, Information and Control	2012	F	Code	Java	Self-organizing maps	Detection/ correction	Technique
30	International Symposium on Symbolic and Numeric Algorithms for scientific Computing	2008	F	Code	Java	Clustering	Detection/ correction	Technique
31	Automated Software Engineering Journal	2013	S	Code	Java	Similarity	Detection	Tool
32	Software Quality Journal	2015	F	UML	Class diagram	Search-based (GA)	Detection	Technique
33	Innovations System Software Engineering	2011	F	Code (classes and packages)	Java	Ranking, hierarchical approach based on metrics	Detection	Tool
34	Software System Modeling	2007	F	UML	Class diagram	Graph transformation	Detection/ correction	Technique
35	ECOOP	2014	F	Code	Java, spring HBM < HQL, W/API, W/HTML	Binding	Detection/ correction	Tool
36	International Conference on Modularity	2014	F	Code	Java	Search-based (SA-like)	Detection/ correction	Tool
37	IEEE Transaction on Software Engineering	2014	F	Code	Open source	Search-based (Parallel Evolution)	Detection	Technique
38	Software Quality Journal	2014	F	Code	Java	Search-based (chemical reaction optimization)	Detection/ correction	Technique
39	IEEE Transaction on Software Engineering	2009	S	Code	Java	Similarity	Detection/ correction	Technique
40	The Journal of Systems and Software	2011	F	Code	Java	Similarity	Detection/ correction	Technique
41	ACM/IEEE international conference on Automated software engineering	2006	F	Code	Java	Similarity	Detection	Technique

Appendix C. Search strings

• 1. IEEE:

In this database, we selected the following string: ((auto* or software or code or uml or model) AND refactor*). which resulted in 165 retrievals. We applied the above search string to be searched in full text and metadata.

When we inserted search-based, the engine returned many search-based algorithms that are pervasive in the literature but do not neccessarily address software engineering problems. But when we dropped it, some relevant papers such as “Search-Based Refactoring Using Recorded Code Changes” did not appear. So we included it and we reduced the search coverage to “metadata” only.

When we tried the following search string:

((refactor*) AND auto* or code or software or uml or model or search-based)

only one paper was retrieved, so we moved search-based to a new field. This resulted in the retrieval of many search-based only papers, but as it returned many relevant papers, we accepted that these would have to be filtered manually.

So our final string was

((auto* or software or code or uml or model) AND refactor*) or search-based

Which returned 1601 and its coverage include metadata only.

1.1 ACM:

We started with the following search string in ACM:

(refactor*) and (auto* or software or code or uml or model) and (Keywords:refactor*)

which returned 781 results.

We then modified it to the follwing:

(refactor*) and (auto* or software or code or uml or model) and (Keywords:refactor* OR Keywords: “search-based”)

which retriveed 821 and more related papers. Surprisingly, if we extended the search string to

(refactor*) and (auto* or software or code or uml or model or anit-pattern or detection or correction) and (Keywords:refactor* OR Keywords: “search-based”), it again retrieved 821 papers.

So we used the shorter version.

Note: We selected the ACM guide to cover ACM and affiliated publishers in addition to other non-affiliated publishers.

• 2. Wiley:

Initially, we used: refactor* in All Fields AND auto* or software or code or uml or model in All Fields AND refactor* or “search-based” in Keywords

But very few results were retrieved, so we changed it to the following to broaden our search:

refactor* in All Fields AND auto* or software or code or uml or model in All Fields AND refactor* or “search-based” in All Fields

However, it returned several non-related papers. We modified to the following

refactor* in Abstract AND auto* or code or uml or model or search-based in Abstract

and it retrieves 35. So, we accepted it.

• 3. Science Direct

We used the following search string:

TITLE-ABSTR-KEY (refactor*) and TITLE-ABSTR-KEY(auto* or software or code or model or uml or search-based). It returned181 related papers. So, we accepted it.

• 4. Springer

We tried “refactor* AND (auto* or software or code or model or search-based)” and it returned 7014. By selecting only English language papers, the number reduced to 6804 but while browsing, we found that many were irrelevant, so we decided to reduce these again by adding the word refactor* in the title, which returned 267 papers, most of which were relevant. By using English papers only, the number of returned papers was reduced to 260.

• 5. Web of Knowledge

We used the following search string:

TOPIC:(refactor*) AND TITLE: (auto* or software or code or uml or model or search-based)

Timespan: All years. Search language=English

It returned 709 papers, which we refined by using only English and excluding patents and abstracts to obtain a final result of 631 papers.