Empirical Software Engineering

, Volume 21, Issue 6, pp 2546–2600 | Cite as

A comparative study of many-objective evolutionary algorithms for the discovery of software architectures

  • Aurora Ramírez
  • José Raúl RomeroEmail author
  • Sebastián Ventura


During the design of complex systems, software architects have to deal with a tangle of abstract artefacts, measures and ideas to discover the most fitting underlying architecture. A common way to structure such complex systems is in terms of their interacting software components, whose composition and connections need to be properly adjusted. Along with the expected functionality, non-functional requirements are key at this stage to guide the many design alternatives to be evaluated by software architects. The appearance of Search Based Software Engineering (SBSE) brings an approach that supports the software engineer along the design process. Evolutionary algorithms can be applied to deal with the abstract and highly combinatorial optimisation problem of architecture discovery from a multiple objective perspective. The definition and resolution of many-objective optimisation problems is currently becoming an emerging challenge in SBSE, where the application of sophisticated techniques within the evolutionary computation field needs to be considered. In this paper, diverse non-functional requirements are selected to guide the evolutionary search, leading to the definition of several optimisation problems with up to 9 metrics concerning the architectural maintainability. An empirical study of the behaviour of 8 multi- and many-objective evolutionary algorithms is presented, where the quality and type of the returned solutions are analysed and discussed from the perspective of both the evolutionary performance and those aspects of interest to the expert. Results show how some many-objective evolutionary algorithms provide useful mechanisms to effectively explore design alternatives on highly dimensional objective spaces.


Software architecture discovery Search based software engineering Many-objective evolutionary algorithms Multi-objective evolutionary algorithms 



Work supported by the Spanish Ministry of Science and Technology, project TIN2011-22408, the Spanish Ministry of Economy and Competitiveness, project TIN2014-55252-P, and FEDER funds. This research was also supported by the Spanish Ministry of Education under the FPU program (FPU13/01466).


  1. Adra S, Fleming P (2011) Diversity management in evolutionary many-objective optimization. IEEE Trans Evol Comput 15(2):183–195CrossRefGoogle Scholar
  2. Aleti A, Buhnova B, Grunske L, Koziolek A, Meedeniya I (2013) Software architecture optimization methods: a systematic literature review. IEEE Trans Softw Eng 39(5):658–683CrossRefGoogle Scholar
  3. Arcuri A, Briand L (2011) A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: Proceedings of the 33rd international conference on software engineering (ICSE’11). IEEE, pp 1–10Google Scholar
  4. Assunção WKG, Colanzi TE, Vergilio SR, Pozo A (2014) A multi-objective optimization approach for the integration and test order problem. Inf Sci 267:119–139MathSciNetCrossRefGoogle Scholar
  5. Bader J, Zitzler E (2011) HypE: an algorithm for fast hypervolume-based many-objective optimization. Evol Comput 19(1):45–76CrossRefGoogle Scholar
  6. Bansiya J, Davis CG (2002) A hierarchical model for object-oriented design quality assessment. IEEE Trans Softw Eng 28(1):4–17CrossRefGoogle Scholar
  7. Bosch J, Molin P (1999) Software architecture design: evaluation and transformation. In: Proceedings of IEEE conference and workshop on engineering of computer-based systems (ECBS’99), pp 4–10Google Scholar
  8. Boussaïd I, Lepagnot J, Siarry P (2013) A survey on optimization metaheuristics. Inf Sci 237:82–117MathSciNetCrossRefzbMATHGoogle Scholar
  9. Bouwers E, Correia J, van Deursen A, Visser J (2011) Quantifying the analyzability of software architectures. In: Proceedings of the 9th working IEEE/IFIP conference on software architecture (WICSA’11), pp 83–92Google Scholar
  10. Bowman M, Briand LC, Labiche Y (2010) Solving the class responsibility assignment problem in object-oriented analysis with multi-objective genetic algorithms. IEEE Trans Softw Eng 36 (6):817–837CrossRefGoogle Scholar
  11. Coello Coello CA, Lamont GB, Van Veldhuizen DA (2007) Evolutionary algorithms for solving multi-objective problems, 2nd edn. SpringerGoogle Scholar
  12. Deb K (2001) Multi-objective optimization using evolutionary algorithms. Wiley, New YorkzbMATHGoogle Scholar
  13. Deb K, Jain H (2014) An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, Part I: solving problems with box constraints. IEEE Trans Evol Comput 18(4):577–601CrossRefGoogle Scholar
  14. Deb K, Mohan M, Mishra S (2003) Towards a quick computation of well-spread pareto-optimal solutions. In: Evolutionary multi-criterion optimization of LNCS, vol 2632. Springer, pp 222–236Google Scholar
  15. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197CrossRefGoogle Scholar
  16. del Sagrado J, del Águila IM, Orellana FJ (2015) Multi-objective ant colony optimization for requirements selection. Empir Softw Eng 20(3):577–610CrossRefGoogle Scholar
  17. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRefGoogle Scholar
  18. Dobrica L, Niemela E (2002) A survey on software architecture analysis methods. IEEE Trans Softw Eng 28(7):638–653CrossRefzbMATHGoogle Scholar
  19. Ducasse S, Pollet D (2009) Software architecture reconstruction: a process-oriented taxonomy. IEEE Trans Softw Eng 35(4):573–591CrossRefGoogle Scholar
  20. Durillo JJ, Zhang Y, Alba E, Harman M, Nebro AJ (2011) A study of the bi-objective next release problem. Empir Softw Eng 16(1):29–60CrossRefGoogle Scholar
  21. Garlan D (2000) Software architecture: a roadmap. In: Proceedings of the 22th international conference of software engineering (ICSE’00), pp 91–101Google Scholar
  22. Grunske L (2006) Identifying ”Good” architectural design alternatives with multi-objective optimization strategies. In: Proceedings of the 28th international conference on software engineering (ICSE’06), pp 849–852Google Scholar
  23. Gupta P, Verma S, Mehlawat M (2012) Optimization model of COTS selection based on cohesion and coupling for modular software systems under multiple applications environment. In: Computational science and its applications (ICCSA) of LNCS, vol 7335. Springer, pp 87–102Google Scholar
  24. Hadka D, Reed P (2013) Borg: an auto-adaptive many-objective evolutionary computing framework. Evol Comput 21(2):231–259CrossRefGoogle Scholar
  25. Harman M, Mansouri SA, Zhang Y (2012) Search-based software engineering: trends, techniques and applications. ACM Comput Surv 45(1):11:1–61CrossRefGoogle Scholar
  26. He Z, Yen G, Zhang J (2014) Fuzzy-based Pareto optimality for many-objective evolutionary algorithms. IEEE Trans Evol Comput 18(2):269–285CrossRefGoogle Scholar
  27. ISO (2011a) ISO/IEC 25010:2011(E). Software product Quality Requirements and Evaluation (SQuaRE) - System and software quality models. ISOGoogle Scholar
  28. ISO (2011b) ISO/IEC/IEEE FDIS 42010/D9. Systems and software engineering - Architecture descriptionGoogle Scholar
  29. Kalboussi S, Bechikh S, Kessentini M, Ben Said L (2013) Preference-based many-objective evolutionary testing generates harder test cases for autonomous agents. In: Proceedings of 5th symposium on search based software engineering (SSBSE’13), pp 245–250Google Scholar
  30. Khare V, Yao X, Deb K (2003) Performance scaling of multi-objective evolutionary algorithms. In: Evolutionary multi-criterion optimization of lecture notes in computer science, vol 2632. Springer, Berlin, pp 376–390CrossRefGoogle Scholar
  31. Koziolek A, Ardagna D, Mirandola R (2013) Hybrid multi-attribute QoS optimization in component based software systems. J Syst Softw 86(10):2542–2558CrossRefGoogle Scholar
  32. Krogmann K (2010) Reconstruction of software component architectures and behaviour models using static and dynamic analysis. KIT Scientific PublishingGoogle Scholar
  33. Li R, Etemaadi R, Emmerich MTM, Chaudron MRV (2011) An evolutionary multiobjective optimization approach to component-based software architecture design. In: Proceedings of the IEEE congress on evolutionary computation (CEC’11), pp 432–439Google Scholar
  34. Luna F, González-Álvarez DL, Chicano F, Vega-Rodríguez MA (2014) The software project scheduling problem: A scalability analysis of multi-objective metaheuristics. Appl Soft Comput 15 :136–148CrossRefGoogle Scholar
  35. Lutz R (2001) Evolving good hierarchical decompositions of complex systems. J Syst Archit 47(7):613–634CrossRefGoogle Scholar
  36. Martin R (1994) OO design quality metrics - An analysis of dependencies. In: Object-Oriented programming systems, languages and applications (OOPSLA), pp 1–8Google Scholar
  37. Mkaouer MW, Kessentini M, Bechikh S, Deb K, Ó Cinnéide M (2014) High dimensional search-based software engineering: finding tradeoffs among 15 objectives for automating software refactoring using NSGA-III. In: Proceedings of the 16th annual genetic and evolutionary computation conference (GECCO’14), pp 1263–1270Google Scholar
  38. Narasimhan VL, Hendradjaya B (2007) Some theoretical considerations for a suite of metrics for the integration of software components. Inf Sci 177(3):844–864MathSciNetCrossRefzbMATHGoogle Scholar
  39. OMG (2010) Unified modeling language 2.4 superstructure specification. OMG. formal/2010-11-14,
  40. Ouni A, Kessentini M, Sahraoui H, Hamdi MS (2013) The use of development history in software refactoring using a multi-objective evolutionary algorithm. In: Proceedings of the 15th annual genetic and evolutionary computation conference (GECCO’13), pp 1461–1468Google Scholar
  41. Praditwong K, Harman M, Yao X (2011) Software module clustering as a multi-objective search problem. IEEE Trans Softw Eng 37(2):264–282CrossRefGoogle Scholar
  42. Praditwong K, Yao X (2007) How well do multi-objective evolutionary algorithms scale to large problems. In: Proceedings of the IEEE congress on evolutionary computation (CEC’07), pp 3959–3966Google Scholar
  43. Purshouse R, Fleming P (2007) On the evolutionary optimization of many conflicting objectives. IEEE Trans Evol Comput 11(6):770–784CrossRefGoogle Scholar
  44. Räihä O (2010) A survey on search-based software design. Comput Sci Rev 4 (4):203–249CrossRefGoogle Scholar
  45. Räihä O, Koskimies K, Makinen E (2011) Generating software architecture spectrum with multi-objective genetic algorithms. In: Proceedings of the 3th world congress on nature and biologically inspired computing (NaBIC’11), pp 29–36Google Scholar
  46. Ramírez A, Romero JR, Ventura S (2014) On the performance of multiple objective evolutionary algorithms for software architecture discovery. In: Proceedings of the 16th annual genetic and evolutionary computation conference (GECCO’14), pp 1287–1294Google Scholar
  47. Ramírez A, Romero JR, Ventura S (2015) An approach for the evolutionary discovery of software architectures. Inf Sci 305:234–255CrossRefGoogle Scholar
  48. Romano J, Kromrey JD, Coraggio J, Showronek J (2006) Appropriate statistics for ordinal level data: should we really be using t-test and cohen’s d for evaluating group differences on the nsse and other surveys? In: Annual meeting of the Florida association of institutional researchGoogle Scholar
  49. Sant’Anna C, Figueiredo E, Garcia A, Lucena CJ (2007) On the modularity of software architectures: a concern-driven measurement framework. In: Software architecture of LNCS, vol 4758. Springer, pp 207–224Google Scholar
  50. Sayyad A, Ammar H (2013) Pareto-optimal search-based software engineering (POSBSE): a literature survey. In: 2nd inter. workshop on realizing artificial intelligence synergies in software engineering (RAISE), pp 21–27Google Scholar
  51. Sayyad AS, Menzies T, Ammar H (2013) On the value of user preferences in search-based software engineering: a case study in software product lines. In: Proceedings of the 35th international conference on software engineering (ICSE’13), pp 492–501Google Scholar
  52. Schutze O, Lara A, Coello Coello CA (2011) On the influence of the number of objectives on the hardness of a multiobjective optimization problem. IEEE Trans Evol Comput 15(4):444–455CrossRefGoogle Scholar
  53. Szyperski C (2002) Component software: beyond object-oriented programming, 2nd edn. Addison-Wesley, BostonzbMATHGoogle Scholar
  54. Ventura S, Romero C, Zafra A, Delgado JA, Hervás C (2008) JCLEC: a Java framework for evolutionary computation. Soft Comput 12(4):381–392CrossRefGoogle Scholar
  55. von Lücken C, Barán B, Brizuela C (2014) A survey on multi-objective evolutionary algorithms for many-objective problems. Comput Optim Appl 58(3):707–756MathSciNetzbMATHGoogle Scholar
  56. Wagner T, Beume N, Naujoks B (2007) Pareto-, aggregation-, and indicator-based methods in many-objective optimization. In: Evolutionary multi-criterion optimization of LNCS, vol 4403. Springer, pp 742–756Google Scholar
  57. Wang H, Jiao L, Yao X (2014) An improved two-archive algorithm for many-objective optimization. IEEE Trans Evol Comput. To appearGoogle Scholar
  58. Yang S, Li M, Liu X, Zheng J (2013) A grid-based evolutionary algorithm for many-objective optimization. IEEE Trans Evol Comput 17(5):721–736CrossRefGoogle Scholar
  59. Yao X (2013) Some recent work on multi-objective approaches to search-based software engineering. In: Proceedings of the 5th symposium on search based software engineering (SSBSE), pp 4–15Google Scholar
  60. Zhang Q, Li H (2007) MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans Evol Comput 11(6):712–731CrossRefGoogle Scholar
  61. Zhang Y, Harman M, Lim SL (2013) Empirical evaluation of search based requirements interaction management. Inf Softw Technol 55(1):126–152CrossRefGoogle Scholar
  62. Zhou A, Qu B-Y, Li H, Zhao S-Z, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49CrossRefGoogle Scholar
  63. Zitzler E, Künzli S (2004) Indicator-based selection in multiobjective search. In: Parallel problem solving from nature - PPSN VIII of LNCS, vol 3242. Springer, pp 832–842Google Scholar
  64. Zitzler E, Laumanns M, Bleuler S (2004) A tutorial on evolutionary multiobjective optimization. In: Metaheuristics for multiobjective optimisation of lecture notes in economics and mathematical systems, vol 535. Springer, Berlin, pp 3–37CrossRefGoogle Scholar
  65. Zitzler E, Laumanns M, Thiele L (2001) SPEA2: improving the strength Pareto evolutionary algorithm. In: Proceedings of the conference on evolutionary methods for design, optimisation and control with applications to industrial problems, pp 95–100Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Aurora Ramírez
    • 1
  • José Raúl Romero
    • 1
    Email author
  • Sebastián Ventura
    • 1
  1. 1.Department of Computer Science and Numerical AnalysisUniversity of CórdobaCórdobaSpain

Personalised recommendations