Empirical Software Engineering

, Volume 16, Issue 1, pp 29–60 | Cite as

A study of the bi-objective next release problem

  • Juan J. DurilloEmail author
  • Yuanyuan Zhang
  • Enrique Alba
  • Mark Harman
  • Antonio J. Nebro


One important issue addressed by software companies is to determine which features should be included in the next release of their products, in such a way that the highest possible number of customers get satisfied while entailing the minimum cost for the company. This problem is known as the Next Release Problem (NRP). Since minimizing the total cost of including new features into a software package and maximizing the total satisfaction of customers are contradictory objectives, the problem has a multi-objective nature. In this work, we apply three state-of-the-art multi-objective metaheuristics (two genetic algorithms, NSGA-II and MOCell, and one evolutionary strategy, PAES) for solving NRP. Our goal is twofold: on the one hand, we are interested in analyzing the results obtained by these metaheuristics over a benchmark composed of six academic problems plus a real world data set provided by Motorola; on the other hand, we want to provide insight about the solution to the problem. The obtained results show three different kinds of conclusions: NSGA-II is the technique computing the highest number of optimal solutions, MOCell provides the product manager with the widest range of different solutions, and PAES is the fastest technique (but with the least accurate results). Furthermore, we have observed that the best solutions found so far are composed of a high percentage of low-cost requirements and of those requirements that produce the largest satisfaction on the customers as well.


Search based software engineering Multi-objective optimization Next release Requirements engineering 



J. J. Durillo, A. Nebro, and E. Alba acknowledge founds from the “Consejería de Innovación, Ciencia y Empresa”, Junta de Andalucía under contract P07-TIC-03044 DIRICOM project (, and the Spanish Ministry of Science and Innovation and FEDER under contract TIN2008-06491-C04-01 (the M* project). J. J. Durillo is also supported by grant AP-2006-03349 from the Spanish Ministry of Education and Science. Mark Harman is partly supported by EPSRC grants EP/G060525 (CREST: Centre for Research on Evolution, Search and Testing, Platform Grant), and EP/D050863 (SEBASE: Software Engineering By Automated SEarch), which also fully supports Yuanyuan Zhang.


  1. Afzal W, Torkar R, Feldt R (2009) A systematic review of search-based testing for non-functional system properties. Inf Softw Technol 51(6):957–976CrossRefGoogle Scholar
  2. Alba E, Chicano JF (2007) Software project management with gas. Inf Sci 177(11):2380–2401CrossRefGoogle Scholar
  3. Alba E, Dorronsoro B (2008) Cellular genetic algorithms. Operations research/computer science interfaces, vol 42. Springer-Verlag, HeidelbergGoogle Scholar
  4. AlBourae T, Ruhe G, Moussavi M (2006) Lightweight replanning of software product releases. In: Proceedings of the 1st international workshop on software product management (IWSPM ’06). IEEE Computer Society, Minneapolis, pp 27–34CrossRefGoogle Scholar
  5. Bagnall AJ, Rayward-Smith VJ, Whittley IM (2001) The next release problem. Inf Softw Technol 43(14):883–890CrossRefGoogle Scholar
  6. Baker P, Harman M, Steinhofel K, Skaliotis A (2006) Search based approaches to component selection and prioritization for the next release problem. In: ICSM ’06: proceedings of the 22nd IEEE international conference on software maintenance. IEEE Computer Society, Washington, pp 176–185Google Scholar
  7. Cortellessa V, Marinelli F, Potena P (2006) Automated selection of software components based on cost/reliability tradeoff. In: Proceedings of the 3rd European workshop on software architecture (EWSA ’06). LNCS, vol 4344. Springer, Nantes, pp 66–81Google Scholar
  8. Cortellessa V, Crnkovic I, Marinelli F, Potena P (2008a) Experimenting the automated selection of COTS components based on cost and system requirements. J Univers Comput Sci 14(8):1228–1255Google Scholar
  9. Cortellessa V, Marinelli F, Potena P (2008b) An optimization framework for “build-or-buy” decisions in software architecture. Comput Oper Res 35(10):3090–3106zbMATHCrossRefGoogle Scholar
  10. Deb K (2001) Multi-objective optimization using evolutionary algorithms, 1st edn. WileyGoogle Scholar
  11. Deb KD, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans Evol Comput 6(2):182–197CrossRefGoogle Scholar
  12. Del Grosso C, Antoniol G, Di Penta M, Galinier P, Merlo E (2005) Improving network applications security: a new heuristic to generate stress testing data. In: GECCO ’05: proceedings of the 2005 conference on Genetic and evolutionary computation. ACM, New York, pp 1037–1043CrossRefGoogle Scholar
  13. Durillo JJ, Nebro AJ, Luna F, Dorronsoro B, Alba E (2006) jMetal: a java framework for developing multi-objective optimization metaheuristics. Technical Report ITI-2006-10, Departamento de Lenguajes y Ciencias de la Computación, University of Málaga, E.T.S.I. Informática, Campus de TeatinosGoogle Scholar
  14. Durillo JJ, Zhang Y, Alba E, Nebro AJ (2009) A study of the multi-objective next release problem. In: SSBSE ’09: proceedings of the 2009 1st international symposium on search based software engineering. IEEE Computer Society, Washington, pp 49–58CrossRefGoogle Scholar
  15. Everson RM, Fieldsend JE (2006) Multiobjective optimization of safety related systems: an application to short-term conflict alert. IEEE Trans Evol Comput 10(2):187–198CrossRefMathSciNetGoogle Scholar
  16. Feather MS, Menzies T (2002) Converging on the optimal attainment of requirements. In: RE ’02: proceedings of the 10th anniversary IEEE joint international conference on requirements engineering. IEEE Computer Society, Washington, pp 263–272CrossRefGoogle Scholar
  17. Finkelstein A, Harman M, Mansouri SA, Ren J, Zhang Y (2008) Fairness analysis” in requirements assignments. In: RE ’08: proceedings of the 2008 16th IEEE international requirements engineering conference. IEEE Computer Society, Washington, pp 115–124CrossRefGoogle Scholar
  18. Finkelstein A, Harman M, Mansouri SA, Ren J, Zhang Y (2009) A search based approach to fairness analysis in requirement assignments to aid negotiation, mediation and decision making. Requir Eng 14(4):231–245CrossRefGoogle Scholar
  19. Greer D, Ruhe G (2004) Software release planning: an evolutionary and iterative approach. Inf Softw Technol 46(4):243–253CrossRefGoogle Scholar
  20. Harman M (2007) The current state and future of search based software engineering. In: FOSE ’07: 2007 future of software engineering. IEEE Computer Society, Washington, pp 342–357CrossRefGoogle Scholar
  21. Harman M, Tratt L (2007) Pareto optimal search based refactoring at the design level. In: GECCO ’07: proceedings of the 9th annual conference on genetic and evolutionary computation. ACM, New York, pp 1106–1113CrossRefGoogle Scholar
  22. Harman M, Skaliotis A, Steinhöfel K (2006) Search-based approaches to the component selection and prioritization problem. In: Proceedings of the 8th annual conference on genetic and evolutionary computation (GECCO ’06). ACM, Seattle, pp 1951–1952CrossRefGoogle Scholar
  23. Harman M, Mansouri SA, Zhang Y (2009) Search based software engineering: a comprehensive analysis and review of trends techniques and applications. Technical Report TR-09-03Google Scholar
  24. Karlsson J (1996) Software requirements prioritizing. In: Proceedings of the second international conference on requirements engineering (RE ’96). IEEE Computer Society, Colorado Springs, CO, pp 110–116CrossRefGoogle Scholar
  25. Karlsson J, Ryan K (1997) A cost-value approach for prioritizing requirements. IEEE Softw 14(5):67–74CrossRefGoogle Scholar
  26. Karlsson J, Wohlin C, Regnell B (1998) An evaluation of methods for prioritizing software requirements. Inf Softw Technol 39(14–15):939–947CrossRefGoogle Scholar
  27. Kendall M, Gibbons JD (1990) Rank correlation methods, 5th edn. A Charles Griffin TitleGoogle Scholar
  28. Khoshgoftaar TM, Liu Y, Seliya N (2004) Module-order modeling using an evolutionary multi-objective optimization approach. In: Proceedings of the 10th IEEE international symposium on software metrics (METRICS ’04). IEEE Computer Society, pp 159–169CrossRefGoogle Scholar
  29. Knowles JD, Corne DW (1999) The Pareto Archived Evolution Strategy : A new baseline algorithm for pareto multiobjective optimisation. In: Proceedings of the 1999 congress on evolutionary computation (CEC’99), pp 98–105Google Scholar
  30. Knowles J, Thiele L, Zitzler E (2006) A tutorial on the performance assessment of stochastic multiobjective optimizers. TIK Report 214, Computer Engineering and Networks Laboratory (TIK), ETH ZurichGoogle Scholar
  31. Korel B (1990) Automated software test data generation. IEEE Trans Softw Eng 16(8):870–879CrossRefGoogle Scholar
  32. Lakhotia K, Harman M, McMinn P (2007) A multi-objective approach to search-based test data generation. In: GECCO ’07: proceedings of the 9th annual conference on genetic and evolutionary computation. ACM, New York, pp 1098–1105CrossRefGoogle Scholar
  33. McMinn P (2004) Search-based software test data generation: a survey: research articles. Softw Test Verif Reliab 14(2):105–156CrossRefGoogle Scholar
  34. Miller W, Spooner D (1976) Automatic generation of floating-point test data. IEEE Trans Softw Eng 2:223–226CrossRefMathSciNetGoogle Scholar
  35. Nebro AJ, Durillo JJ, Luna F, Dorronsoro B, Alba E (2007) Design issues in a multiobjective cellular genetic algorithm. In: Obayashi S, Deb K, Poloni C, Hiroyasu T, Murata T (eds) Evolutionary multi-criterion optimization. 4th international conference, EMO 2007. Lecture notes in computer science, vol 4403. Springer, pp 126–140Google Scholar
  36. Nebro AJ, Durillo JJ, Luna F, Dorronsoro B, Alba E (2009) Mocell: a cellular genetic algorithm for multiobjective optimization. Int J Intell Syst 24(7):726–746zbMATHCrossRefGoogle Scholar
  37. Ruhe G, Greer D (2003) Quantitative studies in software release planning under risk and resource constraints. In: Proceedings of the international symposium on empirical software engineering (ISESE ’03). IEEE, Rome, pp 262–270CrossRefGoogle Scholar
  38. Ruhe G, Ngo-The A (2004) Hybrid intelligence in software release planning. International Journal of Hybrid Intelligent Systems 1(1–2):99–110Google Scholar
  39. Saaty TL (1980) The analytic hierarchy process, planning, priority setting, resource allocation. McGraw-HillGoogle Scholar
  40. Saliu MO, Ruhe G (2007) Bi-objective release planning for evolving software systems. In: ESEC-FSE ’07: proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering. ACM, New York, pp 105–114Google Scholar
  41. Sullivan L (1986) Quality function deployment. Quality progress, pp 39–50Google Scholar
  42. Walcott KR, Soffa ML, Kapfhammer GM, Roos RS (2006) Timeaware test suite prioritization. In: ISSTA ’06: proceedings of the 2006 international symposium on software testing and analysis. ACM, New York, pp 1–12CrossRefGoogle Scholar
  43. Xanthakis S, Ellis C, Skourlas C, Gall, AL, Katsikas S, Karapoulios K (1992) Application of genetic algorithms to software testing. In: Proceedings of the 5th international conference on software engineering and applications. Toulouse, France, pp 625–636Google Scholar
  44. Yeh R, Ng P (1990) Software requirements—a management perspective. In: Thayer RH, Dorfman M (eds) System and software requirements engineering. IEEE Computer Society Press Tutorial, pp 450–461Google Scholar
  45. Yoo S, Harman M (2007) Pareto efficient multi-objective test case selection. In: ISSTA ’07: proceedings of the 2007 international symposium on software testing and analysis. ACM, New York, pp 140–150CrossRefGoogle Scholar
  46. Zhang Y, Finkelstein A, Harman M (2008) Search based requirements optimisation: existing work and challenges. In: REFSQ ’08: proceedings of the 14th international conference on requirements engineering. Springer-Verlag, Berlin, pp 88–94Google Scholar
  47. Zhang Y, Harman M, Mansouri SA (2007) The multi-objective next release problem. In: GECCO ’07: proceedings of the 9th annual conference on genetic and evolutionary computation. ACM, New York, pp 1129–1137CrossRefGoogle Scholar
  48. Zitzler E, Thiele L (1999) Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Trans Evol Comput 3(4):257–271CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Juan J. Durillo
    • 1
    Email author
  • Yuanyuan Zhang
    • 2
  • Enrique Alba
    • 1
  • Mark Harman
    • 2
  • Antonio J. Nebro
    • 1
  1. 1.Computer Science DepartmentUniversity of MálagaMálagaSpain
  2. 2.CREST CentreUniversity College LondonLondonUK

Personalised recommendations