Soft Computing

, Volume 17, Issue 6, pp 1005–1017 | Cite as

Best practices in measuring algorithm performance for dynamic optimization problems

  • Hajer Ben-RomdhaneEmail author
  • Enrique Alba
  • Saoussen Krichen


Dynamic optimization problems (DOPs) have attracted considerable attention due to the wide range of problems they can be applied to. Lots of efforts have been expended in modeling dynamic situations, proposing algorithms, and analyzing the results (too often in a visual way). Numeric performance measurements and their statistical validation have been however barely used in the literature. Most of works in DOPs report only the best-of-generation fitness, due to its simplicity of computation. Although this measure indicates the best algorithm in terms of fitness, it does not provide any details about the actual strength and weakness of each algorithm. In this article, we conduct a comparative study among algorithms of different search modes via several performance measures to demonstrate their relative advantages. We discuss the role of using different performance measures in drawing balanced conclusions about algorithms for DOPs.


Dynamic optimization problems Evolutionary algorithms Genetic algorithms Performance measure 



Authors acknowledge funds from the CICE of the Junta de Andalucia, under contract P07-TIC-03044 (DIRICOM and Spanish Ministry of Sciences and Innovation (MICINN) and FEDER under contracts TIN2011-28194 (RoadMe and TIN2008-06491-C04-01 (M* Also, from the European COADVISE project number 230833.


  1. Alba E (2005) Parallel metaheuristics. Wiley, Inc., USAGoogle Scholar
  2. Alba E, Sarasola B (2010) Abc, a new performance tool for algorithms solving dynamic optimization problems. In: IEEE Congress on evolutionary computation, pp 1–7Google Scholar
  3. Alba E, Sarasola B (2010) Measuring fitness degradation in dynamic optimization problems. In: Applications of evolutionary computation, Lecture notes in computer science, vol 6024, Springer, Heidelberg, pp 572–581Google Scholar
  4. Back T (1998) On the behavior of evolutionary algorithms in dynamic environments. In: The 1998 IEEE International Conference on evolutionary computation proceedings, pp 446–451Google Scholar
  5. Bierwirth C, Mattfeld D (1999) Production scheduling and rescheduling with genetic algorithms. Evol Comput 7(1):1–17CrossRefGoogle Scholar
  6. Branke J (1999) Memory enhanced evolutionary algorithms for changing optimization problems. In: IEEE Proceedings of the 1999 Congress on evolutionary computation, vol 3, pp 1875–1882Google Scholar
  7. Branke J, Mattfeld DC (2000) Anticipation in dynamic optimization: the scheduling case. In: Proceedings of the 6th International Conference on parallel problem solving from nature, pp 253–262Google Scholar
  8. Cheng H, Yang S (2010) Multi-population genetic algorithms with immigrants scheme for dynamic shortest path routing problems in mobile ad hoc networks. In: Proceedings of the 2010 international conference on applications of evolutionary computation, Springer, Verlag, pp 562–571Google Scholar
  9. Cruz C, González J, Pelta D (2011) Optimization in dynamic environments: a survey on problems, methods and measures. Soft Comput 15(7):1427–1448CrossRefGoogle Scholar
  10. Goldberg DE, Smith RE (1987) Nonstationary function optimization using genetic algorithms with dominance and diploidy. In: Proceedings of the Second International Conference on genetic algorithms and their application, pp 59–68Google Scholar
  11. Grefenstette J (1992) Genetic algorithms for changing environments. In: parallel problem solving from nature 2, Elsevier, USA, pp 137–144Google Scholar
  12. Hadad BS, Eick CF (1997) Supporting polyploidy in genetic algorithms using dominance vectors. In: evolutionary programming VI, lecture notes in computer science, vol 1213, Springer, Berlin, pp 223–234Google Scholar
  13. Li C, Yang M, Kang L (2006) A new approach to solving dynamic traveling salesman problems. In: Proceedings of the 6th International Conference on simulated evolution and learning, pp 236–243Google Scholar
  14. Mavrovouniotis M, Yang S (2011) A memetic ant colony optimization algorithm for the dynamic travelling salesman problem. Soft Comput 15:1405–1425CrossRefGoogle Scholar
  15. Morrison R, De Jong K (2000) Triggered hypermutation revisited. In: Proceedings of the 2000 Congress on evolutionary computation 2:1025–1032Google Scholar
  16. Morrison RW (2003) Performance measurement in dynamic environments. In: A.M. Barry (ed.) Proceedings of the bird of a feather workshops, genetic and evolutionary computation conference, AAAI, Chigaco, pp 99–102Google Scholar
  17. Morrison RW, De Jong KA (1999) A test problem generator for non-stationary environments. In: Proceedings of the 1999 Congress on evolutionary computation, pp 2047–2053Google Scholar
  18. Nguyen TT, Yang S, Branke J (2012) Evolutionary dynamic optimization: a survey of the state of the art. Swarm Evol Comput 6(0):1–24CrossRefGoogle Scholar
  19. Pisinger D (1997) A minimal algorithm for the 0–1 knapsack problem. Oper Res 45(5):758–767MathSciNetzbMATHCrossRefGoogle Scholar
  20. Pisinger D (1999) Core problems in knapsack algorithms. Oper Res 47:570–575MathSciNetzbMATHCrossRefGoogle Scholar
  21. Rand W, Riolo R (2005) Measurements for understanding the behavior of the genetic algorithm in dynamic environments: a case study using the shaky ladder hyperplane-defined functions. In: GECCO Workshops, pp 32–38Google Scholar
  22. Richter H (2009) Detecting change in dynamic fitness landscapes. In: Proceedings of the eleventh conference on Congress on evolutionary computation, IEEE Press, pp 1613–1620Google Scholar
  23. Rohlfshagen P, Bullinaria J (2006) Alternative splicing in evolutionary computation: adaptation in dynamic environments. In: Proceedings of the 2006 IEEE Congress on evolutionary computation, pp 2277–2284Google Scholar
  24. Rohlfshagen P, Yao X (2009) The dynamic knapsack problem revisited: a new benchmark problem for dynamic combinatorial optimisation. In: applications of evolutionary computing, lecture notes in computer science 5484:745–754Google Scholar
  25. Simões A, Costa E (2008) Evolutionary algorithms for dynamic environments: prediction using linear regression and markov chains. In: parallel problem solving from NaturePPSN X, lecture notes in computer science, vol 5199, Springer Berlin, pp 306–315Google Scholar
  26. Simões A, Costa E (2009) Improving prediction in evolutionary algorithms for dynamic environments. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, GECCO ’09, ACM, pp 875–882Google Scholar
  27. Tinós R, Yang S (2007) A self-organizing random immigrants genetic algorithm for dynamic optimization problems. Genetic Program Evolvable Mach 8:255–286CrossRefGoogle Scholar
  28. Weicker K (2002) Performance measures for dynamic environments. In: parallel problem solving from nature PPSN VII, Springer, Verlag, pp 64–73Google Scholar
  29. Yang S (2003) Non-stationary problem optimization using the primal-dual genetic algorithm. In: Proceedings of the 2003 Congress on Evol Comput, pp 2246–2253Google Scholar
  30. Yang S (2003) Non-stationary problem optimization using the primal-dual genetic algorithm. In: Proceedings of the 2003 Congress on Evolutionary Computation 3:2246–2253Google Scholar
  31. Yang S (2008) Genetic algorithms with memory-and elitism-based immigrants in dynamic environments. Evol Comput 16:385–416CrossRefGoogle Scholar
  32. Yang S, Tinós R (2007) A hybrid immigrants scheme for genetic algorithms in dynamic environments. Int J Automat Comput 4:243–254CrossRefGoogle Scholar
  33. Yang S, Yao X (2005) Experimental study on population-based incremental learning algorithms for dynamic optimization problems. Soft Comput 9:815–834zbMATHCrossRefGoogle Scholar
  34. Yang S, Yao X (2008) Population-based incremental learning with associative memory for dynamic environments. IEEE Transactions on evolutionary computation, 12(5):542–561Google Scholar
  35. Younes A, Calamai P, Basir O (2005) Generalized benchmark generation for dynamic combinatorial problems. In: Proceedings of the 2005 workshops on genetic and evolutionary computation, GECCO ’05, pp 25–31Google Scholar
  36. Yu X, Jin Y, Tang K, Yao X (2010) Robust optimization over time—a new perspective on dynamic optimization problems. In: IEEE Congress on evolutionary computation, pp 1–6Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Hajer Ben-Romdhane
    • 1
    Email author
  • Enrique Alba
    • 2
  • Saoussen Krichen
    • 3
  1. 1.LARODEC LaboratoryISG of TunisLe BardoTunisia
  2. 2.Universidad de MálagaMálagaSpain
  3. 3.LARODEC Laboratory, ISG of TunisFSJEG de JendoubaJendoubaTunisia

Personalised recommendations