Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Best practices in measuring algorithm performance for dynamic optimization problems

Abstract

Dynamic optimization problems (DOPs) have attracted considerable attention due to the wide range of problems they can be applied to. Lots of efforts have been expended in modeling dynamic situations, proposing algorithms, and analyzing the results (too often in a visual way). Numeric performance measurements and their statistical validation have been however barely used in the literature. Most of works in DOPs report only the best-of-generation fitness, due to its simplicity of computation. Although this measure indicates the best algorithm in terms of fitness, it does not provide any details about the actual strength and weakness of each algorithm. In this article, we conduct a comparative study among algorithms of different search modes via several performance measures to demonstrate their relative advantages. We discuss the role of using different performance measures in drawing balanced conclusions about algorithms for DOPs.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. Alba E (2005) Parallel metaheuristics. Wiley, Inc., USA

  2. Alba E, Sarasola B (2010) Abc, a new performance tool for algorithms solving dynamic optimization problems. In: IEEE Congress on evolutionary computation, pp 1–7

  3. Alba E, Sarasola B (2010) Measuring fitness degradation in dynamic optimization problems. In: Applications of evolutionary computation, Lecture notes in computer science, vol 6024, Springer, Heidelberg, pp 572–581

  4. Back T (1998) On the behavior of evolutionary algorithms in dynamic environments. In: The 1998 IEEE International Conference on evolutionary computation proceedings, pp 446–451

  5. Bierwirth C, Mattfeld D (1999) Production scheduling and rescheduling with genetic algorithms. Evol Comput 7(1):1–17

  6. Branke J (1999) Memory enhanced evolutionary algorithms for changing optimization problems. In: IEEE Proceedings of the 1999 Congress on evolutionary computation, vol 3, pp 1875–1882

  7. Branke J, Mattfeld DC (2000) Anticipation in dynamic optimization: the scheduling case. In: Proceedings of the 6th International Conference on parallel problem solving from nature, pp 253–262

  8. Cheng H, Yang S (2010) Multi-population genetic algorithms with immigrants scheme for dynamic shortest path routing problems in mobile ad hoc networks. In: Proceedings of the 2010 international conference on applications of evolutionary computation, Springer, Verlag, pp 562–571

  9. Cruz C, González J, Pelta D (2011) Optimization in dynamic environments: a survey on problems, methods and measures. Soft Comput 15(7):1427–1448

  10. Goldberg DE, Smith RE (1987) Nonstationary function optimization using genetic algorithms with dominance and diploidy. In: Proceedings of the Second International Conference on genetic algorithms and their application, pp 59–68

  11. Grefenstette J (1992) Genetic algorithms for changing environments. In: parallel problem solving from nature 2, Elsevier, USA, pp 137–144

  12. Hadad BS, Eick CF (1997) Supporting polyploidy in genetic algorithms using dominance vectors. In: evolutionary programming VI, lecture notes in computer science, vol 1213, Springer, Berlin, pp 223–234

  13. Li C, Yang M, Kang L (2006) A new approach to solving dynamic traveling salesman problems. In: Proceedings of the 6th International Conference on simulated evolution and learning, pp 236–243

  14. Mavrovouniotis M, Yang S (2011) A memetic ant colony optimization algorithm for the dynamic travelling salesman problem. Soft Comput 15:1405–1425

  15. Morrison R, De Jong K (2000) Triggered hypermutation revisited. In: Proceedings of the 2000 Congress on evolutionary computation 2:1025–1032

  16. Morrison RW (2003) Performance measurement in dynamic environments. In: A.M. Barry (ed.) Proceedings of the bird of a feather workshops, genetic and evolutionary computation conference, AAAI, Chigaco, pp 99–102

  17. Morrison RW, De Jong KA (1999) A test problem generator for non-stationary environments. In: Proceedings of the 1999 Congress on evolutionary computation, pp 2047–2053

  18. Nguyen TT, Yang S, Branke J (2012) Evolutionary dynamic optimization: a survey of the state of the art. Swarm Evol Comput 6(0):1–24

  19. Pisinger D (1997) A minimal algorithm for the 0–1 knapsack problem. Oper Res 45(5):758–767

  20. Pisinger D (1999) Core problems in knapsack algorithms. Oper Res 47:570–575

  21. Rand W, Riolo R (2005) Measurements for understanding the behavior of the genetic algorithm in dynamic environments: a case study using the shaky ladder hyperplane-defined functions. In: GECCO Workshops, pp 32–38

  22. Richter H (2009) Detecting change in dynamic fitness landscapes. In: Proceedings of the eleventh conference on Congress on evolutionary computation, IEEE Press, pp 1613–1620

  23. Rohlfshagen P, Bullinaria J (2006) Alternative splicing in evolutionary computation: adaptation in dynamic environments. In: Proceedings of the 2006 IEEE Congress on evolutionary computation, pp 2277–2284

  24. Rohlfshagen P, Yao X (2009) The dynamic knapsack problem revisited: a new benchmark problem for dynamic combinatorial optimisation. In: applications of evolutionary computing, lecture notes in computer science 5484:745–754

  25. Simões A, Costa E (2008) Evolutionary algorithms for dynamic environments: prediction using linear regression and markov chains. In: parallel problem solving from NaturePPSN X, lecture notes in computer science, vol 5199, Springer Berlin, pp 306–315

  26. Simões A, Costa E (2009) Improving prediction in evolutionary algorithms for dynamic environments. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, GECCO ’09, ACM, pp 875–882

  27. Tinós R, Yang S (2007) A self-organizing random immigrants genetic algorithm for dynamic optimization problems. Genetic Program Evolvable Mach 8:255–286

  28. Weicker K (2002) Performance measures for dynamic environments. In: parallel problem solving from nature PPSN VII, Springer, Verlag, pp 64–73

  29. Yang S (2003) Non-stationary problem optimization using the primal-dual genetic algorithm. In: Proceedings of the 2003 Congress on Evol Comput, pp 2246–2253

  30. Yang S (2003) Non-stationary problem optimization using the primal-dual genetic algorithm. In: Proceedings of the 2003 Congress on Evolutionary Computation 3:2246–2253

  31. Yang S (2008) Genetic algorithms with memory-and elitism-based immigrants in dynamic environments. Evol Comput 16:385–416

  32. Yang S, Tinós R (2007) A hybrid immigrants scheme for genetic algorithms in dynamic environments. Int J Automat Comput 4:243–254

  33. Yang S, Yao X (2005) Experimental study on population-based incremental learning algorithms for dynamic optimization problems. Soft Comput 9:815–834

  34. Yang S, Yao X (2008) Population-based incremental learning with associative memory for dynamic environments. IEEE Transactions on evolutionary computation, 12(5):542–561

  35. Younes A, Calamai P, Basir O (2005) Generalized benchmark generation for dynamic combinatorial problems. In: Proceedings of the 2005 workshops on genetic and evolutionary computation, GECCO ’05, pp 25–31

  36. Yu X, Jin Y, Tang K, Yao X (2010) Robust optimization over time—a new perspective on dynamic optimization problems. In: IEEE Congress on evolutionary computation, pp 1–6

Download references

Acknowledgments

Authors acknowledge funds from the CICE of the Junta de Andalucia, under contract P07-TIC-03044 (DIRICOM http://diricom.lcc.uma.es) and Spanish Ministry of Sciences and Innovation (MICINN) and FEDER under contracts TIN2011-28194 (RoadMe http://roadme.lcc.uma.es) and TIN2008-06491-C04-01 (M* http://mstar.lcc.uma.es). Also, from the European COADVISE project number 230833.

Author information

Correspondence to Hajer Ben-Romdhane.

Additional information

Communicated by A-A. Tantar.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ben-Romdhane, H., Alba, E. & Krichen, S. Best practices in measuring algorithm performance for dynamic optimization problems. Soft Comput 17, 1005–1017 (2013). https://doi.org/10.1007/s00500-013-0989-7

Download citation

Keywords

  • Dynamic optimization problems
  • Evolutionary algorithms
  • Genetic algorithms
  • Performance measure