Skip to main content

Advertisement

SpringerLink
Log in

Best practices in measuring algorithm performance for dynamic optimization problems

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Dynamic optimization problems (DOPs) have attracted considerable attention due to the wide range of problems they can be applied to. Lots of efforts have been expended in modeling dynamic situations, proposing algorithms, and analyzing the results (too often in a visual way). Numeric performance measurements and their statistical validation have been however barely used in the literature. Most of works in DOPs report only the best-of-generation fitness, due to its simplicity of computation. Although this measure indicates the best algorithm in terms of fitness, it does not provide any details about the actual strength and weakness of each algorithm. In this article, we conduct a comparative study among algorithms of different search modes via several performance measures to demonstrate their relative advantages. We discuss the role of using different performance measures in drawing balanced conclusions about algorithms for DOPs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Alba E (2005) Parallel metaheuristics. Wiley, Inc., USA

  • Alba E, Sarasola B (2010) Abc, a new performance tool for algorithms solving dynamic optimization problems. In: IEEE Congress on evolutionary computation, pp 1–7

  • Alba E, Sarasola B (2010) Measuring fitness degradation in dynamic optimization problems. In: Applications of evolutionary computation, Lecture notes in computer science, vol 6024, Springer, Heidelberg, pp 572–581

  • Back T (1998) On the behavior of evolutionary algorithms in dynamic environments. In: The 1998 IEEE International Conference on evolutionary computation proceedings, pp 446–451

  • Bierwirth C, Mattfeld D (1999) Production scheduling and rescheduling with genetic algorithms. Evol Comput 7(1):1–17

    Article  Google Scholar 

  • Branke J (1999) Memory enhanced evolutionary algorithms for changing optimization problems. In: IEEE Proceedings of the 1999 Congress on evolutionary computation, vol 3, pp 1875–1882

  • Branke J, Mattfeld DC (2000) Anticipation in dynamic optimization: the scheduling case. In: Proceedings of the 6th International Conference on parallel problem solving from nature, pp 253–262

  • Cheng H, Yang S (2010) Multi-population genetic algorithms with immigrants scheme for dynamic shortest path routing problems in mobile ad hoc networks. In: Proceedings of the 2010 international conference on applications of evolutionary computation, Springer, Verlag, pp 562–571

  • Cruz C, González J, Pelta D (2011) Optimization in dynamic environments: a survey on problems, methods and measures. Soft Comput 15(7):1427–1448

    Article  Google Scholar 

  • Goldberg DE, Smith RE (1987) Nonstationary function optimization using genetic algorithms with dominance and diploidy. In: Proceedings of the Second International Conference on genetic algorithms and their application, pp 59–68

  • Grefenstette J (1992) Genetic algorithms for changing environments. In: parallel problem solving from nature 2, Elsevier, USA, pp 137–144

  • Hadad BS, Eick CF (1997) Supporting polyploidy in genetic algorithms using dominance vectors. In: evolutionary programming VI, lecture notes in computer science, vol 1213, Springer, Berlin, pp 223–234

  • Li C, Yang M, Kang L (2006) A new approach to solving dynamic traveling salesman problems. In: Proceedings of the 6th International Conference on simulated evolution and learning, pp 236–243

  • Mavrovouniotis M, Yang S (2011) A memetic ant colony optimization algorithm for the dynamic travelling salesman problem. Soft Comput 15:1405–1425

    Article  Google Scholar 

  • Morrison R, De Jong K (2000) Triggered hypermutation revisited. In: Proceedings of the 2000 Congress on evolutionary computation 2:1025–1032

  • Morrison RW (2003) Performance measurement in dynamic environments. In: A.M. Barry (ed.) Proceedings of the bird of a feather workshops, genetic and evolutionary computation conference, AAAI, Chigaco, pp 99–102

  • Morrison RW, De Jong KA (1999) A test problem generator for non-stationary environments. In: Proceedings of the 1999 Congress on evolutionary computation, pp 2047–2053

  • Nguyen TT, Yang S, Branke J (2012) Evolutionary dynamic optimization: a survey of the state of the art. Swarm Evol Comput 6(0):1–24

    Article  Google Scholar 

  • Pisinger D (1997) A minimal algorithm for the 0–1 knapsack problem. Oper Res 45(5):758–767

    Article  MathSciNet  MATH  Google Scholar 

  • Pisinger D (1999) Core problems in knapsack algorithms. Oper Res 47:570–575

    Article  MathSciNet  MATH  Google Scholar 

  • Rand W, Riolo R (2005) Measurements for understanding the behavior of the genetic algorithm in dynamic environments: a case study using the shaky ladder hyperplane-defined functions. In: GECCO Workshops, pp 32–38

  • Richter H (2009) Detecting change in dynamic fitness landscapes. In: Proceedings of the eleventh conference on Congress on evolutionary computation, IEEE Press, pp 1613–1620

  • Rohlfshagen P, Bullinaria J (2006) Alternative splicing in evolutionary computation: adaptation in dynamic environments. In: Proceedings of the 2006 IEEE Congress on evolutionary computation, pp 2277–2284

  • Rohlfshagen P, Yao X (2009) The dynamic knapsack problem revisited: a new benchmark problem for dynamic combinatorial optimisation. In: applications of evolutionary computing, lecture notes in computer science 5484:745–754

  • Simões A, Costa E (2008) Evolutionary algorithms for dynamic environments: prediction using linear regression and markov chains. In: parallel problem solving from NaturePPSN X, lecture notes in computer science, vol 5199, Springer Berlin, pp 306–315

  • Simões A, Costa E (2009) Improving prediction in evolutionary algorithms for dynamic environments. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, GECCO ’09, ACM, pp 875–882

  • Tinós R, Yang S (2007) A self-organizing random immigrants genetic algorithm for dynamic optimization problems. Genetic Program Evolvable Mach 8:255–286

    Article  Google Scholar 

  • Weicker K (2002) Performance measures for dynamic environments. In: parallel problem solving from nature PPSN VII, Springer, Verlag, pp 64–73

  • Yang S (2003) Non-stationary problem optimization using the primal-dual genetic algorithm. In: Proceedings of the 2003 Congress on Evol Comput, pp 2246–2253

  • Yang S (2003) Non-stationary problem optimization using the primal-dual genetic algorithm. In: Proceedings of the 2003 Congress on Evolutionary Computation 3:2246–2253

  • Yang S (2008) Genetic algorithms with memory-and elitism-based immigrants in dynamic environments. Evol Comput 16:385–416

    Article  Google Scholar 

  • Yang S, Tinós R (2007) A hybrid immigrants scheme for genetic algorithms in dynamic environments. Int J Automat Comput 4:243–254

    Article  Google Scholar 

  • Yang S, Yao X (2005) Experimental study on population-based incremental learning algorithms for dynamic optimization problems. Soft Comput 9:815–834

    Article  MATH  Google Scholar 

  • Yang S, Yao X (2008) Population-based incremental learning with associative memory for dynamic environments. IEEE Transactions on evolutionary computation, 12(5):542–561

    Google Scholar 

  • Younes A, Calamai P, Basir O (2005) Generalized benchmark generation for dynamic combinatorial problems. In: Proceedings of the 2005 workshops on genetic and evolutionary computation, GECCO ’05, pp 25–31

  • Yu X, Jin Y, Tang K, Yao X (2010) Robust optimization over time—a new perspective on dynamic optimization problems. In: IEEE Congress on evolutionary computation, pp 1–6

Download references

Acknowledgments

Authors acknowledge funds from the CICE of the Junta de Andalucia, under contract P07-TIC-03044 (DIRICOM http://diricom.lcc.uma.es) and Spanish Ministry of Sciences and Innovation (MICINN) and FEDER under contracts TIN2011-28194 (RoadMe http://roadme.lcc.uma.es) and TIN2008-06491-C04-01 (M* http://mstar.lcc.uma.es). Also, from the European COADVISE project number 230833.

Author information

Authors and Affiliations

  1. LARODEC Laboratory, ISG of Tunis, 41 Rue de la Liberté, Le Bardo, Tunisia

    Hajer Ben-Romdhane

  2. Universidad de Málaga, Boulevard Louis Pasteur s/n, Málaga, Spain

    Enrique Alba

  3. LARODEC Laboratory, ISG of Tunis, FSJEG de Jendouba, Avenue de l’U.M.A, 8189, Jendouba, Tunisia

    Saoussen Krichen

Authors
  1. Hajer Ben-Romdhane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enrique Alba
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Saoussen Krichen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hajer Ben-Romdhane.

Additional information

Communicated by A-A. Tantar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ben-Romdhane, H., Alba, E. & Krichen, S. Best practices in measuring algorithm performance for dynamic optimization problems. Soft Comput 17, 1005–1017 (2013). https://doi.org/10.1007/s00500-013-0989-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-013-0989-7

Keywords

Search

Navigation