Soft Computing

, Volume 19, Issue 10, pp 2717–2734 | Cite as

On the statistical distribution of the expected run-time in population-based search algorithms

  • David F. Barrero
  • Pablo Muñoz
  • David CamachoEmail author
  • María D. R-Moreno


Run-time analysis is a method that characterizes the run-time behaviour of an algorithm. It has been successfully used to analyse metaheuristics and stochastic local search algorithms. Some studies on genetic programming and related stochastic search algorithms suggested the existence of a rationale behind the run-time behaviour which could be exploited to better understand the algorithm looking at its run-time response. Under that hypothesis, this paper presents empirical evidence suggesting common statistical properties in the run-time of several types of population-based search algorithms, including genetic algorithms, genetic programming, grammatical evolution, differential evolution or particle swarm optimization. In this analysis, only the run-time of runs that were able to find a solution are considered; unsuccessful runs are not included in the analysis. The run-time to find a solution, measured by the number of evaluations, of some well-known evolutionary algorithms is empirically obtained, finding that it usually can be approximated using a lognormal distribution, with the exception of some difficult problems, which are better approximated by an exponential. We also show that the algorithm parameter settings might influence the yielding run-time statistical distribution; in particular, when there is no selective pressure, the run-time, measured as the number of evaluations to find a solution, follows a Weibull distribution instead of a lognormal one. Finally, we outline a framework using a simple theoretical discrete-time model, showing that the geometrically distributed run-time is a consequence of the lack of memory in the algorithm.


RTD Run-time analysis RTD analysis Statistical models 



We would like to thank the comments and suggestions received during the EuroGP 2011 presentation of the early results of this work. This work was supported by the Project of Castilla-La Mancha PEII-2014-015A (PEII11-0079-8929).


  1. Barr R, Golden B, Kelly J, Resende M, Stewart W (1995) Designing and reporting on computational experiments with heuristic methods. J Heuristics 1:9–32zbMATHCrossRefGoogle Scholar
  2. Barrero DF, Camacho D, R-Moreno MD (2010) Confidence intervals of success rates in evolutionary computation. In: Proceedings of the 12th annual conference on genetic and evolutionary computation, GECCO ’10, pp 975–976. ACM, Portland. doi: 10.1145/1830483.1830657
  3. Barrero DF, Castaño B, R-Moreno MD, Camacho D (2011) Statistical distribution of generation-to-success in GP: application to model accumulated success probability. In: Silva S, Foster JA, Nicolau M, Giacobini M, Machado P (eds) Proceedings of the 14th European conference on genetic programming, EuroGP 2011. LNCS, vol 6621, pp 155–166. Springer, Turin (2011)Google Scholar
  4. Chiarandini M, Stützle T (2002) Experimental evaluation of course timetabling algorithms. In: Technical report AIDA-02-05, Intellectics Group, Computer Science Department, Darmstadt University of Technology, DarmstadtGoogle Scholar
  5. Eiben AE, Smith JE (2009) Introduction to evolutionary computing. Chapter: Working with evolutionary algorithms, pp 241–258. Springer, New YorkGoogle Scholar
  6. Epstein S, Yun X (2010) From unsolvable to solvable: an exploration of simple changes. In: Workshops at the 24th AAAI conference on artificial intelligenceGoogle Scholar
  7. Feo T, Resende M, Smith S (1994) A greedy randomized adaptive search procedure for maximum independent set. Oper Res 860–878Google Scholar
  8. Frost D, Rish I, Vila L (1997) Summarizing CSP hardness with continuous probability distributions. In: Proceedings of the 14th national conference on artificial intelligence and 9th conference on innovative applications of artificial intelligence, AAAI conference on artificial intelligence, pp 327–333. AAAI Press, New YorkGoogle Scholar
  9. Gagliolo M, Legrand C (2010) Experimental methods for the analysis of optimization algorithms. Chapter: Algorithm survival analysis, pp 161–184. Springer, New YorkGoogle Scholar
  10. Hoos H, Stützle T (1998) Characterizing the run-time behavior of stochastic local search. In: Proceedings of the AAAI conference on artificial intelligenceGoogle Scholar
  11. Hoos H, Stützle T (1998) Evaluating Las Vegas algorithms—pitfalls and remedies. In: Proceedings of the 14th conference on uncertainty in artificial intelligence (UAI-98), pp 238–245. Morgan Kaufmann, MadisonGoogle Scholar
  12. Hoos H, Stützle T (1999) Towards a characterisation of the behaviour of stochastic local search algorithms for SAT. Artif Intell 112(1–2):213–232zbMATHCrossRefGoogle Scholar
  13. Hoos H, Stützle T (2000) Local search algorithms for SAT: an empirical evaluation. J Autom Reason 24(4):421–481zbMATHCrossRefGoogle Scholar
  14. Koza J (1992) Genetic programming: on the programming of computers by means of natural selection. MIT Press, CambrigezbMATHGoogle Scholar
  15. Koza JR (1994) Genetic programming II: automatic discovery of reusable programs. MIT Press, CambridgezbMATHGoogle Scholar
  16. Limpert E, Stahel WA, Abbt M (2001) Log-normal distributions across the sciences: keys and clues. BioScience 51(5):341–352CrossRefGoogle Scholar
  17. Luke S (2001) When short runs beat long runs. In: Proceedings of the genetic and evolutionary computation conference, pp 74–80. Morgan Kaufmann, San FranciscoGoogle Scholar
  18. McDermott J, De Jong K, OReilly UM, White DR, Luke S, Manzoni L, Castelli M, Vanneschi L, Jaskowski W, Krawiec K et al. (2012) Genetic programming needs better benchmarks, p 791. ACM Press, Philadelphia.
  19. Paterson N, Livesey M (2000) Performance comparison in genetic programming. In: Late breaking papers at the 2000 genetic and evolutionary computation conference, Las Vegas, pp 253–260Google Scholar
  20. Ribeiro CC, Rosseti I, Vallejos R (2009) On the use of run time distributions to evaluate and compare stochastic local search algorithms. In: Proceedings of the second international workshop on engineering stochastic local search algorithms., designing, implementing and analyzing effective heuristics, SLS ’09, pp 16–30. Springer, BerlinGoogle Scholar
  21. Stützle T, Hoos H (1999) Analyzing the run-time behaviour of iterated local search for the TSP. In: III Metaheuristics international conference. Kluwer Academic, Angra dos ReisGoogle Scholar
  22. Wilson EB (1927) Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 22:309–316Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • David F. Barrero
    • 1
  • Pablo Muñoz
    • 1
  • David Camacho
    • 2
    Email author
  • María D. R-Moreno
    • 1
  1. 1.Departamento de AutomáticaUniversidad de AlcaláAlcalá de HenaresSpain
  2. 2.Departamento de InformáticaUniversidad Autonónoma de MadridMadridSpain

Personalised recommendations