Journal of Combinatorial Optimization

, Volume 35, Issue 4, pp 1261–1285 | Cite as

Quantile and mean value measures of search process complexity

  • Jaromír Kukal
  • Matej Mojzeš


Performance measures of metaheuristic algorithms assess the quality of a search process by statistically analysing its performance. Such criteria serve two purposes: they provide the verdict on which algorithm is better for what task, and they help applying an algorithm on a given task in the most effective way. The latter goal may be achieved by an appropriate restart strategy of the search process. Furthermore, these criteria are traditionally based on analysis of the search step mean value. Our aim is to elaborate the mean value analysis as well, but via a novel and more general quantile-based analytic approach, which can be used to define new measures. We prove and demonstrate this purpose on three quantile-based performance measures.


Optimization Performance measure Time complexity Restart strategy Quantile Mean value 


  1. Auger A, Hansen N (2005) Performance evaluation of an advanced local search evolutionary algorithm. In: The 2005 IEEE congress on evolutionary computation, 2005, vol 2, pp 1777–1784. IEEE, 2005Google Scholar
  2. Barrero DF, Muñoz P, Camacho D, R-Moreno MD (2015) On the statistical distribution of the expected run-time in population-based search algorithms. Soft Comput 19(10):2717–2734. ISSN 1433-7479.
  3. Burkard RE, Karisch SE, Rendl F (1997) Qaplib-a quadratic assignment problem library. J Glob Optim 10(4):391–403MathSciNetCrossRefzbMATHGoogle Scholar
  4. Cerny V (1985) Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm. J Optim Theory Appl 45(1):41–51MathSciNetCrossRefzbMATHGoogle Scholar
  5. Chen H, Zhu Y, Hu K, He X (2010) Hierarchical swarm model: a new approach to optimization. Discrete Dyn Nat Soc 2010Google Scholar
  6. Feoktistov V (2006) Differential evolution—in search of solutions. Springer, New YorkzbMATHGoogle Scholar
  7. Gendreau M, Potvin J-Y (2010) Handbook of metaheuristics, vol 2. Springer, New YorkCrossRefzbMATHGoogle Scholar
  8. Golberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, ReadingGoogle Scholar
  9. Hansen N, Auger A, Finck S, Ros R (2010) Real-parameter black-box optimization benchmarking 2010: experimental setupGoogle Scholar
  10. Hilbert D (1933) Ein beitrag zur theorie des legendreschen polynoms. In: Algebra Invariantentheorie Geometrie, pp 367–370. Springer, BerlinGoogle Scholar
  11. Hoos H, Stützle T (1998) Evaluating las vegas algorithms: pitfalls and remedies. In: Proceedings of the fourteenth conference on uncertainty in artificial intelligence, pp 238–245. Morgan Kaufmann Publishers Inc., Los AltosGoogle Scholar
  12. Kharroubi F, He J, Tang J, Chen M, Chen L (2015) Evaluation performance of genetic algorithm and tabu search algorithm for solving the max-rwa problem in all-optical networks. J Comb Optim, 30(4):1042–1061. ISSN 1573-2886.
  13. Neocleous C, Schizas C (2002) Artificial neural network learning: a comparative review. In: Vouros GA, Panayiotopoulos T (eds) Methods and applications of artificial intelligence. Springer, Berlin, pp 300–313CrossRefGoogle Scholar
  14. Senthilkumar P, Shahabudeen P (2006) Ga based heuristic for the open job shop scheduling problem. Int J Adv Manuf Technol 30(3–4):297–301CrossRefGoogle Scholar
  15. Snyman J (2005) Practical mathematical optimization: an introduction to basic optimization theory and classical and new gradient-based algorithms, vol 97. Springer, New YorkzbMATHGoogle Scholar
  16. Stützle T, Hoos H (2004) Stochastic local search: foundations and applications. Elsevier, AmsterdamzbMATHGoogle Scholar
  17. Valsecchi A, Vanneschi L, Mauri G (2014) A study of search algorithms’ optimization speed. J Comb Optim, 27(2):256–270. ISSN 1382-6905.
  18. Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82CrossRefGoogle Scholar
  19. Yang X-S, Deb S (2009) Cuckoo search via lévy flights. In: Nature & biologically inspired computing, 2009. World Congress on NaBIC 2009, pp 210–214. IEEE, 2009Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Software Engineering, Faculty of Nuclear Sciences and Physical EngineeringCzech Technical University in PraguePragueCzech Republic

Personalised recommendations