Abstract
Metaheuristic optimization algorithms are characterized by stochastic behavior, making each optimization run unique. In addition, metaheuristic algorithms are usually governed by a number of control parameters that require problem-specific tuning. Many publications on metaheuristic algorithms lack the kind of rigorous statistical convergence assessment that is needed to compensate for the above uncertainties, making it impossible to assess the optimality of the resulting design or the effectiveness of the optimization method. In this contribution, we propose a method to assess the performance (i.e. the ability to find the best known solution and the associated computational cost) of a metaheuristic algorithm that takes into account the influence of its control parameters. First, a large number of simulations (independent optimization runs) are performed, where the values of the control parameters are randomly selected from predefined sets of realistic possibilities. Next, for every value of every control parameter, the corresponding subset of simulations is considered in order to infer the relevant conditional performance statistics. As a example, the approach is used to assess the performance of the genetic algorithm built into matlab for a 25-bar truss test case. It is observed that, for the algorithm and the test case considered, the majority of the control parameters have little influence on algorithm performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adenso-Diaz, B., Laguna, M.: Fine-tuning of algorithms using fractional experimental designs and local search. Oper. Res. 54(1), 99–114 (2006)
Agresti, A., Caffo, B.: Simple and effective confidence intervals for proportions and differences of proportions result from adding two successes and two failures. Am. Stat. 54(4), 280–288 (2000)
Agresti, A., Coull, B.: Approximate is better than “exact” for interval estimation of binomial proportions. Am. Stat. 52(2), 119–126 (1998)
Bäck, T., Schwefel, H.P.: An overview of evolutionary algorithms for parameter optimization. Evol. Comput. 1(1), 1–23 (1993)
Baghlani, A., Makiabadi, M., Maheri, M.: Sizing optimization of truss structures by an efficient constraint-handling strategy in TLBO. J. Comput. Civil Eng. 31(4), 04017004 (2017)
Bartz-Beielstein, T., Markon, S.: Tuning search algorithms for real-world applications: a regression tree based approach. In: Proceedings of the 2004 Congress on Evolutionary Computation, vol. 1, pp. 1111–1118. IEEE, Portland, Oregon, USA (2004)
Bartz-Beielstein, T., Preuss, M.: Experimental research in evolutionary computation. In: Proceedings of the 9th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 3001–3020. ACM, London, UK (2007)
Bartz-Beielstein, T., Parsopoulos, K., Vrahatis, M.: Design and analysis of optimization algorithms using computational statistics. Appl. Numer. Anal. Comput. Math. 1(2), 413–433 (2004)
Bartz-Beielstein, T., Lasarczyk, C., Preuß, M.: Sequential parameter optimization. In: 2005 IEEE Congress on Evolutionary Computation, vol. 1, pp. 773–780. IEEE, Edinburgh, Scotland, UK (2005)
Birattari, M., Stützle, T., Paquete, L., Varrentrapp, K.: A racing algorithm for configuring metaheuristics. In: Proceedings of the 4th Annual Conference on Genetic and Evolutionary Computation, pp. 11–18. Morgan Kaufmann Publishers Inc., New York City, New York, USA (2002)
Castillo-Valdivieso, P., Merelo, J., Prieto, A., Rojas, I., Romero, G.: Statistical analysis of the parameters of a neuro-genetic algorithm. IEEE Trans. Neural Netw. 13(6), 1374–1394 (2002)
Coy, S., Golden, B., Runger, G., Wasil, E.: Using experimental design to find effective parameter settings for heuristics. J. Heuristics 7(1), 77–97 (2001)
De Jong, K.: Analysis of the behavior of a class of genetic adaptive systems. Ph.D. thesis, University of Michigan, Computer and Communication Sciences Department (1975)
Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. programm. 91(2), 201–213 (2002)
Eiben, A., Smit, S.: Parameter tuning for configuring and analyzing evolutionary algorithms. Swarm Evol. Comput. 1(1), 19–31 (2011)
François, O., Lavergne, C.: Design of evolutionary algorithms - a statistical perspective. IEEE Trans. Evol. Comput. 5(2), 129–148 (2001)
Gomes, W., Beck, A., Lopez, R., Miguel, L.: A probabilistic metric for comparing metaheuristic optimization algorithms. Struct. Saf. 70, 59–70 (2018)
Grefenstette, J.: Optimization of control parameters for genetic algorithms. IEEE Trans. Syst. Man Cybern. 16(1), 122–128 (1986)
Haftka, R.: Requirements for papers focusing on new or improved global optimization algorithms. Struct. Multidiscip. Optim. 54(1), 1–1 (2016)
Hooker, J.: Testing heuristics: we have it all wrong. J. Heuristics 1(1), 33–42 (1995)
Kohavi, R., John, G.: Automatic parameter selection by minimizing estimated error. In: Machine Learning Proceedings 1995, pp. 304–312. Elsevier, Tahoe City, California, USA (1995)
Le Riche, R., Haftka, R.: On global optimization articles in SMO. Struct. Multidiscip. Optim. 46(5), 627–629 (2012)
Moré, J., Wild, S.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)
Myers, R., Hancock, E.: Empirical modelling of genetic algorithms. Evol. Comput. 9(4), 461–493 (2001)
Pedersen, M., Chipperfield, A.: Simplifying particle swarm optimization. Appl. Soft Comput. 10(2), 618–628 (2010)
Rardin, R., Uzsoy, R.: Experimental evaluation of heuristic optimization algorithms: a tutorial. J. Heuristics 7(3), 261–304 (2001)
Rechenberg, I.: Evolution strategy: optimization of technical systems by means of biological evolution. Frommann-Holzboog Stuttg. 104, 15–16 (1973)
Sacks, J., Welch, W., Mitchell, T., Wynn, H.: Design and analysis of computer experiments. Stat. Sci., 409–423 (1989)
Schwefel, H.P.: Evolution and Optimum Seeking. Sixth-Generation Computer Technology Series. Wiley, New York (1995)
Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)
Xu, J., Chiu, S., Glover, F.: Fine-tuning a tabu search algorithm with statistical tests. Int. Trans. Oper. Res. 5(3), 233–244 (1998)
Yuan, B., Gallagher, M.: Combining meta-EAs and racing for difficult EA parameter tuning tasks. In: Parameter Setting in Evolutionary Algorithms. Springer, Berlin, Heidelberg, pp. 121–142 (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Dillen, W., Lombaert, G., Voeten, N., Schevenels, M. (2019). Performance Assessment of Metaheuristic Algorithms for Structural Optimization Taking into Account the Influence of Control Parameters. In: Rodrigues, H., et al. EngOpt 2018 Proceedings of the 6th International Conference on Engineering Optimization. EngOpt 2018. Springer, Cham. https://doi.org/10.1007/978-3-319-97773-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-97773-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97772-0
Online ISBN: 978-3-319-97773-7
eBook Packages: EngineeringEngineering (R0)