Theory of Randomized Search Heuristics
Randomized search heuristics such as evolutionary algorithms, evolution strategies, ant colony optimizers etc. are optimization algorithms that can be applied to a wide class of problems ranging from combinatorial to continuous optimization. They are popular in practice because they are generally easy to implement, their application requires little assumptions on the function to be optimized and provide solutions to a problem in a comparatively short time. Their theoretical analysis has gained momentum over the past decade. Runtime analysis of various randomized search heuristics, using and adapting methods from the analysis of randomized algorithms, has emerged as a new line of research, and general theoretical tools that can be applied to different algorithms have been developed. In 2007, a theory track dedicated to analyses of this kind was established at the Genetic and Evolutionary Computation Conference (GECCO), one of the largest conference on randomized search heuristics.
This special issue on Theory of Randomized Search Heuristics of Algorithmica contains papers that are extensions or follow-ups of papers from the theory track of the GECCO held July 7–11, 2010 in Portland, Oregon. Each article has undergone a new rigorous journal review process from which four papers have been selected. They contribute to improve our theoretical knowledge about randomized search heuristics as well as they enhance general tools and techniques for their analysis.
The paper Black-Box Search by Unbiased Variation by Lehre and Witt (doi: 10.1007/s00453-012-9616-8) reconsiders and refines the black-box model of randomized search heuristics originally proposed by Droste, Jansen and Wegener (Theory Comput. Syst. 39(4):525–544, 2006). As a crucial restriction that is typically met by search heuristics, unbiasedness of the sampling distribution of variation operators is introduced. On classical test problems, upper and lower bounds on the black-box complexity are proved, which match the running time of basic search heuristics more closely than the original model.
The paper A Simple Ant Colony Optimizer for Stochastic Shortest Path Problems by Sudholt and Thyssen (doi: 10.1007/s00453-011-9606-2) puts forward the running time analysis of ant colony optimization in combinatorial optimization. For the first time, optimization under noise is considered in this context. The dependency of running time and noise strength is investigated in different noise models. Moreover, results on the approximation capability are proved.
The paper Multiplicative Drift Analysis by Doerr, Johannsen and Winzen (doi: 10.1007/s00453-012-9622-x) introduces a multiplicative drift theorem for analyzing the runtime of randomized search heuristics. Compared to classical drift theorems, the multiplicative version captures the progress of search heuristics on many problems in a more natural way and leads to improved bounds on the running time. Such results are shown for the linear function problem and combinatorial optimization problems such as minimum spanning trees, shortest paths and Euler tours.
The paper Theoretical Foundation for CMA-ES from Information Geometry Perspective (doi: 10.1007/s00453-011-9564-8) by Akimoto, Nagata, Ono and Kobayashi concerns optimization in continuous domain. It establishes the connection between the state-of-the-art Covariance-Matrix-Adaptation Evolution-Strategy (CMA-ES) algorithm and optimization on statistical manifolds by means of natural gradients. By shedding light on a fundamental design principle of CMA-ES and linking it to a broader class of algorithms, this theoretical result certainly explains a big part of the success of the CMA-ES algorithm in practice.
We would like to thank all authors for contributing in this special issue as well as the referees for their careful work. Finally, we are indebted to Professor Ming-Yang Kao, editor of Algorithmica, and his team for their help and support.