Algorithmica

, Volume 64, Issue 4, pp 623–642 | Cite as

Black-Box Search by Unbiased Variation

Article

Abstract

The complexity theory for black-box algorithms, introduced by Droste, Jansen, and Wegener (Theory Comput. Syst. 39:525–544, 2006), describes common limits on the efficiency of a broad class of randomised search heuristics. There is an obvious trade-off between the generality of the black-box model and the strength of the bounds that can be proven in such a model. In particular, the original black-box model provides for well-known benchmark problems relatively small lower bounds, which seem unrealistic in certain cases and are typically not met by popular search heuristics.

In this paper, we introduce a more restricted black-box model for optimisation of pseudo-Boolean functions which we claim captures the working principles of many randomised search heuristics including simulated annealing, evolutionary algorithms, randomised local search, and others. The key concept worked out is an unbiased variation operator. Considering this class of algorithms, significantly better lower bounds on the black-box complexity are proved, amongst them an Ω(nlogn) bound for functions with unique optimum. Moreover, a simple unimodal function and plateau functions are considered. We show that a simple (1+1) EA is able to match the runtime bounds in several cases.

Keywords

Runtime analysis Black-box complexity 

References

  1. 1.
    Anil, G., Wiegand, R.P.: Black-box search by elimination of fitness functions. In: Proceedings of the 10th International Workshop on Foundations of Genetic Algorithms (FOGA’09), pp. 67–78. ACM Press, New York (2009) Google Scholar
  2. 2.
    Böttcher, S., Doerr, B., Neumann, F.: Optimal fixed and adaptive mutation rates for the leadingones problem. In: Proceedings of the 11th international conference on Parallel Problem Solving from Nature (PPSN’10), pp. 1–10. Springer, Berlin (2010) CrossRefGoogle Scholar
  3. 3.
    Cathabard, S., Lehre, P.K., Yao, X.: Non-uniform mutation rates for problems with unknown solution lengths. In: Proceedings of the 11th International Workshop on Foundations of Genetic Algorithms (FOGA’11), pp. 173–180. ACM, New York (2011) Google Scholar
  4. 4.
    Chvátal, V.: The tail of the hypergeometric distribution. Discrete Math. 25(3), 285–287 (1979) MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Doerr, B., Fouz, M., Witt, C.: Quasirandom evolutionary algorithms. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation (GECCO’10), pp. 1457–1464. ACM, New York (2010) CrossRefGoogle Scholar
  6. 6.
    Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004) MATHCrossRefGoogle Scholar
  7. 7.
    Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) Evolutionary Algorithm. Theor. Comput. Sci. 276, 51–81 (2002) MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Droste, S., Jansen, T., Wegener, I.: Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst. 39(4), 525–544 (2006) MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Droste, S., Wiesmann, D.: Metric based evolutionary algorithms. In: Proceedings of Genetic Programming, European Conference. LNCS, vol. 1802, pp. 29–43. Springer, Berlin (2000) Google Scholar
  10. 10.
    Fournier, H., Teytaud, O.: Lower bounds for comparison based evolution strategies using VC-dimension and sign patterns. Algorithmica 59, 387–408 (2011) MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Giel, O., Wegener, I.: Evolutionary algorithms and the maximum matching problem. In: Proceedings of the 20th Annual Symposium on Theoretical Aspects of Computer Science (STACS’03), pp. 415–426 (2003) Google Scholar
  12. 12.
    Hajek, B.: Hitting-time and occupation-time bounds implied by drift analysis with applications. Adv. Appl. Probab. 13(3), 502–525 (1982) CrossRefGoogle Scholar
  13. 13.
    He, J., Yao, X.: A study of drift analysis for estimating computation time of evolutionary algorithms. Natural Computing, 3(1) (2004) Google Scholar
  14. 14.
    Jägersküpper, J.: Algorithmic analysis of a basic evolutionary algorithm for continuous optimization. Theor. Comput. Sci. 39(3), 329–347 (2007) CrossRefGoogle Scholar
  15. 15.
    Jansen, T., De Jong, K.A., Wegener, I.: On the choice of the offspring population size in evolutionary algorithms. Evol. Comput. 13(4), 413–440 (2005) CrossRefGoogle Scholar
  16. 16.
    Jansen, T., Sudholt, D.: Analysis of an asymmetric mutation operator. Evol. Comput. 18, 1–26 (2010) CrossRefGoogle Scholar
  17. 17.
    Johnson, D.S., Papadimitriou, C.H., Yannakakis, M.: How easy is local search? J. Comput. Syst. Sci. 37(1), 79–100 (1988) MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Kennedy, J., Eberhart, R.C.: Swarm Intelligence. Morgan Kaufmann, San Mateo (2001) Google Scholar
  19. 19.
    Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983) MathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Larrañaga, P., Lozano, J.A.: Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic, Dordrecht (2002) MATHGoogle Scholar
  21. 21.
    Lehre, P.K., Witt, C.: Black-box search by unbiased variation. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation (GECCO’10), pp. 1441–1448. ACM, New York (2010) CrossRefGoogle Scholar
  22. 22.
    Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, Cambridge (1995) MATHGoogle Scholar
  23. 23.
    Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theor. Comput. Sci. 378(1), 32–40 (2007) MathSciNetMATHCrossRefGoogle Scholar
  24. 24.
    Neumann, F., Witt, C.: Ant colony optimization and the minimum spanning tree problem. In: Proceedings of Learning and Intelligent Optimization (LION’08), pp. 153–166 (2008) CrossRefGoogle Scholar
  25. 25.
    Neumann, F., Witt, C.: Bioinspired Computation in Combinatorial Optimization: Algorithms and Their Computational Complexity, 1st edn. Springer, New York (2010) MATHCrossRefGoogle Scholar
  26. 26.
    Oliveto, P.S., Lehre, P.K., Neumann, F.: Theoretical analysis of rank-based mutation—combining exploration and exploitation. In: Proceedings of the 10th IEEE Congress on Evolutionary Computation (CEC’09), pp. 1455–1462. IEEE, New York (2009) CrossRefGoogle Scholar
  27. 27.
    Rowe, J.E., Vose, M.D.: Unbiased black box search algorithms. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (GECCO’11), pp. 2035–2042. ACM, New York (2011) Google Scholar
  28. 28.
    Rowe, J.E., Vose, M.D., Wright, A.H.: Neighborhood graphs and symmetric genetic operators. In: Proceedings of the 9th International Workshop on Foundations of Genetic Algorithms (FOGA’07). LNCS, vol. 4436, pp. 110–122 (2007) Google Scholar
  29. 29.
    Sudholt, D.: General lower bounds for the running time of evolutionary algorithms. In: Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN’10), pp. 124–133. Springer, Berlin (2010) CrossRefGoogle Scholar
  30. 30.
    Teytaud, O., Gelly, S.: General lower bounds for evolutionary algorithms. In: Proceedings of the 9th International Conference on Parallel Problem Solving from Nature (PPSN’06). LNCS, vol. 4193, pp. 21–31. Springer, Berlin (2006) CrossRefGoogle Scholar
  31. 31.
    Teytaud, O., Gelly, S., Mary, J.: On the ultimate convergence rates for isotropic algorithms and the best choices among various forms of isotropy. In: Proceedings of the 9th International Conference on Parallel Problem Solving from Nature (PPSN’06). LNCS, vol. 4193, pp. 32–41. Springer, Berlin (2006) CrossRefGoogle Scholar
  32. 32.
    Wegener, I.: Methods for the analysis of evolutionary algorithms on pseudo-Boolean functions. In: Sarker, R., Mohammadian, M., Yao, X. (eds.) Evolutionary Optimization, pp. 349–369. Kluwer Academic, Dordrecht (2002) Google Scholar
  33. 33.
    Wegener, I., Witt, C.: On the optimization of monotone polynomials by simple randomized search heuristics. Comb. Probab. Comput. 14(1), 225–247 (2005) MathSciNetMATHCrossRefGoogle Scholar
  34. 34.
    Witt, C.: Runtime analysis of the (μ+1) EA on simple pseudo-Boolean functions. Evol. Comput. 14(1), 65–86 (2006) MathSciNetGoogle Scholar
  35. 35.
    Zarges, C.: Theoretical foundations of artificial immune systems. PhD thesis, Technische Universität Dortmund (2011) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.School of Computer ScienceUniversity of NottinghamNottinghamUK
  2. 2.Technical University of DenmarkKgs. LyngbyDenmark

Personalised recommendations