Skip to main content

Ranking-Based Black-Box Complexity

Abstract

Randomized search heuristics such as evolutionary algorithms, simulated annealing, and ant colony optimization are a broadly used class of general-purpose algorithms. Analyzing them via classical methods of theoretical computer science is a growing field. While several strong runtime analysis results have appeared in the last 20 years, a powerful complexity theory for such algorithms is yet to be developed. We enrich the existing notions of black-box complexity by the additional restriction that not the actual objective values, but only the relative quality of the previously evaluated solutions may be taken into account by the black-box algorithm. Many randomized search heuristics belong to this class of algorithms.

We show that the new ranking-based model can give more realistic complexity estimates. The class of all binary-value functions has a black-box complexity of O(logn) in the previous black-box models, but has a ranking-based complexity of Θ(n).

On the other hand, for the class of all OneMax functions, we present a ranking-based black-box algorithm that has a runtime of Θ(n/logn), which shows that the OneMax problem does not become harder with the additional ranking-basedness restriction.

This is a preview of subscription content, access via your institution.

Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Algorithm 7
Algorithm 8
Algorithm 9
Algorithm 10
Algorithm 11
Algorithm 12
Algorithm 13

Notes

  1. 1.

    That is, {g(Om z (x 1)),…,g(Om z (x s ))}∩([g(Om z (x i )),g(Om z (y))]∪[g(Om z (y)),g(Om z (x i ))])={g(Om z (x i )),g(Om z (y))}.

References

  1. 1.

    Aaronson, S.: Lower bounds for local search by quantum arguments. In: Proceedings of the 36th Annual ACM Symposium on Theory of Computing (STOC’04), pp. 465–474. ACM Press, New York (2004)

    Google Scholar 

  2. 2.

    Aldous, D.: Minimization algorithms and random walk on the d-cube. Ann. Probab. 11, 403–413 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  3. 3.

    Anil, G., Wiegand, R.P.: Black-box search by elimination of fitness functions. In: Proceedings of Foundations of Genetic Algorithms (FOGA’09), pp. 67–78. ACM Press, New York (2009)

    Google Scholar 

  4. 4.

    Chvátal, V.: Mastermind. Combinatorica 3, 325–329 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  5. 5.

    Doerr, B., Johannsen, D., Kötzing, T., Lehre, P.K., Wagner, M., Winzen, C.: Faster black-box algorithms through higher arity operators. In: Proceedings of Foundations of Genetic Algorithms (FOGA’11), pp. 163–172. ACM Press, New York (2011)

    Google Scholar 

  6. 6.

    Doerr, B., Kötzing, T., Winzen, C.: Too fast unbiased black-box algorithms. In: Proceedings of the 13th Annual Genetic and Evolutionary Computation Conference (GECCO’11), pp. 2043–2050. ACM Press, New York (2011)

    Chapter  Google Scholar 

  7. 7.

    Doerr, B., Winzen, C.: Towards a complexity theory of randomized search heuristics: ranking-based black-box complexity. In: Proceedings of Computer Science Symposium in Russia (CSR’11), pp. 15–28. Springer, Berlin (2011)

    Google Scholar 

  8. 8.

    Droste, S., Jansen, T., Tinnefeld, K., Wegener, I.: A new framework for the valuation of algorithms for black-box optimization. In: Proceedings of Foundations of Genetic Algorithms (FOGA’03), pp. 253–270. Morgan Kaufmann, San Mateo (2003)

    Google Scholar 

  9. 9.

    Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theor. Comput. Sci. 276, 51–81 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  10. 10.

    Droste, S., Jansen, T., Wegener, I.: Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst. 39, 525–544 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  11. 11.

    Dubhashi, D., Panconesi, A.: Concentration of Measure for the Analysis of Randomized Algorithms. Cambridge University Press, Cambridge (2009)

    Book  MATH  Google Scholar 

  12. 12.

    Erdős, P., Rényi, A.: On two problems of information theory. Magy. Tud. Akad. Mat. Kut. Intéz. Közl. 8, 229–243 (1963)

    Google Scholar 

  13. 13.

    Fournier, H., Teytaud, O.: Lower bounds for comparison based evolution strategies using vc-dimension and sign patterns. Algorithmica 59, 387–408 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  14. 14.

    Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W.H. Freeman, New York (1990)

    Google Scholar 

  15. 15.

    Goodrich, M.T.: On the algorithmic complexity of the mastermind game with black-peg results. Inf. Process. Lett. 109, 675–678 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  16. 16.

    Hromkovič, J.: Algorithmics for Hard Problems: Introduction to Combinatorial Optimization, Randomization, Approximation, and Heuristics. Springer, Berlin (2003)

    Google Scholar 

  17. 17.

    Lehre, P.K., Witt, C.: Black-box search by unbiased variation. In: Proceedings of Genetic and Evolutionary Computation Conference (GECCO’10), pp. 1441–1448. ACM Press, New York (2010). A journal version of this paper is to appear in Algorithmica. doi:10.1007/s00453-012-9616-8

    Google Scholar 

  18. 18.

    Llewellyn, D.C., Tovey, C., Trick, M.: Local optimization on graphs. Discrete Appl. Math. 23, 157–178 (1989). Erratum 46, 93–94, 1993

    Article  MATH  MathSciNet  Google Scholar 

  19. 19.

    Michalewicz, Z., Fogel, D.B.: How to Solve It—Modern Heuristics, 2nd edn. Springer, Berlin (2004)

    Book  MATH  Google Scholar 

  20. 20.

    Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, Cambridge (1995)

    Book  MATH  Google Scholar 

  21. 21.

    Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theor. Comput. Sci. 378, 32–40 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  22. 22.

    Robbins, H.: A remark on Stirling’s formula. Am. Math. Mon. 62, 26–29 (1955)

    Article  MATH  MathSciNet  Google Scholar 

  23. 23.

    Teytaud, O., Gelly, S.: General lower bounds for evolutionary algorithms. In: Proceedings of the 9th International Conference on Parallel Problem Solving from Nature—PPSN IX (PPSN’06), pp. 21–31. Springer, Berlin (2006)

    Chapter  Google Scholar 

  24. 24.

    Yao, A.C.-C.: Probabilistic computations: toward a unified measure of complexity. In: Proceedings of 18th Annual Symposium on Foundations of Computer Science (FOCS’77), pp. 222–227 (1977)

    Chapter  Google Scholar 

  25. 25.

    Zhang, S.: New upper and lower bounds for randomized and quantum local search. In: Proceedings of the 38th Annual ACM Symposium on Theory of Computing (STOC’06), pp. 634–643. ACM Press, New York (2006)

    Google Scholar 

Download references

Acknowledgements

Carola Winzen is a recipient of the Google Europe Fellowship in Randomized Algorithms. This work is supported in part by this Google Fellowship.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Carola Winzen.

Additional information

A preliminary version of the results appeared in [7].

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Doerr, B., Winzen, C. Ranking-Based Black-Box Complexity. Algorithmica 68, 571–609 (2014). https://doi.org/10.1007/s00453-012-9684-9

Download citation

Keywords

  • Query complexity
  • Theory of randomized search heuristics
  • Mastermind
  • Black-box complexity