Performance Assessment of Recursive Probability Matching for Adaptive Operator Selection in Differential Evolution

  • Mudita SharmaEmail author
  • Manuel López-Ibáñez
  • Dimitar Kazakov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11102)


Probability Matching is one of the most successful methods for adaptive operator selection (AOS), that is, online parameter control, in evolutionary algorithms. In this paper, we propose a variant of Probability Matching, called Recursive Probability Matching (RecPM-AOS), that estimates reward based on progress in past generations and estimates quality based on expected quality of possible selection of operators in the past. We apply RecPM-AOS to the online selection of mutation strategies in differential evolution (DE) on the bbob benchmark functions. The new method is compared with two AOS methods, namely, PM-AdapSS, which utilises probability matching with relative fitness improvement, and F-AUC, which combines the concept of area under the curve with a multi-arm bandit algorithm. Experimental results show that the new tuned RecPM-AOS method is the most effective at identifying the best mutation strategy to be used by DE in solving most functions in bbob among the AOS methods.


Parameter control Probability matching Differential evolution Black-box optimisation 


  1. 1.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)CrossRefGoogle Scholar
  2. 2.
    Das, S., Mullick, S.S., Suganthan, P.N.: Recent advances in differential evolution-an updated survey. Swarm Evol. Comput. 27, 1–30 (2016)CrossRefGoogle Scholar
  3. 3.
    Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. CRC Press, Boca Raton (1994)zbMATHGoogle Scholar
  4. 4.
    Fialho, Á., Schoenauer, M., Sebag, M.: Fitness-AUC bandit adaptive strategy selection vs. the probability matching one within differential evolution: an empirical comparison on the BBOB-2010 noiseless testbed. In: Pelikan, M., et al. (eds.) GECCO (Companion), pp. 1535–1542 (2010)Google Scholar
  5. 5.
    Fialho, Á., Schoenauer, M., Sebag, M.: Toward comparison-based adaptive operator selection. In: Pelikan, M., et al. (eds.) 2010 Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), Portland, Oregon, USA, pp. 767–774 (2010)Google Scholar
  6. 6.
    Goldberg, D.E.: Probability matching, the magnitude of reinforcement, and classifier system bidding. Mach. Learn. 5(4), 407–425 (1990)Google Scholar
  7. 7.
    Gong, W., Fialho, Á., Cai, Z.: Adaptive strategy selection in differential evolution. In: Pelikan, M., et al. (eds.) GECCO, pp. 409–416 (2010)Google Scholar
  8. 8.
    Hansen, N., Auger, A., Mersmann, O., Tusar, T., Brockhoff, D.: COCO: a platform for comparing continuous optimizers in a black-box setting. Arxiv preprint arXiv:1603.08785 (2016)
  9. 9.
    Hansen, N., Finck, S., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2009: noiseless functions definitions. Technical report, RR-6829, INRIA, France (2009). (updated February 2010)Google Scholar
  10. 10.
    Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195 (2001)CrossRefGoogle Scholar
  11. 11.
    Karafotias, G., Hoogendoorn, M., Eiben, A.E.: Parameter control in evolutionary algorithms: trends and challenges. IEEE Trans. Evol. Comput. 19(2), 167–187 (2015)CrossRefGoogle Scholar
  12. 12.
    López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Stützle, T., Birattari, M.: The irace package: iterated racing for automatic algorithm configuration. Oper. Res. Perspect. 3, 43–58 (2016)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Price, K., Storn, R.M., Lampinen, J.A.: Differential Evolution: A Practical Approach to Global Optimization. Springer, New York (2005). Scholar
  14. 14.
    Rust, J.: Structural estimation of Markov decision processes. In: Handbook of Econometrics, vol. 4, pp. 3081–3143. Elsevier (1994)Google Scholar
  15. 15.
    Sharma, M., López-Ibáñez, M., Kazakov, D.: Performance assessment of recursive probability matching for adaptive operator selection in differential evolution: supplementary material (2018).
  16. 16.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  17. 17.
    Zhang, J., Sanderson, A.C.: JADE: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput. 13(5), 945–958 (2009)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Mudita Sharma
    • 1
    Email author
  • Manuel López-Ibáñez
    • 2
  • Dimitar Kazakov
    • 1
  1. 1.University of YorkYorkUK
  2. 2.University of ManchesterManchesterUK

Personalised recommendations